Create a queue

A Queue is a processing queue managed by GCP Cloud Batch. The workflow orchestrator — Nextflow or Cromwell — submits jobs to it.

You can register several queues to keep work organised and to manage cost. Queues also configure the instance type used.

AWS and GCP differ here

AWS queues carry more settings: initial disk size, instance families, and whether to use spot or on-demand instances. GCP queues are simpler, because Cloud Batch derives the machine from each task’s declared CPU, memory and disk requirements. See Cloud resources.

Create a spot queue

export ENVIRONMENT_ID=<environment-id>

geni queue create \
  --name "SPOT" \
  --environment-id $ENVIRONMENT_ID \
  --execution-mode spot

Create an on-demand queue

Create a second queue in on-demand mode. Analysts then pick per submission, and can name it as the fallback for spot-capacity failures — see Managing submissions.

geni queue create \
  --name "ONDEMAND" \
  --environment-id $ENVIRONMENT_ID \
  --execution-mode ondemand

Verify

geni queue list
geni engine list

With an environment, an engine and at least one queue in place, the project is ready. An analyst can now register a workflow and submit — see Managing workflows.