Building/Editing a Pipeline

If you want to build your own pipeline, or edit an existing one, there are a few important things to know.

Editing a WDL

If you want to edit the WDL for a pipeline, please ensure that these changes are reflected both in the Gitlab repository and the Firecloud method repository. When uploading the new WDL to the Firecloud method repository, a new snapshot will be stored. Please give some indication of changes that are made between the snapshots (this could, for example, be done in the snapshot comments section).

Editing a Docker image

It may be necessary for you to update a pipeline’s Docker when you add new functionality. This is a multi-step process:

  1. Update the Dockerfile (or included scripts) as necessary.
  2. Build the Docker image. We recommend naming it to be consistent with the pipeline name.
building a Docker image
docker build -t gcr.io/<{genomics-xavier, microbiome-xavier}>/<pipeline_name> /path/to/directory/containing/dockerfile
  1. Push the Docker image to the Google container repository.
pushing a Docker image to the Google Container Registry
docker push gcr.io/<{genomics-xavier, microbiome-xavier}>/<pipeline_name>

Giving a pipeline access to external Google Buckets

By default, FireCloud pipelines will have access to the Google Bucket to which they are attached, but not to buckets associated with the genomics-xavier and microbiome-xavier Google projects. To give a FireCloud group access to one of these Google buckets, you can go to the bucket’s permissions and add <firecloud_group_name>@firecloud.org as a Storage Admin, where firecloud_group_name is either genomics-xavier-users or microbiome-xavier.