Setting Up A Pipeline Configuration

Overview

There are two ways of setting up a pipeline configuration for C-PAC:

  • Using a text editor (useful for remote servers where using the C-PAC GUI is not possible or impractical)
  • Using the pipeline configuration interface in the C-PAC GUI

Definitions

  • Workflow - A workflow accomplishes a particular processing task (e.g. functional preprocessing, scrubbing, nuisance correction). Each workflow can be turned on or off in the pipeline configuration. Sometimes a workflow can be set to both on and off, allowing for pipelines to branch.
  • Pipeline - A pipeline is a combination of workflows.
  • Strategy - A strategy is a set of preprocessing options. Specifically, a strategy is defined by nuisance corrections and scrubbing settings. Strategies can branch depending on which of these workflows are turned on or off and how they are configured. Their names are constructed by concatenating the following parameters:
    • Number of principle components calculated by CompCor (if enabled)
    • Nuisance corrections selected.
    • Scrubbing threshold (if enabled)

For instance, _compcor_ncomponents_5_linear1.motion1.compcor1.SCRUB_0.2 is a strategy with 5 principle components for compcor, linear drift, motion, and compcor corrections applied, and a scrubbing threshold of 0.2 mm.

  • Derivative - Derivatives are the results of processing a participant’s raw data (i.e., connectivity measures).

Using a Text Editor

Pipeline configuration files, like the data configuration and subject list files discussed in the subject list builder section, are stored as YAML files. Similarly, each of the parameters used by C-PAC to assemble your pipeline can be specified as key-value pairs, so a pipeline configuration YAML would have multiple lines of the form:

key : value

An example of a pipeline configuration YAML file can be found here. Tables explaining the keys and their potential values can be found on the individual pages for each of the outputs C-PAC is capable of producing. All pipeline configuration files should have the keys in the Output Settings table defined.

Why a list?

You may notice as you learn about the settings for various outputs that many of the values for C-PAC’s configurable settings are stored in lists (i.e., multiple values are separated by commas and surrounded by square brackets). Such lists containing 1s and 0s (for ‘True’ and ‘False’ respectively) allow you to toggle on multiple options at the same time, and branch a pipeline into two different analysis strategies. See the developer documentation for more information about how lists are used in C-PAC.

Using the GUI

Opening the GUI

If the C-PAC GUI is not open already, type the command cpac_gui in a terminal window as you would with the subject list builder. Then, on the main screen click on New next to Pipelines. For each of the settings in the lefthand pane, refer to the pages linked to below in the Configurable Settings section.

When you have finished configuring your pipeline, click Save. You will be asked to specify a location to save a configuration file containing information about the pipeline, and to specify a name for the pipeline.

Configurable Settings

Data Management and Environment Settings

Pre- and post-processing

Derivatives