Pipeline Development

Validation Schema

C-PAC uses voluptuous to validate pipeline configurations against defined types and value restrictions. Default values should be set in the default pipeline along with comments explaining each option.

Validation schema for C-PAC pipeline configurations

CPAC.pipeline.schema.name_motion_filter(mfilter, mfilters=None)

Given a motion filter, create a short string for the filename

Parameters:

• mfilter (dict) –

• mfliters (list or None) –

Return type:

str

Examples

>>> name_motion_filter({'filter_type': 'notch', 'filter_order': 2,
...     'center_frequency': 0.31, 'filter_bandwidth': 0.12})
'notch2fc0p31bw0p12'
>>> name_motion_filter({'filter_type': 'notch', 'filter_order': 4,
...     'breathing_rate_min': 0.19, 'breathing_rate_max': 0.43})
'notch4fl0p19fu0p43'
>>> name_motion_filter({'filter_type': 'lowpass', 'filter_order': 4,
...     'lowpass_cutoff': .0032})
'lowpass4fc0p0032'
>>> name_motion_filter({'filter_type': 'lowpass', 'filter_order': 2,
...     'breathing_rate_min': 0.19})
'lowpass2fl0p19'
>>> name_motion_filter({'filter_type': 'lowpass', 'filter_order': 2,
...     'breathing_rate_min': 0.19}, [{'Name': 'lowpass2fl0p19'}])
'lowpass2fl0p19dup1'
>>> name_motion_filter({'filter_type': 'lowpass', 'filter_order': 2,
...     'breathing_rate_min': 0.19}, [{'Name': 'lowpass2fl0p19'},
...     {'Name': 'lowpass2fl0p19dup1'}])
'lowpass2fl0p19dup2'

CPAC.pipeline.schema.permutation_message(key, options)

Function to give a clean, human-readable error message for keys that accept permutation values

Parameters:

• key (str) –

• options (list or set) –

Returns:

msg

Return type:

str

CPAC.pipeline.schema.sanitize(filename)

Sanitize a filename and replace whitespaces with underscores

CPAC.pipeline.schema.schema(config_dict)

Validate a pipeline configuration against the latest validation schema by first applying backwards-compatibility patches, then applying Voluptuous validation, then handling complex configuration interaction checks before returning validated config_dict.

Parameters:

config_dict (dict) –

Return type:

dict

CPAC.pipeline.schema.str_to_bool1_1(x)

Convert strings to Booleans for YAML1.1 syntax

Ref https://yaml.org/type/bool.html

Parameters:

x (any) –

Return type:

bool

Pipeline Utilities

C-PAC pipeline engine utilities

CPAC.pipeline.utils.name_fork(resource_idx, cfg, json_info, out_dct)

Create and insert entities for forkpoints

Parameters:

• resource_idx (str) –

• cfg (CPAC.utils.configuration.Configuration) –

• json_info (dict) –

• out_dct (dict) –

Returns:

• resource_idx (str)

• out_dct (dict)

CPAC.pipeline.utils.present_outputs(outputs: dict, keys: list) → dict

Given an outputs dictionary and a list of output keys, returns the subset of outputs that includes all keys in keys that are present. I.e., motion_correct_connections will have different items in its outputs dictionary at different times depending on the motion_correction configuration; func_motion_estimates can then wrap that outputs in this function and provide a list of keys of the desired outputs to include, if they are present in the provided outputs dictionary, eliminating the need for multiple NodeBlocks that differ only by configuration options and relevant output keys.

Parameters:

• outputs (dict) –

• keys (list of str) –

Returns:

outputs filtered down to keys

Return type:

dict

Examples

>>> present_outputs({'a': 1, 'b': 2, 'c': 3}, ['b'])
{'b': 2}
>>> present_outputs({'a': 1, 'b': 2, 'c': 3}, ['d'])
{}
>>> present_outputs({'a': 1, 'b': 2, 'c': 3}, ['a', 'c'])
{'a': 1, 'c': 3}

CPAC.pipeline.utils.source_set(sources: str | list | set) → set

Given a CpacProvenance, return a set of {resource}:{source} strings

Parameters:

sources (str, list, or set) –

Return type:

set

Examples

>>> source_set([[[['bold:func_ingress',
...       'desc-preproc_bold:func_reorient',
...       'desc-preproc_bold:func_truncate'],
...      ['TR:func_metadata_ingress'],
...      ['tpattern:func_metadata_ingress'],
...      'desc-preproc_bold:func_slice_time'],
...     [['bold:func_ingress',
...       'desc-preproc_bold:func_reorient',
...       'desc-preproc_bold:func_truncate'],
...      ['bold:func_ingress', 'desc-reorient_bold:func_reorient'],
...      'motion-basefile:get_motion_ref_fmriprep_reference'],
...     'desc-preproc_bold:motion_correction_only_mcflirt'],
...         [[['bold:func_ingress',
...           'desc-preproc_bold:func_reorient',
...           'desc-preproc_bold:func_truncate'],
...      ['bold:func_ingress', 'desc-reorient_bold:func_reorient'],
...      'motion-basefile:get_motion_ref_fmriprep_reference'],
...     [[['bold:func_ingress',
...        'desc-preproc_bold:func_reorient',
...        'desc-preproc_bold:func_truncate'],
...       ['TR:func_metadata_ingress'],
...       ['tpattern:func_metadata_ingress'],
...       'desc-preproc_bold:func_slice_time'],
...      [['bold:func_ingress',
...        'desc-preproc_bold:func_reorient',
...        'desc-preproc_bold:func_truncate'],
...       ['bold:func_ingress', 'desc-reorient_bold:func_reorient'],
...       'motion-basefile:get_motion_ref_fmriprep_reference'],
...      'desc-preproc_bold:motion_correction_only_mcflirt'],
...     ['FSL-AFNI-bold-ref:template_resample'],
...     ['FSL-AFNI-brain-mask:template_resample'],
...     ['FSL-AFNI-brain-probseg:template_resample'],
...     'space-bold_desc-brain_mask:bold_mask_fsl_afni'],
...     'desc-preproc_bold:bold_masking']) == set({
...     'FSL-AFNI-bold-ref:template_resample',
...     'FSL-AFNI-brain-mask:template_resample',
...     'FSL-AFNI-brain-probseg:template_resample',
...     'TR:func_metadata_ingress',
...     'bold:func_ingress',
...     'desc-preproc_bold:bold_masking',
...     'desc-preproc_bold:func_reorient',
...     'desc-preproc_bold:func_slice_time',
...     'desc-preproc_bold:func_truncate',
...     'desc-preproc_bold:motion_correction_only_mcflirt',
...     'desc-reorient_bold:func_reorient',
...     'motion-basefile:get_motion_ref_fmriprep_reference',
...     'space-bold_desc-brain_mask:bold_mask_fsl_afni',
...     'tpattern:func_metadata_ingress'})
True