Utilities #

CPAC.utils.bids_utils.bids_match_entities(file_list, entities, suffix)[source]#

Function to subset a list of filepaths by a passed BIDS entity.

Parameters:

file_list (list of str) –
entities (str) – BIDS entities joined by underscores (e.g., ‘ses-001_task-PEER1’)
suffix (str) – BIDS suffix (e.g., ‘bold’, ‘T1w’)

Return type:

list of str

Examples

>>> bids_match_entities([
...     's3://fake/data/sub-001_ses-001_task-MSIT_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ], 'task-PEER1', 'bold')
['s3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz']
>>> bids_match_entities([
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ], 'PEER', 'bold')
Traceback (most recent call last):
LookupError: No match found for provided entity "PEER" in
- s3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz
- s3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz
Perhaps you meant one of these?
- task-PEER1
- task-PEER2

CPAC.utils.bids_utils.bids_parse_sidecar()[source]#

Uses the BIDS principle of inheritance to build a data structure that maps parameters in side car .json files to components in the names of corresponding nifti files.

Parameters:

config_dict – dictionary that maps paths of sidecar json files (the key) to a dictionary containing the contents of the files (the values)
dbg – boolean flag that indicates whether or not debug statements should be printed

Returns:

a dictionary that maps parameters to components from BIDS filenames such as sub, sess, run, acq, and scan type

CPAC.utils.bids_utils.bids_remove_entity(name, key)[source]#

Remove an entity from a BIDS string by key

Parameters:

name (str) – BIDS string to remove entity from
key (str) – BIDS key of entity to remove

Returns:

BIDS name with entity removed

Return type:

Examples

>>> bids_remove_entity('atlas-Yeo_space-MNI152NLin6_res-2x2x2', 'space')
'atlas-Yeo_res-2x2x2'
>>> bids_remove_entity('atlas-Yeo_space-MNI152NLin6_res-2x2x2', 'res')
'atlas-Yeo_space-MNI152NLin6'

CPAC.utils.bids_utils.bids_retrieve_params(bids_config_dict, f_dict, dbg=False)[source]#

Retrieve the BIDS parameters from bids_config_dict for BIDS file corresponding to f_dict. If an exact match for f_dict is not found the nearest match is returned, corresponding to the BIDS inheritance principle.

Parameters:

bids_config_dict – BIDS configuration dictionary, this is a multi-level dictionary that maps the components of a bids filename (i.e. sub, ses, acq, run) to a dictionary that contains the BIDS parameters (RepetitionTime, EchoTime, etc). This information is extracted from sidecar json files using the principle of inheritance using the bids_parse_configs function
f_dict – Dictionary built from the name of a file in the BIDS format. This is built using the bids_decode_fname by splitting on “-” and “_” delimeters
dbg – boolean flag that indicates whether or not debug statements should be printed, defaults to “False”

Returns:

returns a dictionary that contains the BIDS parameters

CPAC.utils.bids_utils.bids_shortest_entity(file_list)[source]#

Function to return the single file with the shortest chain of BIDS entities from a given list, returning the first if more than one have the same minimum length.

Parameters:: file_list (list of strings) –
Return type:: str or None

Examples

>>> bids_shortest_entity([
...     's3://fake/data/sub-001_ses-001_task-MSIT_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ])
's3://fake/data/sub-001_ses-001_bold.nii.gz'

CPAC.utils.bids_utils.camelCase(string: str) → str[source]#

Convert a hyphenated string to camelCase

Parameters:: string (str) – string to convert to camelCase
Return type:: str

Examples

>>> camelCase('PearsonNilearn-aCompCor')
'PearsonNilearnACompCor'
>>> camelCase('mean-Pearson-Nilearn-aCompCor')
'meanPearsonNilearnACompCor'

CPAC.utils.bids_utils.cl_strip_brackets(arg_list)[source]#

Removes ‘[’ from before first and ‘]’ from after final arguments in a list of commandline arguments

Parameters:: arg_list (list) –
Return type:: list

Examples

>>> cl_strip_brackets('[a b c]'.split(' '))
['a', 'b', 'c']
>>> cl_strip_brackets('a b c'.split(' '))
['a', 'b', 'c']
>>> cl_strip_brackets('[ a b c ]'.split(' '))
['a', 'b', 'c']

CPAC.utils.bids_utils.collect_bids_files_configs(bids_dir, aws_input_creds='')[source]#

Parameters:

bids_dir –
aws_input_creds –

Returns:

CPAC.utils.bids_utils.combine_multiple_entity_instances(bids_str: str) → str[source]#

Combines mutliple instances of a key in a BIDS string to a single instance by camelCasing and concatenating the values

Parameters:: bids_str (str) –
Return type:: str

Examples

>>> combine_multiple_entity_instances(
...     'sub-1_ses-HBN_site-RU_task-rest_atlas-AAL_'
...     'desc-Nilearn_desc-36-param_suffix.ext')
'sub-1_ses-HBN_site-RU_task-rest_atlas-AAL_desc-Nilearn36Param_suffix.ext'
>>> combine_multiple_entity_instances(
...     'sub-1_ses-HBN_site-RU_task-rest_'
...     'run-1_framewise-displacement-power.1D')
'sub-1_ses-HBN_site-RU_task-rest_run-1_framewiseDisplacementPower.1D'

CPAC.utils.bids_utils.create_cpac_data_config(bids_dir, participant_labels=None, aws_input_creds=None, skip_bids_validator=False, only_one_anat=True)[source]#

Create a C-PAC data config YAML file from a BIDS directory.

Parameters:

bids_dir (str) –
participant_labels (list or None) –
aws_input_creds –
skip_bids_validator (bool) –
only_one_anat (bool) – The “anat” key for a subject expects a string value, but we can temporarily store a list instead by passing True here if we will be filtering that list down to a single string later

Return type:

CPAC.utils.bids_utils.gen_bids_outputs_sublist(base_path, paths_list, key_list, creds_path)[source]#

CPAC.utils.bids_utils.insert_entity(resource, key, value)[source]#

Insert a f’{key}-{value}’ BIDS entity before desc- if present or before the suffix otherwise

Parameters:

resource (str) –
key (str) –
value (str) –

Return type:

Examples

>>> insert_entity('run-1_desc-preproc_bold', 'reg', 'default')
'run-1_reg-default_desc-preproc_bold'
>>> insert_entity('run-1_bold', 'reg', 'default')
'run-1_reg-default_bold'
>>> insert_entity('run-1_desc-preproc_bold', 'filt', 'notch4c0p31bw0p12')
'run-1_filt-notch4c0p31bw0p12_desc-preproc_bold'
>>> insert_entity('run-1_reg-default_bold', 'filt', 'notch4c0p31bw0p12')
'run-1_reg-default_filt-notch4c0p31bw0p12_bold'

CPAC.utils.bids_utils.load_cpac_data_config(data_config_file, participant_labels, aws_input_creds)[source]#

Loads the file as a check to make sure it is available and readable

Parameters:

data_config_file (str) – path to data config
participants_labels (list or None) –
aws_input_creds –

Return type:

CPAC.utils.bids_utils.load_yaml_config(config_filename, aws_input_creds)[source]#

CPAC.utils.bids_utils.res_in_filename(cfg, label)[source]#

Specify resolution in filename

Parameters:

cfg (CPAC.utils.configuration.Configuration) –
label (str) –

Returns:

label

Return type:

Examples

>>> from CPAC.utils.configuration import Configuration
>>> res_in_filename(Configuration({
...     'registration_workflows': {
...         'anatomical_registration': {'resolution_for_anat': '2x2x2'}}}),
...     'sub-1_res-anat_bold')
'sub-1_res-2x2x2_bold'
>>> res_in_filename(Configuration({
...     'registration_workflows': {
...         'anatomical_registration': {'resolution_for_anat': '2x2x2'}}}),
...     'sub-1_res-3mm_bold')
'sub-1_res-3mm_bold'

CPAC.utils.bids_utils.sub_list_filter_by_labels(sub_list, labels)[source]#

Function to filter a sub_list by provided BIDS labels for specified suffixes

Parameters:

sub_list (list) –
labels (dict) –
labels['T1w'] (str or None) – C-PAC currently only uses a single T1w image
labels['bold'] (str, list, or None) –

Return type:

CPAC.utils.bids_utils.with_key(entity: str, key: str) → str[source]#

Return a keyed BIDS entity

Parameters:

entity (str) –
key (str) –

Return type:

Examples

>>> with_key('sub-1', 'sub')
'sub-1'
>>> with_key('1', 'sub')
'sub-1'

CPAC.utils.bids_utils.without_key(entity: str, key: str) → str[source]#

Return a BIDS entity value

Parameters:

entity (str) –
key (str) –

Return type:

Examples

>>> without_key('sub-1', 'sub')
'1'
>>> without_key('1', 'sub')
'1'

BIDS Data Configuration#

Configuration#

C-PAC Configuration module

class CPAC.utils.configuration.Configuration(config_map=None)[source]#

Class to set dictionary keys as map attributes.

If the given dictionary includes the key FROM, that key’s value will form the base of the Configuration object with the values in the given dictionary overriding matching keys in the base at any depth. If no FROM key is included, the base Configuration is the default Configuration.

FROM accepts either the name of a preconfigured pipleine or a path to a YAML file.

Given a Configuration c, and a list or tuple of an attribute name and nested keys keys = ['attribute', 'key0', 'key1'] or keys = ('attribute', 'key0', 'key1'), the value ‘value’ nested in

c.attribute = {'key0': {'key1': 'value'}}

can be accessed (get and set) in any of the following ways (and more):

c.attribute['key0']['key1']
c['attribute']['key0']['key1']
c['attribute', 'key0', 'key1']
c[keys]

Examples

>>> c = Configuration({})
>>> c['pipeline_setup', 'pipeline_name']
'cpac-blank-template'
>>> c = Configuration({'pipeline_setup': {
...     'pipeline_name': 'example_pipeline'}})
>>> c['pipeline_setup', 'pipeline_name']
'example_pipeline'
>>> c['pipeline_setup', 'pipeline_name'] = 'new_pipeline2'
>>> c['pipeline_setup', 'pipeline_name']
'new_pipeline2'

>>> from CPAC.utils.tests.configs import SLACK_420349

# test “FROM: /path/to/file” >>> slack_420349_filepath = Configuration( … yaml.safe_load(SLACK_420349[‘filepath’])) >>> slack_420349_filepath[‘pipeline_setup’, ‘pipeline_name’] ‘slack_420349_filepath’

# test “FROM: preconfig” >>> slack_420349_preconfig = Configuration( … yaml.safe_load(SLACK_420349[‘preconfig’])) >>> slack_420349_preconfig[‘pipeline_setup’, ‘pipeline_name’] ‘slack_420349_preconfig’

check_pattern(orig_key, tags=None)[source]#

dict()[source]#: Show contents of a C-PAC configuration as a dict

get_nested(_d, keys)[source]#

key_type_error(key)[source]#

keys()[source]#: Show toplevel keys of a C-PAC configuration dict

return_config_elements()[source]#

set_nested(d, keys, value)[source]#

sub_pattern(pattern, orig_key)[source]#

switch_is_off(key: str | list[str], exclusive: bool = False) → bool[source]#

Return True if the given key is set to ‘off’ OR ‘on’ and ‘off’ or False otherwise. Used for tracking forking.

Parameters:

key (str or list of str) – key to check
exclusive (bool, optional, default: False) – return False if the key is set to ‘on’ and ‘off’

Returns:

True if key is set to ‘off’, False if not set to ‘off’. If exclusive is set to True, return False if the key is set to ‘on’ and ‘off’.

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'])
True
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'])
True
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'], exclusive=True)
False

switch_is_on(key: str | list[str], exclusive: bool = False) → bool[source]#

Return True if the given key is set to ‘on’ OR ‘on’ and ‘off’ or False otherwise. Used for tracking forking.

Parameters:

key (str or list of str) – key to check
exclusive (bool, optional, default: False) – return False if the key is set to ‘on’ and ‘off’

Returns:

True if key is set to ‘on’, False if not set to ‘on’. If exclusive is set to True, return False if the key is set to ‘on’ and ‘off’.

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'])
False
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'])
True
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'], exclusive=True)
False

switch_is_on_off(key: str | list[str]) → bool[source]#

Return True if the given key is set to both ‘on’ and ‘off’ or False otherwise. Used for tracking forking.

Parameters:: key (str or list of str) – key to check
Returns:: True if key is set to ‘on’ and ‘off’, False otherwise
Return type:: bool

Examples

>>> c = Configuration()
>>> c.switch_is_on_off(['nuisance_corrections',
...                     '2-nuisance_regression', 'run'])
False
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_on_off(['nuisance_corrections',
...                     '2-nuisance_regression', 'run'])
True

update(key, val=ConfigurationDictUpdateConflation())[source]#

class CPAC.utils.configuration.Preconfiguration(preconfig)[source]#

A preconfigured Configuration

Parameters:: preconfig (str) – The canonical name of the preconfig to load

CPAC.utils.configuration.check_pname(p_name: str, pipe_config: Configuration) → str[source]#

Function to check / set p_name, the string representation of a pipeline for use in filetrees

Parameters:

p_name (str or None) –
pipe_config (Configuration) –

Return type:

p_name

Examples

>>> c = Configuration()
>>> check_pname(None, c)
'pipeline_cpac-blank-template'
>>> check_pname('cpac-default-pipeline', c)
'pipeline_cpac-default-pipeline'
>>> check_pname('pipeline_cpac-default-pipeline', c)
'pipeline_cpac-default-pipeline'
>>> check_pname('different-name', Configuration())
'pipeline_different-name'
>>> p_name = check_pname(None, Preconfiguration('blank'))
>>> p_name
'pipeline_cpac-blank-template'
>>> p_name = check_pname(None, Preconfiguration('default'))
>>> p_name
'pipeline_cpac-default-pipeline'

CPAC.utils.configuration.preconfig_yaml(preconfig_name='default', load=False)[source]#

Get the path to a preconfigured pipeline’s YAML file. Raises BadParameter if an invalid preconfig name is given.

Parameters:

preconfig_name (str) –
load (boolean) – return dict if True, str if False

Returns:

path to YAML file or dict loaded from YAML

Return type:

str or dict

CPAC.utils.configuration.set_subject(sub_dict: dict, pipe_config: Configuration, p_name: str | None = None) → tuple[str, str, str][source]#

Function to set pipeline name and log directory path for a given sub_dict

Parameters:

sub_dict (dict) –
pipe_config (CPAC.utils.configuration.Configuration) –
p_name (str, optional) – pipeline name string

Returns:

subject_id (str)
p_name (str) – pipeline name string
log_dir (str) – path to subject log directory

Examples

>>> from tempfile import TemporaryDirectory
>>> from CPAC.utils.configuration import Configuration
>>> sub_dict = {'site_id': 'site1', 'subject_id': 'sub1',
...             'unique_id': 'uid1'}
>>> with TemporaryDirectory() as tmpdir:
...     subject_id, p_name, log_dir = set_subject(
...         sub_dict, Configuration({'pipeline_setup': {'log_directory':
...             {'path': tmpdir}}}))
>>> subject_id
'sub1_uid1'
>>> p_name
'pipeline_cpac-blank-template'
>>> log_dir.endswith(f'{p_name}/{subject_id}')
True

Create FMRIB’s Local Analysis of Mixed Effects (FLAME) Model Files#

CPAC.utils.create_flame_model_files.create_con_file(con_vecs, con_names, col_names, model_name, current_output, out_dir)[source]#

CPAC.utils.create_flame_model_files.create_con_ftst_file(con_file, model_name, current_output, output_dir, column_names, coding_scheme, group_sep)[source]#: Create the contrasts and fts file

CPAC.utils.create_flame_model_files.create_dummy_string(length)[source]#

CPAC.utils.create_flame_model_files.create_flame_model_files(design_matrix, col_names, contrasts_vectors, contrast_names, custom_contrasts_csv, ftest_list, group_sep, grouping_vector, coding_scheme, model_name, output_measure, output_dir)[source]#

CPAC.utils.create_flame_model_files.create_fts_file(ftest_list, con_names, model_name, current_output, out_dir)[source]#

CPAC.utils.create_flame_model_files.create_grp_file(design_matrix, grp_file_vector, output_dir, model_name)[source]#

CPAC.utils.create_flame_model_files.write_mat_file(design_matrix, output_dir, model_name, depatsified_EV_names, current_output=None)[source]#

Create FSL FLAME Preset#

CPAC.utils.create_fsl_flame_preset.create_contrasts_template_df(design_df, contrasts_dct_list=None)[source]#

Create the template Pandas DataFrame for the contrasts matrix CSV.

The headers in the contrasts matrix needs to match the headers of the design matrix.

CPAC.utils.create_fsl_flame_preset.create_design_matrix_df(group_list, pheno_df=None, ev_selections=None, pheno_sub_label=None, pheno_ses_label=None, pheno_site_label=None, ses_id=False)[source]#

Create the design matrix intended for group-level analysis via the FSL FLAME tool.

This does NOT create the final .mat file that FSL FLAME takes in. This is an intermediary design matrix CSV meant for the user to review.

If there is a phenotype CSV provided, this function will align the participant-session ID labels in the CPAC individual-level analysis output directory with the values listed in the phenotype file.

CPAC.utils.create_fsl_flame_preset.preset_paired_two_group(group_list, conditions, condition_type='session', output_dir=None, model_name='two_sample_unpaired_T-test')[source]#

Set up the design matrix and contrasts matrix for running an paired two-group difference (two-sample paired T-test).

group_list: a list of strings- sub_ses unique IDs conditions: a two-item list of strings- session or series/scan names of

the two sessions or two scans (per participant) you wish to compare

condition_type: a string, either “session” or “scan”, depending on what: is in “conditions”

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:

https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide: #Paired_Two-Group_Difference_.28Two-Sample_Paired_T-Test.29

CPAC.utils.create_fsl_flame_preset.preset_single_group_avg(group_list, pheno_df=None, covariate=None, pheno_sub_label=None, output_dir=None, model_name='one_sample_T-test')[source]#: Set up the design matrix CSV for running a single group average (one-sample T-test).

CPAC.utils.create_fsl_flame_preset.preset_tripled_two_group(group_list, conditions, condition_type='Sessions', output_dir=None, model_name='tripled_T-test')[source]#

Set up the design matrix and contrasts matrix for running a tripled two-group difference (‘tripled’ T-test).

group_list: a list of strings- sub_ses unique IDs conditions: a three-item list of strings- session or series/scan names of

the three sessions or three scans (per participant) you wish to compare

condition_type: a string, either “session” or “scan”, depending on what: is in “conditions”

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:

https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide: #Tripled_Two-Group_Difference_.28.22Tripled.22_T-Test.29

CPAC.utils.create_fsl_flame_preset.preset_unpaired_two_group(group_list, pheno_df, groups, pheno_sub_label, output_dir=None, model_name='two_sample_unpaired_T-test')[source]#

Set up the design matrix and contrasts matrix for running an unpaired two-group difference (two-sample unpaired T-test).

group_list: a list of strings- sub_ses unique IDs pheno_df: a Pandas DataFrame object of the phenotypic file CSV/matrix groups: a list of either one or two strings- design matrix EV/covariate

labels to take from the phenotype DF and include in the model

pheno_sub_label: a string of the label name of the column in the phenotype: file that holds the participant/session ID for each row

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:

https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide: #Unpaired_Two-Group_Difference_.28Two-Sample_Unpaired_T-Test.29

Only one “group” will be provided usually if the two groups in the phenotypic information you wish to compare are encoded in one covariate column, as categorical information. Thus, providing this one name will pull it from the phenotype file, and this function will break it out into two columns using dummy-coding.

CPAC.utils.create_fsl_flame_preset.read_group_list_text_file(group_list_text_file)[source]#: Read in the group-level analysis participant-session list text file.

CPAC.utils.create_fsl_flame_preset.read_pheno_csv_into_df(pheno_csv, id_label=None)[source]#: Read the phenotypic file CSV or TSV into a Pandas DataFrame.

CPAC.utils.create_fsl_flame_preset.run(pipeline_dir, derivative_list, z_thresh, p_thresh, preset=None, group_list_text_file=None, pheno_file=None, pheno_sub_label=None, output_dir=None, model_name=None, covariate=None, condition_type=None, run=False)[source]#

CPAC.utils.create_fsl_flame_preset.write_config_dct_to_yaml(config_dct, out_file=None)[source]#: Write out a configuration dictionary into a YAML file.

CPAC.utils.create_fsl_flame_preset.write_dataframe_to_csv(matrix_df, out_file=None)[source]#: Write out a matrix Pandas DataFrame into a CSV file.

CPAC.utils.create_fsl_flame_preset.write_group_list_text_file(group_list, out_file=None)[source]#: Write out the group-level analysis participant list as a text file.

Create FSL create_fsl_model#

CPAC.utils.create_fsl_model.check_multicollinearity(matrix)[source]#

CPAC.utils.create_fsl_model.create_con_file(con_dict, col_names, file_name, current_output, out_dir)[source]#

CPAC.utils.create_fsl_model.create_con_ftst_file(con_file, model_name, current_output, output_dir, column_names, coding_scheme, group_sep)[source]#: Create the contrasts and fts file

CPAC.utils.create_fsl_model.create_design_matrix(pheno_file, ev_selections, formula, subject_id_label, sub_list=None, coding_scheme='Treatment', grouping_var=None, new_regressor_dict=None, roi_means_dict=None, output_dir=None, model_name='design', current_output=None)[source]#

CPAC.utils.create_fsl_model.create_dummy_string(length)[source]#

CPAC.utils.create_fsl_model.create_fts_file(ftest_list, con_dict, model_name, current_output, out_dir)[source]#

CPAC.utils.create_fsl_model.create_grp_file(design_matrix, grouping_var_id_dict, output_dir, model_name, current_output=None)[source]#

CPAC.utils.create_fsl_model.create_pheno_dict(pheno_file_rows, ev_selections, participant_id_label)[source]#

CPAC.utils.create_fsl_model.get_custom_roi_info(roi_means_dict)[source]#

CPAC.utils.create_fsl_model.get_measure_dict(param_file)[source]#

CPAC.utils.create_fsl_model.greater_than(dmat, a, b, coding, group_sep, grouping_var)[source]#

CPAC.utils.create_fsl_model.load_group_participant_list(group_participant_list_file)[source]#

CPAC.utils.create_fsl_model.load_pheno_file(pheno_file)[source]#

CPAC.utils.create_fsl_model.model_group_var_separately(grouping_var, formula, pheno_data_dict, ev_selections, coding_scheme)[source]#

CPAC.utils.create_fsl_model.negative(dmat, a, coding, group_sep, grouping_var)[source]#

CPAC.utils.create_fsl_model.positive(dmat, a, coding, group_sep, grouping_var)[source]#

CPAC.utils.create_fsl_model.process_contrast(parsed_contrast, operator, ev_selections, group_sep, grouping_var, coding_scheme)[source]#

CPAC.utils.create_fsl_model.process_pheno_file(pheno_file_dataframe, group_subs_dataframe, participant_id_label)[source]#

CPAC.utils.create_fsl_model.run(group_config, current_output, param_file=None, derivative_means_dict=None, roi_means_dict=None, model_out_dir=None, CPAC_run=True)[source]#

CPAC.utils.create_fsl_model.write_mat_file(design_matrix, output_dir, model_name, depatsified_EV_names, current_output=None)[source]#

Create Group Analysis Info Files#

CPAC.utils.create_group_analysis_info_files.write_blank_contrast_csv(contrasts_columns, contrast_out_path)[source]#

CPAC.utils.create_group_analysis_info_files.write_custom_readme_file()[source]#

CPAC.utils.create_group_analysis_info_files.write_design_matrix_csv(patsy_dmatrix, participant_column, column_names, outfile_path)[source]#

Datasource#

CPAC.utils.datasource.bidsier_prefix(unique_id)[source]#

Function to return a BIDSier prefix for a given unique_id

Parameters:: unique_id (str) –
Returns:: prefix
Return type:: str

Examples

>>> bidsier_prefix('01_1')
'sub-01_ses-1'
>>> bidsier_prefix('sub-01_ses-1')
'sub-01_ses-1'
>>> bidsier_prefix('sub-01_1')
'sub-01_ses-1'
>>> bidsier_prefix('01_ses-1')
'sub-01_ses-1'

CPAC.utils.datasource.calc_delta_te_and_asym_ratio(effective_echo_spacing: float, echo_times: list) → tuple[float, float][source]#

Calcluate deltaTE and ees_asym_ratio from given metadata

Parameters:

effective_echo_spacing (float) – EffectiveEchoSpacing from sidecar JSON
echo_times (list) –

Returns:

deltaTE (float)
ees_asym_ratio (float)

CPAC.utils.datasource.check_for_s3(file_path, creds_path=None, dl_dir=None, img_type='other', verbose=False)[source]#

CPAC.utils.datasource.check_func_scan(func_scan_dct, scan)[source]#: Run some checks on the functional timeseries-related files for a given series/scan name or label.

CPAC.utils.datasource.create_anat_datasource(wf_name='anat_datasource')[source]#

CPAC.utils.datasource.create_check_for_s3_node(name, file_path, img_type='other', creds_path=None, dl_dir=None, map_node=False)[source]#

CPAC.utils.datasource.create_fmap_datasource(fmap_dct, wf_name='fmap_datasource')[source]#: Return the field map files, from the dictionary of functional files described in the data configuration (sublist) YAML file.

CPAC.utils.datasource.create_func_datasource(rest_dict, rpool, wf_name='func_datasource')[source]#

Return the functional timeseries-related file paths for each series/scan, from the dictionary of functional files described in the data configuration (sublist) YAML file.

Scan input (from inputnode) is an iterable.

CPAC.utils.datasource.create_general_datasource(wf_name)[source]#

CPAC.utils.datasource.create_grp_analysis_dataflow(wf_name='gp_dataflow')[source]#

CPAC.utils.datasource.create_roi_mask_dataflow(masks, wf_name='datasource_roi_mask')[source]#

CPAC.utils.datasource.create_spatial_map_dataflow(spatial_maps, wf_name='datasource_maps')[source]#

CPAC.utils.datasource.extract_scan_params_dct(scan_params_dct)[source]#

CPAC.utils.datasource.gather_echo_times(echotime_1, echotime_2, echotime_3=None, echotime_4=None)[source]#

CPAC.utils.datasource.gather_extraction_maps(c)[source]#

CPAC.utils.datasource.get_fmap_phasediff_metadata(data_config_scan_params)[source]#

CPAC.utils.datasource.get_highest_local_res(template: Path | str, tagname: str) → Path[source]#

Given a reference template path and a resolution string, get all resolutions of that template in the same local path and return the highest resolution.

Parameters:

template (Path or str) –
tagname (str) –

Return type:

https://github.com/neurodata/m2g/blob/v0.1.0/ndmg/graph/graph.py

Raises:

LookupError – If no matching local template is found.

Examples

>>> get_highest_local_res(
...     '/cpac_templates/MacaqueYerkes19_T1w_2mm_brain.nii.gz', '2mm')
PosixPath('/cpac_templates/MacaqueYerkes19_T1w_0.5mm_brain.nii.gz')
>>> get_highest_local_res(
...     '/cpac_templates/dne_T1w_2mm.nii.gz', '2mm')
Traceback (most recent call last):
   ...
LookupError: Could not find template /cpac_templates/dne_T1w_2mm.nii.gz

CPAC.utils.datasource.get_map(map, map_dct)[source]#

CPAC.utils.datasource.get_rest(scan, rest_dict, resource='scan')[source]#

Return the file path of the chosen resource stored in the functional file dictionary, if it exists.

scan: the scan/series name or label rest_dict: the dictionary read in from the data configuration YAML file

(sublist) nested under ‘func:’

resource: the dictionary key: scan - the functional timeseries scan_parameters - path to the scan parameters JSON file, or

a dictionary containing scan parameters information (to be phased out in the future)

CPAC.utils.datasource.ingress_func_metadata(wf, cfg, rpool, sub_dict, subject_id, input_creds_path, unique_id=None, num_strat=None)[source]#

CPAC.utils.datasource.match_epi_fmaps(bold_pedir, epi_fmap_one, epi_fmap_params_one, epi_fmap_two=None, epi_fmap_params_two=None)[source]#

Parse the field map files in the data configuration and determine which ones have the same and opposite phase-encoding directions as the BOLD scan in the current pipeline.

Example - parse the files under the ‘fmap’ level, i.e. ‘epi_AP’:

anat: /path/to/T1w.nii.gz fmap:

epi_AP:
scan: /path/to/field-map.nii.gz scan_parameters: <config dictionary containing phase-encoding

direction>

func:

rest_1:: scan: /path/to/bold.nii.gz scan_parameters: <config dictionary of BOLD scan parameters>

Check PhaseEncodingDirection field in the metadata for the BOLD.
Check whether there are one or two EPI’s in the field map data.
Grab the one or two EPI field maps.

CPAC.utils.datasource.res_string_to_tuple(resolution)[source]#

Converts a resolution string to a tuple of floats.

Parameters:: resolution (str) – Resolution string, e.g. “3.438mmx3.438mmx3.4mm”
Returns:: resolution – Tuple of floats, e.g. (3.438, 3.438, 3.4)
Return type:: tuple

CPAC.utils.datasource.resample_func_roi(in_func, in_roi, realignment, identity_matrix)[source]#

CPAC.utils.datasource.resolve_resolution(resolution, template, template_name, tag=None)[source]#

CPAC.utils.datasource.select_model_files(model, ftest, model_name)[source]#: Method to select model files

Datatypes#

Custom datatypes for C-PAC

class CPAC.utils.datatypes.ItemFromList(list_of_one, msg=None)[source]#

Coerce single-item lists into just the only item in the list. Returns item if item is not a list, set, or tuple. Raises CoerceInvalid if impossible.

Examples

>>> ItemFromList(['seagull'])
'seagull'
>>> ItemFromList(['two', 'seagulls'])
Traceback (most recent call last):
    ...
voluptuous.error.CoerceInvalid: Cannot coerce list of length 2 to item
>>> ItemFromList('string')
'string'

class CPAC.utils.datatypes.ListFromItem(*args, **kwargs)[source]#

Subclass of list to coerce non-lists into lists

Examples

>>> list('one')
['o', 'n', 'e']
>>> ListFromItem('one')
['one']
>>> list(['one'])
['one']
>>> ListFromItem(['one'])
['one']
>>> list()
[]
>>> ListFromItem()
[]
>>> list(None)
Traceback (most recent call last):
...
TypeError: 'NoneType' object is not iterable
>>> ListFromItem(None)
[]

Documentation#

Utilties for documentation.

CPAC.utils.docs.docstring_parameter(*args, **kwargs)[source]#

Decorator to parameterize docstrings. Use double-curly-braces ({{}}) for literal curly braces.

Examples

>>> @docstring_parameter('test', answer='Yes it does.')
... def do_nothing():
...     '''Does this {} do anything? {answer}'''
...     pass
>>> print(do_nothing.__doc__)
Does this test do anything? Yes it does.
>>> @docstring_parameter('test', answer='It should not.')
... def how_about_now():
...     '''How about {{ this }}?'''
...     pass
>>> print(how_about_now.__doc__)
How about { this }?

CPAC.utils.docs.version_report() → str[source]#: A formatted block of versions included in CPAC’s environment

Extract Data#

class CPAC.utils.extract_data.Configuration(config_map)[source]#

CPAC.utils.extract_data.extract_data(c, param_map)[source]#

Method to generate a CPAC input subject list python file. The method extracts anatomical and functional data for each site( if multiple site) and/or scan parameters for each site and put it into a data structure read by python

Example: subjects_list =[

{
‘subject_id’ : ‘0050386’, ‘unique_id’ : ‘session_1’, ‘anat’: ‘/Users/home/data/NYU/0050386/session_1/anat_1/anat.nii.gz’, ‘rest’:{

‘rest_1_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_1/rest.nii.gz’, ‘rest_2_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_2/rest.nii.gz’, }

‘scan_parameters’:{
‘tr’: ‘2’, ‘acquisition’: ‘alt+z2’, ‘reference’: ‘17’, ‘first_tr’: ‘’, ‘last_tr’: ‘’, }

},

]

subjects_list =[

{: ‘subject_id’ : ‘0050386’, ‘unique_id’ : ‘session_1’, ‘anat’: ‘/Users/home/data/NYU/0050386/session_1/anat_1/anat.nii.gz’, ‘rest’:{

‘rest_1_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_1/rest.nii.gz’, ‘rest_2_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_2/rest.nii.gz’, }

},

]

CPAC.utils.extract_data.generate_supplementary_files(data_config_outdir, data_config_name)[source]#: Method to generate phenotypic template file and subject list for group analysis

CPAC.utils.extract_data.read_csv(csv_input)[source]#

Method to read csv file: ‘Acquisition’ ‘Reference’ ‘Site’ ‘TR (seconds)’

CPAC.utils.extract_data.run(data_config)[source]#: Run method takes data_config file as the input argument

Extract Data (Multiscan)#

class CPAC.utils.extract_data_multiscan.Configuration(config_map)[source]#

CPAC.utils.extract_data_multiscan.extract_data(c, param_map)[source]#

Method to generate a CPAC input subject list python file. The method extracts anatomical functional data and scan parameters for each site( if multiple site) and for each scan and put it into a data structure read by python

Note:#

Use this tool only if the scan parameters are different for each scan as shown in the example below.

Example:#

subjects_list = [

{

‘subject_id’: ‘0021001’, ‘unique_id’: ‘session2’, ‘anat’: ‘/home/data/multiband_data/NKITRT/0021001/anat/mprage.nii.gz’, ‘rest’:{

‘RfMRI_mx_1400_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_mx_1400/rest.nii.gz’, ‘RfMRI_mx_645_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_mx_645/rest.nii.gz’, ‘RfMRI_std_2500_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_std_2500/rest.nii.gz’, },

‘scan_parameters’:{

‘TR’:{: ‘RfMRI_mx_1400_rest’: ‘1.4’, ‘RfMRI_mx_645_rest’: ‘1.4’, ‘RfMRI_std_2500_rest’: ‘2.5’, },
‘Acquisition’:{: ‘RfMRI_mx_1400_rest’: ‘/home/data/1400.txt’, ‘RfMRI_mx_645_rest’: ‘/home/data/645.txt’, ‘RfMRI_std_2500_rest’: ‘/home/data/2500.txt’, },
‘Reference’:{: ‘RfMRI_mx_1400_rest’: ‘32’, ‘RfMRI_mx_645_rest’: ‘20’, ‘RfMRI_std_2500_rest’: ‘19’, },
‘FirstTR’:{: ‘RfMRI_mx_1400_rest’: ‘7’, ‘RfMRI_mx_645_rest’: ‘15’, ‘RfMRI_std_2500_rest’: ‘4’, },
‘LastTR’:{: ‘RfMRI_mx_1400_rest’: ‘440’, ‘RfMRI_mx_645_rest’: ‘898’, ‘RfMRI_std_2500_rest’: ‘None’, },

}

]

CPAC.utils.extract_data_multiscan.generate_suplimentary_files(output_path)[source]#: Method to generate phenotypic template file and subject list for group analysis

CPAC.utils.extract_data_multiscan.read_csv(csv_input)[source]#

Method to read csv file: ‘Acquisition’ ‘Reference’ ‘Site’ ‘TR (seconds)’

CPAC.utils.extract_data_multiscan.run(data_config)[source]#: Run method takes data_config file as the input argument

Extract Parameters#

CPAC.utils.extract_parameters.grab(output_dir, scrubbing)[source]#

Method to grab all the motion parameters and power parameters file from each subject for each pipeline and merge them

Parameters:: output_dir (string) – Path to the datasink output directory of CPAC

CPAC.utils.extract_parameters.merge(output_dir, scan_name, threshold, motion_f, power_f, flag)[source]#: Method to merge power parameters and motion parameters file

CPAC.utils.extract_parameters.run(output_path, scrubbing)[source]#

Google Analytics#

CPAC.utils.ga.do_it(data, timeout)[source]#

CPAC.utils.ga.get_or_create_config()[source]#

CPAC.utils.ga.get_uid()[source]#

CPAC.utils.ga.track_config(cpac_interface)[source]#

CPAC.utils.ga.track_event(category, action, uid=None, label=None, value=0, software_version=None, timeout=2, thread=True)[source]#

Record an event with Google Analytics

Parameters:

tracking_id (str) – Google Analytics tracking ID.
category (str) – Event category.
action (str) – Event action.
uid (str) – User unique ID, assigned when popylar was installed.
label (str) – Event label.
value (int) – Event value.
software_version (str) – Records a version of the software.
timeout (float) – Maximal duration (in seconds) for the network connection to track the event. After this duration has elapsed with no response (e.g., on a slow network connection), the tracking is dropped.

CPAC.utils.ga.track_run(level='participant', participants=0)[source]#

Interfaces#

Function Interfaces#

Function interface utilities for C-PAC

class CPAC.utils.interfaces.function.Function(input_names=None, output_names='out', function=None, imports=None, as_module=False, **inputs)[source]#

Runs arbitrary function as an interface

Examples

>>> func = 'def func(arg1, arg2=5): return arg1 + arg2'
>>> fi = Function(input_names=['arg1', 'arg2'], output_names=['out'])
>>> fi.inputs.function_str = func
>>> res = fi.run(arg1=1)
>>> res.outputs.out
6

input_spec#: alias of FunctionInputSpec

output_spec#: alias of DynamicTraitedSpec

static sig_imports(imports: List[str]) → Callable[source]#

Sets an ns_imports attribute on a function for Function-node functions. This can be useful for classes needed for decorators, typehints and for avoiding redefinitions.

Parameters:: imports (list of str) – import statements to import the function in an otherwise empty namespace. If these collide with imports defined via the Function.__init__ initialization method, the imports given as a parameter here will be overridden by those from the initializer.
Returns:: func
Return type:: function

Examples

See the defintion of calculate_FD_J to see the decorator tested here being applied. >>> from CPAC.generate_motion_statistics import calculate_FD_J >>> calc_fdj = Function(input_names=[‘in_file’, ‘calc_from’, ‘center’], … output_names=[‘out_file’], … function=calculate_FD_J, … as_module=True) >>> calc_fdj.imports # doctest: +NORMALIZE_WHITESPACE [‘from CPAC.utils.interfaces.function import Function’,

‘import os’, ‘import sys’, ‘from typing import Optional’, ‘import numpy as np’, ‘from CPAC.utils.pytest import skipif’, ‘from CPAC.utils.typing import LITERAL, TUPLE’]

>>> from inspect import signature
>>> from nipype.utils.functions import (getsource,
...     create_function_from_source)
>>> f = create_function_from_source(getsource(calculate_FD_J),
...                                 calc_fdj.imports)
>>> inspect.signature(calculate_FD_J) == inspect.signature(f)
True

CPAC.utils.interfaces.function.pick_tissue_from_labels_file_interface(input_names=None)[source]#

Function to create a Function interface for CPAC.seg_preproc.utils.pick_tissue_from_labels_file

Parameters:: input_names (list, optional) –
Return type:: nipype.interfaces.base.core.Interface

Tests#

Miscellaneous#

Miscellaneous utilities

CPAC.utils.misc.get_template_specs(in_template, template_spec=None, default_resolution=1)[source]#

Parse template specifications

>>> get_template_specs('MNI152NLin2009cAsym', {'suffix': 'T1w'})[1]  
{'resolution': 1}

>>> get_template_specs('MNI152NLin2009cAsym', {'res': '2', 'suffix': 'T1w'})[1]  
{'resolution': '2'}

>>> get_template_specs('MNIInfant', {'res': '2', 'cohort': '10', 'suffix': 'T1w'})[1]  
{'resolution': '2', 'cohort': '10'}

>>> get_template_specs('MNI152NLin2009cAsym',
...                    {'suffix': 'T1w', 'cohort': 1})[1] 
Traceback (most recent call last):
RuntimeError:
...

>>> get_template_specs('MNI152NLin2009cAsym',
...                    {'suffix': 'T1w', 'res': '1|2'})[1] 
Traceback (most recent call last):
RuntimeError:
...

Monitoring#

Module to customize Nipype’s process monitoring for use in C-PAC

See https://fcp-indi.github.io/docs/developer/nodes for C-PAC-specific documentation. See https://nipype.readthedocs.io/en/latest/api/generated/nipype.utils.profiler.html for Nipype’s documentation.

class CPAC.utils.monitoring.LoggingHTTPServer(pipeline_name, logging_dir='', host='', port=8080, request=<class 'CPAC.utils.monitoring.monitoring.LoggingRequestHandler'>)[source]#

class CPAC.utils.monitoring.LoggingRequestHandler(request, client_address, server)[source]#

handle()[source]#

CPAC.utils.monitoring.failed_to_start(log_dir, exception)[source]#

Launch a failed-to-start logger for a run that failed to start. Must be called from within an except block.

Parameters:

log_dir (str) – path to logging directory
exception (Exception) –

CPAC.utils.monitoring.getLogger(name)[source]#

Function to get a mock logger if one exists, falling back on real loggers.

Parameters:: name (str) –
Returns:: logger
Return type:: CPAC.utils.monitoring.custom_logging.MockLogger or logging.Logger

CPAC.utils.monitoring.log_nodes_cb(node, status)[source]#

Function to record node run statistics to a log file as json dictionaries

Parameters:

node (nipype.pipeline.engine.Node) – the node being logged
status (string) – acceptable values are ‘start’, ‘end’; otherwise it is considered and error

Returns:

this function does not return any values, it logs the node status info to the callback logger

Return type:

None

CPAC.utils.monitoring.log_nodes_initial(workflow)[source]#

CPAC.utils.monitoring.monitor_server(pipeline_name, logging_dir, host='0.0.0.0', port=8080)[source]#

CPAC.utils.monitoring.recurse_nodes(workflow, prefix='')[source]#

CPAC.utils.monitoring.set_up_logger(name, filename=None, level=None, log_dir=None, mock=False, overwrite_existing=False)[source]#

Function to initialize a logger

Parameters:

name (str) – logger name (for subsequent calls to logging.getLogger) to write to the same log file)
filename (str, optional) – filename to write log to. If not specified, filename will be the same as name with the extension log
level (str, optional) – one of {critical, error, warning, info, debug, notset}, case-insensitive
log_dir (str, optional) –
mock (bool, optional) – if True, return a CPAC.utils.monitoring.MockLogger instead of a logging.Logger

Returns:

logger – initialized logging Handler

Return type:

logging.Handler

Examples

>>> lg = set_up_logger('test')
>>> lg.handlers[0].baseFilename.split('/')[-1]
'test.log'
>>> lg.level
0
>>> lg = set_up_logger('second_test', 'specific_filename.custom', 'debug')
>>> lg.handlers[0].baseFilename.split('/')[-1]
'specific_filename.custom'
>>> lg.level
10
>>> lg = set_up_logger('third_test', mock=True)
>>> getLogger('third_test') == lg
True
>>> 'third_test' in MOCK_LOGGERS
True
>>> lg.delete()
>>> 'third_test' in MOCK_LOGGERS
False

NeuroData’s MRI to Graphs (NDMG | m2g) Utilities#

Functions in this file adapted from NeuroData group: STATEMENT OF CHANGES:

This file is derived from sources licensed under the Apache-2.0 terms, and this file has been changed.

CHANGES:

Minor refactoring for compatibility with C-PAC

ORIGINAL WORK’S ATTRIBUTION NOTICE:

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

graph.py Created by Greg Kiar on 2016-01-27. Email: gkiar@jhu.edu

Can be found here:

This file is part of C-PAC.

class CPAC.utils.ndmg_utils.graph(N, rois, attr=None, sens='dwi')[source]#

as_matrix()[source]#: Returns the graph as a matrix.

cor_graph(timeseries, attr=None)[source]#

Takes timeseries and produces a correlation matrix Positional Arguments:

timeseries:

-the timeseries file to extract correlation for.
dimensions are [numrois]x[numtimesteps]

get_graph()[source]#: Returns the graph object created

make_graph(streamlines, attr=None)[source]#

Takes streamlines and produces a graph Positional Arguments:

streamlines:

Fiber streamlines either file or array in a dipy EuDX or compatible format.

save_graph(graphname)[source]#

Saves the graph to disk Positional Arguments:

graphname:

Filename for the graph

summary()[source]#: User friendly wrapping and display of graph properties

CPAC.utils.ndmg_utils.ndmg_create_graphs(ts, labels)[source]#

CPAC.utils.ndmg_utils.ndmg_roi_timeseries(func_file, label_file)[source]#

Function to extract average timeseries for the voxels in each roi of the labelled atlas. Returns the roi timeseries as a numpy.ndarray. Positional Arguments

func_file:

the path to the 4d volume to extract timeseries

label_file:

the path to the labelled atlas containing labels
for the voxels in the fmri image

roits_file:

the path to where the roi timeseries will be saved. If

None, don’t save and just return the roi_timeseries.

Neuroimaging Informatics Technology Initiative (NIfTI) Utilities#

CPAC.utils.nifti_utils.inverse_nifti_values(image)[source]#

Replace zeros by ones and non-zero values by 1 :param image: path to the nifti file to be inverted or

the image already loaded through nibabel

Returns:: output
Return type:: Nibabel Nifti1Image

CPAC.utils.nifti_utils.more_zeros_than_ones(image)[source]#

Return True is there is more zeros than other values in a given nifti image. :param image: path to the nifti file to be inverted or

the image already loaded through nibabel

Returns:: more_zeros
Return type:: boolean

CPAC.utils.nifti_utils.nifti_image_input(image)[source]#

Test if an input is a path or a nifti.image and the image loaded through nibabel :param image: path to the nifti file or the image already loaded through nibabel :type image: str or nibabel.nifti1.Nifti1Image

Returns:: img – load and return the nifti image if image is a path, otherwise simply return image
Return type:: nibabel.nifti1.Nifti1Image

Outputs#

class CPAC.utils.outputs.Outputs[source]#

all_native_filter = 0 True 1 True 2 True 3 True 4 False ... 214 False 215 False 216 True 217 True 218 False Name: Space, Length: 219, dtype: bool#

all_template_filter = 0 False 1 False 2 False 3 False 4 True ... 214 True 215 True 216 False 217 False 218 False Name: Space, Length: 219, dtype: bool#

anat = ['space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'space-template_desc-brain_mask', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'dseg']#

any = ['alff', 'desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-T1w_sbref', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'space-template_desc-brain_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'motion', 'desc-summary_motion', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'space-template_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'space-fsLR_den-32k_bold-dtseries', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'vmhc', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask', 'mdmr', 'desc-zstd-mdmr', 'dseg']#

bold_ts = ['desc-brain_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold']#

debugging = ['desc-brain_bold', 'desc-motion_bold', 'desc-sm_bold', 'space-EPItemplate_desc-brain_bold', 'space-symtemplate_desc-sm_bold', 'space-template_desc-brain_bold', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-reorient_T1w', 'space-template_desc-brain_T1w']#

func = ['alff', 'desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-T1w_sbref', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'motion', 'desc-summary_motion', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'space-template_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'space-fsLR_den-32k_bold-dtseries', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'vmhc', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask']#

functional_timeseries = ['desc-brain_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-preproc_bold']#

native_nonsmooth = ['alff', 'falff', 'reho']#

native_raw = ['alff', 'desc-sm_alff', 'desc-sm_falff', 'falff', 'desc-sm_reho', 'reho', 'desc-zstd-mdmr']#

native_smooth = ['desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-T1w_sbref', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'motion', 'desc-summary_motion', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'space-fsLR_den-32k_bold-dtseries', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'mdmr', 'desc-zstd-mdmr', 'dseg']#

reference = Resource ... Multi-File 0 alff ... 1 desc-sm_alff ... 2 desc-sm-zstd_alff ... 3 desc-zstd_alff ... 4 space-template_alff ... .. ... ... ... 214 space-EPItemplate_label-WM_mask ... 215 space-EPItemplate_label-GM_mask ... 216 mdmr ... 217 desc-zstd-mdmr ... 218 dseg ... [219 rows x 10 columns]#

reference_csv = '/home/circleci/docs_env/lib/python3.10/site-packages/CPAC/resources/cpac_outputs.tsv'#

template_nonsmooth = ['space-template_alff', 'space-template_dcb', 'space-template_dcw', 'space-template_ecb', 'space-template_ecw', 'space-template_falff', 'space-template_lfcdb', 'space-template_lfcdw', 'space-template_reho']#

template_raw = ['space-template_alff', 'space-template_desc-sm_alff', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm_reho', 'space-template_reho']#

template_smooth = ['space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-template_desc-brain_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'space-symtemplate_desc-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'vmhc', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask']#

to_fisherz = []#

to_smooth = ['alff', 'space-template_alff', 'space-template_dcb', 'space-template_dcw', 'space-template_ecb', 'space-template_ecw', 'falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_lfcdw', 'reho', 'space-template_reho']#

to_zstd = ['alff', 'desc-sm_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'desc-sm_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'desc-sm_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_reho', 'desc-zstd-mdmr']#

pytest#

Utilities for Pytest integration

CPAC.utils.pytest.skipif(condition, reason)[source]#: Skip test if we have Pytest, ignore test entirely if not

Serialization#

Strategy#

class CPAC.utils.strategy.Strategy[source]#

append_name(name)[source]#

fork()[source]#

get(resource_key)[source]#

static get_forking_labels(strategies)[source]#

static get_forking_points(strategies)[source]#

get_leaf_properties()[source]#

get_name()[source]#

get_node_from_resource_pool(resource_key)[source]#

get_nodes_names()[source]#

get_resource_pool()[source]#

property resource_pool#: Strategy’s ResourcePool dict

property rpool#: Strategy’s ResourcePool

set_leaf_properties(node, out_file)[source]#

update_resource_pool(resources, override=False)[source]#

Symlinks#

CPAC.utils.symlinks.compile_strategy_name(strategy)[source]#

CPAC.utils.symlinks.create_paths_to_symlinks(output_dir, pipeline_id, subject_id, paths)[source]#

CPAC.utils.symlinks.create_symlinks(output_dir, pipeline_id, subject_id, paths, relative=True, symlink_dir='sym_links')[source]#

CPAC.utils.symlinks.group_files_in_strategies(output_dir, paths)[source]#

Test Initialization#

This module contains functions that assist in initializing CPAC tests resources

CPAC.utils.test_init.concordance(x, y)[source]#

Return the concordance correlation coefficient as defined by Lin (1989)

Parameters:

x (list or array) – a list of array of length N of numbers
y (list or array) – a list of array of length N of numbers

Returns:

rho_c – the concordance value as a float

Return type:

numpy.float32

CPAC.utils.test_init.create_dummy_node(name: str, fields: list[str] | None = None)[source]#

Create a dummy IdentityInterface Node source for resources upstream in a graph from a section to be tested

Parameters:

name (str) – a name for the dummy Node
fields (list of str, optional) – a list of resources to be present in the created Node. If not provided, the only resource will be called ‘resource’

Return type:

Node

CPAC.utils.test_init.default_bucket_name()[source]#

Function to return the default S3 bucket name used in test suite

Parameters:: None –
Returns:: bucket_name – default S3 bucket name for testing
Return type:: string

CPAC.utils.test_init.download_cpac_resources_from_s3(local_base)[source]#

Function to download the CPAC testing resources directory from S3

Parameters:: local_base (string) – the local directory to save the ‘cpac_resources’ contents

CPAC.utils.test_init.download_resource_from_s3(s3_url_path)[source]#

CPAC.utils.test_init.pearson_correlation(nii_1, nii_2)[source]#

CPAC.utils.test_init.populate_all_templates()[source]#

Function to populate all of the template files

Parameters:: None –
Return type:: None

CPAC.utils.test_init.populate_template_config(config_type: str) → str[source]#

Function to read in a template config file from the CPAC_RESOURCE_DIR and populate it with actual filepaths

Parameters:: config_type (string) – config file to populate; accepts ‘data_config’ and ‘pipeline_config’
Returns:: config_test – filepath to the newly written config file for testing
Return type:: string

CPAC.utils.test_init.return_all_niis(base_dir)[source]#

Function to walk through a base directory and all subsequent files and return the filepaths of all nifti files found

Parameters:: base_dir (string) – filepath to the base directory to search through
Returns:: nii_list – a list of filepath strings of the nifti files found in base_dir
Return type:: list

CPAC.utils.test_init.return_aws_creds()[source]#

Function to return the AWS credentials file given by the CPAC_AWS_CREDS environment variable

Parameters:: None –
Returns:: aws_creds – filepath to the AWS credentials with access key id and secret access key
Return type:: string

CPAC.utils.test_init.return_resource_dir()[source]#

Function to return the filepath of the CPAC_RESOURCE_DIR; note the CPAC_RESOURCE_DIR environment variable must be set

Parameters:: None –
Returns:: resource_dir – the file path on disk where the cpac resources folder is
Return type:: string

CPAC.utils.test_init.return_resource_subfolder(subfolder)[source]#

Funnction to return subfolders of the CPAC_RESOURCE_DIR

Parameters:: subfolder (string) – subfolder name to return path of
Returns:: resource_subfolder – filepath to the resource subfolder
Return type:: string

CPAC.utils.test_init.return_strats_obj()[source]#

Function to return the file path of the strategies obj file from the CPAC_RESOURCE_DIR

Parameters:: None –
Returns:: strats_obj – filepath to the strategies obj file
Return type:: string

CPAC.utils.test_init.return_subj_measure_dirs(measure)[source]#

Function to grab the base directories of the test subject’s output files for a given measure or workflow

Parameters:: measure (string) – the measure or workflow or derivative of interest to parse for; this must be the folder name where all of the subject’s test outputs are located (e.g. ‘network_centrality’)
Returns:: subj_measure_dirs – a list of strings of the base directories for each instance of the desired measure folder within the test subjects outputs
Return type:: list

CPAC.utils.test_init.return_subject_list()[source]#

Function to return the file path of the subject list file from the CPAC_RESOURCE_DIR

Parameters:: None –
Returns:: subject_list – filepath to the subject list yaml file
Return type:: string

CPAC.utils.test_init.return_test_subj()[source]#

Function to return the subject id; note the CPAC_RESOURCE_DIR environment variable must be set

Parameters:: None –
Returns:: resource_dir – the file path on disk where the cpac resources folder is
Return type:: string

CPAC.utils.test_init.setup_test_logger(logger_name, log_file, level, to_screen=False)[source]#

Function to initialize and configure a logger that can write to file and (optionally) the screen.

Parameters:

logger_name (string) – name of the logger
log_file (string) – file path to the log file on disk
level (integer) – indicates the level at which the logger should log; this is controlled by integers that come with the python logging package. (e.g. logging.INFO=20, logging.DEBUG=10)
to_screen (boolean (optional)) – flag to indicate whether to enable logging to the screen

Returns:

logger – Python logging.Logger object which is capable of logging run- time information about the program to file and/or screen

Return type:

logging.Logger object

CPAC.utils.test_init.smooth_nii_file(self, nii_file, fwhm, mask_file=None)[source]#

Function to Gaussian smooth nifti files and optionally using a mask on the smoothed data

Parameters:

nii_file (string) – filepath to the nifti file to smooth
fwhm (float) – FWHM for Gaussian smoothing kernel, in mm
mask_file (string (optional); default=None) – filepath to the mask file to use

Returns:

smooth_arr – smoothed nifti image as a numpy array

Return type:

numpy.ndarray

Test Mocks#

CPAC.utils.test_mocks.configuration_strategy_mock(method='FSL')[source]#

CPAC.utils.test_mocks.file_node(path, file_node_num=0)[source]#

Test Resources#

CPAC.utils.test_resources.setup_test_wf(s3_prefix, paths_list, test_name, workdirs_to_keep=None)[source]#: Set up a basic template Nipype workflow for testing single nodes or small sub-workflows.

Tests#

The Trimmer#

CPAC.utils.trimmer.compute_datasink_dirs(graph, datasink, output_dir=None, container=None)[source]#

CPAC.utils.trimmer.expand_workflow(wf)[source]#

CPAC.utils.trimmer.is_datasink(n)[source]#

CPAC.utils.trimmer.list_files(path, s3_creds_path=None)[source]#

CPAC.utils.trimmer.the_trimmer(wf, output_dir=None, container=None, s3_creds_path=None)[source]#

The trimmer: trimming your workflow based on its datasinks.

The assumption here is that all your desired outputs will be placed in an output directory by using a DataSink node.

By analysing a specific output directory, and comparing what is in it with the DataSinks of a workflow, it is possible to audit which Datasinks have already outputted to the output directory. If the DataSink has already fulfilled its function, we infer that previous nodes also had, since they are prerequisites for the DataSink to run. This is the simplest case in which we can prune nodes (i.e. do not execute them), optimizing the execution time.

A brief syntax note: [node] is a node → is a connection, disconsidering which field/attribute →(field)→ indicates a connection considering the field ✓ is a datasink with an existing file ❌ is a datasing witn an missing file

E.g.

[node1] → [node2] → [node3] → [datasink to file.txt ✓]

since file.txt already exist, there is no need to execute the [node3]. Since [node2] only outputs to [node3], and [node3] will not be executed, we can prune [node2]. Same for [node1]. In this case, our workflow will not have any more nodes to run.

There are more complex cases:

1) A node outputs for several nodes, and some of their results are not in the output directory.

[node1] → [node2] → [node3] → [datasink to file1.txt ✔]: ↳ [node4] → [datasink to file2.txt ❌]

for this case, we cannot prune [node1], since its output is used in another branch, for [node4], that is not cached. After trimming, the remaining workflow is:

[node1]: ↳ [node4] → [datasink to file2.txt ❌]

2) The node has several outputs, and an uncached branch down the graph requires one of its outputs.

[registration] →(warped image)→ [datasink to warped.nii.gz ✔]: ↳(transforms)→ [apply transforms] → [datasink to func_warped.nii.gz ❌] [functional] ↗

given func_warped.nii.gz is not cached, we need to perform “apply transforms”, that requires the transforms from the [registration] node. In this case, even that warped.nii.gz is cached, we will reexecute the [registration] again to get the transforms. After trimming, the remaining workflow is:

[registration]: ↳(transforms)→ [apply transforms] → [datasink to func_warped.nii.gz ❌] [functional] ↗

For this implementation, we disregard MapNodes, as their outputs is harder to check.

Iterables are considered in the implementation by expanding the original workflow into what is called an execution graph, creating a node for each iterable value.

Parameters:

wf (Workflow) – A Nipype workflow to be pruned.
output_dir (Path) – The directory in which the outputs are stored. If not provided, value is inferred from the DataSink nodes.
container (Path) – The subdirectory from the output_dir in which the output are stored. If not provided, value is inferred from the DataSink nodes.
s3_creds_path (Path) – Path to S3 credentials, in case output_dir is in a S3 bucket.

Returns:

wf_new (Workflow) – Prunned workflow
(replacement_mapping, deletions) ((Dict, List)) – replacement_mapping contains the nodes replaces with input nodes, pointing to files from the output_dir

deletions contains the nodes removed from the workflow, as they do not need to be executed

Typing#

Helpers and aliases for handling typing in main and variant Python versions

Once all variants (see https://fcp-indi.github.io/docs/latest/user/versions#variants) run Python ≥ 3.10, these global variables can be replaced with the current preferred syntax.

CPAC.utils.typing.ITERABLE#: alias of Iterable

CPAC.utils.typing.LIST#: alias of list

CPAC.utils.typing.TUPLE#: alias of tuple

Utilities#

CPAC.utils.utils.add_afni_prefix(tpattern)[source]#

CPAC.utils.utils.check(params_dct, subject_id, scan_id, val_to_check, throw_exception)[source]#

CPAC.utils.utils.check_command_path(path)[source]#

CPAC.utils.utils.check_config_resources(c)[source]#

CPAC.utils.utils.check_prov_for_motion_tool(prov)[source]#

CPAC.utils.utils.check_prov_for_regtool(prov)[source]#

CPAC.utils.utils.check_random_state(seed)[source]#

Turn seed into a np.random.RandomState instance Code from scikit-learn (https://github.com/scikit-learn/scikit-learn)

Parameters:: seed (None | int | instance of RandomState) – If seed is None, return the RandomState singleton used by np.random. If seed is an int, return a new RandomState instance seeded with seed. If seed is already a RandomState instance, return it. Otherwise raise ValueError.

CPAC.utils.utils.check_system_deps(check_ants=False, check_ica_aroma=False, check_centrality_degree=False, check_centrality_lfcd=False)[source]#: Function to check system for neuroimaging tools AFNI, C3D, FSL, and (optionally) ANTs

CPAC.utils.utils.check_tr(tr, in_file)[source]#

CPAC.utils.utils.compute_fisher_z_score(correlation_file, timeseries_one_d, input_name)[source]#

Computes the fisher z transform of the input correlation map If the correlation map contains data for multiple ROIs then the function returns z score for each ROI as a seperate nifti file

Parameters:: correlation_file (string) – Input correlations file
Returns:: out_file – list of z_scores for mask or ROI
Return type:: list (nifti files)

CPAC.utils.utils.concat_list(in_list1=None, in_list2=None)[source]#

Parameters:

in_list1 (list or str) – file path or a list of file paths
in_list2 (list or str) – file path or a list of file paths

Returns:

out_list – a list of file paths

Return type:

CPAC.utils.utils.correlation(matrix1, matrix2, match_rows=False, z_scored=False, symmetric=False)[source]#

CPAC.utils.utils.create_id_string(cfg, unique_id, resource, scan_id=None, template_desc=None, atlas_id=None, fwhm=None, subdir=None)[source]#

Create the unique key-value identifier string for BIDS-Derivatives compliant file names.

This is used in the file renaming performed during the Datasink connections.

Example

>>> from CPAC.utils.configuration import Configuration
>>> create_id_string(Configuration(), 'sub-1_ses-1',
...                  'res-derivative_desc-Mean-1_timeseries',
...                  scan_id='rest', atlas_id='Yeo_desc-7')
'sub-1_ses-1_task-rest_atlas-Yeo7_res-3mm_desc-Mean1_timeseries'

CPAC.utils.utils.create_log(wf_name='log', scan_id=None)[source]#: Workflow to create log

CPAC.utils.utils.create_output_mean_csv(subject_dir)[source]#: this function finds all of the mean_{output}.txt files in the subject’s output directory, collects the data and organizes them into one .csv file in the subject directory

CPAC.utils.utils.delete_nested_value(d, keys)[source]#

Helper function to delete nested values

Parameters:

d (dict) –
keys (list or tuple) –

Returns:

updated

Return type:

Examples

>>> delete_nested_value(
...     {'nested': {'key1': 'value', 'key2': 'value'}},
...     ['nested', 'key1'])
{'nested': {'key2': 'value'}}

CPAC.utils.utils.extract_one_d(list_timeseries)[source]#

CPAC.utils.utils.extract_output_mean(in_file, output_name)[source]#

function takes ‘in_file’, which should be an intermediary 1D file from individual-level analysis, containing the mean of the output across all voxels

it then parses this value and writes it to a .csv file named output_means.csv located in the subject’s output directory

CPAC.utils.utils.extract_txt(list_timeseries)[source]#: Method to extract txt file containing roi timeseries required for dual regression

CPAC.utils.utils.find_files(directory, pattern)[source]#

CPAC.utils.utils.get_fisher_zscore(input_name, map_node=False, wf_name='fisher_z_score')[source]#: Runs the compute_fisher_z_score function as part of a one-node workflow.

CPAC.utils.utils.get_flag(in_flag)[source]#

CPAC.utils.utils.get_flag_wf(wf_name='get_flag')[source]#

CPAC.utils.utils.get_last_prov_entry(prov)[source]#

CPAC.utils.utils.get_operand_string(mean, std_dev)[source]#

Method to get operand string for Fsl Maths

Parameters:

mean (string) – path to img containing mean
std_dev (string) – path to img containing standard deviation

Returns:

op_string – operand string

Return type:

string

CPAC.utils.utils.get_roi_num_list(timeseries_file, prefix=None)[source]#

CPAC.utils.utils.get_scan_params(subject_id, scan, pipeconfig_start_indx, pipeconfig_stop_indx, data_config_scan_params=None)[source]#

Method to extract slice timing correction parameters and scan parameters.

Parameters:

subject_id (str) – subject id
scan (str) – scan id
pipeconfig_start_indx (int) – starting volume index as provided in the pipeline config yaml file
pipeconfig_stop_indx (int) – ending volume index as provided in the pipeline config yaml file
data_config_scan_params (str) – file path to scan parameter JSON file listed in data config yaml file

Returns:

TR (a string) – TR value
pattern (a string) – slice aquisition pattern string or file path
ref_slice (an integer) – reference slice which is used to allign all other slices
first_tr (an integer) – starting TR or starting volume index
last_tr (an integer) – ending TR or ending volume index
pe_direction (str)
effective_echo_spacing (float)

CPAC.utils.utils.get_tr(tr)[source]#: Method to return TR in seconds

CPAC.utils.utils.get_zscore(map_node=False, wf_name='z_score')[source]#

Workflow to calculate z-scores

Parameters:: wf_name (string) – name of the workflow
Returns:: wf
Return type:: workflow object

Notes

Source

Workflow Inputs:

inputspec.input_file : string
    path to input functional derivative file for which z score has to be calculated
inputspec.mask_file : string
    path to whole brain functional mask file required to calculate zscore

Workflow Outputs:

outputspec.z_score_img : string
     path to image containing Normalized Input Image Z scores across full brain.

High Level Workflow Graph:

Detailed Workflow Graph:

Example

>>> wf = get_zscore('example_input')
>>> wf.inputs.inputspec.input_file = '/home/data/graph_working_dir/calculate_centrality/degree_centrality_binarize.nii.gz'
>>> wf.inputs.inputspec.mask_file = '/home/data/graphs/GraphGeneration/new_mask_3m.nii.gz'
>>> wf.run()  

CPAC.utils.utils.list_item_replace(l, old, new)[source]#

Function to replace an item in a list

Parameters:

l (list or string) –
old (any) – item to replace
new (any) – new item

Returns:

l – updated

Return type:

list or string

Examples

>>> list_item_replace(['AFNI', 'FSL'], 'AFNI', '3dSkullStrip')
['3dSkullStrip', 'FSL']
>>> list_item_replace(['AFNI', 'FSL'], 'FSL', 'BET')
['AFNI', 'BET']

CPAC.utils.utils.lookup_nested_value(d, keys)[source]#

Helper method to look up nested values

Parameters:

d (dict) –
keys (list or tuple) –

Returns:

yaml

Return type:

str or dict

Examples

>>> lookup_nested_value({'nested': {'True': True}}, ['nested', 'True'])
True
>>> lookup_nested_value({'nested': {'None': None}}, ['nested', 'None'])
''

CPAC.utils.utils.ordereddict_to_dict(value)[source]#: this function convert ordereddict into regular dict

CPAC.utils.utils.pick_wm(seg_prob_list)[source]#

CPAC.utils.utils.read_json(json_file)[source]#

CPAC.utils.utils.remove_False(d, k)[source]#

Function to remove “Off” and False from a list at a given nested key.

Parameters:

d (dict) –
k (list) –

Returns:

d – updated

Return type:

Examples

>>> remove_False({'a': {'b': [1, False, 2, "Off", 3]}}, ['a', 'b'])
{'a': {'b': [1, 2, 3]}}

CPAC.utils.utils.remove_None(d, k)[source]#

Function to remove “None” and None from a list at a given nested key.

Parameters:

d (dict) –
k (list) –

Returns:

d – updated

Return type:

Examples

>>> remove_None({'a': {'b': [1, None, 2, "None", 3]}}, ['a', 'b'])
{'a': {'b': [1, 2, 3]}}

CPAC.utils.utils.repickle(directory)[source]#

Function to check all of the pickles in a given directory, recursively, and convert any Python 2 pickles found to Python 3 pickles.

Parameters:: directory (str) –
Return type:: None

CPAC.utils.utils.replace_in_strings(d, replacements=None)[source]#

Helper function to recursively replace substrings.

Parameters:

d (any) –
replacements (list of 2-tuples) –

0str
substring to replace

1str
replacement substring

Returns:

d – same as input, but updated

Return type:

any

Examples

>>> replace_in_strings({'key': 'test${resolution_for_func_preproc}'})
{'key': 'test${func_resolution}'}

CPAC.utils.utils.safe_shape(*vol_data)[source]#

Checks if the volume (first three dimensions) of multiple ndarrays are the same shape.

Parameters:

vol_data0 (ndarray) – Volumes to check
vol_data1 (ndarray) – Volumes to check
... (ndarray) – Volumes to check
vol_datan (ndarray) – Volumes to check

Returns:

same_volume – True only if all volumes have the same shape.

Return type:

bool

CPAC.utils.utils.set_nested_value(d, keys, value)[source]#

Helper method to set nested values

Parameters:

d (dict) –
keys (list or tuple) –
value (any) –

Returns:

updated

Return type:

Examples

>>> set_nested_value({}, ['nested', 'keys'], 'value')
{'nested': {'keys': 'value'}}

CPAC.utils.utils.try_fetch_parameter(scan_parameters, subject, scan, keys)[source]#

CPAC.utils.utils.update_config_dict(old_dict)[source]#

Function to convert an old config dict to a new config dict

Parameters:

old_dict (dict) –

Returns:

new_dict (dict) – 1.8 nested config dictionary
old_dict (dict) – remaining undefined mappings
combined_dict (dict) – 1.8 nested config dictionary plus remaining undefined mappings

Examples

>>> a, b, c = update_config_dict({
...     'pipelineName': 'example-pipeline', '2': None})
>>> a
{'pipeline_setup': {'pipeline_name': 'example-pipeline'}}
>>> b
{'2': None}
>>> c
{'pipeline_setup': {'pipeline_name': 'example-pipeline'}, '2': None}

CPAC.utils.utils.update_nested_dict(d_base, d_update, fully_specified=False)[source]#

Update dictionary of varying depth.

Parameters:

d_base (dict) – original dictionary
d_update (dict) – dictionary with updates
fully_specified (bool) – if True, overwrite instead of update

Returns:

d_base – original dictionary with updates

Return type:

Examples

>>> d_base = {'pipeline_name': 'cpac-default-pipeline',
...     'output_directory': {'path': '/output',
...     'write_func_outputs': False,
...     'write_debugging_outputs': False,
...     'output_tree': 'default',
...     'quality_control': {
...         'generate_quality_control_images': True,
...         'generate_xcpqc_files': True}},
...     'working_directory': {'path': '/tmp', 'remove_working_dir': True},
...     'log_directory': {'run_logging': True, 'path': '/logs'},
...     'system_config': {'maximum_memory_per_participant': 1,
...     'max_cores_per_participant': 1,
...     'num_ants_threads': 4,
...     'num_participants_at_once': 1},
...     'Amazon-AWS': {'aws_output_bucket_credentials': None,
...                    's3_encryption': False}}
>>> d_update = {'pipeline_name': 'cpac_fmriprep-options',
...     'system_config': {'num_ants_threads': 1},
...     'Amazon-AWS': {'s3_encryption': True}}
>>> str(update_nested_dict(d_base, d_update)) == str({
...     'pipeline_name': 'cpac_fmriprep-options', 'output_directory': {
...         'path': '/output', 'write_func_outputs': False,
...         'write_debugging_outputs': False, 'output_tree': 'default',
...         'quality_control': {
...             'generate_quality_control_images': True,
...             'generate_xcpqc_files': True}
...     }, 'working_directory': {
...        'path': '/tmp', 'remove_working_dir': True
...     }, 'log_directory': {'run_logging': True, 'path': '/logs'},
...     'system_config': {
...     'maximum_memory_per_participant': 1,
...     'max_cores_per_participant': 1,
...     'num_ants_threads': 1, 'num_participants_at_once': 1
... }, 'Amazon-AWS': {
...     'aws_output_bucket_credentials': None, 's3_encryption': True}})
True
>>> tse_base = {'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/CC400.nii.gz': 'Avg',
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Avg'
... }, 'realignment': 'ROI_to_func'}}
>>> str(update_nested_dict(tse_base, {})) == str({
...     'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...         '/cpac_templates/CC400.nii.gz': 'Avg',
...         '/cpac_templates/aal_mask_pad.nii.gz': 'Avg'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'tse_roi_paths': {'/cpac_templates/rois_3mm.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/rois_3mm.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'roi_paths_fully_specified': False,
...     'tse_roi_paths': {'/cpac_templates/rois_3mm.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/CC400.nii.gz': 'Avg',
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Avg',
...     '/cpac_templates/rois_3mm.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'roi_paths_fully_specified': False,
...     'tse_roi_paths': {'/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True,
...     'tse_roi_paths': {
...         '/cpac_templates/CC400.nii.gz': 'Avg',
...         '/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'tse_roi_paths': {'/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True

CPAC.utils.utils.update_pipeline_values_1_8(d_old)[source]#

Function to update pipeline config values that changed from C-PAC 1.7 to 1.8.

Parameters:: d_old (dict) –
Returns:: d – updated
Return type:: dict

Examples

>>> update_pipeline_values_1_8({'segmentation': {'tissue_segmentation': {
...     'using': ['FSL-FAST Thresholding', 'Customized Thresholding']}}})
{'segmentation': {'tissue_segmentation': {'using': ['FSL-FAST'], 'FSL-FAST': {'thresholding': {'use': 'Custom'}}}}}
>>> update_pipeline_values_1_8({'segmentation': {'tissue_segmentation': {
...     'using': ['FSL-FAST Thresholding']}}})
{'segmentation': {'tissue_segmentation': {'using': ['FSL-FAST'], 'FSL-FAST': {'thresholding': {'use': 'Auto'}}}}}

CPAC.utils.utils.update_values_from_list(d_old, last_exception=None)[source]#

Function to convert 1-length lists of an expected type to single items of that type, or to convert singletons of an expected list of a type into lists thereof. Also handles some type conversions against the schema.

Parameters:

d_old (dict) –
last_exception (Exception or None) – if the same exception recurs, raise it.

Returns:

d – updated

Return type: