Utilities

BIDS Utilities

CPAC.utils.bids_utils.bids_decode_fname(file_path, dbg=False, raise_error=True)

CPAC.utils.bids_utils.bids_entities_from_filename(filename)

Function to collect a list of BIDS entities from a given filename.

Parameters:

filename (str) –

Returns:

entities

Return type:

list

Examples

>>> bids_entities_from_filename(
...     's3://fake/data/sub-0001/ses-NFB3/func/'
...     'sub-0001_ses-NFB3_task-MSIT_bold.nii.gz')
['sub-0001', 'ses-NFB3', 'task-MSIT', 'bold']

CPAC.utils.bids_utils.bids_gen_cpac_sublist(bids_dir, paths_list, config_dict, creds_path, dbg=False, raise_error=True, only_one_anat=True)

Generates a CPAC formatted subject list from information contained in a BIDS formatted set of data.

Parameters:

• bids_dir (str) – base directory that contains all of the data, this could be a directory that contains data for a multiple BIDS datasets, in which case the intervening directories will be interpreted as site names

• paths_list (str) – lists of all nifti files found in bids_dir, these paths are relative to bids_dir

• config_dict (dict) – dictionary that contains information from the JSON sidecars found in bids_dir, keys are relative paths and values are dictionaries containing all of the parameter information. if config_dict is None, the subject list will be built without the parameters

• creds_path (str) – if using S3 bucket, this path credentials needed to access the bucket, if accessing anonymous bucket, this can be set to None

• dbg (bool) – indicating whether or not the debug statements should be printed

• raise_error (bool) –

• only_one_anat (bool) – The “anat” key for a subject expects a string value, but we can temporarily store a list instead by passing True here if we will be filtering that list down to a single string later

Returns:

a list of dictionaries suitable for use by CPAC to specify data to be processed

Return type:

list

CPAC.utils.bids_utils.bids_match_entities(file_list, entities, suffix)

Function to subset a list of filepaths by a passed BIDS entity.

Parameters:

• file_list (list of str) –

• entities (str) – BIDS entities joined by underscores (e.g., ‘ses-001_task-PEER1’)

• suffix (str) – BIDS suffix (e.g., ‘bold’, ‘T1w’)

Return type:

list of str

Examples

>>> bids_match_entities([
...     's3://fake/data/sub-001_ses-001_task-MSIT_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ], 'task-PEER1', 'bold')
['s3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz']
>>> bids_match_entities([
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ], 'PEER', 'bold')
Traceback (most recent call last):
LookupError: No match found for provided entity "PEER" in
- s3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz
- s3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz
Perhaps you meant one of these?
- task-PEER1
- task-PEER2

CPAC.utils.bids_utils.bids_parse_sidecar()

Uses the BIDS principle of inheritance to build a data structure that maps parameters in side car .json files to components in the names of corresponding nifti files.

Parameters:

• config_dict – dictionary that maps paths of sidecar json files (the key) to a dictionary containing the contents of the files (the values)

• dbg – boolean flag that indicates whether or not debug statements should be printed

Returns:

a dictionary that maps parameters to components from BIDS filenames such as sub, sess, run, acq, and scan type

CPAC.utils.bids_utils.bids_remove_entity(name, key)

Remove an entity from a BIDS string by key

Parameters:

• name (str) – BIDS string to remove entity from

• key (str) – BIDS key of entity to remove

Returns:

BIDS name with entity removed

Return type:

str

Examples

>>> bids_remove_entity('atlas-Yeo_space-MNI152NLin6_res-2x2x2', 'space')
'atlas-Yeo_res-2x2x2'
>>> bids_remove_entity('atlas-Yeo_space-MNI152NLin6_res-2x2x2', 'res')
'atlas-Yeo_space-MNI152NLin6'

CPAC.utils.bids_utils.bids_retrieve_params(bids_config_dict, f_dict, dbg=False)

Retrieve the BIDS parameters from bids_config_dict for BIDS file corresponding to f_dict. If an exact match for f_dict is not found the nearest match is returned, corresponding to the BIDS inheritance principle.

Parameters:

• bids_config_dict – BIDS configuration dictionary, this is a multi-level dictionary that maps the components of a bids filename (i.e. sub, ses, acq, run) to a dictionary that contains the BIDS parameters (RepetitionTime, EchoTime, etc). This information is extracted from sidecar json files using the principle of inheritance using the bids_parse_configs function

• f_dict – Dictionary built from the name of a file in the BIDS format. This is built using the bids_decode_fname by splitting on “-” and “_” delimeters

• dbg – boolean flag that indicates whether or not debug statements should be printed, defaults to “False”

Returns:

returns a dictionary that contains the BIDS parameters

CPAC.utils.bids_utils.bids_shortest_entity(file_list)

Function to return the single file with the shortest chain of BIDS entities from a given list, returning the first if more than one have the same minimum length.

Parameters:

file_list (list of strings) –

Return type:

str or None

Examples

>>> bids_shortest_entity([
...     's3://fake/data/sub-001_ses-001_task-MSIT_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER1_bold.nii.gz',
...     's3://fake/data/sub-001_ses-001_task-PEER2_bold.nii.gz'
... ])
's3://fake/data/sub-001_ses-001_bold.nii.gz'

CPAC.utils.bids_utils.camelCase(string: str) → str

Convert a hyphenated string to camelCase

Parameters:

string (str) – string to convert to camelCase

Return type:

str

Examples

>>> camelCase('PearsonNilearn-aCompCor')
'PearsonNilearnACompCor'
>>> camelCase('mean-Pearson-Nilearn-aCompCor')
'meanPearsonNilearnACompCor'

CPAC.utils.bids_utils.cl_strip_brackets(arg_list)

Removes ‘[’ from before first and ‘]’ from after final arguments in a list of commandline arguments

Parameters:

arg_list (list) –

Return type:

list

Examples

>>> cl_strip_brackets('[a b c]'.split(' '))
['a', 'b', 'c']
>>> cl_strip_brackets('a b c'.split(' '))
['a', 'b', 'c']
>>> cl_strip_brackets('[ a b c ]'.split(' '))
['a', 'b', 'c']

CPAC.utils.bids_utils.collect_bids_files_configs(bids_dir, aws_input_creds='')

Parameters:

• bids_dir –

• aws_input_creds –

Returns:

CPAC.utils.bids_utils.combine_multiple_entity_instances(bids_str: str) → str

Combines mutliple instances of a key in a BIDS string to a single instance by camelCasing and concatenating the values

Parameters:

bids_str (str) –

Return type:

str

Examples

>>> combine_multiple_entity_instances(
...     'sub-1_ses-HBN_site-RU_task-rest_atlas-AAL_'
...     'desc-Nilearn_desc-36-param_suffix.ext')
'sub-1_ses-HBN_site-RU_task-rest_atlas-AAL_desc-Nilearn36Param_suffix.ext'
>>> combine_multiple_entity_instances(
...     'sub-1_ses-HBN_site-RU_task-rest_'
...     'run-1_framewise-displacement-power.1D')
'sub-1_ses-HBN_site-RU_task-rest_run-1_framewiseDisplacementPower.1D'

CPAC.utils.bids_utils.create_cpac_data_config(bids_dir, participant_labels=None, aws_input_creds=None, skip_bids_validator=False, only_one_anat=True)

Create a C-PAC data config YAML file from a BIDS directory.

Parameters:

• bids_dir (str) –

• participant_labels (list or None) –

• aws_input_creds –

• skip_bids_validator (bool) –

• only_one_anat (bool) – The “anat” key for a subject expects a string value, but we can temporarily store a list instead by passing True here if we will be filtering that list down to a single string later

Return type:

list

CPAC.utils.bids_utils.gen_bids_outputs_sublist(base_path, paths_list, key_list, creds_path)

CPAC.utils.bids_utils.insert_entity(resource, key, value)

Insert a f’{key}-{value}’ BIDS entity before desc- if present or before the suffix otherwise

Parameters:

• resource (str) –

• key (str) –

• value (str) –

Return type:

str

Examples

>>> insert_entity('run-1_desc-preproc_bold', 'reg', 'default')
'run-1_reg-default_desc-preproc_bold'
>>> insert_entity('run-1_bold', 'reg', 'default')
'run-1_reg-default_bold'
>>> insert_entity('run-1_desc-preproc_bold', 'filt', 'notch4c0p31bw0p12')
'run-1_filt-notch4c0p31bw0p12_desc-preproc_bold'
>>> insert_entity('run-1_reg-default_bold', 'filt', 'notch4c0p31bw0p12')
'run-1_reg-default_filt-notch4c0p31bw0p12_bold'

CPAC.utils.bids_utils.load_cpac_data_config(data_config_file, participant_labels, aws_input_creds)

Loads the file as a check to make sure it is available and readable

Parameters:

• data_config_file (str) – path to data config

• participants_labels (list or None) –

• aws_input_creds –

Return type:

list

CPAC.utils.bids_utils.load_yaml_config(config_filename, aws_input_creds)

CPAC.utils.bids_utils.res_in_filename(cfg, label)

Specify resolution in filename

Parameters:

• cfg (CPAC.utils.configuration.Configuration) –

• label (str) –

Returns:

label

Return type:

str

Examples

>>> from CPAC.utils.configuration import Configuration
>>> res_in_filename(Configuration({
...     'registration_workflows': {
...         'anatomical_registration': {'resolution_for_anat': '2x2x2'}}}),
...     'sub-1_res-anat_bold')
'sub-1_res-2x2x2_bold'
>>> res_in_filename(Configuration({
...     'registration_workflows': {
...         'anatomical_registration': {'resolution_for_anat': '2x2x2'}}}),
...     'sub-1_res-3mm_bold')
'sub-1_res-3mm_bold'

CPAC.utils.bids_utils.sub_list_filter_by_labels(sub_list, labels)

Function to filter a sub_list by provided BIDS labels for specified suffixes

Parameters:

• sub_list (list) –

• labels (dict) –

• labels['T1w'] (str or None) – C-PAC currently only uses a single T1w image

• labels['bold'] (str, list, or None) –

Return type:

list

CPAC.utils.bids_utils.with_key(entity: str, key: str) → str

Return a keyed BIDS entity

Parameters:

• entity (str) –

• key (str) –

Return type:

str

Examples

>>> with_key('sub-1', 'sub')
'sub-1'
>>> with_key('1', 'sub')
'sub-1'

CPAC.utils.bids_utils.without_key(entity: str, key: str) → str

Return a BIDS entity value

Parameters:

• entity (str) –

• key (str) –

Return type:

str

Examples

>>> without_key('sub-1', 'sub')
'1'
>>> without_key('1', 'sub')
'1'

BIDS Data Configuration

Configuration

C-PAC Configuration module

class CPAC.utils.configuration.Configuration(config_map: dict | None = None, skip_env_check: bool = False)

Class to set dictionary keys as map attributes.

If the given dictionary includes the key FROM, that key’s value will form the base of the Configuration object with the values in the given dictionary overriding matching keys in the base at any depth. If no FROM key is included, the base Configuration is the default Configuration.

FROM accepts either the name of a preconfigured pipleine or a path to a YAML file.

Given a Configuration c, and a list or tuple of an attribute name and nested keys keys = ['attribute', 'key0', 'key1'] or keys = ('attribute', 'key0', 'key1'), the value ‘value’ nested in

c.attribute = {'key0': {'key1': 'value'}}

can be accessed (get and set) in any of the following ways (and more):

c.attribute['key0']['key1']
c['attribute']['key0']['key1']
c['attribute', 'key0', 'key1']
c[keys]

Examples

>>> c = Configuration({})
>>> c['pipeline_setup', 'pipeline_name']
'cpac-blank-template'
>>> c = Configuration({'pipeline_setup': {
...     'pipeline_name': 'example_pipeline'}})
>>> c['pipeline_setup', 'pipeline_name']
'example_pipeline'
>>> c['pipeline_setup', 'pipeline_name'] = 'new_pipeline2'
>>> c['pipeline_setup', 'pipeline_name']
'new_pipeline2'

>>> from CPAC.utils.tests.configs import SLACK_420349

# test “FROM: /path/to/file” >>> slack_420349_filepath = Configuration( … yaml.safe_load(SLACK_420349[‘filepath’])) >>> slack_420349_filepath[‘pipeline_setup’, ‘pipeline_name’] ‘slack_420349_filepath’

# test “FROM: preconfig” >>> slack_420349_preconfig = Configuration( … yaml.safe_load(SLACK_420349[‘preconfig’])) >>> slack_420349_preconfig[‘pipeline_setup’, ‘pipeline_name’] ‘slack_420349_preconfig’

check_pattern(orig_key, tags=None)

dict()

Show contents of a C-PAC configuration as a dict

get_nested(_d, keys)

key_type_error(key)

keys()

Show toplevel keys of a C-PAC configuration dict

return_config_elements()

set_from_ENV(conf)

Replace strings like $VAR and ${VAR} with environment variable values.

Parameters:

conf (any) –

Returns:

conf

Return type:

any

Examples

>>> import os
>>> os.environ['SAMPLE_VALUE_SFE'] = '/example/path'
>>> c = Configuration()
>>> c.set_from_ENV({'key': {'nested_list': [
...     1, '1', '$SAMPLE_VALUE_SFE/extended']}})
{'key': {'nested_list': [1, '1', '/example/path/extended']}}
>>> c.set_from_ENV(['${SAMPLE_VALUE_SFE}', 'SAMPLE_VALUE_SFE'])
['/example/path', 'SAMPLE_VALUE_SFE']
>>> del os.environ['SAMPLE_VALUE_SFE']

set_nested(d, keys, value)

set_without_ENV(conf)

Retain strings like $VAR and ${VAR} when setting attributes.

Parameters:

conf (any) –

Returns:

conf

Return type:

any

Examples

>>> import os
>>> os.environ['SAMPLE_VALUE_SFE'] = '/example/path'
>>> c = Configuration()
>>> c.set_without_ENV({'key': {'nested_list': [
...     1, '1', '$SAMPLE_VALUE_SFE/extended']}})
{'key': {'nested_list': [1, '1', '$SAMPLE_VALUE_SFE/extended']}}
>>> c.set_without_ENV(['${SAMPLE_VALUE_SFE}', 'SAMPLE_VALUE_SFE'])
['${SAMPLE_VALUE_SFE}', 'SAMPLE_VALUE_SFE']
>>> del os.environ['SAMPLE_VALUE_SFE']

sub_pattern(pattern, orig_key)

switch_is_off(key: str | list[str], exclusive: bool = False) → bool

Return True if the given key is set to ‘off’ OR ‘on’ and ‘off’ or False otherwise. Used for tracking forking.

Parameters:

• key (str or list of str) – key to check

• exclusive (bool, optional, default: False) – return False if the key is set to ‘on’ and ‘off’

Returns:

True if key is set to ‘off’, False if not set to ‘off’. If exclusive is set to True, return False if the key is set to ‘on’ and ‘off’.

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'])
True
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'])
True
>>> c.switch_is_off(['nuisance_corrections', '2-nuisance_regression',
...                  'run'], exclusive=True)
False

switch_is_on(key: str | list[str], exclusive: bool = False) → bool

Return True if the given key is set to ‘on’ OR ‘on’ and ‘off’ or False otherwise. Used for tracking forking.

Parameters:

• key (str or list of str) – key to check

• exclusive (bool, optional, default: False) – return False if the key is set to ‘on’ and ‘off’

Returns:

True if key is set to ‘on’, False if not set to ‘on’. If exclusive is set to True, return False if the key is set to ‘on’ and ‘off’.

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'])
False
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'])
True
>>> c.switch_is_on(['nuisance_corrections', '2-nuisance_regression',
...                 'run'], exclusive=True)
False

switch_is_on_off(key: str | list[str]) → bool

Return True if the given key is set to both ‘on’ and ‘off’ or False otherwise. Used for tracking forking.

Parameters:

key (str or list of str) – key to check

Returns:

True if key is set to ‘on’ and ‘off’, False otherwise

Return type:

bool

Examples

>>> c = Configuration()
>>> c.switch_is_on_off(['nuisance_corrections',
...                     '2-nuisance_regression', 'run'])
False
>>> c = Configuration({'nuisance_corrections': {
...     '2-nuisance_regression': {'run': [True, False]}}})
>>> c.switch_is_on_off(['nuisance_corrections',
...                     '2-nuisance_regression', 'run'])
True

update(key, val=ConfigurationDictUpdateConflation())

class CPAC.utils.configuration.Preconfiguration(preconfig, skip_env_check=False)

A preconfigured Configuration

Parameters:

preconfig (str) – The canonical name of the preconfig to load

CPAC.utils.configuration.check_pname(p_name: str, pipe_config: Configuration) → str

Function to check / set p_name, the string representation of a pipeline for use in filetrees

Parameters:

• p_name (str or None) –

• pipe_config (Configuration) –

Return type:

p_name

Examples

>>> c = Configuration()
>>> check_pname(None, c)
'pipeline_cpac-blank-template'
>>> check_pname('cpac-default-pipeline', c)
'pipeline_cpac-default-pipeline'
>>> check_pname('pipeline_cpac-default-pipeline', c)
'pipeline_cpac-default-pipeline'
>>> check_pname('different-name', Configuration())
'pipeline_different-name'
>>> p_name = check_pname(None, Preconfiguration('blank'))
>>> p_name
'pipeline_cpac-blank-template'
>>> p_name = check_pname(None, Preconfiguration('default'))
>>> p_name
'pipeline_cpac-default-pipeline'

CPAC.utils.configuration.preconfig_yaml(preconfig_name='default', load=False)

Get the path to a preconfigured pipeline’s YAML file. Raises BadParameter if an invalid preconfig name is given.

Parameters:

• preconfig_name (str) –

• load (boolean) – return dict if True, str if False

Returns:

path to YAML file or dict loaded from YAML

Return type:

str or dict

CPAC.utils.configuration.set_subject(sub_dict: dict, pipe_config: Configuration, p_name: str | None = None) → tuple[str, str, str]

Function to set pipeline name and log directory path for a given sub_dict

Parameters:

• sub_dict (dict) –

• pipe_config (CPAC.utils.configuration.Configuration) –

• p_name (str, optional) – pipeline name string

Returns:

• subject_id (str)

• p_name (str) – pipeline name string

• log_dir (str) – path to subject log directory

Examples

>>> from tempfile import TemporaryDirectory
>>> from CPAC.utils.configuration import Configuration
>>> sub_dict = {'site_id': 'site1', 'subject_id': 'sub1',
...             'unique_id': 'uid1'}
>>> with TemporaryDirectory() as tmpdir:
...     subject_id, p_name, log_dir = set_subject(
...         sub_dict, Configuration({'pipeline_setup': {'log_directory':
...             {'path': tmpdir}}}))
>>> subject_id
'sub1_uid1'
>>> p_name
'pipeline_cpac-blank-template'
>>> log_dir.endswith(f'{p_name}/{subject_id}')
True

Create FMRIB’s Local Analysis of Mixed Effects (FLAME) Model Files

CPAC.utils.create_flame_model_files.create_con_file(con_vecs, con_names, col_names, model_name, current_output, out_dir)

CPAC.utils.create_flame_model_files.create_con_ftst_file(con_file, model_name, current_output, output_dir, column_names, coding_scheme, group_sep)

Create the contrasts and fts file

CPAC.utils.create_flame_model_files.create_dummy_string(length)

CPAC.utils.create_flame_model_files.create_flame_model_files(design_matrix, col_names, contrasts_vectors, contrast_names, custom_contrasts_csv, ftest_list, group_sep, grouping_vector, coding_scheme, model_name, output_measure, output_dir)

CPAC.utils.create_flame_model_files.create_fts_file(ftest_list, con_names, model_name, current_output, out_dir)

CPAC.utils.create_flame_model_files.create_grp_file(design_matrix, grp_file_vector, output_dir, model_name)

CPAC.utils.create_flame_model_files.write_mat_file(design_matrix, output_dir, model_name, depatsified_EV_names, current_output=None)

Create FSL FLAME Preset

CPAC.utils.create_fsl_flame_preset.create_contrasts_template_df(design_df, contrasts_dct_list=None)

Create the template Pandas DataFrame for the contrasts matrix CSV.

The headers in the contrasts matrix needs to match the headers of the design matrix.

CPAC.utils.create_fsl_flame_preset.create_design_matrix_df(group_list, pheno_df=None, ev_selections=None, pheno_sub_label=None, pheno_ses_label=None, pheno_site_label=None, ses_id=False)

Create the design matrix intended for group-level analysis via the FSL FLAME tool.

This does NOT create the final .mat file that FSL FLAME takes in. This is an intermediary design matrix CSV meant for the user to review.

If there is a phenotype CSV provided, this function will align the participant-session ID labels in the CPAC individual-level analysis output directory with the values listed in the phenotype file.

CPAC.utils.create_fsl_flame_preset.preset_paired_two_group(group_list, conditions, condition_type='session', output_dir=None, model_name='two_sample_unpaired_T-test')

Set up the design matrix and contrasts matrix for running an paired two-group difference (two-sample paired T-test).

group_list: a list of strings- sub_ses unique IDs conditions: a two-item list of strings- session or series/scan names of

the two sessions or two scans (per participant) you wish to compare

condition_type: a string, either “session” or “scan”, depending on what

is in “conditions”

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:

https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

#Paired_Two-Group_Difference_.28Two-Sample_Paired_T-Test.29

CPAC.utils.create_fsl_flame_preset.preset_single_group_avg(group_list, pheno_df=None, covariate=None, pheno_sub_label=None, output_dir=None, model_name='one_sample_T-test')

Set up the design matrix CSV for running a single group average (one-sample T-test).

CPAC.utils.create_fsl_flame_preset.preset_tripled_two_group(group_list, conditions, condition_type='Sessions', output_dir=None, model_name='tripled_T-test')

Set up the design matrix and contrasts matrix for running a tripled two-group difference (‘tripled’ T-test).

group_list: a list of strings- sub_ses unique IDs conditions: a three-item list of strings- session or series/scan names of

the three sessions or three scans (per participant) you wish to compare

condition_type: a string, either “session” or “scan”, depending on what

is in “conditions”

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:

https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

#Tripled_Two-Group_Difference_.28.22Tripled.22_T-Test.29

CPAC.utils.create_fsl_flame_preset.preset_unpaired_two_group(group_list, pheno_df, groups, pheno_sub_label, output_dir=None, model_name='two_sample_unpaired_T-test')

Set up the design matrix and contrasts matrix for running an unpaired two-group difference (two-sample unpaired T-test).

group_list: a list of strings- sub_ses unique IDs pheno_df: a Pandas DataFrame object of the phenotypic file CSV/matrix groups: a list of either one or two strings- design matrix EV/covariate

labels to take from the phenotype DF and include in the model

pheno_sub_label: a string of the label name of the column in the phenotype

file that holds the participant/session ID for each row

output_dir: (optional) string of the output directory path model_name: (optional) name/label of the model to run

Sets up the model described here:

https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FEAT/UserGuide

#Unpaired_Two-Group_Difference_.28Two-Sample_Unpaired_T-Test.29

Only one “group” will be provided usually if the two groups in the phenotypic information you wish to compare are encoded in one covariate column, as categorical information. Thus, providing this one name will pull it from the phenotype file, and this function will break it out into two columns using dummy-coding.

CPAC.utils.create_fsl_flame_preset.read_group_list_text_file(group_list_text_file)

Read in the group-level analysis participant-session list text file.

CPAC.utils.create_fsl_flame_preset.read_pheno_csv_into_df(pheno_csv, id_label=None)

Read the phenotypic file CSV or TSV into a Pandas DataFrame.

CPAC.utils.create_fsl_flame_preset.run(pipeline_dir, derivative_list, z_thresh, p_thresh, preset=None, group_list_text_file=None, pheno_file=None, pheno_sub_label=None, output_dir=None, model_name=None, covariate=None, condition_type=None, run=False)

CPAC.utils.create_fsl_flame_preset.write_config_dct_to_yaml(config_dct, out_file=None)

Write out a configuration dictionary into a YAML file.

CPAC.utils.create_fsl_flame_preset.write_dataframe_to_csv(matrix_df, out_file=None)

Write out a matrix Pandas DataFrame into a CSV file.

CPAC.utils.create_fsl_flame_preset.write_group_list_text_file(group_list, out_file=None)

Write out the group-level analysis participant list as a text file.

Create FSL create_fsl_model

CPAC.utils.create_fsl_model.check_multicollinearity(matrix)

CPAC.utils.create_fsl_model.create_con_file(con_dict, col_names, file_name, current_output, out_dir)

CPAC.utils.create_fsl_model.create_con_ftst_file(con_file, model_name, current_output, output_dir, column_names, coding_scheme, group_sep)

Create the contrasts and fts file

CPAC.utils.create_fsl_model.create_design_matrix(pheno_file, ev_selections, formula, subject_id_label, sub_list=None, coding_scheme='Treatment', grouping_var=None, new_regressor_dict=None, roi_means_dict=None, output_dir=None, model_name='design', current_output=None)

CPAC.utils.create_fsl_model.create_dummy_string(length)

CPAC.utils.create_fsl_model.create_fts_file(ftest_list, con_dict, model_name, current_output, out_dir)

CPAC.utils.create_fsl_model.create_grp_file(design_matrix, grouping_var_id_dict, output_dir, model_name, current_output=None)

CPAC.utils.create_fsl_model.create_pheno_dict(pheno_file_rows, ev_selections, participant_id_label)

CPAC.utils.create_fsl_model.get_custom_roi_info(roi_means_dict)

CPAC.utils.create_fsl_model.get_measure_dict(param_file)

CPAC.utils.create_fsl_model.greater_than(dmat, a, b, coding, group_sep, grouping_var)

CPAC.utils.create_fsl_model.load_group_participant_list(group_participant_list_file)

CPAC.utils.create_fsl_model.load_pheno_file(pheno_file)

CPAC.utils.create_fsl_model.model_group_var_separately(grouping_var, formula, pheno_data_dict, ev_selections, coding_scheme)

CPAC.utils.create_fsl_model.negative(dmat, a, coding, group_sep, grouping_var)

CPAC.utils.create_fsl_model.positive(dmat, a, coding, group_sep, grouping_var)

CPAC.utils.create_fsl_model.process_contrast(parsed_contrast, operator, ev_selections, group_sep, grouping_var, coding_scheme)

CPAC.utils.create_fsl_model.process_pheno_file(pheno_file_dataframe, group_subs_dataframe, participant_id_label)

CPAC.utils.create_fsl_model.run(group_config, current_output, param_file=None, derivative_means_dict=None, roi_means_dict=None, model_out_dir=None, CPAC_run=True)

CPAC.utils.create_fsl_model.write_mat_file(design_matrix, output_dir, model_name, depatsified_EV_names, current_output=None)

Create Group Analysis Info Files

CPAC.utils.create_group_analysis_info_files.write_blank_contrast_csv(contrasts_columns, contrast_out_path)

CPAC.utils.create_group_analysis_info_files.write_custom_readme_file()

CPAC.utils.create_group_analysis_info_files.write_design_matrix_csv(patsy_dmatrix, participant_column, column_names, outfile_path)

Datasource

CPAC.utils.datasource.bidsier_prefix(unique_id)

Function to return a BIDSier prefix for a given unique_id

Parameters:

unique_id (str) –

Returns:

prefix

Return type:

str

Examples

>>> bidsier_prefix('01_1')
'sub-01_ses-1'
>>> bidsier_prefix('sub-01_ses-1')
'sub-01_ses-1'
>>> bidsier_prefix('sub-01_1')
'sub-01_ses-1'
>>> bidsier_prefix('01_ses-1')
'sub-01_ses-1'

CPAC.utils.datasource.calc_delta_te_and_asym_ratio(effective_echo_spacing: float, echo_times: list) → tuple[float, float]

Calcluate deltaTE and ees_asym_ratio from given metadata

Parameters:

• effective_echo_spacing (float) – EffectiveEchoSpacing from sidecar JSON

• echo_times (list) –

Returns:

• deltaTE (float)

• ees_asym_ratio (float)

CPAC.utils.datasource.check_for_s3(file_path, creds_path=None, dl_dir=None, img_type='other', verbose=False)

CPAC.utils.datasource.check_func_scan(func_scan_dct, scan)

Run some checks on the functional timeseries-related files for a given series/scan name or label.

CPAC.utils.datasource.create_anat_datasource(wf_name='anat_datasource')

CPAC.utils.datasource.create_check_for_s3_node(name, file_path, img_type='other', creds_path=None, dl_dir=None, map_node=False)

CPAC.utils.datasource.create_fmap_datasource(fmap_dct, wf_name='fmap_datasource')

Return the field map files, from the dictionary of functional files described in the data configuration (sublist) YAML file.

CPAC.utils.datasource.create_func_datasource(rest_dict, rpool, wf_name='func_datasource')

Return the functional timeseries-related file paths for each series/scan, from the dictionary of functional files described in the data configuration (sublist) YAML file.

Scan input (from inputnode) is an iterable.

CPAC.utils.datasource.create_general_datasource(wf_name)

CPAC.utils.datasource.create_grp_analysis_dataflow(wf_name='gp_dataflow')

CPAC.utils.datasource.create_roi_mask_dataflow(masks, wf_name='datasource_roi_mask')

CPAC.utils.datasource.create_spatial_map_dataflow(spatial_maps, wf_name='datasource_maps')

CPAC.utils.datasource.extract_scan_params_dct(scan_params_dct)

CPAC.utils.datasource.gather_echo_times(echotime_1, echotime_2=None, echotime_3=None, echotime_4=None)

CPAC.utils.datasource.gather_extraction_maps(c)

CPAC.utils.datasource.get_fmap_phasediff_metadata(data_config_scan_params)

CPAC.utils.datasource.get_highest_local_res(template: Path | str, tagname: str) → Path

Given a reference template path and a resolution string, get all resolutions of that template in the same local path and return the highest resolution.

Parameters:

• template (Path or str) –

• tagname (str) –

Return type:

str

Raises:

LookupError – If no matching local template is found.

Examples

>>> get_highest_local_res(
...     '/cpac_templates/MacaqueYerkes19_T1w_2mm_brain.nii.gz', '2mm')
PosixPath('/cpac_templates/MacaqueYerkes19_T1w_0.5mm_brain.nii.gz')
>>> get_highest_local_res(
...     '/cpac_templates/dne_T1w_2mm.nii.gz', '2mm')
Traceback (most recent call last):
   ...
LookupError: Could not find template /cpac_templates/dne_T1w_2mm.nii.gz

CPAC.utils.datasource.get_map(map, map_dct)

CPAC.utils.datasource.get_rest(scan, rest_dict, resource='scan')

Return the file path of the chosen resource stored in the functional file dictionary, if it exists.

scan: the scan/series name or label rest_dict: the dictionary read in from the data configuration YAML file

(sublist) nested under ‘func:’

resource: the dictionary key

scan - the functional timeseries scan_parameters - path to the scan parameters JSON file, or

a dictionary containing scan parameters information (to be phased out in the future)

CPAC.utils.datasource.ingress_func_metadata(wf, cfg, rpool, sub_dict, subject_id, input_creds_path, unique_id=None, num_strat=None)

CPAC.utils.datasource.match_epi_fmaps(bold_pedir, epi_fmap_one, epi_fmap_params_one, epi_fmap_two=None, epi_fmap_params_two=None)

Parse the field map files in the data configuration and determine which ones have the same and opposite phase-encoding directions as the BOLD scan in the current pipeline.

Example - parse the files under the ‘fmap’ level, i.e. ‘epi_AP’:

anat: /path/to/T1w.nii.gz fmap:

epi_AP:

scan: /path/to/field-map.nii.gz scan_parameters: <config dictionary containing phase-encoding

direction>

func:

rest_1:

scan: /path/to/bold.nii.gz scan_parameters: <config dictionary of BOLD scan parameters>

1. Check PhaseEncodingDirection field in the metadata for the BOLD.

2. Check whether there are one or two EPI’s in the field map data.

3. Grab the one or two EPI field maps.

CPAC.utils.datasource.res_string_to_tuple(resolution)

Converts a resolution string to a tuple of floats.

Parameters:

resolution (str) – Resolution string, e.g. “3.438mmx3.438mmx3.4mm”

Returns:

resolution – Tuple of floats, e.g. (3.438, 3.438, 3.4)

Return type:

tuple

CPAC.utils.datasource.resample_func_roi(in_func, in_roi, realignment, identity_matrix)

CPAC.utils.datasource.resolve_resolution(resolution, template, template_name, tag=None)

CPAC.utils.datasource.select_model_files(model, ftest, model_name)

Method to select model files

Datatypes

Custom datatypes for C-PAC

class CPAC.utils.datatypes.ItemFromList(list_of_one, msg=None)

Coerce single-item lists into just the only item in the list. Returns item if item is not a list, set, or tuple. Raises CoerceInvalid if impossible.

Examples

>>> ItemFromList(['seagull'])
'seagull'
>>> ItemFromList(['two', 'seagulls'])
Traceback (most recent call last):
    ...
voluptuous.error.CoerceInvalid: Cannot coerce list of length 2 to item
>>> ItemFromList('string')
'string'

class CPAC.utils.datatypes.ListFromItem(*args, **kwargs)

Subclass of list to coerce non-lists into lists

Examples

>>> list('one')
['o', 'n', 'e']
>>> ListFromItem('one')
['one']
>>> list(['one'])
['one']
>>> ListFromItem(['one'])
['one']
>>> list()
[]
>>> ListFromItem()
[]
>>> list(None)
Traceback (most recent call last):
...
TypeError: 'NoneType' object is not iterable
>>> ListFromItem(None)
[]

Documentation

Utilties for documentation.

CPAC.utils.docs.docstring_parameter(*args, **kwargs)

Decorator to parameterize docstrings. Use double-curly-braces ({{}}) for literal curly braces.

Examples

>>> @docstring_parameter('test', answer='Yes it does.')
... def do_nothing():
...     '''Does this {} do anything? {answer}'''
...     pass
>>> print(do_nothing.__doc__)
Does this test do anything? Yes it does.
>>> @docstring_parameter('test', answer='It should not.')
... def how_about_now():
...     '''How about {{ this }}?'''
...     pass
>>> print(how_about_now.__doc__)
How about { this }?

CPAC.utils.docs.version_report() → str

A formatted block of versions included in CPAC’s environment

Extract Data

class CPAC.utils.extract_data.Configuration(config_map)

CPAC.utils.extract_data.extract_data(c, param_map)

Method to generate a CPAC input subject list python file. The method extracts anatomical and functional data for each site( if multiple site) and/or scan parameters for each site and put it into a data structure read by python

Example: subjects_list =[

{

‘subject_id’ : ‘0050386’, ‘unique_id’ : ‘session_1’, ‘anat’: ‘/Users/home/data/NYU/0050386/session_1/anat_1/anat.nii.gz’, ‘rest’:{

‘rest_1_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_1/rest.nii.gz’, ‘rest_2_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_2/rest.nii.gz’, }

‘scan_parameters’:{

‘tr’: ‘2’, ‘acquisition’: ‘alt+z2’, ‘reference’: ‘17’, ‘first_tr’: ‘’, ‘last_tr’: ‘’, }

},

]

or

subjects_list =[

{

‘subject_id’ : ‘0050386’, ‘unique_id’ : ‘session_1’, ‘anat’: ‘/Users/home/data/NYU/0050386/session_1/anat_1/anat.nii.gz’, ‘rest’:{

‘rest_1_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_1/rest.nii.gz’, ‘rest_2_rest’ : ‘/Users/home/data/NYU/0050386/session_1/rest_2/rest.nii.gz’, }

},

]

CPAC.utils.extract_data.generate_supplementary_files(data_config_outdir, data_config_name)

Method to generate phenotypic template file and subject list for group analysis

CPAC.utils.extract_data.read_csv(csv_input)

Method to read csv file

‘Acquisition’ ‘Reference’ ‘Site’ ‘TR (seconds)’

CPAC.utils.extract_data.run(data_config)

Run method takes data_config file as the input argument

Extract Data (Multiscan)

class CPAC.utils.extract_data_multiscan.Configuration(config_map)

CPAC.utils.extract_data_multiscan.extract_data(c, param_map)

Method to generate a CPAC input subject list python file. The method extracts anatomical functional data and scan parameters for each site( if multiple site) and for each scan and put it into a data structure read by python

Note:

Use this tool only if the scan parameters are different for each scan as shown in the example below.

Example:

subjects_list = [

{

‘subject_id’: ‘0021001’, ‘unique_id’: ‘session2’, ‘anat’: ‘/home/data/multiband_data/NKITRT/0021001/anat/mprage.nii.gz’, ‘rest’:{

‘RfMRI_mx_1400_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_mx_1400/rest.nii.gz’, ‘RfMRI_mx_645_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_mx_645/rest.nii.gz’, ‘RfMRI_std_2500_rest’: ‘/home/data/multiband_data/NKITRT/0021001/session2/RfMRI_std_2500/rest.nii.gz’, },

‘scan_parameters’:{

‘TR’:{

‘RfMRI_mx_1400_rest’: ‘1.4’, ‘RfMRI_mx_645_rest’: ‘1.4’, ‘RfMRI_std_2500_rest’: ‘2.5’, },

‘Acquisition’:{

‘RfMRI_mx_1400_rest’: ‘/home/data/1400.txt’, ‘RfMRI_mx_645_rest’: ‘/home/data/645.txt’, ‘RfMRI_std_2500_rest’: ‘/home/data/2500.txt’, },

‘Reference’:{

‘RfMRI_mx_1400_rest’: ‘32’, ‘RfMRI_mx_645_rest’: ‘20’, ‘RfMRI_std_2500_rest’: ‘19’, },

‘FirstTR’:{

‘RfMRI_mx_1400_rest’: ‘7’, ‘RfMRI_mx_645_rest’: ‘15’, ‘RfMRI_std_2500_rest’: ‘4’, },

‘LastTR’:{

‘RfMRI_mx_1400_rest’: ‘440’, ‘RfMRI_mx_645_rest’: ‘898’, ‘RfMRI_std_2500_rest’: ‘None’, },

}

},

]

CPAC.utils.extract_data_multiscan.generate_suplimentary_files(output_path)

Method to generate phenotypic template file and subject list for group analysis

CPAC.utils.extract_data_multiscan.read_csv(csv_input)

Method to read csv file

‘Acquisition’ ‘Reference’ ‘Site’ ‘TR (seconds)’

CPAC.utils.extract_data_multiscan.run(data_config)

Run method takes data_config file as the input argument

Extract Parameters

CPAC.utils.extract_parameters.grab(output_dir, scrubbing)

Method to grab all the motion parameters and power parameters file from each subject for each pipeline and merge them

Parameters:

output_dir (string) – Path to the datasink output directory of CPAC

CPAC.utils.extract_parameters.merge(output_dir, scan_name, threshold, motion_f, power_f, flag)

Method to merge power parameters and motion parameters file

CPAC.utils.extract_parameters.run(output_path, scrubbing)

Google Analytics

CPAC.utils.ga.do_it(data, timeout)

CPAC.utils.ga.get_or_create_config()

CPAC.utils.ga.get_uid()

CPAC.utils.ga.track_config(cpac_interface)

CPAC.utils.ga.track_event(category, action, uid=None, label=None, value=0, software_version=None, timeout=2, thread=True)

Record an event with Google Analytics

Parameters:

• tracking_id (str) – Google Analytics tracking ID.

• category (str) – Event category.

• action (str) – Event action.

• uid (str) – User unique ID, assigned when popylar was installed.

• label (str) – Event label.

• value (int) – Event value.

• software_version (str) – Records a version of the software.

• timeout (float) – Maximal duration (in seconds) for the network connection to track the event. After this duration has elapsed with no response (e.g., on a slow network connection), the tracking is dropped.

CPAC.utils.ga.track_run(level='participant', participants=0)

Interfaces

Function Interfaces

Function interface utilities for C-PAC

class CPAC.utils.interfaces.function.Function(input_names=None, output_names='out', function=None, imports=None, as_module=False, **inputs)

Runs arbitrary function as an interface

Examples

>>> func = 'def func(arg1, arg2=5): return arg1 + arg2'
>>> fi = Function(input_names=['arg1', 'arg2'], output_names=['out'])
>>> fi.inputs.function_str = func
>>> res = fi.run(arg1=1)
>>> res.outputs.out
6

input_spec

alias of FunctionInputSpec

output_spec

alias of DynamicTraitedSpec

static sig_imports(imports: List[str]) → Callable

Sets an ns_imports attribute on a function for Function-node functions. This can be useful for classes needed for decorators, typehints and for avoiding redefinitions.

Parameters:

imports (list of str) – import statements to import the function in an otherwise empty namespace. If these collide with imports defined via the Function.__init__ initialization method, the imports given as a parameter here will be overridden by those from the initializer.

Returns:

func

Return type:

function

Examples

See the defintion of calculate_FD_J to see the decorator tested here being applied. >>> from CPAC.generate_motion_statistics import calculate_FD_J >>> calc_fdj = Function(input_names=[‘in_file’, ‘calc_from’, ‘center’], … output_names=[‘out_file’], … function=calculate_FD_J, … as_module=True) >>> calc_fdj.imports # doctest: +NORMALIZE_WHITESPACE [‘from CPAC.utils.interfaces.function import Function’,

‘import os’, ‘import sys’, ‘from typing import Optional’, ‘import numpy as np’, ‘from CPAC.utils.pytest import skipif’, ‘from CPAC.utils.typing import LITERAL, TUPLE’]

>>> from inspect import signature
>>> from nipype.utils.functions import (getsource,
...     create_function_from_source)
>>> f = create_function_from_source(getsource(calculate_FD_J),
...                                 calc_fdj.imports)
>>> inspect.signature(calculate_FD_J) == inspect.signature(f)
True

CPAC.utils.interfaces.function.pick_tissue_from_labels_file_interface(input_names=None)

Function to create a Function interface for CPAC.seg_preproc.utils.pick_tissue_from_labels_file

Parameters:

input_names (list, optional) –

Return type:

nipype.interfaces.base.core.Interface

Tests

Miscellaneous

Miscellaneous utilities

CPAC.utils.misc.get_template_specs(in_template, template_spec=None, default_resolution=1)

Parse template specifications

>>> get_template_specs('MNI152NLin2009cAsym', {'suffix': 'T1w'})[1]  
{'resolution': 1}

>>> get_template_specs('MNI152NLin2009cAsym', {'res': '2', 'suffix': 'T1w'})[1]  
{'resolution': '2'}

>>> get_template_specs('MNIInfant', {'res': '2', 'cohort': '10', 'suffix': 'T1w'})[1]  
{'resolution': '2', 'cohort': '10'}

>>> get_template_specs('MNI152NLin2009cAsym',
...                    {'suffix': 'T1w', 'cohort': 1})[1] 
Traceback (most recent call last):
RuntimeError:
...

>>> get_template_specs('MNI152NLin2009cAsym',
...                    {'suffix': 'T1w', 'res': '1|2'})[1] 
Traceback (most recent call last):
RuntimeError:
...

Monitoring

Module to customize Nipype’s process monitoring for use in C-PAC

See https://fcp-indi.github.io/docs/developer/nodes for C-PAC-specific documentation. See https://nipype.readthedocs.io/en/latest/api/generated/nipype.utils.profiler.html for Nipype’s documentation.

class CPAC.utils.monitoring.LoggingHTTPServer(pipeline_name, logging_dir='', host='', port=8080, request=<class 'CPAC.utils.monitoring.monitoring.LoggingRequestHandler'>)

class CPAC.utils.monitoring.LoggingRequestHandler(request, client_address, server)

handle()

CPAC.utils.monitoring.failed_to_start(log_dir, exception)

Launch a failed-to-start logger for a run that failed to start. Must be called from within an except block.

Parameters:

• log_dir (str) – path to logging directory

• exception (Exception) –

CPAC.utils.monitoring.getLogger(name)

Function to get a mock logger if one exists, falling back on real loggers.

Parameters:

name (str) –

Returns:

logger

Return type:

CPAC.utils.monitoring.custom_logging.MockLogger or logging.Logger

CPAC.utils.monitoring.log_nodes_cb(node, status)

Function to record node run statistics to a log file as json dictionaries

Parameters:

• node (nipype.pipeline.engine.Node) – the node being logged

• status (string) – acceptable values are ‘start’, ‘end’; otherwise it is considered and error

Returns:

this function does not return any values, it logs the node status info to the callback logger

Return type:

None

CPAC.utils.monitoring.log_nodes_initial(workflow)

CPAC.utils.monitoring.monitor_server(pipeline_name, logging_dir, host='0.0.0.0', port=8080)

CPAC.utils.monitoring.recurse_nodes(workflow, prefix='')

CPAC.utils.monitoring.set_up_logger(name, filename=None, level=None, log_dir=None, mock=False, overwrite_existing=False)

Function to initialize a logger

Parameters:

• name (str) – logger name (for subsequent calls to logging.getLogger) to write to the same log file)

• filename (str, optional) – filename to write log to. If not specified, filename will be the same as name with the extension log

• level (str, optional) – one of {critical, error, warning, info, debug, notset}, case-insensitive

• log_dir (str, optional) –

• mock (bool, optional) – if True, return a CPAC.utils.monitoring.MockLogger instead of a logging.Logger

Returns:

logger – initialized logging Handler

Return type:

logging.Handler

Examples

>>> lg = set_up_logger('test')
>>> lg.handlers[0].baseFilename.split('/')[-1]
'test.log'
>>> lg.level
0
>>> lg = set_up_logger('second_test', 'specific_filename.custom', 'debug')
>>> lg.handlers[0].baseFilename.split('/')[-1]
'specific_filename.custom'
>>> lg.level
10
>>> lg = set_up_logger('third_test', mock=True)
>>> getLogger('third_test') == lg
True
>>> 'third_test' in MOCK_LOGGERS
True
>>> lg.delete()
>>> 'third_test' in MOCK_LOGGERS
False

NeuroData’s MRI to Graphs (NDMG | m2g) Utilities

Functions in this file adapted from NeuroData group: STATEMENT OF CHANGES:

This file is derived from sources licensed under the Apache-2.0 terms, and this file has been changed.

CHANGES:

• Minor refactoring for compatibility with C-PAC

ORIGINAL WORK’S ATTRIBUTION NOTICE:

Copyright 2016 NeuroData (http://neurodata.io)

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

graph.py Created by Greg Kiar on 2016-01-27. Email: gkiar@jhu.edu

Can be found here:

https://github.com/neurodata/m2g/blob/v0.1.0/ndmg/graph/graph.py

Modifications Copyright (C) 2022 C-PAC Developers

This file is part of C-PAC.

class CPAC.utils.ndmg_utils.graph(N, rois, attr=None, sens='dwi')

as_matrix()

Returns the graph as a matrix.

cor_graph(timeseries, attr=None)

Takes timeseries and produces a correlation matrix Positional Arguments:

timeseries:

-the timeseries file to extract correlation for.

dimensions are [numrois]x[numtimesteps]

get_graph()

Returns the graph object created

make_graph(streamlines, attr=None)

Takes streamlines and produces a graph Positional Arguments:

streamlines:

• Fiber streamlines either file or array in a dipy EuDX or compatible format.

save_graph(graphname)

Saves the graph to disk Positional Arguments:

graphname:

• Filename for the graph

summary()

User friendly wrapping and display of graph properties

CPAC.utils.ndmg_utils.ndmg_create_graphs(ts, labels)

CPAC.utils.ndmg_utils.ndmg_roi_timeseries(func_file, label_file)

Function to extract average timeseries for the voxels in each roi of the labelled atlas. Returns the roi timeseries as a numpy.ndarray. Positional Arguments

func_file:

• the path to the 4d volume to extract timeseries

label_file:

• the path to the labelled atlas containing labels

  for the voxels in the fmri image

roits_file:

• the path to where the roi timeseries will be saved. If

None, don’t save and just return the roi_timeseries.

# Adapted from ndmg v0.1.1 # Copyright 2016 NeuroData (http://neurodata.io)

Neuroimaging Informatics Technology Initiative (NIfTI) Utilities

CPAC.utils.nifti_utils.inverse_nifti_values(image)

Replace zeros by ones and non-zero values by 1 :param image: path to the nifti file to be inverted or

the image already loaded through nibabel

Returns:

output

Return type:

Nibabel Nifti1Image

CPAC.utils.nifti_utils.more_zeros_than_ones(image)

Return True is there is more zeros than other values in a given nifti image. :param image: path to the nifti file to be inverted or

the image already loaded through nibabel

Returns:

more_zeros

Return type:

boolean

CPAC.utils.nifti_utils.nifti_image_input(image)

Test if an input is a path or a nifti.image and the image loaded through nibabel :param image: path to the nifti file or the image already loaded through nibabel :type image: str or nibabel.nifti1.Nifti1Image

Returns:

img – load and return the nifti image if image is a path, otherwise simply return image

Return type:

nibabel.nifti1.Nifti1Image

Outputs

class CPAC.utils.outputs.Outputs

all_native_filter = 0       True 1       True 2       True 3       True 4      False        ...   306    False 307    False 308    False 309    False 310    False Name: Space, Length: 311, dtype: bool

all_template_filter = 0      False 1      False 2      False 3      False 4       True        ...   306    False 307    False 308    False 309    False 310    False Name: Space, Length: 311, dtype: bool

anat = ['space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'space-template_desc-brain_mask', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'dseg']

any = ['alff', 'desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-T1w_sbref', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'space-template_desc-brain_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'motion', 'desc-summary_motion', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'space-template_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'space-fsLR_den-32k_bold-dtseries', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'vmhc', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask', 'mdmr', 'desc-zstd-mdmr', 'dseg', 'AtlasSubcortical-s2', 'space-fsLR_den-32k_bold', 'goodvoxels', 'ribbon-only', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-L_space-native_bold', 'hemi-R_space-native_bold', 'space-fsLR_den-32k_wb-spec', 'space-native_wb-spec', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion', 'space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion', 'space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-L_space-fsLR_den-32k_curv', 'hemi-R_space-fsLR_den-32k_curv', 'space-fsLR_den-32k_curv', 'hemi-L_space-fsLR_den-32k_flat', 'hemi-R_space-fsLR_den-32k_flat', 'hemi-L_space-fsLR_den-32k_inflated', 'hemi-R_space-fsLR_den-32k_inflated', 'hemi-L_space-fsLR_den-32k_veryinflated', 'hemi-R_space-fsLR_den-32k_veryinflated', 'hemi-L_space-native_inflated', 'hemi-R_space-native_inflated', 'hemi-L_space-native_veryinflated', 'hemi-R_space-native_veryinflated', 'hemi-L_space-fsLR_den-164k_midthickness', 'hemi-R_space-fsLR_den-164k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-native_midthickness', 'hemi-R_space-native_midthickness', 'hemi-L_space-fsLR_den-32k_pial', 'hemi-R_space-fsLR_den-32k_pial', 'hemi-L_space-native_den-32k_pial', 'hemi-R_space-native_den-32k_pial', 'hemi-L_space-fsLR_den-32k_sphere', 'hemi-R_space-fsLR_den-32k_sphere', 'hemi-L_space-native_desc-MSMSulc_sphere', 'hemi-R_space-native_desc-MSMSulc_sphere', 'hemi-L_space-native_sphere', 'hemi-R_space-native_sphere', 'hemi-L_space-native_desc-reg_sphere', 'hemi-R_space-native_desc-reg_sphere', 'hemi-L_space-native_desc-reg-reg_sphere', 'hemi-R_space-native_desc-reg-reg_sphere', 'hemi-L_space-native_desc-rot_sphere', 'hemi-R_space-native_desc-rot_sphere', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ', 'space-fsLR_den-32k_desc-FS_strainJ', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR', 'space-fsLR_den-32k_desc-FS_strainR', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR', 'space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-L_space-fsLR_den-32k_sulc', 'hemi-R_space-fsLR_den-32k_sulc', 'space-fsLR_den-32k_sulc', 'hemi-L_space-fsLR_den-32k_thickness', 'hemi-R_space-fsLR_den-32k_thickness', 'space-fsLR_den-32k_thickness', 'hemi-L_space-fsLR_den-164k_white', 'hemi-R_space-fsLR_den-164k_white', 'hemi-L_space-fsLR_den-32k_white', 'hemi-R_space-fsLR_den-32k_white', 'hemi-L_space-native_white', 'hemi-R_space-native_white', 'atlas-DesikanKilliany_space-fsLR_den-32k', 'atlas-Destrieux_space-fsLR_den-32k', 'atlas-DesikanKilliany_space-fsLR_den-164k', 'atlas-Destrieux_space-fsLR_den-164k', 'space-fsLR_den-32k_bold_surf_falff', 'space-fsLR_den-32k_bold_surf_alff', 'space-fsLR_den-32k_bold_surf-L_reho', 'space-fsLR_den-32k_bold_surf-R_reho', 'space-fsLR_den-32k_bold_surf-correlation_matrix']

bold_ts = ['desc-brain_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold']

ciftis = {'atlas-DesikanKilliany_space-fsLR_den-164k': 'dlabel', 'atlas-DesikanKilliany_space-fsLR_den-32k': 'dlabel', 'atlas-Destrieux_space-fsLR_den-164k': 'dlabel', 'atlas-Destrieux_space-fsLR_den-32k': 'dlabel', 'space-fsLR_den-32k_bold': 'dtseries', 'space-fsLR_den-32k_bold_surf-L_reho': 'dscalar', 'space-fsLR_den-32k_bold_surf-R_reho': 'dscalar', 'space-fsLR_den-32k_bold_surf-correlation_matrix': 'pconn', 'space-fsLR_den-32k_bold_surf_alff': 'dscalar', 'space-fsLR_den-32k_bold_surf_falff': 'dscalar', 'space-fsLR_den-32k_curv': 'dscalar', 'space-fsLR_den-32k_desc-FS_arealdistortion': 'dscalar', 'space-fsLR_den-32k_desc-FS_edgedistortion': 'dscalar', 'space-fsLR_den-32k_desc-FS_strainJ': 'dscalar', 'space-fsLR_den-32k_desc-FS_strainR': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_strainJ': 'dscalar', 'space-fsLR_den-32k_desc-MSMSulc_strainR': 'dscalar', 'space-fsLR_den-32k_sulc': 'dscalar', 'space-fsLR_den-32k_thickness': 'dscalar'}

debugging = ['desc-brain_bold', 'desc-motion_bold', 'desc-sm_bold', 'space-EPItemplate_desc-brain_bold', 'space-symtemplate_desc-sm_bold', 'space-template_desc-brain_bold', 'space-symtemplate_desc-brain_T1w', 'desc-brain_T1w', 'desc-reorient_T1w', 'space-template_desc-brain_T1w']

func = ['alff', 'desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-T1w_sbref', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'motion', 'desc-summary_motion', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'space-template_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'space-fsLR_den-32k_bold-dtseries', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'vmhc', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask', 'AtlasSubcortical-s2', 'space-fsLR_den-32k_bold', 'goodvoxels', 'ribbon-only', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-L_space-native_bold', 'hemi-R_space-native_bold', 'space-fsLR_den-32k_wb-spec', 'space-native_wb-spec', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion', 'space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion', 'space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-L_space-fsLR_den-32k_curv', 'hemi-R_space-fsLR_den-32k_curv', 'space-fsLR_den-32k_curv', 'hemi-L_space-fsLR_den-32k_flat', 'hemi-R_space-fsLR_den-32k_flat', 'hemi-L_space-fsLR_den-32k_inflated', 'hemi-R_space-fsLR_den-32k_inflated', 'hemi-L_space-fsLR_den-32k_veryinflated', 'hemi-R_space-fsLR_den-32k_veryinflated', 'hemi-L_space-native_inflated', 'hemi-R_space-native_inflated', 'hemi-L_space-native_veryinflated', 'hemi-R_space-native_veryinflated', 'hemi-L_space-fsLR_den-164k_midthickness', 'hemi-R_space-fsLR_den-164k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-native_midthickness', 'hemi-R_space-native_midthickness', 'hemi-L_space-fsLR_den-32k_pial', 'hemi-R_space-fsLR_den-32k_pial', 'hemi-L_space-native_den-32k_pial', 'hemi-R_space-native_den-32k_pial', 'hemi-L_space-fsLR_den-32k_sphere', 'hemi-R_space-fsLR_den-32k_sphere', 'hemi-L_space-native_desc-MSMSulc_sphere', 'hemi-R_space-native_desc-MSMSulc_sphere', 'hemi-L_space-native_sphere', 'hemi-R_space-native_sphere', 'hemi-L_space-native_desc-reg_sphere', 'hemi-R_space-native_desc-reg_sphere', 'hemi-L_space-native_desc-reg-reg_sphere', 'hemi-R_space-native_desc-reg-reg_sphere', 'hemi-L_space-native_desc-rot_sphere', 'hemi-R_space-native_desc-rot_sphere', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ', 'space-fsLR_den-32k_desc-FS_strainJ', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR', 'space-fsLR_den-32k_desc-FS_strainR', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR', 'space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-L_space-fsLR_den-32k_sulc', 'hemi-R_space-fsLR_den-32k_sulc', 'space-fsLR_den-32k_sulc', 'hemi-L_space-fsLR_den-32k_thickness', 'hemi-R_space-fsLR_den-32k_thickness', 'space-fsLR_den-32k_thickness', 'hemi-L_space-fsLR_den-164k_white', 'hemi-R_space-fsLR_den-164k_white', 'hemi-L_space-fsLR_den-32k_white', 'hemi-R_space-fsLR_den-32k_white', 'hemi-L_space-native_white', 'hemi-R_space-native_white', 'atlas-DesikanKilliany_space-fsLR_den-32k', 'atlas-Destrieux_space-fsLR_den-32k', 'atlas-DesikanKilliany_space-fsLR_den-164k', 'atlas-Destrieux_space-fsLR_den-164k', 'space-fsLR_den-32k_bold_surf_falff', 'space-fsLR_den-32k_bold_surf_alff', 'space-fsLR_den-32k_bold_surf-L_reho', 'space-fsLR_den-32k_bold_surf-R_reho', 'space-fsLR_den-32k_bold_surf-correlation_matrix']

functional_timeseries = ['desc-brain_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-preproc_bold']

giftis = {'hemi-L_space-fsLR_den-164k_midthickness': 'surf', 'hemi-L_space-fsLR_den-164k_white': 'surf', 'hemi-L_space-fsLR_den-32k_curv': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ': 'shape', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ': 'shape', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR': 'shape', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_bold': 'func', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask': 'shape', 'hemi-L_space-fsLR_den-32k_flat': 'surf', 'hemi-L_space-fsLR_den-32k_inflated': 'surf', 'hemi-L_space-fsLR_den-32k_midthickness': 'surf', 'hemi-L_space-fsLR_den-32k_pial': 'surf', 'hemi-L_space-fsLR_den-32k_sphere': 'surf', 'hemi-L_space-fsLR_den-32k_sulc': 'shape', 'hemi-L_space-fsLR_den-32k_thickness': 'shape', 'hemi-L_space-fsLR_den-32k_veryinflated': 'surf', 'hemi-L_space-fsLR_den-32k_white': 'surf', 'hemi-L_space-native_bold': 'func', 'hemi-L_space-native_den-32k_pial': 'surf', 'hemi-L_space-native_desc-MSMSulc_sphere': 'surf', 'hemi-L_space-native_desc-reg-reg_sphere': 'surf', 'hemi-L_space-native_desc-reg_sphere': 'surf', 'hemi-L_space-native_desc-rot_sphere': 'surf', 'hemi-L_space-native_inflated': 'surf', 'hemi-L_space-native_midthickness': 'surf', 'hemi-L_space-native_sphere': 'surf', 'hemi-L_space-native_veryinflated': 'surf', 'hemi-L_space-native_white': 'surf', 'hemi-R_space-fsLR_den-164k_midthickness': 'surf', 'hemi-R_space-fsLR_den-164k_white': 'surf', 'hemi-R_space-fsLR_den-32k_curv': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ': 'shape', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ': 'shape', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR': 'shape', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold': 'func', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask': 'shape', 'hemi-R_space-fsLR_den-32k_flat': 'surf', 'hemi-R_space-fsLR_den-32k_inflated': 'surf', 'hemi-R_space-fsLR_den-32k_pial': 'surf', 'hemi-R_space-fsLR_den-32k_sphere': 'surf', 'hemi-R_space-fsLR_den-32k_sulc': 'shape', 'hemi-R_space-fsLR_den-32k_thickness': 'shape', 'hemi-R_space-fsLR_den-32k_veryinflated': 'surf', 'hemi-R_space-fsLR_den-32k_white': 'surf', 'hemi-R_space-native_bold': 'func', 'hemi-R_space-native_den-32k_pial': 'surf', 'hemi-R_space-native_desc-MSMSulc_sphere': 'surf', 'hemi-R_space-native_desc-reg-reg_sphere': 'surf', 'hemi-R_space-native_desc-reg_sphere': 'surf', 'hemi-R_space-native_desc-rot_sphere': 'surf', 'hemi-R_space-native_inflated': 'surf', 'hemi-R_space-native_midthickness': 'surf', 'hemi-R_space-native_sphere': 'surf', 'hemi-R_space-native_veryinflated': 'surf', 'hemi-R_space-native_white': 'surf', 'space-fsLR_den-32k_wb-spec': 'spec', 'space-native_wb-spec': 'spec'}

native_nonsmooth = ['alff', 'falff', 'reho']

native_raw = ['alff', 'desc-sm_alff', 'desc-sm_falff', 'falff', 'desc-sm_reho', 'reho', 'desc-zstd-mdmr']

native_smooth = ['desc-sm_alff', 'desc-sm-zstd_alff', 'desc-zstd_alff', 'desc-brain_bold', 'desc-mean_bold', 'desc-motion_bold', 'desc-preproc_bold', 'desc-sm_bold', 'sbref', 'space-T1w_sbref', 'desc-sm_falff', 'desc-sm-zstd_falff', 'desc-zstd_falff', 'space-bold_desc-brain_mask', 'space-bold_desc-eroded_mask', 'space-bold_label-CSF_desc-eroded_mask', 'space-bold_label-CSF_mask', 'space-bold_label-GM_desc-eroded_mask', 'space-bold_label-GM_mask', 'space-bold_label-WM_desc-eroded_mask', 'space-bold_label-WM_mask', 'space-longitudinal_desc-brain_mask', 'space-longitudinal_label-CSF_desc-preproc_mask', 'space-longitudinal_label-CSF_mask', 'space-longitudinal_label-GM_desc-preproc_mask', 'space-longitudinal_label-GM_mask', 'space-longitudinal_label-WM_desc-preproc_mask', 'space-longitudinal_label-WM_mask', 'label-CSF_desc-eroded_mask', 'label-CSF_desc-preproc_mask', 'label-CSF_mask', 'label-GM_desc-eroded_mask', 'label-GM_desc-preproc_mask', 'label-GM_mask', 'label-WM_desc-eroded_mask', 'label-WM_desc-preproc_mask', 'label-WM_mask', 'space-T1w_desc-acpcbrain_mask', 'space-T1w_desc-brain_mask', 'space-T1w_desc-eroded_mask', 'motion', 'desc-summary_motion', 'motion-filter-plot', 'desc-movementParameters_motion', 'desc-movementParametersUnfiltered_motion', 'label-CSF_probseg', 'label-GM_probseg', 'label-WM_probseg', 'desc-T1wAxial_quality', 'desc-T1wSagittal_quality', 'desc-dsegAxial_quality', 'desc-dsegSagittal_quality', 'desc-boldAxial_quality', 'desc-boldSagittal_quality', 'desc-boldCarpet_quality', 'desc-framewiseDisplacementJenkinsonPlot_quality', 'desc-movementParametersTrans_quality', 'desc-movementParametersRot_quality', 'desc-boldSnrAxial_quality', 'desc-boldSnrSagittal_quality', 'desc-boldSnrHist_quality', 'desc-boldSnr_quality', 'space-template_desc-xcp_quality', 'desc-confounds_timeseries', 'desc-sm_reho', 'desc-sm-zstd_reho', 'desc-zstd_reho', 'hemi-L_desc-surfaceMap_thickness', 'hemi-R_desc-surfaceMap_thickness', 'hemi-L_desc-surfaceMap_volume', 'hemi-R_desc-surfaceMap_volume', 'hemi-L_desc-surfaceMesh_pial', 'hemi-R_desc-surfaceMesh_pial', 'raw-average', 'hemi-L_desc-surfaceMesh_smoothwm', 'hemi-R_desc-surfaceMesh_smoothwm', 'atlas-DesikanKilliany_space-fsLR_den-32k_dlabel', 'atlas-Destrieux_space-fsLR_den-32k_dlabel', 'atlas-DesikanKilliany_space-fsLR_den-164k_dlabel', 'atlas-Destrieux_space-fsLR_den-164k_dlabel', 'space-fsLR_den-32k_bold-dtseries', 'hemi-L_desc-surfaceMesh_sphere', 'hemi-R_desc-surfaceMesh_sphere', 'hemi-L_desc-surfaceMap_sulc', 'hemi-R_desc-surfaceMap_sulc', 'hemi-L_desc-surface_curv', 'hemi-R_desc-surface_curv', 'hemi-L_desc-surfaceMesh_white', 'hemi-R_desc-surfaceMesh_white', 'wmparc', 'desc-brain_T1w', 'desc-head_T1w', 'desc-preproc_T1w', 'desc-reorient_T1w', 'desc-restore_T1w', 'desc-restore-brain_T1w', 'space-template_desc-Mean_timeseries', 'desc-MeanSCA_timeseries', 'desc-SpatReg_timeseries', 'desc-Voxel_timeseries', 'space-longitudinal_label-CSF_probseg', 'space-longitudinal_label-GM_probseg', 'space-longitudinal_label-WM_probseg', 'blip-warp', 'from-bold_to-EPItemplate_mode-image_desc-linear_xfm', 'from-bold_to-EPItemplate_mode-image_desc-nonlinear_xfm', 'from-bold_to-EPItemplate_mode-image_xfm', 'from-bold_to-symtemplate_mode-image_xfm', 'from-bold_to-T1w_mode-image_desc-linear_xfm', 'from-bold_to-template_mode-image_xfm', 'from-EPItemplate_to-bold_mode-image_desc-linear_xfm', 'from-EPItemplate_to-bold_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-linear_xfm', 'from-longitudinal_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-symtemplate_mode-image_xfm', 'from-longitudinal_to-template_mode-image_desc-linear_xfm', 'from-longitudinal_to-template_mode-image_desc-nonlinear_xfm', 'from-longitudinal_to-template_mode-image_xfm', 'from-symtemplate_to-bold_mode-image_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-linear_xfm', 'from-symtemplate_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-longitudinal_mode-image_xfm', 'from-symtemplate_to-T1w_mode-image_desc-linear_xfm', 'from-symtemplate_to-T1w_mode-image_desc-nonlinear_xfm', 'from-symtemplate_to-T1w_mode-image_xfm', 'from-T1w_to-symtemplate_mode-image_desc-linear_xfm', 'from-T1w_to-symtemplate_mode-image_desc-nonlinear_xfm', 'from-T1w_to-symtemplate_mode-image_xfm', 'from-T1w_to-template_mode-image_desc-linear_xfm', 'from-T1w_to-template_mode-image_desc-nonlinear_xfm', 'from-T1w_to-template_mode-image_xfm', 'from-template_to-bold_mode-image_xfm', 'from-template_to-longitudinal_mode-image_desc-linear_xfm', 'from-template_to-longitudinal_mode-image_desc-nonlinear_xfm', 'from-template_to-longitudinal_mode-image_xfm', 'from-template_to-T1w_mode-image_desc-linear_xfm', 'from-template_to-T1w_mode-image_desc-nonlinear_xfm', 'from-template_to-T1w_mode-image_xfm', 'mdmr', 'desc-zstd-mdmr', 'dseg', 'AtlasSubcortical-s2', 'space-fsLR_den-32k_bold', 'goodvoxels', 'ribbon-only', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_bold', 'hemi-L_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-R_space-fsLR_den-32k_desc-atlasroi_mask', 'hemi-L_space-native_bold', 'hemi-R_space-native_bold', 'space-fsLR_den-32k_wb-spec', 'space-native_wb-spec', 'hemi-L_space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_arealdistortion', 'space-fsLR_den-32k_desc-FS_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'space-fsLR_den-32k_desc-MSMSulc_arealdistortion', 'hemi-L_space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-FS_edgedistortion', 'space-fsLR_den-32k_desc-FS_edgedistortion', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'space-fsLR_den-32k_desc-MSMSulc_edgedistortion', 'hemi-L_space-fsLR_den-32k_curv', 'hemi-R_space-fsLR_den-32k_curv', 'space-fsLR_den-32k_curv', 'hemi-L_space-fsLR_den-32k_flat', 'hemi-R_space-fsLR_den-32k_flat', 'hemi-L_space-fsLR_den-32k_inflated', 'hemi-R_space-fsLR_den-32k_inflated', 'hemi-L_space-fsLR_den-32k_veryinflated', 'hemi-R_space-fsLR_den-32k_veryinflated', 'hemi-L_space-native_inflated', 'hemi-R_space-native_inflated', 'hemi-L_space-native_veryinflated', 'hemi-R_space-native_veryinflated', 'hemi-L_space-fsLR_den-164k_midthickness', 'hemi-R_space-fsLR_den-164k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-fsLR_den-32k_midthickness', 'hemi-L_space-native_midthickness', 'hemi-R_space-native_midthickness', 'hemi-L_space-fsLR_den-32k_pial', 'hemi-R_space-fsLR_den-32k_pial', 'hemi-L_space-native_den-32k_pial', 'hemi-R_space-native_den-32k_pial', 'hemi-L_space-fsLR_den-32k_sphere', 'hemi-R_space-fsLR_den-32k_sphere', 'hemi-L_space-native_desc-MSMSulc_sphere', 'hemi-R_space-native_desc-MSMSulc_sphere', 'hemi-L_space-native_sphere', 'hemi-R_space-native_sphere', 'hemi-L_space-native_desc-reg_sphere', 'hemi-R_space-native_desc-reg_sphere', 'hemi-L_space-native_desc-reg-reg_sphere', 'hemi-R_space-native_desc-reg-reg_sphere', 'hemi-L_space-native_desc-rot_sphere', 'hemi-R_space-native_desc-rot_sphere', 'hemi-L_space-fsLR_den-32k_desc-FS_strainJ', 'hemi-R_space-fsLR_den-32k_desc-FS_strainJ', 'space-fsLR_den-32k_desc-FS_strainJ', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainJ', 'space-fsLR_den-32k_desc-MSMSulc_strainJ', 'hemi-L_space-fsLR_den-32k_desc-FS_strainR', 'hemi-R_space-fsLR_den-32k_desc-FS_strainR', 'space-fsLR_den-32k_desc-FS_strainR', 'hemi-L_space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-R_space-fsLR_den-32k_desc-MSMSulc_strainR', 'space-fsLR_den-32k_desc-MSMSulc_strainR', 'hemi-L_space-fsLR_den-32k_sulc', 'hemi-R_space-fsLR_den-32k_sulc', 'space-fsLR_den-32k_sulc', 'hemi-L_space-fsLR_den-32k_thickness', 'hemi-R_space-fsLR_den-32k_thickness', 'space-fsLR_den-32k_thickness', 'hemi-L_space-fsLR_den-164k_white', 'hemi-R_space-fsLR_den-164k_white', 'hemi-L_space-fsLR_den-32k_white', 'hemi-R_space-fsLR_den-32k_white', 'hemi-L_space-native_white', 'hemi-R_space-native_white', 'atlas-DesikanKilliany_space-fsLR_den-32k', 'atlas-Destrieux_space-fsLR_den-32k', 'atlas-DesikanKilliany_space-fsLR_den-164k', 'atlas-Destrieux_space-fsLR_den-164k', 'space-fsLR_den-32k_bold_surf_falff', 'space-fsLR_den-32k_bold_surf_alff', 'space-fsLR_den-32k_bold_surf-L_reho', 'space-fsLR_den-32k_bold_surf-R_reho', 'space-fsLR_den-32k_bold_surf-correlation_matrix']

reference = Resource  ... Multi-File 0                                               alff  ...            1                                       desc-sm_alff  ...            2                                  desc-sm-zstd_alff  ...            3                                     desc-zstd_alff  ...            4                                space-template_alff  ...            ..                                               ...  ...        ... 306               space-fsLR_den-32k_bold_surf_falff  ...            307                space-fsLR_den-32k_bold_surf_alff  ...            308              space-fsLR_den-32k_bold_surf-L_reho  ...            309              space-fsLR_den-32k_bold_surf-R_reho  ...            310  space-fsLR_den-32k_bold_surf-correlation_matrix  ...             [311 rows x 10 columns]

reference_csv = '/home/circleci/docs_env/lib/python3.10/site-packages/CPAC/resources/cpac_outputs.tsv'

template_nonsmooth = ['space-template_alff', 'space-template_dcb', 'space-template_dcw', 'space-template_ecb', 'space-template_ecw', 'space-template_falff', 'space-template_lfcdb', 'space-template_lfcdw', 'space-template_reho']

template_raw = ['space-template_alff', 'space-template_desc-sm_alff', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'space-template_desc-sm_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm_reho', 'space-template_reho']

template_smooth = ['space-template_desc-sm_alff', 'space-template_desc-sm-zstd_alff', 'space-template_desc-zstd_alff', 'space-EPItemplate_bold', 'space-EPItemplate_desc-brain_bold', 'space-EPItemplate_desc-mean_bold', 'space-EPItemplate_desc-preproc_bold', 'space-symtemplate_desc-sm_bold', 'space-template_bold', 'space-template_desc-brain_bold', 'space-template_desc-head_bold', 'space-template_desc-mean_bold', 'space-template_desc-preproc_bold', 'space-template_desc-scout_bold', 'space-template_sbref', 'space-template_desc-DualReg_correlations', 'space-template_desc-MeanSCA_correlations', 'space-template_desc-MultReg_correlations', 'space-template_desc-ndmg_correlations', 'space-template_desc-PearsonAfni_correlations', 'space-template_desc-PartialAfni_correlations', 'space-template_desc-PearsonNilearn_correlations', 'space-template_desc-PartialNilearn_correlations', 'space-template_desc-sm_dcb', 'space-template_desc-sm-zstd_dcb', 'space-template_desc-zstd_dcb', 'space-template_desc-sm_dcw', 'space-template_desc-sm-zstd_dcw', 'space-template_desc-zstd_dcw', 'space-template_desc-sm_ecb', 'space-template_desc-sm-zstd_ecb', 'space-template_desc-zstd_ecb', 'space-template_desc-sm_ecw', 'space-template_desc-sm-zstd_ecw', 'space-template_desc-zstd_ecw', 'space-template_desc-sm_falff', 'space-template_desc-sm-zstd_falff', 'space-template_desc-zstd_falff', 'space-template_desc-sm_lfcdb', 'space-template_desc-sm-zstd_lfcdb', 'space-template_desc-zstd_lfcdb', 'space-template_desc-sm_lfcdw', 'space-template_desc-sm-zstd_lfcdw', 'space-template_desc-zstd_lfcdw', 'space-EPItemplate_desc-bold_mask', 'space-EPItemplate_res-derivative_desc-bold_mask', 'space-template_desc-brain_mask', 'space-template_desc-bold_mask', 'space-template_res-derivative_desc-bold_mask', 'space-template_desc-sm_reho', 'space-template_desc-sm-zstd_reho', 'space-template_desc-zstd_reho', 'desc-DualReg_statmap', 'desc-MultReg_statmap', 'space-symtemplate_desc-brain_T1w', 'space-template_desc-brain_T1w', 'space-template_desc-preproc_T1w', 'space-template_desc-head_T1w', 'space-template_desc-T1w_mask', 'vmhc', 'space-template_label-CSF_mask', 'space-template_label-WM_mask', 'space-template_label-GM_mask', 'space-EPItemplate_label-CSF_mask', 'space-EPItemplate_label-WM_mask', 'space-EPItemplate_label-GM_mask']

to_fisherz = []

to_smooth = ['alff', 'space-template_alff', 'space-template_dcb', 'space-template_dcw', 'space-template_ecb', 'space-template_ecw', 'falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_lfcdw', 'reho', 'space-template_reho']

to_zstd = ['alff', 'desc-sm_alff', 'space-template_alff', 'space-template_desc-sm_alff', 'space-template_dcb', 'space-template_desc-sm_dcb', 'space-template_dcw', 'space-template_desc-sm_dcw', 'space-template_ecb', 'space-template_desc-sm_ecb', 'space-template_ecw', 'space-template_desc-sm_ecw', 'desc-sm_falff', 'falff', 'space-template_desc-sm_falff', 'space-template_falff', 'space-template_lfcdb', 'space-template_desc-sm_lfcdb', 'space-template_lfcdw', 'space-template_desc-sm_lfcdw', 'desc-sm_reho', 'reho', 'space-template_desc-sm_reho', 'space-template_reho', 'desc-zstd-mdmr']

pytest

Utilities for Pytest integration

CPAC.utils.pytest.skipif(condition, reason)

Skip test if we have Pytest, ignore test entirely if not

Serialization

Strategy

class CPAC.utils.strategy.Strategy

append_name(name)

fork()

get(resource_key)

static get_forking_labels(strategies)

static get_forking_points(strategies)

get_leaf_properties()

get_name()

get_node_from_resource_pool(resource_key)

get_nodes_names()

get_resource_pool()

property resource_pool

Strategy’s ResourcePool dict

property rpool

Strategy’s ResourcePool

set_leaf_properties(node, out_file)

update_resource_pool(resources, override=False)

Symlinks

CPAC.utils.symlinks.compile_strategy_name(strategy)

CPAC.utils.symlinks.create_paths_to_symlinks(output_dir, pipeline_id, subject_id, paths)

CPAC.utils.symlinks.create_symlinks(output_dir, pipeline_id, subject_id, paths, relative=True, symlink_dir='sym_links')

CPAC.utils.symlinks.group_files_in_strategies(output_dir, paths)

Test Initialization

This module contains functions that assist in initializing CPAC tests resources

CPAC.utils.test_init.concordance(x, y)

Return the concordance correlation coefficient as defined by Lin (1989)

Parameters:

• x (list or array) – a list of array of length N of numbers

• y (list or array) – a list of array of length N of numbers

Returns:

rho_c – the concordance value as a float

Return type:

numpy.float32

CPAC.utils.test_init.create_dummy_node(name: str, fields: list[str] | None = None)

Create a dummy IdentityInterface Node source for resources upstream in a graph from a section to be tested

Parameters:

• name (str) – a name for the dummy Node

• fields (list of str, optional) – a list of resources to be present in the created Node. If not provided, the only resource will be called ‘resource’

Return type:

Node

CPAC.utils.test_init.default_bucket_name()

Function to return the default S3 bucket name used in test suite

Parameters:

None –

Returns:

bucket_name – default S3 bucket name for testing

Return type:

string

CPAC.utils.test_init.download_cpac_resources_from_s3(local_base)

Function to download the CPAC testing resources directory from S3

Parameters:

local_base (string) – the local directory to save the ‘cpac_resources’ contents

CPAC.utils.test_init.download_resource_from_s3(s3_url_path)

CPAC.utils.test_init.pearson_correlation(nii_1, nii_2)

CPAC.utils.test_init.populate_all_templates()

Function to populate all of the template files

Parameters:

None –

Return type:

None

CPAC.utils.test_init.populate_template_config(config_type: str) → str

Function to read in a template config file from the CPAC_RESOURCE_DIR and populate it with actual filepaths

Parameters:

config_type (string) – config file to populate; accepts ‘data_config’ and ‘pipeline_config’

Returns:

config_test – filepath to the newly written config file for testing

Return type:

string

CPAC.utils.test_init.return_all_niis(base_dir)

Function to walk through a base directory and all subsequent files and return the filepaths of all nifti files found

Parameters:

base_dir (string) – filepath to the base directory to search through

Returns:

nii_list – a list of filepath strings of the nifti files found in base_dir

Return type:

list

CPAC.utils.test_init.return_aws_creds()

Function to return the AWS credentials file given by the CPAC_AWS_CREDS environment variable

Parameters:

None –

Returns:

aws_creds – filepath to the AWS credentials with access key id and secret access key

Return type:

string

CPAC.utils.test_init.return_resource_dir()

Function to return the filepath of the CPAC_RESOURCE_DIR; note the CPAC_RESOURCE_DIR environment variable must be set

Parameters:

None –

Returns:

resource_dir – the file path on disk where the cpac resources folder is

Return type:

string

CPAC.utils.test_init.return_resource_subfolder(subfolder)

Funnction to return subfolders of the CPAC_RESOURCE_DIR

Parameters:

subfolder (string) – subfolder name to return path of

Returns:

resource_subfolder – filepath to the resource subfolder

Return type:

string

CPAC.utils.test_init.return_strats_obj()

Function to return the file path of the strategies obj file from the CPAC_RESOURCE_DIR

Parameters:

None –

Returns:

strats_obj – filepath to the strategies obj file

Return type:

string

CPAC.utils.test_init.return_subj_measure_dirs(measure)

Function to grab the base directories of the test subject’s output files for a given measure or workflow

Parameters:

measure (string) – the measure or workflow or derivative of interest to parse for; this must be the folder name where all of the subject’s test outputs are located (e.g. ‘network_centrality’)

Returns:

subj_measure_dirs – a list of strings of the base directories for each instance of the desired measure folder within the test subjects outputs

Return type:

list

CPAC.utils.test_init.return_subject_list()

Function to return the file path of the subject list file from the CPAC_RESOURCE_DIR

Parameters:

None –

Returns:

subject_list – filepath to the subject list yaml file

Return type:

string

CPAC.utils.test_init.return_test_subj()

Function to return the subject id; note the CPAC_RESOURCE_DIR environment variable must be set

Parameters:

None –

Returns:

resource_dir – the file path on disk where the cpac resources folder is

Return type:

string

CPAC.utils.test_init.setup_test_logger(logger_name, log_file, level, to_screen=False)

Function to initialize and configure a logger that can write to file and (optionally) the screen.

Parameters:

• logger_name (string) – name of the logger

• log_file (string) – file path to the log file on disk

• level (integer) – indicates the level at which the logger should log; this is controlled by integers that come with the python logging package. (e.g. logging.INFO=20, logging.DEBUG=10)

• to_screen (boolean (optional)) – flag to indicate whether to enable logging to the screen

Returns:

logger – Python logging.Logger object which is capable of logging run- time information about the program to file and/or screen

Return type:

logging.Logger object

CPAC.utils.test_init.smooth_nii_file(self, nii_file, fwhm, mask_file=None)

Function to Gaussian smooth nifti files and optionally using a mask on the smoothed data

Parameters:

• nii_file (string) – filepath to the nifti file to smooth

• fwhm (float) – FWHM for Gaussian smoothing kernel, in mm

• mask_file (string (optional); default=None) – filepath to the mask file to use

Returns:

smooth_arr – smoothed nifti image as a numpy array

Return type:

numpy.ndarray

Test Mocks

CPAC.utils.test_mocks.configuration_strategy_mock(method='FSL')

CPAC.utils.test_mocks.file_node(path, file_node_num=0)

Test Resources

CPAC.utils.test_resources.setup_test_wf(s3_prefix, paths_list, test_name, workdirs_to_keep=None)

Set up a basic template Nipype workflow for testing single nodes or small sub-workflows.

Tests

The Trimmer

CPAC.utils.trimmer.compute_datasink_dirs(graph, datasink, output_dir=None, container=None)

CPAC.utils.trimmer.expand_workflow(wf)

CPAC.utils.trimmer.is_datasink(n)

CPAC.utils.trimmer.list_files(path, s3_creds_path=None)

CPAC.utils.trimmer.the_trimmer(wf, output_dir=None, container=None, s3_creds_path=None)

The trimmer: trimming your workflow based on its datasinks.

The assumption here is that all your desired outputs will be placed in an output directory by using a DataSink node.

By analysing a specific output directory, and comparing what is in it with the DataSinks of a workflow, it is possible to audit which Datasinks have already outputted to the output directory. If the DataSink has already fulfilled its function, we infer that previous nodes also had, since they are prerequisites for the DataSink to run. This is the simplest case in which we can prune nodes (i.e. do not execute them), optimizing the execution time.

A brief syntax note: [node] is a node → is a connection, disconsidering which field/attribute →(field)→ indicates a connection considering the field ✓ is a datasink with an existing file ❌ is a datasing witn an missing file

E.g.

[node1] → [node2] → [node3] → [datasink to file.txt ✓]

since file.txt already exist, there is no need to execute the [node3]. Since [node2] only outputs to [node3], and [node3] will not be executed, we can prune [node2]. Same for [node1]. In this case, our workflow will not have any more nodes to run.

There are more complex cases:

1) A node outputs for several nodes, and some of their results are not in the output directory.

[node1] → [node2] → [node3] → [datasink to file1.txt ✔]

↳ [node4] → [datasink to file2.txt ❌]

for this case, we cannot prune [node1], since its output is used in another branch, for [node4], that is not cached. After trimming, the remaining workflow is:

[node1]

↳ [node4] → [datasink to file2.txt ❌]

2) The node has several outputs, and an uncached branch down the graph requires one of its outputs.

[registration] →(warped image)→ [datasink to warped.nii.gz ✔]

↳(transforms)→ [apply transforms] → [datasink to func_warped.nii.gz ❌] [functional] ↗

given func_warped.nii.gz is not cached, we need to perform “apply transforms”, that requires the transforms from the [registration] node. In this case, even that warped.nii.gz is cached, we will reexecute the [registration] again to get the transforms. After trimming, the remaining workflow is:

[registration]

↳(transforms)→ [apply transforms] → [datasink to func_warped.nii.gz ❌] [functional] ↗

For this implementation, we disregard MapNodes, as their outputs is harder to check.

Iterables are considered in the implementation by expanding the original workflow into what is called an execution graph, creating a node for each iterable value.

Parameters:

• wf (Workflow) – A Nipype workflow to be pruned.

• output_dir (Path) – The directory in which the outputs are stored. If not provided, value is inferred from the DataSink nodes.

• container (Path) – The subdirectory from the output_dir in which the output are stored. If not provided, value is inferred from the DataSink nodes.

• s3_creds_path (Path) – Path to S3 credentials, in case output_dir is in a S3 bucket.

Returns:

• wf_new (Workflow) – Prunned workflow

• (replacement_mapping, deletions) ((Dict, List)) – replacement_mapping contains the nodes replaces with input nodes, pointing to files from the output_dir

  deletions contains the nodes removed from the workflow, as they do not need to be executed

Typing

Helpers and aliases for handling typing in main and variant Python versions.

Once all variants (see https://fcp-indi.github.io/docs/nightly/user/versions#variants) run Python ≥ 3.10, these global variables can be replaced with the current preferred syntax.

CPAC.utils.typing.DICT

alias of dict

CPAC.utils.typing.ITERABLE

alias of Iterable

CPAC.utils.typing.LIST

alias of list

CPAC.utils.typing.TUPLE

alias of tuple

Utilities

CPAC.utils.utils.add_afni_prefix(tpattern)

CPAC.utils.utils.check(params_dct, subject_id, scan_id, val_to_check, throw_exception)

CPAC.utils.utils.check_command_path(path)

CPAC.utils.utils.check_config_resources(c)

CPAC.utils.utils.check_prov_for_motion_tool(prov)

CPAC.utils.utils.check_prov_for_regtool(prov)

CPAC.utils.utils.check_random_state(seed)

Turn seed into a np.random.RandomState instance Code from scikit-learn (https://github.com/scikit-learn/scikit-learn)

Parameters:

seed (None | int | instance of RandomState) – If seed is None, return the RandomState singleton used by np.random. If seed is an int, return a new RandomState instance seeded with seed. If seed is already a RandomState instance, return it. Otherwise raise ValueError.

CPAC.utils.utils.check_system_deps(check_ants=False, check_ica_aroma=False, check_centrality_degree=False, check_centrality_lfcd=False)

Function to check system for neuroimaging tools AFNI, C3D, FSL, and (optionally) ANTs

CPAC.utils.utils.check_tr(tr, in_file)

CPAC.utils.utils.compute_fisher_z_score(correlation_file, timeseries_one_d, input_name)

Computes the fisher z transform of the input correlation map If the correlation map contains data for multiple ROIs then the function returns z score for each ROI as a seperate nifti file

Parameters:

correlation_file (string) – Input correlations file

Returns:

out_file – list of z_scores for mask or ROI

Return type:

list (nifti files)

CPAC.utils.utils.concat_list(in_list1=None, in_list2=None)

Parameters:

• in_list1 (list or str) – file path or a list of file paths

• in_list2 (list or str) – file path or a list of file paths

Returns:

out_list – a list of file paths

Return type:

list

CPAC.utils.utils.correlation(matrix1, matrix2, match_rows=False, z_scored=False, symmetric=False)

CPAC.utils.utils.create_id_string(cfg, unique_id, resource, scan_id=None, template_desc=None, atlas_id=None, fwhm=None, subdir=None, extension=None)

Create the unique key-value identifier string for BIDS-Derivatives compliant file names.

This is used in the file renaming performed during the Datasink connections.

Example

>>> from CPAC.utils.configuration import Configuration
>>> create_id_string(Configuration(), 'sub-1_ses-1',
...                  'res-derivative_desc-Mean-1_timeseries',
...                  scan_id='rest', atlas_id='Yeo_desc-7')
'sub-1_ses-1_task-rest_atlas-Yeo7_res-3mm_desc-Mean1_timeseries'

CPAC.utils.utils.create_log(wf_name='log', scan_id=None)

Workflow to create log

CPAC.utils.utils.create_output_mean_csv(subject_dir)

this function finds all of the mean_{output}.txt files in the subject’s output directory, collects the data and organizes them into one .csv file in the subject directory

CPAC.utils.utils.delete_nested_value(d, keys)

Helper function to delete nested values

Parameters:

• d (dict) –

• keys (list or tuple) –

Returns:

updated

Return type:

dict

Examples

>>> delete_nested_value(
...     {'nested': {'key1': 'value', 'key2': 'value'}},
...     ['nested', 'key1'])
{'nested': {'key2': 'value'}}

CPAC.utils.utils.extract_one_d(list_timeseries)

CPAC.utils.utils.extract_output_mean(in_file, output_name)

function takes ‘in_file’, which should be an intermediary 1D file from individual-level analysis, containing the mean of the output across all voxels

it then parses this value and writes it to a .csv file named output_means.csv located in the subject’s output directory

CPAC.utils.utils.extract_txt(list_timeseries)

Method to extract txt file containing roi timeseries required for dual regression

CPAC.utils.utils.find_files(directory, pattern)

CPAC.utils.utils.get_fisher_zscore(input_name, map_node=False, wf_name='fisher_z_score')

Runs the compute_fisher_z_score function as part of a one-node workflow.

CPAC.utils.utils.get_flag(in_flag)

CPAC.utils.utils.get_flag_wf(wf_name='get_flag')

CPAC.utils.utils.get_last_prov_entry(prov)

CPAC.utils.utils.get_operand_string(mean, std_dev)

Method to get operand string for Fsl Maths

Parameters:

• mean (string) – path to img containing mean

• std_dev (string) – path to img containing standard deviation

Returns:

op_string – operand string

Return type:

string

CPAC.utils.utils.get_roi_num_list(timeseries_file, prefix=None)

CPAC.utils.utils.get_scan_params(subject_id, scan, pipeconfig_start_indx, pipeconfig_stop_indx, data_config_scan_params=None)

Method to extract slice timing correction parameters and scan parameters.

Parameters:

• subject_id (str) – subject id

• scan (str) – scan id

• pipeconfig_start_indx (int) – starting volume index as provided in the pipeline config yaml file

• pipeconfig_stop_indx (int) – ending volume index as provided in the pipeline config yaml file

• data_config_scan_params (str) – file path to scan parameter JSON file listed in data config yaml file

Returns:

• TR (a string) – TR value

• pattern (a string) – slice aquisition pattern string or file path

• ref_slice (an integer) – reference slice which is used to allign all other slices

• first_tr (an integer) – starting TR or starting volume index

• last_tr (an integer) – ending TR or ending volume index

• pe_direction (str)

• effective_echo_spacing (float)

CPAC.utils.utils.get_tr(tr)

Method to return TR in seconds

CPAC.utils.utils.get_zscore(map_node=False, wf_name='z_score')

Workflow to calculate z-scores

Parameters:

wf_name (string) – name of the workflow

Returns:

wf

Return type:

workflow object

Notes

Source

Workflow Inputs:

inputspec.input_file : string
    path to input functional derivative file for which z score has to be calculated
inputspec.mask_file : string
    path to whole brain functional mask file required to calculate zscore

Workflow Outputs:

outputspec.z_score_img : string
     path to image containing Normalized Input Image Z scores across full brain.

High Level Workflow Graph:

Detailed Workflow Graph:

Example

>>> wf = get_zscore('example_input')
>>> wf.inputs.inputspec.input_file = '/home/data/graph_working_dir/calculate_centrality/degree_centrality_binarize.nii.gz'
>>> wf.inputs.inputspec.mask_file = '/home/data/graphs/GraphGeneration/new_mask_3m.nii.gz'
>>> wf.run()  

CPAC.utils.utils.list_item_replace(l, old, new)

Function to replace an item in a list

Parameters:

• l (list or string) –

• old (any) – item to replace

• new (any) – new item

Returns:

l – updated

Return type:

list or string

Examples

>>> list_item_replace(['AFNI', 'FSL'], 'AFNI', '3dSkullStrip')
['3dSkullStrip', 'FSL']
>>> list_item_replace(['AFNI', 'FSL'], 'FSL', 'BET')
['AFNI', 'BET']

CPAC.utils.utils.lookup_nested_value(d, keys)

Helper method to look up nested values

Parameters:

• d (dict) –

• keys (list or tuple) –

Returns:

yaml

Return type:

str or dict

Examples

>>> lookup_nested_value({'nested': {'True': True}}, ['nested', 'True'])
True
>>> lookup_nested_value({'nested': {'None': None}}, ['nested', 'None'])
''

CPAC.utils.utils.ordereddict_to_dict(value)

this function convert ordereddict into regular dict

CPAC.utils.utils.pick_wm(seg_prob_list)

CPAC.utils.utils.read_json(json_file)

CPAC.utils.utils.remove_False(d, k)

Function to remove “Off” and False from a list at a given nested key.

Parameters:

• d (dict) –

• k (list) –

Returns:

d – updated

Return type:

dict

Examples

>>> remove_False({'a': {'b': [1, False, 2, "Off", 3]}}, ['a', 'b'])
{'a': {'b': [1, 2, 3]}}

CPAC.utils.utils.remove_None(d, k)

Function to remove “None” and None from a list at a given nested key.

Parameters:

• d (dict) –

• k (list) –

Returns:

d – updated

Return type:

dict

Examples

>>> remove_None({'a': {'b': [1, None, 2, "None", 3]}}, ['a', 'b'])
{'a': {'b': [1, 2, 3]}}

CPAC.utils.utils.repickle(directory)

Function to check all of the pickles in a given directory, recursively, and convert any Python 2 pickles found to Python 3 pickles.

Parameters:

directory (str) –

Return type:

None

CPAC.utils.utils.replace_in_strings(d, replacements=None)

Helper function to recursively replace substrings.

Parameters:

• d (any) –

• replacements (list of 2-tuples) –

  0str

  substring to replace

  1str

  replacement substring

Returns:

d – same as input, but updated

Return type:

any

Examples

>>> replace_in_strings({'key': 'test${resolution_for_func_preproc}'})
{'key': 'test${func_resolution}'}

CPAC.utils.utils.safe_shape(*vol_data)

Checks if the volume (first three dimensions) of multiple ndarrays are the same shape.

Parameters:

• vol_data0 (ndarray) – Volumes to check

• vol_data1 (ndarray) – Volumes to check

• ... (ndarray) – Volumes to check

• vol_datan (ndarray) – Volumes to check

Returns:

same_volume – True only if all volumes have the same shape.

Return type:

bool

CPAC.utils.utils.set_nested_value(d, keys, value)

Helper method to set nested values

Parameters:

• d (dict) –

• keys (list or tuple) –

• value (any) –

Returns:

updated

Return type:

dict

Examples

>>> set_nested_value({}, ['nested', 'keys'], 'value')
{'nested': {'keys': 'value'}}

CPAC.utils.utils.try_fetch_parameter(scan_parameters, subject, scan, keys)

CPAC.utils.utils.update_config_dict(old_dict)

Function to convert an old config dict to a new config dict

Parameters:

old_dict (dict) –

Returns:

• new_dict (dict) – 1.8 nested config dictionary

• old_dict (dict) – remaining undefined mappings

• combined_dict (dict) – 1.8 nested config dictionary plus remaining undefined mappings

Examples

>>> a, b, c = update_config_dict({
...     'pipelineName': 'example-pipeline', '2': None})
>>> a
{'pipeline_setup': {'pipeline_name': 'example-pipeline'}}
>>> b
{'2': None}
>>> c
{'pipeline_setup': {'pipeline_name': 'example-pipeline'}, '2': None}

CPAC.utils.utils.update_nested_dict(d_base, d_update, fully_specified=False)

Update dictionary of varying depth.

Parameters:

• d_base (dict) – original dictionary

• d_update (dict) – dictionary with updates

• fully_specified (bool) – if True, overwrite instead of update

Returns:

d_base – original dictionary with updates

Return type:

dict

Examples

>>> d_base = {'pipeline_name': 'cpac-default-pipeline',
...     'output_directory': {'path': '/output',
...     'write_func_outputs': False,
...     'write_debugging_outputs': False,
...     'output_tree': 'default',
...     'quality_control': {
...         'generate_quality_control_images': True,
...         'generate_xcpqc_files': True}},
...     'working_directory': {'path': '/tmp', 'remove_working_dir': True},
...     'log_directory': {'run_logging': True, 'path': '/logs'},
...     'system_config': {'maximum_memory_per_participant': 1,
...     'max_cores_per_participant': 1,
...     'num_ants_threads': 4,
...     'num_participants_at_once': 1},
...     'Amazon-AWS': {'aws_output_bucket_credentials': None,
...                    's3_encryption': False}}
>>> d_update = {'pipeline_name': 'cpac_fmriprep-options',
...     'system_config': {'num_ants_threads': 1},
...     'Amazon-AWS': {'s3_encryption': True}}
>>> str(update_nested_dict(d_base, d_update)) == str({
...     'pipeline_name': 'cpac_fmriprep-options', 'output_directory': {
...         'path': '/output', 'write_func_outputs': False,
...         'write_debugging_outputs': False, 'output_tree': 'default',
...         'quality_control': {
...             'generate_quality_control_images': True,
...             'generate_xcpqc_files': True}
...     }, 'working_directory': {
...        'path': '/tmp', 'remove_working_dir': True
...     }, 'log_directory': {'run_logging': True, 'path': '/logs'},
...     'system_config': {
...     'maximum_memory_per_participant': 1,
...     'max_cores_per_participant': 1,
...     'num_ants_threads': 1, 'num_participants_at_once': 1
... }, 'Amazon-AWS': {
...     'aws_output_bucket_credentials': None, 's3_encryption': True}})
True
>>> tse_base = {'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/CC400.nii.gz': 'Avg',
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Avg'
... }, 'realignment': 'ROI_to_func'}}
>>> str(update_nested_dict(tse_base, {})) == str({
...     'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...         '/cpac_templates/CC400.nii.gz': 'Avg',
...         '/cpac_templates/aal_mask_pad.nii.gz': 'Avg'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'tse_roi_paths': {'/cpac_templates/rois_3mm.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/rois_3mm.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'roi_paths_fully_specified': False,
...     'tse_roi_paths': {'/cpac_templates/rois_3mm.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/CC400.nii.gz': 'Avg',
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Avg',
...     '/cpac_templates/rois_3mm.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'roi_paths_fully_specified': False,
...     'tse_roi_paths': {'/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True,
...     'tse_roi_paths': {
...         '/cpac_templates/CC400.nii.gz': 'Avg',
...         '/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True
>>> str(update_nested_dict(tse_base, {'timeseries_extraction': {
...     'tse_roi_paths': {'/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'}
... }})) == str({'timeseries_extraction': {'run': True, 'tse_roi_paths': {
...     '/cpac_templates/aal_mask_pad.nii.gz': 'Voxel'
... }, 'realignment': 'ROI_to_func'}})
True

CPAC.utils.utils.update_pipeline_values_1_8(d_old)

Function to update pipeline config values that changed from C-PAC 1.7 to 1.8.

Parameters:

d_old (dict) –

Returns:

d – updated

Return type:

dict

Examples

>>> update_pipeline_values_1_8({'segmentation': {'tissue_segmentation': {
...     'using': ['FSL-FAST Thresholding', 'Customized Thresholding']}}})
{'segmentation': {'tissue_segmentation': {'using': ['FSL-FAST'], 'FSL-FAST': {'thresholding': {'use': 'Custom'}}}}}
>>> update_pipeline_values_1_8({'segmentation': {'tissue_segmentation': {
...     'using': ['FSL-FAST Thresholding']}}})
{'segmentation': {'tissue_segmentation': {'using': ['FSL-FAST'], 'FSL-FAST': {'thresholding': {'use': 'Auto'}}}}}

CPAC.utils.utils.update_values_from_list(d_old, last_exception=None)

Function to convert 1-length lists of an expected type to single items of that type, or to convert singletons of an expected list of a type into lists thereof. Also handles some type conversions against the schema.

Parameters:

• d_old (dict) –

• last_exception (Exception or None) – if the same exception recurs, raise it.

Returns:

d – updated

Return type:

dict

Examples

>>> update_values_from_list({'pipeline_setup': {
...     'pipeline_name': ['one_string']}})
{'pipeline_setup': {'pipeline_name': 'one_string'}}
>>> update_values_from_list({'nuisance_corrections': {
...     '1-ICA-AROMA': {'run': [False]}}})
{'nuisance_corrections': {'1-ICA-AROMA': {'run': [False]}}}

CPAC.utils.utils.write_output_json(json_data, filename, indent=3, basedir=None)

CPAC.utils.utils.write_to_log(workflow, log_dir, index, inputs, scan_id)

Method to write into log file the status of the workflow run.

CPAC.utils.utils.zscore(data, axis)

Versioning

Gather and report versions