Pre-processing

Convert CI data to NWB file format

Data to NWB

cicada.preprocessing.cicada_data_to_nwb.convert_data_to_nwb(data_to_convert_dir, default_convert_to_nwb_yml_file, nwb_files_dir)

Convert all default_config_data_for_conversion located in dir_path and put it in NWB format then create the file. Use the yaml file contains in dir_path to convert the default_config_data_for_conversion. A yaml file with in its name session_data and one with subject_data must be in directory. Otherwise nothing will happend. A yaml file with abf in its name will need to be present to convert the abf default_config_data_for_conversion.

Parameters

• data_to_convert_dir (str) – Absolute path to the directory containing all data

• default_convert_to_nwb_yml_file (str) – Absolute path to the default YAML file to convert an NWB file

• nwb_files_dir (str) – Absolute path to the directory where to save the nwb file created

cicada.preprocessing.cicada_data_to_nwb.create_convert_class(class_name, config_dict, converter_dict, nwb_file, yaml_path, files, dir_path)

Parameters

• class_name –

• config_dict –

• converter_dict –

• nwb_file –

• yaml_path –

• files –

• dir_path –

Returns:

cicada.preprocessing.cicada_data_to_nwb.create_nwb_file(subject_data_yaml_file, session_data_yaml_file)

Create an NWB file object using all metadata containing in YAML file

Parameters

• subject_data_yaml_file (str) – Absolute path to YAML file containing the subject metadata

• session_data_yaml_file (str) – Absolute path to YAML file containing the session metadata

cicada.preprocessing.cicada_data_to_nwb.filter_list_according_to_keywords(list_to_filter, keywords, keywords_to_exclude)

Conditional loop to remove all files or directories not containing the keywords # or containing excluded keywords. Inplace list modification

Parameters

• list_to_filter (list) – List containing all files/directories to be filtered (full_path)

• keywords (str) – If the list doesn’t contain the keyword, remove it from list

• keywords_to_exclude (str) – If the list contains the keyword, remove it from list

Exemples:

>>> print(filter_list_of_files(["file1.py", "file2.c", "file2.h"],"2","h"))
["file2.c"]

cicada.preprocessing.cicada_data_to_nwb.filter_list_of_files(dir_path, files, extensions, directory=None)

Take a list of file names and either no extensions (empty list or None) and remove the directory that starts by “.” or a list of extension and remove the files that are not with this extension. It returns a new list with full paths

Parameters

• dir_path (str) – path in which the files ares

• files (list) – List of files to be filtered

• extensions (str) – File extension to use as a filter

• directory (in which looking for files with a given extensions or return files in this) – directory keyword, allows to identify directories

• directory –

Exemples:

>>> print(filter_list_of_files(["file1.py", "file2.c", "file3.h"],"py"))
["file1.py"]

Run preprocessing

NWB file class

class cicada.preprocessing.convert_to_nwb.ConvertToNWB(nwb_file)

NWB file object class

convert(**kwargs)

Convert the data and add to the nwb_file

Parameters

**kwargs – arbitrary arguments

Suite 2P ROIs

2D series

class cicada.preprocessing.convert_processed_2d_series_to_nwb.ConvertProcessed2dSeriesToNWB(nwb_file)

Class to convert 2D series to NWB

convert(**kwargs)

Convert the data and add to the nwb_file

Parameters

**kwargs – arbitrary arguments

static load_rasterplot_in_memory(rasterplot_file_name, matlab_string, frames_to_add=None)

Get 2D series data from file

Parameters

• rasterplot_file_name (str) – Absolute path to file containing 2D series

• matlab_string (str) – Key to the data from matlab and npz files

• frames_to_add (dict) – Key is the frame where you add blank frames and value is the number of frames to add

• None. (Default is) –

Returns

Raster data as a 2d array

Return type

raster (np.array)

Calcium imaging movie

class cicada.preprocessing.convert_ci_movie_to_nwb.ConvertCiMovieToNWB(nwb_file)

Class to convert Calcium Imaging movies to NWB

convert(**kwargs)

Convert the data and add to the nwb_file

Parameters

**kwargs – arbitrary arguments

ABF

class cicada.preprocessing.convert_abf_to_nwb.ConvertAbfToNWB(nwb_file)

Class to convert ABF data to NWB

convert(**kwargs)

The goal of this function is to extract from an Axon Binary Format (ABF) file its content and make it accessible through the NWB file. The content can be: LFP signal, piezzo signal, speed of the animal on the treadmill. All, None or a few of these informations could be available. One information always present is the timestamps, at the abf sampling_rate, of the frames acquired by the microscope to create the calcium imaging movie. Such movie could be the concatenation of a few movies, such is the case if the movie need to be saved every x frames for memory issue for ex. If the movie is the concatenation of many, then there is an option to choose to extract the information as if 2 frames concatenate are contiguous in times (such as then LFP signal or piezzo would be match movie), or to add interval_times indicating at which time the recording is on pause and at which time it’s starting again. The time interval containing this information is named “ci_recording_on_pause” and you can get it doing: if ‘ci_recording_on_pause’ in nwb_file.intervals: pause_intervals = nwb_file.intervals[‘ci_recording_on_pause’]

Parameters

• **kwargs (dict) – kwargs is a dictionary, potentials keys and values types are:

• abf_yaml_file_name – mandatory parameter. The value is a string representing the path

• abf (and file_name of the yaml file associated to this abf file. In the) –

• frames_channel – mandatory parameter. The value is an int representing the channel

• data. (of the abf in which is the frames timestamps) –

• abf_file_name – mandatory parameter. The value is a string representing the path

• file. (and file_name of the abf) –

determine_ci_frames_indices()

Using the frames data channel, estimate the timestamps of each frame of the calcium imaging movie. If there are breaks between each recording (the movie being a concatenation of different movies), then there is an option to either skip those non registered frames that will be skept in all other data (lfp, piezzo, …) or to determine how many frames to add in the movie and where so it matches the other data recording in the abf file

double_sign_transi(chan1, chan2, thr=1)

Extract signed ticks from two signals.

Parameters

• chan1 (npt.NDArray[np.float64]) – First analog signal

• chan2 (npt.NDArray[np.float64]) – Second analog signal

• thr (float) – Value for the transition detections

Returns

The signed transitions

Return type

npt.NDArray[np.int64]

position_ticks(chan1, chan2, thr=1)

Computes the position transitions.

Parameters

• chan1 (npt.NDArray[np.float64]) – First position channel

• chan2 (npt.NDArray[np.float64] | None, optional) – Second channel if present

• thr (float) – Crossing threshold value

Returns

The transition array

Return type

npt.NDArray[np.int64]

Utils

class cicada.preprocessing.utils.ComparableItem(value)

Make it possible to sort a list of items of different types, such as int and string

cicada.preprocessing.utils.class_name_to_module_name(class_name)

Transform the string representing a class_name, by removing the upper case letters, and inserting before them an underscore if 2 upper case letters don’t follow. Underscore are also inserted before numbers ex: ConvertAbfToNWB -> convert_abf_to_nwb :param class_name: string :return:

cicada.preprocessing.utils.flatten(list)

Flatten a nested list no matter the nesting level

Parameters

list (list) – List to flatten

Returns

List without nest

Examples

>>> flatten([1,2,[[3,4],5],[7]])
[1,2,3,4,5,7]

cicada.preprocessing.utils.get_continous_time_periods(binary_array)

take a binary array and return a list of tuples representing the first and last position(included) of continuous positive period :param binary_array: :return:

cicada.preprocessing.utils.load_tiff_movie_in_memory(tif_movie_file_name, frames_to_add=None)

Load tiff movie from filename using Scan Image Tiff

Parameters

tif_movie_file_name (str) – Absolute path to tiff movie

Returns

Tiff movie as 3D-array

Return type

tiff_movie (array)

cicada.preprocessing.utils.load_tiff_movie_in_memory_using_pil(tif_movie_file_name, frames_to_add=None)

Load tiff movie from filename using PIL library

Parameters

• tif_movie_file_name (str) – Absolute path to tiff movie

• frames_to_add – dict with key an int representing the frame index after which add frames. the value is the number of frames to add (integer)

Returns

Tiff movie as 3D-array

Return type

tiff_movie (array)

cicada.preprocessing.utils.merging_time_periods(time_periods, min_time_between_periods)

Take a list of pair of values representing intervals (periods) and a merging thresholdd represented by min_time_between_periods. If the time between 2 periods are under this threshold, then we merge those periods. It returns a new list of periods. :param time_periods: list of list of 2 integers or floats. The second value represent the end of the period, the value being included in the period. :param min_time_between_periods: a float or integer value :return: a list of pair of list.

cicada.preprocessing.utils.module_name_to_class_name(module_name)

Transform the string representing a module_name, by removing underscores, , and transforming as upper cases the following letter. ex: convert_abf_to_nwb -> ConvertAbfToNwb :param module_name: string :return:

cicada.preprocessing.utils.sort_by_param(nwb_path_list, param_list)

Sort NWB files depending on a list of parameters

Parameters

• nwb_path_list (list) – List of absolute path to NWB files

• param_list (list) – List of parameters to sort by

Returns

List of NWB files sorted

Return type

nwb_sorted_list (list)

cicada.preprocessing.utils.update_frames_to_add(frames_to_add, nwb_file, ci_sampling_rate)

Update frames_to_add (dict), based on pause_intervals and ci_frames_time_series :param frames_to_add: dict, with key an int representing the frame index after which add frames. :param the value is the number of frames to add: :type the value is the number of frames to add: integer :param nwb_file: nwb file, will get nwb_file.intervals[‘ci_recording_on_pause’] and :param nwb_file.get_acquisition: :type nwb_file.get_acquisition: “ci_frames”

Returns:

Appendix