Pre-processingΒΆ

In pre-processing, we include everything that is done to the data between acquisition and analysis (although the dividing line between acquisition and analysis is not always clear). This can include assuring the quality of the data through checks and inspections, data cleaning and data selection, e.g. exclusion of cells or periods of time where there was some problem with the recording (50 Hz noise, etc.).

The most important best practice is this:

Important

never change a raw data file (e.g. to remove artifacts, bad recordings), instead copy the file and edit the copy, recording how you got from one to the other. To enforce this, make raw data read-only.

Ideally the transformation should be scripted/automated, i.e. set objective standards for when to exclude data from further analysis, and write a program to implement the application of these standards. If you can’t easily automate the process, keep detailed notes of what you did and why.

Consider maintaining separate directory trees for the raw and cleaned-up data files.