Best practices for data management in neurophysiology

Andrew Davison

Unité de Neurosciences, Information et Complexité (UNIC), Centre National de la Recherche Scientifique, 91198 Gif sur Yvette, France. http://andrewdavison.info

Notes for a tutorial given at the GDR Multielectrodes minischool. Available to download in PDF and Epub formats.

Version 0.1

October 13th 2014

Contents

• Why data management?
• The data life cycle
• Data acquisition
  • Where possible, store data in nonproprietary software formats
  • Keep backups on stable media, and in geographically separated locations
  • Have a clear organisation for your data files
  • Always maintain effective metadata
• Pre-processing
• Data analysis
  • Keep code and data separate
  • Use version control
  • Test your code
  • Split commonly used code off into functions/classes, and put these into libraries
  • Prioritize code robustness
  • Maintain a consistent, repeatable computing environment
  • Separate code from configuration
  • Share your code
• Version control
  • Basic ideas
  • Examples of version control systems
  • The importance of tracking projects, not individual files
  • Advantages of formal version control systems
  • Installing Mercurial
  • Creating a repository
  • Adding files to the repository
  • Committing changes
  • Viewing the history of changes
  • Seeing what’s changed
  • Switching between versions
  • Giving informative names to versions
  • Recap #1
  • Making backups
  • Working on multiple computers
  • Collaborating with others
  • Recap #2
  • A comparison of Git and Mercurial
  • A comparison of Subversion and Mercurial
  • Graphical tools
  • Web-based tools
• Verification (testing)
  • How confident are you that your code is doing what you think it’s doing?
  • Automated testing
  • Terminology
  • Test frameworks
  • Examples
  • Test coverage measurement
  • Test-driven development
• Provenance tracking
  • What is provenance?
  • The lab notebook
  • Software tools for provenance tracking
  • An introduction to Sumatra
  • References
• Conclusions and outlook

Licence

This document is licenced under a Creative Commons Attribution 3.0 licence. You are free to copy, adapt or reuse these notes, provided you give attribution to the authors, and include a link to this web page.

Sources

https://bitbucket.org/apdavison/reproducible_research_cns - feel free to fork the repository!