This document describes Pypet, a Python parameter exploration toolbox. Pypet allows for easy exploration of parameter spaces and storage of simulation results and parameters. It revolves around a trajectory container, which uses a tree data structure to manage parameters and results in a natural naming scheme. Pypet supports a variety of data formats and storage via HDF5. It provides tools for disentangling simulations from I/O, logging, version control integration, and parallelization. Pypet is open source, well tested, and documented.

1. pypet
 PyData 2016
 Robert Meyer
 Neural Information Processing, TU-Berlin

2. Motivation
 The cycle of simulations and proto-typing
Idea
Coding
Simulations

3. Motivation
 The cycle of simulations (ground truth)
Idea
Spaghetti
code
Parameter
exploration
bash scripts
Hacky I/O
functionality
to store data
Data Analysis
that only works with
my current I/O

4. Motivation
 The cycle of simulations (ground truth)
Uhm, maybe I should
try something else
Spaghetti
code
Parameter
exploration
bash scripts
Hacky I/O
functionality
to store data
Data Analysis
that only works with
my current I/O

5. Motivation
 The cycle of simulations (ground truth)
Spaghetti
code
Parameter
exploration
bash scripts
Hacky I/O
functionality
to store data
Data Analysis
that only works with
my current I/OLet's repeat an old simulation
Uhm, maybe I should
try something else

6. Motivation
 The cycle of simulations (ground truth)
Spaghetti
code
Parameter
exploration
bash scripts
Hacky I/O
functionality
to store data
Data Analysis
that only works with
my current I/OLet's repeat an old simulation
Uhm, maybe I should
try something else

7. Motivation
 The cycle of simulations (ground truth)
Uhm, maybe I should
try something else
Spaghetti
code
Parameter
exploration
bash scripts
Hacky I/O
functionality
to store data
Data Analysis
that only works with
my current I/O
???

8. Motivation
 The cycle of simulations (ground truth)
Uhm, maybe I should
try something else
Spaghetti
code
Parameter
exploration
bash scripts
Hacky I/O
functionality
to store data
Data Analysis
that only works with
my current I/O
pypet

9. pypet
 python parameter exploration toolbox
 Suitable for any numerical simulation in python

10. pypet
 python parameter exploration toolbox
 Suitable for any numerical simulation in python
 Main objectives
 Allow easy exploration of the parameter space
 Easy storage and retrieval of computed results
and the underlying set of parameters

11. pypet
 Evolves around the trajectory container
 Tree Data Structure to manage parameters
and results
 Group your parameters according to meaningful categories
 >>> traj.f_add_parameter('traffic.ncars', 100, comment = 'Number of vehicles')
>>> traj.f_add_parameter('traffic.cyclists', 42, comment='Number of cyclists')
>>> traj.f_add_parameter('simulation.dt', 0.01, comment='Timestep')

12. pypet
 Evolves around the trajectory container
 Tree Data Structure to manage parameters
and results
 Group your parameters according to meaningful categories
 >>> traj.f_add_parameter('traffic.ncars', 100, comment = 'Number of vehicles')
>>> traj.f_add_parameter('traffic.cyclists', 42, comment='Number of cyclists')
>>> traj.f_add_parameter('simulation.dt', 0.01, comment='Timestep')
 Access your data with Natural Naming
 >>> traj.traffic.ncars
100
 Or cut short through the tree
 >>> traj.cyclists
42

13. pypet
 Evolves around the trajectory container
 Handles a variety of data formats
 Python Natives (bool, int, float, complex, str)
 Python Containers (dict, list, tuple)
 NumPy Arrays and Matrices, Scipy Sparse Matrices
 Pandas DataFrames
 BRIAN(2)_ Quantities and Monitors
 (Everything you can pickle)
_BRIAN(2): http://briansimulator.org/

14. pypet
 Evolves around the trajectory container
 Handles a variety of data formats
 Python Natives (bool, int, float, complex, str)
 Python Containers (dict, list, tuple)
 NumPy Arrays and Matrices, Scipy Sparse Matrices
 Pandas DataFrames
 BRIAN(2)_ Quantities and Monitors
 (Everything you can pickle)
 Your work with something else?
 Easily extendible for other parameters and results
_BRIAN(2): http://briansimulator.org/

15. pypet
 Evolves around the trajectory container
 Tree Data Structure to manage parameters
and results
 Allows arbitrary exploration of the parameter
space
 Beyond grids (Cartesian product space) and
straight lines

16. pypet
 Evolves around the trajectory container
 Tree Data Structure to manage parameters
and results
 Allows arbitrary exploration of the parameter
space beyond grids (cartesian product lines
ncars
cyclists

17. pypet
 Evolves around the trajectory container
 Tree Data Structure to manage parameters
and results
 Allows arbitrary exploration of the parameter
space beyond grids (cartesian pr lines
ncars
cyclists

18. pypet
 Evolves around the trajectory container
 Tree Data Structure to manage parameters
and results
 Allows arbitrary exploration of the parameter
space beyond grids (cartesian produ lines
cyclists
ncars

19. pypet
 HDF5_ (Hierarchical Data Format)
 Portable file format for managing and storing data
 Can handle large amounts of data
 Human readable data
 You can browse within your data as in a regular file
system
 Your data is automatically stored using the PyTables_
interface
_HDF5: http://www.hdfgroup.org/HDF5
_PyTables: http://www.pytables.org/

20. pypet
 Standing on the shoulder of giants
 HDF5_ (Hierarchical Data Format)
 Portable file format for managing and storing data
 You can browse within your data as in a regular file system

21. pypet
 Provides and environment to disentangle
your actual scientific simulation from I/O
management and execution
 Manages logging

22. pypet
 Provides and environment to disentangle
your actual scientific simulation from I/O
management and execution
 Manages logging
 Automatic integration with git version control

23. pypet
 Provides and environment to disentangle
your actual scientific simulation from I/O
management and execution
 Manages logging
 Automatic integration with git version control
 Allows adaptive exploration (optimization with DEAP_)
_DEAP: http://deap.readthedocs.org

24. pypet
 Provides and environment to disentangle
your actual scientific simulation from I/O
management and execution
 Manages logging
 Automatic integration with git version control
 Allows adaptive exploration (optimization with DEAP_)
 Allows parallel simulations and multiprocessing
(on multiple serves or clusters like SGE/qsub using SCOOP_)
_DEAP: http://deap.readthedocs.org _SCOOP: http://scoop.readthedocs.org

25. pypet
 Live Demo (although you shouldn't do any)!
>>> from pypet import Environment

26.  Even more important!
 Open source: project available on github_
 Fully documented on readthedocs_
pypet
_github: https://github.com/SmokinCaterpillar/pypet
_readthedocs: https://pypet.readthedocs.org/en/latest/

27.  Even more important!
 Open source: project available on github_
 Fully documented on readthedocs_
 Thoroughly tested
 More than 1000 tests, 91% code coverage
 Continuous integration testing with Travis_ (Linux) and
Appveyor_ (Windows)
pypet
_github: https://github.com/SmokinCaterpillar/pypet
_readthedocs: https://pypet.readthedocs.io
_Travis: https://travis-ci.org/SmokinCaterpillar/pypet
_Appveyor:https://ci.appveyor.com/project/SmokinCaterpillar/pypet

28. pypet
 Where to get pypet (for python 2.6 till 3.5) ?
 https://pypi.python.org/pypi/pypet
 Or >>> pip install pypet
 Or https://github.com/SmokinCaterpillar/pypet
 Documentation?!
 https://pypet.readthedocs.io
 Tutorial
 More than 20 comprehensive examples

29. Bonus Track
 I am looking for a job!
 Soon to be PhD with lack of motivation/self-delusion
to stay in academia

31. Bonus Track
 I am looking for a job!
 Soon to be PhD with lack of motivation/self-delusion
to stay in academia
 Will do (Python) Coding, Data Science, and
Machine Learning for food!

32. Thank You!
from xkcd.com
And thanks to the BCCN Berlin,
my supervisor Prof. Klaus Obermayer,
Robert Pröpper, Phillip Meier,
Owen Mackwood, and Mehmet Nevvaf Timur