Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Which transient, when?
A utility function for transient follow-up scheduling
Tim Staley
(4 Pi Sky group)
Southampton Wedne...
Automate all the things Forecasting transients Decision theory Future work
Outline
Automate all the things
Forecasting tra...
Automate all the things Forecasting transients Decision theory Future work
LSST predicted transient rates
(LSST science bo...
Automate all the things Forecasting transients Decision theory Future work
Transient rates today
Notices / events in a 30 ...
Automate all the things Forecasting transients Decision theory Future work
Robotic follow-up facilities
Slow rise of autom...
Automate all the things Forecasting transients Decision theory Future work
The (wider) problem
How do we begin to ‘close t...
Automate all the things Forecasting transients Decision theory Future work
The (wider) problem
How do we begin to ‘close t...
Automate all the things Forecasting transients Decision theory Future work
The (wider) problem
How do we begin to ‘close t...
Automate all the things Forecasting transients Decision theory Future work
Diversion on DG-CVn superflare
Fender 2014, http...
Automate all the things Forecasting transients Decision theory Future work
Missing pieces
Transient
discovery
Observation
...
Automate all the things Forecasting transients Decision theory Future work
We found a transient!
What now?
Two implicit go...
Automate all the things Forecasting transients Decision theory Future work
We found a transient!
What now?
Two implicit go...
Automate all the things Forecasting transients Decision theory Future work
Outline
Automate all the things
Forecasting tra...
Automate all the things Forecasting transients Decision theory Future work
Working with tiny data
We’ve found a transient....
Automate all the things Forecasting transients Decision theory Future work
The set-up
How do we predict the possible futur...
Automate all the things Forecasting transients Decision theory Future work
Assumption: Parametric models
Deterministic, fin...
Automate all the things Forecasting transients Decision theory Future work
Result: Line of best fit
(Maximum likelihood)
−1...
Automate all the things Forecasting transients Decision theory Future work
Assumption: Multivar-Normal priors
Known priors...
Automate all the things Forecasting transients Decision theory Future work
Construct: Model lightcurve ensembles
−20 0 20 ...
Automate all the things Forecasting transients Decision theory Future work
Result: MAP fit
−10 −5 0 5 10 15 20 25 30
Time
0...
Automate all the things Forecasting transients Decision theory Future work
But...
−10 −5 0 5 10 15 20 25 30
Time
0
2
4
6
8...
Automate all the things Forecasting transients Decision theory Future work
Constrained parameter distributions
Take our tw...
Automate all the things Forecasting transients Decision theory Future work
Constrained lightcurve ensembles
−30 −20 −10 0 ...
Automate all the things Forecasting transients Decision theory Future work
Constrained lightcurve ensembles
−30 −20 −10 0 ...
Automate all the things Forecasting transients Decision theory Future work
Constrained lightcurve ensembles
−30 −20 −10 0 ...
Automate all the things Forecasting transients Decision theory Future work
Constrained lightcurve ensembles
−30 −20 −10 0 ...
Automate all the things Forecasting transients Decision theory Future work
Constrained lightcurve ensembles
−30 −20 −10 0 ...
Automate all the things Forecasting transients Decision theory Future work
Constrained lightcurve ensembles
−30 −20 −10 0 ...
Automate all the things Forecasting transients Decision theory Future work
Constrained lightcurve ensembles
−30 −20 −10 0 ...
Automate all the things Forecasting transients Decision theory Future work
Comparing model ensembles
−30 −20 −10 0 10 20 3...
Automate all the things Forecasting transients Decision theory Future work
Comparing model ensembles
−30 −20 −10 0 10 20 3...
Automate all the things Forecasting transients Decision theory Future work
Comparing model ensembles
−30 −20 −10 0 10 20 3...
Automate all the things Forecasting transients Decision theory Future work
Comparing model ensembles
−30 −20 −10 0 10 20 3...
Automate all the things Forecasting transients Decision theory Future work
Comparing model ensembles
−30 −20 −10 0 10 20 3...
Automate all the things Forecasting transients Decision theory Future work
Outline
Automate all the things
Forecasting tra...
Automate all the things Forecasting transients Decision theory Future work
The crux of the problem
(Couldn’t get past the
...
Automate all the things Forecasting transients Decision theory Future work
The crux of the problem
(Couldn’t get past the
...
Automate all the things Forecasting transients Decision theory Future work
−6 −4 −2 0 2 4 6
Epoch
0.0
0.2
0.4
0.6
0.8
1.0
...
Automate all the things Forecasting transients Decision theory Future work
−6 −4 −2 0 2 4 6
Epoch
0.0
0.2
0.4
0.6
0.8
1.0
...
Automate all the things Forecasting transients Decision theory Future work
−6 −4 −2 0 2 4 6
Epoch
−0.5
0.0
0.5
1.0
1.5
Rel...
Automate all the things Forecasting transients Decision theory Future work
−6 −4 −2 0 2 4 6
Epoch
−0.5
0.0
0.5
1.0
1.5
Rel...
Automate all the things Forecasting transients Decision theory Future work
0 1
−0.5
0.0
0.5
1.0
1.5
Relativeflux
T=-5.0
0 ...
Automate all the things Forecasting transients Decision theory Future work
PDF value
−0.5
0.0
0.5
1.0
1.5
Relativeflux
T=-...
Automate all the things Forecasting transients Decision theory Future work
The problem with information content
PDF value
...
Automate all the things Forecasting transients Decision theory Future work
Introducing: confusion matrices
Common or garde...
Automate all the things Forecasting transients Decision theory Future work
Confusion matrices
Label True class
A B C
ˆA P(...
Automate all the things Forecasting transients Decision theory Future work
Probabilistic confusion matrices
Example
0.0 0....
Automate all the things Forecasting transients Decision theory Future work
Probabilistic confusion matrices
Example
0.0 0....
Automate all the things Forecasting transients Decision theory Future work
PDF value
−0.5
0.0
0.5
1.0
1.5
Relativeflux
T=-...
Automate all the things Forecasting transients Decision theory Future work
PDF value
−0.5
0.0
0.5
1.0
1.5
Relativeflux
T=-...
Automate all the things Forecasting transients Decision theory Future work
As applied to the previous example . . .
−30 −2...
Automate all the things Forecasting transients Decision theory Future work
Summary
Data + models + Bayesian analysis =⇒
En...
Automate all the things Forecasting transients Decision theory Future work
Outline
Automate all the things
Forecasting tra...
Automate all the things Forecasting transients Decision theory Future work
Scheduler, simulation, testing
Still need to im...
Automate all the things Forecasting transients Decision theory Future work
Optimization / Automated Planning
Optimizing fo...
Automate all the things Forecasting transients Decision theory Future work
Optimization / Automated Planning
Multi-armed b...
Automate all the things Forecasting transients Decision theory Future work
Model refinement (and basic validity)
Multivaria...
Automate all the things Forecasting transients Decision theory Future work
Culture
Pretty sure we can make this work. . . ...
Automate all the things Forecasting transients Decision theory Future work
Code packages used
astropy.modeling — for the m...
Automate all the things Forecasting transients Decision theory Future work
Fin
Plenty more to do . . . watch this space.
Upcoming SlideShare
Loading in …5
×

Which transient when? - A utility function for transient follow-up scheduling

227 views

Published on

Next-generation astronomical facilities such as the LSST and the SKA will be game-changers, allowing us to observe the entire southern sky and track changing sources in near real-time. Keeping up with their alert-streams represents a significant challenge - how do we make the most of our limited telescope resources to follow up 100000 sources per night?

The biggest problem here is classification - we want to find the really interesting transients and spend our time watching those. However, classification based on the initial survey data can only get you so far - we'll need to use robotic follow-up telescopes for rapid-response observations, to give us more information on the most promising targets. To get the most science done, we need to be smart about scheduling that follow-up.

We're exploring use of active learning algorithms (AKA Bayesian Decision Theory) to solve this problem, building a framework that allows for iterative refinement of a probabilistic classification state. Because there are no algorithms that fit this problem 'out-of-the-box', we've built our own analysis framework using the emcee and PyMultiNest packages to power the underlying Bayesian inference. I'll give an overview of how our proposed system fits into the wider context of an automated astronomy ecosystem, then give a gentle introduction to Bayesian Decision Theory and how it can be applied to this problem.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Which transient when? - A utility function for transient follow-up scheduling

  1. 1. Which transient, when? A utility function for transient follow-up scheduling Tim Staley (4 Pi Sky group) Southampton Wednesday Seminar, May 2015 WWW: 4pisky.org , timstaley.co.uk/talks
  2. 2. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
  3. 3. Automate all the things Forecasting transients Decision theory Future work LSST predicted transient rates (LSST science book, 2009) Orphan GRBS: ∼ 1000 per year. “The reader should be cautioned that many of these rates are very rough.” Guesstimates: 104 – 106 alerts per night (depending on your definition).
  4. 4. Automate all the things Forecasting transients Decision theory Future work Transient rates today Notices / events in a 30 day period around April 2015: GCN: 106 (perhaps 10 new events) ATEL: 146 (probably similar ratio of new / follow-up) GAIA: 17 (public) PTF: 299 (internal) CRTS: 81(CSS) + 105 (MLS) (automatic detections) Just counting initial events: ∼500 / month, or ∼17 /day
  5. 5. Automate all the things Forecasting transients Decision theory Future work Robotic follow-up facilities Slow rise of automated sites / networks, e.g. LCOGT: 2 × 2m + 9 × 1m telescopes, over six sites. (http://lcogt.net/observatory)
  6. 6. Automate all the things Forecasting transients Decision theory Future work The (wider) problem How do we begin to ‘close the loop’ of observe =⇒ analyse =⇒ observe?
  7. 7. Automate all the things Forecasting transients Decision theory Future work The (wider) problem How do we begin to ‘close the loop’ of observe =⇒ analyse =⇒ observe? Transient discovery Observation prioritization Classification estimates Interesting? Yes Schedule optimization External alerts Survey data Telescope agents Follow-up data
  8. 8. Automate all the things Forecasting transients Decision theory Future work The (wider) problem How do we begin to ‘close the loop’ of observe =⇒ analyse =⇒ observe? Transient discovery Observation prioritization Classification estimates Interesting? Yes Schedule optimization External alerts Survey data Telescope agents Follow-up data Science!
  9. 9. Automate all the things Forecasting transients Decision theory Future work Diversion on DG-CVn superflare Fender 2014, http://adsabs.harvard.edu/abs/2014arXiv1410.1545F Osten et al (in prep)
  10. 10. Automate all the things Forecasting transients Decision theory Future work Missing pieces Transient discovery Observation prioritization Classification estimates Interesting? Yes Schedule optimization External alerts Survey data Telescope agents Follow-up data We’ve found a potentially interesting new transient. Looks like it could be one of class A, B, or C. What now?
  11. 11. Automate all the things Forecasting transients Decision theory Future work We found a transient! What now? Two implicit goals of follow-up observation: Improving classification (Let’s see if it’s really class A.) Further observation (Tell me more! / Boring!)
  12. 12. Automate all the things Forecasting transients Decision theory Future work We found a transient! What now? Two implicit goals of follow-up observation: Improving classification (Let’s see if it’s really class A.) Further observation (Tell me more! / Boring!) . . . I’ll mainly be talking about the former.
  13. 13. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
  14. 14. Automate all the things Forecasting transients Decision theory Future work Working with tiny data We’ve found a transient. But, very few datapoints: −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux Detection threshold Data
  15. 15. Automate all the things Forecasting transients Decision theory Future work The set-up How do we predict the possible futures for a given transient? For now, make some simplifying assumptions: Parametric lightcurve models for each class of transient. Each transient class has a known prior distribution over the morphological parameters, and this is multivariate Normal.
  16. 16. Automate all the things Forecasting transients Decision theory Future work Assumption: Parametric models Deterministic, finite number of parameters e.g. y = ƒ(t, t0, , τrse, τdecy) −40 −20 0 20 40 60 80 100 Time 0 2 4 6 8 10 Flux
  17. 17. Automate all the things Forecasting transients Decision theory Future work Result: Line of best fit (Maximum likelihood) −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux ML fit Detection threshold Data
  18. 18. Automate all the things Forecasting transients Decision theory Future work Assumption: Multivar-Normal priors Known priors, normally distributed, covariant 234 rise_tau 8 16 24 32 a 6121824 decay_tau 2 3 4 rise_tau 6 12 18 24 decay_tau
  19. 19. Automate all the things Forecasting transients Decision theory Future work Construct: Model lightcurve ensembles −20 0 20 40 60 80 Time 0 2 4 6 8 10 12 14 16 18 Flux
  20. 20. Automate all the things Forecasting transients Decision theory Future work Result: MAP fit −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux ML fit MAP fit Detection threshold Data
  21. 21. Automate all the things Forecasting transients Decision theory Future work But... −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux ML fit MAP fit True Detection threshold Data
  22. 22. Automate all the things Forecasting transients Decision theory Future work Constrained parameter distributions Take our two datapoints, run some MCMC fitting... 3.0 3.6 4.2 rise_tau 12151821 decay_tau 10 15 20 25 30 a 04812 t0 3.0 3.6 4.2 rise_tau 12 15 18 21 decay_tau 0 4 8 12 t0
  23. 23. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 14 16 18 Flux True Observations Comparison, 2 datapoints
  24. 24. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 Flux True Forecast epoch Observations 0.00 0.05 0.10 0.15 Prob. 0 5 10 15 Comparison, 2 datapoints
  25. 25. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 14 16 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 2 4 6 8 10 12 14 16 Comparison, 1 datapoints
  26. 26. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 Flux True Forecast epoch Observations 0.00 0.05 0.10 0.15 Prob. 0 5 10 15 Comparison, 2 datapoints
  27. 27. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 2 4 6 8 10 12 Comparison, 3 datapoints
  28. 28. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 2 4 6 8 10 12 Comparison, 4 datapoints
  29. 29. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 Flux True Forecast epoch Observations 0.00 0.25 0.50 0.75 Prob. 0 2 4 6 8 10 12 Comparison, 6 datapoints
  30. 30. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 14 16 18 Flux Prior ensemble, Type 1
  31. 31. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux Prior ensemble, Type 2
  32. 32. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 40 Flux Comparison, 0 datapoints
  33. 33. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux True Forecast epoch Observations 0.00 0.05 0.10 0.15 0.20 Prob. 0 5 10 15 20 25 30 35 Comparison, 2 datapoints
  34. 34. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 5 10 15 20 25 30 35 Comparison, 2 datapoints
  35. 35. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
  36. 36. Automate all the things Forecasting transients Decision theory Future work The crux of the problem (Couldn’t get past the crux)
  37. 37. Automate all the things Forecasting transients Decision theory Future work The crux of the problem (Couldn’t get past the crux) Computers are good at optimisation. e.g., picking out a good observing schedule. But need a way to assign value.
  38. 38. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch 0.0 0.2 0.4 0.6 0.8 1.0 Relativeflux stable logistic null Intrinsic lightcurves
  39. 39. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch 0.0 0.2 0.4 0.6 0.8 1.0 Relativeflux stable logistic null Intrinsic LC's with noise estimates
  40. 40. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch −0.5 0.0 0.5 1.0 1.5 Relativeflux Sampling with noise
  41. 41. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch −0.5 0.0 0.5 1.0 1.5 Relativeflux Sampling with noise
  42. 42. Automate all the things Forecasting transients Decision theory Future work 0 1 −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 0 1 T=-4.0 0 1 PDF value T=-3.0 0 1 T=-2.0 0 1 T=-1.0 0 1 T=0.0 0 1 T=1.0 0 1 T=2.0 stable logistic null Class PDF at each epoch
  43. 43. Automate all the things Forecasting transients Decision theory Future work PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch −0.55 −0.50 −0.45 −0.40 −0.35 −0.30 −0.25 −0.20 FoM Information content Evaluating each epoch
  44. 44. Automate all the things Forecasting transients Decision theory Future work The problem with information content PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch −0.55 −0.50 −0.45 −0.40 −0.35 −0.30 −0.25 −0.20 FoM Information content Evaluating each epoch Information content weights all classes equally. What if we’re more interested in identifying one particular class?
  45. 45. Automate all the things Forecasting transients Decision theory Future work Introducing: confusion matrices Common or garden empirical confusion matrix: (Automatic classification of time-variable X-ray sources; K. Lo et al, 2014)
  46. 46. Automate all the things Forecasting transients Decision theory Future work Confusion matrices Label True class A B C ˆA P( ˆA | A ) P( ˆA | B ) P( ˆA | C ) ˆB P( ˆB | A ) P( ˆB | B ) P( ˆC | C ) ˆC P( ˆC | A ) P( ˆC | B ) P( ˆC | C )
  47. 47. Automate all the things Forecasting transients Decision theory Future work Probabilistic confusion matrices Example 0.0 0.5 1.0 1.5 2.0 2.5 PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-2 stable logistic null True class stable logistic null Label stable 0.549 0.449 0.002 logistic 0.449 0.541 0.010 null 0.002 0.010 0.988
  48. 48. Automate all the things Forecasting transients Decision theory Future work Probabilistic confusion matrices Example 0.0 0.5 1.0 1.5 2.0 2.5 PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-2 stable logistic null True class stable logistic null Label stable 0.549 0.449 0.002 logistic 0.449 0.541 0.010 null 0.002 0.010 0.988 Diagonal entries represent recall for each class.
  49. 49. Automate all the things Forecasting transients Decision theory Future work PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 FoM Information content (shifted) Total Recall Evaluating each epoch
  50. 50. Automate all the things Forecasting transients Decision theory Future work PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch 0.5 0.6 0.7 0.8 0.9 1.0 FoM Total Recall Null Recall Evaluating each epoch
  51. 51. Automate all the things Forecasting transients Decision theory Future work As applied to the previous example . . . −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux True Forecast epoch Observations 0.00 0.08 0.16 0.24 0.32 Prob. 0 5 10 15 20 25 30 35 −30 −20 −10 0 10 20 30 40 50 Time −0.70 −0.65 −0.60 −0.55 −0.50 −0.45 −0.40 ICscore Comparison, 2 datapoints
  52. 52. Automate all the things Forecasting transients Decision theory Future work Summary Data + models + Bayesian analysis =⇒ Ensemble forecasts Ensemble forecasts + utility function =⇒ Figure-of-merit for evaluating possible actions Using the figure-of-merit as basis for a decision = Applied Bayesian decision theory (AKA active machine learning)
  53. 53. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
  54. 54. Automate all the things Forecasting transients Decision theory Future work Scheduler, simulation, testing Still need to implement a (basic) scheduler using figure-of-merit as input. Then: simulate (easy, already have models!)
  55. 55. Automate all the things Forecasting transients Decision theory Future work Optimization / Automated Planning Optimizing for ASAP classification (Future-discounted weighting schemes?) Non-myopic (better-than-greedy) scheduling. Multi-armed bandit problem.
  56. 56. Automate all the things Forecasting transients Decision theory Future work Optimization / Automated Planning Multi-armed bandit problem Image credit: Wikipedia/Yamaguchi (CC BY-SA 3.0)
  57. 57. Automate all the things Forecasting transients Decision theory Future work Model refinement (and basic validity) Multivariate normal prior — valid? Sum of multiple multivariate normals? Non-parametric modelling (Gaussian processes)?
  58. 58. Automate all the things Forecasting transients Decision theory Future work Culture Pretty sure we can make this work. . . but there’s a culture / chicken-and-egg problem.
  59. 59. Automate all the things Forecasting transients Decision theory Future work Code packages used astropy.modeling — for the models interface. Numpy — efficient lightcurve-model calculations. pandas — (Python ANalysis of DAta Series) for handling time-series data. statsmodels — Kernel density estimates. emcee — for MCMC (Py)MultiNest — For model-evidence calculations. Seaborn — Neat plotting tools, aesthetically pleasing defaults for Matplotlib.
  60. 60. Automate all the things Forecasting transients Decision theory Future work Fin Plenty more to do . . . watch this space.

×