## Just for you: FREE 60-day trial to the world’s largest digital library.

The SlideShare family just got bigger. Enjoy access to millions of ebooks, audiobooks, magazines, and more from Scribd.

Cancel anytime.Free with a 14 day trial from Scribd

- 1. Which transient, when? A utility function for transient follow-up scheduling Tim Staley (4 Pi Sky group) Southampton Wednesday Seminar, May 2015 WWW: 4pisky.org , timstaley.co.uk/talks
- 2. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
- 3. Automate all the things Forecasting transients Decision theory Future work LSST predicted transient rates (LSST science book, 2009) Orphan GRBS: ∼ 1000 per year. “The reader should be cautioned that many of these rates are very rough.” Guesstimates: 104 – 106 alerts per night (depending on your deﬁnition).
- 4. Automate all the things Forecasting transients Decision theory Future work Transient rates today Notices / events in a 30 day period around April 2015: GCN: 106 (perhaps 10 new events) ATEL: 146 (probably similar ratio of new / follow-up) GAIA: 17 (public) PTF: 299 (internal) CRTS: 81(CSS) + 105 (MLS) (automatic detections) Just counting initial events: ∼500 / month, or ∼17 /day
- 5. Automate all the things Forecasting transients Decision theory Future work Robotic follow-up facilities Slow rise of automated sites / networks, e.g. LCOGT: 2 × 2m + 9 × 1m telescopes, over six sites. (http://lcogt.net/observatory)
- 6. Automate all the things Forecasting transients Decision theory Future work The (wider) problem How do we begin to ‘close the loop’ of observe =⇒ analyse =⇒ observe?
- 7. Automate all the things Forecasting transients Decision theory Future work The (wider) problem How do we begin to ‘close the loop’ of observe =⇒ analyse =⇒ observe? Transient discovery Observation prioritization Classification estimates Interesting? Yes Schedule optimization External alerts Survey data Telescope agents Follow-up data
- 8. Automate all the things Forecasting transients Decision theory Future work The (wider) problem How do we begin to ‘close the loop’ of observe =⇒ analyse =⇒ observe? Transient discovery Observation prioritization Classification estimates Interesting? Yes Schedule optimization External alerts Survey data Telescope agents Follow-up data Science!
- 9. Automate all the things Forecasting transients Decision theory Future work Diversion on DG-CVn superﬂare Fender 2014, http://adsabs.harvard.edu/abs/2014arXiv1410.1545F Osten et al (in prep)
- 10. Automate all the things Forecasting transients Decision theory Future work Missing pieces Transient discovery Observation prioritization Classification estimates Interesting? Yes Schedule optimization External alerts Survey data Telescope agents Follow-up data We’ve found a potentially interesting new transient. Looks like it could be one of class A, B, or C. What now?
- 11. Automate all the things Forecasting transients Decision theory Future work We found a transient! What now? Two implicit goals of follow-up observation: Improving classiﬁcation (Let’s see if it’s really class A.) Further observation (Tell me more! / Boring!)
- 12. Automate all the things Forecasting transients Decision theory Future work We found a transient! What now? Two implicit goals of follow-up observation: Improving classiﬁcation (Let’s see if it’s really class A.) Further observation (Tell me more! / Boring!) . . . I’ll mainly be talking about the former.
- 13. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
- 14. Automate all the things Forecasting transients Decision theory Future work Working with tiny data We’ve found a transient. But, very few datapoints: −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux Detection threshold Data
- 15. Automate all the things Forecasting transients Decision theory Future work The set-up How do we predict the possible futures for a given transient? For now, make some simplifying assumptions: Parametric lightcurve models for each class of transient. Each transient class has a known prior distribution over the morphological parameters, and this is multivariate Normal.
- 16. Automate all the things Forecasting transients Decision theory Future work Assumption: Parametric models Deterministic, ﬁnite number of parameters e.g. y = ƒ(t, t0, , τrse, τdecy) −40 −20 0 20 40 60 80 100 Time 0 2 4 6 8 10 Flux
- 17. Automate all the things Forecasting transients Decision theory Future work Result: Line of best ﬁt (Maximum likelihood) −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux ML fit Detection threshold Data
- 18. Automate all the things Forecasting transients Decision theory Future work Assumption: Multivar-Normal priors Known priors, normally distributed, covariant 234 rise_tau 8 16 24 32 a 6121824 decay_tau 2 3 4 rise_tau 6 12 18 24 decay_tau
- 19. Automate all the things Forecasting transients Decision theory Future work Construct: Model lightcurve ensembles −20 0 20 40 60 80 Time 0 2 4 6 8 10 12 14 16 18 Flux
- 20. Automate all the things Forecasting transients Decision theory Future work Result: MAP ﬁt −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux ML fit MAP fit Detection threshold Data
- 21. Automate all the things Forecasting transients Decision theory Future work But... −10 −5 0 5 10 15 20 25 30 Time 0 2 4 6 8 10 12 Flux ML fit MAP fit True Detection threshold Data
- 22. Automate all the things Forecasting transients Decision theory Future work Constrained parameter distributions Take our two datapoints, run some MCMC ﬁtting... 3.0 3.6 4.2 rise_tau 12151821 decay_tau 10 15 20 25 30 a 04812 t0 3.0 3.6 4.2 rise_tau 12 15 18 21 decay_tau 0 4 8 12 t0
- 23. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 14 16 18 Flux True Observations Comparison, 2 datapoints
- 24. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 Flux True Forecast epoch Observations 0.00 0.05 0.10 0.15 Prob. 0 5 10 15 Comparison, 2 datapoints
- 25. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 14 16 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 2 4 6 8 10 12 14 16 Comparison, 1 datapoints
- 26. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 Flux True Forecast epoch Observations 0.00 0.05 0.10 0.15 Prob. 0 5 10 15 Comparison, 2 datapoints
- 27. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 2 4 6 8 10 12 Comparison, 3 datapoints
- 28. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 2 4 6 8 10 12 Comparison, 4 datapoints
- 29. Automate all the things Forecasting transients Decision theory Future work Constrained lightcurve ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 Flux True Forecast epoch Observations 0.00 0.25 0.50 0.75 Prob. 0 2 4 6 8 10 12 Comparison, 6 datapoints
- 30. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 2 4 6 8 10 12 14 16 18 Flux Prior ensemble, Type 1
- 31. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux Prior ensemble, Type 2
- 32. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 40 Flux Comparison, 0 datapoints
- 33. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux True Forecast epoch Observations 0.00 0.05 0.10 0.15 0.20 Prob. 0 5 10 15 20 25 30 35 Comparison, 2 datapoints
- 34. Automate all the things Forecasting transients Decision theory Future work Comparing model ensembles −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux True Forecast epoch Observations 0.0 0.2 0.4 0.6 Prob. 0 5 10 15 20 25 30 35 Comparison, 2 datapoints
- 35. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
- 36. Automate all the things Forecasting transients Decision theory Future work The crux of the problem (Couldn’t get past the crux)
- 37. Automate all the things Forecasting transients Decision theory Future work The crux of the problem (Couldn’t get past the crux) Computers are good at optimisation. e.g., picking out a good observing schedule. But need a way to assign value.
- 38. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch 0.0 0.2 0.4 0.6 0.8 1.0 Relativeflux stable logistic null Intrinsic lightcurves
- 39. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch 0.0 0.2 0.4 0.6 0.8 1.0 Relativeflux stable logistic null Intrinsic LC's with noise estimates
- 40. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch −0.5 0.0 0.5 1.0 1.5 Relativeflux Sampling with noise
- 41. Automate all the things Forecasting transients Decision theory Future work −6 −4 −2 0 2 4 6 Epoch −0.5 0.0 0.5 1.0 1.5 Relativeflux Sampling with noise
- 42. Automate all the things Forecasting transients Decision theory Future work 0 1 −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 0 1 T=-4.0 0 1 PDF value T=-3.0 0 1 T=-2.0 0 1 T=-1.0 0 1 T=0.0 0 1 T=1.0 0 1 T=2.0 stable logistic null Class PDF at each epoch
- 43. Automate all the things Forecasting transients Decision theory Future work PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch −0.55 −0.50 −0.45 −0.40 −0.35 −0.30 −0.25 −0.20 FoM Information content Evaluating each epoch
- 44. Automate all the things Forecasting transients Decision theory Future work The problem with information content PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch −0.55 −0.50 −0.45 −0.40 −0.35 −0.30 −0.25 −0.20 FoM Information content Evaluating each epoch Information content weights all classes equally. What if we’re more interested in identifying one particular class?
- 45. Automate all the things Forecasting transients Decision theory Future work Introducing: confusion matrices Common or garden empirical confusion matrix: (Automatic classiﬁcation of time-variable X-ray sources; K. Lo et al, 2014)
- 46. Automate all the things Forecasting transients Decision theory Future work Confusion matrices Label True class A B C ˆA P( ˆA | A ) P( ˆA | B ) P( ˆA | C ) ˆB P( ˆB | A ) P( ˆB | B ) P( ˆC | C ) ˆC P( ˆC | A ) P( ˆC | B ) P( ˆC | C )
- 47. Automate all the things Forecasting transients Decision theory Future work Probabilistic confusion matrices Example 0.0 0.5 1.0 1.5 2.0 2.5 PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-2 stable logistic null True class stable logistic null Label stable 0.549 0.449 0.002 logistic 0.449 0.541 0.010 null 0.002 0.010 0.988
- 48. Automate all the things Forecasting transients Decision theory Future work Probabilistic confusion matrices Example 0.0 0.5 1.0 1.5 2.0 2.5 PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-2 stable logistic null True class stable logistic null Label stable 0.549 0.449 0.002 logistic 0.449 0.541 0.010 null 0.002 0.010 0.988 Diagonal entries represent recall for each class.
- 49. Automate all the things Forecasting transients Decision theory Future work PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 FoM Information content (shifted) Total Recall Evaluating each epoch
- 50. Automate all the things Forecasting transients Decision theory Future work PDF value −0.5 0.0 0.5 1.0 1.5 Relativeflux T=-5.0 T=-4.0 T=-3.0 T=-2.0 T=-1.0 T=0.0 T=1.0 T=2.0 stable logistic null −5 −4 −3 −2 −1 0 1 2 Epoch 0.5 0.6 0.7 0.8 0.9 1.0 FoM Total Recall Null Recall Evaluating each epoch
- 51. Automate all the things Forecasting transients Decision theory Future work As applied to the previous example . . . −30 −20 −10 0 10 20 30 40 50 Time 0 5 10 15 20 25 30 35 Flux True Forecast epoch Observations 0.00 0.08 0.16 0.24 0.32 Prob. 0 5 10 15 20 25 30 35 −30 −20 −10 0 10 20 30 40 50 Time −0.70 −0.65 −0.60 −0.55 −0.50 −0.45 −0.40 ICscore Comparison, 2 datapoints
- 52. Automate all the things Forecasting transients Decision theory Future work Summary Data + models + Bayesian analysis =⇒ Ensemble forecasts Ensemble forecasts + utility function =⇒ Figure-of-merit for evaluating possible actions Using the ﬁgure-of-merit as basis for a decision = Applied Bayesian decision theory (AKA active machine learning)
- 53. Automate all the things Forecasting transients Decision theory Future work Outline Automate all the things Forecasting transients Decision theory Future work
- 54. Automate all the things Forecasting transients Decision theory Future work Scheduler, simulation, testing Still need to implement a (basic) scheduler using ﬁgure-of-merit as input. Then: simulate (easy, already have models!)
- 55. Automate all the things Forecasting transients Decision theory Future work Optimization / Automated Planning Optimizing for ASAP classiﬁcation (Future-discounted weighting schemes?) Non-myopic (better-than-greedy) scheduling. Multi-armed bandit problem.
- 56. Automate all the things Forecasting transients Decision theory Future work Optimization / Automated Planning Multi-armed bandit problem Image credit: Wikipedia/Yamaguchi (CC BY-SA 3.0)
- 57. Automate all the things Forecasting transients Decision theory Future work Model reﬁnement (and basic validity) Multivariate normal prior — valid? Sum of multiple multivariate normals? Non-parametric modelling (Gaussian processes)?
- 58. Automate all the things Forecasting transients Decision theory Future work Culture Pretty sure we can make this work. . . but there’s a culture / chicken-and-egg problem.
- 59. Automate all the things Forecasting transients Decision theory Future work Code packages used astropy.modeling — for the models interface. Numpy — efﬁcient lightcurve-model calculations. pandas — (Python ANalysis of DAta Series) for handling time-series data. statsmodels — Kernel density estimates. emcee — for MCMC (Py)MultiNest — For model-evidence calculations. Seaborn — Neat plotting tools, aesthetically pleasing defaults for Matplotlib.
- 60. Automate all the things Forecasting transients Decision theory Future work Fin Plenty more to do . . . watch this space.