Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Training your astronomy
robots to work as a team
Tim Staley, Amr Hussein & the
4 Pi Sky group
(R. Fender, A. Stewart, G. P...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Outline
The “secon...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Outline
The “secon...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
The astronomer’s t...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
The astronomer’s t...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
The astronomer’s t...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
The astronomer’s t...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
The astronomer’s t...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
The GRB microcosm
...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
How does VOEvent h...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Outline
The “secon...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
VOEvent-Parse
A su...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
VOEvent-Parse
impo...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
VOEvent-Parse
http...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Other VOEvent tool...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Outline
The “secon...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Loud noises!
Faste...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
How can we facilit...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
How can we facilit...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
How can we facilit...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Outline
The “secon...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Negotiation is har...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
An optimal respons...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Agent based system...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Agent based system...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
One block is missi...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Outline
The “secon...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
What are we optimi...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
What are we optimi...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
What are we optimi...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
What are we optimi...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
What are we optimi...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Guessing at scienc...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Representing infor...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
Start with the tra...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0 5 10 15 20 25 30...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0 5 10 15 20 25 30...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0 5 10 15 20 25 30...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0 5 10 15 20 25 30...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0 5 10 15 20 25 30...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0.00 0.01 0.0210-1...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0 5 10 15 20 25 30...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
0 5 10 15 20 25 30...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
This is the end
Op...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
This is the end
Op...
The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
This is the end
Op...
Upcoming SlideShare
Loading in …5
×

Training your astronomy robots to work as a team

246 views

Published on

I present a case that the astronomy community is missing a part of the puzzle for the next era of automated big-survey astronomy: we currently have very little published work on target prioritization and optimized observation scheduling. I discuss the sociological issues surrounding the sort of open collaboration needed to make optimal use of globally distributed observatories, and show some preliminary work on generally-applicable classification methods.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Training your astronomy robots to work as a team

  1. 1. Training your astronomy robots to work as a team Tim Staley, Amr Hussein & the 4 Pi Sky group (R. Fender, A. Stewart, G. Pietka, T. Munoz-Darias) Radio Transients with SKA Pathfinders South Africa, July 2013 WWW: 4pisky.org , timstaley.co.uk
  2. 2. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Outline The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
  3. 3. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Outline The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
  4. 4. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers The astronomer’s telegram
  5. 5. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers The astronomer’s telegram
  6. 6. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers The astronomer’s telegram Most of transient astronomy still operates using telegraph (well, email) and telephone.
  7. 7. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers The astronomer’s telegram Most of transient astronomy still operates using telegraph (well, email) and telephone. High latency results in significant duplication of effort.
  8. 8. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers The astronomer’s telegram Most of transient astronomy still operates using telegraph (well, email) and telephone. High latency results in significant duplication of effort. Requires significant astronomer / observation time ratio.
  9. 9. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers The GRB microcosm GRB community has overcome latency issues using GCN. Works well, but narrow in scope. Automated system is one way (NASA −→ World); follow-up reverts to manual.
  10. 10. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers How does VOEvent help? Fast. Flexible. Allows for automated creation, collation, filtering . . .
  11. 11. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Outline The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
  12. 12. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers VOEvent-Parse A super-lightweight python wrapper / library built on lxml. https://github.com/timstaley/ voevent-parse pip install voevent-parse
  13. 13. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers VOEvent-Parse import voeparse v = voeparse.load(’path/to/voevent_packet.xml’) #Prints ivo://nasa.gsfc.tan/gcn print "AuthorIVORN:", v.Who.AuthorIVORN #Alters the XML value. v.Who.AuthorIVORN = ’ivo://i.heart.python/lxml’
  14. 14. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers VOEvent-Parse http: //voevent-parse.readthedocs.org/
  15. 15. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Other VOEvent tools See also: http://comet.transientskp.org/ — Connect to the VOEvent backbone! https: //github.com/timstaley/pysovo — Email people! Do stuff! Generate a VOEvent, (and a templated GCN / Atel!) from your web browser— Coming soon? (Would anyone use it?)
  16. 16. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Outline The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
  17. 17. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Loud noises! Faster communication by itself only gets you so far (HFT disasters?).
  18. 18. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers How can we facilitate collaboration? At a minimum: Need to be open about what we are doing. Better yet: Share / trade observations.
  19. 19. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers How can we facilitate collaboration? At a minimum: Need to be open about what we are doing. Better yet: Share / trade observations. Minor hurdle: Astronomy community has no universal standard for ’observation requests.’
  20. 20. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers How can we facilitate collaboration? At a minimum: Need to be open about what we are doing. Better yet: Share / trade observations. Minor hurdle: Astronomy community has no universal standard for ’observation requests.’ Major hurdle: Political will / rewards system for observers releasing open data. (Or; only share with your friends? Trust vs. potential benefits.)
  21. 21. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Outline The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
  22. 22. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Negotiation is hard Negotiation and decentralization adds complexity. Trade off: Freedom and robustness vs. efficiency, single set of priorities at any one time. e.g. GCN community vs. PTF, LCOGT.
  23. 23. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers An optimal response system Something like this, perhaps?
  24. 24. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Agent based systems These have been seen before... A. Allan, E. Saunders et al. c.f. 2003–2007 (RIP Estar)
  25. 25. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Agent based systems These have been seen before... A. Allan, E. Saunders et al. c.f. 2003–2007 (RIP Estar) Large collaboration Scheduling: E. Saunders (small world) Currently being deployed
  26. 26. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers One block is missing
  27. 27. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Outline The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers
  28. 28. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers What are we optimizing for? Science value
  29. 29. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers What are we optimizing for? Science value { Subject to cost constraints }
  30. 30. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers What are we optimizing for? Science value { Subject to cost constraints } { And telescope availability }
  31. 31. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers What are we optimizing for? Science value { Subject to cost constraints } { And telescope availability } { And . . . }
  32. 32. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers What are we optimizing for? Science value { Subject to cost constraints } { And telescope availability } { And . . . } How do we assign a science value — when we don’t know what it is yet?
  33. 33. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Guessing at science value Suppose we have a tentative set of classification probabilities. Choose follow-up observations with best chance of refining probabilities. Assign expected science values to transients based on those potential outcomes. Hence, assign science value to planned observations, feed to scheduler for optimization.
  34. 34. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Representing information gain Assign ‘confusion matrices’ to a given observation. . . . and the rest is ‘just’ computer science (hopefully). Need to determine e.g. how well an optical observation separates classes.
  35. 35. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers Start with the training data 0 5 10 15 20 25 30 35 Optical mag 10-2 10-1 100 101 102 103 104 105 106 RadiofluxmJy Quasar (Optical)(8629) Quasar (Radio)(976) Stellar(707) Quasar (Type II)(531) GRB(14) Pulsar(10) SN(6) X-ray binary(4) Magnetar(3) CV(3) M101(1) Bell#1(1) RRAT(1)
  36. 36. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0 5 10 15 20 25 30 35 10−2 10−1 100 101 102 103 104 105 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 PDF 10−2 10−1 100 101 102 103 104 105 Radioflux[mJy] 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Optical magnitude 0.00 0.02 0.04 0.06 0.08 0.10 PDF Stellar Stellar
  37. 37. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0 5 10 15 20 25 30 35 10−2 10−1 100 101 102 103 104 105 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 PDF 10−2 10−1 100 101 102 103 104 105 Radioflux[mJy] 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Optical magnitude 0.00 0.05 0.10 0.15 0.20 0.25 PDF GRB GRB
  38. 38. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0 5 10 15 20 25 30 35 10−2 10−1 100 101 102 103 104 105 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 PDF 10−2 10−1 100 101 102 103 104 105 Radioflux[mJy] 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Optical magnitude 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 PDF Quasar (Radio) Quasar (Radio)
  39. 39. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0 5 10 15 20 25 30 35 10−2 10−1 100 101 102 103 104 105 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 PDF 10−2 10−1 100 101 102 103 104 105 Radioflux[mJy] 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Optical magnitude 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 PDF Pulsar Pulsar
  40. 40. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0 5 10 15 20 25 30 35 10−2 10−1 100 101 102 103 104 105 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 PDF 10−2 10−1 100 101 102 103 104 105 Radioflux[mJy] 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Optical magnitude 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 PDF Stellar GRB Quasar (Radio) Pulsar Combined Stellar, GRB, Quasar (Radio), Pulsar
  41. 41. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0.00 0.01 0.0210-1 100 101 102 103 104 Conditionals 0.0 0.2 0.4 0.6 0.8 1.010-1 100 101 102 103 104 Relative Stellar Pulsar Quasar (Radio) GRB Radio probabilities: flux err = 0.5mJy
  42. 42. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0 5 10 15 20 25 30 350.00 0.02 0.04 0.06 0.08 0.10 0.12 Conditionals Stellar Pulsar Quasar (Radio) GRB 0 5 10 15 20 25 300.0 0.2 0.4 0.6 0.8 1.0 Relative Non-detection Opt probabilities: flux err = 0.5mag
  43. 43. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers 0 5 10 15 20 25 30 350.000 0.005 0.010 0.015 0.020 0.025 Conditionals Stellar Pulsar Quasar (Radio) GRB 0 5 10 15 20 25 300.0 0.2 0.4 0.6 0.8 1.0 Relative Non-detection Opt probabilities: flux err = 0.5mag
  44. 44. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers This is the end Open source VOEvent tools are here, now. Do what you want with them.
  45. 45. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers This is the end Open source VOEvent tools are here, now. Do what you want with them. (Automated) Transient astronomy is clearly nascent. The rise of the robots will likely be a slow and progressive one. Political will and manpower may be the limiting factors? (cf. eStar)
  46. 46. The “second grade soccer” problem A quick plug But... The centralized approach Classifiers to schedulers This is the end Open source VOEvent tools are here, now. Do what you want with them. (Automated) Transient astronomy is clearly nascent. The rise of the robots will likely be a slow and progressive one. Political will and manpower may be the limiting factors? (cf. eStar) If you have experience in schedulers I’d like to hear the gritty details.

×