Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Perfect Conference: Using stochastic optimization to bring people together

302 views

Published on

One of the most valuable parts of attending a conference is the opportunity to meet others and build your network—and potentially even find new partners. The explicit goal of the Research Corporation for Scientific Advancement’s (RCSA) Scialog conferences is to foster collaboration between scientists with different specialties and approaches, and, working together with Datascope, the company has been doing so in a quantitative way for the last six years.

Datascope designed a survey to run before and after each conference to determine the level of familiarity between each attendee, as well as the topics they were most interested in and the other attendees they’d like to discuss those topics with. After the survey, Datascope implemented and continues to adapt an optimization tool that takes the survey data along with other metadata to choose optimal large topic discussion groups and small breakout groups for the conference. Afterward, a second survey is taken to see the effects on the network of attendees.

Brian Lange discusses how Datasope and RCSA arrived at the problem, the design choices made in the survey and optimization, and how the results were visualized. Along the way, Brian covers lessons learned and other problems where optimization may prove to be fruitful.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

The Perfect Conference: Using stochastic optimization to bring people together

  1. 1. Using stochastic optimization to bring people together The perfect conference Brian Lange @ Strata San José 2017
  2. 2. who is this guy?
  3. 3. who is this guy?
  4. 4. who are these folks?
  5. 5. who are these folks?
  6. 6. Today
  7. 7. Today - Case study: our work with Research Corporation for Scientific Advancement (RCSA)
  8. 8. Today - Case study: our work with Research Corporation for Scientific Advancement (RCSA) - Some high level explanation of the technique we used (simulated annealing)
  9. 9. Today - Case study: our work with Research Corporation for Scientific Advancement (RCSA) - Some high level explanation of the technique we used (simulated annealing) - Our vision of the role of a data scientist, and how they interact with the people they’re serving
  10. 10. Meet Richard* *(bad) artist’s rendering
  11. 11. By Hbarrison (Own work) [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons Scialog
  12. 12. “Some of the goals of Scialog conferences are to facilitate the formation of new collaborative teams, encourage sharing insights and catalyze novel lines of research.” Richard’s needs
  13. 13. Scialog Conferences Solar Energy Conversion Molecules Come to Life Time Domain Astronomy 2010 2011 2012 2015 2016
  14. 14. What’s different about Scialog
  15. 15. What’s different about Scialog - big picture, cross disciplinary topics
  16. 16. What’s different about Scialog - big picture, cross disciplinary topics - small conferences (40–50 junior scientists, 10–15 senior. scientists)
  17. 17. What’s different about Scialog - big picture, cross disciplinary topics - small conferences (40–50 junior scientists, 10–15 senior. scientists) - invite only
  18. 18. What’s different about Scialog - big picture, cross disciplinary topics - small conferences (40–50 junior scientists, 10–15 senior. scientists) - invite only - unconventional format
  19. 19. What’s different about Scialog - big picture, cross disciplinary topics - small conferences (40–50 junior scientists, 10–15 senior. scientists) - invite only - unconventional format How well does this work?
  20. 20. Quantifying connection
  21. 21. Quantifying connection
  22. 22. Quantifying connection
  23. 23. Quantifying connection
  24. 24. Quantifying connection
  25. 25. before after
  26. 26. Scialog 2015: Molecules Come to Life
  27. 27. Scialog 2015: Molecules Come to Life - 118 new conversations
  28. 28. Scialog 2015: Molecules Come to Life - 118 new conversations - 17 new collaborations
  29. 29. Scialog 2015: Molecules Come to Life - 118 new conversations - 17 new collaborations - 100% growth from pre-conference
  30. 30. Scialog 2015: Molecules Come to Life - 118 new conversations - 17 new collaborations - 100% growth from pre-conference - 53.6% of attendees formed a new collaboration with an attendee
  31. 31. measure improve
  32. 32. Mini Breakout Groups - round 1 A D G J B E H K C F I L conference attendees
  33. 33. Mini Breakout Groups - round 2 A D G J B E H KC F I L
  34. 34. Mini Breakout Groups - round 3 A D G J B E H K C F I L
  35. 35. Are these good groups? A D G J B E H K C F I L
  36. 36. Is this a good group? DJ F L
  37. 37. Is this a good group? DJ F L
  38. 38. Is this a good group? DJ F L 1
  39. 39. Is this a good group? DJ F L 1 2
  40. 40. Is this a good group? DJ F L 4 1 2
  41. 41. Is this a good group? DJ F L 4 1 2 terribleness: 7
  42. 42. Is this a good group? J F L 4 1 2 terribleness: 7
  43. 43. Is this a good group? J F L 2
  44. 44. Is this a good group? BJ F L 2
  45. 45. Is this a good group? BJ F L 2 terribleness: 2
  46. 46. 14778213400262135041705388361938994140625 possible configurations
  47. 47. 14778213400262135041705388361938994140625 possible configurations 1.47 x 1040, or 14 duodecillion
  48. 48. quadrillions of years :(
  49. 49. terribleness slope intercept optimization
  50. 50. DJ F L 4 1 2 terribleness: 7 0 1.75 3.5 5.25 7 JDFL
  51. 51. BJ F L 2 terribleness: 2 0 1.75 3.5 5.25 7 JDFL JBFL
  52. 52. 1 BJ C L 4 terribleness: 6 0 1.75 3.5 5.25 7 JDFL JBFL JBCL
  53. 53. 1 BJ C E terribleness:1 0 1.75 3.5 5.25 7 JDFL JBFL JBCL JBCE
  54. 54. 4 4 KJ C E terribleness:8 0 2 4 6 8 JDFL JBFL JBCL JBCE JKCE
  55. 55. slope intercept terribleness
  56. 56. slope intercept terribleness 🤔
  57. 57. enter: simulated annealing
  58. 58. simulated annealing
  59. 59. simulated annealing a type of stochastic optimization
  60. 60. simulated annealing a type of stochastic optimization stochastic optimization algorithm with randomness introduced to avoid getting caught in local minima
  61. 61. temperature: 100 A D G J B E H K C F I L terribleness: 350 previous terribleness: 350
  62. 62. temperature: 1001. make a random “move” from current state A D G J B E H K C F I L terribleness: 350 previous terribleness: 350
  63. 63. temperature: 100 A D G JB E H K C F I L terribleness: 380 previous terribleness: 350 1. make a random “move” from current state
  64. 64. temperature: 100 A D G JB E H K C F I L terribleness: 380 previous terribleness: 350 1. make a random “move” from current state 2. check if the new state is better than the previous
  65. 65. temperature: 100 A D G JB E H K C F I L terribleness: 380 previous terribleness: 350 1. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead.
  66. 66. temperature: 100 A D G JB E H K C F I L terribleness: 380 previous terribleness: 350 1. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature
  67. 67. temperature: 1001. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature A D G JB E H K C F I L terribleness: 380 previous terribleness: 380
  68. 68. temperature: 1001. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature 5. Stop when temperature is too low. Otherwise, reduce temperature and repeat. A D G JB E H K C F I L terribleness: 380 previous terribleness: 380
  69. 69. temperature: 901. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature 5. Stop when temperature is too low. Otherwise, reduce temperature and repeat. A D G JB E H K C F I L terribleness: 243 previous terribleness: 380
  70. 70. temperature: 901. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature 5. Stop when temperature is too low. Otherwise, reduce temperature and repeat. A D G JB E H K C F I L terribleness: 243 previous terribleness: 243
  71. 71. ⏩ fast forward
  72. 72. temperature: 31. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature 5. Stop when temperature is too low. Otherwise, reduce temperature and repeat. A D G J B E HK C FI L terribleness: 63 previous terribleness: 63
  73. 73. temperature: 31. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature 5. Stop when temperature is too low. Otherwise, reduce temperature and repeat. A D G J B E H K C FI L terribleness: 194 previous terribleness: 63
  74. 74. temperature: 31. make a random “move” from current state 2. check if the new state is better than the previous 3. If it is, keep it and skip ahead. 4. If not, there’s a chance we’ll still keep it, depending on the temperature 5. Stop when temperature is too low. Otherwise, reduce temperature and repeat. A D G J B E HK C FI L terribleness: 63 previous terribleness: 63
  75. 75. Not just for groups of things
  76. 76. Not just for groups of things - circuit board designs
  77. 77. Not just for groups of things - circuit board designs - stock trading rules
  78. 78. Not just for groups of things - circuit board designs - stock trading rules - anything which has a defined state, a “move” (aka transition to another state), and a measure of goodness
  79. 79. very, very tweakable
  80. 80. very, very tweakable - make “smarter-than-random” moves
  81. 81. very, very tweakable - make “smarter-than-random” moves - change the cooling function (reheat and cool)
  82. 82. very, very tweakable - make “smarter-than-random” moves - change the cooling function (reheat and cool) - save best state so far and restart from it periodically
  83. 83. very, very tweakable - make “smarter-than-random” moves - change the cooling function (reheat and cool) - save best state so far and restart from it periodically - repeat many times from different random starting states
  84. 84. very, very tweakable - make “smarter-than-random” moves - change the cooling function (reheat and cool) - save best state so far and restart from it periodically - repeat many times from different random starting states - etc…
  85. 85. DATA SCIENCE PITFALL WARNING very, very tweakable Icons by Andrea Novoa, Chris Kerr, and Ananth from The Noun Project - make “smarter-than-random” moves - change the cooling function (reheat and cool) - save best state so far and restart from it periodically - repeat many times from different random starting states - etc…
  86. 86. let’s talk more about terribleness
  87. 87. client needs
  88. 88. client needs - minimize how “connected” the people in each group are
  89. 89. client needs - minimize how “connected” the people in each group are - don’t want people to be in the same group with somebody twice
  90. 90. client needs - minimize how “connected” the people in each group are - don’t want people to be in the same group with somebody twice - etc…
  91. 91. client needs - minimize how “connected” the people in each group are - don’t want people to be in the same group with somebody twice - etc… math/code
  92. 92. generate ideas build prototypeevaluation iterative problem solving anticipate refining your solution “what does Richard need?”
  93. 93. generate ideas build prototypeevaluation iterative problem solving anticipate refining your solution “these could be better”
  94. 94. generate ideas build prototypeevaluation iterative problem solving anticipate refining your solution “let’s tweak the optimization technique”
  95. 95. generate ideas build prototypeevaluation iterative problem solving anticipate refining your solution “much better, what do you think Richard?”
  96. 96. generate ideas build prototypeevaluation iterative problem solving anticipate refining your solution “pretty good, but I don’t like ______”
  97. 97. generate ideas build prototypeevaluation iterative problem solving anticipate refining your solution “cool, let’s tweak the objective function”
  98. 98. generate ideas build prototypeevaluation iterative problem solving anticipate refining your solution “these look great!”
  99. 99. generate ideas build prototypeevaluation iterative problem solving “these are great!” also an optimization algorithm of sorts…
  100. 100. so, did it work?
  101. 101. Scialog 2015: Molecules Come to Life
  102. 102. Scialog 2015: Molecules Come to Life - In 30 out of 39 group discussions (77%), none of the people in those groups had even heard of each other before
  103. 103. Scialog 2015: Molecules Come to Life - In 30 out of 39 group discussions (77%), none of the people in those groups had even heard of each other before - In all group discussions, no one had spoken or collaborated with anyone else in their group
  104. 104. Scialog 2015: Molecules Come to Life - In 30 out of 39 group discussions (77%), none of the people in those groups had even heard of each other before - In all group discussions, no one had spoken or collaborated with anyone else in their group - On average, a group had 2.6 different disciplines (physics, biology, etc)
  105. 105. Scialog 2015: Molecules Come to Life - In 30 out of 39 group discussions (77%), none of the people in those groups had even heard of each other before - In all group discussions, no one had spoken or collaborated with anyone else in their group - On average, a group had 2.6 different disciplines (physics, biology, etc) - Each group had at least one theorist and one experimentalist
  106. 106. Scialog 2015: Molecules Come to Life __________ is important to the success of Scialog 0 7.5 15 22.5 30 Very Much Disagree Neutral Very Much Agree Mini Breakouts Regular Discussions
  107. 107. Scialog 2015: Molecules Come to Life “We received 20 collaborative proposals, the most ever at a Scialog.”
  108. 108. further iterations
  109. 109. A D G J B E HK C FI L Tissue mechanics and evolution The physics of genetic networks In vivo mechanics Topic groups/preferences
  110. 110. Topic A Topic B Topic C Topic groups/preferences
  111. 111. Desired connections
  112. 112. Nominated topic experts
  113. 113. how might this work at a bigger conference? - survey including EVERYONE not tenable - use latent sources like social media - use RFID or other hardware to infer connections at the conference - use a survey with a representative sample and similarity metrics to project preferences
  114. 114. how might this work at a bigger conference? - different format - separate opt-in discussion track - rather than groups, make email introductions
  115. 115. what to learn from this
  116. 116. optimization isn’t just for straightforward mathematical problems
  117. 117. data science isn’t just for straightforward mathematical problems
  118. 118. a great data scientist acts as a translator
  119. 119. iteration drives improvement on both methods and data collection
  120. 120. if you can measure it, you have the data to actively improve it
  121. 121. .co@bjlange thanks! questions? brian [at] datascope.co

×