Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Don't let this happen to you! Cloud, complexity and drift

25,809 views

Published on

Presentation given at Gluecon 2012 on Thursday morning at 9AM MT. Covers complex adaptive systems and systems thinking as applied to cloud computing and the API economy.

Published in: Business, Economy & Finance

Don't let this happen to you! Cloud, complexity and drift

  1. Don’t let this happen to you! Cloud, complexity and drift James Urquhart@jamesurquhart http://gigaom.com/cloud
  2. May 6, 2010
  3. ”[A] large fundamental trader chose to execute [a$4.1B] sell program via an automated executionalgorithm (Sell Algorithm)." - Findings Regarding The Market Events of May 6, 2010 http://www.sec.gov/news/studies/2010/marketevents-report.pdf
  4. "...the Sell Algorithm…executed the sell program extremelyrapidly in just 20 minutes.”The market responded, and trading volume increased…"... [The Sell Algorithm] responded to the increased volume byincreasing the rate at which it was feeding the orders into themarket." - Findings Regarding The Market Events of May 6, 2010 http://www.sec.gov/news/studies/2010/marketevents-report.pdf
  5. Quick!What was the root cause?
  6. Automatic Trading Market AAlgorithm 1
  7. Automatic Trading Market AAlgorithm 1 Automatic Automatic Automatic Trading Trading Trading Algorithm 2 Algorithm 3 Algorithm 4
  8. Market BAutomatic Trading Market AAlgorithm 1 Automatic Automatic Automatic Trading Trading Trading Algorithm 2 Algorithm 3 Algorithm 4
  9. Automatic Automatic Trading Trading Algorithm 6 Algorithm 5 Market BAutomatic Trading Market AAlgorithm 1 Automatic Automatic Automatic Trading Trading Trading Algorithm 2 Algorithm 3 Algorithm 4
  10. Automatic Automatic Trading Trading Algorithm 6 Algorithm 5 Market BAutomatic Trading Market AAlgorithm 1 Automatic Automatic Automatic Trading Trading Trading Algorithm 2 Algorithm 3 Algorithm 4
  11. Automatic Automatic Trading Trading Algorithm 6 Algorithm 5 Automatic Automatic Trading Market B Automatic TradingAlgorithm N Trading Market A Algorithm N Algorithm N Automatic Automatic Automatic Trading Trading Trading Algorithm 2 Algorithm 3 Algorithm 4
  12. “Root cause” is not an answer—it’s a clue.
  13. "May 6 was…an important reminder of theinterconnectedness of our derivatives andsecurities markets, particularly with respect toindex products." - Findings Regarding The Market Events of May 6, 2010 http://www.sec.gov/news/studies/2010/marketevents-report.pdf
  14. Interconnectedness
  15. The Internet is aboutinterconnectedness.
  16. APIs are aboutinterconnectedness.
  17. “The cloud” is aboutinterconnectedness.
  18. Both cloud computingand stock markets are complex adaptive systems.
  19. COMPLEX?!?
  20. What areComplex Adaptive Systems?
  21. A large number of individual agents
  22. AgentsOther Information/A ctions StateAgents Learning Information/A Other Rules ctions Agents
  23. A large number of individual agents+ dynamic interactions between agents
  24. Dynamic Interaction From NetLogo 4.1.3
  25. A large number of individual agents+ dynamic interactions between agents+ rules for reacting to/interacting withother agents
  26. Rules
  27. Equals
  28. A system that:demonstrates emergent behavior,
  29. Emergent Behavior
  30. A system that:demonstrates emergent behavior,can be modeled,
  31. Models TimeFlocking model fromNetLogo 4.1.3http://ccl.northwestern.edu/netlogo/
  32. A system that:demonstrates emergent behavior,can be modeled,butmakes precise prediction of futurebehavior impossible.
  33. Unpredictability
  34. Unpredictability
  35. Unpredictability
  36. Unpredictability
  37. So…what can you do?
  38. EMBRACECOMPLEXITY
  39. Never!
  40. EMBRACECOMPLEXITY SYSTEMS
  41. Embrace Systems Thinking
  42. Practical advice?
  43. Do your homework!
  44. Design for resilience
  45. “The goal of producing a maximumsustained yield may result in a more stablesystem of reduced resilience.” “Command and Control and the Pathology of Natural Resource Management”, C. S. Holling and Gary K. Meffe http://landscape.forest.wisc.edu/courses/Landscape565spr01/Holling_Meffe1996.pdf
  46. “[T]he resilience-stability tradeoff is morethan just a simple transformation indistribution. …[A]gents adapt to aprolonged period of stability in such amanner that the system cannot ‘withstandeven modest adverse shocks.’” “The Euro and the Resilience-Stability Tradeoff”, Ashwin Parameswaran http://www.macroresilience.com/2011/11/14/the-euro-and-the-resilience-stability-tradeoff/
  47. Focus on relationships
  48. Automatic Automatic Trading Trading Algorithm 6 Algorithm 5 Automatic Automatic Trading Market B Automatic TradingAlgorithm N Trading Market A Algorithm N Algorithm N Automatic Automatic Automatic Trading Trading Trading Algorithm 2 Algorithm 3 Algorithm 4
  49. Automatic Automatic Trading Trading Algorithm 6 Algorithm 5 Automatic Automatic Trading Market B Automatic TradingAlgorithm N Trading Market A Algorithm N Algorithm N Automatic Automatic Automatic Trading Trading Trading Algorithm 2 Algorithm 3 Algorithm 4
  50. Circuit Breaker
  51. http://techblog.netflix.com/2011/12/making-netflix-api-more-resilient.html
  52. Avoid Drift
  53. Drift
  54. Avoiding Drift:Test production
  55. Release themonkeys!!!• Failure• Degredation• Usage
  56. Avoiding Drift:The agent is not the system
  57. App-centric System-centric• Monitor each app • Monitor system at many separately levels, and use as feedback• Dig for root cause • Search for system weakness• Attempt to stabilize • Focus on resilience
  58. Avoiding Drift: YOUare part of the system
  59. Don’t let this happen to you! Cloud, complexity and drift

×