Download to read offline
Uploaded byAllison Miller
When Algorithms Are Our Co-Pilots
The document discusses the role of algorithms in information risk management and decision-making systems, highlighting their benefits in simplifying, automating, and optimizing various processes. It emphasizes the complexity and risks associated with algorithm-driven systems, the importance of effective control measures, and the implications of automation in decision-making. Key themes include the relationship between algorithms and human operators, the potential for both efficiency and inefficiency through automation, and the necessity for understanding the behaviors of these systems and their interactions.
More Related Content
Similar to When Algorithms Are Our Co-Pilots
![[DSC Europe 25] Srba Markovic - From Pilot to Production: Overcoming AI Deplo...](https://cdn.slidesharecdn.com/ss_thumbnails/yjjmrtytmwbalxlba7px-4-srba-markovic-from-pilot-to-production-overcoming-ai-deployment-blockers-with-260114111931-4a892d44-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Danilo Djukanovic - From Vibes to KPIs: Turning Culture Into ...](https://cdn.slidesharecdn.com/ss_thumbnails/inqestws5wf0cik2glgv-3-danilo-djukanovic-from-vibes-to-kpis-presentation-260114111931-dacff81f-thumbnail.jpg?width=640&height=640&fit=bounds)
When Algorithms Are Our Co-Pilots
- 1. #SIRACon14 When Algorithms Are Our Co-Pilots Allison Miller President, SIRA 1
- 2. #SIRACon14 @selenakyle Outline ❖ What are algorithms? ❖ OMG, I LOVE Algorithms! ❖ Algorithms as the foundation of information risk control systems ❖ Algorithms VS (Adversity + Adversaries) ❖ Control vs Complexity 2
- 3.
- 4. #SIRACon14 @selenakyle Namshub ❖Tie Your Shoes ❖Common core ❖$15.51 4
- 5. #SIRACon14 @selenakyle Areyou a good algorithm or a bad 1,000,000,000,000 -1 algorithm? 5
- 6. #SIRACon14 @selenakyle Namshub ❖Tie Your Shoes ❖Common core ❖$15.51 6
- 7.
- 8. #SIRACon14 @selenakyle w(e)<3 Math ❖ Gauss: What’s Normal Anyway ❖ Bernoulli: Making Heads or Tails of things ❖ Plenty of Poisson in the Sea ❖ Euler: Graph theory, topology ❖ All about that Bayes ❖ Groves of Decision Trees, Nets of Neurons, Totally Random Forests 8
- 9.
- 10. #SIRACon14 @selenakyle WhatAlgorithms Do ❖Simplify ❖Match ❖Price ❖Automate ❖Predict ❖Compete 10
- 11.
- 12. #SIRACon14 @selenakyle ThePlayer is the Game ❖ Tic Tac Toe ❖ Chess ❖ Trivia ❖ Poker ❖ Tit for Tat 12 Kasparov vs X3D Fritz
- 13. #SIRACon14 @selenakyle DiscussingGames Mechanics of decision trees UP DOWN CIRCLE RED BLUE MARIO LUIGI KIRBY GIZMO 10, 3 2, 5 -3, 3 2, 10 A B B A A A 13
- 14. #SIRACon14 @selenakyle #WTF (What, Thou Failest?) * Es tu, Bot — eh? 14
- 15. #SIRACon14 @selenakyle AlgorithmsGone Wild ❖Bidding/auctions ❖Targeting ❖Too Many Likes 15
- 16.
- 17. #SIRACon14 @selenakyle Speakingof $$$ ❖Black Scholes ❖Automation of trading ❖HFT: Faster than the speed of light 17
- 18.
- 19. #SIRACon14 @selenakyle ManagingDecisions Risk Management = Decision Management - Algorithms are useful for “outsourcing” decisions to our systems - Systems enforce risk strategies Defenders and operators design control systems that make decisions - Where risks/issues manifest in observable behavior - They can make moves/counter-moves depending on the context and understanding of an actor’s identity or intent - Where system or individual costs/payoffs depend on the outcome of an actor’s actions - Decision strategies can be optimized 19
- 20. 20 UP DOWN CIRCLE RED BLUE MARIO LUIGI KIRBY GIZMO 10, 3 2, 5 -3, 3 2, 10 A B B A A A Payoffs!
- 21. 21 Optimization! %of population Cost Operations Total Cost Cost of Defects Cost of Control Number of Defects Produced Risk/Loss
- 22. #SIRACon14 @selenakyle AlgorithmsRule ❖Defenses ❖DevOps ❖Devices 22
- 23. #SIRACon14 @selenakyle Operatinga Learning System Disposition & Time Email CC# Items Total Submit Maybe No! Yes!! OutScuome AttSeumpt Black & Whitelists Machine Learning Velocity & Spend caps Geo & IP Logic Linking Data • Reporting • Metrics • Analysis • Modeling Good Bad Indeterminate 23
- 24. #SIRACon14 @selenakyle TheBetter Mousetrap Automates defensive action x-platform In Real Time - Fast In Time to Minimize Loss - Accurate - Cheap Reasonable False Positives As good as a human specialist Reduces Reduction > Cost Created Cheaper than Manual intervention 24
- 25. #SIRACon14 @selenakyle TheJoyof… #If/Then 25
- 26. #SIRACon14 @selenakyle Thefirst rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency. –Bill Gates 26
- 27. #SIRACon14 @selenakyle ManagingSocial Systems “…all policy systems…pose incentives that are reacted to by groups of agents acting in their own interest, and often those reactions are unexpected and act counter to the policy’s intentions…” — Brian Arthur, All Systems will be Gamed: Exploitive Behavior in Economic and Social Systems 27
- 28. #SIRACon14 @selenakyle Sendin the Bots ❖ Convenience vs dependence ❖ Transparency vs obfuscation ❖ Simplicity vs complexity 28
- 29.
- 30. #SIRACon14 @selenakyle Asimov’s3 Laws of Robotics 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. 2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law. 3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law. 30
- 31. #SIRACon14 @selenakyle SIRAtonin3 Bot Guidelines 1. Bots are thoughtless but purposeful 2. Bots compete but will also collude 3. The behavior of a network of bots is more complex than the behavior of a single bot 31
- 32. #SIRACon14 @selenakyle TheComplexity Dimension Embedded system controls are part of the system Instrument your instrumentation Learning systems are hungry Intelligent agents are dumb Fatal feedback is fatal Risk hides in complexity 32
- 33. 33 Photo: MichaelRubenstein/Harvard University https://www.youtube.com/watch?v=G1t4M2XnIhI#action=share
- 34.
Editor's Notes
- #3 What’s your favorite algorithm? My algorithm can beat up your algorithm. Shout out to Brian Arthur’s “All Systems Will Be Gamed”
- #5 Tying shoes (learning, shortcuts) Common core —> cashier (explicit vs implicit)
- #6 Common Core example: http://blogs.edweek.org/teachers/teaching_now/common-core-standard-math-addition.jpg
- #7 Tying shoes (learning, shortcuts) Common core —> cashier (explicit vs implicit)
- #9 All about that Bayes Spam Normal (Gauss & least squares) / Poisson the probability of a given number of events occurring in a fixed interval of time 1) if events occur @ a known average rate & 2) independently (rate-wise) (cars at a stoplight, telephone calls arriving, claims received…also radioactive decay) Bernoulli (law of large numbers, probability) Euler (graph theory, topology) Decision trees
- #13 Algorithms compete: Against humans, against each other Chess: Deep Blue defeated Kasparov in 1997, insane decision tree (1.4 tons 256 processors, 200M chess positions/second) Trivia: Watson in 2011 beat ll human contenders 200M pages of content, 2800 processor cores, 16TB of RAM Tit for Tat Poker (irrational)
- #14 Algorithms: the ultimate rational agents #FTW
- #16 Bidding/auctions: eBay (snipes) —> Amazon book The Making of a Fly in April 2011 listed for ~1.7M &2.1M, w/in 2 weeks price peaked for almost $24M, then dropped down to around $100 http://www.geek.com/news/amazon-algorithm-freaks-out-sells-book-for-23-6-million-1347813/ Profiling (creepy): Target marketing As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window…Take a fictional Target shopper named Jenny Ward, who is 23, lives in Atlanta and in March bought cocoa-butter lotion, a purse large enough to double as a diaper bag, zinc and magnesium supplements and a bright blue rug. There’s, say, an 87 percent chance that she’s pregnant and that her delivery date is sometime in late August. via How Companies Learn Your Secrets – NYTimes.com. http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/ Preferences (Too many likes) http://www.wired.com/2014/08/i-liked-everything-i-saw-on-facebook-for-two-days-heres-what-it-did-to-me/
- #17 But why? Algorithms do what they are told to do, they are context-less. Designers have expectations about features, implementation — may have no expectations about upstream/downstream interactions, interconnections, or failure conditions. <Unit tests>
- #18 Valuation/Arbitrage: Finance —> Flash Crash —> HFT May 2010 Dow dropped ~1000 points w/in 5 minutes, roughly $1Trillion http://money.cnn.com/2010/05/06/markets/markets_newyork/ Growth of options/derivatives (70’s) leading to algorithms to predict value (prediction, bc strike price, expiration, volatility) —> Black Scholes (partial differential equations…Nobel prize…calculate the value as values change & find opportunities to profit off of mispriced securities). Math at speed. Automation of trading, disappearance of pits People were the bottleneck, queues jammed, then the speed of light became the bottleneck, (private) dark fiber & the growth of HFT (the straightest line between Chicago Board Options Exchange & Wall Street)
- #23 As Riskafarians, Algorithms literally rule our systems in the sense that they are tools we use to measure AND MANAGE our systems Software defined Networking as an analogy Software defined Defense, Administration, Hardware Key is algorithmically driven and INTEGRATED into systems
- #26 Automation & the tyranny of hard-coded “policies”! Death via skew
- #27 Scaling-up (Efficiency): Be careful what you wish for.
- #28 Furthermore: Adversity vs Adversaries Also: Groups of agents, self-interested
- #29 Botter beware
- #33 Analog: Vulns in security software Data: Nutrients vs big “empty” calories MECE: Mutually Exclusive, Collectively Exhaustive More unknown unknowns