Belief: learning about new problems from old things
•Download as PPT, PDF•
1 like•599 views
Talk given in NYC, February 2017, about what belief is and how it's manipulated across humans and machines
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (20)
Similar to Belief: learning about new problems from old things
Similar to Belief: learning about new problems from old things (20)
More from Sara-Jayne Terp
More from Sara-Jayne Terp (20)
Recently uploaded
Recently uploaded (20)
Belief: learning about new problems from old things
- 1. Belief Learning about new problems from old things
- 2. Why am I interested in belief? • Long-time love of autonomy • Most data science is based on belief • Lean, Agile etc are based on belief • Deep learning systems are creating assholes • America’s belief systems are being hacked
- 3. Agenda • Understanding beliefs • Why do techs care about this? • What goes wrong? • What could help?
- 5. Belief • Pistis: confidence in a ‘fact’ • Doxa: acceptance of a ‘fact’ • Normative belief: what you think other people expect you to believe…
- 6. You don’t escape the machine Sensor Effector
- 9. Learning = Forming Beliefs
- 10. Beliefs can be shared
- 11. Beliefs can be shared
- 12. Beliefs can be hacked A Facebook ‘like’, he said, was their most “potent weapon”. “Because using artificial intelligence, as we did, tells you all sorts of things about that individual and how to convince them with what sort of advert. And you knew there would also be other people in their network who liked what they liked, so you could spread. And then you follow them. The computer never stops learning and it never stops monitoring.” Sensor Effector Contagion Adaptation
- 13. Beliefs can be ‘wrong’
- 14. The Internet is made of beliefs
- 15. Why do techs care about this?
- 16. Lean Enterprise
- 17. Lean Enterprise includes beliefs ts All About Value Hypotheses
- 18. Hypothesis • A testable definite statement • (Null hypothesis: a hypothesis that you’re trying to prove false, e.g. “these results have the same underlying statistics”) • “Our mailing lists are too wide” • “More people will vote for us if we target our advertisements to them”
- 19. Lean Value Trees • Mission: transform the way the US Government does business • Goal: get elected to power • Strategic bet: use data analytics to increase ‘friendly’ voter numbers • Promise of value: larger turnout in ‘friendly’ demographics • Hypotheses and experiments • Promise of value: smaller turnout in ‘unfriendly’ • Strategic bet: use behavioral computing to increase approval ratings • Strategic bet: use propaganda techniques to destroy opposing views • Strategic bet: change the way votes are counted • Goal: reallocate wealth
- 20. Trimming the tree: beliefs • Mission: transform the way the US Government does business • Goal: get elected to power • Strategic bet: use data analytics to increase ‘friendly’ voter numbers • Strategic bet: use behavioral computing to increase approval ratings • Strategic bet: use propaganda techniques to destroy opposing views • Goal: reallocate wealth
- 21. Hypotheses and experiments: totally beliefs!
- 22. = Optimising for other peoples’ beliefs
- 23. And potentially adjusting them…
- 24. What goes wrong?
- 25. Ways to go wrong with a model ❖ Bad inputs ❖ Biased classifications ❖ Missing demographics ❖ Bad models ❖ Unclean inputs, assumptions etc ❖ Lazy interpretations (eg. clicks == interest) ❖ Trained once in a changing world ❖ Willful abuse ❖ gaming with ‘wrong’ data (propaganda etc)
- 26. Ways to go wrong with a human ❖ Manipulation ❖ Bias ❖ Censorship ❖ Privacy violations ❖ Social discrimination ❖ Property right violations ❖ Market power abuses ❖ Cognitive effects ❖ Heteronomy
- 27. Ways humans go wrong • Imperfect recall • Unconscious bias • Confirmation bias • Mental immune systems • Familiarity backfire effect • Memory traces • Emotions = stronger traces
- 30. Human bias in big data
- 31. Not always wrong: drifts and shifts
- 32. What if somebody lies?
- 33. (PS Make sure it isn’t you)
- 34. What could help?
- 35. –Unknown “Ignore everything they say, watch everything they do.”
- 38. –Richard Heuer (“Psychology of Intelligence Analysis”) “Conflicting information of uncertain reliability is endemic to intelligence analysis, as is the need to make rapid judgments on current events even before all the evidence is in” Find People Already Doing This
- 39. Statistics as a coding for ‘belief’
- 41. Comparing multiple hypotheses: ACH • Hypothesis. Create a set of potential hypotheses. • Evidence. List evidence and arguments for each hypothesis. • Diagnostics. List evidence against each hypothesis • Refinement. Review findings so far, find gaps in knowledge • Inconsistency. Estimate relative likelihood of each hypothesis • Sensitivity. Run sensitivity analysis on evidence, arguments • Conclusions and evidence. Present most likely explanation, and reasons why other explanations were rejected.
- 42. ACH in practice
- 43. What could help with changing beliefs?
- 44. Humans as adjustable systems
- 45. Human networks as adjustable systems
- 46. Countering political ‘beliefs’ • Teaching people about disinformation / questioning • Making belief differences visible • Breaking down belief ‘silos’ • Building credibility standards • New belief carriers (e.g. meme wars)
Editor's Notes
- Hi, I’m Sara, and I’m going to talk today about belief. Not religious belief, but what it means to believe something, when accuracy does and doesn’t matter, and why “strongly held belief” is a better concept to aim at than “true”. It’s something I’ve been thinking about for a while, and IMHO it’s a conversation we need to have both as technologists who base many of our work decisions on beliefs, and often work on a belief-based system (the internet), and in the current climate of “fake news”, uncertainty and deliberate disinformation.
- And a smaller one: Aspergers people view the world differently to neurotypical people.
- Image: clipart library http://clipart-library.com/clipart/pT7rMdb8c.htm Philosophy warning: we will probably never totally know our objective truths. We’re probably not in the matrix, but we humans are all systems whose beliefs in the world are completely shaped by our physical senses, and those senses are imperfect. We’ll rarely have complete information either (e.g. there are always outside influences that we can’t see), so what we really have are very strong to much weaker beliefs. There are some beliefs that we accept as truths (e.g. I have a bruise on my leg because I walked into a table today), but mostly we’re basing what we believe on a combination of evidence and personal viewpoint (e.g. “it’s not okay to let people die because they don’t have healthcare”). Try to make both of those as strong as you can.
- Image: wikipedia article on belief networks
- Image: wikipedia article on belief networks
- Images: two news stories about systems that may/do have human bias built into them. Yes, computers have beliefs too. We train our algorithms like we train our children. Except we don’t do that. We install culture, correct ‘errors’ and deviations from that, etc. What’s the algorithmic equivalent of that?
- Image: clipart library http://clipart-library.com/clipart/pT7rMdb8c.htm We spend a lot of our time talking about other people’s beliefs. But we also need to talk about what it means to understand what someone else’s beliefs are. Communication is a form of passing or sharing beliefs. When we talk about belief sharing, we’re talking about communication: the ways in which we pass or share beliefs. And communication is totally and routinely hackable, either in-message through the messages used, messages adapted, subtexts created, or by adapting communications channels and old-school network effects like simple and complex contagion. When we talk about other people’s beliefs, we’re really talking about our beliefs about their beliefs which may be offered through a communication channel (e.g. surveys) which is subject to their beliefs about our beliefs and how we’re likely to react to them. This is important, complicated, and something that I had many fascinating meetings about when I worked on intelligence analysis… which also has a set of techniques that could help (structured analytics and friends).
- This thing is the PACT framework, used to talk about the interactions between responsibility and control in UAV systems. (image from book Human Factors in Simulation and Training https://books.google.com/books?id=cgT56UW6aPUC)
- The text is from an article about Cambridge Analytica’s work on Trump’s election campaign. https://www.theguardian.com/politics/2017/feb/26/robert-mercer-breitbart-war-on-media-steve-bannon-donald-trump-nigel-farage You can also hack computer beliefs. And this - this looks suspiciously like a human-carried computer virus.
- Even when you know better, it’s hard to manage bias. This is the Muller-Lyer illusion. The lines are the same length, but even if you know that, it’s hard to believe it. Psychology of illusion; traces left behind in your mind even when you’ve been told the ‘truth’; car crash lie = unsafe driver problem; pretty much any piece of advertising content ever (“you’re so beautiful if you have this…”) Bias, implicit bias…
- Image: websearch result for “fake news” The internet is also made of beliefs. The internet is made of many things: pages and and comment boxes and ports and protocols and tubes (for a given value of ‘tubes’). But it’s also made of belief: it’s a virtual space that’s only tangentially anchored in reality, and to navigate that virtual space, we all build mental models of who is out there, where they’re coming from, who or what to trust, and how to verify that they are who they say they are, and what they’re saying is true (or untrue but entertaining, or fantasy, or… you get the picture). The internet is (mostly) location-independent. That affects perception: if I say my favorite color is green, then the option to follow me and view the colours I like is only available to a few people; others must either follow my digital traces and my friends’ traces (which can be faked), believe me or decide to hold an open mind until more evidence appears. Location independence makes verification hard…
- image: https://steveblank.com/2015/05/06/build-measure-learn-throw-things-against-the-wall-and-see-if-they-work/ Most people think of lean as a cycle of build-measure-learn, trying the smallest things that we can learn from, pivoting, repeating.
- “Value: Understand clearly what value the customer wants for the product or service. Value Stream: The entire flow of a product's or service's life cycle. In other words, from raw materials, production of the product or service, customer delivery, customer use, and final disposal. Flow: Keep the value stream moving. If it's not moving, it's creating waste and less value for the customer. Pull: Do not make anything until the customer orders it. Perfection: Systematically and continuously remove root causes of poor quality from production processes.”
- In Lean Enterprise, metrics exist as part of a Lean Value Tree (LVT). At the top of this tree is the organisation’s vision for itself. It says who we are and what our purpose is, is stable and owned by executive level. Then: goals that support that vision. These describe desired outcomes, are stable and measurable. Then: *strategic bets on those goals. *These are different ways that we could satisfy a goal, are stable in the medium term, have metrics and are owned below the executive level. Then: Promises of value (value-driven initiatives against those bets). These are component parts of each strategic bet, are stable in the medium-term and are owned at the product level. Then: hypotheses and experiments to support those initiatives. These are measurable, testable hypotheses, with experiments to test them (hypothesis creation and testing is one of the big things that data scientists do). These are very short term, often quick and dirty ways to learn about the problem space and potential solutions as quickly as possible, and are owned by the team.
- You never have the resources to do everything on your tree. Which means allocating those resources to the ‘important’ things on it. Which means knowing how big the market in each branch is, and how big the potential gains.
- Image: https://barryoreilly.com/2013/10/21/how-to-implement-hypothesis-driven-development/ Hypotheses and experiments support those initiatives. These are measurable, testable hypotheses, with experiments to test them (hypothesis creation and testing is one of the big things that data scientists do). These are very short term, often quick and dirty ways to learn about the problem space and potential solutions as quickly as possible, and are owned by the team. And this is all about learning… Ahem… An astronomer, a physicist and a mathematician are on a train in Scotland. The astronomer looks out of the window, sees a black sheep standing in a field, and remarks, "How odd. All the sheep in Scotland are black!" "No, no, no!" says the physicist. "Only some Scottish sheep are black." The mathematician rolls his eyes at his companions' muddled thinking and says, "In Scotland, there is at least one sheep, at least one side of which appears to be black from here some of the time.
- Image: clipart library http://clipart-library.com/clipart/pT7rMdb8c.htm
- Image: http://www.rosebt.com/blog/descriptive-diagnostic-predictive-prescriptive-analytics
- Biased data in; either biased connections between cause and effect, missing demographics, misleading interpretations of inputs (e.g. number of people with phones). Algorithms that don’t explain the data. Misuse of algorithms and data (including inappropriate reuse, e.g. survey results being used to target jewish people in WW2). NB misinformation vs disinformation: the former is usually accidenta, the latter usually deliberate.
- These are Lazar’s categories of algorithm risk… heteronomy = individuals no longer have agency over the algorithms influencing them.
- Confirmation bias = you believe more in things that fit your existing beliefs than things that don’t Memory traces: Gilbert’s work on the way we hold beliefs as true whilst we examine whether they’re false http://www.danielgilbert.com/Gillbert%20(How%20Mental%20Systems%20Believe).PDF Immune systems include doubling down Backfire: repeated = more true Emotions include fear, surprise etc.
- Image: http://users.cs.cf.ac.uk/Dave.Marshall/AI2/node145.html The frame problem says that you can’t measure the whole world, so there are always parts of your system potentially affected by things you haven’t measured. We spent a lot of the early 80s talking about children’s building blocks (Winston’s blocks world) and things like the “naughty baby problem”. Which was basically that sometime during our careful deliberation, a naughty baby could come into the frame and move all the block orders around. http://users.cs.cf.ac.uk/Dave.Marshall/AI2/node145.html
- Image: https://www.quora.com/Whats-the-difference-between-a-cow-a-bull-a-buffalo-and-an-ox Uncertainty, missing data, incorrect understanding of the question, fat-fingered answers… remember to score both the individual and the answer separately!
- Image: news article. A long time ago, if you wanted to study AI, you inevitably ended up reading up on cognitive psychology. Big data didn’t exist (well it did, but it was mainly things like the real-time sonar signal processing systems that people like me cut their parallel processing teeth on), the Cyc system was a Sisyphean attempt to gather enough ‘facts’ to make a large expert system, and most of our efforts revolved around how to make sense of the world given a limited number of ‘facts’ about it (and also how to make sense of the world, given a limited number of uncertain data points and connections between them). And then the internet, and wikipedia, and internet 2.0 where we all became content providers, and suddenly we had so much data that we could just point machine learning algorithms at it, and start finding useful patterns. We relied on two things: having a lot of data, and not needing to be specific: all we cared about was that, on average, the patterns worked. So we could sell more goods because the patterns that we used to target the types of goods to the types of people most likely to buy them, with the types of prompts that they were most likely to respond to. But we forgot the third thing: that almost all data collection has human bias in it - either in the collection, the models used or the assumptions made when using them. Prison recidivism scores etc.
- Image: http://www.thwink.org/sustain/glossary/KuhnCycle.htm Kuhn’s structure of scientific revolutions. * http://theconversation.com/thomas-kuhns-the-structure-of-scientific-revolutions-50-years-on-6586
- IMage: 2010 red cross t-shirt. The other “V”: Veracity. Or seven Vs: https://datafloq.com/read/3vs-sufficient-describe-big-data/166 Ihub 3 Vs of crowdsourcing: pects of crowdsourcing – what we’re calling the ‘3Vs of Crowdsourcing’ – with the following objectives: 1. Viability: In what situations, or during which events, is crowdsourcing a viable venture like- ly to offer worthwhile results and outcomes? We aim to understand and outline the features that make a particular election in a particular country viable for crowdsourcing. 2. Validity: Can crowdsourced information offer a true reflection of the reality on the ground? We aim to identify that conditions that might make real-time data validation possible. 3. Verification: Is there a way in which we can verify that the information provided through crowdsourcing is indeed valid? If so, can the verification process be automated? If so, we aim to devise a tool for doing so.
- Image: Fox news screenshot This is misinformation, not disinformation. The numbers are correct, but the impression people get from the graphs (a large difference in values) is wrong.
- My favorite quote… Be aware of what people are saying, but also watch their actions. Follow the money, and follow the data; everything leaves a trace somewhere if you know how to look for it (again, something that perhaps is best done as a group).
- Image: Findyr screenshot Verification means going there. For most of us, verification is something we might do up front, but rarely do as a continuing practice. Which, apart from making people easy to phish, also makes us vulnerable to deliberate misinformation. We want to believe stuff? We need to do the leg-work of cross-checking that the source is real (crisismappers had a bunch of techniques for this, including checking how someone’s social media profile had grown, looking at activity patterns), finding alternate sources, getting someone to physically go look at something and send photos (groups like findyr still do this). Validation: is this a true representation of what’s happening?
- Image: crisis mapping deployment over Afgooye corridor, Somalia.
- Image: screenshot from SaraTerp lecture on AB testing and significance. Statistics makes me uncomfortable. It makes a lot of people uncomfortable. My personal theory is because statistics tries to describe uncertainty, and there’s that niggling feeling always that there’s something else we haven’t quite considered when we use them to do this. Images: asking if two distributions are statistically significant means you’re dealing with and coding for uncertainty.
- Image: Screenshot from Heuer book “Structured Analytic Techniques”.
- Hypothesis. Create a set of potential hypotheses. This is similar to the hypothesis generation used in hypothesis-driven development (HDD), but generally has graver potential consequences than how a system's usrs are likely to respond to a new website, and is generally encouraged to be wilder and more creative (DHCP theory might come in useful for that). Evidence. List evidence and arguments for each hypothesis. Diagnostics. List evidence against each hypothesis; use that evidence to compare the relatively llikelihood of different hypotheses. Refinement. Review findings so far, find gaps in knowledge, collect more evidence (especially evidence that can remove hypotheses). Inconsistency. Estimate the relative likelihood of each hypothesis; remove the weakest hypotheses. Sensitivity. Run sensitivity analysis on evidence and arguments. Conclusions and evidence. Present most likely explanation, and reasons why other explanations were rejected.
- Image: Screenshot from Heuer book “Structured Analytic Techniques”. The point of ACH is refutation: you’re trying to remove hypotheses from the list, not support the ‘strongest’ one.
- Image: clipart library http://clipart-library.com/clipart/pT7rMdb8c.htm Yes, we’re all individuals…
- Image: John Boyd’s original OODA loop. There are many pressure points in this loop that can be used. But do this gently: go too quickly with belief change, and you get shut down by cognitive dissonance.
- Image: Screenshot of Sara’s Facebook friends. Can borrow ideas from biostatistics, and create idea ‘infections’ across groups.