Successfully reported this slideshow.

The Crisis of Self Sovereignty in The Age of Surveillance Capitalism



Upcoming SlideShare
SCIVE 2010
SCIVE 2010
Loading in …3
1 of 50
1 of 50

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

The Crisis of Self Sovereignty in The Age of Surveillance Capitalism

  1. 1. The Crisis of Self Sovereignty - With A Brief Overview of “The Age of Surveillance Capitalism” Jonathan Kim ( 2019.4.25
  2. 2. It’s Time to Panic about Privacy
  3. 3. What is Privacy? Privacy is a fundamental right, essential to autonomy and the protection of human dignity, serving as the foundation upon which many other human rights are built. <Source:>
  4. 4. Debates on Privacy ü Companies and governments are gaining newpowers to followpeople across the internet and around the world, and even to peer into their genomes. ü The benefitsofsuch advances have been apparent for years; the costs are nowbecoming clearer. ü The boundariesof privacy are in dispute, and its future isin doubt. Citizens, politicians and business leaders are asking if societies are making the wisest tradeoffs. <Source:>
  5. 5. Privacy is Too Big to Understand <Source:> §“Privacy” is an impoverished word — far too small a word to describe what we talk about when we talk about the mining, transmission, storing, buying,selling, use and misuse ofour personal information. §You are losing controlover your life.When technologygovernsso many aspects of our lives — and when that technologyis powered by the exploitation of ourdata — privacy isn’t just about knowing your secrets, it’s about autonomy. §Privacy is really about being able to define for ourselveswho we are forthe world and on our own terms. §“Maintaining privacy will be integral to the internet’sfuture, if only because consumers need to feelsafe enoughto participate.”
  6. 6. How Capitalism Betrayed Privacy <Source:> §The forcesof wealth creation no longer favor the expansion of privacy but work to undermine it. - Therise of “attention merchants” and “surveillancecapitalism”; thecommodificationofourpersonal data by techgiants likeFacebook,Googleand theirimitators. §We face a future in which active surveillance is such a routine part of business that for most people it is nearly inescapable. We are on the road back to serfdom. §Many employers also nowconstantly watch their employees. There is good reason to believe that, if nothing is done,gratuitous surveillance will be built into nearly every business and business model. §Those who want privacy should support and reward the companies who respect it. The economicsof privacy would change if enough consumers bought fromcompanies that don’t spy on us and whose products actually help people avoid an unwanted gaze.
  7. 7. Facebook Under Fire <Source:> §Cambridge Analytica illicitly procured the data of50 million Facebook users — without their knowledge or consent — and then enlisted that to informvoter- targeting strategies forDonald Trump’s presidential campaign. §Both Facebook and CAclaim they were duped by the researcher who originally harvested the data, who used an innocuous-seemingpersonality quiz in 2013 to access infoon friends of people who used the app. à Facebook is expecting to pay as much as $5bn to the US Federal Trade Commission.(2019.4.25)
  8. 8. Facebook is "Surveillance Machine" <Source:> You don't use Facebook. Facebook uses you. “Facebook is in the surveillance business. Facebook, in fact, is the biggest surveillance-based enterprise in the history of mankind. It knows far, far more about youthan the most intrusive government has ever known about its citizens.” “Note that the company’s knowledge about its users isn’t used merely to target ads but to shape the flowof news to them.” When the service is free, you're not the customer.You're the product.
  9. 9. The ‘Data Labor Union’ <Source:> Facebook usersunite! 'Data Labour Union'launches in Netherlands §The Data Union is going to demand the transparency and readability of algorithms used corporationsand governments. §The Data Union is going to help members demand their data rights from corporations and governments. §The Data Union will focus attention on the need for the Data Union. §The Data Union will inform its members about the newest tools to protect your data and develop these ourselves,to end our dependency.
  10. 10. Facebook’s Privacy Policies <Source:> ü Facebook founderMark Zuckerberg told that if he were to create Facebook again today, user information would by default be public, not private as it was for years until the company changed dramatically in December. (2010. 1. 9) ü Facebook’s mission is to make the world more open and connected. ü The main way we do this is by giving people the tools to map out their relationships with the people and things they care about. We call this map the graph. (2013. 1. 15) The digital equivalent of a town square à The digital equivalent of the living room - Private interactions - Encryption - ReducingPermanence - Safety - Interoperability - Securedata storage (2019. 3.7) “The Age of Privacyis Over” “GraphAPI/Graph Search” “A Privacy-FocusedVision”
  11. 11. Four Essential Privacy Principles(by Tim Cook) Ø Companies should challenge themselves to de-identify customer data or not collect that data in the first place. Ø Users should always knowwhat data is being collected from them and what it’s being collected for.This is the only way to empower users to decide what collection is legitimate and what isn’t. Ø Companies should recognizethat data belongs to users and we should make it easy for people to get acopy of their personal data, as well as correct and delete it. Ø Everyonehas a right to the security of their data. Security is at the heart ofall data privacy and privacy rights. <Source:>
  12. 12. “The Age of Surveillance Capitalism”
  13. 13. The Foundations of Surveillance Capitalism
  14. 14. Sur-veil-lance Cap-i-tal-ism, n. 1. Anew economicorder that claims human experienceas free raw material for hidden commercial practices ofextraction, prediction, and sales; 2. Aparasitic economiclogic in which the productionof goods and services is subordinated to a newglobal architecture ofbehavioral modification; 3. Arogue mutation of capitalism marked by concentrations of wealth, knowledge, and power unprecedented in human history; 4. The foundationalframework ofa surveillance economy; 5. The origin ofa new instrumentarian power that asserts dominance over society and presents startling challenges to market democracy; 6. Amovement that aims to impose a new collective order based on total certainty; 7. An expropriation of critical human rights that is best understoodas a coup from above: an overthrowofthe people’s sovereignty. <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019>
  15. 15. Behavioral Surplus <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Behavioral Data Human Experience appliedto product or service improvement a proprietary behavioralsurplus Although some ofbehavioral data are applied to product or service improvement, the rest are fed into advanced manufacturing processes known as“machine intelligence,” and fabricated into prediction products that anticipate what you will do.
  16. 16. Data Extraction and Analysis <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Data The raw material necessary for surveillance capitalism’s novel manufacturing processes Extraction The social relations and material infrastructure with which the firm asserts authority over those raw materials to achieve economiesof scale in its raw-material supply operations Analysis The complex ofhighly specialized computational systems that is generally referred to as “machine intelligence”
  17. 17. Behavioral Value Reinvestment Cycle <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> ü Users provides the raw material in the form ofbehavioral data, and those data are harvested to improve speed, accuracy, and relevance and to help build ancillary productssuch as translation. ü In the Behavioral Value Reinvestment Cycle, all the behavioral data are reinvested in the improvement ofthe product or service.
  18. 18. Discovery of Behavior Surplus <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> ü Surveillance Capitalism begins with the discovery of behavioral surplus. More behavioral data are rendered than required for service improvement. ü This surplus feeds machine intelligence that fabricates predictions ofuser behavior.These Productsare sold to business customersin newbehavioral futures markets. ü The Behavioral Value Reinvestment Cycle is subordinated to this new logic.
  19. 19. The Logic and Operation of Surveillance Capitalism <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Logic The Means of Production The Products The Marketplace The translation of behavioral surplus from outside to inside the market finally enabled Google to convert investment into revenue. Google’s machine intelligence capabilities feed on behavioral surplus, and the more surplus they consume, the more accurate the prediction products that result. Machine intelligence processesbehavioral surplus into prediction products designed to forecast what we will feel, think, and do. Prediction products are sold into a new kind ofmarket that trades exclusively in future behavior. Surveillance capitalism’s profits derive from behavioral futuresmarkets.
  20. 20. The Dynamic of Behavioral Surplus Accumulation <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> ü Surveillance Capitalism master motionis the accumulation of newsources of behavioral surplus with more predictive power. ü The goalis predictionscomparable to guaranteed outcomes in real-life behavior. ü Extraction begins online,but the prediction imperative increases the momentum,driving extraction toward new sourcesin the real world.
  21. 21. Accumulation by Dispossession <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Accumulationby dispossession is a conceptpresented by the Marxist geographer David Harvey, which defines the neoliberal capitalist policies in many western nations as resulting in a centralization ofwealth and power in the hands of a fewby dispossessing the public and private entitiesof their wealth or land. Human experience could be extracted at no extra cost online and at very low cost out in the real world. Once extracted, it is rendered as behavioral data, producing a surplus that forms the basis ofa wholly newclass ofmarket exchange. Surveillance capitalism originatesin this act of digital dispossession, brought to life by the impatience of over-accumulated investment.This isthe lever that moved Google’s world and shifted it toward profit.
  22. 22. The Dispossession Cycle <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Incursion Habituation Adaptation Redirection The incursion is when dispossession operations rely ontheir virtual capabilities to kidnap behavioral surplus from the nonmarket spaces of everyday life where it lives. The incursion itself, once unthinkable, slowly worms its way into the ordinary. Worse still, it gradually comes to seem inevitable. New dependencies develop. when Google is forced to alter its practices, its engineers produce superficial but tactically effective adaptations that satisfy the immediate demands of government authorities and public opinion. In afinal stage the corporation regroups to cultivate new rhetoric, methods, and design elements that redirect contested supply operations just enough so that they appear to be compliant with social/legal demands.
  23. 23. Scrapes Your Site Activity <Source:> “People will give up their privacy to get something they want.” (Steve Thornhill)
  24. 24. Who Knows? Who Decides? Who Decides Who Decides? <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Who Knows? Who Decides? Who Decides Who Decides? • ‘This is aquestion about the distribution of knowledge and whether one is included or excluded from the opportunityto learn. • This is a question about power. What is the sourceof power that undergirds the authority to share or withhold knowledge? • This is a question about authority : which people, institutions, or processesdetermine who is included in learning, what they are able to learn, and how they are able to act ontheir knowledge.
  25. 25. Merely Human Natural Resources <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> In this future we are exiles from ourown behavior, denied access to or controlover knowledge derived from our experience. Knowledge, authority, and power rest with surveillance capital, for which we are merely “human natural resources.” The commodificationof behavior under the conditionsof Surveillance capitalism pivotsustoward a societal future in which an exclusive division oflearning is protected by secrecy,indecipherability, and expertise.
  26. 26. A Fight over Surveillance Capitalism <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> ü Let it be an insistence that raw surveillance capitalism is as much a threat to society as it is to capitalism itself. ü This is not a technical undertaking, not a program for advanced encryption,improved data anonymity,or data ownership. ü Such strategies only acknowledge the inevitability ofcommercial surveillance. ü Surveillance capitalism dependson the social, and it is only in and through collective social action that the larger promise of an information capitalism aligned with a flourishing third modernitycan be reclaimed. ‘If there is to be a fight, let it be a fight over capitalism.
  27. 27. The Advances of Surveillance Capitalism
  28. 28. Economic Imperative <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Extraction Imperative Prediction Imperative l The first wave of prediction productsenabled targeted online advertising. These productsdepended upon surplus derived at scale from the internet. l I have summarized the competitive forcesthat drive the need for surplus at scale as the “extraction imperative.” l The next threshold was definedby the quality of prediction products. l In the race for higher degrees of certainty, it became clear that the best predictionswould have to approximate observation. l The prediction imperative is the expression ofthese competitive forces. what forms of surplus enable the fabrication of predictionproducts that most reliably foretell the future?
  29. 29. Prediction Imperative <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Economiesof Scale Economiesof Scope Economiesof Action • Extension ofExtraction • Depth
  30. 30. Rendition : From Experience to Data <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> why is our experience rendered as behavioraldata in the first place? Rendition: the specific operations that target the gap between experience and data on a mission to transform the one into the other Rendition describes the concrete operationalpractices through which dispossession is accomplished, as human experienceis claimed as raw material for datafication and all that follows, from manufacturing to sales.
  31. 31. Big Data and Big Profits <Source:> §It is the data—which we give away for free—that is the coreof the vast profits of the world’smost valuable companies. - Facebook, pays only 1% ofits value to its workers(programmers), because the rest of its work it getsfree from us - Walmart pays 40% of its value in wages. §Afewvisionaries have been trying to introduce a radical new concept. —“Data as Labor(DaL)”— that will make for a far more economically fair society in the digital age.
  32. 32. Data as Labor <Source:> ü Imagine a world in which your personal data, currently hooveredup by tech companiesand repurposed for their profit, were honoredas your dignified work and compensated as such. ü Data as Labor(DaL) : Because data suppliers are not properly rewarded for their digital contributions,they lack the incentive or freedomto contributethe high-quality data that would most empower technologyor develop their personal capacities to maximize their earnings and contributions to the digital economy.
  33. 33. Economies of Action <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> “The new power is action.” The prediction imperative phase represents the completion of the newmeans of behavior modification, a decisive and necessary evolution of the surveillance capitalist “means ofproduction” toward a more complex,iterative, and muscular operational system. Under surveillance capitalism the objectives and operations of automated behavioral modification are designed and controlled by companies to meet their own revenueand growth objectives.
  34. 34. Three Key Approaches to Economies of Action <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Tuning Herding Conditioning Nudge any aspect of a choice architecture that alters people’sbehavior in a predictable way Orchestration of the human situation foreclosingaction alternatives and thus movingbehavioralong a path of heightenedprobability Reinforcement “Behaviormodification” or “behavioral engineering,” in whichbehavioris continuously shaped to amplify some actions at the expense of others
  35. 35. Behavioral Economics for Surveillance Capitalism Extrinsic Motivation Intrinsic Motivation 《ChoiceArchitect》based on ‘LibertarianPaternalism’ Libertarianpaternalismis the idea that it is both possible and legitimate for private and public institutions to affect behavior while also respecting freedom ofchoice, as well as the implementation of that idea. <Source:>
  36. 36. Behavioral Modification <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> Behavioral modification relies upon a variety of machine processes, techniques, and tactics (tuning, herding,conditioning) to shape individual, group, and population behavior in ways that continuouslyimprove their approximation to guaranteed outcomes. Surveillance capitalists’ interests have shifted from using automated machine processes to know about your behavior to using machine processes to shape yourbehavior according to their interests. This decade-and-a-half trajectory has taken us from automating information flows about youto automating you.
  37. 37. Instrumentarianismas a New Species of Power <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> What is Instrumentarianism? The instrumentation and instrumentalization of behavior for the purposes of modification, prediction,monetization, and control ü Totalitarianism was apolitical projectthat converged with economics to overwhelm society. ü Instrumentarianism is amarket project that convergeswith the digital to achieve its own unique brand ofsocial domination. ü Thanks to Big Other’s capabilities, instrumentarian power aims for a conditionof certainty without terror in the form of“guaranteed outcomes.” *Big Other:It is thesensate, computational,connectedpuppetthatrenders,monitors, computes, and modifies human behavior.
  38. 38. Two Species of Power : Totalitarianism vs. Instrumentarianism <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019>
  39. 39. A Market Project of Total Certainty <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> ü Totalitarianism was atransformation of the state into a project oftotal possession. ü Instrumentarianism and Big Other signal the transformation ofthe market into a project of total certainty, an undertaking that isunimaginable outside the digital milieu, but also unimaginable outside the logic of accumulation that is surveillance capitalism. ü This new power is the spawn of an unprecedentedconvergence:the surveillance and actuation capabilities of Big Other in combination with the discovery and monetization of behavioral surplus. ü we can imagine economicprinciples that instrumentalize and controlhuman experience to systematically and predictably shape behavior toward others’ profitable ends. First, machinesare not individuals, andwe should be more like machines.
  40. 40. Crisis of Self-Sovereignty
  41. 41. What is Self-Sovereignty? <Source:> Sovereigntyis the full right and power of a governingbody over itself, without any interference fromoutside sources or bodies. sovereignty is a substantive term designating supreme authority over some polity. ü Self(orindividual)-Sovereigntyis the concept of propertyin one's own person, expressed as the natural right of aperson to have bodily integrity and be the exclusive controller ofone's own bodyand life. ü Self-Sovereigntyis a central idea in several political philosophies that emphasize individualism, such as liberalism and anarchism.
  42. 42. What is Self-Determination? <Source:> ü The right of a people to self-determination is a cardinal principle in modern international law. ü It states that people, based on respect for the principle of equal rights and fair equality of opportunity,have the right to freely choose their sovereignty and international political status with no interference. ü By extension, the term self-determination has come to mean the freechoice of one's own acts without external compulsion.
  43. 43. The End of Self-Determination surveillance capitalists declare their right to modify others’behavior for profit according to methodsthat bypass human awareness, individual decision rights, and the entire complex of self-regulatory processes (autonomy and self-determination). §Human consciousness itself is a threat to surveillance revenues,as awareness endangers the larger project ofbehavior modification. §Philosophers recognize“self-regulation,” “self-determination,” and “autonomy” as “freedom of will.” §The competitive necessity of economiesof action means that surveillance capitalists must use all means available to supplant autonomousaction with heteronomousaction. <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019>
  44. 44. Solutions to the Problem of Self-Sovereignty
  45. 45. The Right to Sanctuary § A sanctuary is a sacred place. By extension the term has come to be used forany place of safety. This secondary use can be categorizedinto human sanctuary, a safe place for humans, such as a political sanctuary. § The sanctuary privilege has stood as an antidote to power since the beginning of the human story.There was an exit fromtotalizing power, and that exit was the entrance to a sanctuary in the formof a city, a community,or a temple. <Source:ShoshanaSuboff,<TheAgeofSurveillance Capitalism> PublicAffairs,2019> ü Big Other outruns society and law in a self-authorizeddestruction of the right to sanctuary as it overwhelms considerations of justice with its tactical mastery of shock. ü “No exit” is the necessary conditionfor Big Other to flourish. -That is the tides ofbehavioralsurplus and their transformation into revenue, the certainty that willmeet everymarket player withguaranteed outcomes, the actuation and modificationthat quietlydrains the willto will.
  46. 46. Obfuscation : as a Strategy for Privacy Protection Obfuscation is the deliberateaddition of ambiguous,confusing, or misleadinginformation to interfere with surveillanceand data collection. <Source:Finn Brunton, Helen Nissenbaum, <Obfuscation:AUser'sGuideforPrivacy andProtest> TheMITPress,2015> Operation Vula Uploads to leak sites TrackMeNot Chaf Identical confederates and objects False tells Excessive documentation Group identity Shuffling SIM cards Tor relays Babble tapes CacheCloak
  47. 47. Privacy Itself is a Solution to Societal Challenges <Source:Finn Brunton, Helen Nissenbaum, <Obfuscation:AUser'sGuideforPrivacy andProtest> TheMITPress,2015> Privacy does not mean stopping the flow of data; It meanschannelingit wisely andjustly to serve societal ends andvalues and the individualswho are its subjects, particularlythe vulnerable andthe disadvantaged.
  48. 48. Privacy Technologies for our Struggle for Self-Sorvereignty Decentralized Identifier Zero Knowledge Proof MimbleWimble Secure MPC Self-Sovereigntyis a Guide to Privacy … It’s time for “Privacy Revolution” with the Blockchain Technology!
  49. 49. “What is at stake here is the human expectation of sovereignty over one’s own life and authorship of one’s own experience.” - Shoshana Zuboff,<TheAge of SurveillanceCapitalism>