Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Альфонсо де Грегорио (script) - Уязвимости и сопутствующие им этические вопросы - кодекс этики для частного сектора


Published on

Уязвимость нулевого дня - недостатки программного обеспечения, которые известны некоторым,кто мог бы уменьшить их конкретные негативные последствия - приобретают заметную роль в современной разведке, национальной безопасности и правоохранительных операциях. В то же время, отсутствие прозрачности и подотчетности в их торговле и адаптаци, их возможная чрезмерная эксплуатации или злоупотребление, скрытый конфликт интересов со стороны субъектов обращения с ними, а также их потенциальный двойной эффект могут представлять социальные риски или приводят к нарушению прав человека. Если оставить без внимания эти проблемы связанные с использованием 0-day, то это ставит под сомнение законность уязвимостей нулевого дня в качестве инструментов реализации национальных операций по обеспечению безопасности и правоохранительных органов и приводят к явному уменьшению пользы, чтобы их адекватно применяли для целей судебной системы, обороны и разведки. Эта работа исследует то, что частный сектор участвует в торговле уязвимости нулевого дня может сделать так, чтобы было обеспечено соблюдение прав человека и доброкачественное и полезное использования обществом этих возможностей. После рассмотрения того, что может пойти не так в приобретении уязвимости нулевого дня, статья вносит свой вклад в первый кодекс этики, ориентированный на торговлю информации об уязвимостях, в которой автор излагает шесть принципов и восемь соответствующих этических норм, направленных соответственно на руководство и на регулирование проведения этого бизнеса.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Альфонсо де Грегорио (script) - Уязвимости и сопутствующие им этические вопросы - кодекс этики для частного сектора

  1. 1. HackIT 2016, 7 October, Kharkiv, Ukraine — Nominal delivery draft Vulnerabilities and Their Surrounding Ethical Questions: A Code of Ethics for the Private Sector Opening Good afternoon, everybody. Thank you for the invitation to speak and thank you for coming. I am so excited to be in Kharkiv with you for the annual HackIT conference. Introduction Over the last year, I presented to multiple audiences around the globe the results on a study on extortion and cooperation in the market for vulnerabilities through the lens of game-theory and listened to the remarks and comments ensued. Most notably, a colleague in Japan asked me what I thought about the ethical questions related to the trading of vulnerability information and if they were meaningless to me. While they are far from being meaningless to me, at that time I sidestepped providing to this friend a fully satisfactory answer. In fact, I was not directly involved into this business and my focus was on the economics of vulnerability markets. Hence, I suggested that there were different sensibilities around the world with regard to the applicable ethical dilemmas and that if we worked towards aligning the incentives in the industry we would have, in turn, lead to business practices of higher ethicality. Later, and after careful consideration, I decided to go from theory to practice and I entered this space. But in so doing, the ethical questions related to the trade of vulnerability information could not be postponed any further. In fact, zero-day vulnerabilities — weaknesses in software or hardware that are unknown to the parties who can mitigate their specific negative effects — are gaining a prominent role in modern-day intelligence, national-security, and low-enforcement operations. At the same time, the lack of transparency and accountability in their trade and adoption, their possible overexploitation or abuse, the latent conflict of interests by entities handling them, and their potential double effect may pose societal risks or lead to the breach of human rights. If left unaddressed, these usage-related challenges call into question the legitimacy of zero-day vulnerabilities as enablers of national security and law enforcement operations and erode the benefits that their proportionate use have for the judiciary, defense, and intelligence purposes. With your help, what I want to do today is to briefly review the vulnerability supply chain, its main actors, and their surrounding ethical questions. After setting the context, I will share with you how I decided to approach these questions in my occupation and report on the ethical principles and standards I set forth in Code of Business Ethics that I adopted in my day-to-day business operations.
  2. 2. In doing so, it is not my intention neither to lecture anybody about how we are expected to behave, nor to provide definitive answers to pressing challenges. I will be happy if will have asked more questions than given answers, or if I will have contributed some meaningful ethical principles and standards to build upon. So, let’s get started. Dramatis Personae and the Vulnerability Supply Chain How many of you are familiar with the Philip K. Dick’s 1968 novel ‘Do Androids Dream of Electric Sheep’? Anyone? And how many of you are familiar with the Ridley Scott’s 1982 film ‘Blade Runner’? The story I want to tell you today is a story that finds inspiration in a science fiction novel, and, as we are about to overtake the period in which the novel is set, informs us about the present we are living in, about some of the ethical questions it raises, and how I decided to approach them in my occupation. Shortly I will introduce you to the dramatis personae of our story. Their story is the story of the supply chain of vulnerabilities, composed, among others, of: software makers, creating vulnerabilities during the products’ development lifecycle; vulnerability researchers, finding existing vulnerabilities and creating exploits to take-advantage of them; brokers, trading vulnerability intelligence or zero-day exploits; and, organisations using, or misusing, the resulting capabilities. Their story is also our story. And it is my contention that wherever we look into the vulnerability supply-chain, each and every industry actor face its respective ethical issues related to the knowledge of zero-day vulnerabilities. But let’s proceed in order and let’s try to draw a parallel between the fictional A.D. 2019 depicted in Blade Runner and the present we are living in. The A.D. 2019 depicted in Blade Runner “[i]t is a time of societal decline, where technology has polluted the earth and seized control of the cities.” [1] Enormous power is in the hands of corporations. Los Angeles “appears to have turned into one of Jeremy Bentham's Panopticons, whereby one cannot tell if one is being watched, but it is possible that one is being watched at all times, which means extreme caution must be exercised at all times.” [2] “The […] roving spotlight, present throughout the film, suggest constant surveillance.” [2] And the replicants need to stay ‘in character’ even when alone. This is what Harvard psychologist Shoshana Zuboff called in 1988 “anticipatory conformity”. [3] Interestingly, though, surveillance is not the only point of contact between fiction and reality. Let’s consider pollution. Vulnerabilities are like pollutants: the private up-front cost of insecure software — as it happens for the cost of waste dumping, for instance — is near zero to most, but the social cost of it is quite high, almost unbearable. That is to say that, as for today, software security is an (negative) externality, because the market does not provide significant or compelling incentives for developing secure
  3. 3. software. As a result, software makers are practising unrestrained vulnerability dumping onto downstream market participants. [4] This comes as no surprise. Building security into our products is in stark contrast with the "ship, then test" paradigm [5] and the "don't worry, be crappy" mantra [6], advocated by entrepreneurs innovating the most. In the absence of policy discouraging the dumping of vulnerabilities onto the downstream market participants, defenders are too busy mopping the floor to turn off the faucet. [7] That is to say that “[t]he market participants in doing patching, filtering, and protecting their systems will do little to stop the dump of pollution that occurs upstream.” [7] Hence, “[p]oorly written, insecure software is no longer a technology issue; it is a public policy issue. Software vulnerabilities leave consumers, businesses, national infrastructures, government and the military susceptible to […] attacks.” [7] Even worse, there is no such thing as bug-free software. Every software of non trivial complexity contains bugs. This means we will need to patch the software we entrust our business. Yet, patching plays as a perverse incentive, allowing software manufacturers to optimize market and legal protections by re-negotiating contract terms buyers could not negotiate in the first place. This is how, every time a vulnerability comes out, we find ourselves signing a new licensing agreement. But this gives the manufacturers the ability to re-negotiate contract terms we could not negotiate in the first place. So we have a choice. We have a take it or leave it choice. I can either accept the license agreement, so I can keep patching the vulnerabilities affecting the software I rely on. Or, not take it and risk exploitation. It's a deal I can't refuse — and neither can you. All of which is to say, that corporate power is as much a key-feature of Blade Runner as much as a signature of our industry. Meet Rick Deckard. Deckard is a selfish and self-involved specialist plainclothes ex police officer, or an officially sanctioned bounty hunter, who goes after renegade androids, also known as “andys”. Here surely, is where the parallel breaks down. Officially or tacitly sanctioned, the bounty hunters in the information security industry are certainly not faced with “retiring” six escaped Nexus-6 androids. We are after the pollutants dumped by the software makers upstream, and we write code to constructively prove their risks. Organisations, both in the government and in the industry sector, demand the findings of these researches, to enable their security strategies. And various types of marketplaces compete with each other in order to win the preference of bounty hunters towards their vulnerability disclosure policy of choice. Ethical Questions As I said, wherever we turn our attention in the vulnerabilities supply chain, from software vendors, to vulnerability researchers, to government agencies, all industry actors face their respective ethical issues related to the  vulnerabilities affecting networked devices and the knowledge of their existence.
  4. 4. Therefore, I want to ask you: Who holds the moral low ground: the ruthless malefactors profiting from yet another remote code execution vulnerability, or the vendors practicing unrestrained vulnerability dumping onto the downstream market participants? Who are the ones that exploit us the most: the foreign security services taking total control of our mobile handsets, or the vendors using patching to optimize market and legal protections by re-negotiating contract terms users could not negotiate in the first place and from which the users have no satisfactory way to escape? If our governments introduce trade controls to administer the export of intrusion software, should we demand software manufacturers to internalise the cost of the insecure software that we import into our lives, for reasons of symmetry? Should we make them liable for the defects and flaws that allow the intrusion in the first place? With incomplete knowledge about the real-world security of systems we entrust our business, is it ethical to refrain us from hunting vulnerabilities or prevent others from doing likewise? And, what should do a security researcher with the vulnerabilities when they get found? Is full disclosure an acceptable course of action? Does full disclosure becomes more acceptable if the affected vendor ignores the vulnerabilities that were reported responsibly or fails to provide a timely patch? Does coordinated vulnerability disclosure provide a more ethically sound path to be taken? Does the same path remains morally preferable if one of the parties, who receives the vulnerability information from the Coordinator prior to its public disclosure, decides to use it to exploit vulnerable entities? Are bug bounty programs exploiting bounty hunters? What are fair terms? Should bug hunters pretend to get paid if the other party has not invited them to do their work? What government security agencies should do with vulnerabilities: should they exploit them or should they let everybody else mitigate them, in the way they already do? Should they take advantage of those vulnerabilities to benefit a limited number of stakeholders, or should they disclose them to all affected constituents? Has the power inequity in the vulnerability equation to be balanced? With entities affected by vulnerabilities spread all around the world, how to inform the public? With vendors threatening legal action and supported by their significant financial resources, how to protect the security researchers? With our society growing more data intensive, how to oversee not only material and technology but also knowledge? “How do the attempts to strike a balance between scientific openness and national security […] redefine science-security relations? How does scientific knowledge become subject to security governance? And how does this dynamic affect the links among scientific knowledge, security expertise and political decision?” Can we regard hacking to be an ethical practice and condemn, at the same time, the trade of capabilities enabling this practice as immoral? Today, I want to explore with you what the private sector involved in the trade of zero-day vulnerabilities can do to ensure the respect human rights and the benign and societally beneficial use of those capabilities. After reviewing what can go wrong in the acquisition
  5. 5. of zero-day vulnerabilities, I want to offer up to your comments and criticism the first code of ethics focused on the trade of vulnerability information, where the I set forth six principles and eight corresponding ethical standards aimed respectively at guiding and regulating the conduct of this business. Zero-days and their Non-Zero Societal Risks Zero-days vulnerabilities are not ipso facto beneficial or harmful. They can be used for beneficial purposes and misused for harmful aims, and, as such, are a dual-purpose knowledge enabling dual-use technology. Yet, in the context of Computer Network Operations (CNOs), what makes the use of any given capability beneficial or harmful is very much a matter of perspective. Depending from which side of the playing field we look at things, the use of the same capability might be considered differently if it goes towards the creation or the detriment of political, military, diplomatic, economic, or business advantages. Notwithstanding the weak or uneven regulatory global landscape, the public international law and the international treaties form a backbone of principles that should be followed by the private sector in the acquisition of zero-day vulnerabilities. To begin with, traders of vulnerability information and security capabilities shall mitigate the risk to enable with their tools or knowledge the cyber security strategies of entities willing to abuse human rights. Also, they should contribute to the realisation to the right to health, by controlling which capabilities they provide to whom, if those capabilities may pose a direct danger to the health of human beings. Finally, in consideration of the time- sensitiveness and value of the traded commodities, the suppliers of zero-day vulnerabilities will need to honour the highest integrity standards, avoiding latent conflicts of interest that may erode the asymmetric advantage of the customers against targets that heavily dependent on IT, and preserving the confidentiality of the entities they do business with and the confidentiality of the acquired capabilities. With the code of ethics it is my intention to address these usage-related challenges and to contribute towards the establishment of a greater culture of responsibility. Code of Business Ethics As an ethically concerned founder of an acquisition platform for vulnerability information and security capabilities, I established a code of business ethics and I hold to its principles and standards in the conduct of my business. I set forth six principles and eight corresponding ethical standards.The principles are aspirational goals aimed at guiding and inspiring the conduct of business, and they underpin the ethical standards. The ethical standards are enforceable rules for the day-to- day business operations. Let’s give a look to them. The first and foremost governing principle that today I want to offer to you today is the clean hands principle, which asks the private sector, and me, to…
  6. 6. Principle A: Respect Human Rights — Clean Hands Respect all human rights proclaimed by international human rights treaties, including The International Bill of Human Rights, and strive to ensure no complicity in any human rights abuses. No, it is not my aspiration to run a company involved in any human rights abuse. Therefore I vet and monitor my Customers. Standard 1: Vetting and Monitoring of Customers Do not engage in any business with entities known for abusing human rights and reserves the right to suspend or cease business operations with entities found to be involved at a later time in human rights abuses. Principle B: Do Not Pose a Danger to Human Health Champion the health of human beings and commit to do not enable your Customer entities with capabilities that may pose a direct danger to human health. Standard 2: Inadmissible Capabilities Do not engage in any trade of capabilities that exploit vulnerabilities in medical devices or in systems to which human life is entrusted, unless the Vendor of the affected device or system is the Acquiring Entity or the Acquiring Entity was authorised by the Vendor to be the recipient of the vulnerability disclosure process, vulnerability information, or risk mitigation strategy. Standard 3: Trade Secrets You will never trade in stolen trade secrets, and require your suppliers to certify that they have independently discovered the vulnerability and autonomously developed any related technology, and that they are not employees of the targeted software manufacturer, nor have they received access to the confidential information through a disclosure by the same. All of which is to say that, no, I don’t want software maker employees to join the bug bonanza and to write them a new minivan this afternoon Principle C: Avoid Conflicts of Interest Strive to benefit those with whom you do business and take care to avoid possible conflicts of interest that could cause your Company, its Employees, or Contractors to pursue goals not in the interest of the Company business peers.
  7. 7. Standard 4: Conflict of interests and overexploitation You will protect the value of the traded capabilities. You will specify the maximum number of entities to which the same capabilities may be sold, within a given time-frame (unless in case the capabilities are intended for risk prevention). Furthermore, you shall strive not to sell a vulnerability to one party, and the technology to defend against that vulnerability to another party which is a likely target of the first. Standard 5: Unintended Use Prohibit yourself, employees and contractors to use the information or the capabilities, traded in the fulfilment of the service, for the pursuit of personal goals. Authorised personnel shall use such capabilities only to test and validate them, and more generally only for research and development purposes. Principle D: Obey the Law Comply with all applicable legal requirements and understands the major laws and regulations that apply to your business, including laws related to: trade controls, anti- bribery, competition, trade secret, money laundering and insider trading. Standard 6: Exporting Comply with trade laws controlling where the you can send products and services, strive to meet the criteria required to hold export licenses, where applicable, and stay alert to changes to the applicable export licensing systems. Principle E: Preserve Confidentiality Protect the confidentiality of the identity of entities you do business with and the the confidentiality of the information and intellectual properties received from, or provided to, your business peers in the fulfilment of your Service. At the same time, recognize that the extent and limits of confidentiality may be regulated by applicable laws and regulations. Standard 7: Maintaining Confidentiality At the extent and limits regulated by applicable laws and regulations, preserve the confidentiality of the identity of entities you do business with. Restrict access to the information and the intellectual property received from or provided to your business partners on a need-to-know basis, enforcing a principle of least privilege.
  8. 8. Principle F: Doctrine of Double Effect and Dual Use Acknowledge that the capabilities you provide may be used within goods that, just like any and all information security tools, are inherently dual purpose and potentially dual-use and therefore may serve also military purposes, police investigations and the like; the military use of the traded capabilities may have a double effect: the intended effect and the foreseen but genuinely unintended consequence. While discouraging against harmful side effects, you acknowledge the inherent duality of the effects resulting from the use of those capabilities and you trade them, unless they are in conflict with other principles set forth in the present Ethics Code. Standard 8: Duality Acknowledge that the capabilities you provide can be used within goods that are inherently dual use and accept to supply them, as long as it is foreseeable that those capabilities will be used only for legitimate purposes in line with international standards for the respect of human rights, and unless their trade is in conflict with principles set out in the present Ethics Code. And that is pretty much it as far as my code of ethics goes. Concluding Remarks Today I am honoured to be with you in Washington D.C. and I am reminded about the Russian-born Али́ са Зино́ вьевна Розенба́ ум, who once remarked: “Every aspect of Western culture needs a new code of ethics — a rational ethics — as a precondition of rebirth.” — Ayn Rand I feel similarly with regard to the debate surrounding vulnerabilities: Every aspect of the vulnerabilities supply chain needs a new code of ethics — a rational ethics — as a precondition of rebirth. In this spirit, I established the first code of ethics focused on the trade of vulnerability information and, today, I offered its principles and standards up for your comments and criticism. If, as noted by Earl Warren, “[i]n a civilised life, law floats in a sea of ethics”, it is both my hope and wish that our reflections will inform policy markers. As recently remarked by Steven Aftergood on Nature, “Expanding the scope of ethical deliberation over new technology may seem like a daunting prospect bound to impede innovation.” Today, there is no doubt, that I raised questions more quickly than they can be answered. “But experience suggests that many such questions will be worth asking.” I welcome all your thoughts. Thank you.
  9. 9. References [1] [2] [3] Shoshana Zuboff, In The Age Of The Smart Machine, new/books/in-the-age-of-the-smart-machine/ [4] David Rice, Geekonomics [5] [6] [7] David Rice, Geekonomics: The Real Cost of Insecure Software, A Keynote for Everyone