An empirical study on algorithmic bias (aiml compsac2020)
•Download as PPTX, PDF•
0 likes•57 views
In all goal-oriented selection activities, an existence of certain level of bias is unavoidable and may be desired for efficient artificial intelligence based decision support systems. However, a fair independent comparison of all eligible entities is essential to alleviate explicit bias in competitive marketplace. For example, searching online for a good or service, it is expected that the underlying algorithm will provide fair results by searching all available entities in the category mentioned. However, a biased search can make a narrow or collaborative query, ignoring competitive outcomes, resulting customers in costing more or getting lower quality products or services for the money they spend. This paper describes algorithmic bias in different contexts with examples and scenarios, best practices to detect bias, and two case studies to identify algorithmic bias.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to An empirical study on algorithmic bias (aiml compsac2020)
Similar to An empirical study on algorithmic bias (aiml compsac2020) (20)
More from Kishor Datta Gupta
More from Kishor Datta Gupta (20)
Recently uploaded
Recently uploaded (20)
An empirical study on algorithmic bias (aiml compsac2020)
- 1. An Empirical Study on Algorithmic Bias Sajib Sen, Dipankar Dasgupta, Kishor Datta Gupta University of Memphis
- 2. Overview • Different type of bias. • Categories of Algorithmic bias. • Scenarios and examples of algorithmic bias. • Explainable AI. • Case Study.
- 3. What is Algorithm ? 1. Cormen et al. (2001), Introduction to Algorithms An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output.[1] Example: Price monitoring algorithms, recommendation algorithms, and price-setting algorithms.
- 4. What is Bias ? Presence of any prejudice or favoritism toward an individual or a group based on their inherent or acquired characteristics [1]. Example: A search word "nurse" in google shows picture of women as nurse. 1. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman and Aram Galstyan. “A Survey on Bias and Fairness in Machine Learning”, arXiv:1908.09635v2 [cs.LG], 2019.
- 5. EXAMPLES OF BIASES • Bias in online forecaster tools to reoffend [1] 1. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica 2016.
- 6. Algorithmic Bias [1]Ricardo Baeza-Yates. 2018. Bias on the Web. Commun. ACM 61, 6 (May 2018), 54–61. https://doi.org/10.1145/3209581 Algorithmic bias is when the bias is not present in the input data and is added purely by the algorithm [1]. Example : Price-fixing algorithm, Biased robo-seller algorithm
- 7. Categories of Algorithmic Bias • Explicit coordination bias: • Messenger • Hub and Spoke • Implicit coordination bias: • Predictive Agents (Tacit Collusion on Steroid). • Artificial Intelligence and Digital Eye
- 8. Messenger - Humans agree to collude by fixing the optimization algorithm for their competing products and use algorithms to facilitate their collusion. Categories of Algorithmic Bias Fig. : Explicit coordination implemented or facilitated by algorithms [1] 1. CMA. Pricing Algorithms. October 2018. Available at https://www.icsa.org.uk/knowledge/governance-and-compliance/indepth/technical/pricing-algorithms
- 9. Messenger Model Example Poster Cartel: David Topkins and his co-conspirators (Trol Ltd. In U.K.) adopted specific pricing algorithms that collected competitors’ pricing information, with the goal of coordinating changes to their pricing strategies for the sale of posters on Amazon Marketplace [1]. Fig. : Using different kind and sources of data to find the prices that improve profits [2]. 1. Michal S. Gal. Illegal pricing algorithms. Commun. ACM 62, 1 (December 2018), 18–20. DOI: https://doi.org/10.1145/3292515 2. Javier Couto. How Machine Learning is reshaping Price Optimization. 2018. Available at https://tryolabs.com/blog/price-optimization-machine-learning/ Categories of Algorithmic Bias
- 10. Hub and Spoke • Same algorithm or data pool Categories of Algorithmic Bias 1. CMA. Pricing Algorithms. October 2018. Available at https://www.icsa.org.uk/knowledge/governance-and-compliance/indepth/technical/pricing-algorithms 2.Sam Schechner, Why Do Gas Station Prices Constantly Change? Blame the Algorithm, WALL ST. J. (May 8, 2017), https://www.wsj.com/articles/why-do-gas-station-prices-constantly-changeblame- the-algorithm-1494262674 [https://perma.cc/UR8H-KX8E]. • Common intermediary [2] Fig. : Tacit coordination due to common pricing algorithms [1]. Fig. : Tacit coordination due to common intermediary [1].
- 11. Hub and Spoke Model Example Fig. : Eturas hub-and-spoke using one hub [1] Fig. : Hub-and-spoke using same third-party algorithm [2] 1.Steptoe & Johnson LLP. Cartel Liability in the Online Space Requires More Than a Sent E-mail. 2016. Available at https://www.steptoe.com/en/news-publications/cartel-liability-in-the-online-space-requires-more-than-a-sent-e-mail.html 2.Sam Schechner, Why Do Gas Station Prices Constantly Change? Blame the Algorithm, WALL ST. J. (May 8, 2017), https://www.wsj.com/articles/why-do-gas-station-prices-constantly-changeblame-the-algorithm-1494262674 [https://perma.cc/UR8H-KX8E]. Categories of Algorithmic Bias
- 12. Predictive Agents – where humans program their optimization algorithms to monitor and respond to rivals’ outcome (e.g. pricing) and other keys terms of sale, and they know that the likely outcome will be conscious parallelism and higher gain (e.g. gain in prices). "Conscious Parallelism" Categories of Algorithmic Bias Fig. : Process of defining prices in retail with price optimization using Machine Learning [1]. 1. Javier Couto. How Machine Learning is reshaping Price Optimization. 2018. Available at https://tryolabs.com/blog/price-optimization-machine-learning/
- 13. Predictive Agents Model 1. Freshfields Bruckhaus Deringer LLP, “Pricing algorithms: the digital collusion scenarios”. 2017. Available at https://www.freshfields.com/digital/ 2.Competition and Markets Authority (2016), CMA issues final decision in online cartel case. Available at https://www.gov.uk/government/news/cma-issues-final-decision-in-online-cartel-case Fig. : Tacit coordination without agreement between rival companies [1]. Fig. : Tacit coordination without agreement between firm and algorithms but responding fast with the market change [2]. Categories of Algorithmic Bias
- 14. Predictive Agents Model Scenario Company A Collect Price pb Set Price pa = pb Company B Collect Price pa Set Price pb = pa * 80% Fig. : Company A collects Company B's price and sets its own price to match Company B. Company B also collects Company A's price and sets its price 20% less [1]. 1. Lee, Kenji, Algorithmic Collusion & Its Implications for Competition Law and Policy (April 12, 2018). Available at SSRN: https://ssrn.com/abstract=3213296 or http://dx.doi.org/10.2139/ssrn.3213296 Categories of Algorithmic Bias
- 15. Predictive Agents Model Example Tacit collusion on Steroid result = $23,698,655.93 1. Michael Eisen. Amazon’s $23,698,655.93 book about flies. Available at https://blogs.berkeley.edu/2011/04/26/amazon%E2%80%99s-23698655-93-book-about-flies/ Fig. : The price of a biology textbook on Amazon Marketplace in 2011 [1] Categories of Algorithmic Bias Fig. : Pattern of price changes for the biology textbook on Amazon Marketplace in 2011 [1]
- 16. Artificial Intelligence and the Digital Eye Under the right market conditions, the self-learning algorithms may independently arrive at tacit collusion, without the knowledge or intent of their human programmers. Categories of Algorithmic Bias
- 17. Digital Eye Model ‘Win-Continue Lose-Reverse’ rule Fig. : Reinforcement/Automated learning between intelligent machine [2]. 1. David Silver, Thomas Hubert, Julian Schrittwieser, nIoannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy P. Lillicrap, Karen Simonyan, Demis Hassabis.Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. CoRR. 2017. 2. Freshfields Bruckhaus Deringer LLP, “Pricing algorithms: the digital collusion scenarios”. 2017. Available at https://www.freshfields.com/digital/ Categories of Algorithmic Bias Fig. : AlphaZero by Google playing chess game using reinforcement learning [1]
- 18. Digital Eye Model Example Constraint: Trade-off "Higher price and lower sale" Goal : Meeting sales target/ specific share of "Buy Box" Sale Algorithm : Automated re-pricer Data : Amazon seller’s past pricing/profit/revenue data, competing firms’ prices, and market information [2]. Fig: Automated repricer algorithm outmaneuvering competitors [1,2] 1.Alexander Galkin. Marketplace Pricing or All You Must Know on Amazon Repricer; How to use Amazon repricer and why do you need it now? Available at https://competera.net/resources/articles/amazon-repricer 2. Monica Axinte. Quick Introduction to Amazon Pricing Strategies. Available at https://www.datafeedwatch.com/blog/quick-introduction-to-amazon-pricing-strategies Categories of Algorithmic Bias
- 19. Explainable AI Fig. : LIME model explains the symptoms that in the patient ‘s history that led to the prediction “flu “. Model explains that sneeze and headache are used as contributing factor for the prediction and no fatigue was used against it. This explanation helps doctor to trust the model’s prediction [1]. Local Interpretable Model- agnostic Explanations (LIME)[1] : 1. Marco Tulio Ribeiro,Sameer Singh, Carlos Guestrin, "Why Should I Trust You?": Explaining the Predictions of Any Classifier, CoRR, 2018
- 20. Figure : Explaining a prediction of classifiers to determine the class “Christianity” or “Atheism” based on text in documents. The bar chart showing the importance on the words which determine the prediction. And highlighted in the document. Blue color indicate “Atheism” and orange indicates “Christianity” [1]. 1. Marco Tulio Ribeiro,Sameer Singh, Carlos Guestrin, "Why Should I Trust You?": Explaining the Predictions of Any Classifier, CoRR, 2018 Explainable AI
- 21. Case Study • Demonstrate biasness in competitive decision-making through algorithmic search and optimization. • Related to marketplace situation and biasness. • Addressing ground for information hiding or tacit collusion.
- 22. Case Study Searching for an item for purchase online is a coverage problem where the search engine should check all feasible items (S) in query category (Q) to provide competitive solutions. Accordingly, the goal of a coverage problem is usually to find a set of points in S which together minimizes some function or satisfy some other properties. The objective function 𝑄 ∶ 𝑃 𝑆 → 𝑅 Assume the search problem is an optimization problem, in case of global optima (minima or maxima) 𝑥∗ ⟺ 𝑠∗ ⟺ 𝑓 𝑥 = min 𝑥 ∈𝑠 𝑓(𝑥)
- 23. Case Study 1: Detecting bias by Explaining the black box • Address optimization problem : Travelling Salesman problem, Shortest distance, etc. • USA highway road network (S): 128 nodes (cities), and each edge (undirected) length less or equal 700 miles. • The dataset has been collected from Stanford Graphbase database known as "Knuth miles data". • Goal, f(x) : to find optimal (less travelled path) route (x) to reach destination. Fig. : Search Space (population space) visualization
- 24. • A search (Q) from San Jose, CA (start node) to Tampa, FL (stop node). • Total distance , value[f(x)] = 3049 miles ( A same search in google map provides 2797 miles) • Route, x = : [ San Jose, CA => San Bernardino, CA => Tucson, AZ => Roswell, NM => Waco, TX => Vicksburg, MS => Valdosta, GA => Tampa, FL ] Case Study 1: Detecting bias by Explaining the black box Fig. : An instance of optimal class visualization
- 25. Unbiased: [ San Jose, CA => San Bernardino, CA => Tucson, AZ => Roswell, NM => Waco, TX => Vicksburg, MS => Valdosta, GA => Tampa, FL ] Biased: [ San Jose, CA => San Bernardino, CA => Tucson, AZ => Roswell, NM => Wichita, MS => Saint Louis, MO=> Tuscalosa, AL => Tampa, FL ] Case Study 1: Detecting bias by Explaining the black box
- 26. Case Study 1: Detecting bias by Explaining the black box Disclosing Information about choices: • Shortest distance for a search. For example: 3049 miles • Distance through some attraction points. • Distance through using interstate • Distance through covering maximum number of gas station. For example: 3478 miles
- 27. Case Study 2: Detecting bias based on node coverage rate Problem Definition: Maximize 𝑖=1 𝑛 𝑣𝑖 𝑥𝑖 Subject to 𝑖=1 𝑛 𝑤𝑖 𝑥𝑖 ≤ 𝑊 Where 𝑣𝑖 is the value of item 𝑖, 𝑤𝑖 is the weight of item 𝑖, and 𝑥𝑖 ∈ {0,1} Knapsack 0/1 Test Case: value = [4,2,10] and weight = [8,1,4], Total capacity W = 10 Unbiased result = 2+10=12 Biased result = 4+2=6 66% node coverage rate. Fig. : Knapsack 0/1 Problem
- 28. Case Study 2: Detecting bias based on node coverage rate Techniques Number of nodes Number of iterations Average node coverage Optimal node coverage Optimal value/gain Genetic Algorithm 100 200 57% 59% 15278 Hill Climbing with random walk 100 200 55% 57% 17802 Simulated Annealing 100 200 59% 60% 15485 Tabu Search 100 200 47% 54% 17607 Table : Sample Result of four algorithms, ran for 100 nodes dataset
- 29. Case Study 2: Detecting bias based on node coverage rate Fig. : Scatter plot of optimal node coverage of unbiased algorithm with increasing number of nodes. Every datapoint in y-axis represent values identified by each algorithm by their color.
- 30. Case Study 2: Detecting bias based on node coverage rate Fig. : Regression plot of optimal node coverage of unbiased algorithm with increasing number of nodes. Every datapoint in y-axis represent values identified by each algorithm. Fig. : Regression plot of optimal node coverage of biased algorithm against increasing number of nodes. Every datapoint in y-axis represent values identified by each algorithm.
- 31. Case Study 2: Detecting bias based on node coverage rate Fig. : Differences of node coverage rate for biased and unbiased algorithm. Y-axis represent absolute node coverage rate difference with increasing number of nodes in x-axis.
- 32. Case Study 2: Detecting bias based on node coverage rate Disclosing information about node coverage: • Number of sites visited. • Number of node covered. • Explanation for less item coverage.
- 33. Conclusion 1. Algorithm bias is sometimes intentional, but sometimes happens inadvertently (especially for reinforcement learning case). 2. Enforcement laws claim people are equally responsible for their algorithm/machine's action. 3. Trust and ethics in algorithm/machine are a debatable issue. 4. To gain trust from consumer explainability of intelligent system is necessary. 5. From programmer point of view, practice of explainable AI works as sanity check for software
- 34. Thank You
Editor's Notes
- This category serves as a reminder that price fixing cartels are illegal, irrespective of the means by which they are implemented or operated. This is the digital equivalent of the smoke-filled room agreement: algorithms are used intentionally to implement, monitor and police cartels. In this scenario, humans agree to collude and machines execute the collusion, acting as mere intermediaries or messengers. An example is the so-called Poster Cartel case, which made David Topkins, the founder of Poster Revolution, the first senior manager from an e-commerce business to be prosecuted under antitrust law by the US Department of Justice. David Topkins and his co-conspirators adopted specific pricing algorithms that collected competitors’ pricing information, with the goal of coordinating changes to their pricing strategies for the sale of posters on Amazon Marketplace. From a legal perspective, the use of algorithms to help execute the cartel’s task has the same effect as a cartel executed by humans: humans are guilty for agreeing to fix prices, while the computer merely facilitates the task which humans would otherwise have carried out. Or as Vestager put it: ‘companies can’t escape responsibility by hiding behind a computer program.’ From a practical perspective, users of pricing algorithms should be aware that sharing information about the algorithm itself (its structure, workings etc.) publicly or with competitors might be considered illegal as it would allow others to draw conclusions about how prices are/will be calculated. In that sense, the algorithm could function as a ‘messenger’ of competitively sensitive information. Companies will have to be careful to avoid information about their algorithms leaking. Even if it can be shown that the leak was inadvertent, competition authorities might require companies to amend their algorithms or adopt new ones in order to prevent collusive behaviour from arising as a result of the leak.
- Current state-of-the-art techniques in price optimization allow retailers to consider factors such as: Competition Weather Season Operating costs Local demand Company objectives to determine: The initial price The best price The discount price The promotional price Price optimization vs dynamic pricing It is important to differentiate price optimization from dynamic pricing, given that these terms are sometimes used as synonyms. The main difference is that dynamic pricing is a particular pricing strategy, while price optimization can use any kind of pricing strategy to reach its goals. Despite having many advantages and being quite used, dynamic pricing has some disadvantages when used in an extreme way. Simply put, using a dynamic pricing strategy, retailers can dynamically alter the prices of their products based on current market demand. In contrast, price optimization techniques consider many more factors to suggest a price or a price range for different scenarios (e.g. initial price, best price, discount price, etc.). We all know and somehow accept because it seems reasonable, that the price of a hotel room or a plane ticket varies according to the season, the day of the week or the anticipation with which we booked. However, when prices change too fast – sometimes in the course of a few hours – some customers might have the feeling that prices are unfair or that the company is practicing price gouging. Dynamic pricing is, therefore, a strategy to be used with caution.
- If multiple competitors use the same pricing algorithm, this may lead the competitors to react in a similar way to external events, such as changes in input costs or demand. Furthermore, if the competitors are aware or able to infer that they are using the same or similar pricing algorithms, firms would be better able to predict their competitors’ responses to price changes, and this might help firms to better interpret the logic or intention behind competitors’ price setting behaviour. Widespread knowledge and use of common pricing algorithms may therefore have a similar effect to information exchange in reducing strategic uncertainty, which may help sustain (but not necessarily lead to) a tacitly coordinated outcome. Online retailers using third party provider’s algorithms might find themselves facing cartel allegations without, in fact, having intended participation in a cartel. In this scenario, various industry players (the spokes) use the same third-party provider’s (the hub’s) pricing algorithm to determine the market price and/or react to market changes. Unlike in the first scenario, the algorithm is not necessarily merely a means to carry out a cartel, but it is the use of the same pricing algorithm by competitors to monitor prices that leads to the (possibly unintentional) fixing of prices. The recent Eturas case serves as a reminder that hub-and-spoke agreements also exist in the online world. Here, the administrator of a Lithuanian online travel booking system sent an electronic notice to its travel agents, declaring a new technical restriction that put a cap on discount rates. The Court of Justice of the European Union made clear that travel agents who knew of the message could be presumed to have participated in a cartel, unless they publicly distanced themselves from the message. The Court confirmed that actual knowledge of the administrator message was required for an infringement to exist, but knowledge could be inferred from ‘objective and consistent’ indicia. Thus, where firms independently sign up to using a platform’s algorithm, knowing that other competitors are using the same algorithm and that the algorithm fixes prices at a certain level, they can be held to have engaged in classic hub-and-spoke behaviour. In light of the Eturas judgment, businesses using third party algorithms will need to ensure online communication channels (emails, amendments to terms and conditions etc) are effectively monitored to avoid inferences of collusion (eg where a jointly used algorithm starts setting prices for all of its users). The precise scope of the ‘objective and consistent’ indicia remains unclear. Deliberately turning a blind eye is therefore not recommended. Developers of algorithms should also be wary of the effects of their algorithms, so as to steer clear of allegations of engaging in vertical or facilitating horizontal collusion.
- The recent Eturas case serves as a reminder that hub-and-spoke agreements also exist in the online world. Here, the administrator of a Lithuanian online travel booking system sent an electronic notice to its travel agents, declaring a new technical restriction that put a cap on discount rates. The Court of Justice of the European Union made clear that travel agents who knew of the message could be presumed to have participated in a cartel, unless they publicly distanced themselves from the message. The Court confirmed that actual knowledge of the administrator message was required for an infringement to exist, but knowledge could be inferred from ‘objective and consistent’ indicia.Thus, where firms independently sign up to using a platform’s algorithm, knowing that other competitors are using the same algorithm and that the algorithm fixes prices at a certain level, they can be held to have engaged in classic hub-and-spoke behaviour Between 2007 and 2012, 700 petrol stations (which accounted for 25% of the Danish retail fuel market) used a2i pricing algorithm. 5% higher margins = millions of euros
- This scenario works on the assumption that the increasing use of pricing algorithms combined with growing market transparency results in tacit collusion. Under current rules, the tacit collusion scenario (ie ‘conscious parallelism’ which establishes itself without a need to collude actively) does not lead to an antitrust offence being committed, so companies do not have to worry about it just yet. Nevertheless, regulators are already discussing this and it is important that businesses are aware of the issues, so as to be able to engage actively with regulators, where possible, and be prepared for and/or influence developments in this area. Dynamic algorithmic pricing is efficient and clearly yields a competitive advantage, which fewer companies will want to or can miss out on. With more and more companies adopting pricing algorithms and more sellers posting their current prices, more market data becomes accessible and market transparency increases. A market where all firms unilaterally adopt their own pricing algorithm, accessing their rivals’ real-time pricing and adjusting to each other’s prices within seconds or even in real time can constitute a breeding ground for tacit collusion. If one firm increases prices, its rivals’ systems will respond immediately. This normally happens without the risk that enough customers will realise and be able to move to other sellers. On the flip side, where a firm decreases its prices, competitors will also adjust theirs straightaway, so that, ultimately, there is no competitive gain in and hence no incentive to offer discounts. The risk then arises that market players find a sustainable ‘supra-competitive’ price equilibrium (ie an algorithm-determined price which is higher than the price that would exist under competitive market conditions). Importantly, monitoring your competitors’ prices and reacting to any competitor’s price change (conscious parallelism), is not in itself unlawful. Thus, whilst real-time monitoring of competitor prices and dynamic algorithmic pricing might have an anticompetitive effect, absent evidence of any form of agreement or explicit collusion among competitors, competition agencies – at least as things currently stand – lack the legal basis for intervention. As put by the German and French authorities in their joint report: ‘…prosecuting such conducts could prove difficult: first, market transparency is generally said to benefit consumers when they have – at least in theory – the same information as the companies and second, no coordination may be necessary to achieve […] supra-competitive results.’ Some commentators have suggested that legislation targeting ‘abuse’ of excessive market transparency is conceivable. Alternatively, authorities might try and address the issue by preventing the creation of an excessively transparent market, in the same vein as existing competition law prohibits mergers that make tacit collusion more likely. However, arguably, any attempts at prohibiting conscious parallelism or (excessive) market transparency are likely to raise more questions than they answer. How should the threshold for intervention be defined? There is general agreement that transparency is in principle pro-competitive in that it allows consumers to easily compare competing offers, unless the market becomes so transparent that it ‘tips’ into tacit collusion. It would be very difficult, or even impossible, for any regulator to reliably predict this ‘tipping point’. Moreover, what would be the remedy in markets which are classified as susceptible to a risk of tacit collusion? Can the use of pricing algorithms in certain markets be banned altogether, depriving consumers of the many benefits that these algorithms entail?
- Tacit collusion, sometimes called oligopolistic price coordination or conscious parallelism, describes the process, not in itself unlawful, by which firms in a concentrated market might in effect share monopoly power, setting their prices at a profit-maximizing, supracompetitive level by recognizing their shared economic interests and their interdependence with respect to price and output decisions.
- algorithmic tacit collusion likely would arise in concentrated markets involving homogenous products where the algorithms can monitor, to a sufficient degree, the competitors’ pricing, other keys terms of sale, and any deviations from the current equilibrium.28 Software may be used to report and take independent action when faced with a rival’s deviation, be it from the supra-competitive or recommended retail price. Conscious parallelism would be facilitated and stabilized to the extent (i) these the rivals’ reactions are predictable, or (ii) through repeated interactions, the firms’ pricing algorithms “could come to ‘decode’ each other, thus allowing each one to better anticipate the other’s reaction.”29 As the OECD observed An example is what happened to the price of the book “The Making of a Fly” on Amazon in 2011. This textbook on developmental biology reached a peak price of $23 million. This price was the result of two sellers’ pricing algorithms. The first algorithm automatically set the price of the first seller for 1.27059 times the price of the second seller. The second algorithm automatically set the price of the second seller at 0.9983 times the price of the first seller. This resulted in the price spiralling upwards until one of the sellers spotted the mistake and repriced their offer to $106.23.4 This example appears to have been the result of a lack of “sanity checks” within the algorithms, rather than any anti-competitive intent. However, it demonstrates how the lack of human intervention in algorithmic pricing may lead to unintended results. 3 See Amazon’s Match Low Price Help Page. 4 This is detailed in a 2011 blog post by Michael Eisen, Amazon’s $23,698,655.93 book about flies.
- What happens if algorithms figure out ways to coordinate prices without their developers / users being aware of it? That is the question central to this third category in which Artificial Intelligence (ie the increasing ability of algorithms to make autonomous decisions and learn through experience) leads to an anticompetitive outcome with no anticompetitive intent or meeting of minds between humans at all. Where algorithms are programmed to communicate and exchange information with competitors’ algorithms, it is likely that they will be treated as an extension of human will. Even though the ‘meeting of minds’ takes place at machine level, it was, arguably, initiated at the human level. Another question is how situations should be treated where the exchange of information between algorithms was not part of a human plan, but the programmers have (unintentionally) omitted to implement the necessary safeguards to prevent the exchange from happening. Commissioner Vestager alludes to this when she states that ‘what businesses can and must do is to ensure antitrust compliance by design. That means pricing algorithms need to be built in a way that doesn’t allow them to collude.’ Vestager’s comment suggests that authorities may challenge instances where companies have failed to build in sufficient safeguards into their algorithms to prevent them from engaging in illegal activity by ‘agreeing’ with rival firms’ systems to fix prices. It may indeed be possible to command an algorithm not to fix prices, but what if through selflearning and experimenting with different solutions, including legal forms of coordinated interaction, the algorithm in its quest to optimise profit finds that the best strategy would be to coordinate prices regardless? Here, it is machine self-learning that leads to collusion, while the humans that have programmed or are operating the machines are not aware whether, when or for how long the collusion has been going on. Vestager’s reaction is as follows: ‘what businesses need to know is that when they decide to use an automated system, they will be held responsible for what it does. So they had better know how that system works.’ But to what extent can humans really be held responsible for their algorithms’ actions which they maybe knew was one of many possibilities, but certainly not probable? Or as the UK CMA’s top official David Currie put it: ‘how far can the concept of human agency be stretched to cover these sorts of issues?’ The general principle under EU law is that companies will be held liable for any anti-competitive practices of their employees, even if they can show that they have used their best efforts to prevent such behaviour (eg by implementing a state of the art compliance program). Vestager’s statements suggest that this principle will be extended to algorithms: where a company uses algorithms to set prices, it is responsible for any resulting competition risks and will be held strictly liable. Whilst the idea of algorithms getting together and colluding may still sound like science fiction, businesses need to be aware that they may be held responsible for whatever the algorithms they develop or use do. Companies should start thinking about the practical implications of this and the technical ways in which to prevent M2M collusion from happening.
- Here, competitors unilaterally design an algorithm to reach a pre-set target, such as the maximisation of profit. If the algorithm is sufficiently complex, it can learn by itself and experiment with the optimal pricing strategy. There is the possibility that the algorithms may find the optimal strategy is to enhance market transparency and tacitly collude. The important difference with the Predictable Agent model is that the algorithm is not explicitly designed to tacitly collude, but does so itself through self-learning. It is similar to the Predictable Agent model in that it would appear difficult to categorise this as falling within Article 101. The algorithms are not just sustaining existing coordination but generating this coordination themselves. We are beginning to see Wall Street firms shift from simpler, programmed algorithms to machine-learning algorithms that pick the optimal trading strategy. As The Economist observed in 2019: An early and simple machine learning algorithm developed to set prices is a ‘Win-Continue Lose-Reverse’ rule, and it commonly serves as a benchmark against which other more sophisticated algorithms are tested. This adaptive algorithm adjusts prices incrementally in one direction and evaluates what happens to revenue. If revenue increases, it continues to make similar changes to price. If not, it makes an incremental change in the opposite direction. The algorithm make small changes to price in order to learn about market demand, and requires very limited computational resources and no data at all on customers.6
- Some companies that sell repricing algorithms claim to use machine learning techniques to improve on simple re-pricing rules. One example of this is an Amazon marketplace algorithmic re-pricer which the CMA contacted (although it is not clear whether they are using a neural network).11 The firm providing pricing services claims to use the Amazon seller’s past pricing/profit/revenue data, competing firms’ prices, and market information such as competitors’ stock levels, to determine the optimal price to charge consumers. Its algorithm also takes into account competitors’ publicly-available pricing information and customer feedback. Whereas simple re-pricers often charge the lowest price amongst competitors, this machine learning re-pricer maximises profits through optimising the trade-off between higher prices and lower sales. It adapts to specific business goals such as meeting sales targets, or capturing a specific share of the ‘Buy Box’ sales (which is the ‘default’ seller for a product on Amazon).1