The document discusses standards and co-regulation of AI under EU law and lessons that may apply in Australia. It summarizes key events involving medical AI disasters like Babylon Health and proposes that co-regulation, which involves standard-setting organizations, could help balance flexibility with accountability when regulating rapidly changing technologies like AI. However, it notes challenges like potential conflicts of interest and questions whether the EU's proposed AI Act will be enforceable given its broad scope and delayed timeline.

1. The role of standards and evidence
Professor Chris Marsden,
Monash Digital Law Group
29 November 2023
AI Co-regulation –
Lessons from the EU?
Victorian Heart Institute Innovation Conference

2. Associate Director,
Monash Data Futures Institute
Director, Monash Digital Law Group
Professor of AI,
Technology and the Law
Visiting Professor UKRI TAS Regulation & Governance Hub

3. A lesson from Babylon’s Tower of Babel
Don’t believe the hype!
AI chatbots

4. 1927
REGULATORY
CONCERNS
ABOUT AI
CHATBOTS

5. CO-REGULATION OF AI
UNDER EU LAW
• Some medical AI/chatbot disasters
• EU proposal for a Regulation on AI, known as the ‘AI Act’,
• Use of co-regulation of standards in the EU AI Act
• At the end of the talk, Chris will offer some (very premature)
observations on what Australia may learn from the process.

6. Marsden and Tambini
“There is a danger that some aspects of internet self-
regulation fail to conform to accepted standards.
We recommend co-regulatory audit as the best balance of
fundamental rights and responsive regulation.”
Directorate-General for Communications Networks, Content and Technology (European Commission) , Programme
in Comparative Law and Policy (2004) Self-Regulation of Digital Media Converging on the Internet: Industry Codes of
Conduct in Sectoral Analysis, Final Report of IAPCODE Project for European Commission DG Information Society
Safer Internet Action Plan, 30 April, Section 12.7 at https://publications.europa.eu/en/publication-detail/-
/publication/b7c998d9-75d6-464d-9d91-d59aa90a543c/language-en
2004 EC selfregulation.info
We still collaborate – see Regulating Big Tech (OUP, 2021)

7. The Story of a Morally Bankrupt Unicorn & a Medical AI Disaster
Babylon – the $4billion disaster
Dr David Watkins: Babylon Health (2013 - 2023)
• Fraudulent claims
• Dangerously flawed technology
• Regulatory failings
• 21 October 2021, Babylon valued at $4.2B
• May 2023, Babylon bankrupt.

8. BABYLON HEALTH AI?
 Why did Babylon fail? Babylon business model was always based on lies.
 “There’s an element of ‘fake it until you make it’ in Silicon Valley”
 Venture Capitalist who didn't invest in @babylonhealth.
 “I don’t think I’ve ever had a meeting when he has not said something provably untrue”
 Blitzscaling Babylon: Ali Parsa’s break-neck scramble for success (thetimes.co.uk)

9. MEDICINES & HEALTHCARE PRODUCTS
REGULATORY AGENCY: FAILED
1. Babylon was an unethical corporation that knowingly put patients at risk of harm.
2. UK regulators ignored serious corporate failings in 2017-18.
3. Babylon business model was based on AI tech that didn’t exist.
 UK Health Secretary Matt Hancock supported it all the way
 Despite existing safety concerns of regulators
 Care Quality Commission & MHRA
 May 2018 Babylon ‘upgraded’ their AI Chatbot
 managed to make it even more dangerous

10. BABYLON AI “CAPABLE OF AUTOMATING DIAGNOSIS &
CONSULTATION TRANSCRIPTION WITH COST SAVING OF 70%”
Ali Parsa made AI technology claims including;
- Alexa Integration
- AI Emotion recognition
- AI Voice transcription
- AI Diagnosis
all AI vapourware

11. THEIR NEW AI CHATBOT CLAIMED TO BE AS ACCURATE AS A
DOCTOR, WHICH WAS A REAL PAIN IN THE CHEST
The patient was a 67 yr old obese 20/day male smoker –
with typical symptoms of a #HeartAttack.
This negligent advice was a known flaw in #AskBabylon #Chatbot
@MHRAdevices formally raised it as incident with them in May 2018

13. CQC CLAIMED THEY COULD
NOT STOP AI CHATBOT HYPE
 Complaints against individual unethical doctors go
through General Medical Council
 Not renowned for its lightning speed
 Medical Act 1858 charity co-regulator
 Council for Healthcare Regulatory Excellence
December 2011 paper 'right-touch regulation'
 Advertising ‘Standards’ (sic) Authority is a self
regulator for ad sales

14. HERE’S WHAT HAPPENED FROM ITS 2021
SHARE SALE ONWARDS

15. THERANOS: SMARTEST WOMAN
IN THE ROOM
 Estafadora, absolutemente!
 Stanford dropout 2004 as with later AI luminaries @FTX & OpenAI
 Faked a low voice and Steve Jobs look from 2007
 2014: Theranos valued at $9 billion; raised $400 million
 By the end of 2014, her name on 18 U.S. & 66 foreign patents
 2015 appointed to Harvard Medical School Board of Fellows
 2016 –regulator Centers for Medicare/Medicaid Services intervened
 Conviction after case 2018-22: Wire fraud (3 counts) Conspiracy to
commit wire fraud (1 count). Serving 135 months) in prison
 Holmes’ father, Christian Rasmus Holmes IV, VP at Enron,
 bankrupt energy company after a giant accounting fraud

16. 2023: OPENAI DEVELOPS
CHATGPT –CLAIMS IT CAN
REPLACE DOCTORS
 Babylon 5?
 Save us from more idiot tech bros
 Yes, it can help doctors write notes…

18. UK NHS USE OF PATIENT DATA FOR AI
• Ben Goldacre Review (2022)
• https://www.gov.uk/government/publications/better-broader-safer-using-health-
data-for-research-and-analysis
• Health Secretary Hancock in 2019 – AI will save NHS
• Replacing doctors’ unions…
• PM advisor Cummings & Palantir
• Faculty AI (start-up) and COVID tracking app
• UK app too centralised – spyware & cybersecurity trap
• rejected by Google & Apple on privacy grounds

19. $BILLION PALANTIR NHS
FEDERATED DATA CONTRACT
• Agreed this week
• Isaac Levito – NSW lobbyist extraordinaire

20. TRUST & VERIFY: BIG DATA CAN HELP
BUT…

21. EU AI ACT: MY VIEW WITH GEORGE CHRISTOU

22. CO-REGULATION INVOLVES DELEGATING THE
CREATION OF STANDARDS TO COMPLY WITH
LEGISLATION TO STANDARD-SETTING
ORGANIZATIONS
• Can be technical, normative or both
• (STS scholars will insist all are both)
• Note formal requirement in legislation –
• this is NOT self-regulation
• Potential advantages:
• flexibility and adaptability to rapidly-changing technology,
• as well as opportunities for expert input and industry participation.
• Risks: conflicts of interest, lack of transparency/accountability,
• potential for under- or over-regulation.

23. Is this a rise from self-regulation to ethics washing acceptance?
Enhanced and Meta Regulation
Co-regulation vs enhanced?
Meta regulation
• Regulated self-regulation
• Beaufort Scale of Co-regulation (Marsden 2011)
• Grabowsky (2016) identifies 1983 as the first use of ‘meta regulation’
• ch09.pdf (anu.edu.au)

24. IS THIS AN ACT WORTH EMULATING?
• Neither Artificial –
• definition may be too broad for AI
• Nor Intelligent
• Devolves to existing standards bodies
• Nor an Act
• Still 3 years from enforcement even after Trilogue
• which may end in September for a plenary vote
• So 2027 enforcement? AI Board proposal?

25. Council of Europe
Strasbourg effect?
As with data protection – Convention 108+
‘
• CAI - Committee on Artificial Intelligence - Artificial
Intelligence (coe.int)
• Convention on Regulation of Artificial Intelligence
• Agreed November 2023?
• Treats human rights as integral to regulation – unlike EU
• EU negotiating as a party:
• https://futurium.ec.europa.eu/en/european-ai-
alliance/blog/eu-participates-actively-council-europe-
negotiations-development-new-convention-artificial
‘

26. AUSTRALIA MAIN LESSON: REPENT AT
LEISURE
#BrusselsEffect has been #PortarlingtonEffect (Ireland not VIC)
• EU laws are unenforceable which suits tech companies
• Not only GAFAM from US/Ireland/Lux – also SAP and other EU based
• But undermines trust in tech in society more generally

27. MINISTER HUSIC SPOKE AT UTS NEW
INSTITUTE LAUNCH OCTOBER 2022
• “I want Australia to become the world leader in responsible AI.
• This includes setting reasonable regulations and standards...
• providing citizens with confidence that technologies like AI are being developed
in ways that's trusted, secure and to their benefit.
• Our existing legal frameworks need to be fit for purpose”
So he wants to be judged on comparative regulation
• Suggests co-regulation with standards – perhaps incorporating CSIRO (and
other) local audit standards
• Husic, Ed (2022) Remarks of the Minister at Launch of the Human Technologies Institute, UTS, 20
October, at https://uts.edu.au/sites/default/files/2022-
10/Transcript%20Ed%20Husic%20UTS%20Human%20Technologies%20Institute%20launch.pdf

28. WHAT WOULD BE BETTER AI
REGULATION?
• UK and Australia using sectoral regulation and sand boxes?
• UK regulatory strategy for trustworthy AI, 29 March 2023, relies on national
sectoral regulators and the Competition Markets Authority: AI regulation: a
pro-innovation approach - GOV.UK (www.gov.uk)
• Australia released its own National AI Strategy, in June 2021
https://wp.oecd.ai/app/uploads/2021/12/Australia_AI_Action_Plan_2021.pd
f
• Petition launched, supported by more than 1,000 AI luminaries, that called
for a six month moratorium on the training of “AI systems more powerful
than GPT-4” (https://futureoflife.org/, March 2023).
• United Nations: 193 member states agreed to a Global Agreement on
Ethics of AI 25 November 2021
• Big summits this summer – AIforGood, Global Digital Compact