This document summarizes a presentation given by Prof Chris Marsden on responsible content moderation and the law. It discusses issues around disinformation, including defining it. It analyzes different regulatory options for addressing disinformation, from self-regulation to statutory regulation. It provides recommendations, including that governments should avoid internet shutdowns in response to disinformation alone. It discusses the Christchurch Call agreement signed by many countries to address terrorist and violent extremist content online, including a focus on understanding recommendation algorithms. Former PM Jacinda Ardern of New Zealand now promotes the Christchurch Call as a special envoy.

1. 5 DECEMBER 2023

2. GENERATIVE AI,
RESPONSIBLE
CONTENT
MODERATION
AND THE LAW
PROF CHRIS MARSDEN
MONASH UNIVERSITY
6 DECEMBER ASIA INTERNET COALITION
@PROF_MARSDEN

3. ACKNOWLEDGEMENT OF COUNTRY

4. VOICE CONSTITUTIONAL
AMENDMENT
• Voting in Australia is mandatory (ballots can be spoilt)
• 90% turnout is normal
• Amendment has to have majority in 4/6 states
• Voice consultative assembly failed in all 6

5. DISINFORMATION ENVIRONMENT
• Victoria (where I live in Melbourne) was closest to majority
• This was a traditional ‘big media’ campaign
• With some AI-generated deep fakes included….

7. COMPLETELY DIFFERENT…
AI IN HEALTHCARE

10. SPEECH
TO
TEXT?
CHATBOT

11. AI
POWERED
SEARCH?

12. REGULATORY CONCERNS INCREASED ABOUT:
• Fake news, hate speech spread by bots, and
• their impact on fair elections and civil society.
• But these 'Great Hack' concerns are much older,
• going back to 2010 Trinidad and Ukraine (2011) elections.
As in the case of privacy regulation with GDPR,
• Europe led the way in willingness to legislate,
• Germany (2017 NetzDG Law) France (2018) legislated against disinformation

13. DEFINING DISINFORMATION (‘FAKE NEWS’)
• “False inaccurate or misleading information
• designed, presented and promoted to
• intentionally cause public harm or for profit”
• European Commission High Level Expert Group 2018
Distinguish disinformation from misinformation,
• which refers to unintentionally false or inaccurate information.

14. DISINFORMATION A RAPIDLY MOVING TARGET
• 2018 European Parliament study
• analysed 250 articles, papers and reports
• strengths and weaknesses of those focussed on AI disinformation
solutions on freedom of expression, media pluralism & democracy
• Agree with other experts: evidence of harm inconclusive in 2018
• 2016 US Presidential election/UK ‘Brexit’ referendum
• Note this report was pre-Muller (2018) and pre-fines in UK (2019-20)
• US Department of Justice and UK Parliamentary Committee

15. CHRIS MARSDEN, TRISHA MEYER, IAN BROWN
● Platform values and democratic elections:
● How can the law regulate digital disinformation?
Computer Law & Security Review, Volume 36, 2020, ISSN 0267-3649, https://doi.org/10.1016/j.clsr.2019.105373

16. 2018 EUROPEAN PARLIAMENT REPORT:
REGULATING DISINFORMATION WITH ARTIFICIAL INTELLIGENCE
• Presented in Strasbourg 13 December
• https://www.europarl.europa.eu/cmsdata/161725/STOA%20Panel%20meeting%2013-12-2018%20-%20Minutes.pdf
• 2 days after terrorists attack Xmas market
• Finally officially published 13 March 2019

17. INTERDISCIPLINARY STUDY ANALYSES IMPLICATIONS OF AI
DISINFORMATION INITIATIVES
Policy options based on literature, 10 expert interviews & mapping
We warn against technocentric optimism as a solution to disinformation,
• that proposes use of automated detection, (de)prioritisation, blocking and
removal by online intermediaries without human intervention.
• More independent, transparent and effective appeal and oversight mechanisms
are necessary in order to minimise inevitable inaccuracies

18. WE STUDIED CAMBRIDGE ANALYTICA, RUSSIAN (AND
MANY OTHER ACTORS) HACKING OF ELECTIONS
• Wider issue of regulating disinformation and election cybersecurity
• Euronews (9 Jan 2019) How Can Europe Tackle Fake News in the Digital Age? 3 minutes' video
• https://www.euronews.com/2019/01/09/how-can-europe-tackle-fake-news-in-the-digital-age
• saves you reading 100 pages’ European Parliament report I co-authored.
• But you should read the 4-page regulatory annex –
• How to regulate AI for disinformation.
• Good luck employing anyone with human rights law qualifications in India
• for much less than $100 an hour...much less a dollar!

19. 2021: LEAVE.EU LOST THEIR CASE, LONG AFTER BREXIT
• Leave.EU and Eldon v Information Commissioner [2021] UKUT 26 (AAC) (8 February 2021)
• https://www.bailii.org/uk/cases/UKUT/AAC/2021/26.pdf
• UK Information Commissioner issued Leave.EU and Eldon with
• monetary penalty notices and assessment notices (and an enforcement notice in the case of Eldon)
• under Data Protection Acts 1998 and 2018.
• First-tier Tribunal dismissed all appeals – appeal to the Upper Tribunal concerned the scope of Reg 22 PECR,
• the criteria for making a MPN (`serious contravention' and knowledge of risk of breach),
• the relevance of the Commissioner's regulatory action policy (RAP),
• proportionality and the criteria for an assessment notice, and unfair process –
• All five appeals dismissed by Upper Tribunal.

20. COMMONWEALTH STUDY:
CYBERSECURITY & DISINFORMATION
● Brown, Ian, Chris Marsden, James Lee, Michael Veale [2020]
● Electoral Cybersecurity in the Commonwealth: A Good Practice Guide
● London: Commonwealth Secretariat, 162pp. at
● https://thecommonwealth.org/sites/default/files/inline/Commonwealth%20cybersecurity%20for%20elections%20guide.pdf
○ based on an in-depth questionnaire sent to all Commonwealth election management bodies;
○ research missions in Ghana, Pakistan, Trinidad and the UK; and
○ regional training workshops in Africa (Jo’burg), Asia-Pacific (Sydney) and Caribbean (Trinidad)
○ It is the chief outcome of the Strengthening Election Cybersecurity project that is part of the Commonwealth Cyber Capability Programme.
Implementation of the Commonwealth Cyber Declaration agreed by heads of government at their 2018 meeting in London.
○ The declaration commits “to a cyberspace that supports economic and social development and rights online; to build the foundations of an
effective national cybersecurity response; and to promote stability in cyberspace through international cooperation.’

21. LAUNCH FOR CWEALTH ELECTORAL CYBERSECURITY GUIDE
3 March 2020: technical systems, laws, policies,
● capabilities across the whole electoral cycle
● recommendations to national contexts
● to help professionals who run elections
Lead author Dr Ian Brown said:
○ “It’s really important electoral authorities build up their links with government agencies dealing with
cyber security, data protection, public procurement, to respond more effectively together.
○ “especially in the Caribbean and Pacific there are a number of small Commonwealth countries,
○ really helpful if electoral authorities can co-operate in terms of sharing training and learning,
○ thinking about collaborative procurement and sharing information about specific attacks on their
election infrastructures
○ because that will make the response of each country together much stronger.”

22. DID COMMONWEALTH MEMBERS ACT ON
RECOMMENDATIONS FOR CYBERSECURITY EXPERTISE?
● Kenya election declared 16 August 2022
● 4 out of 7 Electoral Commissioners refused to certify the incumbent ‘victory’
○ Commonwealth: problems “use of the Kenya Integrated Elections Management System
(KIEMS) kit, which is used to register and identify voters using biometrics”
https://thecommonwealth.org/news/kenya-elections-largely-peaceful-and-transparent-say-commonwealth-observers
● Official observers:
○ Commonwealth Observer Group –Bruce Golding, Former Prime Minister of Jamaica
○ EISA Election Observation Mission –Goodluck Ebele Jonathan, Former President of Nigeria
○ Joint AU-COMESA Election Mission –Dr. Ernest Bai Koroma, Former President of Sierra Leone
○ IGAD Election Observation Mission –Dr. Mulatu Teshome, Former President of Ethiopia
● Ethiopian People's Revolutionary Democratic Front – not elected, nominated
○ IRI/NDI Election Observation Mission –Joaquim Chissano, Former President of Mozambique
○ Carter Center Election Expert Mission – Ben Graham Jones, Team Leader
○ EU Election Observation Mission –Ivan Stefanec, Member of the European Parliament

23. COMMONWEALTH FINAL REPORT – AUGUST 2023
● Report officially conveyed to the Government of Kenya and relevant stakeholders,
○ builds on the findings of the interim statement
○ issued shortly after the elections on 11 August 2022
● The Group recommended implementation of key legislative provisions,
● Including 2013 Elections Campaign Financing Act.
● need for the IEBC to receive its funding earlier on in the electoral cycle,
● to allow it to adequately plan and undertake its electoral activities,
● including boundary delimitation, in accordance with its own desired timeframe
● https://production-new-commonwealth-files.s3.eu-west-2.amazonaws.com/s3fs-public/2023-
08/Kenya%20COG%20Report%20Final%20(2).pdf?VersionId=yj2INsQTT0LrdfixdctCsAgJw4yt3cpg

24. RECOMMENDS THAT CMCA SS22-23 BE REPEALED, LAW
ADDRESSING DISINFORMATION THAT DO NOT UNDULY
CURTAIL OTHER CONSTITUTIONAL FREEDOMS
Section 22 of the CMCA 2018 places limitations on Article 33 of the Constitution (freedom of expression)
● ‘in respect of the publication of false, misleading or fictitious data or information that is likely to propagate
● war; incite persons to violence; constitutes hate speech; advocates hatred that constitutes ethnic
● incitement, vilification of others or incitement to cause harm.’
In 2021, a bill was tabled in the National Assembly that sought to amend the CMCA
● to provide the National Computer and Cybercrimes Coordination Committee (NC4) with greater powers to
recommend websites that should be made inaccessible,
● and to prohibit the use of electronic mediums to promote terrorism, extreme religious or cult activities.
Media stakeholders’ concern with this law extended beyond freedom of expression,
● with some noting an increased tendency by the Government to use ‘security concerns’ –
● in this case, terrorism – as a pretext for such media freedom violations.
● raised concerns that efforts to amend the CMCA so close to the election
● represented an effort to stifle freedom of expression in the election year.

25. ALWAYS READ FOOTNOTE 58!
● “As noted, only around 12 million people (21%) of Kenyan citizens use social media.”
● www.statista.com/statistics/1029198/facebook-user-share-in-kenyaby-age/
Well, that’s just nonsense. Why did the Observer group write that claim?
Freedom House (2023) Freedom on the Net 2023: The Repressive Power of Artificial Intelligence
● “During Kenya’s August 2022 election,
● influencers gamed social media platforms’ trending functions to boost misleading political hashtags.
● “For instance, the hashtag #ChebukatiCannotBeTrusted sought to undermine the country’s independent electoral
authority by suggesting that its leader supported one presidential candidate over the others.
● “Similar networks of influencers were found to have coordinated disinformation campaigns against Kenyan
journalists, judges, and members of civil society.”
● https://freedomhouse.org/report/freedom-net/2023/repressive-power-artificial-intelligence

26. REAL PICTURE OF SOCIAL MEDIA USE IN KENYA
● Total Internet users were almost 50million in 2022!
● Communications Authority (CA) October-December 2022,
● mobile data/internet subscriptions 47.76Million,
● 66.8% mobile broadband.
● smartphones 60.2%,
● total population of 49.4 Million at the end of 2022
● https://www.kictanet.or.ke/state-of-internet-penetration-in-kenya/
● Social media use therefore 40million?

27. KENYA ENVIRONMENT VERY DIFFICULT BY JULY 2023
● “While economic concerns are the root of the conflict,
● it is troubling that the protests have seen an increase in the use of hate speech and
● we call on leaders of all sides of the political divide to deescalate the situation.”
● https://thecommonwealth.org/news/commonwealth-statement-recent-developments-kenya

28. SPECIFIC POLICY OPTIONS:
THE CO-REGULATORY TRIANGLE

29. NOTE: EU COMMISSIONER TO COMBAT
FOREIGN DISINFORMATION IS CZECH
• Prague Spring 1968 – she remembers
• Russian tanks occupying her country

30. 2018: FIVE RULE OF LAW
RECOMMENDATIONS
1. Media literacy and user choice
2. Strong human review and appeal processes where
AI is used
3. Independent appeal and audit of platforms
4. Standardizing notice and appeal procedures
Creating a multistakeholder body for appeals
5. Transparency in AI disinformation techniques

31. Option and form Typology of regulation Implications/Notes
0 Status quo Corporate Social Responsibility,
single-company initiatives
Note that enforcement of General Data Protection Regulation and the
proposed revised ePrivacy Regulation, plus agreed text for new AVMS
Directive, would all continue and likely expand
1 Non-audited
self-regulation
Industry code of practice,
transparency reports, self-
reporting
Corporate agreement on principles for common technical solutions and
Santa Clara Principles
2 Audited self-
regulation
European Code of Practice of Sept
2018; Global Network Initiative
published audit reports
Open interoperable publicly available standard e.g. commonly
engineered/designed standard for content removal to which platforms
could certify compliance
3 Formal self-
regulator
Powers to expel non-performing
members, Dispute Resolution
ruling/arbitration on cases
Commonly engineered standard for content filtering or algorithmic
moderation. Requirement for members of self-regulatory body to conform
to standard or prove equivalence. Particular focus on content ‘Put Back’
metrics and efficiency/effectiveness of appeal process
4 Co-regulation Industry code approved by
Parliament or regulator(s) with
statutory powers to supplant
Government-approved technical standard – for filtering or other forms of
moderation. Examples from broadcast and advertising regulation
5 Statutory
regulation
Formal regulation - tribunal with
judicial review
National Regulatory Agencies – though note many overlapping powers
between agencies on e.g. freedom of expression, electoral advertising and

32. 2021: TEN RECOMMENDATIONS
• 1.Electoral management boards (EMBs) should not request the operation of
Internet shutdowns during election periods,
• not objectively assessed a national emergency and sanctioned by a superior court.
• Such an injunction may be achieved with great speed, and
• the need for procedural legitimacy before such an extreme response is essential.
• 2.Governments should avoid shutdowns in response to disinformation concerns
• while ensuring false announcements are responded to
• where defamatory, fraudulent, or unjustifiably casting doubt on official EMB results.
• such action may not appear disinterested or legitimate prior to a court decision.

33. CHRISTCHURCH CALL 2ND ANNIVERSARY, UNITED
STATES JOINED BRINGING THE TOTAL TO 55 NATIONS
• 'Christchurch Call' response signatories issued this message:
• "We will improve transparency from Governments and companies on terrorist and violent extremist content, including
Government information on flagging and content removal requests; and increase the number and variety of companies
providing transparency reporting.
• We will work to improve the quality and content of that reporting over time;
• We will establish a multi-stakeholder process to ensure that transparency reporting across Government and industry is
responsive to the concerns of civil society participants and informative in demonstrating progress on the Call’s commitments;
• As a Community, we will devote resources towards building understanding of recommendation algorithms and user journeys,
including the role they may play in radicalisation or amplification of terrorist and violent extremist content;
• We have agreed to work together this year to design methods that can safely be used to build a better understanding of
algorithmic outcomes.
• This will help us address the question of amplification and identify more effective intervention points".
• Algorithmic analysis is thus now at the centre of the response.

34. “WE WILL BUILD UNDERSTANDING OF
RECOMMENDATION ALGORITHMS AND USER JOURNEYS”
• “including the role they may play in radicalisation or amplification of terrorist
• and violent extremist content;
• We have agreed to work together this year
• to design methods that can safely be
• used to build a better understanding of algorithmic outcomes.
• This will help us address the question of amplification and
• identify more effective intervention points"

35. ALGORITHMIC ANALYSIS IS AT THE CENTRE OF ANALYSIS
• Amplification?
• Should social media as default
• prevent live video from reaching a wide audience without prior approval?
• Does that remove the immediacy of e.g. TikTok?
• Isn’t that what broadcast rules are in place to do?
• 15 second delay for profanity/violence/banned speech

36. FORMER PM JACINTA ARDEN NOW A SPECIAL ENVOY ,
RECENTLY AT STANFORD ON CHRISTCHURCH CALL
• Jacinda Ardern Assembles Stanford Scholars for Discussion on Technology Governance and Regulation
• Led by former Prime Minister of New Zealand Rt. Hon. Dame Jacinda Ardern,
• a delegation from the Christchurch Call joined Stanford scholars
• to discuss how to address the challenges posed by emerging technologies.
• https://fsi.stanford.edu/news/special-envoy-jacinda-ardern-assembles-stanford-scholars-discussion-
technology-governance-and

37. BUT FOR NON-ENGLISH LANGUAGE CONTENT….
• Almost non-existent moderation – and
• none at all on WhatsApp
• which is where ‘angry uncle’ content is
• more likely to be spread in family groups
• Brazil 2018 was bad…2022 may be worse
• https://blogs.lse.ac.uk/polis/2018/10/27/2018-brazil-elections-the-power-of-social-media-and-the-
threat-to-journalism/

38. NEW EU LEGISLATION: 2022, ENFORCED AUGUST 2023
• 22 April 2022, European policymakers reached an agreement on the Digital Services Act.
• European Parliament approved the DSA along with the Digital Markets Act on 5 July 2022
• In September, the Council of the European Union (member states) is expected to formally adopt the DSA,
• Published in the Official Journal of the European Union
• comes into force 20 days after publication.
• apply fifteen months after coming into force or on 1 January 2024.
• Very Large Online Platforms (VLOPs) and search engines will need to comply with their obligations
• four months after they have been designated as such by the EU Commission.

39. DSA IMPLEMENTED IN PART:
25 AUGUST 2023

40. DSA ALREADY BEING FLOUTED BY X?
• European Commission, Directorate-General for
Communications Networks, Content and Technology,
• Digital Services Act – Application of the risk management
framework to Russian disinformation campaigns,
• Publications Office of European Union, 2023
https://data.europa.eu/doi/10.2759/764631

41. DIGITAL SERVICES ACT, RECITAL 68:
“IT IS APPROPRIATE THIS REGULATION IDENTIFY CERTAIN
AREAS OF CONSIDERATION FOR SUCH CODES OF CONDUCT”
• In particular, risk mitigation measures concerning
• specific types of illegal content should be explored via self- and co-regulatory agreements.
• Another area for consideration is the possible negative impacts of systemic risks on society and democracy,
• such as disinformation or manipulative and abusive activities.
• This includes coordinated operations aimed at amplifying information, including disinformation,
• such as the use of bots or fake accounts for the creation of fake or misleading information,
• sometimes with a purpose of obtaining economic gain,
• which are particularly harmful for vulnerable recipients of the service, such as children.

42. HOW AUSTRALIA COPED WITH
DISINFORMATION IN THE 2022 ELECTION
• Tom Rogers (Australian Election Commissioner) Official agreement with social media
• ACMA (2023) 2nd report on the Code of Practice on Disinformation
• https://www.acma.gov.au/sites/default/files/2023-
07/Digital%20platforms%20efforts%20under%20Code%20of%20Practice%20on%20Disinformation%20and%20Misinformation.pdf
• eSafety Commissioner has a guide on disinformation online.
• Marsden, C. and T. Meyer [2019] How can the law regulate removal of fake news?
• https://www.scl.org/articles/10425-how-can-the-law-regulate-removal-of-fake-news

43. 2022 AUSTRALIAN ELECTION WAS FAIR?
• Interference with the Trump (2016) election
• Muller Report to Department of Justice (2019) heavily redacted by Trump’s Attorney general Bill Barr – we’re
not sure what was in it
• Report On The Investigation Into Russian Interference In The 2016 Presidential Election
• 2020 Report more extensive SELECT COMMITTEE ON INTELLIGENCE ON RUSSIAN ACTIVE MEASURES
CAMPAIGNS AND INTERFERENCE IN THE 2016 U.S. ELECTION
• VOLUME 5: COUNTERINTELLIGENCE THREATS AND VULNERABILITIES
• https://www.intelligence.senate.gov/sites/default/files/documents/report_volume5.pdf
• 2016 Brexit referendum in the UK
• No Commons ‘pandemic of disinformation inquiry – left to Lord Puttnam in Lords
• https://committees.parliament.uk/committee/407/democracy-and-digital-technologies-committee/

44. CO-REGULATION OF AUSTRALIAN CODE OF PRACTICE
ON DISINFORMATION
• 2021 voluntary Code on Disinformation
• https://www.oecd.org/stories/dis-misinformation-hub/voluntary-code-of-practice-on-misinformation-and-disinformation-1fe0be59/
• 2023 amended law to make ACMA co-regulator of the Code?
• Communications Legislation Amendment (Combatting Misinformation and Disinformation) Bill 2023
• Department of Infrastructure, Transport, Regional Development, Communications and the Arts
• Consultation June-August. Bill in spring?
• Communications Legislation Amendment (Combatting Misinformation and Disinformation) Bill 2023—Guidance
Note (infrastructure.gov.au)

45. OFFICE OF THE E-SAFETY COMMISSIONER ESTABLISHED 2015
• to promote safety for children only
• Problem grew during the pandemic
• Affects all children
• sextortion of boys growing problem

46. ONLINE SAFETY ACT 2021 (CTH) TO COVER ADULTS:
• promoting online safety for all Australians
• complaints system for cyber bullying material targeted at an Australian child [s 30]
• complaints system for cyber-abuse material targeted at an Australian adult (that meets the threshold of
serious harm) [s 36]
• complaints and objections system for non-consensual sharing of intimate images [s 32]
• online content scheme (for illegal and restricted online content, including a complaints system) [ss 38-40]
• requiring internet service providers to block access to material showing abhorrent violent conduct (e.g.
terrorist acts) [s 39]
• issue civil penalties, enforceable undertakings, injunctions for breaches of the Act [Part 10, OSA 2021]

47. E-SAFETY COMMISSIONER CAN APPLY TO FEDERAL COURT
• person may be ordered to cease providing a
social media service
• if civil penalty provisions of the Act continue
to be breached,
• continued operation of the social media
service represents a significant community
safety risk [s.156 OSA 2021]

48. S.34 DSA COMPARABLE TO S.156 ONLINE SAFETY ACT?
• When does content or conduct cross the threshold,
• resulting in an
• “actual or foreseeable negative effect”
• severe enough to be considered systemic
• in relation to the risk factors specified in DSA?