Copyright Management in the Web 3

Copyright Management
in the Web 3.0
Semantic Web and Blockchain for Decentralized and Web-wide
Content Management in the Era of Social Media
Rosa Gil & Roberto García
Department of Commercial Law Seminar
Business School, University of Auckland
July 12th, 2018

GRIHO
Human-Computer Interaction
& Data Integration Research Group
Department of Computer Science and Industrial Engineering
Polytechnic Institute of Research and Innovation in Sustainability
Universitat de Lleida, Spain
The Interface adapts the
System to the User
From

Team Roberto García
Director
Associate Professor
Rosa Gil
Associate Professor
Juan Manuel Gimeno
Assistant Professor
Toni Granollers
Associate Professor
Marta Oliva
Associate Professor
Montserrat Sendín
Associate Professor
Juan Enrique Garrido
Visiting Professor
Jordi Virgili
PhD Student
part-time
Aitor Corchero
PhD Student
in-company (Eurecat)
Albert Berga
Researcher
Gerard Rovira
Researcher
MSc Student
Marc Ribalta
Researcher
BSc Student

Research Lines
Human-
Computer
Interaction
User
Experience
Data
Integration
(Big Data)
Information
Architecture
Web 3.0
(Semantic)
Knowledge
Engineering
Interaction
Paradigms
Usability and
Accessibility
Adaptative
User
Interfaces
User Centred
Design

Some Recent Projects and Research Lines
• InVID: In Video Veritas
Verification of Social Media Video Content for the News Industry
European Union’s Horizon 2020 – 2016-2018
• MedISys Media Monitoring for Plant Health Threats Identification and Reporting
European Food Safety Authority – 2013-2016
• MediaMixer
Community and Networking for the Remixing of Online Media
European Union’s FP7 – 2012-2014
• InDAGuS
Infrastructures for Sustainable Open Government Data with Geospatial Features
Spanish Government – 2013-2015
• Emotion Recognition using biometrics
• Ethics and Artificial Intelligence
• Web 3.0 – Semantic Web + Blockchain

Motivation
• Why copyright management on the Web is more important
than ever?
• And why Digital Rights Management isn’t enough?
• Why semantic technologies seem a good choice for Web-
scale copyright management?
• What can blockchain technologies, and particularly smart
contracts, contribute to Web-wide copyright management?

Scenario
• Web media business models go beyond digital version of “copy/distribute”
vinyl, cassette, CD, DVD,…
• New revenue streams:
• Growing: streaming (Spotify, Pandora, Netflix, Hulu…)
• Promising: big scale remixing (make money if others reuse your content),
User Generated Content (UGC)

Scenario
• UGC remixing copyrighted media
©
©

Scenario
AudioID ©
©VideoID
match
match
match

Scenario
“As of July 2016, YouTube has paid out two billion dollars to rightsholders
who have chosen to monetize claims since Content ID first launched in 2007”
YouTube by the numbers - https://www.youtube.com/yt/about/press/
©
Block
Choices:
Track
Monetize
match

Scenario
• What if more than one ownership claim?
• If disputed, no one can monetize
©© match match

Scenario
• Are we sure we can claim?
• Do we own that particular copy?
• In that territory?
• Also streaming on YouTube?
• Does the artist authorise YouTube?
• …
DRM  Copyright Management

Scenario
…1000s of pieces of registered content…
…1000s of videos on YouTube…
• Copyright Decision Support System:
• Disputes with Media.com on A and B, can we claim?
• Trade A for B with Media.com (both win, start to get revenue)
A B

Scenario
• Copyright Decision Support System requirements:
• Fine grained
• Scalable (largely automatized)
• Takes into account:
• Contracts
“…all rights on the live version but studio version just in the US.”
• Policies
“…artist does not want his music together with violent images”.
• Rights Expression Languages
DDEX metadata:
<UseType>OnDemandStream</UseType>
<TerritoryCode>Worldwide</TerritoryCode>
...

Media Fragment
Green Day’s Contract:
“Avoid
making available
our work
together with
war images”
Digital Operations
(DDEX)
“Bullet In A Bible”
– Green Day
<isrc>USREV0500293</isrc>
<DealTerms>
<ValidityPeriod>
<StartDate>2005-11-15</StartDate>
</ValidityPeriod>
<Usage>
<UseType>OnDemandStream
</UseType>
<DistributionChannelType>
Internet</DistributionChannelType>
</Usage>
<TerritoryCode>US</TerritoryCode>...
Monetize “Bullet in a
Bible” in YouTube.com?
Digital Operations
says YES but we
should check Green
Day’s contract…
Scenario

Digital Operations
(DDEX)
<DealTerms>
<ValidityPeriod>
<StartDate>2005-11-15
</StartDate>
</ValidityPeriod>
<Usage>
<UseType>AdSupportedStreaming
</UseType>
Internet
</Usage>
<TerritoryCode>US
</TerritoryCode>...
ddex:AdSupported
Streaming with
http://youtube.com
what
Isrc:USREV0500293 ?
Scenario

Digital Operations
(DDEX)
<DealTerms>
<ValidityPeriod>
</StartDate>
</ValidityPeriod>
<Usage>
</UseType>
Internet
</Usage>
<TerritoryCode>US
</TerritoryCode>...
ddex:AdSupported
Streaming
dbpedia:Green_Day
what
Isrc:USREV0500293
with
ddex:Internet
Rights Language
Mapping
ddex:AdSupported
Streaming with
http://youtube.com
author
what
Isrc:USREV0500293 ?
is a
Semantic
Reasoner
Scenario

“Avoid
making available
our work
together with
war content”
Digital Operations
(DDEX)
<DealTerms>
<ValidityPeriod>
</StartDate>
</ValidityPeriod>
<Usage>
</UseType>
Internet
</Usage>
<TerritoryCode>US
</TerritoryCode>...
ddex:AdSupported
Streaming
dbpedia:Green_Day
what
Isrc:USREV0500293
with
ddex:Internet
Rights Language
Mapping
Rights Builder
User Interface
ddex:AdSupported
Streaming with
http://youtube.com
author
what
Isrc:USREV0500293 ?
is a
Semantic
Reasoner
Scenario

“Avoid
making available
our work
together with
war content”
Digital Operations
(DDEX)
<DealTerms>
<ValidityPeriod>
</StartDate>
</ValidityPeriod>
<Usage>
</UseType>
Internet
</Usage>
<TerritoryCode>US
</TerritoryCode>...
dbpedia:Category:War
cro:MakeAvailable ddex:AdSupported
Streaming
what
schema:CreativeWork
author
dbpedia:Green_Day
part ofschema:CreativeWork
topic
what
Isrc:USREV0500293
with
ddex:Internet
Rights Language
Mapping
Rights Builder
User Interface
ddex:AdSupported
Streaming with
http://youtube.com
author
what
Isrc:USREV0500293
subClassOf
?
is a
Semantic
Reasoner

“Avoid
making available
our work
together with
war content”
Digital Operations
(DDEX)
“Bullet In A Bible” – Green Day
<DealTerms>
<ValidityPeriod>
</StartDate>
</ValidityPeriod>
<Usage>
</UseType>
Internet
</Usage>
<TerritoryCode>US
</TerritoryCode>...
part of
dbpedia:Category:Syrian_civil_war
Semantic
Media
Annotation
Seeing Syria's War Through the Lens
by SOPHIA JONES - October 09, 2012
http://www.npr.org
dbpedia:Category:War
topic
broader
cro:MakeAvailable ddex:AdSupported
Streaming
what
schema:CreativeWork
author
dbpedia:Green_Day
part ofschema:CreativeWork
topic
what
Isrc:USREV0500293
sameAs
with
ddex:Internet
Rights Language
Mapping
Rights Builder
User Interface
ddex:AdSupported
Streaming with
http://youtube.com
author
what
Isrc:USREV0500293
subClassOf
?
is a
Semantic
Reasoner

Proposal
• Post-DRM standardisation difficulties (e.g. ISO/IEC or W3C)
• Web open and heterogeneous
• Business models beyond copy and distribute
• DRM just about access control, Copyright is much more complex
• Requires higher level of abstraction (not bits or pixels)
• Concentrate on the concepts, not how they are expressed
formalise meaning  SEMANTICS
World Wide Web Consortium (W3C) - ODRL “Duplicate”
Reproduction Right
Copy
ISO/IEC MPEG-21 “Adapt”
Creative Commons “Reproduction”
Copyright

From Controlled Vocabularies…to Ontologies
INCREASE EXPRESSIVENESS
Features
Controlled
Vocabularies
Synonyms Taxonomies Thesaurus Ontologies
Control
Ambiguity X X X X
Control
Synonym X X X X
Hierarchical
Relations X X X
Associative
Relations X X
Custom
Relations X

Creation lifecycle
• Sample: broadcast of a serial adapted from a literary work
• A creator adapts the original literary work Victor Hugo’s “Les Miserables”, to
produce a serial. The resulting adaptation to a serial is realised as a script that
is performed by some actors, including Gerard Depardieu, and recorded into
a motion picture. This motion picture is finally broadcasted to users who can
tune the resulting communication.

Creation lifecycle
• Sample: webcast of a composition recording
• A composer writes down a music sheet for her new composition, which is
published as sheet prints. Moreover, it is played in a studio to record
a master which is later made available as a webcast.

IFLA's FRBR Creation Model
• International Federation of Library Associations and Institutions (IFLA)
Functional Requirements for Bibliographic Records (FRBR)
• Most commonly referenced creation model
• Work: abstract entity; no material object. Recognised through its realisations
or expressions
• Expression: the intellectual or artistic realisation of a Work
• In the form of alpha-numeric, musical, or choreographic notation, sound, image, object,
movement, etc., or any combination of such forms
• Manifestation: the physical embodiment of an Expression of a Work
• Manuscripts, books, periodicals, maps, posters, sound recordings, films,...
• Item: a single exemplar of a Manifestation
• Enables us to identify individual copies of a Manifestation separately

IFLA's FRBR Creation Model Limitations
• Other proposals, most based on FRBR, same problems
• Example: MPEG-21 Media Value Chain Ontology
Recording of the broadcast?
!
!?
?
?

Copyright Ontology
Creation Model
• Based on fundamental
ontological distinction
• Abstract: intangible
• Object: can be defined
independent of time
(includes digital objects)
• Process: happens,
temporal stages
(action, event,…)
Victor Hugo’s
Les Misérables
Abstract
Objects
Processes

Copyright Ontology
Creation Model
• Capable of modelling the
previous value chain
examples

Copyright Ontology
Action Model
• Also capture the dynamic
part of the value chain
• The actions performed
by value chain participants
• Plus consumer actions:
• Buy, Attend, Access,
Play, Tune,…
• Plus licensing actions:
• Agree/Disagree
• Transfer, Attribute,…
Victor Hugo’s
Les Misérables

Copyright Ontology
Action Model
Creator Actor Producer Broadcaster User
Motion PictureScript
Adaptation Performance
manifest perform record
Communication
broadcastadapt
Literary Work
tune

Copyright Ontology
Action Model
Role
Kind
Main Role Description
who schema:agent
The direct performer or driver of
the action (animate or
inanimate)
schema:participant
Other co-agents that participated
in the action indirectly, for
instance a recipient
what schema:object
The object upon which the action
is carried out
schema:result The result produced in the action
where schema:location Where an action takes place
schema:fromLocation
The original location of the object
or the agent before the action
schema:toLocation
The final location of the object or
the agent after the action
when schema:startTime
When the action started or the
time it is expected to start
schema:endTime
When the action finished or the
time it is expected to end
pointInTime
The point in time when the
action happens
duration
The amount of time the action
requires to complete
with schema:instrument
The object that helps the agent
perform the action
why aim
The reason or objective of the
action
how manner The way the action is carried out
if condition
Something that must hold or
happen before the action starts
then consequence
Something that must hold or
happen after the action is
completed
Who?
What?
When? Where?
What?
With?
• Model full details of an action
• Its dimensions like as a verb in a sentence:
who performs it, what is manipulated, when, where…

Copyright Ontology
Rights Model
• Basic Rights Model based on
generic set of rights inspired by
WIPO1 agreements
• Can fit more specific
rights systems
• Define restrictions on associated
copyright actions and dimensions
(who? what? when? how? …)
EU Copyright Directive
Publisher’s Rights?
1 World Intellectual Property Organization

Copyright Ontology
Rights Model
• Basic association of rights to
governed actions

Copyright Ontology
USA Music Rights
William Fisher (February, 2016) CopyrightX Lecture 3: The Subject Matter of Copyright, Slide 4. Harvard Law School. Licensed CC-BY 4.0
Agree
Economic
Rights
what
who
Composer Publisher
who
what Manifestation:
Composition
Transfer
who
recipient
condition
what Amount:
Currency

Copyright Ontology
USA Music Rights
Agree
Reproduction
Right
what
who
Publisher
who
what Manifestation:
Composition
Sheet Music
Printer
Distribution
Right
what
who
what Instance:
Sheet Music
result
condition

Copyright Ontology
USA Music Rights
Agree
Economic
Rights
what
who
Publisher
who
what Manifestation:
Composition
Foreign
Subpublisher
Territory:
New Zealand
where
condition

Copyright Ontology
USA Music Rights
Agree
Adaptation
Right
what
who
Publisher
who
what Manifestation:
Composition
Movie
Studio
Manifestation:
Movie Master
result
condition

Copyright Ontology
USA Music Rights
Agree Performwhat
who
what Manifestation:
Composition
Record
Company
Fixation
Right
what
who
what Performance:
Music
result
result
Performer
Recording:
Master
Reproduction
Right
what
who
what
Instance:
Copy result
condition

Copyright Ontology
USA Music Rights
• Streaming:
• Blanket License – Broadcast
• Non-interactive streams,
• Pandora, Sirius XM,…
• Mechanical License – Reproduction
• Interactive streams
• Spotify, Soundcloud, Apple Music,…

Copyright Ontology
USA Music Rights – Public Performance Right
William Fisher (March, 2015) CopyrightX Lecture 8: Rights of Distribution and Performance, Slide 8. Harvard Law School. Licensed CC-BY 4.0

Semantic Web Approach
• Capable of representing
rights statements but…
• Main limitations of the
Copyright Ontology:
• Trust on statements
• Incentives to use it to
make copyright statements
Kingsley Uyi Idehen (Jul, 2017) Semantic Web Layer Cake Tweak, Explained.
https://medium.com/openlink-software-blog/semantic-web-layer-cake-tweak-explained-6ba5c6ac3fab

Verification of Social Media Video Content
for the News Industry
Objective: identify truthful videos (not “fake news”)
for its reuse by the news industry
Universitat de Lleida
Participation: GRIHO & Law Department researchers
Role: Leading WP4 – Rights Management
H2020 ICT 19(b)-2015
Technologies for creative industries,
social media and convergence
InVID
• MediaMixer: video fragmentation, media fragment applications & industry outreach
Significant Infrastructure
Through its involvement in the aforementioned projects, the team has built an infrastructure of considerable
computational capacity (150+ cores, 900+GB RAM, 70+ TB storage) and developed a sophisticated
distributed architecture for data collection and indexing, as well as a variety of cutting edge data mining and
retrieval, video fragmentation and annotation, and machine learning algorithms. The team is therefore in
excellent position to support a wide range of data collection, mining, analysis and indexing needs within
InVID.
2 Modul Technology GmbH (MOD)
Organisation Profile
MODUL Technology GmbH is the non-profit technology and innovation company of MODUL University
Vienna, a private Austrian university founded in 2007. MODUL University Vienna is funded by the Austrian
Chamber of Commerce and its goal is to educate the next generation of business leaders in the domains of
new media, tourism, governance, sustainability and international management. The Technology GmbH
enables both university faculty (as technology experts) and hired researchers/developers to work on near-to-
market R&D with a close collaboration with the University itself (including sharing of MODUL University
IP) and support of existing and future University spin-offs (by agreements to license IP with
commercialisation potential to for-profit companies). Technology experts will contribute via the company to
TV InSight from the Department of New Media Technology, which conducts cross-disciplinary research on
knowledge acquisition, semantic Web annotation, human-computer interaction, data analytics, natural
language processing and multimedia description and linking.
MODUL Technology GmbH is 100% owned by MODUL University Vienna and shares IP from and to the
University. University faculty can participate in innovation projects through the company and high value IP
could be licensed to a new for-profit spin-off.
Role in the project
This proposal version was submitted by Vasileios Mezaris on 14/04/2015 16:45:55 CET . Issued by the Participant Portal Submission Service.
support the extraction of marketing information, and aid decision makers in optimizing their branding and
marketing strategies. (Swiss KTI | CHF 480,000 | 2013-2015 | www.htwchur.ch/comet)
Significant Infrastructure
webLyzard operates a modular, layered and highly scalable infrastructure that has proven its accuracy and
reliability over many years. The system architecture includes a portfolio of backend services including content
acquisition, document enrichment and annotation, filtering, service orchestration, load balancing, etc. Frontend
services include rendering and portlet synchronisation, data export in multiple formats, search and
crowdsourcing Application Programming Interfaces (APIs), visualization modules based on SVG and the D3.js.
These services operate in a virtualized Linux Ubuntu environment on top of Intel Xeon-based industry standard
servers, which is straightforward to scale up or migrate to different host environments. The system architecture
supports but does not require cloud-based solutions.
6 Condat AG (Condat)
The Condat AG is a medium sized company located in the centre of Berlin developing innovative solutions
for leading European companies of the Media/TV sector. Condat is one of the main German providers for
program planning, newsroom support, video on demand and media asset management for major public and
private TV-broadcasters (e.g. RBB, ARD, MDR, WDR, NDR, n-tv, arte, Deutsche Welle, ZDF). Condat has
participated (co-ordinating or member) in 8 EU-Projects from FP3 to FP7 as partner and coordinator. In
addition, Condat provides for TV broadcasters video on-demand via internet, integration of feedback and
content from social networks, advanced retrieval, and user profile evaluation. We apply semantic search
engines to retrieve and analyze large, heterogeneous data sources distributed over the network. Condat
covers the entire Plan – Build – Run cycle and offers scalable cloud based solutions. The development of
server and client side applications uses J2EE, Web- and Open Source – technology. Project and quality
management is certified according DIN EN ISO 9001:2000.
8 Agence France-Presse (AFP)
Agence France-Presse (AFP) is a global news agency, delivering fast, accurate, in-depth coverage of the
events shaping our world from wars and conflicts to politics, sports, entertainment and the latest
breakthroughs in health, science and technology. With 2,900 staff and stringers of 80 nationalities, spread
across 165 countries, AFP covers the world 24 hours a day in six languages, delivering the news in video,
text, pictures, multimedia and graphics. AFP produces roughly 5.000 text dispatches, 2.000 photos, 80
• Denis Teyssoum, “GLOCAL: Pro-am collaboration in the news production”, Proc. Workshop on
RecognisingandTrackingEvents ontheWeb and in RealLife, 6thHellenic Conference on Artificial
Intelligence(SETN2010),Athens,Greece.
RelevantProjects
• Glocal–FP7projectonorganizingmediabyeventswithlocalandaglobaldimensions.Content-based
imageretrievaltechniques wereusedamongothers duringtheproject tocluster picturesrelatedtothe
sameevent.
9 DeutscheWelle(DW)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 687786
– In Video Veritas
Jan 2016 - Dec 2018

InVID Copyright Management
• Support reuse request generation and negotiation
• User interface to model request terms as Copyright Ontology actions
• Content owners can respond with updated conditions
• Keep track of negotiation steps
• Check intended uses against existing Reuse Agreements
• Export agreement terms as semantic representations (JSON-LD)
• Register agreements in blockchain
• Independent parties monitor blockchain for agreements
• Load agreements into Semantic Repository
• Check intended uses against agreements
• Reasoning about reuse terms
Rights
Database
InVID Rights
Management
Semantic
Repository
Semantic
Copyright
Management
Blockchain Events
Agreements
Time Stamping
Accountability
Auditability
Tamper Proof
Blockchain

InVID and Blockchain
• Use Ethereum Smart Contracts
• Blockchain as a global shared computer
• Immutable transactions (executed in all nodes)
• Encode rules guaranteed to execute
• Smart contract to store semantic agreements
• Participants digitally sign them
• Identity management using uPort mobile app
• Optional: remuneration using cryptocurrency wallet
• Proof of existence of the agreement, timestamp
• Verifiable independently of InVID
https://www.uport.me
Icons by https://fontawesome.com/license

Future Work
• Smart contract for copyright registration
• Upload manifestation and facilitate content identification and disputes resolution
• Smart contract for usage tracking
• Register copyright use
• Track uses so they can be independently verified against registered agreements
• Creators can also keep contact with fans
• E.g.: loyalty programs based on tokens as proof of purchase
• Automatic royalties splitting based on chain of registered agreements
• Payments using cryptocurrency
• Smart contracts for dispute resolution
• Enable law experts participation through incentive and reputation mechanisms
Icons by https://fontawesome.com/license

Related Work
• Artists’ initiatives to regain connection
with fans and fair revenues
• Bjork, Imogen Heap, Pitbull,…
• Spotify, Pandora, Apple, YouTube,…
shield artists from fans and provide low revenues per stream
• YouTube 25% of music streaming, pays $1 per 1000 plays (Spotify $7) 1
• Some blockchain music start-ups
• Ujo Creator’s Portal, upload and purchase music using cryptocurrency
• Jaak global registry of copyright data curated by specialised guardians,
rewarded for the authority provided
1 The Washington Post (July, 2017) Why musicians are so angry at the world’s most popular music streaming service. http://wapo.st/2t09dM2
https://imogen2.surge.sh

Blockchain Can Legally Authenticate Evidence, Chinese Judge Rules
• A court in China's Hangzhou city has ruled that evidence authenticated with blockchain technology can be
presented in legal disputes. The judge commented:
• "The court thinks it should maintain an open and neutral stance on using blockchain to analyze individual
cases. We can't exclude it just because it's a complex technology. Nor can we lower the standard just
because it is tamper-proof and traceable. ... In this case, the usage of a third-party blockchain platform that
is reliable without conflict of interests provides the legal ground for proving the intellectual infringement."
• According to data from the court, the case was filed in January by Huatai Yimei, a Hangzhou-based media
company, against a Shenzhen-based technology firm for copyright infringement.
• During the legal procedure, the plaintiff showed screen-captured images of web pages and text that it
considered unauthorized usage by the Shenzhen company. Previously, Huatai Yimei had encoded the
images, website sources codes and other evidence through a third-party site named baoquan.com – a
blockchain-based evidence deposition platform – and attempted to use that evidence in the proceedings.
• Even if the disputed media assets should be taken down at a later stage, the court decided that evidence
stored on the blockchain is sufficient to be legally accepted by the court. As such, the judge ruled in favor of
the plaintiff.
• The decision marks one of the first officially sanctioned use cases for blockchain in legal proceedings.
Wolfie Zhao (Jun 28, 2018)
https://www.coindesk.com/blockchain-can-legally-authenticate-evidence-chinese-judge-rules/

Baidu Launches Blockchain-Based Image Rights Protection Platform
• Chinese internet search giant Baidu has launched a digital image property rights management
platform based on Blockchain, the company announced Wednesday, April 11.
• The service, called Totem, timestamps each submitted original image with a real-time identity
and other user data, storing it on a traceable and immutable Blockchain.
• According to the Baidu’s announcement, the company’s existing artificial intelligence capacities –
including image analysis and semantic understanding – will feed into the construction of unique
image tags, with the underlying Blockchain system allowing circulating images to be traced,
reproduced, and monitored in order to substantiate or refute allegations of copyright
infringement.
• Baidu announced that traditional stock photo services – including Getty Images partner, Visual
China Group – have already moved onto the platform.
• Baidu’s Totem comes hot on the trail of Kodak’s own image rights management Blockchain
platform, announced in Jan. 2018. It similarly uses a digital ledger to establish rights ownership,
and also offers a KODAKCoin token for the online photo community to pay for and receive
dividends on their image licenses.
Marie Huillet (Apr 12, 2018)
https://cointelegraph.com/news/chinas-google-baidu-launches-blockchain-based-image-rights-protection-platform

Beyond: Organisations in the Blockchain
https://blog.aragon.one

Beyond: life forms in the Blockchain?
http://www.okhaos.com/plantoids/

Thank you for your attention
Questions?
Contact:
roberto.garcia@udl.cat rgil@diei.udl.cat
http://rhizomik.net/~roberto/ http://rhizomik.net/~rosa/
Slides:
https://www.slideshare.net/rogargon/copyright-management-in-the-web-3

Copyright Management in the Web 3

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Copyright Management in the Web 3

Similar to Copyright Management in the Web 3 (20)

More from Roberto García

More from Roberto García (20)

Recently uploaded

Recently uploaded (13)

Copyright Management in the Web 3