SlideShare a Scribd company logo
1 of 48
Download to read offline
A Network Analysis of Dutch
Regulations
Using the MetaLex Document Server
Rinke Hoekstra
VU University Amsterdam/University of Amsterdam
rinke.hoekstra@vu.nl
A Network Analysis of Dutch Regulations - Using the MetaLex Document Server
Licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Who knows what Linked Data is?
Linked Data
• Everything gets a URI
• Everything is connected as much as possible
• Everything is assigned to a category (including connections)
• When we know two things are the same, we say so
• We might even publish our data on the Web (but don’t have to)
+ =
Why this paper?
Why this paper?
Because we can ... and because it is interesting!
Why this paper?
Because we can ... and because it is interesting!
Demonstrate
the power of Linked Data
for Network Analysis
Do the outcomes
of Network Analysis correspond to
our intuitions?
Can we apply
the results of Network
Analysis in practice?
Does the Law
resemble the Web?
A Mathematical Approach to the Study of the United States Code
Michael J. Bommarito II
Center for the Study of Complex Systems, University of Michigan, Ann Arbor
Department of Political Science, University of Michigan, Ann Arbor
Department of Financial Engineering, University of Michigan, Ann Arbor
Daniel M. Katz
Center for the Study of Complex Systems, University of Michigan, Ann Arbor
Department of Political Science, University of Michigan, Ann Arbor
University of Michigan Law School
Abstract
The United States Code (Code) is a document containing over 22 million words that represents a large and important
source of Federal statutory law. Scholars and policy advocates often discuss the direction and magnitude of changes
in various aspects of the Code. However, few have mathematically formalized the notions behind these discussions or
directly measured the resulting representations. This paper addresses the current state of the literature in two ways.
First, we formalize a representation of the United States Code as the union of a hierarchical network and a citation
network over vertices containing the language of the Code. This representation reflects the fact that the Code is a
hierarchically organized document containing language and explicit citations between provisions. Second, we use this
formalization to measure aspects of the Code as codified in October 2008, November 2009, and March 2010. These
measurements allow for a characterization of the actual changes in the Code over time. Our findings indicate that in
the recent past, the Code has grown in its amount of structure, interdependence, and language.
Keywords: United States Code, hierarchical network, citation network, language, computational legal studies
1. Formalizing the Code
The United States Code (Code) is a document con-
taining over 22 million words that represents a large and
important source of Federal statutory law. The Code is a
concise and conveniently organized compilation of all rati-
fied legislation and treaties, and is often the first source of
information for lawyers, judges, and legal academics.1
The
Code is compiled through a process known as codification,
which is carried out by the O ce of the Law Revision
Counsel (LRC), an organization within the U.S. House
of Representatives. The LRC’s goal in this codification
process is to transform the incremental and chronological
Statutes at Large into the Code, a current snapshot of the
law that is organized into hierarchical categories.2
Email addresses: mjbommar@umich.edu (Michael J. Bommarito
II), dmartink@umich.edu (Daniel M. Katz)
1The complete set of all ratified legislation and treaties is known
as the Statutes at Large. As a legal technicality, the Code is only
prima facie evidence of Federal law. In the event of a discrepancy,
the Statutes at Large are the final authority. Furthermore, additional
sources such as the Code of Federal Regulations contains clarifica-
tions issued by other Federal agencies or bodies.
22 U.S.C. §285- §285g outlines the purpose, policy and functions
of the O ce of Law Revision Counsel.
This hierarchical organization is an important quali-
tative feature of the Code. At the first level of the hi-
erarchy, the Code is divided into 49 titles that represent
the broadest categories of law. Well-known titles include
the Tax Code, formally known as Title 26 - Internal Rev-
enue Code, Title 20 - Education, and Title 18 - Crimes
and Criminal Punishment. Each title is also hierarchi-
cally subdivided into some combination of subtitles, chap-
ters, subchapters, parts, subparts, sections, subsections,
paragraphs, subparagraphs, clauses, or subclauses. Out of
these vertex types, only sections, subsections, paragraphs,
subparagraphs, clauses and subclauses can contain the ac-
tual substantive legal text.
The text within these vertices can also contain explicit
citations to other portions of the Code. These citations
may be used to reference definitions, highlight qualifying
conditions, or point to well-understood processes. It is
critical to recognize that these citations are not restricted
by the organizational hierarchy. For example, sections
within Title 26, the Tax Code, can and do contain ci-
tations to Title 18, the Criminal Code. Thus, though the
LRC attempts to organize the Code into a cleanly divided
hierarchical network, there is also a citation network em-
Preprint submitted to Elsevier March 23, 2010
arXiv:1003.4146v1[cs.IR]22Mar2010
Related Work
A Mathematical Approach to the Study of the United States Code
Michael J. Bommarito II
Center for the Study of Complex Systems, University of Michigan, Ann Arbor
Department of Political Science, University of Michigan, Ann Arbor
Department of Financial Engineering, University of Michigan, Ann Arbor
Daniel M. Katz
Center for the Study of Complex Systems, University of Michigan, Ann Arbor
Department of Political Science, University of Michigan, Ann Arbor
University of Michigan Law School
Abstract
The United States Code (Code) is a document containing over 22 million words that represents a large and important
source of Federal statutory law. Scholars and policy advocates often discuss the direction and magnitude of changes
in various aspects of the Code. However, few have mathematically formalized the notions behind these discussions or
directly measured the resulting representations. This paper addresses the current state of the literature in two ways.
First, we formalize a representation of the United States Code as the union of a hierarchical network and a citation
network over vertices containing the language of the Code. This representation reflects the fact that the Code is a
hierarchically organized document containing language and explicit citations between provisions. Second, we use this
formalization to measure aspects of the Code as codified in October 2008, November 2009, and March 2010. These
measurements allow for a characterization of the actual changes in the Code over time. Our findings indicate that in
the recent past, the Code has grown in its amount of structure, interdependence, and language.
Keywords: United States Code, hierarchical network, citation network, language, computational legal studies
1. Formalizing the Code
The United States Code (Code) is a document con-
taining over 22 million words that represents a large and
important source of Federal statutory law. The Code is a
concise and conveniently organized compilation of all rati-
fied legislation and treaties, and is often the first source of
information for lawyers, judges, and legal academics.1
The
Code is compiled through a process known as codification,
which is carried out by the O ce of the Law Revision
Counsel (LRC), an organization within the U.S. House
of Representatives. The LRC’s goal in this codification
process is to transform the incremental and chronological
Statutes at Large into the Code, a current snapshot of the
law that is organized into hierarchical categories.2
Email addresses: mjbommar@umich.edu (Michael J. Bommarito
II), dmartink@umich.edu (Daniel M. Katz)
1The complete set of all ratified legislation and treaties is known
as the Statutes at Large. As a legal technicality, the Code is only
prima facie evidence of Federal law. In the event of a discrepancy,
the Statutes at Large are the final authority. Furthermore, additional
sources such as the Code of Federal Regulations contains clarifica-
tions issued by other Federal agencies or bodies.
22 U.S.C. §285- §285g outlines the purpose, policy and functions
of the O ce of Law Revision Counsel.
This hierarchical organization is an important quali-
tative feature of the Code. At the first level of the hi-
erarchy, the Code is divided into 49 titles that represent
the broadest categories of law. Well-known titles include
the Tax Code, formally known as Title 26 - Internal Rev-
enue Code, Title 20 - Education, and Title 18 - Crimes
and Criminal Punishment. Each title is also hierarchi-
cally subdivided into some combination of subtitles, chap-
ters, subchapters, parts, subparts, sections, subsections,
paragraphs, subparagraphs, clauses, or subclauses. Out of
these vertex types, only sections, subsections, paragraphs,
subparagraphs, clauses and subclauses can contain the ac-
tual substantive legal text.
The text within these vertices can also contain explicit
citations to other portions of the Code. These citations
may be used to reference definitions, highlight qualifying
conditions, or point to well-understood processes. It is
critical to recognize that these citations are not restricted
by the organizational hierarchy. For example, sections
within Title 26, the Tax Code, can and do contain ci-
tations to Title 18, the Criminal Code. Thus, though the
LRC attempts to organize the Code into a cleanly divided
hierarchical network, there is also a citation network em-
Preprint submitted to Elsevier March 23, 2010
arXiv:1003.4146v1[cs.IR]22Mar2010
Related Work
Disclaimer: we’re just scratching the surface!
Regulation A at time t1
Article A1 at time t1
reference to Article B1 at time t2
Regulation B at time t2
Article B1 at time t2
Regulation A
Article A1
realizes
realizes
Regulation B
Article B1
realizes
realizes
Linked Data in MDS
Linked Data in MDS
Regulation A at time t1
Article A1 at time t1
reference to Article B1
Regulation B at time t2
Article B1 at time t2
Regulation A
Article A1
realizes
realizes
Regulation B
Article B1
realizes
realizes
Linked Data in MDS
Every regulation expression is an RDF Named Graph
Regulation A at time t1
Article A1 at time t1
reference to Article B1
Regulation B at time t2
Article B1 at time t2
Regulation A
Article A1
realizes
realizes
Regulation B
Article B1
realizes
realizes
Linked Data in MDS
Every regulation expression is an RDF Named Graph
Regulation A at time t1
Article A1 at time t1
reference to Article B1
Regulation B at time t2
Article B1 at time t2
Regulation A
Article A1
realizes
realizes
Regulation B
Article B1
realizes
realizes
A citation from an expression level to a work is
represented as a citation between works
Two citations to separate expressions of a single
work will only be counted once
The highest level of detail of a citation is the
article level
Document Level
Text
Document Level
Text
Article Level
http://doc.metalex.eu/id/BWBR0002634/hoofdstuk/XII/artikel/33/lid/2/al/2/extref/1/nl/2012-01-01
http://doc.metalex.eu/id/BWBR0002634/hoofdstuk/XII/artikel/33
Article Level
http://doc.metalex.eu/id/BWBR0002634/hoofdstuk/XII/artikel/33/lid/2/al/2/extref/1/nl/2012-01-01
http://doc.metalex.eu/id/BWBR0002634/hoofdstuk/XII/artikel/33
Article Level
http://doc.metalex.eu/id/BWBR0002634/hoofdstuk/XII/artikel/33/lid/2/al/2/extref/1/nl/2012-01-01
http://doc.metalex.eu/id/BWBR0002634/hoofdstuk/XII/artikel/33
Measure Document Article Factor
Number of nodes 14935 64018 4.286
Number of edges 33819 80082 2.368
Average degree 2.264 1.251 0.553
Avg. Weighted degree 9.117 3.749 0.411
Network diameter 16 8 0.5
Average path length 5.479 1.316 0.240
Avg. Clustering Coe cient 0.09 0.0021 0.023
Connected Components 492 7262 14.76
Number of SCC’s 14019 63303 4.516
Table 1. Network properties
1.1 Regulations in the MDS
Regulations in the MDS are represented using the CEN MetaLex ontology, an au-
tomatically generated ontology of the Basiswettenbestand (BWB, the database
Measure Document Article Factor
Number of nodes 14935 64018 4.286
Number of edges 33819 80082 2.368
Average degree 2.264 1.251 0.553
Avg. Weighted degree 9.117 3.749 0.411
Network diameter 16 8 0.5
Average path length 5.479 1.316 0.240
Avg. Clustering Coe cient 0.09 0.0021 0.023
Connected Components 492 7262 14.76
Number of SCC’s 14019 63303 4.516
Table 1. Network properties
1.1 Regulations in the MDS
Regulations in the MDS are represented using the CEN MetaLex ontology, an au-
tomatically generated ontology of the Basiswettenbestand (BWB, the database
that all citations are from one leaf of the hierarchical net-
work to another leaf of the hierarchical network. This is
a consequence of our above choice to combine any subsec-
tions and their text into their respective section vertices.
Date |V (G)| |E(G)| |V (GC
)| |E(GC
)|
Oct 2008 57947 140154 33503 82208
Nov 2009 59684 144758 34473 85075
Mar 2010 59988 145908 34674 85921
Table 1: Summary of Code snapshots. 10
−4
10
−3
10
−2
Pr(X≥x) 2008
2009
2010
Document Level
Spatial planning, environment (17%)
Administrative law, privacy, elections (11%)
Welfare, social insurance (9%)
Taxes (6%)
Education (6%)
Penal code, procedural law, constitution (5.5%)
Table 2. Top-10 Betweenness Centrality
Rank Name Value
1 Algemene wet bestuursrecht 7007741
2 Wet milieubeheer 2172441
3 Besluit omgevingsrecht 1667495
4 Besluit algemene regels voor inrichtingen milieubeheer 948497
5 Wet op de economische delicten 770968
6 Wetboek van Burgerlijke Rechtsvordering 696456
7 Wet op het hoger onderwijs en wetenschappelijk onderzoek 687873
8 Wet op het financieel toezicht 664934
9 Algemene douanewet 616671
10 Circulaire bodemsanering 2009 561465
Table 3. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht 0.0152
2 Wetboek van Burgerlijke Rechtsvordering 0.0117
3 Wet gemeenschappelijke regelingen” 0.00803
4 Wet openbaarheid van bestuur 0.00785
5 Wetboek van Strafvordering 0.00723
2 Wet milieubeheer 2172441
3 Besluit omgevingsrecht 1667495
4 Besluit algemene regels voor inrichtingen milieubeheer 948497
5 Wet op de economische delicten 770968
6 Wetboek van Burgerlijke Rechtsvordering 696456
7 Wet op het hoger onderwijs en wetenschappelijk onderzoek 687873
8 Wet op het financieel toezicht 664934
9 Algemene douanewet 616671
10 Circulaire bodemsanering 2009 561465
Table 3. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht 0.0152
2 Wetboek van Burgerlijke Rechtsvordering 0.0117
3 Wet gemeenschappelijke regelingen” 0.00803
4 Wet openbaarheid van bestuur 0.00785
5 Wetboek van Strafvordering 0.00723
6 Grondwet 0.00712
7 Algemene termijnenwet 0.00668
8 Wet structuur uitvoeringsorganisatie werk en inkomen 0.00638
9 Kaderwet zelfstandige bestuursorganen 0.00623
10 Vreemdelingenwet 2000 0.00597
Table 4. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 426
2 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984 336
3 Archiefwet 1995 278
4 Werkloosheidswet 265
5 Wet op de arbeidsongeschiktheidsverzekering 236
6 Ziektewet 220
1 Algemene wet bestuursrecht 0.0152
2 Wetboek van Burgerlijke Rechtsvordering 0.0117
3 Wet gemeenschappelijke regelingen” 0.00803
4 Wet openbaarheid van bestuur 0.00785
5 Wetboek van Strafvordering 0.00723
6 Grondwet 0.00712
7 Algemene termijnenwet 0.00668
8 Wet structuur uitvoeringsorganisatie werk en inkomen 0.00638
9 Kaderwet zelfstandige bestuursorganen 0.00623
10 Vreemdelingenwet 2000 0.00597
Table 4. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 426
2 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984 336
3 Archiefwet 1995 278
4 Werkloosheidswet 265
5 Wet op de arbeidsongeschiktheidsverzekering 236
6 Ziektewet 220
7 Warenwet 210
8 Algemene Wet Bijzondere Ziektekosten 207
9 Wet op het voortgezet onderwijs 204
10 Zorgverzekeringswet 119
Hubs & Authorities
Algemene wet bestuursrecht 0.008432735
Werkloosheidswet 0.0062917816
Ziektewet 0.0062043956
Wet op de arbeidsongeschiktheidsverzekering 0.005985931
Warenwet 0.0050683795
Wet milieubeheer 0.0050683795
Wet structuur uitvoeringsorganisatie werk en inkomen 0.0049809935
Wet op het voortgezet onderwijs 0.004849915
Wet werk en inkomen naar arbeidsvermogen 0.0046314504
Hubs & Authorities
Algemene wet bestuursrecht 0.008432735
Werkloosheidswet 0.0062917816
Ziektewet 0.0062043956
Wet op de arbeidsongeschiktheidsverzekering 0.005985931
Warenwet 0.0050683795
Wet milieubeheer 0.0050683795
Wet structuur uitvoeringsorganisatie werk en inkomen 0.0049809935
Wet op het voortgezet onderwijs 0.004849915
Wet werk en inkomen naar arbeidsvermogen 0.0046314504
Algemene wet bestuursrecht 0.010200669
Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984 0.008050645
Archiefwet 1995 0.0066650743
Werkloosheidswet 0.006354515
Wet op de arbeidsongeschiktheidsverzekering 0.0056617297
Ziektewet 0.0052795033
Warenwet 0.0050406116
Algemene Wet Bijzondere Ziektekosten 0.004968944
Wet op het voortgezet onderwijs 0.0048972764
Evaluation
315 Wikipedia pages about Dutch regulations
196 candidate pages that have a potential match
(compared to 14935 work level identifiers)
Evaluation
315 Wikipedia pages about Dutch regulations
Table 5. Comparision to laws listed on Wikipedia
Measure Recall (0.5x) Recall (1x) Recall (2x) Precision (2x) F-Score (2x)
PageRank 0.311 0.444 0.617 0.309 0.411
Indegree 0.296 0.474 0.612 0.306 0.408
Degree 0.260 0.423 0.551 0.276 0.367
Betweenness 0.240 0.388 0.536 0.268 0.357
a very imprecise measure, but it indicates at least a basic notion of importance.
We normalized all names to lower case, and removed those laws from the target
set Wikipedia that do not have a direct match with any regulation in our set.
196 candidate pages that have a potential match
(compared to 14935 work level identifiers)
Article Level
Table 6. Top-10 Betweenness Centrality
Rank Name Value
1 Wet op de omzetbelasting 1968, Bijlage I 829.5
2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili-
taire luchthavens), Artikel X
504
3 Warenwet, Artikel 1 492.5
4 Warenwet, Artikel 3 436.5
5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423
6 Pensioenwet BES, Artikel 1 373
7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362
8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319
9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder-
landse Antillen, Artikel I
306
10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens,
Artikel 2
294
Table 7. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht” 0.00262
2 Archiefwet 1995 0.00242
3 Wet op het financieel toezicht 0.00196
4 Zorgverzekeringswet 0.00175
5 Algemene Wet Bijzondere Ziektekosten 0.00167
6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162
7 Wet op het voortgezet onderwijs 0.00159
8 Wet bescherming persoonsgegevens 0.00150
9 Wet op de omzetbelasting 1968 0.00148
10 Werkloosheidswet 0.00147
Table 8. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 558
2 Werkloosheidswet 453
3 Wet op de arbeidsongeschiktheidsverzekering 453
4 Ziektewet 493
5 Archiefwet 1995 398
6 Wet op het voortgezet onderwijs 364
7 Wet op het financieel toezicht 361
Table 6. Top-10 Betweenness Centrality
Rank Name Value
1 Wet op de omzetbelasting 1968, Bijlage I 829.5
2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili-
taire luchthavens), Artikel X
504
3 Warenwet, Artikel 1 492.5
4 Warenwet, Artikel 3 436.5
5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423
6 Pensioenwet BES, Artikel 1 373
7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362
8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319
9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder-
landse Antillen, Artikel I
306
10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens,
Artikel 2
294
Table 7. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht” 0.00262
2 Archiefwet 1995 0.00242
3 Wet op het financieel toezicht 0.00196
4 Zorgverzekeringswet 0.00175
5 Algemene Wet Bijzondere Ziektekosten 0.00167
6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162
7 Wet op het voortgezet onderwijs 0.00159
8 Wet bescherming persoonsgegevens 0.00150
9 Wet op de omzetbelasting 1968 0.00148
10 Werkloosheidswet 0.00147
Table 8. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 558
2 Werkloosheidswet 453
3 Wet op de arbeidsongeschiktheidsverzekering 453
4 Ziektewet 493
5 Archiefwet 1995 398
6 Wet op het voortgezet onderwijs 364
7 Wet op het financieel toezicht 361
8 Algemene Wet Bijzondere Ziektekosten 342
9 Wet werk in inkomen naar arbeidsvermogen 327
10 Zorgverzekeringswet 326
Table 6. Top-10 Betweenness Centrality
Rank Name Value
1 Wet op de omzetbelasting 1968, Bijlage I 829.5
2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili-
taire luchthavens), Artikel X
504
3 Warenwet, Artikel 1 492.5
4 Warenwet, Artikel 3 436.5
5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423
6 Pensioenwet BES, Artikel 1 373
7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362
8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319
9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder-
landse Antillen, Artikel I
306
10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens,
Artikel 2
294
Table 7. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht” 0.00262
2 Archiefwet 1995 0.00242
3 Wet op het financieel toezicht 0.00196
4 Zorgverzekeringswet 0.00175
5 Algemene Wet Bijzondere Ziektekosten 0.00167
6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162
7 Wet op het voortgezet onderwijs 0.00159
8 Wet bescherming persoonsgegevens 0.00150
9 Wet op de omzetbelasting 1968 0.00148
10 Werkloosheidswet 0.00147
Table 8. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 558
2 Werkloosheidswet 453
3 Wet op de arbeidsongeschiktheidsverzekering 453
4 Ziektewet 493
5 Archiefwet 1995 398
6 Wet op het voortgezet onderwijs 364
7 Wet op het financieel toezicht 361
8 Algemene Wet Bijzondere Ziektekosten 342
9 Wet werk in inkomen naar arbeidsvermogen 327
10 Zorgverzekeringswet 326
Table 6. Top-10 Betweenness Centrality
Rank Name Value
1 Wet op de omzetbelasting 1968, Bijlage I 829.5
2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili-
taire luchthavens), Artikel X
504
3 Warenwet, Artikel 1 492.5
4 Warenwet, Artikel 3 436.5
5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423
6 Pensioenwet BES, Artikel 1 373
7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362
8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319
9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder-
landse Antillen, Artikel I
306
10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens,
Artikel 2
294
Table 7. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht” 0.00262
2 Archiefwet 1995 0.00242
3 Wet op het financieel toezicht 0.00196
4 Zorgverzekeringswet 0.00175
5 Algemene Wet Bijzondere Ziektekosten 0.00167
6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162
7 Wet op het voortgezet onderwijs 0.00159
8 Wet bescherming persoonsgegevens 0.00150
9 Wet op de omzetbelasting 1968 0.00148
10 Werkloosheidswet 0.00147
Table 8. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 558
2 Werkloosheidswet 453
3 Wet op de arbeidsongeschiktheidsverzekering 453
4 Ziektewet 493
5 Archiefwet 1995 398
6 Wet op het voortgezet onderwijs 364
7 Wet op het financieel toezicht 361
Table 6. Top-10 Betweenness Centrality
Rank Name Value
1 Wet op de omzetbelasting 1968, Bijlage I 829.5
2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili-
taire luchthavens), Artikel X
504
3 Warenwet, Artikel 1 492.5
4 Warenwet, Artikel 3 436.5
5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423
6 Pensioenwet BES, Artikel 1 373
7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362
8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319
9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder-
landse Antillen, Artikel I
306
10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens,
Artikel 2
294
Table 7. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht” 0.00262
2 Archiefwet 1995 0.00242
3 Wet op het financieel toezicht 0.00196
4 Zorgverzekeringswet 0.00175
5 Algemene Wet Bijzondere Ziektekosten 0.00167
6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162
7 Wet op het voortgezet onderwijs 0.00159
8 Wet bescherming persoonsgegevens 0.00150
9 Wet op de omzetbelasting 1968 0.00148
10 Werkloosheidswet 0.00147
Table 8. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 558
2 Werkloosheidswet 453
3 Wet op de arbeidsongeschiktheidsverzekering 453
4 Ziektewet 493
5 Archiefwet 1995 398
6 Wet op het voortgezet onderwijs 364
7 Wet op het financieel toezicht 361
8 Algemene Wet Bijzondere Ziektekosten 342
9 Wet werk in inkomen naar arbeidsvermogen 327
10 Zorgverzekeringswet 326
Table 6. Top-10 Betweenness Centrality
Rank Name Value
1 Wet op de omzetbelasting 1968, Bijlage I 829.5
2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili-
taire luchthavens), Artikel X
504
3 Warenwet, Artikel 1 492.5
4 Warenwet, Artikel 3 436.5
5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423
6 Pensioenwet BES, Artikel 1 373
7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362
8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319
9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder-
landse Antillen, Artikel I
306
10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens,
Artikel 2
294
Table 7. Top-10 PageRank
Rank Name Value
1 Algemene wet bestuursrecht” 0.00262
2 Archiefwet 1995 0.00242
3 Wet op het financieel toezicht 0.00196
4 Zorgverzekeringswet 0.00175
5 Algemene Wet Bijzondere Ziektekosten 0.00167
6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162
7 Wet op het voortgezet onderwijs 0.00159
8 Wet bescherming persoonsgegevens 0.00150
9 Wet op de omzetbelasting 1968 0.00148
10 Werkloosheidswet 0.00147
Table 8. Top-10 Indegree
Rank Name Value
1 Algemene wet bestuursrecht 558
2 Werkloosheidswet 453
3 Wet op de arbeidsongeschiktheidsverzekering 453
4 Ziektewet 493
5 Archiefwet 1995 398
6 Wet op het voortgezet onderwijs 364
7 Wet op het financieel toezicht 361
8 Algemene Wet Bijzondere Ziektekosten 342
9 Wet werk in inkomen naar arbeidsvermogen 327
10 Zorgverzekeringswet 326
Measure Article
PageRank 0.6996583
Degree 2.896214
Indegree 2.19982
Betweenness 3.658579
Table 10. Indegree per part of the Algemene wet bestuursrecht (AWB)
Part Indegree
Algemene wet bestuursrecht 558
Chapter 9 52
Chapter 6 48
Chapter 8 37
Chapter 7 33
Chapter 10 9
Chapter 5 9
Appendix 2 5
Chapter 3 4
Chapter 4 4
Chapter 2 3
Appendix 3 3
4 Is the Law like the Web?
Both the law and the Web are man-made networks of interlinked documents. It
is a valid question to ask whether the graph properties of both networks resemble
eachother. Or, put in another way, is the distribution of information within the
body of Dutch regulations specific for the domain of law, or is it similar to the
Does Law resemble the Web?
• The Web is a scale free network, ultra small world
• The Web contains a single giant SCC
would be interesting to see whether citations to articles indicate e.g. a high
representation of definitions in the cited article [5].
Secondly, we compared network properties of the document- and article level
networks to that of the Web, and concluded that both networks are scale free, but
the connectedness of regulations is much lower than that of the Web. The lower
connectedness makes it easier to distinguish modules in the set of regulations. It
would be very interesting to see how the results of generic module recognition
algorithms correspond to actual topics in legislation.
Acknowledgments
This publication was supported by the Dutch national program COMMIT.
References
1. Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Ra-
jagopalan, Raymie Stata, Andrew Tomkins, and Janet Wiener. Graph structure in
the web. In Proceedings of the 9th international World Wide Web conference on
Computer networks : the international journal of computer and telecommunications
netowrking, pages 309–320, Amsterdam, The Netherlands, The Netherlands, 2000.
North-Holland Publishing Co.
2. Rinke Hoekstra. The MetaLex Document Server - Legal Documents as Versioned
Linked Data. In Harith Alani and Jamie Tailor, editors, Proceedings of the 10th
International Semantic Web Conference (ISWC 2011), page 16. Springer, 2011.
3. Rinke Hoekstra. A Network Analysis of Dutch Regulations. Figshare.com, 2013.
doi:10.6084/m9.figshare.689880.
4. Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank
citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford
Does Law resemble the Web?
• The Web is a scale free network, ultra small world
• The Web contains a single giant SCC
would be interesting to see whether citations to articles indicate e.g. a high
representation of definitions in the cited article [5].
Secondly, we compared network properties of the document- and article level
networks to that of the Web, and concluded that both networks are scale free, but
the connectedness of regulations is much lower than that of the Web. The lower
connectedness makes it easier to distinguish modules in the set of regulations. It
would be very interesting to see how the results of generic module recognition
algorithms correspond to actual topics in legislation.
Acknowledgments
This publication was supported by the Dutch national program COMMIT.
References
1. Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Ra-
jagopalan, Raymie Stata, Andrew Tomkins, and Janet Wiener. Graph structure in
the web. In Proceedings of the 9th international World Wide Web conference on
Computer networks : the international journal of computer and telecommunications
netowrking, pages 309–320, Amsterdam, The Netherlands, The Netherlands, 2000.
North-Holland Publishing Co.
2. Rinke Hoekstra. The MetaLex Document Server - Legal Documents as Versioned
Linked Data. In Harith Alani and Jamie Tailor, editors, Proceedings of the 10th
International Semantic Web Conference (ISWC 2011), page 16. Springer, 2011.
3. Rinke Hoekstra. A Network Analysis of Dutch Regulations. Figshare.com, 2013.
doi:10.6084/m9.figshare.689880.
4. Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank
citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford
Type # Largest >1 #
Document 14019 816 74
Article 63303 12 501
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server
A Network Analysis of Dutch Regulations - Using the Metalex Document Server

More Related Content

More from Rinke Hoekstra

Provenance and Reuse of Open Data (PILOD 2.0 June 2014)
Provenance and Reuse of Open Data (PILOD 2.0 June 2014)Provenance and Reuse of Open Data (PILOD 2.0 June 2014)
Provenance and Reuse of Open Data (PILOD 2.0 June 2014)Rinke Hoekstra
 
Prov-O-Viz: Interactive Provenance Visualization
Prov-O-Viz: Interactive Provenance VisualizationProv-O-Viz: Interactive Provenance Visualization
Prov-O-Viz: Interactive Provenance VisualizationRinke Hoekstra
 
Linkitup: Link Discovery for Research Data
Linkitup: Link Discovery for Research DataLinkitup: Link Discovery for Research Data
Linkitup: Link Discovery for Research DataRinke Hoekstra
 
Linked (Open) Data - But what does it buy me?
Linked (Open) Data - But what does it buy me?Linked (Open) Data - But what does it buy me?
Linked (Open) Data - But what does it buy me?Rinke Hoekstra
 
Linked Science - Building a Web of Research Data
Linked Science - Building a Web of Research DataLinked Science - Building a Web of Research Data
Linked Science - Building a Web of Research DataRinke Hoekstra
 
Semantic Representations for Research
Semantic Representations for ResearchSemantic Representations for Research
Semantic Representations for ResearchRinke Hoekstra
 
A Slightly Different Web of Data
A Slightly Different Web of DataA Slightly Different Web of Data
A Slightly Different Web of DataRinke Hoekstra
 
The Knowledge Reengineering Bottleneck
The Knowledge Reengineering BottleneckThe Knowledge Reengineering Bottleneck
The Knowledge Reengineering BottleneckRinke Hoekstra
 
Concept- en Definitie Extractie
Concept- en Definitie ExtractieConcept- en Definitie Extractie
Concept- en Definitie ExtractieRinke Hoekstra
 
SIKS 2011 Semantic Web Languages
SIKS 2011 Semantic Web LanguagesSIKS 2011 Semantic Web Languages
SIKS 2011 Semantic Web LanguagesRinke Hoekstra
 
Querying the Web of Data
Querying the Web of DataQuerying the Web of Data
Querying the Web of DataRinke Hoekstra
 
History of Knowledge Representation (SIKS Course 2010)
History of Knowledge Representation (SIKS Course 2010)History of Knowledge Representation (SIKS Course 2010)
History of Knowledge Representation (SIKS Course 2010)Rinke Hoekstra
 
Making Sense of Design Patterns
Making Sense of Design PatternsMaking Sense of Design Patterns
Making Sense of Design PatternsRinke Hoekstra
 
Publicatie van Linked Open Overheids Data
Publicatie van Linked Open Overheids DataPublicatie van Linked Open Overheids Data
Publicatie van Linked Open Overheids DataRinke Hoekstra
 
ODaF 2010 Linked Data in the Netherlands
ODaF 2010 Linked Data in the NetherlandsODaF 2010 Linked Data in the Netherlands
ODaF 2010 Linked Data in the NetherlandsRinke Hoekstra
 
Overzicht BEST Project - NWO Site Visit
Overzicht BEST Project - NWO Site VisitOverzicht BEST Project - NWO Site Visit
Overzicht BEST Project - NWO Site VisitRinke Hoekstra
 
Semantic Modelling using Semantic Web Technology
Semantic Modelling using Semantic Web TechnologySemantic Modelling using Semantic Web Technology
Semantic Modelling using Semantic Web TechnologyRinke Hoekstra
 
BestPortal: Lessons Learned in Lightweight Semantic Access to Court Proceedings
BestPortal: Lessons Learned in Lightweight Semantic Access to Court ProceedingsBestPortal: Lessons Learned in Lightweight Semantic Access to Court Proceedings
BestPortal: Lessons Learned in Lightweight Semantic Access to Court ProceedingsRinke Hoekstra
 

More from Rinke Hoekstra (20)

Provenance and Reuse of Open Data (PILOD 2.0 June 2014)
Provenance and Reuse of Open Data (PILOD 2.0 June 2014)Provenance and Reuse of Open Data (PILOD 2.0 June 2014)
Provenance and Reuse of Open Data (PILOD 2.0 June 2014)
 
Prov-O-Viz: Interactive Provenance Visualization
Prov-O-Viz: Interactive Provenance VisualizationProv-O-Viz: Interactive Provenance Visualization
Prov-O-Viz: Interactive Provenance Visualization
 
Linkitup: Link Discovery for Research Data
Linkitup: Link Discovery for Research DataLinkitup: Link Discovery for Research Data
Linkitup: Link Discovery for Research Data
 
Linked (Open) Data - But what does it buy me?
Linked (Open) Data - But what does it buy me?Linked (Open) Data - But what does it buy me?
Linked (Open) Data - But what does it buy me?
 
Linked Science - Building a Web of Research Data
Linked Science - Building a Web of Research DataLinked Science - Building a Web of Research Data
Linked Science - Building a Web of Research Data
 
COMMIT/VIVO
COMMIT/VIVOCOMMIT/VIVO
COMMIT/VIVO
 
Semantic Representations for Research
Semantic Representations for ResearchSemantic Representations for Research
Semantic Representations for Research
 
A Slightly Different Web of Data
A Slightly Different Web of DataA Slightly Different Web of Data
A Slightly Different Web of Data
 
The Knowledge Reengineering Bottleneck
The Knowledge Reengineering BottleneckThe Knowledge Reengineering Bottleneck
The Knowledge Reengineering Bottleneck
 
Linked Census Data
Linked Census DataLinked Census Data
Linked Census Data
 
Concept- en Definitie Extractie
Concept- en Definitie ExtractieConcept- en Definitie Extractie
Concept- en Definitie Extractie
 
SIKS 2011 Semantic Web Languages
SIKS 2011 Semantic Web LanguagesSIKS 2011 Semantic Web Languages
SIKS 2011 Semantic Web Languages
 
Querying the Web of Data
Querying the Web of DataQuerying the Web of Data
Querying the Web of Data
 
History of Knowledge Representation (SIKS Course 2010)
History of Knowledge Representation (SIKS Course 2010)History of Knowledge Representation (SIKS Course 2010)
History of Knowledge Representation (SIKS Course 2010)
 
Making Sense of Design Patterns
Making Sense of Design PatternsMaking Sense of Design Patterns
Making Sense of Design Patterns
 
Publicatie van Linked Open Overheids Data
Publicatie van Linked Open Overheids DataPublicatie van Linked Open Overheids Data
Publicatie van Linked Open Overheids Data
 
ODaF 2010 Linked Data in the Netherlands
ODaF 2010 Linked Data in the NetherlandsODaF 2010 Linked Data in the Netherlands
ODaF 2010 Linked Data in the Netherlands
 
Overzicht BEST Project - NWO Site Visit
Overzicht BEST Project - NWO Site VisitOverzicht BEST Project - NWO Site Visit
Overzicht BEST Project - NWO Site Visit
 
Semantic Modelling using Semantic Web Technology
Semantic Modelling using Semantic Web TechnologySemantic Modelling using Semantic Web Technology
Semantic Modelling using Semantic Web Technology
 
BestPortal: Lessons Learned in Lightweight Semantic Access to Court Proceedings
BestPortal: Lessons Learned in Lightweight Semantic Access to Court ProceedingsBestPortal: Lessons Learned in Lightweight Semantic Access to Court Proceedings
BestPortal: Lessons Learned in Lightweight Semantic Access to Court Proceedings
 

A Network Analysis of Dutch Regulations - Using the Metalex Document Server

  • 1. A Network Analysis of Dutch Regulations Using the MetaLex Document Server Rinke Hoekstra VU University Amsterdam/University of Amsterdam rinke.hoekstra@vu.nl A Network Analysis of Dutch Regulations - Using the MetaLex Document Server Licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
  • 2.
  • 3.
  • 4.
  • 5. Who knows what Linked Data is?
  • 6. Linked Data • Everything gets a URI • Everything is connected as much as possible • Everything is assigned to a category (including connections) • When we know two things are the same, we say so • We might even publish our data on the Web (but don’t have to) + =
  • 8. Why this paper? Because we can ... and because it is interesting!
  • 9. Why this paper? Because we can ... and because it is interesting! Demonstrate the power of Linked Data for Network Analysis Do the outcomes of Network Analysis correspond to our intuitions? Can we apply the results of Network Analysis in practice? Does the Law resemble the Web?
  • 10. A Mathematical Approach to the Study of the United States Code Michael J. Bommarito II Center for the Study of Complex Systems, University of Michigan, Ann Arbor Department of Political Science, University of Michigan, Ann Arbor Department of Financial Engineering, University of Michigan, Ann Arbor Daniel M. Katz Center for the Study of Complex Systems, University of Michigan, Ann Arbor Department of Political Science, University of Michigan, Ann Arbor University of Michigan Law School Abstract The United States Code (Code) is a document containing over 22 million words that represents a large and important source of Federal statutory law. Scholars and policy advocates often discuss the direction and magnitude of changes in various aspects of the Code. However, few have mathematically formalized the notions behind these discussions or directly measured the resulting representations. This paper addresses the current state of the literature in two ways. First, we formalize a representation of the United States Code as the union of a hierarchical network and a citation network over vertices containing the language of the Code. This representation reflects the fact that the Code is a hierarchically organized document containing language and explicit citations between provisions. Second, we use this formalization to measure aspects of the Code as codified in October 2008, November 2009, and March 2010. These measurements allow for a characterization of the actual changes in the Code over time. Our findings indicate that in the recent past, the Code has grown in its amount of structure, interdependence, and language. Keywords: United States Code, hierarchical network, citation network, language, computational legal studies 1. Formalizing the Code The United States Code (Code) is a document con- taining over 22 million words that represents a large and important source of Federal statutory law. The Code is a concise and conveniently organized compilation of all rati- fied legislation and treaties, and is often the first source of information for lawyers, judges, and legal academics.1 The Code is compiled through a process known as codification, which is carried out by the O ce of the Law Revision Counsel (LRC), an organization within the U.S. House of Representatives. The LRC’s goal in this codification process is to transform the incremental and chronological Statutes at Large into the Code, a current snapshot of the law that is organized into hierarchical categories.2 Email addresses: mjbommar@umich.edu (Michael J. Bommarito II), dmartink@umich.edu (Daniel M. Katz) 1The complete set of all ratified legislation and treaties is known as the Statutes at Large. As a legal technicality, the Code is only prima facie evidence of Federal law. In the event of a discrepancy, the Statutes at Large are the final authority. Furthermore, additional sources such as the Code of Federal Regulations contains clarifica- tions issued by other Federal agencies or bodies. 22 U.S.C. §285- §285g outlines the purpose, policy and functions of the O ce of Law Revision Counsel. This hierarchical organization is an important quali- tative feature of the Code. At the first level of the hi- erarchy, the Code is divided into 49 titles that represent the broadest categories of law. Well-known titles include the Tax Code, formally known as Title 26 - Internal Rev- enue Code, Title 20 - Education, and Title 18 - Crimes and Criminal Punishment. Each title is also hierarchi- cally subdivided into some combination of subtitles, chap- ters, subchapters, parts, subparts, sections, subsections, paragraphs, subparagraphs, clauses, or subclauses. Out of these vertex types, only sections, subsections, paragraphs, subparagraphs, clauses and subclauses can contain the ac- tual substantive legal text. The text within these vertices can also contain explicit citations to other portions of the Code. These citations may be used to reference definitions, highlight qualifying conditions, or point to well-understood processes. It is critical to recognize that these citations are not restricted by the organizational hierarchy. For example, sections within Title 26, the Tax Code, can and do contain ci- tations to Title 18, the Criminal Code. Thus, though the LRC attempts to organize the Code into a cleanly divided hierarchical network, there is also a citation network em- Preprint submitted to Elsevier March 23, 2010 arXiv:1003.4146v1[cs.IR]22Mar2010 Related Work
  • 11. A Mathematical Approach to the Study of the United States Code Michael J. Bommarito II Center for the Study of Complex Systems, University of Michigan, Ann Arbor Department of Political Science, University of Michigan, Ann Arbor Department of Financial Engineering, University of Michigan, Ann Arbor Daniel M. Katz Center for the Study of Complex Systems, University of Michigan, Ann Arbor Department of Political Science, University of Michigan, Ann Arbor University of Michigan Law School Abstract The United States Code (Code) is a document containing over 22 million words that represents a large and important source of Federal statutory law. Scholars and policy advocates often discuss the direction and magnitude of changes in various aspects of the Code. However, few have mathematically formalized the notions behind these discussions or directly measured the resulting representations. This paper addresses the current state of the literature in two ways. First, we formalize a representation of the United States Code as the union of a hierarchical network and a citation network over vertices containing the language of the Code. This representation reflects the fact that the Code is a hierarchically organized document containing language and explicit citations between provisions. Second, we use this formalization to measure aspects of the Code as codified in October 2008, November 2009, and March 2010. These measurements allow for a characterization of the actual changes in the Code over time. Our findings indicate that in the recent past, the Code has grown in its amount of structure, interdependence, and language. Keywords: United States Code, hierarchical network, citation network, language, computational legal studies 1. Formalizing the Code The United States Code (Code) is a document con- taining over 22 million words that represents a large and important source of Federal statutory law. The Code is a concise and conveniently organized compilation of all rati- fied legislation and treaties, and is often the first source of information for lawyers, judges, and legal academics.1 The Code is compiled through a process known as codification, which is carried out by the O ce of the Law Revision Counsel (LRC), an organization within the U.S. House of Representatives. The LRC’s goal in this codification process is to transform the incremental and chronological Statutes at Large into the Code, a current snapshot of the law that is organized into hierarchical categories.2 Email addresses: mjbommar@umich.edu (Michael J. Bommarito II), dmartink@umich.edu (Daniel M. Katz) 1The complete set of all ratified legislation and treaties is known as the Statutes at Large. As a legal technicality, the Code is only prima facie evidence of Federal law. In the event of a discrepancy, the Statutes at Large are the final authority. Furthermore, additional sources such as the Code of Federal Regulations contains clarifica- tions issued by other Federal agencies or bodies. 22 U.S.C. §285- §285g outlines the purpose, policy and functions of the O ce of Law Revision Counsel. This hierarchical organization is an important quali- tative feature of the Code. At the first level of the hi- erarchy, the Code is divided into 49 titles that represent the broadest categories of law. Well-known titles include the Tax Code, formally known as Title 26 - Internal Rev- enue Code, Title 20 - Education, and Title 18 - Crimes and Criminal Punishment. Each title is also hierarchi- cally subdivided into some combination of subtitles, chap- ters, subchapters, parts, subparts, sections, subsections, paragraphs, subparagraphs, clauses, or subclauses. Out of these vertex types, only sections, subsections, paragraphs, subparagraphs, clauses and subclauses can contain the ac- tual substantive legal text. The text within these vertices can also contain explicit citations to other portions of the Code. These citations may be used to reference definitions, highlight qualifying conditions, or point to well-understood processes. It is critical to recognize that these citations are not restricted by the organizational hierarchy. For example, sections within Title 26, the Tax Code, can and do contain ci- tations to Title 18, the Criminal Code. Thus, though the LRC attempts to organize the Code into a cleanly divided hierarchical network, there is also a citation network em- Preprint submitted to Elsevier March 23, 2010 arXiv:1003.4146v1[cs.IR]22Mar2010 Related Work Disclaimer: we’re just scratching the surface!
  • 12. Regulation A at time t1 Article A1 at time t1 reference to Article B1 at time t2 Regulation B at time t2 Article B1 at time t2 Regulation A Article A1 realizes realizes Regulation B Article B1 realizes realizes Linked Data in MDS
  • 13. Linked Data in MDS Regulation A at time t1 Article A1 at time t1 reference to Article B1 Regulation B at time t2 Article B1 at time t2 Regulation A Article A1 realizes realizes Regulation B Article B1 realizes realizes
  • 14. Linked Data in MDS Every regulation expression is an RDF Named Graph Regulation A at time t1 Article A1 at time t1 reference to Article B1 Regulation B at time t2 Article B1 at time t2 Regulation A Article A1 realizes realizes Regulation B Article B1 realizes realizes
  • 15. Linked Data in MDS Every regulation expression is an RDF Named Graph Regulation A at time t1 Article A1 at time t1 reference to Article B1 Regulation B at time t2 Article B1 at time t2 Regulation A Article A1 realizes realizes Regulation B Article B1 realizes realizes A citation from an expression level to a work is represented as a citation between works Two citations to separate expressions of a single work will only be counted once The highest level of detail of a citation is the article level
  • 21. Measure Document Article Factor Number of nodes 14935 64018 4.286 Number of edges 33819 80082 2.368 Average degree 2.264 1.251 0.553 Avg. Weighted degree 9.117 3.749 0.411 Network diameter 16 8 0.5 Average path length 5.479 1.316 0.240 Avg. Clustering Coe cient 0.09 0.0021 0.023 Connected Components 492 7262 14.76 Number of SCC’s 14019 63303 4.516 Table 1. Network properties 1.1 Regulations in the MDS Regulations in the MDS are represented using the CEN MetaLex ontology, an au- tomatically generated ontology of the Basiswettenbestand (BWB, the database
  • 22. Measure Document Article Factor Number of nodes 14935 64018 4.286 Number of edges 33819 80082 2.368 Average degree 2.264 1.251 0.553 Avg. Weighted degree 9.117 3.749 0.411 Network diameter 16 8 0.5 Average path length 5.479 1.316 0.240 Avg. Clustering Coe cient 0.09 0.0021 0.023 Connected Components 492 7262 14.76 Number of SCC’s 14019 63303 4.516 Table 1. Network properties 1.1 Regulations in the MDS Regulations in the MDS are represented using the CEN MetaLex ontology, an au- tomatically generated ontology of the Basiswettenbestand (BWB, the database that all citations are from one leaf of the hierarchical net- work to another leaf of the hierarchical network. This is a consequence of our above choice to combine any subsec- tions and their text into their respective section vertices. Date |V (G)| |E(G)| |V (GC )| |E(GC )| Oct 2008 57947 140154 33503 82208 Nov 2009 59684 144758 34473 85075 Mar 2010 59988 145908 34674 85921 Table 1: Summary of Code snapshots. 10 −4 10 −3 10 −2 Pr(X≥x) 2008 2009 2010
  • 23. Document Level Spatial planning, environment (17%) Administrative law, privacy, elections (11%) Welfare, social insurance (9%) Taxes (6%) Education (6%) Penal code, procedural law, constitution (5.5%)
  • 24. Table 2. Top-10 Betweenness Centrality Rank Name Value 1 Algemene wet bestuursrecht 7007741 2 Wet milieubeheer 2172441 3 Besluit omgevingsrecht 1667495 4 Besluit algemene regels voor inrichtingen milieubeheer 948497 5 Wet op de economische delicten 770968 6 Wetboek van Burgerlijke Rechtsvordering 696456 7 Wet op het hoger onderwijs en wetenschappelijk onderzoek 687873 8 Wet op het financieel toezicht 664934 9 Algemene douanewet 616671 10 Circulaire bodemsanering 2009 561465 Table 3. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht 0.0152 2 Wetboek van Burgerlijke Rechtsvordering 0.0117 3 Wet gemeenschappelijke regelingen” 0.00803 4 Wet openbaarheid van bestuur 0.00785 5 Wetboek van Strafvordering 0.00723
  • 25. 2 Wet milieubeheer 2172441 3 Besluit omgevingsrecht 1667495 4 Besluit algemene regels voor inrichtingen milieubeheer 948497 5 Wet op de economische delicten 770968 6 Wetboek van Burgerlijke Rechtsvordering 696456 7 Wet op het hoger onderwijs en wetenschappelijk onderzoek 687873 8 Wet op het financieel toezicht 664934 9 Algemene douanewet 616671 10 Circulaire bodemsanering 2009 561465 Table 3. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht 0.0152 2 Wetboek van Burgerlijke Rechtsvordering 0.0117 3 Wet gemeenschappelijke regelingen” 0.00803 4 Wet openbaarheid van bestuur 0.00785 5 Wetboek van Strafvordering 0.00723 6 Grondwet 0.00712 7 Algemene termijnenwet 0.00668 8 Wet structuur uitvoeringsorganisatie werk en inkomen 0.00638 9 Kaderwet zelfstandige bestuursorganen 0.00623 10 Vreemdelingenwet 2000 0.00597 Table 4. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 426 2 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984 336 3 Archiefwet 1995 278 4 Werkloosheidswet 265 5 Wet op de arbeidsongeschiktheidsverzekering 236 6 Ziektewet 220
  • 26. 1 Algemene wet bestuursrecht 0.0152 2 Wetboek van Burgerlijke Rechtsvordering 0.0117 3 Wet gemeenschappelijke regelingen” 0.00803 4 Wet openbaarheid van bestuur 0.00785 5 Wetboek van Strafvordering 0.00723 6 Grondwet 0.00712 7 Algemene termijnenwet 0.00668 8 Wet structuur uitvoeringsorganisatie werk en inkomen 0.00638 9 Kaderwet zelfstandige bestuursorganen 0.00623 10 Vreemdelingenwet 2000 0.00597 Table 4. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 426 2 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984 336 3 Archiefwet 1995 278 4 Werkloosheidswet 265 5 Wet op de arbeidsongeschiktheidsverzekering 236 6 Ziektewet 220 7 Warenwet 210 8 Algemene Wet Bijzondere Ziektekosten 207 9 Wet op het voortgezet onderwijs 204 10 Zorgverzekeringswet 119
  • 27. Hubs & Authorities Algemene wet bestuursrecht 0.008432735 Werkloosheidswet 0.0062917816 Ziektewet 0.0062043956 Wet op de arbeidsongeschiktheidsverzekering 0.005985931 Warenwet 0.0050683795 Wet milieubeheer 0.0050683795 Wet structuur uitvoeringsorganisatie werk en inkomen 0.0049809935 Wet op het voortgezet onderwijs 0.004849915 Wet werk en inkomen naar arbeidsvermogen 0.0046314504
  • 28. Hubs & Authorities Algemene wet bestuursrecht 0.008432735 Werkloosheidswet 0.0062917816 Ziektewet 0.0062043956 Wet op de arbeidsongeschiktheidsverzekering 0.005985931 Warenwet 0.0050683795 Wet milieubeheer 0.0050683795 Wet structuur uitvoeringsorganisatie werk en inkomen 0.0049809935 Wet op het voortgezet onderwijs 0.004849915 Wet werk en inkomen naar arbeidsvermogen 0.0046314504 Algemene wet bestuursrecht 0.010200669 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984 0.008050645 Archiefwet 1995 0.0066650743 Werkloosheidswet 0.006354515 Wet op de arbeidsongeschiktheidsverzekering 0.0056617297 Ziektewet 0.0052795033 Warenwet 0.0050406116 Algemene Wet Bijzondere Ziektekosten 0.004968944 Wet op het voortgezet onderwijs 0.0048972764
  • 29. Evaluation 315 Wikipedia pages about Dutch regulations 196 candidate pages that have a potential match (compared to 14935 work level identifiers)
  • 30. Evaluation 315 Wikipedia pages about Dutch regulations Table 5. Comparision to laws listed on Wikipedia Measure Recall (0.5x) Recall (1x) Recall (2x) Precision (2x) F-Score (2x) PageRank 0.311 0.444 0.617 0.309 0.411 Indegree 0.296 0.474 0.612 0.306 0.408 Degree 0.260 0.423 0.551 0.276 0.367 Betweenness 0.240 0.388 0.536 0.268 0.357 a very imprecise measure, but it indicates at least a basic notion of importance. We normalized all names to lower case, and removed those laws from the target set Wikipedia that do not have a direct match with any regulation in our set. 196 candidate pages that have a potential match (compared to 14935 work level identifiers)
  • 32. Table 6. Top-10 Betweenness Centrality Rank Name Value 1 Wet op de omzetbelasting 1968, Bijlage I 829.5 2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili- taire luchthavens), Artikel X 504 3 Warenwet, Artikel 1 492.5 4 Warenwet, Artikel 3 436.5 5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423 6 Pensioenwet BES, Artikel 1 373 7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362 8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319 9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder- landse Antillen, Artikel I 306 10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens, Artikel 2 294 Table 7. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht” 0.00262 2 Archiefwet 1995 0.00242 3 Wet op het financieel toezicht 0.00196 4 Zorgverzekeringswet 0.00175 5 Algemene Wet Bijzondere Ziektekosten 0.00167 6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162 7 Wet op het voortgezet onderwijs 0.00159 8 Wet bescherming persoonsgegevens 0.00150 9 Wet op de omzetbelasting 1968 0.00148 10 Werkloosheidswet 0.00147 Table 8. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 558 2 Werkloosheidswet 453 3 Wet op de arbeidsongeschiktheidsverzekering 453 4 Ziektewet 493 5 Archiefwet 1995 398 6 Wet op het voortgezet onderwijs 364 7 Wet op het financieel toezicht 361 Table 6. Top-10 Betweenness Centrality Rank Name Value 1 Wet op de omzetbelasting 1968, Bijlage I 829.5 2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili- taire luchthavens), Artikel X 504 3 Warenwet, Artikel 1 492.5 4 Warenwet, Artikel 3 436.5 5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423 6 Pensioenwet BES, Artikel 1 373 7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362 8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319 9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder- landse Antillen, Artikel I 306 10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens, Artikel 2 294 Table 7. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht” 0.00262 2 Archiefwet 1995 0.00242 3 Wet op het financieel toezicht 0.00196 4 Zorgverzekeringswet 0.00175 5 Algemene Wet Bijzondere Ziektekosten 0.00167 6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162 7 Wet op het voortgezet onderwijs 0.00159 8 Wet bescherming persoonsgegevens 0.00150 9 Wet op de omzetbelasting 1968 0.00148 10 Werkloosheidswet 0.00147 Table 8. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 558 2 Werkloosheidswet 453 3 Wet op de arbeidsongeschiktheidsverzekering 453 4 Ziektewet 493 5 Archiefwet 1995 398 6 Wet op het voortgezet onderwijs 364 7 Wet op het financieel toezicht 361 8 Algemene Wet Bijzondere Ziektekosten 342 9 Wet werk in inkomen naar arbeidsvermogen 327 10 Zorgverzekeringswet 326 Table 6. Top-10 Betweenness Centrality Rank Name Value 1 Wet op de omzetbelasting 1968, Bijlage I 829.5 2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili- taire luchthavens), Artikel X 504 3 Warenwet, Artikel 1 492.5 4 Warenwet, Artikel 3 436.5 5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423 6 Pensioenwet BES, Artikel 1 373 7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362 8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319 9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder- landse Antillen, Artikel I 306 10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens, Artikel 2 294 Table 7. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht” 0.00262 2 Archiefwet 1995 0.00242 3 Wet op het financieel toezicht 0.00196 4 Zorgverzekeringswet 0.00175 5 Algemene Wet Bijzondere Ziektekosten 0.00167 6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162 7 Wet op het voortgezet onderwijs 0.00159 8 Wet bescherming persoonsgegevens 0.00150 9 Wet op de omzetbelasting 1968 0.00148 10 Werkloosheidswet 0.00147 Table 8. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 558 2 Werkloosheidswet 453 3 Wet op de arbeidsongeschiktheidsverzekering 453 4 Ziektewet 493 5 Archiefwet 1995 398 6 Wet op het voortgezet onderwijs 364 7 Wet op het financieel toezicht 361 8 Algemene Wet Bijzondere Ziektekosten 342 9 Wet werk in inkomen naar arbeidsvermogen 327 10 Zorgverzekeringswet 326
  • 33. Table 6. Top-10 Betweenness Centrality Rank Name Value 1 Wet op de omzetbelasting 1968, Bijlage I 829.5 2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili- taire luchthavens), Artikel X 504 3 Warenwet, Artikel 1 492.5 4 Warenwet, Artikel 3 436.5 5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423 6 Pensioenwet BES, Artikel 1 373 7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362 8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319 9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder- landse Antillen, Artikel I 306 10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens, Artikel 2 294 Table 7. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht” 0.00262 2 Archiefwet 1995 0.00242 3 Wet op het financieel toezicht 0.00196 4 Zorgverzekeringswet 0.00175 5 Algemene Wet Bijzondere Ziektekosten 0.00167 6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162 7 Wet op het voortgezet onderwijs 0.00159 8 Wet bescherming persoonsgegevens 0.00150 9 Wet op de omzetbelasting 1968 0.00148 10 Werkloosheidswet 0.00147 Table 8. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 558 2 Werkloosheidswet 453 3 Wet op de arbeidsongeschiktheidsverzekering 453 4 Ziektewet 493 5 Archiefwet 1995 398 6 Wet op het voortgezet onderwijs 364 7 Wet op het financieel toezicht 361 Table 6. Top-10 Betweenness Centrality Rank Name Value 1 Wet op de omzetbelasting 1968, Bijlage I 829.5 2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili- taire luchthavens), Artikel X 504 3 Warenwet, Artikel 1 492.5 4 Warenwet, Artikel 3 436.5 5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423 6 Pensioenwet BES, Artikel 1 373 7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362 8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319 9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder- landse Antillen, Artikel I 306 10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens, Artikel 2 294 Table 7. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht” 0.00262 2 Archiefwet 1995 0.00242 3 Wet op het financieel toezicht 0.00196 4 Zorgverzekeringswet 0.00175 5 Algemene Wet Bijzondere Ziektekosten 0.00167 6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162 7 Wet op het voortgezet onderwijs 0.00159 8 Wet bescherming persoonsgegevens 0.00150 9 Wet op de omzetbelasting 1968 0.00148 10 Werkloosheidswet 0.00147 Table 8. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 558 2 Werkloosheidswet 453 3 Wet op de arbeidsongeschiktheidsverzekering 453 4 Ziektewet 493 5 Archiefwet 1995 398 6 Wet op het voortgezet onderwijs 364 7 Wet op het financieel toezicht 361 8 Algemene Wet Bijzondere Ziektekosten 342 9 Wet werk in inkomen naar arbeidsvermogen 327 10 Zorgverzekeringswet 326 Table 6. Top-10 Betweenness Centrality Rank Name Value 1 Wet op de omzetbelasting 1968, Bijlage I 829.5 2 Wijzigingswet Wet luchtvaart (Regelgeving burgerluchthavens en mili- taire luchthavens), Artikel X 504 3 Warenwet, Artikel 1 492.5 4 Warenwet, Artikel 3 436.5 5 Wet vergoedingen adviescolleges en commissies, Artikel 2 423 6 Pensioenwet BES, Artikel 1 373 7 Administratiebesluit Bijzondere Ziektekostenverzekering, Artikel 1 362 8 Besluit inbeslaggenomen voorwerpen, Artikel 1 319 9 Rijkswet wijziging Statuut in verband met de ophe ng van de Neder- landse Antillen, Artikel I 306 10 Wet openbaarmaking uit publieke middelen gefinancierde topinkomens, Artikel 2 294 Table 7. Top-10 PageRank Rank Name Value 1 Algemene wet bestuursrecht” 0.00262 2 Archiefwet 1995 0.00242 3 Wet op het financieel toezicht 0.00196 4 Zorgverzekeringswet 0.00175 5 Algemene Wet Bijzondere Ziektekosten 0.00167 6 Bezoldigingsbesluit Burgerlijke Rijksambtenaren 1984, Bijlage B 0.00162 7 Wet op het voortgezet onderwijs 0.00159 8 Wet bescherming persoonsgegevens 0.00150 9 Wet op de omzetbelasting 1968 0.00148 10 Werkloosheidswet 0.00147 Table 8. Top-10 Indegree Rank Name Value 1 Algemene wet bestuursrecht 558 2 Werkloosheidswet 453 3 Wet op de arbeidsongeschiktheidsverzekering 453 4 Ziektewet 493 5 Archiefwet 1995 398 6 Wet op het voortgezet onderwijs 364 7 Wet op het financieel toezicht 361 8 Algemene Wet Bijzondere Ziektekosten 342 9 Wet werk in inkomen naar arbeidsvermogen 327 10 Zorgverzekeringswet 326 Measure Article PageRank 0.6996583 Degree 2.896214 Indegree 2.19982 Betweenness 3.658579 Table 10. Indegree per part of the Algemene wet bestuursrecht (AWB) Part Indegree Algemene wet bestuursrecht 558 Chapter 9 52 Chapter 6 48 Chapter 8 37 Chapter 7 33 Chapter 10 9 Chapter 5 9 Appendix 2 5 Chapter 3 4 Chapter 4 4 Chapter 2 3 Appendix 3 3 4 Is the Law like the Web? Both the law and the Web are man-made networks of interlinked documents. It is a valid question to ask whether the graph properties of both networks resemble eachother. Or, put in another way, is the distribution of information within the body of Dutch regulations specific for the domain of law, or is it similar to the
  • 34. Does Law resemble the Web? • The Web is a scale free network, ultra small world • The Web contains a single giant SCC would be interesting to see whether citations to articles indicate e.g. a high representation of definitions in the cited article [5]. Secondly, we compared network properties of the document- and article level networks to that of the Web, and concluded that both networks are scale free, but the connectedness of regulations is much lower than that of the Web. The lower connectedness makes it easier to distinguish modules in the set of regulations. It would be very interesting to see how the results of generic module recognition algorithms correspond to actual topics in legislation. Acknowledgments This publication was supported by the Dutch national program COMMIT. References 1. Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Ra- jagopalan, Raymie Stata, Andrew Tomkins, and Janet Wiener. Graph structure in the web. In Proceedings of the 9th international World Wide Web conference on Computer networks : the international journal of computer and telecommunications netowrking, pages 309–320, Amsterdam, The Netherlands, The Netherlands, 2000. North-Holland Publishing Co. 2. Rinke Hoekstra. The MetaLex Document Server - Legal Documents as Versioned Linked Data. In Harith Alani and Jamie Tailor, editors, Proceedings of the 10th International Semantic Web Conference (ISWC 2011), page 16. Springer, 2011. 3. Rinke Hoekstra. A Network Analysis of Dutch Regulations. Figshare.com, 2013. doi:10.6084/m9.figshare.689880. 4. Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford
  • 35. Does Law resemble the Web? • The Web is a scale free network, ultra small world • The Web contains a single giant SCC would be interesting to see whether citations to articles indicate e.g. a high representation of definitions in the cited article [5]. Secondly, we compared network properties of the document- and article level networks to that of the Web, and concluded that both networks are scale free, but the connectedness of regulations is much lower than that of the Web. The lower connectedness makes it easier to distinguish modules in the set of regulations. It would be very interesting to see how the results of generic module recognition algorithms correspond to actual topics in legislation. Acknowledgments This publication was supported by the Dutch national program COMMIT. References 1. Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Ra- jagopalan, Raymie Stata, Andrew Tomkins, and Janet Wiener. Graph structure in the web. In Proceedings of the 9th international World Wide Web conference on Computer networks : the international journal of computer and telecommunications netowrking, pages 309–320, Amsterdam, The Netherlands, The Netherlands, 2000. North-Holland Publishing Co. 2. Rinke Hoekstra. The MetaLex Document Server - Legal Documents as Versioned Linked Data. In Harith Alani and Jamie Tailor, editors, Proceedings of the 10th International Semantic Web Conference (ISWC 2011), page 16. Springer, 2011. 3. Rinke Hoekstra. A Network Analysis of Dutch Regulations. Figshare.com, 2013. doi:10.6084/m9.figshare.689880. 4. Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford Type # Largest >1 # Document 14019 816 74 Article 63303 12 501