Learn how Congress is starting to use Elasticsearch to modernize search, and how you can help to accelerate this change. Until just a decade ago, some of the most important precedents of Congress were stored and indexed on paper cards. This system was replaced by a slow, clunky SQL search and there was initially resistance when Xcential proposed to convert this to Elasticsearch (v1.7.2), because lawyers feared that ‘relevance’-based search would be less precise. With accurate results, 50 times faster, users were won over. Now there is a potential to use modern search, not only for precedents, but throughout the work of Congress and in state legislatures as well. We’ll talk about some of these projects, and how you could get involved (we’re hiring!).

1. Elasticsearch Goes to Congress
Indexing law
1
ari@xcential.com
xcential.com
@arihersh
Ari Hershowitz

2. 2
Who am I?
Ari Hershowitz
• Neuroscience, environmental
advocacy, etc (prehistory - 2007)
• Lawyer (Georgetown Law 2007)
• Legal technology (2008 - Now)
• Dad
ari@xcential.com
@arihersh
github.com/aih/FlatGov
blog.linkedlegislation.com

3. 3
My goals today
1. Show how we’re using search in law.
2. Get you to help us do it better.

4. 4
Overview
• A brief history of legal ‘search’
• Unique challenges of search for law
• Our work in Congress and beyond
• A vision for modern legal search

5. 5

6. 6
Lawyers like lists
https://www.loc.gov/resource/rbpe.24203100/

7. 7
Lawyers like lists

8. 8
Incremental indexing
1800’s Shepard’s citator
printed gummed paper
to add to law books to
track new references to
prior law

9. 9

10. 10
Legal search
• Lawyers hate relevancy
Complete >>> sort order
Precision >>> efficiency
• Lawyers hate stop-words: every ‘and’ is sacred
• Legal researchers love booleans

11. 11

12. Our work
12
Office of the House
Parliamentarian
•Cards listing daily
precedents for more
than 200 years
•Thousands of
precedents
•About 1 Gb of data
Index cards -> SQL -> Elasticsearch

13. Our work
13
Bill similarity
•Open source
•“Plagiarism” software for
bills in Congress
•Using
Elasticsearch/Lucene
‘more like this’

14. Bill similarity: one section
14
SEC. 2. IMPROVING ACCESS TO INFLUENTIAL VISITOR ACCESS RECORDS.
(a) Definitions.—In this section:
(1) COVERED LOCATION.—The term “covered location” means—
(A) the White House;
(B) the residence of the Vice President; and
(C) any other location at which the President or the Vice President regularly conducts official business.
(2) COVERED RECORDS.—The term “covered records” means information relating to a visit at a covered location, which shall include—
(A) the name of each visitor at the covered location;

15. Bill similarity: many sections
15
Bill 116hr1500ih
Sections
1
100
101 => similar sections in other bills
102
200
…
Many ways to calculate cumulative similarity.

16. Our work
16
State law
•Ingest XML
•Link to sources
•Translate user query to
ES DSL
•Create a user interface
for legal search

17. Summary
• Law is a natural fit for indexing, search
and similarity metrics
• Small data (1 GB) is important, too
• Stop working on logs and work with us
on legislative text
17