Quantitative Legal Prediction - Legal Tech 2012 - Professor Daniel Martin Katz

Quantitative Legal Prediction ( Or How I Learned to Stop Worrying and Embrace Disruptive Technology) Daniel Martin Katz Michigan State University - College of Law

Do Not Fool Yourself

The LegalEmployment Crisis

Is More ThanJust a Recession

Two TrendsAre In Play

Structural and Cyclical

StructuralStarted Before The Recession

Those Jobs Will Not Return

Welcome to Law’sInformation Revolution

Revolution Already In Progress

My Background Assistant Professor of Law PhD Political Science & Public Policy Fellow in Empirical Legal Studies 2009 - 2011 University of Michigan Center for the Study of Complex Systems ( Fellow 2009-2010 )

Today I WouldLike to Sketch Out Where I Believe Our Industry is Heading

Start By Talking About SomeBroad Trends...

This is the Era of “Big Data”Increasing Computing PowerDecreasing Data Storage CostsFundamentally Altering the Scope of Scientific Inquiry

Highlighting the Data Deluge 2008 2009 2010

Highlighting the Data Deluge 2011 2011

Highlighting the Data Deluge

So How Big is‘Big’?

How Much Data Is a Petabyte?

$600 to Buy a Disk that canStore All of the World’s Music$1000 Harddrive Could Easily Store Every Written Judicial Decision in U.S. History

Okay But ...

Data is only half the story

Computation andArtificial Intelligence

The Artificial Intelligence Revolution is On

The Artificial Intelligence Revolution is On ....But it is not what we thought

‘Soft’ Artificial Intelligence

“Practically every financial transaction,from someone buying a cup of coffee tosomeone trading a trillion dollars ofcredit default derivatives, is done insoftware ....Health care and education, in my view, arenext up for fundamental software-basedtransformation. My venture capital firmis backing aggressive start-ups in both ofthese gigantic and critical industries.We believe both of these industries, whichhistorically have been highly resistant toentrepreneurial change, are primed fortipping by great new software-centricentrepreneurs ...Companies in every industry need toassume that a software revolution iscoming.”

The Artificial Intelligence Revolution is On

Play Videohttp://www.youtube.com/watch?v=lI-M7O_bRNg&feature=relmfu

Play Videohttp://www.youtube.com/watch?v=DywO4zksfXw&feature=related

That’s Me

Some Applicable Terms

Natural Language ProcessingKnowledge Representation Feature Extraction Feature Selection Machine Learning Classification Clustering

http://www.drewconway.com/zia/?p=2378

A More Meaningful‘Soft’ AI Example?

E-Discovery

Yes LegalTech Is Full of E-Discovery Firms

Just Remember there was a Day When E-Discovery was not mainstream

Even Within LegalTechnology Circles ...

So What is the Next Big Thing... ?

Quantitative Legal Prediction

The First ResponseI Typically Encounter

You Cannot Replace What I Do ...

It is Useful

To Consider an Industry

Where Human Reasoning

Was Paramount

Finance was an Industry

Where Human Reasoning

Reigned Supreme

But Not Anymore ...

The Rise of the Quants...

http://www.cbsnews.com/video/watch/?id=6945451n

Prediction

A Few WordsAbout Prediction

Imagine Two Different Complex Systems

Weather

vs.Easy/ Predictable Difficult / Chaotic

The Caliber of Prediction is

A Function ofVarious Factors

Including ...

Underlying SystemVariability

Quality of Inputs

Formal Treatment of the question of prediction inalternative Domains

THE THESIS OFTHIS PRESENTATION

Human Prediction is the Hallmark of theLegal Services Industry

The Race for the Future of this Industry is Developing Processes

That Leverage the DataStreams Being Used by “Expert Reasoners”

To Mimic their Processesbut do so at scale

What Does Mean By This?

Human are GoodPattern Detectors

But Aggregation is the Question ...

How Does a HumanReasoner Evaluate 10,000 100,000 1,000,000 Data Points?

Machine Aided Prediction

Will Come Before Wide Spread

Machine Only Prediction

In the Language of E-Discovery ...

Supervised Methods vs.Unsupervised Methods

It Already Exists ...

Legal Procurement &Legal Supply Chain Mgmt.

Data Driven Legal Supply Chain Mgmt.Different Form of PredictionHow Does a CompanyStaff for Peak Load?What sort of InternalPolicies are Efficient?

Legal Supply Chain Mgmt. (High End of Market)Data and Logistics =General Counsels asMaestrosLaw Firms Point ofSale Providers

Legal Procurement (High End of Market)General Counsels asLegal ProcurementSpecialistsTyMetrix - Using$4billion in LegalSpend Data to HelpGC’s Look for ArbitrageOpportunities

Legal Procurement (High End of Market)Driving Downyour Legal BillsYeah there is anApp for That City Firm Size Partner Calculate Experience

Predicting JudicialDecisions

This Was ~10 Years Ago

Model Leverages ‘Soft’ AI (i.e. Classification Tree)

This is aGreat Example

But What Does the Market Really Care About?

Predicting Outcomesin Disputes

Disputes v. Decisions

Disputes, Filings, etc.

Bargaining in theShadow of the Law

What is the Client’s First Question?

do i have a case?

how is that assessment generated?

Mental Models vsAggregated Data

How Does the Human Reasoner Arrive at Their Conclusion?

lots of factors matter

Time Scales Matter

inherent systemvariability matters

‘complexity’ matters

Some Potential Inputs

what is the relevantinformation contained therein?

text, citations, votes, etc.

other metadata

what form might the outputs take?

standard client memo +statistical portrait of10,000 ‘similar’ cases

IPLITIGATION

https://lexmachina.com/

TAXLITIGATION

Which Cases Settle? A Large-ScaleEmpirical Study of U.S. Tax Court Cases!Leandra Lederman !William W. Oliver Professor of Tax Law!Indiana University Maurer School of Law!Daniel Martin Katz !Assistant Professor of Law!Michigan State University College of Law! !Michael J. Bommarito II!Ph.D. Pre-Candidate!University of Michigan, !Department of Political Science! (Forthcoming)

TAX COURT Data Acquisition LITIGATION ! 4.5+618,053! ! Million! ! Filings! Docket Entries!(1986-2008)! (1986-2008)!

standard client memo +statistical portrait of10,000 ‘similar’ cases

But How do weassess similarity?

Analogical Reasoning and the Science of Similarity

similarity measures =~distance measures

developing thesedistance functions

is an important part of my research

just one example ...

Six Degrees of Marbury v. Madison A Sink Based Visualization

My Primary Motivation is to Leverage the Citation Network (and Text)

To Better UnderstandAnalogical Reasoning...

Which is the Hallmarkof the Socratic Method

Learning to Think Like a Lawyer

Big Strides Have Been Made in the‘Science of Similarity’

The Science of Similarity What Makes a Set of High Dimensional Objects ‘Similar”?Movies People Words Music Books Crowd Sourced Human Reasoning is Helping Develop a ‘New’ Science of Similarity

A Brief Word About Data Driven Theory BuildingThe Hypothesis Testing Frame isnot the only way to do ScienceOne Billion Clicks Can Be theBasis for a Theory Generation

A Brief Word About Data Driven Theory BuildingValidation Can Be AchievedThrough Either: (1) Out of Sample Testing (2) Forward PredictionExample: http://en.wikipedia.org/wiki/Netflix_Prize http://www.wired.com/epicenter/2009/09/ how-the-netflix-prize-was-won/

Additional Examples This is the Age of Aspirational Spelling (Spelling is 1.0 Thinking)

Additional Examples Collaborative FilteringPeople Who Bought XAlso Bought Y

Science of Similarity is not QuiteAnalogical Reasoning

But it is in theSame Family ...

In Law We Have a Massive Recommender System

Chains of CitationsChains of Precedent

Whose DynamicsShould be Leveraged

As It ContainsValuable Information

United States Supreme Court Case to Case Citation NetworkNodes = Cases Decided by the Supreme CourtDirected Edges = Citations between Cases

Case to Case Citation Network of theUnited States Supreme Court Oregon v. Elstad 530 U.S. 428 (2000) Miranda v. Arizona Dickerson v. U.S. 384 U.S. 436 (1966) 530 U.S. 428 (2000)

Exploring the Path of Precedent?Dynamic Visualization of the Early Supreme Court Citation Network

PLAY MOVIE! http://computationallegalstudies.com/ 2010/02/11/the-development-of-structure-in- the-citation-network-of-the-united-states- supreme-court-now-in-hd/Cases Decided by Citations in the Citations fromthe Supreme Court Current Year prior years

What is theDynamic Signature of Analogical Reasoning?

Dynamic Document Relations:The Judicial Citation Network Re- +* Gergely Palla, Albert-Laszlo Barabasi & Tamas Vicsek, Quantifying Social Group Evolution, 446 Nature 664 (2007).

Text +Citation Signature

http://computationallegalstudies.com/2011/04/02/six-degrees-of-marbury-v- madison-a-sink-based-visualization-2/

It Is Not Clear Exactly ...What Metadata or Configuration ofMetadata is Useful

Let Me Just MakeSure We Are Clear

This Stuff is Technical But Also Very Practical

Lawyers are in thePrediction Business (in Part)

Soft AI to Mimic (Perhaps Improve Upon)Human Prediction Processes

What Metadata Is Relevantto Prediction?

Text, Citations, Votes, etc.“Similar” Prior Disputes, etc.

What Weighting Scheme Over These Objects?

My Lab Group isActively Engaged in Research that is Relevant to Developing Legal Information Products

Here Are a FewThings from Our Lab Law as a Seamless Web? Electronic World3D HD Visualization of Supreme Treaty Court Citation Network Index Six Degrees of Marbury v. Madison The United States Code

LegalLanguageExplorer.com

Some Additional Thoughts

Study of Law CanBe a Scientific Exercise

The Practice of Law Can be Closer to Engineering

The Market forLegal Services

The Market forLegal Education

Have a Date With Destiny

We Already HaveQuantitative Legal Prediction

There Is Going to Lots More

In Other Words,

Welcome To Law’sInformation Revolution

And Yes There Is Goingto Be Math on the Exam

Primary ContactInformation katzd@law.msu.edu http://www.law.msu.edu/ faculty_staff/profile.php? prof=780http://computationallegalstudies.com/

AdditionalMaterialsAvailable Online!

>>> Print “Welcome to Computational Legal Studies”Welcome to Computational Legal Studies

http://www.legalethicsforum.com	4327
http://computationallegalstudies.com	2164
http://legalethicsforum.typepad.com	736
http://rattsinfo.collected.info	30
http://www.typepad.com	5
http://translate.googleusercontent.com	4
http://www.legalethicsforum.typepad.com	3
http://www.newsblur.com	2
http://paper.li	2
http://ec2-107-21-222-181.compute-1.amazonaws.com	2
http://www.linkedin.com	2
http://127.0.0.1	1
http://webcache.googleusercontent.com	1
http://www.twylah.com	1

Quantitative Legal Prediction - Legal Tech 2012 - Professor Daniel Martin Katz

by Daniel Katz on Feb 01, 2012

Accessibility

Categories

Upload Details

Usage Rights

14 Embeds 7,280

Statistics

Quantitative Legal Prediction - Legal Tech 2012 - Professor Daniel Martin Katz — Presentation Transcript