The Three Forms of (Legal) Prediction: Experts, Crowds and Algorithms -- Professors Daniel Martin Katz & Michael J. Bommarito - Illinois Tech Law / Univ of Michigan CSCS (Updated Version)
Artificial Intelligence and Law - A Primer Daniel Katz
Artificial Intelligence in Law (and beyond) including Machine Learning as a Service, Quantitative Legal Prediction / Legal Analytics, Experts + Crowds + Algorithms
Fin (Legal) Tech – Law’s Future from Finance’s Past (Some Thoughts About the ...Daniel Katz
Fin (Legal) Tech – Law’s Future from Finance’s Past (Some Thoughts About the Financialization of the Law) – Professors Daniel Martin Katz + Michael J Bommarito
Artificial Intelligence and Law - A Primer Daniel Katz
Artificial Intelligence in Law (and beyond) including Machine Learning as a Service, Quantitative Legal Prediction / Legal Analytics, Experts + Crowds + Algorithms
Fin (Legal) Tech – Law’s Future from Finance’s Past (Some Thoughts About the ...Daniel Katz
Fin (Legal) Tech – Law’s Future from Finance’s Past (Some Thoughts About the Financialization of the Law) – Professors Daniel Martin Katz + Michael J Bommarito
Why We Are Open Sourcing ContraxSuite and Some Thoughts About Legal Tech and ...Daniel Katz
Why We Are Open Sourcing ContraxSuite and Some Thoughts About Legal Tech and the Modern Information Economy - By Michael Bommarito + Daniel Martin Katz from LexPredict
Exploring the Physical Properties of Regulatory Ecosystems - Professors Danie...Daniel Katz
Exploring the Physical Properties of Regulatory Ecosystems: Regulatory Dynamics Revealed by Securities Filings — Professors Daniel Martin Katz + Michael J Bommarito
Introduction to artificial intelligence and lawLawScienceTech
Presentation at Seminar on Artificial Intelligence and Law (15/03/2018) at the Norwegian Research Center for Computers and Law (NRCCL), University of Oslo
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (KD...Krishnaram Kenthapadi
Researchers and practitioners from different disciplines have highlighted the ethical and legal challenges posed by the use of machine learned models and data-driven systems, and the potential for such systems to discriminate against certain population groups, due to biases in algorithmic decision-making systems. This tutorial presents an overview of algorithmic bias / discrimination issues observed over the last few years and the lessons learned, key regulations and laws, and evolution of techniques for achieving fairness in machine learning systems. We motivate the need for adopting a "fairness by design" approach (as opposed to viewing algorithmic bias / fairness considerations as an afterthought), when developing machine learning based models and systems for different consumer and enterprise applications. Then, we focus on the application of fairness-aware machine learning techniques in practice by presenting non-proprietary case studies from different technology companies. Finally, based on our experiences working on fairness in machine learning at companies such as Facebook, Google, LinkedIn, and Microsoft, we present open problems and research directions for the data mining / machine learning community.
These are slides from my presentation at the Law Firm Leaders Forum in New York, Nov. 6-7, 2014. Part I covers Substantive Hints of Change: Innovative Technology Popping Up and Part II covers Legal Design: Structured Innovation Process and Focus on Client/User Needs.
Privacy in AI/ML Systems: Practical Challenges and Lessons LearnedKrishnaram Kenthapadi
How do we protect the privacy of users when building large-scale AI based systems? How do we develop machine learning models and systems taking fairness, accuracy, explainability, and transparency into account? Model fairness and explainability and protection of user privacy are considered prerequisites for building trust and adoption of AI systems in high stakes domains. We will first motivate the need for adopting a “fairness, explainability, and privacy by design” approach when developing AI/ML models and systems for different consumer and enterprise applications from the societal, regulatory, customer, end-user, and model developer perspectives. We will then focus on the application of privacy-preserving AI techniques in practice through industry case studies. We will discuss the sociotechnical dimensions and practical challenges, and conclude with the key takeaways and open challenges.
What AI is and examples of how it is used in legalBen Gardner
This presentation was given at Legal Geek on 10th Dec 2015. It is a scenesetting peice that looks to de-mystify artificial intelligence by looking beyond the hype.
Algorithms are taking control of our information rich world. As the twin sibling to Big Data, increasingly they decide how society views us via constructed profiles (as criminals? as terrorists? as rich or poor consumers?); what we see as important, newsworthy, cool or profitable (eg Twitter trending topics, automated stock selling, Amazon recommendations, BBC website top news topics etc); and indeed what we see at all as algorithms are increasingly used to filter our illegal or undesirable content as tools of public policy. Algorithms are peceived by virtue of their automation as neutral, objective and fair, unlike human decision makers - yet evidence increasingly shows the opposite - eg a series of legal complaints assert that Google games its own search results to promote its own economic interests and demote those of competitors or annoyances; while in the defamation field, French, German and Italian courts have decided that algorithmically generated autosuggests in search can be libellous (eg "Bettina Wolf prostitute"). . This paper asks if any legal remedies do or should exist to *audit* proprietary algorithms , given their importance, and asks if one way forward might be via existing and future subject access rights to personal data in EU data protection law. The transformation of these rights as proposed in the draft Data Protection Regulation is not however hopeful.
Why We Are Open Sourcing ContraxSuite and Some Thoughts About Legal Tech and ...Daniel Katz
Why We Are Open Sourcing ContraxSuite and Some Thoughts About Legal Tech and the Modern Information Economy - By Michael Bommarito + Daniel Martin Katz from LexPredict
Exploring the Physical Properties of Regulatory Ecosystems - Professors Danie...Daniel Katz
Exploring the Physical Properties of Regulatory Ecosystems: Regulatory Dynamics Revealed by Securities Filings — Professors Daniel Martin Katz + Michael J Bommarito
Introduction to artificial intelligence and lawLawScienceTech
Presentation at Seminar on Artificial Intelligence and Law (15/03/2018) at the Norwegian Research Center for Computers and Law (NRCCL), University of Oslo
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (KD...Krishnaram Kenthapadi
Researchers and practitioners from different disciplines have highlighted the ethical and legal challenges posed by the use of machine learned models and data-driven systems, and the potential for such systems to discriminate against certain population groups, due to biases in algorithmic decision-making systems. This tutorial presents an overview of algorithmic bias / discrimination issues observed over the last few years and the lessons learned, key regulations and laws, and evolution of techniques for achieving fairness in machine learning systems. We motivate the need for adopting a "fairness by design" approach (as opposed to viewing algorithmic bias / fairness considerations as an afterthought), when developing machine learning based models and systems for different consumer and enterprise applications. Then, we focus on the application of fairness-aware machine learning techniques in practice by presenting non-proprietary case studies from different technology companies. Finally, based on our experiences working on fairness in machine learning at companies such as Facebook, Google, LinkedIn, and Microsoft, we present open problems and research directions for the data mining / machine learning community.
These are slides from my presentation at the Law Firm Leaders Forum in New York, Nov. 6-7, 2014. Part I covers Substantive Hints of Change: Innovative Technology Popping Up and Part II covers Legal Design: Structured Innovation Process and Focus on Client/User Needs.
Privacy in AI/ML Systems: Practical Challenges and Lessons LearnedKrishnaram Kenthapadi
How do we protect the privacy of users when building large-scale AI based systems? How do we develop machine learning models and systems taking fairness, accuracy, explainability, and transparency into account? Model fairness and explainability and protection of user privacy are considered prerequisites for building trust and adoption of AI systems in high stakes domains. We will first motivate the need for adopting a “fairness, explainability, and privacy by design” approach when developing AI/ML models and systems for different consumer and enterprise applications from the societal, regulatory, customer, end-user, and model developer perspectives. We will then focus on the application of privacy-preserving AI techniques in practice through industry case studies. We will discuss the sociotechnical dimensions and practical challenges, and conclude with the key takeaways and open challenges.
What AI is and examples of how it is used in legalBen Gardner
This presentation was given at Legal Geek on 10th Dec 2015. It is a scenesetting peice that looks to de-mystify artificial intelligence by looking beyond the hype.
Algorithms are taking control of our information rich world. As the twin sibling to Big Data, increasingly they decide how society views us via constructed profiles (as criminals? as terrorists? as rich or poor consumers?); what we see as important, newsworthy, cool or profitable (eg Twitter trending topics, automated stock selling, Amazon recommendations, BBC website top news topics etc); and indeed what we see at all as algorithms are increasingly used to filter our illegal or undesirable content as tools of public policy. Algorithms are peceived by virtue of their automation as neutral, objective and fair, unlike human decision makers - yet evidence increasingly shows the opposite - eg a series of legal complaints assert that Google games its own search results to promote its own economic interests and demote those of competitors or annoyances; while in the defamation field, French, German and Italian courts have decided that algorithmically generated autosuggests in search can be libellous (eg "Bettina Wolf prostitute"). . This paper asks if any legal remedies do or should exist to *audit* proprietary algorithms , given their importance, and asks if one way forward might be via existing and future subject access rights to personal data in EU data protection law. The transformation of these rights as proposed in the draft Data Protection Regulation is not however hopeful.
INSIDER'S PERSPECTIVE: Three Trends That Will Define the Next Horizon in Lega...LexisNexis
In a recent Information Today article, Sean Fitzpatrick of LexisNexis discusses trends that will define the future of legal research as we know it.
Humans create 2.5 quintillion bytes of data each day, and the cost of storing and maintaining each byte of data is declining. In fact, the growth of stored data is outpacing the ability of most people to manage it.
Powerful tools, such as natural language processing and machine learning, are helping professionals bridge the gap between information overload and the ability to harvest the power of Big Data.
Millennials now make up nearly one-third of the U.S. workforce and they are our most educated generation.
This presentation by the John O. McGinnis, Northwestern Pritzker School of Law was made during a roundtable discussion on Disruptive innovations in legal services held at the 61st meeting of the Working Party No. 2 on Competition and Regulation on 13 June 2014. More papers, presentations and contributions from delegations on the topic can be found out at www.oecd.org/daf/competition/disruptive-innovations-in-legal-services.htm
3222020 Prediction, persuasion, and the jurisprudence of beh.docxlorainedeserre
3/22/2020 Prediction, persuasion, and the jurisprudence of behaviourism: UC MegaSearch
eds.a.ebscohost.com/eds/detail/detail?vid=1&sid=80a53932-b932-4bf6-926e-093727bceef6%40sessionmgr4007&bdata=JkF1dGhUeXBlPXNoaWIm… 1/14
Title:
Database:
Prediction, persuasion, and the jurisprudence of behaviourism By: Frank Pasquale, Glyn Cashwell,
17101174, , Vol. 68, Issue 1
ProjectMUSE
Prediction, persuasion, and the jurisprudence of
behaviourism
There is a growing literature critiquing the unreflective application of big data, predictive analytics, artificial intelligence, and
machine-learning techniques to social problems. Such methods may reflect biases rather than reasoned decision making. They also
may leave those affected by automated sorting and categorizing unable to understand the basis of the decisions affecting them.
Despite these problems, machine-learning experts are feeding judicial opinions to algorithms to predict how future cases will be
decided. We call the use of such predictive analytics in judicial contexts a jurisprudence of behaviourism as it rests on a
fundamentally Skinnerian model of cognition as a black-boxed transformation of inputs into outputs. In this model, persuasion is
passé; what matters is prediction. After describing and critiquing a recent study that has advanced this jurisprudence of behaviourism,
we question the value of such research. Widespread deployment of prediction models not based on the meaning of important
precedents and facts may endanger the core rule-of-law values.
artificial intelligence; cyber law; machine learning; jurisprudence; predictive analysis
I Introduction
A growing chorus of critics are challenging the use of opaque (or merely complex) predictive analytics programs to monitor,
influence, and assess individuals’ behaviour. The rise of a ‘black box society’ portends profound threats to individual autonomy;
when critical data and algorithms cannot be a matter of public understanding or debate, both consumers and citizens are unable to
comprehend how they are being sorted, categorized, and influenced.[ 2]
A predictable counter-argument has arisen, discounting the comparative competence of human decision makers. Defending opaque
sentencing algorithms, for instance, Christine Remington (a Wisconsin assistant attorney general) has stated: ‘We don’t know what’s
going on in a judge’s head; it’s a black box, too.’[ 3] Of course, a judge must (upon issuing an important decision) explain why the
decision was made; so too are agencies covered by the Administrative Procedure Act obliged to offer a ‘concise statement of basis
and purpose’ for rule making.[ 4] But there is a long tradition of realist commentators dismissing the legal justifications adopted by
judges as unconvincing fig leaves for the ‘real’ (non-legal) bases of their decisions.
In the first half of the twentieth century, the realist disdain for stated rationales for decisions led in at least two directions: toward
more rigorous ...
3222020 Prediction, persuasion, and the jurisprudence of beh.docxpriestmanmable
3/22/2020 Prediction, persuasion, and the jurisprudence of behaviourism: UC MegaSearch
eds.a.ebscohost.com/eds/detail/detail?vid=1&sid=80a53932-b932-4bf6-926e-093727bceef6%40sessionmgr4007&bdata=JkF1dGhUeXBlPXNoaWIm… 1/14
Title:
Database:
Prediction, persuasion, and the jurisprudence of behaviourism By: Frank Pasquale, Glyn Cashwell,
17101174, , Vol. 68, Issue 1
ProjectMUSE
Prediction, persuasion, and the jurisprudence of
behaviourism
There is a growing literature critiquing the unreflective application of big data, predictive analytics, artificial intelligence, and
machine-learning techniques to social problems. Such methods may reflect biases rather than reasoned decision making. They also
may leave those affected by automated sorting and categorizing unable to understand the basis of the decisions affecting them.
Despite these problems, machine-learning experts are feeding judicial opinions to algorithms to predict how future cases will be
decided. We call the use of such predictive analytics in judicial contexts a jurisprudence of behaviourism as it rests on a
fundamentally Skinnerian model of cognition as a black-boxed transformation of inputs into outputs. In this model, persuasion is
passé; what matters is prediction. After describing and critiquing a recent study that has advanced this jurisprudence of behaviourism,
we question the value of such research. Widespread deployment of prediction models not based on the meaning of important
precedents and facts may endanger the core rule-of-law values.
artificial intelligence; cyber law; machine learning; jurisprudence; predictive analysis
I Introduction
A growing chorus of critics are challenging the use of opaque (or merely complex) predictive analytics programs to monitor,
influence, and assess individuals’ behaviour. The rise of a ‘black box society’ portends profound threats to individual autonomy;
when critical data and algorithms cannot be a matter of public understanding or debate, both consumers and citizens are unable to
comprehend how they are being sorted, categorized, and influenced.[ 2]
A predictable counter-argument has arisen, discounting the comparative competence of human decision makers. Defending opaque
sentencing algorithms, for instance, Christine Remington (a Wisconsin assistant attorney general) has stated: ‘We don’t know what’s
going on in a judge’s head; it’s a black box, too.’[ 3] Of course, a judge must (upon issuing an important decision) explain why the
decision was made; so too are agencies covered by the Administrative Procedure Act obliged to offer a ‘concise statement of basis
and purpose’ for rule making.[ 4] But there is a long tradition of realist commentators dismissing the legal justifications adopted by
judges as unconvincing fig leaves for the ‘real’ (non-legal) bases of their decisions.
In the first half of the twentieth century, the realist disdain for stated rationales for decisions led in at least two directions: toward
more rigorous .
The impact of AI and Blockchain technologies in the Legal IndustryHunter Thompson
This is a paper I wrote for my final semester of my Bachelor of Law (Honours) for the subject Innovation and intellectual Property Law, for which I received a high distinction (56/60). I wanted to share this paper with my Linkedin colleagues in the hope that it might provide an overview of two areas of disruption in law that I believe are highly relevant and interesting.
The role we play as creators - A designer's take on AIGiuseppe de Cesare
There is a growing interest in AI, but the field is understood only by a few, often by technicians. What’s our involvement as designers? What do we know about these new disruptive technologies? Often we know little. This talk will unveil benefits and pitfalls of AI through concrete cases offering designers a framework for understanding. It will introduce a toolkit of ethical principles, that will make you reflect upon the potential of AI to change what is our conventional understanding of a product or a service. Design is about developing a higher awareness of the role we play as creators and builders.
Similar to The Three Forms of (Legal) Prediction: Experts, Crowds and Algorithms -- Professors Daniel Martin Katz & Michael J. Bommarito (Updated) (20)
LexPredict - Empowering the Future of Legal Decision MakingDaniel Katz
LexPredict is an enterprise legal technology and consulting firm, specializing in the application of best-in-class processes and technologies from the technology, financial services, and logistics industries to the practice of law, compliance, insurance, and risk management.
We focus on the goals of prediction, optimization, and risk management to enable holistic organizational changes that empower legal decision-making.
These changes span people and processes, software and data, and execution and education.
Legal Analytics Course - Class 9 - Clustering Algorithms (K-Means & Hierarchical Clustering) - Professor Daniel Martin Katz + Professor Michael J Bommarito
Legal Analytics Course - Class 5 - Quantitative Legal Prediction + Data Drive...Daniel Katz
Legal Analytics Course - Class 5 - Quantitative Legal Prediction + Data Driven Future of Law Practice - Professor Daniel Martin Katz + Professor Michael J Bommarito
NATURE, ORIGIN AND DEVELOPMENT OF INTERNATIONAL LAW.pptxanvithaav
These slides helps the student of international law to understand what is the nature of international law? and how international law was originated and developed?.
The slides was well structured along with the highlighted points for better understanding .
Introducing New Government Regulation on Toll Road.pdfAHRP Law Firm
For nearly two decades, Government Regulation Number 15 of 2005 on Toll Roads ("GR No. 15/2005") has served as the cornerstone of toll road legislation. However, with the emergence of various new developments and legal requirements, the Government has enacted Government Regulation Number 23 of 2024 on Toll Roads to replace GR No. 15/2005. This new regulation introduces several provisions impacting toll business entities and toll road users. Find out more out insights about this topic in our Legal Brief publication.
ASHWINI KUMAR UPADHYAY v/s Union of India.pptxshweeta209
transfer of the P.I.L filed by lawyer Ashwini Kumar Upadhyay in Delhi High Court to Supreme Court.
on the issue of UNIFORM MARRIAGE AGE of men and women.
WINDING UP of COMPANY, Modes of DissolutionKHURRAMWALI
Winding up, also known as liquidation, refers to the legal and financial process of dissolving a company. It involves ceasing operations, selling assets, settling debts, and ultimately removing the company from the official business registry.
Here's a breakdown of the key aspects of winding up:
Reasons for Winding Up:
Insolvency: This is the most common reason, where the company cannot pay its debts. Creditors may initiate a compulsory winding up to recover their dues.
Voluntary Closure: The owners may decide to close the company due to reasons like reaching business goals, facing losses, or merging with another company.
Deadlock: If shareholders or directors cannot agree on how to run the company, a court may order a winding up.
Types of Winding Up:
Voluntary Winding Up: This is initiated by the company's shareholders through a resolution passed by a majority vote. There are two main types:
Members' Voluntary Winding Up: The company is solvent (has enough assets to pay off its debts) and shareholders will receive any remaining assets after debts are settled.
Creditors' Voluntary Winding Up: The company is insolvent and creditors will be prioritized in receiving payment from the sale of assets.
Compulsory Winding Up: This is initiated by a court order, typically at the request of creditors, government agencies, or even by the company itself if it's insolvent.
Process of Winding Up:
Appointment of Liquidator: A qualified professional is appointed to oversee the winding-up process. They are responsible for selling assets, paying off debts, and distributing any remaining funds.
Cease Trading: The company stops its regular business operations.
Notification of Creditors: Creditors are informed about the winding up and invited to submit their claims.
Sale of Assets: The company's assets are sold to generate cash to pay off creditors.
Payment of Debts: Creditors are paid according to a set order of priority, with secured creditors receiving payment before unsecured creditors.
Distribution to Shareholders: If there are any remaining funds after all debts are settled, they are distributed to shareholders according to their ownership stake.
Dissolution: Once all claims are settled and distributions made, the company is officially dissolved and removed from the business register.
Impact of Winding Up:
Employees: Employees will likely lose their jobs during the winding-up process.
Creditors: Creditors may not recover their debts in full, especially if the company is insolvent.
Shareholders: Shareholders may not receive any payout if the company's debts exceed its assets.
Winding up is a complex legal and financial process that can have significant consequences for all parties involved. It's important to seek professional legal and financial advice when considering winding up a company.
Car Accident Injury Do I Have a Case....Knowyourright
Every year, thousands of Minnesotans are injured in car accidents. These injuries can be severe – even life-changing. Under Minnesota law, you can pursue compensation through a personal injury lawsuit.
RIGHTS OF VICTIM EDITED PRESENTATION(SAIF JAVED).pptxOmGod1
Victims of crime have a range of rights designed to ensure their protection, support, and participation in the justice system. These rights include the right to be treated with dignity and respect, the right to be informed about the progress of their case, and the right to be heard during legal proceedings. Victims are entitled to protection from intimidation and harm, access to support services such as counseling and medical care, and the right to restitution from the offender. Additionally, many jurisdictions provide victims with the right to participate in parole hearings and the right to privacy to protect their personal information from public disclosure. These rights aim to acknowledge the impact of crime on victims and to provide them with the necessary resources and involvement in the judicial process.
Responsibilities of the office bearers while registering multi-state cooperat...Finlaw Consultancy Pvt Ltd
Introduction-
The process of register multi-state cooperative society in India is governed by the Multi-State Co-operative Societies Act, 2002. This process requires the office bearers to undertake several crucial responsibilities to ensure compliance with legal and regulatory frameworks. The key office bearers typically include the President, Secretary, and Treasurer, along with other elected members of the managing committee. Their responsibilities encompass administrative, legal, and financial duties essential for the successful registration and operation of the society.
DNA Testing in Civil and Criminal Matters.pptxpatrons legal
Get insights into DNA testing and its application in civil and criminal matters. Find out how it contributes to fair and accurate legal proceedings. For more information: https://www.patronslegal.com/criminal-litigation.html
ALL EYES ON RAFAH BUT WHY Explain more.pdf46adnanshahzad
All eyes on Rafah: But why?. The Rafah border crossing, a crucial point between Egypt and the Gaza Strip, often finds itself at the center of global attention. As we explore the significance of Rafah, we’ll uncover why all eyes are on Rafah and the complexities surrounding this pivotal region.
INTRODUCTION
What makes Rafah so significant that it captures global attention? The phrase ‘All eyes are on Rafah’ resonates not just with those in the region but with people worldwide who recognize its strategic, humanitarian, and political importance. In this guide, we will delve into the factors that make Rafah a focal point for international interest, examining its historical context, humanitarian challenges, and political dimensions.
PRECEDENT AS A SOURCE OF LAW (SAIF JAVED).pptxOmGod1
Precedent, or stare decisis, is a cornerstone of common law systems where past judicial decisions guide future cases, ensuring consistency and predictability in the legal system. Binding precedents from higher courts must be followed by lower courts, while persuasive precedents may influence but are not obligatory. This principle promotes fairness and efficiency, allowing for the evolution of the law as higher courts can overrule outdated decisions. Despite criticisms of rigidity and complexity, precedent ensures similar cases are treated alike, balancing stability with flexibility in judicial decision-making.
The Three Forms of (Legal) Prediction: Experts, Crowds and Algorithms -- Professors Daniel Martin Katz & Michael J. Bommarito (Updated)
1. The Three Forms
of (Legal) Prediction
professor daniel martin katz
home | Illinois tech - chicago kent
blog | ComputationalLegalStudies
corp | LexPredict
experts, crowds & algorithms
professor michael j bommarito
4. play “whack-a-mole”, reacting to
problems by creating fear and
friction within organizations and
the impression that there is a
legal risk around every corner.
Mediocre Lawyers
5. can help clients shape
(perhaps distort)
external perception of risk.
Merely Clever Lawyers
6. design systems that
balance risk and improve
transparency, helping clients
correctly price risk internally
Great Lawyers
15. Quantitative Legal Prediction
- or -
How I Learned to Stop Worrying and Start
Preparing for the Data Driven Future of the
Legal Services Industry
Professor Daniel Martin Katz
#LegalAnalyics
#LegalData #LegalPrediction
18.
Daniel Martin Katz, Joshua Gubler, Jon Zelner, Michael Bommarito, Eric Provins
& Eitan Ingall, Reproduction of Hierarchy? A Social Network Analysis of the American
Law Professoriate, 61 Journal of Legal Education 76 (2011)
23. Acyclic digraphs arise in many natural and artificial processes. Among the
broader set, dynamic citation networks represent a substantively important
form of acyclic digraphs. For example, the study of such networks includes the
spread of ideas through academic citations, the spread of innovation
through patent citations, and the development of precedent in common law
systems.
26. Daniel Martin Katz, The MIT
School of Law? A Perspective on
Legal Education in the 21st
Century, University of Illinois
Law Review 1431 (2014)
New York Times - August 1, 2014
Daniel Martin Katz, an associate professor with
expertise in big data and powerful computing and their
applications to legal studies. He hopes to give his
students a leg up in a job market that seems
increasingly bleak, and to help them become “T-
shaped,” by which he means having deep knowledge —
the downward swipe of the letter T — as well as a
broadened set of abilities. So providing them with
information on seemingly arcane subjects like data
analytics can be a career builder. “Analytics plus law
gets you into a niche,” he said.
42. Quantitative Legal Prediction
- or -
How I Learned to Stop Worrying and Start
Preparing for the Data Driven Future of the
Legal Services Industry
Professor Daniel Martin Katz
43.
44.
45.
46.
47.
48.
49.
50. Today we are going to
talk about one key
idea in prediction
54. For today we will apply
these approaches to the
decisions of the
Supreme Court of United States
55. Every year, law reviews, magazine and
newspaper articles, television and radio
time, conference panels, blog posts, and
tweets are devoted to questions such as:
How will the Court rule in particular cases?
56.
57. There are only 3 ways
to predict something
Experts
Crowds
Algorithms
59. Columbia Law Review
October, 2004
Theodore W. Ruger, Pauline T. Kim,
Andrew D. Martin, Kevin M. Quinn
Legal and Political Science
Approaches to Predicting
Supreme Court Decision
Making
The Supreme Court
Forecasting Project:
84. “Software developers were asked on two
separate days to estimate the completion
time for a given task, the hours they
projected differed by 71%, on average.
When pathologists made two assessments of
the severity of biopsy results, the correlation
between their ratings was only .61 (out of a
perfect 1.0), indicating that they made
inconsistent diagnoses quite frequently.
Judgments made by different people are
even more likely to diverge.”
93. Our approach is a special version
of random forest
Black
Reed
Frankfurter
Douglas
Jackson
Burton
Clark
Minton
Warren
Harlan
Brennan
Whittaker
Stewart
White
Goldberg
Fortas
Marshall
Burger
Blackmun
Powell
Rehnquist
Stevens
OConnor
Scalia
Kennedy
Souter
Thomas
Ginsburg
Breyer
Roberts
Alito
Sotomayor
Kagan
1953 1963 1973 1983 1993 2003 2013
9-0 Reverse
8-1, 7-2, 6-3
19 19 19 19 19 20 20
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
- Reverse
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
-
8-1, 7-2, 6-3
9-0
19 19 19 19 19 20 20
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2463244
http://arxiv.org/abs/1407.6333
available at
Revise and Resubmit @ PloS One
94.
95. we have developed an
algorithm that we call
{Marshall}+
random forest
96. Benchmarking
since 1953
+
Using only data
available prior to
the decision
Mean Court Direction [FE]
Mean Court Direction 10 [FE]
Mean Court Direction Issue [FE]
Mean Court Direction Issue 10 [FE]
Mean Court Direction Petitioner [FE]
Mean Court Direction Petitioner 10 [FE]
Mean Court Direction Respondent [FE]
Mean Court Direction Respondent 10 [FE]
Mean Court Direction Circuit Origin [FE]
Mean Court Direction Circuit Origin 10 [FE]
Mean Court Direction Circuit Source [FE]
Mean Court Direction Circuit Source 10 [FE]
Difference Justice Court Direction [FE]
Abs. Difference Justice Court Direction [FE]
Difference Justice Court Direction Issue [FE]
Abs. Difference Justice Court Direction Issue [FE]
Z Score Difference Justice Court Direction Issue [FE]
Difference Justice Court Direction Petitioner [FE]
Abs. Difference Justice Court Direction Petitioner [FE]
Difference Justice Court Direction Respondent [FE]
Abs. Difference Justice Court Direction Respondent [FE]
Z Score Justice Court Direction Difference [FE]
Justice Lower Court Direction Difference [FE]
Justice Lower Court Direction Abs. Difference [FE]
Justice Lower Court Direction Z Score [FE]
Z Score Justice Lower Court Direction Difference [FE]
Agreement of Justice with Majority [FE]
Agreement of Justice with Majority 10 [FE]
Difference Court and Lower Ct Direction [FE]
Abs. Difference Court and Lower Ct Direction [FE]
Z-Score Difference Court and Lower Ct Direction [FE]
Z-Score Abs. Difference Court and Lower Ct Direction [FE]
Justice [S]
Justice Gender [FE]
Is Chief [FE]
Party President [FE]
Natural Court [S]
Segal Cover Score [SC]
Year of Birth [FE]
Mean Lower Court Direction Circuit Source [FE]
Mean Lower Court Direction Circuit Source 10 [FE]
Mean Lower Court Direction Issue [FE]
Mean Lower Court Direction Issue 10 [FE]
Mean Lower Court Direction Petitioner [FE]
Mean Lower Court Direction Petitioner 10 [FE]
Mean Lower Court Direction Respondent [FE]
Mean Lower Court Direction Respondent 10 [FE]
Mean Justice Direction [FE]
Mean Justice Direction 10 [FE]
Mean Justice Direction Z Score [FE]
Mean Justice Direction Petitioner [FE]
Mean Justice Direction Petitioner 10 [FE]
Mean Justice Direction Respondent [FE]
Mean Justice Direction Respondent 10 [FE]
Mean Justice Direction for Circuit Origin [FE]
Mean Justice Direction for Circuit Origin 10 [FE]
Mean Justice Direction for Circuit Source [FE]
Mean Justice Direction for Circuit Source 10 [FE]
Mean Justice Direction by Issue [FE]
Mean Justice Direction by Issue 10 [FE]
Mean Justice Direction by Issue Z Score [FE]
Admin Action [S]
Case Origin [S]
Case Origin Circuit [S]
Case Source [S]
Case Source Circuit [S]
Law Type [S]
Lower Court Disposition Direction [S]
Lower Court Disposition [S]
Lower Court Disagreement [S]
Issue [S]
Issue Area [S]
Jurisdiction Manner [S]
Month Argument [FE]
Month Decision [FE]
Petitioner [S]
Petitioner Binned [FE]
Respondent [S]
Respondent Binned [FE]
Cert Reason [S]
Mean Agreement Level of Current Court [FE]
Std. Dev. of Agreement Level of Current Court [FE]
Mean Current Court Direction Circuit Origin [FE]
Std. Dev. Current Court Direction Circuit Origin [FE]
Mean Current Court Direction Circuit Source [FE]
Std. Dev. Current Court Direction Circuit Source [FE]
Mean Current Court Direction Issue [FE]
Z-Score Current Court Direction Issue [FE]
Std. Dev. Current Court Direction Issue [FE]
Mean Current Court Direction [FE]
Std. Dev. Current Court Direction [FE]
Mean Current Court Direction Petitioner [FE]
Std. Dev. Current Court Direction Petitioner [FE]
Mean Current Court Direction Respondent [FE]
Std. Dev. Current Court Direction Respondent [FE]
0.00781
0.00205
0.00283
0.00604
0.00764
0.00971
0.00793
TOTAL 0.04403
Justice and Court Background Information
Case Information
0.00978
0.00971
0.00845
0.00953
0.01015
0.01370
0.01190
0.01125
0.00706
0.01541
0.01469
0.00595
0.02014
0.01349
0.01406
0.01199
0.01490
0.01179
0.01408
TOTAL 0.22814
Overall Historic Supreme Court Trends
0.00988
0.01997
0.01546
0.00938
0.00863
0.00904
0.00875
0.00925
0.00791
0.00864
0.00951
0.01017
TOTAL 0.12663
Lower Court Trends
0.00962
0.01017
0.01334
0.00933
0.00949
0.00874
0.00973
0.00900
TOTAL 0.07946
0.00955
0.00936
0.00789
0.00850
0.00945
0.01021
0.01469
0.00832
0.01266
0.00918
0.00942
0.00863
0.00894
0.00882
0.00888
Current Supreme Court Trends
TOTAL 0.14456
Individual Supreme Court Justice Trends
0.01248
0.01530
0.00826
0.00732
0.01027
0.00724
0.01030
0.00792
0.00945
0.00891
0.00970
0.01881
0.00950
0.00771
TOTAL 0.14323
0.01210
0.00929
0.01167
0.00968
0.01055
0.00705
0.00708
0.00690
0.00699
0.01280
0.01922
0.02494
0.01126
0.00992
0.00866
0.01483
0.01522
0.01199
0.01217
0.01150
TOTAL 0.23391
Differences in Trends
101. Columbia Law Review
October, 2004
Theodore W. Ruger, Pauline T. Kim,
Andrew D. Martin, Kevin M. Quinn
Legal and Political Science
Approaches to Predicting
Supreme Court Decision
Making
The Supreme Court
Forecasting Project:
102. Given Some Data:
(X1, Y1), ... , (Xn, Yn)
Now We Have a New Set of X’s
We Want to Predict the Y
103. Form a BinaryTree that
Minimizes the Error
in each leaf of the tree
CART
(Classification & RegressionTrees)
112. 1
0
1
1
1
0
0
0
0
0
1
1 1
1
0
0
1
1
1
1
0
01
0
Xi1
Xi2
0
1 2
1
2
Adapted from Example
By Mathematical Monk
If No - then we are in zone (a) ...
we tally the number of zeros and ones
Using Majority Rule do we assign a
classification to this rule this leaf
split 1
(a)
Xi1 > 1 ?
YesNo
(0,5)
Classify as 1
zone (a)
138. expert crowd algorithm
via back testing we can learn the
weights to apply for particular problems
ensemble method
learning problem is to discover when to use a given stream of intelligence
152. Myriad Genetics
“Myriad employs a number of proprietary
technologies that permit doctors and patients
to understand the genetic basis of human
disease and the role that genes play in the
onset, progression and treatment of disease.”
153. Myriad Genetics
“Myriad was the subject of scrutiny
after it became involved in a lengthy
lawsuit over its controversial patenting
practices” which including the
patenting of human gene sequences ....
173. lots of litigation decisions
are just a version of this basic idea
law = finance
174. this is a part of the
industry where you
need rigorous
#LegalUnderwriting
175. but lots of litigation decisions
are actually implicit litigation finance
(or self insurance)
#fin(legal)tech
176. however most implicit litigation
finance is not based upon
rigorous underwriting …
law =! finance
(but it will)
177. we expand on this theme in this presentation
http://computationallegalstudies.com/2015/10/fin-legal-tech-laws-future-from-finances-past-katz-bommartio/
186. Michael J. Bommarito II
@ mjbommar
computationallegalstudies.com
lexpredict.com
bommaritollc.com
university of michigan center for the study of complex systems@
187. Daniel Martin Katz
@ computational
computationallegalstudies.com
lexpredict.com
danielmartinkatz.com
illinois tech - chicago kent college of law@