Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz

1,512 views

Published on

Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz

Published in: Law
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz

  1. 1. Can Librarians Help Law Become More Data Driven ?  an open question in need of a solution daniel martin katz blog | ComputationalLegalStudies.com corp | LexPredict.com page | DanielMartinKatz.com edu | illinois tech - chicago kent law lab | TheLawLab.com
  2. 2. A Long History Innovation in Law
  3. 3. One of the First to Teach Law with Computers
  4. 4. Helped Launch the Premier State Wide Legal Aid Website
  5. 5. Helped 3.5 million+ Users Seek Access to Justice Guided Interview Completed Document A2J AUTHOR www.a2jauthor.org LOGIC Used over 3.5 Million times 2.1 Million Documents generated IMPACT
  6. 6. Building Upon Our Tradition
  7. 7. thelawlab.com
  8. 8. #LegalScience
  9. 9. American Federal Judiciary American Law Professoriate Building New Algorithms Large Scale Judicial Studies Scientific Research
  10. 10. 3D HD Visualization of Supreme Court Citation Network Campaign Contributions and Legislative Ecosystems Six Degrees of Marbury v. Madison Electronic World Treaty Index Radial SCOTUS Citation Network Scientific Research
  11. 11. Scientific Research
  12. 12. Scientific Research
  13. 13. Polytechnic Legal Training
  14. 14. http://www.quantitativemethodsclass.com/Professor Daniel Martin Katz Intro Class
  15. 15. http://www.legalanalyticscourse.com/Professor Daniel Martin Katz Professor Michael J. Bommarito II Advanced Class
  16. 16. Building Ties with the Legal Industry
  17. 17. TheLawLabChannel.com
  18. 18. FinLegalTechConference.comNovember 4, 2016
  19. 19. Fin(Legal)Tech Conference finlegaltechconference.com @Illinois Tech - Chicago Kent College of Law
  20. 20. Fin(Legal)Tech Conference October 19 2017 FinLegalTechConference.com @LawLaboratory Chicago, IL
  21. 21. Can Librarians Help Law Become More Data Driven ?  an open question in need of a solution daniel martin katz blog | ComputationalLegalStudies.com corp | LexPredict.com page | DanielMartinKatz.com edu | illinois tech - chicago kent law lab | TheLawLab.com
  22. 22. Today — a session in five parts …
  23. 23. A Reset on Robot LawyersI. The Rise of #LegalAnalyticsII. The Killer Use Case(s) - Fin (Legal) Tech)III. The Infrastructure for #LegalAnalyticsIV. Building a Legal Data StrategyV.
  24. 24. A Reset on Robolawyers Part I< >
  25. 25. There has been lots of recent interest in applying artificial intelligence to law
  26. 26. and there is a bit of confusion as to where we stand today and where we are headed
  27. 27. data driven AI rules based AI Competing Orientations in Artificial Intelligence
  28. 28. expert systems Computational Law Data Driven Rules Based prediction models and methods network analytic methods natural language processing self executing law visual law computable codes
  29. 29. we see a decent amount of rules based AI in legal industry
  30. 30. Three Examples of Rules Based A.I.
  31. 31. tax preparation software
  32. 32. Rules Based A.I.
  33. 33. Rules Based A.I.
  34. 34. Among other things Neota has been used to create decision trees to support lawyers / non lawyers
  35. 35. What do I do if there has been An issue in Human Resources ? A potential FCPA violation? A potential data breach?
  36. 36. Decision Trees are a step by step memorialization of best practices
  37. 37. At my home institution - Illinois Tech Chicago-Kent Law has a platform that allows individuals to automate various legal forms, etc.
  38. 38. used by a variety of legal aid organizations
  39. 39. A2J AUTHOR www.a2jauthor.org
  40. 40. PROCESS Guided Interview Completed Document
  41. 41. LOGIC
  42. 42. DECISION TREE
  43. 43. Used over 3.5 Million times 2.1 Million Documents generated IMPACT
  44. 44. Expert Systems 
 (together with data) 
 will eventually become Chatbots …
  45. 45. Client Intake More Seamless Client Interaction via Tech Platform Providing Legal Information to Non-Lawyers in Large Organizations
  46. 46. so although we see a decent amount of rules based AI in legal industry
  47. 47. I am pretty bearish on Rules Based A.I. for most (but not all) applications …
  48. 48. my views are informed by the history of A.I. in general
  49. 49. lots of issues with expert systems and/or rules based A.I. (without data or an evolutionary dynamic)
  50. 50. rules based A.I. data driven A.I. 1980’s, 1990’s, Early 2000’s >
  51. 51. rules based A.I. data driven A.I. 1980’s, 1990’s, Early 2000’s > rules based A.I. data driven A.I. 2005 - Present < ~
  52. 52. Ultimately we are trying to learn the rules / dynamics that underlie some class of activity
  53. 53. With that understanding we want to be able to mimic / predict
  54. 54. There are some use cases for Rules Based AI / Expert Systems
  55. 55. Practically ZERO Top Tier Companies Building Expert Systems
  56. 56. expert systems Computational Law Data Driven Rules Based prediction models and methods network analytic methods natural language processing self executing law visual law computable codes
  57. 57. A.I. State of the Art
  58. 58. A.I. State of the Art purely data centric
  59. 59. A.I. State of the Art purely data centric augment expert forecasts w/ data
  60. 60. iterative data < > rules A.I. State of the Art purely data centric augment expert forecasts w/ data
  61. 61. Again - I like Chatbots because they end up being a massive data collection effort … iterative data < > rules
  62. 62. But as a general matter …
  63. 63. In the Rules vs. Data Debate in A.I.
  64. 64. Data Won the War (Terms of Surrender are Available)
  65. 65. The Rise of #LegalAnalytics Part II< >
  66. 66. Law is a relatively small vertical and there is lots of diversity among tasks lawyers undertake …
  67. 67. Given large fixed costs infrastructure + human capital (data scientists)
  68. 68. harder to successfully deploy high quality enterprise applications for relatively narrow (sub)verticals
  69. 69. in addition there is a borderline pathological numerophobia among lawyers
  70. 70. plus the implicit (explicit) challenge of partnership as the dominant form of the organization within our market
  71. 71. taken together this has challenged the deployment 
 of analytics in legal
  72. 72. Analytics / Quant Legal Prediction has come to law Notwithstanding these head winds—
  73. 73. #LegalAnalytics Quantitative Legal Prediction
  74. 74. #LegalAnalytics Quantitative Legal Prediction
  75. 75. #LegalAnalytics Quantitative Legal Prediction
  76. 76. #LegalAnalytics Quantitative Legal Prediction
  77. 77. #LegalAnalytics Quantitative Legal Prediction
  78. 78. #LegalAnalytics Quantitative Legal Prediction
  79. 79. Some Commercial Applications
  80. 80. In a real sense, this represents just a narrow set of products
  81. 81. #ContractAnalytics Quantitative Legal Prediction
  82. 82. #JudicialAnalytics Quantitative Legal Prediction
  83. 83. #PredictiveCoding #E-Discovery Quantitative Legal Prediction
  84. 84. General Counsels as Legal Procurement Specialists TyMetrix/ELM - Using $50 billion+ in Legal Spend Data to Help GC’s Look for Arbitrage Opportunities, Value Propositions in Hiring Law Firms #LegalSpendAnalytics Quantitative Legal Prediction
  85. 85. #LegalAnalytics Quantitative Legal Prediction https://lexsemble.com/
  86. 86. #NegotiationAnalytics Quantitative Legal Prediction
  87. 87. Here are just a subset of the tasks that we are trying to accomplish in legal …
  88. 88. #Predict Case Outcomes Data Driven Legal Underwriting
  89. 89. #Predict Case Outcomes Data Driven Legal Underwriting #Predict Legal Costs Data Driven Legal Operations
  90. 90. #Predict Relevant Documents #Predict Case Outcomes Data Driven Legal Underwriting Data Driven EDiscovery/Due Diligence (Predictive Coding) #Predict Legal Costs Data Driven Legal Operations
  91. 91. #Predict Relevant Documents #Predict Case Outcomes Data Driven Legal Underwriting Data Driven EDiscovery/Due Diligence (Predictive Coding) #Predict Legal Costs #Predict Rogue Behavior Data Driven Legal Operations Data Driven Compliance
  92. 92. #Predict Relevant Documents #Predict Case Outcomes Data Driven Legal Underwriting Data Driven EDiscovery/Due Diligence (Predictive Coding) #Predict Legal Costs Data Driven Legal Operations Data Driven Compliance #Predict Contract Terms/Outcomes Data Driven Transactional Work #Predict Rogue Behavior
  93. 93. A Deeper Dive on Predicting Predicting Case Outcomes (other problems can be solved using similar methods)
  94. 94. Supreme Court of United States #PredictSCOTUS
  95. 95. There are only 3 ways 
 to predict something Experts Crowds Algorithms
  96. 96. Experts
  97. 97. Columbia Law Review October, 2004 Theodore W. Ruger, Pauline T. Kim, Andrew D. Martin, Kevin M. Quinn Legal and Political Science Approaches to Predicting Supreme Court Decision Making The Supreme Court Forecasting Project:
  98. 98. experts
  99. 99. Case Level Prediction Justice Level Prediction 67.4% experts 58% experts From the 68 Included Cases for the 2002-2003 Supreme Court Term
  100. 100. these experts probably overfit
  101. 101. they fit to the noise and not the signal
  102. 102. if this were finance this would be trading worse than S&P500
  103. 103. #NoiseTrading
  104. 104. #BuffetChallenge
  105. 105. #BuffetChallenge
  106. 106. like many other forms human endeavor law is full of 
 noise predictors …
  107. 107. we need to evaluate experts and somehow benchmark their expertise
  108. 108. from a pure forecasting standpoint
  109. 109. the best known SCOTUS predictor is
  110. 110. the law version of superforecasting
  111. 111. Crowds
  112. 112. crowds
  113. 113. https://fantasyscotus.lexpredict.com/case/list/ We can generate Crowd Sourced Predictions
  114. 114. Just like the Market the Crowd is collectively terrible … < No Alpha >
  115. 115. however, not all members of crowd are made equal
  116. 116. we maintain a ‘supercrowd’ which is the top n% of predictors up to time t
  117. 117. the ‘supercrowd’ outperforms the overall crowd (and the best single player)
  118. 118. For the 2015-2016 term
  119. 119. not enough crowd based decision making in institutions (law included)
  120. 120. “Software developers were asked on two separate days to estimate the completion time for a given task, the hours they projected differed by 71%, on average. W h e n p a t h o l o g i s t s m a d e t wo assessments of the severity of biopsy results, the correlation between their ratings was only .61 (out of a perfect 1.0), indicating that they made inconsistent diagnoses quite frequently. Judgments made by different people are even more likely to diverge.”
  121. 121. not enough crowd based decision making in institutions (aka manual underwriting)
  122. 122. here is a commercial offering
  123. 123. https://lexsemble.com/
  124. 124. Brief Aside About Crowd Sourced Prediction #LegalCrowdSourcing
  125. 125. (most pundits did not identify as a serious candidate him until mid-January 2017) Neil Gorsuch was #1 o n o u r F a n t a s y Platform 12 Days after Donald Trump was elected President (i.e Nov 20)
  126. 126. #FantasySCOTUS
  127. 127. Algorithms
  128. 128. Our algorithm is a special version of random forest (time evolving) http://journals.plos.org/ plosone/article?id=10.1371/ journal.pone.0174698 available at RESEARCH ARTICLE A general approach for predicting the behavior of the Supreme Court of the United States Daniel Martin Katz1,2 *, Michael J. Bommarito II1,2 , Josh Blackman3 1 Illinois Tech - Chicago-Kent College of Law, Chicago, IL, United States of America, 2 CodeX - The Stanford Center for Legal Informatics, Stanford, CA, United States of America, 3 South Texas College of Law Houston, Houston, TX, United States of America * dkatz3@kentlaw.iit.edu Abstract Building on developments in machine learning and prior work in the science of judicial pre- diction, we construct a model designed to predict the behavior of the Supreme Court of the United States in a generalized, out-of-sample context. To do so, we develop a time-evolving random forest classifier that leverages unique feature engineering to predict more than 240,000 justice votes and 28,000 cases outcomes over nearly two centuries (1816-2015). Using only data available prior to decision, our model outperforms null (baseline) models at both the justice and case level under both parametric and non-parametric tests. Over nearly two centuries, we achieve 70.2% accuracy at the case outcome level and 71.9% at the jus- tice vote level. More recently, over the past century, we outperform an in-sample optimized null model by nearly 5%. Our performance is consistent with, and improves on the general level of prediction demonstrated by prior work; however, our model is distinctive because it can be applied out-of-sample to the entire past and future of the Court, not a single term. Our results represent an important advance for the science of quantitative legal prediction and portend a range of other potential applications. Introduction As the leaves begin to fall each October, the first Monday marks the beginning of another term for the Supreme Court of the United States. Each term brings with it a series of challenging, important cases that cover legal questions as diverse as tax law, freedom of speech, patent law, administrative law, equal protection, and environmental law. In many instances, the Court’s decisions are meaningful not just for the litigants per se, but for society as a whole. Unsurprisingly, predicting the behavior of the Court is one of the great pastimes for legal and political observers. Every year, newspapers, television and radio pundits, academic jour- nals, law reviews, magazines, blogs, and tweets predict how the Court will rule in a particular case. Will the Justices vote based on the political preferences of the President who appointed them or form a coalition along other dimensions? Will the Court counter expectations with an unexpected ruling? PLOS ONE | https://doi.org/10.1371/journal.pone.0174698 April 12, 2017 1 / 18 a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Katz DM, Bommarito MJ, II, Blackman J (2017) A general approach for predicting the behavior of the Supreme Court of the United States. PLoS ONE 12(4): e0174698. https://doi. org/10.1371/journal.pone.0174698 Editor: Luı´s A. Nunes Amaral, Northwestern University, UNITED STATES Received: January 17, 2017 Accepted: March 13, 2017 Published: April 12, 2017 Copyright: © 2017 Katz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: Data and replication code are available on Github at the following URL: https://github.com/mjbommar/scotus-predict-v2/. Funding: The author(s) received no specific funding for this work. Competing interests: All Authors are Members of a LexPredict, LLC which provides consulting services to various legal industry stakeholders. We received no financial contributions from LexPredict or anyone else for this paper. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
  129. 129. From a Pure Machine Learning Perspective — Much of this is not novel EXCEPT the time evolving element of the Random Forest
  130. 130. https://github.com/mjbommar/ scotus-predict-v2/
  131. 131. 243,882 28,009 Case Outcomes JusticeVotes Final Version of #PredictSCOTUS 1816-2015 http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174698
  132. 132. Final Version of #PredictSCOTUS 1816-2015 case accuracy 70.2% 71.9% justice accuracy http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174698
  133. 133. Experts, Crowds, Algorithms
  134. 134. http://www.sciencemag.org/news/ 2017/05/artificial-intelligence-prevails- predicting-supreme-court-decisions Professor Katz noted that in the long term …“We believe the blend of experts, crowds, and algorithms is the secret sauce for the whole thing.” May 2nd 2017
  135. 135. For most problems ... ensembles of these streams outperform any single stream
  136. 136. the non-trivial question is how to optimally assemble such streams for particular problems
  137. 137. Humans + Machines Humans or Machines >
  138. 138. Here is what we are working on right now …
  139. 139. expert forecast crowd forecast learning problem is to discover how to blend streams of intelligence algorithm forecast ensemble method ensemble model
  140. 140. expert forecast crowd forecast learning problem is to discover how to blend streams of intelligence algorithm forecast ensemble method ensemble model via back testing we can learn the weights to apply for particular problems
  141. 141. By the way, you might ask why does one care about marginal improvements in prediction ? #Fin(Legal)Tech
  142. 142. It is a fair question because in the private market … improvements in performance must be linked up to an actual business model …
  143. 143. Fin (Legal) Tech is the killer use case Part III< >
  144. 144. Given our ability to offer forecasts of judicial outcomes, we wondered if this information could inform an event driven trading strategy ?
  145. 145. Revise + Resubmit @ http://arxiv.org/abs/1508.05751 available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2649726
  146. 146. lots of litigation decisions are just a version of this basic idea law = finance
  147. 147. from an asset valuation standpoint lots of litigation decisions are actually implicit litigation finance (or self insurance) #fin(legal)tech
  148. 148. Consider for example … Litigation Reserves Setting Under FASB ASC 450-20-25 #fin(legal)tech
  149. 149. But there are many other places where … law = finance
  150. 150. Fin (Legal) Tech
  151. 151. Three Types of Lawyers (as described by paul lippe)
  152. 152. play “whack-a-mole”, reacting to problems by creating fear and friction within organizations and the impression that there is a legal risk around every corner. Mediocre Lawyers
  153. 153. can help clients shape (perhaps distort) external perception of risk. Merely Clever Lawyers
  154. 154. design systems that balance risk and improve transparency, helping clients correctly price risk internally Great Lawyers
  155. 155. law = finance (insuranceaswell)
  156. 156. law < > finance many elements in law look like finance did 25 - 50 years ago (on the long road from Black-Scholes to algorithmic trading)
  157. 157. Lawyer VALUE PROPOSITION (From the Client’s Perspective) (internal or external client)
  158. 158. help price risk / help reduce information asymmetries transactional =
  159. 159. litigation = characterize (predict) risk/exposure shift the expected value of a lawsuit help price risk / help reduce information asymmetries transactional =
  160. 160. litigation = characterize (predict) risk/exposure shift the expected value of a lawsuit compliance = identify + prevent rogue behavior monitor behavior in (near) real time help price risk / help reduce information asymmetries transactional =
  161. 161. litigation = characterize (predict) risk/exposure shift the expected value of a lawsuit compliance = identify + prevent rogue behavior monitor behavior in (near) real time help price risk / help reduce information asymmetries transactional = regulatory = help identify (predict) the decisions of regulators / law makers and the risk associated with various outcomes
  162. 162. Dominant Model in Law expert centered pricing of risk
  163. 163. Dominant Model in Law lots of unintentional self insurance rarely (if ever) based upon explicit risk models
  164. 164. Cult of one 
 (or very small # of) person(s) thinking drives decisions with serious financial consequences
  165. 165. Claim: fin(tech) offers lessons for many areas in law
  166. 166. thesis statement: the financialization of the law will be an important vector of the next decade(s)
  167. 167. #Fin(Legal)Tech application of those ideas and technologies to a wide range of law related spheres including litigation, transactional work and compliance.
  168. 168. Here are just a few of many examples
  169. 169. www.burfordcapital.com/ http://www.gerchenkeller.com/ http://www.fulbrookmanagement.com/ http://www.longfordcapital.com/ http://www.benthamimf.com/ Litigation Finance
  170. 170. Litigation Finance
  171. 171. Litigation Finance
  172. 172. Event Driven Legal Trading
  173. 173. M&A Insurance
  174. 174. Outside of M+A Requires Mapping of Deal Terms to actual substantive outcomes #legaldata #legalanalytics
  175. 175. Being able to compute the change in risk as a function of a change in deal terms
  176. 176. Trading Desk is all about alpha - using data, predictions, process, etc.
  177. 177. Not about simply buying tools off the shelf and deploying them …
  178. 178. The Infrastructure for Legal Analytics - #MLaaS and the Enterprise Open Source Movement Part IV< >
  179. 179. Lots of folks ask me what is next in legal analytics …
  180. 180. A big part of the answer comes from one of the most dominant vectors in tech
  181. 181. both those in positions of leadership and those in technical positions need to take stock
  182. 182. the democratization of machine learning is underway
  183. 183. Emerging Business Model - Machine Learning as a Service #MLaaS
  184. 184. IBM Watson (per se) IBM Watson (as early #MLaaS) vs.
  185. 185. IBM WATSON First major effort at #MLaaS Machine Learning as a Service
  186. 186. The Cloud Wars
  187. 187. Commercial Examples
  188. 188. Machine Learning as a Service #MLaaS
  189. 189. Machine Learning as a Service #MLaaS
  190. 190. Machine Learning as a Service #MLaaS
  191. 191. Machine Learning as a Service #MLaaS
  192. 192. But wait there is more …
  193. 193. Machine Learning as a Service #MLaaS
  194. 194. Machine Learning as a Service #MLaaS Enterprise Open Source Movement #OpenSource +
  195. 195. Enterprise Open Source Movement #OpenSource
  196. 196. https://techcrunch.com/2016/06/19/the-next-wave-in-software-is-open-adoption-software/
  197. 197. historically one needed to build the full stack (i.e end to end) for an application
  198. 198. Standing on 
 the Shoulders of Giants
  199. 199. The (Emerging) Last Mile Problem in (Legal) Analytics
  200. 200. Off the Shelf #MLaaS, etc. (perhaps with some configuration and/or customization) Unique Domain Specific Offering
  201. 201. The New Ball Game
  202. 202. Piece together the combinations of 
 #MLaaS + open source
  203. 203. to build enterprise applications which are unique combinations drawn from across the #MLaaS / open source spectrum
  204. 204. First Wave vs. Second Wave Legal Tech
  205. 205. Second Movers can catch up faster …
  206. 206. Second Movers need less capital …
  207. 207. Second Movers who start now will have lower fixed costs …
  208. 208. Major Implication The Best Legal Tech is Yet to Be Built …
  209. 209. We are beginning to see the first wave of #MLaaS Implementation Companies in General
  210. 210. https://computationallegalstudies.com/2017/05/07/machine-learning- service-mlaas-ecosystem-grows-bonsai-mlaas-implentation-company/ “AI startup Bonsai has raised $7.6 million to grow its platform that simplifies open-source machine learning library TensorFlow to help businesses construct their own artificial intelligence models and incorporate AI into their business.”
  211. 211. And this is in Part What My Company LexPredict will be (already is) doing within law …
  212. 212. https://www.slideshare.net/lexpredict/ contraxsuite-why-were-opensourcing- contraxsuite-and-product-overview #OpenSourceLegal
  213. 213. "We are increasingly thinking that there's room in legal tech for a Red Hat in legal — companies that really focus on development of software by providing wraparound services, but offer their software open source," Michael J Bommarito II said. Michael J. Bommarito Co-Founder CEO @ LexPredict
  214. 214. contraxsuite.com
  215. 215. Will you resell the software to third parties? YES%NO% How much does ContraxSuite cost? Will you keep derivative work open?Free% YES%NO% Free%$12K/year% 50% in trust for open source grants ! 50% for ContraxSuite, LLC!
  216. 216. If you are just buying tools from vendors you likely have no alpha
  217. 217. Building a Legal Data Strategy Part V< > (A Role for Law Librarians?)
  218. 218. every organization in law needs a data strategy
  219. 219. Capture, Clean, Regularize Data to support a range of tasks
  220. 220. Deploy Data for Specific Enterprise Applications Develop a data roadmap
  221. 221. What%is%a%data%strategy?%
  222. 222. Statement and Framework Data Strategy: Defined ! ! ! ! A! data! strategy! combines! a! top2down! mission! statement! acknowledging! the! value! of! an! organiza(on’s+ data! with! a! framework!for!developing!data.driven+capabili(es.+ ! ! ! ! ! While! data! strategies! are! built! on! lists! of! principles! and! technologies,! they! address! much! more:! strategic! communica=on! and! change! management,! process! improvement,!knowledge!management,!and!risk!management,! to!name!a!few.!
  223. 223. MAY–JUNE 2017 ISSUE!
  224. 224. D - I - K - W From Data Strategy to Wisdom Data$ Informa+on$ Knowledge$ Wisdom$ Direct'record'of'fact,' signal,'symbol' Indirect'record'or' descrip6on$ Interpreta6on'of' informa6on$ Ac6onable'inference'or' heuris6c$ Data-Information-Knowledge-Wisdom Data$ Readings'from'a'temperature' sensor'in'Tahoe.$ Informa+on$ The'average'temperature'in'the' month'of'December'is'32.2F.$ Knowledge$ Snow'is'likely'to'accumulate'in' December.' Wisdom$ January'is'a'good'month'to'plan' a'ski'trip'to'Tahoe.$
  225. 225. When is Data Valuable? Even when it’s not LOW$ HIGH$ HIGH$LOW$ IMPACT' FREQUENCY' High3frequency,'high3impact'3'best'use'case'for'data' •  Systema/c'understanding'and'treatment' •  Standardized$reporEng'and'sta/s/cal$treatment' •  PotenEal'for'automaEon'and'predicEon' Example:' •  Labor'&'Employment'for'a'large'employer' •  Patent'Defense'for'a'large'tech'company'
  226. 226. Some%Organiza-ons%Have%Publically% Commi8ed%Themselves%to%Use%Data% to%Become%‘Best%in%Class’%% Legal%Departments%%
  227. 227. 33! “Now! we! have! program! managers,! data! analysts,! business! analysts,! data! scien9sts,! opera9ons! managers,!I!mean,!we!have!a!ton!of! stuff.! That's! the! key! for! me,! is! thinking! about! the! right! people! doing! the! right! tasks.! That's! the! people!part.!And!then!how!they!do! them,! is! the! process,! and! then,! automa9ng! parts,! is! kind! of! that! next,!final!step.!! " And$ all$ of$ that$ is$ underpinned$ by$ d a t a ." Y o u$ c a n ' t$ d o$ a n y$ improvements$ unless$ you$ have$ data.$ You$ can't$ automate$ unless$ you$have$good$data.”!
  228. 228. 36! “From!se)lement!informa0on!and! contracts! to! sensi0ve! client! data! and! beyond,! Liberty! Mutual! creates! and! stores! ever:growing! volumes! of! unorganized! data! across! its! worldwide! offices! and! databases.”! “I've!seen!a!real!transforma0on!in! the! legal! department! just! having! t h a t! i n f o r m a 0 o n! v i s u a l l y! available."! “The' legal' department' is' now' w o r k i n g' p r e d i c 7 v e' a n d' prescrip7ve' analy7cs,"' i.e.' ways' to' analyze' data' that' enable' forecas7ng'for'legal'issues.”'
  229. 229. 34!
  230. 230. 37! “I"believe"strongly"that"data"analy2cs"is" a"new"fron2er"in"the"legal"space.”" Susie!Lees! General!Counsel!! Allstate!! “Leveraging" data," not" only" that" we" possess" but" that" our" law" firms" have" amassed"over"the"years,"offers"a"plethora" of" un<tapped" opportuni=es—not" simply" to" help" us" forecast" and" manage" legal" expenses," but" also" to" help" our" clients" make"more"informed"business"decisions.”"
  231. 231. Why$a$legal%data$strategy?$
  232. 232. Five reasons to care Can you answer these questions? 1. How&many&legal&ma.ers&did&you&handle&last&year?& 2. How&much&poten:al&legal&liability&did&you&handle&last&year?& 3. How&many&hours&per&legal&ma.er&did&you&spend&last&year?& 4. How&many&dollars&per&legal&ma.er&did&you&spend&last&year?& 5. How&much&value&did&you&protect&or&create&last&year?&
  233. 233. 47! Methods for Using (Legal) Data Historical reporting in legal Historical analytics in legal Predictive analytics in legal
  234. 234. 48! Historical reporting in legal Ques'on:+ What! did! we! spend! on! se.lements! and! legal!expenses!last!quarter?! $1.2M+ Ques'on:+ On! average,! how! many!effort!hours!does!staff! counsel! spend! on! the! discovery! phase! of! a! non> compete!dispute?! 25+ hours+
  235. 235. 49! Ques&on:!What! factors!drove! se3lement! amounts!last! quarter?! •  F o r ! l a b o r ! a n d! employment!disputes,!the! length! of! employment! a n d! p r e s e n c e! o f! retaliatory! or! sexual! harassment! claims! are! posi&vely! related! to! se3lement!amount! •  Disputes! origina&ng! in! region!X!have!abnormally! higher! se3lements! than! expected,! given! their! facts! Ques&on:!What! factors!drove! legal!expenses! last!quarter?! •  An! increase! in! ma3ers! in! highCcost! jurisdic&ons! is! posi&vely! related! to! total! legal!expenses! •  A! decrease! in! arbitra&on/ media&on! u&liza&on! is! posi&vely! related! to! total! legal!expenses! Historical analytics in legal
  236. 236. 50! Ques'on:!Should!we!se,le!this!dispute!at!outset?! •  The!counterparty!is!expected!to!accept!an!ini8al!offer! •  The!dispute!is!predicted!to!se,le!for!$100k,!with!legal! expenses!of!$15k! •  If!an!ini8al!offer!is!not!made,!this!dispute!is!expected!to! cost!$50k!in!legal!expenses!and!has!a!25%!chance!of!going! to!jury!trial.! Ques'on:!How!many!effort!hours!will! we!spend!on!this!ma,er?! •  An!es8mate!of!18!hours,!with!90%!confidence!that!the! dispute!will!fall!between!13!and!30!hours! Predictive analytics in legal
  237. 237. 37! Stages!of!Legal!! Data!Strategy!Maturity! Chaotic Managed Defined Data-Driven Continuously Improvin 5!1! 2! 3! 4!
  238. 238. 51! (be able to do so without a herculean effort) 1. !Measure,!monitor,!and!manage!your!resources!and!service!providers.! ! 2. !Using!data!+!experts,!model!and!improve!the!processes!you!execute.! 3. !Allocate!tasks!across!internal/external!resources!and!assess!cost!and!quality.! 4. !Manage!risk!and!be!able!to!formally!characterize!the!risks!avoided.! 5. !Jus&fy)and)explain)performance)to)the)clients.! Five Goals for Every Legal Organization
  239. 239. Deploying a Legal Data Strategy for a Discrete Problem
  240. 240. Y = βo +/- β1 ( X1 ) +/- β2 ( X2 ) +/- β3 ( X3 ) +/- β4 ( X3 ) +/- β5 ( X3 ) + ε Y = $151 + $15 ( ) + 161 ( ) + 95 ( ) + 34 ( ) +/- β5 ( ) + ε Per 100 Lawyers If Tier 1 Market is True Partner Status is True Per 10 Years Practice Area
  241. 241. 1. Define the Parameter Space 3. Select a Model/Method 4. Validate Out of Sample 2. Collect / Normalize Data (typically using experts)
  242. 242. Work with experts to define relevant variables that drive outcomes on some problem (experts are strong at identifying relevant variables but have trouble applying weights)
  243. 243. Figure out how to collect or normalize relevant data
  244. 244. Yes No f( ) Outcome? binary f( ) Outcome? continuous machine learning is the approach to ‘learn’ the best performing f ( ) select a model/method then validated out of sample
  245. 245. https://www.slideshare.net/lexpredict/ developing-a-legal-data-strategy-learning- to-see-data-as-a-strategic-business-asset https://www.slideshare.net/lexpredict/ legal-data-strategy-maturity-assessing- capabilities-and-planning-improvements
  246. 246. Daniel Martin Katz @ computational computationallegalstudies.com lexpredict.com danielmartinkatz.com illinois tech - chicago kent college of law@ thelawlab.com

×