Recommended
Recommended
More Related Content
Similar to Addressing the Great Bottleneck in Legal AI: Harnessing Law Students to Produce Open Access Human-annotated Data Sets
Similar to Addressing the Great Bottleneck in Legal AI: Harnessing Law Students to Produce Open Access Human-annotated Data Sets (20)
Recently uploaded
Recently uploaded (20)
Addressing the Great Bottleneck in Legal AI: Harnessing Law Students to Produce Open Access Human-annotated Data Sets
- 1. Addressing the Great Bottle- neck in Legal AI: Harness- ing Law Students to Pro- duce Open Access Human- annotated Data Sets Arthur Dyevre SAIL 1 June 2021
- 2. 0 About me
- 3. 1 Outline 1 The Challenge: Supervised Learning in Law 2 Solution 1: Small Is Beautiful 3 Solution 2: Legal Education and Legal AI 4 Conclusion: Clearing the Bottleneck 3 Harnessing Law School
- 4. Most important end-user tasks in legal AI require human-labelled data: I Information retrieval
- 5. Most important end-user tasks in legal AI require human-labelled data: I Information retrieval I Document classification
- 6. Most important end-user tasks in legal AI require human-labelled data: I Information retrieval I Document classification I Argumentation mining
- 7. Most important end-user tasks in legal AI require human-labelled data: I Information retrieval I Document classification I Argumentation mining I Overruling and jurisprudential change
- 10. 1 Where is the legal equivalent of ImageNet? I Typical datasets in published research small
- 11. 1 Where is the legal equivalent of ImageNet? I Typical datasets in published research small I Often proprietary
- 12. 1 Where is the legal equivalent of ImageNet? I Typical datasets in published research small I Often proprietary I Annotating legal documents is expensive and time-consuming
- 13. 1 Where is the legal equivalent of ImageNet? I Typical datasets in published research small I Often proprietary I Annotating legal documents is expensive and time-consuming I Requires domain expertise
- 14. 1 Turkers?
- 16. 2 Outline 1 The Challenge: Supervised Learning in Law 2 Solution 1: Small Is Beautiful 3 Solution 2: Legal Education and Legal AI 4 Conclusion: Clearing the Bottleneck 10 Harnessing Law School
- 17. Transformer and Transfer Learning
- 22. 2 Does pre-training work for legal tasks?
- 23. Improving pre-trained models for legal domain
- 25. 2 Complexity and sequence length
- 26. 2 BigBird
- 27. 2 BigBird
- 28. 3 Outline 1 The Challenge: Supervised Learning in Law 2 Solution 1: Small Is Beautiful 3 Solution 2: Legal Education and Legal AI 4 Conclusion: Clearing the Bottleneck 22 Harnessing Law School
- 35. Legal education in Europe: 1 Large student numbers
- 36. Legal education in Europe: 1 Large student numbers 2 Low instructor/student ratio
- 37. Legal education in Europe: 1 Large student numbers 2 Low instructor/student ratio 3 Ex cathedra teaching
- 38. Legal education in Europe: 1 Large student numbers 2 Low instructor/student ratio 3 Ex cathedra teaching 4 Little skills learning
- 39. Proof of Concept: EU Law Classifier
- 42. "...this course was a very nice change from the classic practical class. A regular seminar normally means doing assignments and then it is done. This class however was a completely new way of working. For once it felt like our work was serving a purpose other than just grading. It felt like we were con- tributing something to legal research. This was very new and refreshing and gave me motivation for the class. Lastly, I be- lieve that this class has made me a better jurist. We have had to analyze judgments before in other classes, but they were always handpicked decisions about a certain topic. This class has given us a wider variety of judgments to read, no matter the topic. Reading and annotating these judgments has given me a better understanding of Belgian decisions in general."
- 43. Next step: I Develop integrated app
- 44. Next step: I Develop integrated app I Broaden scope: contract clauses, terms of service...
- 45. Next step: I Develop integrated app I Broaden scope: contract clauses, terms of service... I More tasks: entity recognition, arguments, overruling/distinguishing...
- 46. Next step: I Develop integrated app I Broaden scope: contract clauses, terms of service... I More tasks: entity recognition, arguments, overruling/distinguishing... I Create annotation-ready tasks
- 47. Next step: I Develop integrated app I Broaden scope: contract clauses, terms of service... I More tasks: entity recognition, arguments, overruling/distinguishing... I Create annotation-ready tasks I Scoring and certification
- 48. 4 Outline 1 The Challenge: Supervised Learning in Law 2 Solution 1: Small Is Beautiful 3 Solution 2: Legal Education and Legal AI 4 Conclusion: Clearing the Bottleneck 35 Harnessing Law School
- 49. 4 Conclusion I Progress in Legal AI depend on advances in supervised learning
- 50. 4 Conclusion I Progress in Legal AI depend on advances in supervised learning I Transformer and transfer learning are part of solution
- 51. 4 Conclusion I Progress in Legal AI depend on advances in supervised learning I Transformer and transfer learning are part of solution I Opportunity to create win-win collaborations with law schools to create new data sets
- 52. 4 Conclusion I Progress in Legal AI depend on advances in supervised learning I Transformer and transfer learning are part of solution I Opportunity to create win-win collaborations with law schools to create new data sets I Thanks!