ABA Business Law Section Meeting, New Orleans, April 8, 2017 Data is useful only if you can act on it. This program will help you develop data plans and capture "small data" tailored to your organization’s needs. We will also introduce tools that will help move you from ad hoc, anecdote-driven decision making to methodical, data-driven decision making. Speakers: Connie Brenton, Senior Director of Legal Operations for NetApp Daniel W. Linna, Jr., Professor of Law and Director of Legal RnD – The Center for Legal Services Innovation at Michigan State University College of Law Jamie May, Associate General Counsel, University Hospitals Health System

1. 1Connie Brenton - NetApp Chief of Staff & Sr. Director of Legal Operations
Dan Linna - MSU Law Professor & Director of LegalRnD – The Center for Legal Services Innovation
Jamie May - University Hospitals Associate General Counsel
ABA Business Law Section Meeting
New Orleans, Louisiana
April 8, 2017

2.  You shouldn’t strive to collect data without having clear goals for its
use.
 The right data can help lawyers and their clients manage risk more
effectively, achieve better outcomes, and manage legal operations (in
legal departments and law firms) more effectively.
 You will probably have to collect and analyze useful data yourself.
2

3.  Transactions
 Litigation
 Managing a Legal Department and External Lawyers
3

4. 4

5. Research has confirmed that in many tasks, experts’ decisions are
highly variable … .
[P]rofessionals often make decisions that deviate significantly
[1] from those of their peers,
[2] from their own prior decisions, and
[3] from rules that they themselves claim to follow.
5

6. 6
eBilling & Matter
Management

7.  Establish consistency and standardization in risk assessment
 Standardized Risk Assessment demo:
https://uhhsandbox.logicmanager.com/
 Know the price of providing legal services
7

8.  Identification
 Estimation (likelihood and value)
 Evaluation
 Treatment
Mahler, Tobias, Tool-supported Legal Risk Management: A Roadmap, European Journal of Legal Studies (2010), available at:
http://cadmus.eui.eu/handle/1814/15122
8

9.  Specify scope and target
 Client objectives?
 How will risk be documented and measured?
 Criteria for risk evaluation?
 Risk identification
 For each clause:
 How will subsequent facts and decisions negatively affect
objectives?
 Risk estimation and evaluation
 likelihood and value
 Risk treatment and cost-benefit analysis
9

10. Source: Just Get PMP http://www.justgetpmp.com/2012/02/probability-and-impact-matrix.html
10

11.  Keep Score: Capture Small Data
 Use external data sources
 Systematically assess dispute outcomes and costs
 Communicate with the client early and often
 Use visual tools
11

12.  Why decision trees?
 Systematic approach
 Identify uncertainty
 “Level set” lawyers and clients
12

13.  parties
 lawyers
 courts
 judges
13

14. • Matter type
• Claim type
• Jurisdiction
• Judge
• Lead lawyer
• Exposure
• Expected value at Early Case Assessment
• Fees estimated at Early Case Assessment
• Settlement amount
• Days to decision on Motion to Dismiss
• Days to decision on Summary Judgment
• Days to trial completion
• Total fees
14

15. 15

16. 16

17. 17

18. 18

19. 19

20. 20

21.  Update and reassess probabilities and expected values
 Remember: Expected value assumes risk neutrality
 Update fee and cost forecasts
 Continue to communicate with your client
21

22.  Incorporate fees & costs (including opportunity costs)
 Create dependency diagrams and lists of reasons
 Conduct sensitivity analyses – Where do you get best ROI?
 Consult outside data sources
 Review Glidden, Robertson, & Victor article
 Systematically refine factual and legal issues
22

23. 23

24. Performance Summary
Quantitative Portfolio Analysis
Category Performance Details Score
Staffing
Models
Staffing
Efficiency
Fees/Costs
Qualitative Analysis
Category Score
Understands and Aligns with our
Business
SharedValues
Quality of Work Product
Responsiveness / Timeliness
Effective Use of Technology
Innovation / Creativity
Service Delivery
Results / Outcomes
Cost Effectiveness
Transparency and Information Sharing
Response Score
Unsatisfactory 1.0
Improvement Needed 2.0
Meets Expectations 3.0
Exceeds Expectations 4.0
Exceptional 5.0

25. Qualitative Review – CLOC.ORG
• In-House requested performance feedback from in-house attorneys on [Firm Name]. [Insert result of likelihood in
recommending the firm for future work.]
Category Description Score
Understands and
Aligns with our
Business
• Demonstrate knowledge of our business and industry
• Strategic and proactive, early involvement
• Firm acts like they are part of our team and invested in our success
• Few if any requests for waivers
Shared Values
• Demonstrated commitment to diversity and inclusion
• Sponsors and/or contributes to pro bono and community service projects
Quality of Work
Product
• Demonstrates in-depth subject matter expertise
• Provides actionable legal opinions that help solve business problems
• Delivers crisp and high quality work product and presentations
Responsiveness /
Timeliness
• Provides timely and concise communications
• Quick to provide meaningful responses in time sensitive situations
• Responsive to phone calls/emails and strives to be available when needed
Effective Use of
Technology
• Demonstrates working knowledge of available tech to improve service delivery and/or reduce cost
• Actively trains staff on effective use of tools
• Presents opinions to employ technology on our matters
Innovation /
Creativity
• Thinks outside the box to resolve business and legal challenges
• Understands company cost constraints and responds with creative solutions
• Value-added service offerings (CLE, free/discounted secondments)
Service Delivery
• Consistently staffs the best available talent and subject matter expertise
• Staffing levels are appropriately matched to the size and complexity of the matter
• Partners/works well with other firms to serve the company
Results / Outcomes
• Consistently achieves favorable interim and final outcomes for the company
• Positive results relative to liabilities averted, reduced settlements, recoveries attained
Cost Effectiveness
• Cost relative to value delivered/outcomes
• Develops useful, realistic budgets and manages to them with regular communication
• Effective use of tools and technology to reduce the cost of service
• Provides fee arrangements which share risk and/or reduce cost
Transparency and
Information Sharing
• Provides metrics and data on case statistics, performance, and outcomes
• Firm shares trends, benchmarks

26. Best in Class Metrics (draft)
 Total Legal Spend as % of Revenue
 Inside Spend as % of Revenue
 External Spend as % of Revenue
 Outside Counsel Spend vs Other Legal Spend
 Litigation / IP / Other as % of External Spend
 Lawyers per $B
 Hourly spend vs Fixed
 # of active firms