Metrics for law firms


Published on

"How is a Managing Partner, Head of Knowledge or indeed anyone invlved in a knowledge project or system (in the wide sense) to know whether it is value for money?"

This is an article from December 2012/January 2013 Managing Partner Magazine about using measurement techniques to improve your law firm's efficacy.

Published in: Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Metrics for law firms

  1. 1. KM Healthcheck “Not everything that counts can be counted: not everything that can be counted, counts” Einstein.Three things you will learn from this masterclass 1. Why you need to measure the performance of your firm’s KM systems 2. Which metrics to combine to obtain a balanced measurement of performance 3. How to manage the potential “gaming” of results by fee earners and staffHow is a Managing Partner, Head of Knowledge or indeed anyone involved in a knowledgeproject or system1 to know whether it is value for money?This is an enormously difficult question. It is probably tempting to institute a simple time-related measurement system with time-based targets, similar to those used to measure feeearners. Unfortunately whilst this may confirm the activity levels of your knowledgeprofessionals, it may not tell you anything about the value of those actions. There is nopoint a professional support lawyer working diligently for long hours on a particular methodof sharing, say, current awareness when that method itself is ineffective for fee earners.Can you measure the efficacy of knowledge systems?Some would say that it is impossible to measure knowledge systems: “knowledge” is whathappens inside someone’s head as they synthesize new information with existingexperiences, so how can that be measured? The effects of knowledge systems are alsodifficult to measure because they tend to be intangible and intricately entwined with othersystems and affected by serendipitous events. For example, a fee earner is mulling over acomplex issue and is planning to look at the firm’s knowledge bank after making a coffee.At the coffee machine he bumps into another fee earner, chats to her for a while anddiscovers that she has recently dealt with the same issue and found a neat solution. Is this asuccess for the knowledge-sharing culture, a failure of the knowledge bank, neither or both?Similarly if you are trying to understand whether it is more cost-effective for lawyers towork in X way than in Y way, this may have a more complicated result than a change inchargeable hours, i.e. there could be lower chargeable hours per matter and lower billinglevels in the short term, but higher quality advice, higher customer satisfaction, lower1 I use “knowledge systems” in a broad sense, to include all kinds of knowledge-focused work, not just ITsystems. This could include formal and informal teaching, networking events, efforts to encouragecollaboration and knowledge-based marketing such as newsletters and seminars, amongst other knowledgework. Managing Partner Magazine December 2012/January 2013 page 52
  2. 2. professional negligence claims, more profitable fixed fees, lower attrition rates and morematters undertaken or higher profit fixed fees.That isn’t to say that firms shouldn’t try to get some understanding of the value of theirknowledge projects. Managing Partners need some understanding of the return oninvestment offered by different knowledge projects and so whether to expand a pilot orditch something unproductive. What it does mean is that measuring the success orotherwise of knowledge projects is complex and needs a multifaceted approach.Importantly, before embarking on designing measurement systems for knowledge assets,you must ask yourself why you need this measurement. What is the business benefit ofknowing whether this knowledge system (in the wide sense) is working well or not? In anideal world you would have multiple measurement systems on all your knowledge systemsand act on those results adjusting and replacing systems continuously, but many businessescannot afford to work this way. It is therefore helpful to have an inkling of what you mightdo about the result, before you start measuring. For example, you hear on the grapevinethat your competitors are replacing their old off-the-shelf CRM systems and you wonder ifyou ought to have a better understanding of how user-friendly your CRM system is, howvaluable staff are finding it compared to a newer version which is supposed to be betteradapted to how lawyers work. This is an excellent idea, but if you simply do not have thebudget to replace your CRM system in the near future, but perhaps do have the budget thisyear to tackle your training systems, then leave measurement of your CRM for now, until itbecomes more of a priority and concentrate on those aspects that you both need to and canafford to improve.Choosing measurement systems (1) balancing measurements and avoiding “gaming”There is a saying, often attributed to Peter Drucker, that what gets measured gets done.This means different things to different people, but I take it to mean that: 1. once measurement systems are in place staff will take a project more seriously; and 2. people consciously or subconsciously try to “game” systems to improve their score, whatever the effect on the system that is being measured, causing unexpected distortions.For example, if you try to ascertain the value of a webpage containing current awarenessmanaged by a Knowledge Lawyer by measuring the number of views or page-loads of thatwebpage, you may find that the lawyer adapts the information put on the webpage to makeit more interesting to readers. This is obviously not a problem if a fascinating hookencourages the team to read useful current awareness which makes them better informedabout client-needs, helps them produce higher quality legal advice and/or reducesprofessional negligence claims. This is a problem if staff read the funny story and not the Managing Partner Magazine December 2012/January 2013 page 52
  3. 3. current awareness information, or the quality of the current awareness is poor, or thereturn from the information for the business doesn’t justify the amount of time spent oncollating it.Gaming is difficult to avoid because people are almost always smarter than systems. Thebest way to smooth out any effects is to measure results from a number of different angles:qualitative and quantitative, leading and lagging, and from different perspectives.QualitativeRelating to/based on the quality or character of something.Examples: focus groups, questionnaires with open questions, observationsQuantitativeRelating to/based on the amount or number of something.Examples: page-loads, usage statistics, numbers of complaints, number of knowledgestories, outage levels, sales earned following seminarQuantitative measurements are often simple and cheap to collect and provide numericresults which are naturally easier to compare and contrast. For these reasons they are oftenthe first thing that managers tend to measure. Unfortunately they are particularlysusceptible to “gaming” by staff. Qualitative measurements are more difficult and costly tocollect, give less clear results, but are far more difficult to “game”. For example, the feeearner whose current awareness work is measured by numbers of page-loads may easilysubconsciously game the results, but would find it far harder to affect the result of a focusgroup of fee earners asked about the value of current awareness, or the result of a clientsurvey with open questions about the quality of fee earners’ advice and awareness ofcurrent industry events. The difficulty with solely using qualitative measurements is that, aswell as the cost and time to the business in collecting them, those offering their opinionscan quickly get tired of giving feedback, so they need to be used judiciously.Leading measurementsIndicators that drive the performance of lagging measurements and tend to be forwardlooking.Examples: surveys, usage, web stats, numbers of documents available, focus groupsLagging measurementsMeasurements of what has already happened at the end of a set time period.Examples: PPEP, chargeable and non-chargeable time recordedMost businesses tend to measure things that have happened in a set period in the past:financial results, time recorded, PPEP, effects on share price. The difficulty with only taking Managing Partner Magazine December 2012/January 2013 page 52
  4. 4. this approach is that, just as with the stock market, past performance isn’t always a goodindicator of future performance. Just because your precedent database has served you wellenough in the past, does not mean that it will suit your business in the future or that it is themost effective means of managing precedents. Leading measurements concentrate on thecomponents that make up the lagging indicator and so can be used as early warningsystems. Unfortunately, they can be difficult to identify and capture and are often moresusceptible to gaming. As with qualitative and quantitative, the best solution is to have amixture of leading and lagging indicators.Balanced perspectivesKaplan and Norton first proposed The Balanced Scorecard in 1992. They understood themisleading signals that traditional financial accounting measures gave and so proposed thatbusinesses should ask four basic questions: 1. How do customers see us? (customer perspective) 2. What must we excel at? (internal perspective) 3. Can we continue to improve and create value? (innovation and learning perspective) 4. How do we look to shareholders? (financial perspective)As well as being a useful means of developing a strategy and keeping a whole business oncourse, these perspectives are useful for those measuring the benefit of different projects.For example, there is no point in continuing with the pilot of a new Knowledge system whichprovides great customer experience, but which doesn’t translate into increased profitabilityin some way, whether that is through increased new work from existing customers, awidening of the customer base or making existing work levels more profitable. Similarly,there is no point in a Knowledge system which satisfies financial requirements, butnegatively affects attrition rates and the firm’s ability to attract talented staff, or the firmwill soon have no one to fulfil client-needs.Choosing measurement systems (2) know your audienceWhen designing a measurement system, the other key aspect you should bear in mind is theaudience for the results. If you are the Managing Partner who wants to understand aknowledge system’s efficacy for your own benefit, this isn’t a problem, but if you are a Headof Knowledge who has to report on the efficacy of systems to a number of differentaudiences, you will probably need different measurements or descriptions of measurementsfor different audiences. If you need to report to the investment/finance board to explainthe return on investment of a new system you will use different measurement systems topersuade them and explain results in a different language to that you use for junior Managing Partner Magazine December 2012/January 2013 page 52
  5. 5. members of staff to explain why changes have to be made and perhaps encourage somecreative thinking about solutions to knowledge problems.ConclusionMeasuring the business benefit of knowledge systems is far from straightforward, but thereare strategies that you can adopt to ameliorate the difficulties, focus your efforts and makethe most of your results. The key to successful measurement is to understand its limitationsand to take a multifaceted approach.Checklist 1. Why do you need the measurement? What is the business benefit of knowing? 2. What will you can do about the result? Can you afford to change things if the results are negative? 3. What can you afford to spend on measuring? 4. Be clear who will collect the data and how they will do it. Who will analyse the data and who will present it to the audience? 5. Use existing measurements where they already exist and give insight/add value. 6. Have a mix of quantitative and qualitative measurements, leading and lagging measures and remember that not all measurements are equal, some results are more important than others. 7. Don’t measure everything or get distracted by what other businesses are measuring. Focus on what is important to your business. 8. Look at long term trends primarily, but keep an eye on the present. 9. Speak in the language of your audience when presenting results. 10. Keep measurement methods and results under review and act on the results. Managing Partner Magazine December 2012/January 2013 page 52
  6. 6. Author profileHélène Russell is a solicitor, now non-practising, and specialist in knowledge management.She is the author of “Knowledge Management Handbook” for the Law Society, published inJuly 2012 and has spoken for Ark, LexisNexis, UWE and BLS.She is the founder of Knowledge Network West, the knowledge-sharing group for PSLs,information professionals and Heads of Knowledge in the West. Through The KnowledgeBusiness she offers advice, training and implementation help to law firms wishing toimprove their businesses through their knowledge systems, including a fixed price healthcheck package.Find out more in touch email tel 07548 912 779 twitter @heleneadby Managing Partner Magazine December 2012/January 2013 page 52