30. Typical Intangibles Job Satisfaction Productivity Job Commitment Work Environment Morale Employee Retention Innovation Competencies Leadership Team work Cooperation Decision-making Communication Client satisfaction Client complaints Client response time
31.
32. 1. Does an acceptable, standard metric exist for the measure? 2. Can the conversion be accomplished with minimum resources? 3. Can the conversion process be described to an executive director and secure their buy-in in five minutes? Conversion Test
33. “ Anything can be measured in a way that is superior to not measuring it at all“
38. Staff Surveys Staff Interviews Client surveys Clients Interviews Data from system Task Analysis Times Sheet Analysis Articles Field research Observation Secondary research reports Benchmark studies Budget and Financial Analysis
91. How can Pauline use ROI analysis to get buy-in from Boss and others? Type into chat
92. Focus on benefits, not features Resist fondling the hammer Don’t ignore intangibles Use ROI Story Telling techniques Compare costs of alternatives, cost/benefit, and cost of not doing Pilot project to get credible numbers Collect the data you need Use ROI to drive change/buy-in, if needed Summary
93. Thank You! Beth’s Blog http://beth.typepad.com http://www.meetyourmission.org/
Editor's Notes
Welcome to the session
A little bit about me – blogger and consultant. I’m the 2009 Visiting scholar for the Packard Foundation. I’m also the author of Chapter 3 on ROI.
The next hour will begin with a primer on ROI. Then we’re going to have some fun inspired by the Cliff Hanger “perils of pauline” where in each installment the central character, Pauline, was left in a situation that would result in near death, only to be rescued at the beginning of the next installment. So, I’m going to give you three different scenarios in the lives of three nonprofit techies – Pauline, Peter, and Paul – and you’re going to get a chance to save them with ROI.
Let’s cover the basics
http://www.flickr.com/photos/seychelles88/351409399/ An analysis that looks at the benefits, costs, and value of a technology intiative over time. Moves us away from thinking of technology as a tool to an investment that is results based. I’m going to explain in more detail about the what and how
A couple of quick points about makes a successful ROI process
Simple and Ecnomical http://www.flickr.com/photos/donsolo/314184695/ Should not take you more time to implement than the technology project itself The data collection should be not be a burden A business case as a memo or simple report may due.
“ Not a method to BS your executive director so you can get a Iphone.” – interviewee from a Legal Services Organization http://www.flickr.com/photos/biz/ Credible The process should include a representative group from the agency/organization – not just the techie Include both internal information gathering and outside research –keeping an eye out for published studies from credible sources that provide acceptable benchmarks about efficiency or effectiveness
Micro versus Macro This is a hard one – because you may be applying an ROI process to a core system that could impact your entire organization – like a client management system or document automation system. The point is that you need to take an in-depth look at how these systems improve efficiency – and that means getting a little granular – tasks analysis, etc. and you’ll be looking at how the technology improves effectiveness in the delivery of your program/service to clients. http://www.flickr.com/photos/thomashawk/340185708/ http://www.flickr.com/photos/67818678@N00/442136095/
http://www.flickr.com/photos/kruggg6/410536827/ Accounts for other factors than direct costs Not all benefits – efficiency and effectiveness –may translate those into dollar amounts …. You ROI process will look at both intangible and tangible benefits
When is ROI used?
http://www.flickr.com/photos/look_ma_im_flying_pictures/2204579311/ Anticipating the future – commonly used prospectively. ROI analysis – use the results used to inform a decision about a IT purchase. Typically cost and performance estimates are based on assumptions about the future that may involve some uncertainty.
http://www.flickr.com/photos/johnlinwood/ Video Conferencing System A statewide legal services organization wanted to invest in a video conference system to facilitate communication, training, and program delivery between regional offices. It was a large upfront investment over $100,000. They analyzed the staff cost of travel time and gas against the cost of the system over 3 year period. They also estimated the income that could be generated if the “saved staff time” were applied to direct program work versus travel. This same organization looked at the outcomes of being able to do more trainings between offices. They also do a lot of staff mentoring. The video conference system allowed them to assign mentors based on expertise versus geographic location. That provided a huge value. They also saw value in improved client service. For example, if a client came into the office with a Medicaid issue, but the person with expertise was in an office an hour way. The client would have had to wait to schedule an appointment or drive over. With the video conference system, they are able to serve the client faster. They also look at the cost of the investment and ask what is the impact if they spent this money on something else? For the conference system, they compared the cost/benefit of investing in more staff.
Ed Marks shared a story about their migrant program where their staff attorneys would travel out the farms and meet with clients. They would discover that they needed a document they didn’t have on them, the client would have a question that needed Internet research, or they’d gather some notes, but needed to go back to the office and stay late to type them up. There were efficiency issues -- they have to drive back to the office, reschedule, and the clients don’t have phones. Or effectiveness issues – the attorneys had to work late, felt stressed because it took multiple steps to complete the work.
Learning from the past – http://www.flickr.com/photos/graciepoo/23544217/ Rather than invest in laptops for everyone in the agency, they did a pilot with a small group of attorneys that work with migrant farm workers. They sent them out with laptops and cell phones and a portable printers. They can now serve clients in one stop.
http://www.flickr.com/photos/drachenspinne/212500261/ The ROI process in the pilot looked at: Analyzed the cost of travel time and mileage from monthly time sheets Analyzed what they could be doing with that time Estimated that took the attorneys 2-3 trips to complete the work with a single client and with the laptops it would only take 1. This information will be used to analyze the ROI for other departments to use laptops. In George, they’re launching a laptop pilot This is the list of questions they answer as part of the pilot: 1. How easy (or not) it is for staff to set up and use the equipment laptop, printer, scanner, cables, etc.? 2. Will staff use the equipment? 3. What will staff use the equipment for? 4. Can we make setup and maintenance simple enough that staff will be able to troubleshoot minor technical problems unassisted? 5. Will advocates consider it helpful to be able to input data as they are interviewing the client? 6. What impact, if any, will the use of the laptop have on good attorney client relationships? 7. Are laptop more useful for doing work
Projects with a long life cycle Project is essential to meeting the organization’s mission and linked to strategic goals Project is very expensive Project is highly visible or controversial Projects that have a large target audience
Your executive director asks you … any of these questions .. “How much does it cost?” “Can we afford it?” “Why do we need it?”
Let’s unpack the four components of an ROI analysis Benefits, Metrics, Values, Costs http://www.flickr.com/photos/toniblay/52445415/
http://www.flickr.com/photos/roadsidepictures/1791904093/ BENEFITS What benefit does the technology offer our staff or clients? Efficiency: · How much time or expenses can we save? · Analysis of work processes, estimated costs _ eliminates errors in figuring and dispensing change, saves time in correction Eliminates undercharging Speed up the check out time Effectiveness: · How will this investment contribute to program goals and results? · How will this investment contribute to effectiveness of staff to serve clients? Customer service improves Return visits Might even purchase more .. Efficiency and effectiveness go hand and hand – hard to separate ..
These are some ways to think about efficiency .. Let’s play some match the cheese .. Let’s take a few of these and match them with a technology investment .. Improves the ability to maintain a system Eliminates duplicate systems: Accommodates increases in workload or demand without additional costs: Reduces manual operations: Improves efficiency Reduces Travel Cost/Time:
Effectiveness: Better Use of Time To Serve Clients or Improve Quality of Work Improves ability to deliver Improves access to services Improves access to information: Improves accuracy: Improves effectiveness of information delivered:
http://www.flickr.com/photos/blah_oh_well/766801412/ Tangible: Benefits that are required for organization operation and are readily visible, rigorously quantified, and are presented as a line item on a budget.
Objective Common measures Easy to quantify Easy to assign money values And … http://www.flickr.com/photos/missioncontrol/1435691544/ http://www.flickr.com/photos/zombie/161977835/
Most Credible with executive directors
http://www.flickr.com/photos/kanaka/199702352/ Intangible: These are key to organizational effectiveness, but may be difficult to quantify and are not tracked through budgeting or accounting procedures.
http://www.flickr.com/photos/ratterrell/104724821/ Subjective Difficult to measure and quantify Difficult to assign monetary value Less credible
Less Credible
Usually behaviorally oriented
Some Typical Intangible Measures These measures are often presented as intangibles. For each measure, there may be exceptions where the organization can convert the data using metrics and possibly calculate a monetary value. Job Satisfaction Productivity Job Commitment Work Environment Morale Employee Retention Innovation Competencies Leadership Team work Cooperation Decision-making Communication Client satisfaction Client complaints Client response time
O
1. Does an acceptable, standard metric exist for the measure? 2. Can the conversion be accomplished with minimum resources? 3. Can the conversion process be described to an executive director and secure their buy-in in five minutes
"Anything can be measured in a way that is superior to not measuring it at all" - Gilbs Law Conversion from Intangible to Tangible In Glib's Scales of Measures : How To Quantify, he suggests: You should learn the art of developing your own tailored scales of measure for the ROI metrics and values,, which are important to your organization or system. You cannot rely on being ‘given the answer’ about how to quantify. You would soon lose control over your current vital concerns if you waited for that!. His point that some intangible may be quantified, but it takes some discussion around the metrics. That the discussion itself is valuable to your thought process.
Hard and soft data – surveys, budget analysis, task analysis as well as interviews, observations, secondary data http://www.flickr.com/photos/ian-s/2152798588/
And you don’t have to be an Einstein for these calculations … Compare the total costs of different solutions Compare the total costs with the benefits Compare the costs of not doing the project Look at doing a small proof of concept pilot with a smaller investment with the goal of identify cost/benefit Look at the cost of not doing the investment. If possible see if there is a “proof of concept” pilot you can do with a smaller investment before rolling out a larger project.
Spreadsheet aerobics There’s no perfect spreadsheet. You may be doing a “benefits analysis” This worksheet from a spreadsheet for YMCA to analyze the revenue increases, cost savings, and productivity from switching from a manual system to an automated CRM. To get to this spreadsheet, they need to analyze work flow tasks, amount of time, translate into lost revenue where there process wasn’t efficient, etc.
http://www.flickr.com/photos/rberteig/254726379/ Centralized In-Take System The legal services organization invested in a centralized in-take system several years ago that ultimately decreased costs and improved services to clients. The service was so successful that they saw a major increase in calls. So much so that clients were being put on hold for too long a time. Complaints soared.
http://www.flickr.com/photos/silvers/34941260/ They further analyzed their complaint log to determine the nature of the complaints. It was the fact that clients had to wait on the phone on hold for over 20-30 minutes until they got served – they were pretty angry – had to hold a baby
Researched software solutions that allowed the client to push a button and hold their place in the line and the system would call them back. That way they didn’t have to sit there with the phone plastered to their ear. Researched how many additional screeners required to reduce waiting time to an acceptable level.
You want a committee – a mix of technology and program/management people Important for buy-in – particularly for a mission critical system like client management system You may have a smaller or more informal group for smaller investments To help you create a business case …… a written report of your ROI analysis – that is more than just a budget.
A well prepared business case incorporates both the numbers and non-financial factors into a concise and informative presentation The purpose is to sell the value of the proposed project. The business case presents the key issues and facts while spotlighting the investment’s contribution the organization’s mission. Good business cases are calculated based on numbers, but they are approved based on stories. ROI numbers, no matter how compelling, won’t speak for themselves. It’s up to you to explain what they mean to enterprise visions and goals. Clever storytelling is one of the quickest and most effective ways to gain executive director understanding, buy-in and funding. It also helps attract support and cooperation from reluctant users during project implementation and operation. Creating effective stories is not hard, provided you clearly understand who these decision influencers are, know well what they most care about and understand where to place stories for the most impact.
http://www.flickr.com/photos/ucumari/389105418/ Don’t Show it all, but have it read
http://www.flickr.com/photos/53537358@N00/296260969/ What happens if ROI is negative? What if the upfront cost is huge and the benefits are all intangible and can’t be quantified? This strikes terror in the hearts of not only technology staff, but also program staff and executive directors. It may be wasted resources that could be used in some other area with better results, like hiring more staff. If you following have followed all the steps, your analysis provides an opportunity for learning and change. Some tips: Use your research to identify a “proof of concept” project that is less of risk and use that to get a better handle on the costs/benefits and to convert, if possible, intangibles. Manage expectations through discussion. Use your information to drive change. Focus on the cost of not implementing to your client service or staff.
http://www.flickr.com/photos/marcelgermain/2078076913/ This phrase was coin by Jeremiah Owyang, a web strategy analyst from Forester to refer to situations where we get seduced by the technology tools, but don’t think about how might get used. Fondling the hammer We all have our term for that – and I know I’m guilty of it … but sometimes it isn’t only the techies in an organization who do it .. http://www.web-strategist.com/blog/2008/01/17/stop-focusing-on-the-hammer-and-think-about-the-house/
Meet Charlie, he is the IT staff person at a legal services organization. http://www.flickr.com/photos/kt/274574974/
That’s Walter, his boss, who is on his way to a business lunch with board members, all attorneys in program practice.
While waiting for the check http://www.flickr.com/photos/85182154@N00/96504518/ Generation X-Ray
One http://www.flickr.com/photos/hirefrank/1988305062/
By one http://www.flickr.com/photos/voorn/468634955/
pull out their mobile devices ... http://www.flickr.com/photos/thomashawk/384835614/
Walter heads back to the office and heads straight to Charlie’s office and says … http://www.flickr.com/photos/deadair/206948514/
Paul is thinking http://www.flickr.com/photos/kt/274574974/
http://www.youtube.com/watch?v=qg1ckCkm8YI About that video on YouTube where the guy throws a Iphone into a blender …
The technology staff person does some research. Asks the executive director and attorneys what the value that a mobile device offers? Would it make them more efficient or effective? He does a work flow analysis of how much work is done remotely over the Internet versus in the office. Analyzes the total cost of the devices, including hardware purchase, subscription fee, extended warranties, etc. He discovers the monthly service fees to be as high as $80 per month. He makes a few calls to colleagues who work for other legal services organizations to find out if and how they use mobile devices in their program delivery. He learns that programs where staff are working remotely in the field are using the devices and mobile laptops and wireless Internet access. That’s the biggest benefit. Cost of the devices, plus the $80 per month service fee for all 10 attorneys exceeded the benefit of mobility because 8 out of the 10 do their most of their work in the office. The other two attorneys worked in a migrant program so the mobile devices might be a good investment. He interviews the attorneys and learns: He performs a cost/benefit ROI calculation using the following: Analyzed the cost of travel time and mileage from monthly time sheets Analyzed what they could be doing with that time Estimated that took the attorneys 2-3 trips to complete the work with a single client and with the laptops it would only take 1. He shares the information in a memo with his Executive Director. http://www.flickr.com/photos/kt/274574974/
http://www.flickr.com/photos/netdance/48462878/ Lost in Translation – its when we the geeks describe technology investments in terms of the features of the tools and the price – only – and don’t articulate or explain the benefits – particularly efficiency and effectiveness – and attempt to translate those into some value ..
Meet Jim ..who works for a legal services organizations in IT .. With more than 40 staff members http://www.flickr.com/photos/johnjoh/376275352/
spends a lot of time in front of his 12 inch computer. http://www.flickr.com/photos/joshrholloway/335349250/
It works just fine but http://www.flickr.com/photos/vanzandt/33083439/
He often needs to have many program application windows at once
Works on these huge spreadsheets http://www.flickr.com/photos/poluz/1412304766/in/pool-387261@N22
Jim knows he would be more productive with a larger monitor http://www.flickr.com/photos/cafeholly/429622457/
He’s not the only person in the office who would be more productive with a larger monitor
He does some research … spends hours evaluating large screen monitors in search of the best on. He sends a memo to his executive director detailing the monitors’ features like refresh rate, screen real estate, and price. It goes on for pages and pages and pages talking about large monitors and their technical specifications. He includes an estimate for what it would cost to get large monitors for all 50 people on staff … along with quantity discounts
And what’s his ed’s response?
Meet Jim ..who works for a legal services organizations in IT .. With more than 40 staff members http://www.flickr.com/photos/johnjoh/376275352/
http://www.flickr.com/photos/stitch/408731177/ http://www.flickr.com/photos/stewart/106963896/ This situation happens when the technology project proposal is designed and researched by one person, usually the techie, without a committee or getting feedback from those impacted by the technology. Also, the proposal isn’t much more than budget and description of the technology used. Sometimes this happens large systems projects ..
http://www.flickr.com/photos/gregpc/1043493772/ She works at a legal services office that is still using paper-based systems for procedures, scheduling, job reviews, etc. She’s noticed that moral is so good lately …
Attorneys, paralegals, and administrative assistants in her office are fighting over the copy machine.
She also over hears some attorneys complaining about how the new attorneys on staff aren’t following the case handling procedures
… .because the case handling procedures in the three-ring binders on the clogged bookshelf are hopelessly out of date.
HR staff person, is really annoyed because he just spent the afternoon copying the updated vacation policy and putting them in 3-ring binders
he’s a green activist http://www.flickr.com/photos/zachstern/62320099/
It gets worse. This results in lots of lost time and frustration ..
Everyone is frustrated and not productive …
Pauline has an idea! She works on it all afternoon, puts together a budget and description of the Intranet technical specs, hardware, software, etc. and hands it to the executive director Guess what the executive director’s reaction is?
He looks it over and asks …
There is no single "right" way to conduct a return on investment analysis. Nor are there standardized ROI analysis processes or benchmark calculations for nonprofit technology. The best advice is to focus on the analysis to the organization's goals and business processes. This focus will help guide decisions about the resources and methods to use to conduct a sound and valuable ROI analysis.
