Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Measuring Business Analyst Impact


Published on

In business it is all about measurement. If something can’t be measured it is deemed unimportant or relegated to “cost” status. Anything put in the “cost” bucket has no chance to grow or prosper, especially in the current economic climate. For the Business Analyst position to grow in respect and requirement within a business, measurement tools must be established.

Currently, there are no generally accepted standards for measuring the success for failure of a BA. In this web seminar we will discuss what measurement tools could be established and what tools organizations are currently using “out in the wild” to measure their BA performance. The goal is to give you a concert idea on how to start developing a BA measurement solution to validate you or your team’s success and worth. The ultimate goal is to push the IIBA to drive the development of generally accepted measuring tools for the BA position.

You will learn:
·Reasons why measurement in business is important
·Why it is critical the BA role doesn’t get lumped into the “cost” bucket
·Potential tools for measuring BA performance
·Options that other organizations are currently using in the wild
·Ideas on how to start implementing a measurement solution
·Importance of IIBA support in driving measurement standards

Published in: Business, Technology

Measuring Business Analyst Impact

  1. 1. “ Measuring BA Impact” Web Seminar Thursday, February 19, 2009 David Mantica
  2. 2. Presentation Agenda <ul><li>Why measurement is important </li></ul><ul><li>The Reality </li></ul><ul><li>Survey Results </li></ul><ul><li>Measurement / Metric theory </li></ul><ul><li>What to measure </li></ul><ul><li>It all starts with customer evaluation </li></ul><ul><li>Metric ideas / possible options </li></ul><ul><li>Final thoughts </li></ul>
  3. 3. Why is Measurement Important? <ul><ul><li>If it can be measured, it can be monitored </li></ul></ul><ul><ul><li>If it can be monitored, it becomes important </li></ul></ul><ul><li>Is the BA position a cost or does the BA position provide value and ROI </li></ul><ul><li>To survive and prosper, the BA position must show value and return </li></ul><ul><li>Standard measurements will provide the tool to show value </li></ul>
  4. 4. The Reality <ul><li>There are no standard measurements available </li></ul><ul><li>From survey conducted there is no consistency in how BA performance is measured and few if any metrics </li></ul><ul><li>Management doesn’t know where to start </li></ul><ul><li>Need to understand </li></ul><ul><ul><li>What you should measure </li></ul></ul><ul><ul><li>When you should measure </li></ul></ul><ul><ul><li>How you should report </li></ul></ul><ul><li>Going back to the last slide </li></ul><ul><li>Need to measure to show importance. You can measure just about anything. The goal is to figure out what the right things are to measure and ultimately what metrics to set. </li></ul><ul><li>We have a blank sheet of paper to start with </li></ul>
  5. 5. Survey Results <ul><li>Demographics of responders </li></ul><ul><li>-All Business Analysts </li></ul><ul><ul><ul><li>23.5% from companies of 5000+ people </li></ul></ul></ul><ul><ul><ul><li>32.4% from companies of 1000-4999 people </li></ul></ul></ul><ul><ul><ul><li>17.6% from companies of 500 to 999 people </li></ul></ul></ul><ul><ul><ul><li>20.6% from companies of 100 to 499 people </li></ul></ul></ul><ul><ul><ul><li>5.9% from companies of 1 to 99 people </li></ul></ul></ul><ul><li>BA position tends to be a “large company” position </li></ul>
  6. 6. Survey Results <ul><li>Using or Planning to Use Metrics </li></ul><ul><ul><li>Currently using Metrics </li></ul></ul><ul><ul><ul><li>71.9% are not and 28.1% are </li></ul></ul></ul><ul><ul><li>Planning on using Metrics in the future </li></ul></ul><ul><ul><ul><li>65.2% are not and 34.8% are </li></ul></ul></ul><ul><li>Types of metrics currently deployed </li></ul><ul><ul><li>Defect tracking (number one choice) </li></ul></ul><ul><ul><li>Performance against set objectives </li></ul></ul><ul><ul><li>Hours worked versus estimated hours </li></ul></ul><ul><ul><li>Supervisor and business sponsor evaluation </li></ul></ul>
  7. 7. Survey Results <ul><li>For those who do not use set measurement or metrics, these performance measures are used: </li></ul><ul><ul><li>Annual Review (most heavily used) </li></ul></ul><ul><ul><li>Project team feedback </li></ul></ul><ul><ul><li>MBOs </li></ul></ul><ul><ul><li>Number of system bugs (can be turned to a measurement/metric) </li></ul></ul><ul><ul><li>Peer review </li></ul></ul><ul><ul><li>Business feedback </li></ul></ul>
  8. 8. Survey Results <ul><li>General data points </li></ul><ul><ul><li>Gather User feedback </li></ul></ul><ul><ul><ul><li>45.5% No </li></ul></ul></ul><ul><ul><ul><li>54.4% yes </li></ul></ul></ul><ul><ul><li>Requirements process as a separate cost </li></ul></ul><ul><ul><ul><li>68.8% No </li></ul></ul></ul><ul><ul><ul><li>31.3% Yes </li></ul></ul></ul><ul><ul><li>How are Business Analysts seen in an organization </li></ul></ul><ul><ul><ul><li>Business alignment tool </li></ul></ul></ul><ul><ul><ul><li>Project quality </li></ul></ul></ul><ul><ul><ul><li>Project cost </li></ul></ul></ul><ul><ul><ul><li>ROI position </li></ul></ul></ul>
  9. 9. Measurement and Metric Theory <ul><li>Balance Score card approach </li></ul><ul><ul><li>Pros and cons </li></ul></ul><ul><ul><li>Still need specific data points </li></ul></ul><ul><li>Initial Key “need to measure what is important” </li></ul><ul><li>Need to build metrics off of data and be flexible </li></ul><ul><ul><li>What are the primary goals of your Business Analysts? </li></ul></ul><ul><ul><ul><li>Tie measurement to primary goals </li></ul></ul></ul><ul><ul><ul><ul><li>Set initial metrics up based on goals/plans and adjust accordingly </li></ul></ul></ul></ul><ul><ul><ul><li>Remember to think clearly about what can be measured, how you would measure, and at what effort </li></ul></ul></ul><ul><li>Wealth of information from Call Center Industry </li></ul><ul><ul><li>Check out the article Improving Performance Through Customer Service Metrics, www.thinkhdi.com Jan/Feb 2009 issue </li></ul></ul>
  10. 10. Measurement and Metric Theory <ul><li>Two items become important </li></ul><ul><ul><li>Evaluation data, getting feedback from the system users </li></ul></ul><ul><ul><li>Cost data, getting the requirements process budgeted and reported as a separate cost of the project </li></ul></ul><ul><li>It is so much better to be measured as a cost than to not be measured at all. If the BA cost is rolled into some other aspect of the project, it might be hidden in the short term but long term it can have serious consequences. </li></ul>
  11. 11. What to Measure <ul><li>What functions of a BA are important/critical? </li></ul><ul><ul><li>“ Nice to do” versus “need to do” </li></ul></ul><ul><ul><li>All organizations are going to be different </li></ul></ul><ul><li>Some industry recognized direct functions </li></ul><ul><ul><li>Eliciting and defining need </li></ul></ul><ul><ul><li>Modeling process (current and future) </li></ul></ul><ul><ul><li>Analyzing whether what is thought to be needed is really needed </li></ul></ul><ul><li>Some indirect impact of BA work </li></ul><ul><ul><li>Project cost changes </li></ul></ul><ul><ul><li>Productivity changes </li></ul></ul><ul><ul><li>Competitive advantage changes </li></ul></ul>
  12. 12. Survey Tool Options <ul><li>To get user data you need a survey tool option </li></ul><ul><ul><li>A lot of options are available </li></ul></ul><ul><ul><ul><li>http://stellersurvey.com </li></ul></ul></ul><ul><ul><ul><li>http://questionpro.com </li></ul></ul></ul><ul><ul><ul><li>http://www.surveygizmo.com </li></ul></ul></ul><ul><ul><ul><li>http://www.constantcontact/index.jsp </li></ul></ul></ul><ul><ul><ul><li>http://zoomerang.com </li></ul></ul></ul><ul><ul><ul><li>http://www.esurveypro.com </li></ul></ul></ul><ul><ul><ul><li>http://www.surveymonkey.com </li></ul></ul></ul><ul><ul><ul><li>http://polldaddy.com/features </li></ul></ul></ul><ul><li>See other document to view the pros and cons of each tool. </li></ul>
  13. 13. Measuring Use/Productivity <ul><li>BA role has the ultimate goal of process and system improvement </li></ul><ul><ul><li>Increase process of system usage </li></ul></ul><ul><ul><li>Increase process of system speed </li></ul></ul><ul><ul><li>Increase overall worker productivity </li></ul></ul><ul><li>End user evaluation data can provide a wealth of information </li></ul><ul><ul><li>During requirements phase set a metric (goal) </li></ul></ul><ul><ul><ul><li>Examples </li></ul></ul></ul><ul><ul><ul><ul><li>Automation will speed up processes by x% </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Upgrade of system will speed up y process by x% </li></ul></ul></ul></ul><ul><ul><li>Build survey that can be used pre and post </li></ul></ul><ul><ul><li>Conduct survey on same people, match results to metric </li></ul></ul><ul><ul><ul><li>Example: ASPE instructor evaluations </li></ul></ul></ul>
  14. 14. Measuring Need Fulfillment <ul><li>BA top priority from survey results is to align business with IT --- So the BA needs to ensure IT solutions are meeting the needs of the business </li></ul><ul><li>Easiest way to do this is measure customer satisfaction with solution deployment </li></ul><ul><ul><li>Post deployment questions only, no need for pre survey, should be a month or two after deployment </li></ul></ul><ul><ul><ul><li>Simple questions </li></ul></ul></ul><ul><ul><ul><ul><li>Do you use the new system enhancements? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Are the enhancements what you needed? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Are you more productive because of the new system enhancements? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>How would you grade the overall functionality of the system based on your needs? </li></ul></ul></ul></ul><ul><ul><li>Should have a baseline scoring system </li></ul></ul><ul><ul><ul><li>poor to excellent </li></ul></ul></ul><ul><ul><ul><li>1-4 </li></ul></ul></ul><ul><ul><ul><li>an average acceptable grade </li></ul></ul></ul>
  15. 15. Measuring Cost: Estimate versus Actual Hours <ul><li>At minimum, a BA organization needs to be able to show their estimated amount of hours for a project and tie it back to actual. </li></ul><ul><li>Break down by </li></ul><ul><ul><li>Business case development </li></ul></ul><ul><ul><li>Process analysis </li></ul></ul><ul><ul><li>Requirements elicitation and development </li></ul></ul><ul><ul><li>Or just roll it all up </li></ul></ul><ul><li>Why? This is part of a balanced scorecard perspective </li></ul><ul><ul><li>Even if extra time was needed over estimates, you can show how extra time correlates to much higher use and value scores </li></ul></ul><ul><ul><li>Even if you beat estimates you can show how less time correlates to lower usage and less system acceptance scores </li></ul></ul><ul><li>As you know, it is not always about making your cost budget </li></ul>
  16. 16. Measuring Cost: Estimated vs. Actual Hours correlated to project time, change orders, and defects <ul><li>Project time is measured </li></ul><ul><li>Change orders are measured </li></ul><ul><li>Defects are measured </li></ul><ul><li>These are all items that the BA role has impact on </li></ul><ul><li>Analyst consistently say more time spent on requirements significantly reduces overall project costs </li></ul><ul><ul><li>By investing only 10% more effort in requirements before freezing them, large, complex projects experienced significantly lower cost overruns — 30% to 130% overruns fell to 10% to 20% overruns (NASA Comptroller Office, reported in Hooks and Farry, 2001). </li></ul></ul><ul><li>By using your hour measurements you can compare requirement hours to overall project time, number, extent of change orders, and even to system rework </li></ul>
  17. 17. Possible Measurements Tied to Stakeholder Productivity “ Is our work more productive because of our IT systems” Delivery time “ did the project deliver on time” Speed “ Did we get what we needed before irrelevant” Return on Investment “ Does the cost of the BA team tie back to a return multiplier ” Work Quality “ did the work match expectations & how did the requirements test ” Use “ Can I work faster and better now” Competitive Advantage “ Do IT systems make us more competitive” Estimates “ Were the estimates within reason” Need “ Did we get what we wanted” Business Return Project Leaders Cost Users Daily Activity
  18. 18. Final Thoughts <ul><li>Measurement is critical for continuation and growth </li></ul><ul><li>BA role’s growth is tied to keeping out of the “cost” bucket </li></ul><ul><li>There are currently NO measurement standards available </li></ul><ul><li>Need to get creative and look at how other industries and positions measure themselves </li></ul><ul><li>It all starts with figuring out what to measure </li></ul><ul><li>Survey tools can help </li></ul><ul><li>There are options, don’t have to create all measurements from scratch </li></ul><ul><li>Evaluating needs met and use can be powerful tools </li></ul>