In business it is all about measurement. If something can’t be measured it is deemed unimportant or relegated to “cost” status. Anything put in the “cost” bucket has no chance to grow or prosper, especially in the current economic climate. For the Business Analyst position to grow in respect and requirement within a business, measurement tools must be established. Currently, there are no generally accepted standards for measuring the success for failure of a BA. In this web seminar we will discuss what measurement tools could be established and what tools organizations are currently using “out in the wild” to measure their BA performance. The goal is to give you a concert idea on how to start developing a BA measurement solution to validate you or your team’s success and worth. The ultimate goal is to push the IIBA to drive the development of generally accepted measuring tools for the BA position. You will learn: ·Reasons why measurement in business is important ·Why it is critical the BA role doesn’t get lumped into the “cost” bucket ·Potential tools for measuring BA performance ·Options that other organizations are currently using in the wild ·Ideas on how to start implementing a measurement solution ·Importance of IIBA support in driving measurement standards

1. “Measuring BA Impact”
Web Seminar
Thursday, February 19, 2009
David Mantica

2. Presentation Agenda
• Why measurement is important
• The Reality
• Survey Results
• Measurement / Metric theory
• What to measure
• It all starts with customer evaluation
• Metric ideas / possible options
• Final thoughts

3. Why is Measurement Important?
– If it can be measured, it can be monitored
– If it can be monitored, it becomes important
• Is the BA position a cost or does the BA position provide
value and ROI
• To survive and prosper, the BA position must show value
and return
Standard measurements will provide the tool to show value

4. The Reality
• There are no standard measurements available
• From survey conducted there is no consistency in how BA
performance is measured and few if any metrics
• Management doesn’t know where to start
• Need to understand
– What you should measure
– When you should measure
– How you should report
Going back to the last slide
Need to measure to show importance. You can measure just about
anything. The goal is to figure out what the right things are to measure
and ultimately what metrics to set.
We have a blank sheet of paper to start with

5. Survey Results
Demographics of responders
-All Business Analysts
• 23.5% from companies of 5000+ people
• 32.4% from companies of 1000-4999 people
• 17.6% from companies of 500 to 999 people
• 20.6% from companies of 100 to 499 people
• 5.9% from companies of 1 to 99 people
BA position tends to be a “large company” position

6. Survey Results
Using or Planning to Use Metrics
– Currently using Metrics
• 71.9% are not and 28.1% are
– Planning on using Metrics in the future
• 65.2% are not and 34.8% are
Types of metrics currently deployed
– Defect tracking (number one choice)
– Performance against set objectives
– Hours worked versus estimated hours
– Supervisor and business sponsor evaluation

7. Survey Results
For those who do not use set measurement or metrics, these
performance measures are used:
– Annual Review (most heavily used)
– Project team feedback
– MBOs
– Number of system bugs (can be turned to a measurement/metric)
– Peer review
– Business feedback

8. Survey Results
General data points
– Gather User feedback
• 45.5% No
• 54.4% yes
– Requirements process as a separate cost
• 68.8% No
• 31.3% Yes
– How are Business Analysts seen in an organization
• Business alignment tool
• Project quality
• Project cost
• ROI position

9. Measurement and Metric Theory
• Balance Score card approach
– Pros and cons
– Still need specific data points
• Initial Key “need to measure what is important”
• Need to build metrics off of data and be flexible
– What are the primary goals of your Business Analysts?
• Tie measurement to primary goals
– Set initial metrics up based on goals/plans and adjust accordingly
• Remember to think clearly about what can be measured, how you
would measure, and at what effort
• Wealth of information from Call Center Industry
– Check out the article Improving Performance Through Customer Service
Metrics, www.thinkhdi.com Jan/Feb 2009 issue

10. Measurement and Metric Theory
Two items become important
– Evaluation data, getting feedback from the system users
– Cost data, getting the requirements process budgeted and
reported as a separate cost of the project
It is so much better to be measured as a cost than to not be
measured at all. If the BA cost is rolled into some other
aspect of the project, it might be hidden in the short term
but long term it can have serious consequences.

11. What to Measure
What functions of a BA are important/critical?
– “Nice to do” versus “need to do”
– All organizations are going to be different
• Some industry recognized direct functions
– Eliciting and defining need
– Modeling process (current and future)
– Analyzing whether what is thought to be needed is really needed
• Some indirect impact of BA work
– Project cost changes
– Productivity changes
– Competitive advantage changes

12. Survey Tool Options
To get user data you need a survey tool option
– A lot of options are available
• http://stellersurvey.com
• http://questionpro.com
• http://www.surveygizmo.com
• http://www.constantcontact/index.jsp
• http://zoomerang.com
• http://www.esurveypro.com
• http://www.surveymonkey.com
• http://polldaddy.com/features
See other document to view the pros and cons of
each tool.

13. Measuring Use/Productivity
BA role has the ultimate goal of process and system
improvement
– Increase process of system usage
– Increase process of system speed
– Increase overall worker productivity
End user evaluation data can provide a wealth of information
– During requirements phase set a metric (goal)
• Examples
– Automation will speed up processes by x%
– Upgrade of system will speed up y process by x%
– Build survey that can be used pre and post
– Conduct survey on same people, match results to metric
• Example: ASPE instructor evaluations

14. Measuring Need Fulfillment
BA top priority from survey results is to align business with
IT --- So the BA needs to ensure IT solutions are meeting
the needs of the business
Easiest way to do this is measure customer satisfaction with solution
deployment
– Post deployment questions only, no need for pre survey, should be a
month or two after deployment
• Simple questions
– Do you use the new system enhancements?
– Are the enhancements what you needed?
– Are you more productive because of the new system enhancements?
– How would you grade the overall functionality of the system based on your needs?
– Should have a baseline scoring system
• poor to excellent
• 1-4
• an average acceptable grade

15. Measuring Cost: Estimate versus Actual Hours
At minimum, a BA organization needs to be able to show their estimated
amount of hours for a project and tie it back to actual.
Break down by
– Business case development
– Process analysis
– Requirements elicitation and development
– Or just roll it all up
Why? This is part of a balanced scorecard perspective
– Even if extra time was needed over estimates, you can show how extra
time correlates to much higher use and value scores
– Even if you beat estimates you can show how less time correlates to lower
usage and less system acceptance scores
As you know, it is not always about making your cost budget

16. Measuring Cost: Estimated vs. Actual Hours
correlated to project time, change orders, and defects
• Project time is measured
• Change orders are measured
• Defects are measured
These are all items that the BA role has impact on
• Analyst consistently say more time spent on requirements significantly
reduces overall project costs
– By investing only 10% more effort in requirements before freezing
them, large, complex projects experienced significantly lower cost
overruns — 30% to 130% overruns fell to 10% to 20% overruns
(NASA Comptroller Office, reported in Hooks and Farry, 2001).
By using your hour measurements you can compare
requirement hours to overall project time, number, extent
of change orders, and even to system rework

17. Possible Measurements Tied to Stakeholder
Users Daily
Activity
Project Leaders
Cost
Business Return
Need
“Did we get what we
wanted”
Estimates
“Were the estimates
within reason”
Competitive Advantage
“Do IT systems make us
more competitive”
Use
“Can I work faster and
better now”
Work Quality
“did the work match
expectations & how did
the requirements test”
Return on Investment
“Does the cost of the BA
team tie back to a return
multiplier ”
Speed
“Did we get what we
needed before irrelevant”
Delivery time
“did the project deliver
on time”
Productivity
“Is our work more
productive because of our
IT systems”

18. Final Thoughts
• Measurement is critical for continuation and growth
• BA role’s growth is tied to keeping out of the “cost” bucket
• There are currently NO measurement standards available
• Need to get creative and look at how other industries and
positions measure themselves
• It all starts with figuring out what to measure
• Survey tools can help
• There are options, don’t have to create all measurements
from scratch
• Evaluating needs met and use can be powerful tools