Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2016 metrics-as-culture

1,995 views

Published on

We don't always think of it this way, but your metrics *are* your culture... Your metrics shape behavior and incentives, which really is the heart of culture.

Published in: Business

2016 metrics-as-culture

  1. 1. @nicolefv HOW METRICS SHAPE YOUR CULTURE NICOLE FORSGREN, PHD @NICOLEFV DIRECTOR, ORGANIZATIONAL PERFORMANCE AND ANALYTICS, CHEF CO-FOUNDER, DEVOPS RESEARCH AND ASSESSMENT
  2. 2. @nicolefv WE DON’T ALWAYS THINK OF IT THIS WAY, BUT… measurement is culture
  3. 3. @nicolefv
  4. 4. @nicolefv "As good as Amazon was, it did not yet have a culture that fully embraced measurement and debate. I think building this culture is the key to innovation. Creativity must flow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate. Position, obedience, and tradition should hold no power. For innovation to flourish, measurement must rule.” - Greg Linden http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html
  5. 5. @nicolefv YOU CAN’T IMPROVE WHAT YOU DON’T MEASURE @nicolefv
  6. 6. @nicolefv Evaluating well-designed and executed experiments that were designed to improve a key metric, only about 1/3 were successful at improving the key metric! Online Experimentation at Microsoft, Kohavi et al. http://stanford.io/130uW6X
  7. 7. @nicolefv ALWAYS MEASURE THINGS THAT MATTER
  8. 8. @nicolefv@nicolefv THAT WHICH IS MEASURED GETS MANAGED In business, you are what you measure
  9. 9. @nicolefv IF YOU CAPTURE ONLY ONE METRIC, YOU KNOW WHAT WILL BE GAMED @nicolefv
  10. 10. @nicolefv@nicolefv Sometimes, we just measure ALL THE THINGS. But STOP and ask yourself: • What does this tell me? • How can I use this to make a decision? • How does this relate to an outcome? • Does it still make sense to measure and track this metric? • What are the unintended consequences of implementing this? NOT EVERYTHING THAT CAN BE COUNTED… COUNTS
  11. 11. @nicolefv METRICS INFORM OUR INCENTIVES… AND SHAPE BEHAVIOR
  12. 12. @nicolefv METRICS INFORM OUR INCENTIVES… AND SHAPE BEHAVIOR Metric = deliver code on schedule Behavior = sprint to deliver code at 5pm on Friday Unintended consequence = ops team spends the weekend putting out fires
  13. 13. @nicolefv
  14. 14. @nicolefv JUST BECAUSE SOMETHING IS HARD TO MEASURE DOESN’T MEAN IT ISN’T WORTH MEASURING
  15. 15. @nicolefv WESTRUM ORGANIZATIONAL CULTURE PATHOLOGICAL Power-oriented BUREACRATIC Rule-oriented GENERATIVE Performance-oriented Low cooperation Modest cooperation High cooperation Messengers shot Messengers neglected Messengers trained Responsibilities shirked Narrow responsibilities Risks are shared Bridging discouraged Bridging tolerated Bridging encouraged Failure leads to scapegoating Failure leads to justice Failure leads to inquiry Novelty crushed Novelty leads to problems Novelty implemented Westrum, R. (2004). A typology of organisational cultures. Quality and safety in health care, 13(suppl 2), ii22-ii27.
  16. 16. @nicolefv HOW TO MEASURE WESTRUM ORGANIZATIONAL CULTURE IN A STATISTICALLY VALID & RELIABLE WAY Rate how strongly you agree (7) or disagree (1) to the following statements: • On my team, information is actively sought. • On my team, failures are learning opportunities, and messengers of them are not punished. • On my team, responsibilities are shared. • On my team, cross-functional collaboration is encouraged and rewarded. • On my team, failure causes enquiry. • On my team, new ideas are welcomed. Forsgren, N., J. Humble (2016). “The Role of Continuous Delivery in IT and Organizational Performance.” In the Proceedings of the Western Decision Sciences Institute (WDSI) 2016, Las Vegas, NV.
  17. 17. @nicolefv WHAT CAN I DO? • Read and share Data Driven, by @dpatil and @hmason • Ask others about metrics: • Is it happening? Where? • Is it important? • Think about your own metrics: • Start with outcomes: what’s important? • Then think about what influences your outcomes • Define and communicate
  18. 18. @nicolefv METRICS HELP US COMMUNICATE KEY: • Give them meaningful names • Define them well • Communicate them often and across boundaries
  19. 19. @nicolefv WE’VE TALKED ABOUT: MEASUREMENT IS CULTURE • You can’t improve what you don’t measure • Always measure things that matter • That which is measured gets managed • If you capture only one metric, you know exactly what will be gamed • Metrics inform our incentives… and shape behavior • Not everything that can be counted... Counts • Just because something is hard to measure doesn’t mean it isn’t worth measuring
  20. 20. @nicolefv IF MEASUREMENT IS CULTURE & YOU AREN’T MEASURING YOUR WORK... how is it affecting your culture?
  21. 21. @nicolefv THANK YOU NICOLEFV.COM

×