2019 AFITC Fantastic outcomes and how to measure them by Cheryl Hammond

•

0 likes•100 views

Do your metrics track what matters most to your organization, or do they merely quantify your adherence to a process? Is that process a good proxy for real results? In your environment? How do you know? Discover where to look for elusive, real outcomes. Join Cheryl Hammond to learn how to study indicators for your important metrics so you can recognize them when you encounter them in the wild. Understand how to monitor the health and relevance of your outcomes, and commit to the constant care needed to keep them vibrant. You'll take back a renewed appreciation for the beauty of a wild, untamed outcome and how it can motivate and focus your teams.

Technology

@bsktcase #AFITC #fantasticoutcomes
fantastic outcomes
@bsktcase #AFITC #fantasticoutcomes

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
fantastic outcomes and
how to measure them
Cheryl Hammond
Agile Practice Leadership Enablement
she/her
chammond@pivotal.io
@bsktcase

@bsktcase #AFITC #fantasticoutcomes
Cheryl Hammond
she/her
20+ years in software development, executive
leadership, and organizational transformation
Passionate about agile, lean, and making
software delivery humane; not afraid to look
silly doing it
Fourth-generation Seattleite
College recruiter & mentor; traveler; bolonka
lover; genealogist; feminist

@bsktcase #AFITC #fantasticoutcomes
Invitations and asks
slideshare.net/bsktcase
Today’s slides and more
@bsktcase #AFITC #fantasticoutcomes
Tweet your feedback and questions during and after and whenever

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
why measure?

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
To recognize eﬀectiveness

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
To focus attention

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
To inﬂuence very large, slow-moving, opinionated entities

@bsktcase #AFITC #fantasticoutcomes
is there a problem?

@bsktcase #AFITC #fantasticoutcomes
An evergreen problem

@bsktcase #AFITC #fantasticoutcomes
We need good proxies… and to recognize them as proxies

@bsktcase #AFITC #fantasticoutcomes
One metric can help us, but it isn’t enough

@bsktcase #AFITC #fantasticoutcomes
Time to feedback is still just a proxy for real outcomes!

@bsktcase #AFITC #fantasticoutcomes
What even is an “outcome”?
What we really want to know
Did we build the right thing?
Was the value worth the cost?
Will this gamble pay oﬀ?
Where should we invest?
And yet, we’re measuring
“Productivity”
“Predictability/reliability”

@bsktcase #AFITC #fantasticoutcomes
outcomes are elusive
@bsktcase #AFITC #fantasticoutcomes

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
Welcome changing requirements,
even late in development.
Agile processes harness change for the
customer’s competitive advantage.

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
Outcomes that deliver value

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
Outcomes that support people

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
Outcomes that solve diﬃcult problems

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
It’s “whys” all the way down

@bsktcase #AFITC #fantasticoutcomes
Calibrating our “whys”
If we “why” too far
We lose our correlation to measurable proxies
Confounding variables contaminate our data
We don’t know whether we aﬀected the outcome
If we don’t “why” far enough
We don’t correlate to actual value delivered
Every metric becomes a vanity metric
We don’t know whether we aﬀected an outcome

@bsktcase #AFITC #fantasticoutcomes
Zeroing in on a useful level of “why”
Beneﬁt
Why the warﬁghter needs the outcome; their unfair advantage
🎯 Outcome
A problem that your product solves for the warﬁghter
Output
The product you create
Feature
An activity the airman can perform within your product

@bsktcase #AFITC #fantasticoutcomes
are metrics to blame?
@bsktcase #AFITC #fantasticoutcomes

@bsktcase #AFITC #fantasticoutcomes
How outcomes (should) cascade and inform us
Vision
The WHAT and WHY
Decades +
Strategy
HOW we will REALIZE the vision
Years
Roadmap
TIMELINE and TACTICS for how
we intend to execute the strategy
Months
Backlog
The DETAILS we need to create
the product
Weeks

@bsktcase #AFITC #fantasticoutcomes
What often happens instead
Vision
Seems unattainable or even insincere!
Epics? Features?
Wait, where did these come from?
Warﬁghter focus may have been here once but
we’ve misplaced it now!
Backlog
What happened to our warﬁghter?
Or even our airman?

@bsktcase #AFITC #fantasticoutcomes
where to begin
@bsktcase #AFITC #fantasticoutcomes

@bsktcase #AFITC #fantasticoutcomes
Outcome-driven metrics
Identify outcomes
Follow the “whys” out until you ﬁnd warﬁghter beneﬁts and unfair advantages
Then work backwards to ﬁnd warﬁghter outcomes, i.e., their problems you can solve
Build strategies and roadmaps around outcomes
Be cautious about reframing existing backlogs, as you may miss faulty assumptions
Ask, “how will we know?”
Start qualitatively; keep the focus on the warﬁghter
Then, and only then, ask, “how will we measure?”
Be ever-vigilant against measuring outputs instead of outcomes!

@bsktcase #AFITC #fantasticoutcomes@bsktcase #AFITC #fantasticoutcomes
Remembering why we do this work in the ﬁrst place

What's hot

Resistance to scrum

Jon McNestrie

How to talk to your Product Owner

Lieke van der Poel

It seems quite well understood who a ScrumMaster is, who a developer or tester is but what about the Product Owner. People often seem to fall into this role without a full understanding of what it's all about. I frequently find myself wondering what the perfect Product Owner would look like. Where do they come from, what qualities and experience do they have and why are they so hard to find! What makes a good product owner and try and where can we find or how can we create this rare creature.

The Mythical Product Owner

Jon McNestrie

Agile manifesto 17 authors - who made it? what do they do?

Jae-Hee Lee

Everyone wants a good architecture, so most of us learned all about design patterns, microservices and all the other important things. But with the raise of DDD it is getting more obvious again: We need to focus more on understanding the functional requirements and specifications instead of thinking too much about the solution. This doesn’t mean we need to go back to the "Big Design Up Front" days of waterfall, but it seems that agile made us forget that some planning is actually quite useful. Most of the time we haven’t even understood the problem before we start coding. I want to show how Architectural Katas can help us train methods like events storming or to focus more on requirements in general.

Introduction to architecture katas

Frank Sons

Architecture Katas - How can we learn to create better architectures?

Frank Sons

Five Things I've Learned in Ten Years of Agile

Flavius Stef

Experience Report from UX Cambridge 9th September 2015 ---- About the Experience Report ---- Skyscanner, the world's travel search engine, launched in 2003 and has grown at significant pace, with over 600 employees spanning 53 nationalities across nine global offices. In any company experiencing this level of growth, it's often challenging to create and maintain a unified user experience. In this session we'l look at some of the challenges we've faced as designers. You'll gain insight into some of the pitfalls that a rapidly growing company can encounter when trying to manage both brand and user experience at scale. Touching on how creating a product shapes the modern day brand, we'll look at how we've approached designing relevant experiences for the world's travellers. We will also take a brief look at some of the technical challenges of delivering a coherent design experience across multiple platforms, technology stacks and co-located teams. ---- About the Speaker ---- James is a full stack designer specialising in crafting user experiences for the mobile web, with a love for combining design and code to create ideas that delight and come alive in the browser. He is currently working for meta travel search engine Skyscanner, solving complex problems for the world's travellers. --- @jamesf3rguson @skyscanner @uxcambridge #uxcam

Designing at Scale: Creating a Global User Experience

James Ferguson

Continuous Learning - The New ALM

Gil Zilberfeld

What's hot (9)

Resistance to scrum

How to talk to your Product Owner

The Mythical Product Owner

Agile manifesto 17 authors - who made it? what do they do?

Introduction to architecture katas

Architecture Katas - How can we learn to create better architectures?

Five Things I've Learned in Ten Years of Agile

Designing at Scale: Creating a Global User Experience

Continuous Learning - The New ALM

Similar to 2019 AFITC Fantastic outcomes and how to measure them by Cheryl Hammond

When the broader organization isn’t agile, teams can get stuck. They’re waiting for their knight in shining armor – the ‘enlightened’ executive – to transform it all. What happens when the hero isn’t coming? Is it possible to realistically accept an organization’s limitations without killing morale? Can a manager or coach ‘shield’ a team without losing momentum? Is there any point in making incremental change at the team or division level when ‘real’ progress seems eternally blocked by external impediments? This session explores practical tips to care for the humans in your realm; provide them a safe, comfortable, and defensible space in which to work; and make it inviting enough to win over (some of) the forces outside. Help your teams become self-rescuing princesses, and create the happily-ever-after that your people deserve!

2022 Fortress Agile by Cheryl Hammond at Agile Tour Vilnius.pdf

Cheryl M Hammond

If you're fortunate enough to have convinced your organization that "quality" matters at all, congratulations! That's the first step. You've probably walked right into the next pitfall, and you may or may not know it yet: when all roles are passionate about "quality", but no one agrees on what it is. Or worse, everyone agrees, but you don't realize you're all missing something. In this session, we'll expose how the use, overuse, and abuse of the word "quality" can mislead even the best-intentioned agile teams and organizations. It's not about role vs. role—no one is immune, not even people with "quality" right there in their job title. You'll learn about a specific common "quality" anti-pattern, the harm it causes, how to spot it, and how to solve it. We'll practice communication strategies that will help you understand and influence your colleagues more effectively. We all want to make our products better. It's time to stop talking about "quality" and learn to say—and more importantly, do—what we really mean.

I Do Not Think It Means What You Think It Means: "Quality" and Why We Should ...

Cheryl M Hammond

Birst Webinar Slides: "Build vs. Buy - Making the Right Choice for a Great Da...

Birst

A crash course into SEO and what moves the needle with scalable processes

patrickstox

Even more than other methodologies ("frameworks", whatever), Kanban requires deep understanding in order to bring about real change and measurable results. Too many teams struggle, even after they understand the "rules" of Kanban—failing to recognize visible problems on their own boards, or not knowing how to replace old bad habits with better ones. I see the same struggle, in microcosm, when I train teams using the open-source getKanban Version 2.0. Even though the game urges teams to limit work in progress (WIP)—the linchpin of Lean and Kanban—I found that my players inevitably wanted to raise it. Over numerous classes and diverse groups of players, I tried hints, then rants, about how limiting WIP reduces cycle time. But there was a problem: play teams who took my advice and lowered their WIP weren't seeing better outcomes. In this session, I'll show you the adjustments I've made to getKanban Version 2.0 to guide players to their "a-ha" moment—a deep, personal experience they feel and remember. Now, instead of just the basics of how a Kanban board works, my students understand how to detect and solve problems that impede their flow. I see the difference when they leave the game and go back to work. For players and coaches who are already familiar with the basics of Kanban, my new open-source hacks to Version 2.0 will help you go deeper and illuminate Lean principles for your clients and teams—and yourselves!—so you can achieve real results faster.

Hacking getKanban v2.0: Advanced Game Mechanics to Illuminate Lean by Cheryl ...

Cheryl M Hammond

Design as Strategy (or, Intro to Competitive Strategy for Designers)

Skot Carruth

User stories are one of the most visible artefacts of most agile methods and, as such, have generated large quantities of expert advice. In my experience, much of that advice is open to misinterpretation. In this session, we'll explore several classic pieces of advice, to see how misunderstandings can cause problems, despite the best intentions. The examples we'll look at are: - an acronym: INVEST, created by Bill Wake - a technique: relative estimation using story points, created by Ron Jeffries or Joseph Pelrine - a template: Connextra (As-A/I-Want/So-That), created by Rachel Davies Expert advice taken in good faith, that leads to bad outcomes, can cause us to become distrustful. It's time to reiterate that there is no magic formula, no silver bullet. At best, experts can lend you a framework within which to think, but their advice will never make thinking unnecessary.

User stories: from good intentions to bad advice - Lean Agile Scotland 2019

Seb Rose

Stayin' Alive! Feature Disco Your Way to PI Planning

Em Campbell-Pretty

What the Bot!? How Salesforce Geared Up for Chatbots

Gavin Austin

A Research Study into DevOps Bottlenecks as presented at Codemash 2018

Baruch Sadogursky

Content creation is hard. In spite of all the tools and data that marketers have available we still generate content which, after launch, doesn't always resonate with users and search engines in the way we'd hoped. During my talk, I dove into Aira's content generation process that's helped one client get over 100,000 organic search clicks by using a repeatable methodology based off thorough keyword research and content brief creation.

Pint-Sized Marketing - A Repeatable Content Generation Process that Generated...

Matthew Kay

Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdf

Sophie Gibson

Why your tech optimisations are still sat in the backlog

Vicky481083

Content for 2015 and Beyond by Matt Beswick, SEM Days 2015

SEO monitor

BrightEdge Share15 - CM203: Scaling Content: Production, Process & Culture - ...

BrightEdge Technologies

In the workshop, Digital Clarity Group's Cathy McKnight and Scott Liewehr explored the customer experience management (CEM) Imperative and how it impacts the enterprise within the firewall, and the need to shift from an “inside-out” to “outside-in” focus for how employees work in order to support and drive CEM for their organization. In the second part, they analyzed the technologies, skills, and processes that will help employees engage and collaborate, and how to evaluate, select, and assemble the team involved in executing the CEM strategy (internal resources, outside service providers, and vendors).

J boye14 workshop CEM Imperative - the right team tools and tactics

Digital Clarity Group

Milestones, SHUV, Roadmaps - Oh My!

Jacqueline K. Sanders-Blackman, Agilist,CBAP,SSBB,CSM,SAFe

Milestones, SHUV, Roadmaps - Oh My!

Technology Expresso Expresso-Consulting

How Personality and Culture Determine Collaboration Success

Christian Buckley

The Zachman Framework and enterprise architecture. An enterprise consists of many components. They are categorized and organized into groups like organizations, locations and so on. The components also relate to each other in some manner. Each enterprise has a unique set of components and set of relationships that differentiate one enterprise from another. There needs to be some framework that that provides rules for how the components of an enterprise are organized.

Zachman Framework and the Periodic Table: A Fun Romp Through Some Basic Frame...

KnowledgeConsultants

Similar to 2019 AFITC Fantastic outcomes and how to measure them by Cheryl Hammond (20)

2022 Fortress Agile by Cheryl Hammond at Agile Tour Vilnius.pdf

I Do Not Think It Means What You Think It Means: "Quality" and Why We Should ...

Birst Webinar Slides: "Build vs. Buy - Making the Right Choice for a Great Da...

A crash course into SEO and what moves the needle with scalable processes

Hacking getKanban v2.0: Advanced Game Mechanics to Illuminate Lean by Cheryl ...

Design as Strategy (or, Intro to Competitive Strategy for Designers)

User stories: from good intentions to bad advice - Lean Agile Scotland 2019

Stayin' Alive! Feature Disco Your Way to PI Planning

What the Bot!? How Salesforce Geared Up for Chatbots

A Research Study into DevOps Bottlenecks as presented at Codemash 2018

Pint-Sized Marketing - A Repeatable Content Generation Process that Generated...

Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdf

Why your tech optimisations are still sat in the backlog

Content for 2015 and Beyond by Matt Beswick, SEM Days 2015

BrightEdge Share15 - CM203: Scaling Content: Production, Process & Culture - ...

J boye14 workshop CEM Imperative - the right team tools and tactics

Milestones, SHUV, Roadmaps - Oh My!

How Personality and Culture Determine Collaboration Success

Zachman Framework and the Periodic Table: A Fun Romp Through Some Basic Frame...

More from Cheryl M Hammond

Why did your latest agile/cloud/DevOps/big data/digital/crypto/AIML transformation stall or fail? Along with the massive layoffs impacting big tech generally, it seems we all know someone whose agile coaching role has been impacted or outright eliminated. Famously, Capital One had an agile transformation effort said to have been going on for more than TWENTY YEARS — and completely abandoned it. What drew you to agile? Was it pride in successful delivery with high quality? The thrill of innovation and experimentation? A passion for people — respecting them through servant leadership, self-organization, and sustainable pace? If you’re feeling a disconnect between your employer’s priorities and your agile values, you’re not alone. In this session, we’ll talk about the business benefits of agile and how some conventional wisdom leads us down wrong and frustrating paths. We DON’T need to wait for the mythical executive sponsor who isn’t coming and can’t help anyway. Sometimes we’re all we’ve got — let’s work together to be all we need.

2023 'We Just Need Executive Support' and Other Lies of Transformation by Che...

Cheryl M Hammond

Whether you’re excited to collaborate in-person or you dread having to wear shoes again—or both—it’s clear the great Return To Office is a big adjustment for you and your teams. In the past 2-3 years, we all tried to tailor our home office environments to help us work at our best. Now we have to re-learn how to share space with others whose working styles, communication styles, needs, and preferences are very different. In some cases, our companies’ Return To Office policies are creating new problems that we might not be able to fix right away. Teamwork and self-organization are the heart of any agile practice, and this fast-paced session will teach you several concrete techniques for resetting team norms in our new normal. We’ll talk about working agreements and conflict resolution that can help fully co-located teams as well as hybrid teams. Return To Office might be challenging, but if your team can pull together and help each other through the transition, you’ll be set up to tackle any problem that comes next!

2023 Using Agile Self-Organization to Help Your Team's 'Return To Office' Suc...

Cheryl M Hammond

Does ‘subsistence’ carry a negative connotation for you—scarcity, poverty, barely scraping by? In this era of enterprise scaling and big digital transformations, our trusty old agile often feels the same. It’s easy to get stuck when it seems like every retrospective action item worth doing requires an executive sponsor who isn’t coming. Our planetary challenges require us to think of ‘subsistence’ differently. The ability to produce what we need, sustainably, using the resources available in our communities, is powerful! Are you getting the best from the ecosystem you already have? This session explores practical tips for nurturing and growing your agile practice hyper-locally, using techniques accessible at the team level. Learning outcomes: - Build and strengthen the supportive agile community already around you - Reduce your dependence on expensive faraway ‘imports’ - Expand your influence and capabilities by developing ‘trade’ networks nearby - Focus on what’s most important for you and your team to thrive!

2022 Subsistence Agile - Sustainability and Self-Sufficiency for Thriving Tea...

Cheryl M Hammond

Organizational diversity isn't just about the attributes we can see. Every team has members with a near-infinite spectrum of needs, some of which we know about and a lot more we probably don't (and might never). How do we create a safe, accommodating space for things we aren’t aware of? It's possible. For our teams to reflect the diversity of our worlds, HR policies are necessary but not sufficient. Our teams can and should take concrete steps, large and small, to make ourselves more welcoming. You're probably doing some of them already, so let's talk about what works and how we can scale it out. From stairs, ramps, and elevators to menu planning to planking and wall sits during meetings, Cheryl Hammond discusses how to rethink our norms, retire our assumptions, ask the right questions without prying, and create space for our teammates to self-accommodate. You might never know the difference you've made, but your teams will be healthier and wiser for it.

2019 Full-spectrum diversity: rethinking our norms by Cheryl Hammond

Cheryl M Hammond

2019 Fantastic Outcomes and How to Measure Them by Cheryl Hammond

Cheryl M Hammond

If you don't know how to measure what you want, you'll end up wanting what you can measure. Most often the thing you want to know isn't easily quantifiable, and common proxy metrics are usually poorly correlated with the information you actually need. Measuring the wrong things is worse than nothing—a toxic metric can damage your teams' performance. With the right data, you can change the conversation. Tell your team's authentic story to management, your customers, and beyond. Step away from dangerous metrics that punish unfairly. Quit wasting time with metrics that are easily gamed. Instead, choose effective metrics to get everyone on the same page about what's important. Whether you're the measurer or the measuree, in this session, you'll learn not just which metrics work, but why and how. Our examples will focus mainly on team, project, and program metrics, with theoretical guidance to inform all kinds of measures including portfolio and organization. Understand the difference between true metrics and proxy metrics, and good proxies and evil ones. Discover a framework for evaluating any metric, a Hall of Shame covering some of the worst most popular benchmarks, and one true guide to point you to the very best metrics of all. See some great examples of visualization that make metrics sing, and leave with several concrete measures you can begin tracking as soon as you get back to your desk.

2018 One Metric to Rule Them All - Effectively Measure Your Teams Without Sub...

Cheryl M Hammond

2017 One Metric to Rule Them All: Effectively Measure Your Teams without Subj...

Cheryl M Hammond

You put up your Big Visible Display. Maybe you even limited WIP, or argu^H^H^H^Htalked about limiting it. Something more was supposed to happen. Why didn't more happen? If your Kanban implementation feels like this... Phase 1: Collect underpants Phase 2: ? Phase 3: Profit ... you're not alone. The Underpants Gnomes can be a scourge of Kanban. Fortunately, they don't have to have the last word. In this session, you'll learn how to detect Underpants Gnomes in your organization's Kanban process, and how to combat them. We'll investigate three concrete solutions—by-the-book, and beyond—and fill your toolbox with simple, proven strategies for restarting your Kanban after it stalls. You're ready for the real Phase 2. Downsize the Underpants Gnomes and let's get started!

Beyond the Underpants Gnomes: Why Kanban Stalls and How to Restart It by Cher...

Cheryl M Hammond

Lean Coffee is a structured format for a completely crowdsourced discussion. Attendees suggest topics, stack rank them via a voting process, and then discuss each one in order, using short timeboxes until they reach consensus to move to the next one. We reconvene at the end of the session time to share takeaways. We tried Lean Coffee as an experiment at Seattle Code Camp 2016, focusing on topics of interest around women in tech, and the collaboration was fantastic! Here are the slides we used—setting the stage with principles from Open Space Technology and finishing with a Lean Coffee quick outline—so you can learn more and facilitate a session on your own.

Women in Tech - Quick Reference Guide for Facilitating a Lean Coffee by Chery...

Cheryl M Hammond

Measuring the wrong things is worse than nothing; a toxic metric can damage your teams' performance—or your bonus. Whether you're the measurer or the measuree, in this session, you'll learn not just which metrics work, but why and how. Understand the difference between true metrics and proxy metrics, and good proxies and evil ones. Discover a framework for evaluating any metric, a Hall of Shame covering some of the worst most popular benchmarks, and one true guide to point you to the very best metrics of all. This updated session was presented at Seattle Code Camp 2016.

Defense Against the Dark Arts: Protect Yourself from Evil Metrics by Learning...

Cheryl M Hammond

Beyond the Full Stack: Growing as a Technology Leader by Cheryl Hammond

Cheryl M Hammond

Lean for Agilists by Cheryl Hammond

Cheryl M Hammond

One Metric to Rule Them All: Effectively Measure Your Teams Without Subjugati...

Cheryl M Hammond

More from Cheryl M Hammond (13)