Downloaded 99 times
![Data Driven Coaching
Safely turning team data into coaching insights (Troy Magennis)
These slides available here:
[link]
@t_magennis
troy.magennis@FocusedObjective.com](https://image.slidesharecdn.com/datadrivencoaching-agile2016troymagennis-160720220114/85/Data-driven-coaching-Agile-2016-troy-magennis-1-320.jpg)
![65
Probability Density Function
Histogram Gamma (3P) Lognormal Rayleigh Weibull
x
1301201101009080706050403020100-10
f(x)
0.32
0.28
0.24
0.2
0.16
0.12
0.08
0.04
0
1997: Industrial Strength Software
by Lawrence H.
Putnam , IEEE , Ware Myers
2002: Metrics and Models in
Software Quality Engineering
(2nd Edition) [Hardcover]
Stephen H. Kan (Author)
Paper: http://bit.ly/14eYFM2](https://image.slidesharecdn.com/datadrivencoaching-agile2016troymagennis-160720220114/85/Data-driven-coaching-Agile-2016-troy-magennis-63-320.jpg)
Uploaded byTroy Magennis
Data driven coaching - Agile 2016 (troy magennis)
The document discusses the importance of using data-driven insights for coaching teams, emphasizing that data should tell a story and be presented in a meaningful way. It includes various metrics and frameworks, such as cycle time analysis and the Software Development Performance Index (SDPI), to measure aspects like productivity, quality, and predictability. Additionally, it provides multiple resources and tools for effective data visualization and forecasting in team performance management.
More Related Content
Similar to Data driven coaching - Agile 2016 (troy magennis)
![The Future of HR: From Metrics to Analytics [Webcast]](https://cdn.slidesharecdn.com/ss_thumbnails/hrmetricstopredictiveanalyticsslideshare-151118001524-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Laila Kakar - Leveraging AI for Strategic Excellence: Enhanci...](https://cdn.slidesharecdn.com/ss_thumbnails/eykmhrtsqmaaftwkexh7-dsc-lailakakar-1-260119101520-5f3b5616-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Jovan Sumarac - Real-World Applications of Computer Vision in...](https://cdn.slidesharecdn.com/ss_thumbnails/fiksms22smcpopvvld03-jovan-sumarac-real-life-applications-of-computer-vision-in-automotive-systems-260120105855-de622abb-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Srdj Stanisic - Local and Private AI in UX.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/vwmetykqmztgmokmmkfa-3-srdjan-stanisic-local-and-small-ai-in-ux-260120105855-55a31869-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Tamas Srancsik - How To Teach Your AI Football? An Argument f...](https://cdn.slidesharecdn.com/ss_thumbnails/bcjh1m9xtbosv20ucftb-tamas-srancsik-how-to-teach-your-ai-football-260121115910-08b53e9e-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Borko Kozomora - Optimizing business workflows with advances ...](https://cdn.slidesharecdn.com/ss_thumbnails/hbgekyb0txw0xpo4yfml-borko-kozomora-leading-ai-transformation-260122103838-cc29ee38-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Tali Fulman - Guild Meetings, Then What? Building Data Commun...](https://cdn.slidesharecdn.com/ss_thumbnails/fgohhi33rwmhqdowdj5k-tali-fulman-guild-meetings-then-what-building-data-communities-that-actually-ch-260120105855-528492c3-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Marcos Heidemann - Beyond the Hype: Making AI Coding Assistan...](https://cdn.slidesharecdn.com/ss_thumbnails/eexkhvldrjsopspdjbur-marcos-heidemann-beyond-the-hype-getting-real-value-out-of-ai-assisted-coding-260121115910-7e9d41ec-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Gordana Milutinovic Dumbelovic - From Insight to Oversight: A...](https://cdn.slidesharecdn.com/ss_thumbnails/t7dkjsfxqwwzceropjv4-gordana-milutinovicdumbelovic-from-insight-to-oversight-ai-driven-power-bi-moni-260119121559-9e0bf11b-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Josip Saban - Career building for data professionals.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/zroflcttkm1vmli0txea-josip-saban-career-building-for-data-professionals-260123083019-587cdb8c-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Mikhail Rozhkov - AI Product Canvas: From Business Goals to T...](https://cdn.slidesharecdn.com/ss_thumbnails/d53doddtpgfqivmzqel6-mikhail-rozhkov-ai-product-canvas-v1-260121115910-9dd517a7-thumbnail.jpg?width=640&height=640&fit=bounds)
Data driven coaching - Agile 2016 (troy magennis)
- 1. Data Driven Coaching Safelyturning team data into coaching insights (Troy Magennis) These slides available here: [link] @t_magennis troy.magennis@FocusedObjective.com
- 2. 2 If it walkslike a duck, and quacks like a duck, it could still be a rabbit.
- 3. Data is E VI L
- 4. Never coerce Never embarrass Beingjudged unfairly is un-bearable…
- 5. Make a difference, notjust make a point
- 6. Use data to tella story…
- 7. Without a story, data isboring…
- 8.
- 9.
- 10.
- 12.
- 13.
- 14.
- 16. Not compared withanything meaningful (to me.) Boring.
- 17. http://www.datarevelations.com /are-you-over-the-hill-in-the-usa “Me” On the 7thAugust I WANT TO KNOW But, I may not want you to know. You are YOUNGER than 37.3%
- 18.
- 20.
- 21. Time and Pacerelated questions 1. Is it taking us longer to do the same type of work? 2. What is a good commitment cycle time to others? (SLA) 3. What is and how stable is our completed work rate? 4. Where should we focus improvement efforts? • Compared to what? • Compared to the same type of work versus all work • Compared to the same time period last week/month/year • My work compares to others (only seen by me so I can improve)
- 22. “If anyone adjustsa stable process, the output that follows will be worse than if (s)he had left the process alone” Attributed to William J Latzko. Source: Out of the Crisis. Deming. Q. Is the process stable? First, do no harm.
- 25. Demand on thisteam decreasing Cycle-time stable Bulk close? Stable distribution
- 29. Source: JumpPlot.com (TomVanBuskirk and Chris DeMartini )
- 30. Its too hardand we don’t have the data
- 31. Q. What couldI do with just start and completed date? http://bit.ly/Throughput Or follow @t_magennis
- 32.
- 33. 17 charts sofar… Throughput (planned & un-planned) Throughput Histogram(s) Cycle Time (planned & un-planed) Cycle Time Histogram(s) Work In Process Cumulative Flow Arrival vs Departure Rate Un-planned work Percentage Cycle Time Distribution Fitting
- 34.
- 35.
- 36.
- 37.
- 38. 1930 to 2012 NationalLeague MVP 23% = 19 out of 82 (last time 1988 - Carston) 1930 to 2013 All-American League MVP 19 out of 82 (last time 1984 - Hernandez) 1955-56 to 2015-16 NBA MVP 37% = 23 out of 62 (last time 2014 – Curry) Source: NBA Most Valuable Player Award. (2016, June 24). In Wikipedia, The Free Encyclopedia. Retrieved 18:28, July 3, 2016, from https://en.wikipedia.org/w/index.php?title=NBA_Most_Valuable_Player _Award&oldid=726766319Source: ESPN Playbook - SportsData (infographic at end of this deck) 23% 37%
- 39.
- 40. You can’t WINby just having the BEST goalkeeper A 0-0 draw is the best you can HOPE The goalkeeper can’t win SOLO
- 41.
- 42.
- 45. Find balance… Inchanging conditions And competing forces
- 46. 4. Predictability (how repeatable) 3.Responsiveness (how fast) 1. Quality (how well) 2. Productivity (how much, delivery pace) • Escaped defect counts • Forecast to complete defects • Measure of release “readiness” • Test count (passing) • Throughput • Velocity • Releases per day • Lead time • Cycle time • Defect resolution time • Coefficient of variation (SD/Mean) • Standard deviation of the SD • “Stability” of team & process
- 47.
- 48. Quality • Goal isto keep the TEAMS within 10 days of releasable • Forecast has to be personal for the team • Days = Open Bugs x Avg(recent cycle time samples) Number of Devs on team “If OUR entire TEAM did nothing else but fix bugs this sprint, at OUR historical rate, we would have x days of work”
- 49.
- 50. Parallel and better Startedworse, but corrected Improving, against company trend
- 51. Better and with companytrend Oops. Still good, but trending adversely Creeping up…
- 52. Don’t Make itPersonal Beautiful + Engaging Tell a Story Compared to What Keep it Simple Balanced Metrics Make GREAT tradeoff Decisions
- 53.
- 54.
- 55. Forecasting and Risk Helpingteams see and understand risk impacts
- 56. Q. Could Imake a simple forecast tool that worked? http://bit.ly/ThroughputForecast Or follow @t_magennis Without macros or add-ins!
- 57.
- 58.
- 59.
- 60. http://bit.ly/ThroughputForecast 2 ½ MonthRange ~ 20 Day Range
- 61. Cycle time analysis Howto interpret cycle time distributions in coaching @t_magennis | Bit.Ly/SimResources 63
- 62. Q. Can historicalcycle-time be used for coaching advice? http://conferences.computer.org/hicss/2015/papers/7367f055.pdf
- 63. 65 Probability Density Function HistogramGamma (3P) Lognormal Rayleigh Weibull x 1301201101009080706050403020100-10 f(x) 0.32 0.28 0.24 0.2 0.16 0.12 0.08 0.04 0 1997: Industrial Strength Software by Lawrence H. Putnam , IEEE , Ware Myers 2002: Metrics and Models in Software Quality Engineering (2nd Edition) [Hardcover] Stephen H. Kan (Author) Paper: http://bit.ly/14eYFM2
- 64. 1970-1990’s Waterfall Rayleigh Distribution, Weibull shapeparameter = 2 66 Approx 2000 Weibull shape parameter = 1.5 Approx. 2008 Lean Weibull shape parameter = 1.25 Approx 2010 Exponential Distribution, Weibull shape parameter = 1 Work Item Cycle Time or Lead Time Distribution Through the Ages Cycle Time in Days Paper: http://bit.ly/14eYFM2
- 65. 67 Shape=2 Scale = 30 ~1 month Scale = 15 ~ 2 week sprint Scale = 5 < 1 week Shape=1.5Shape=1 Work Item Cycle Time or Lead Time Batch Size / Iteration Length ProcessExternalFactors
- 66. Lean, Few dependencies •Higher work item count • More granular work items • Lower WIP • Team Self Sufficient • Internal Impediments • Do: Automation • Do: Task Efficiency Sprint, Many dependencies • Lower work item count • Chunkier work items • Higher WIP • External Dependencies • External Impediments • Do: Collapse Teams • Do: Impediment analysis 68Paper: http://bit.ly/14eYFM2
- 67. 69 0 to 1010 to 30 1.3to2 (WeibullRange) 1to1.3 (ExponentialRange) Traits: Small or repetitive work items. Low WIP. Few external dependencies. Good predictability. Process advice: Automation of tasks, focus on task efficiency. Lean/Kanban optimal. Traits: Larger unique work items. High WIP. Low predictability. Many external dependencies. Process advice: Focus on identification and removal of impediments and delays, and quality. Scrum optimal. Traits: Small unique work items. Medium WIP. Few external impediments. Fair predictability. Traits: Larger work items. Large WIP. Many external dependencies. Poor predictability. Weibull Scale Parameter WeibullShapeParameter @t_magennis | Bit.Ly/SimResources
- 68.
- 69. Tools • Excel orGoogle Sheets Spreadsheets (all free) • General metrics spreadsheet (17 charts) – • Team Capability Matrix - • Forecasting – • 10+ other spreadsheets tools all free - • Visualization Tools • Tableau ($995-$1995) – Tableau.com • PowerBI (free) – • Plotly (free) – • Online Lean/Kanban Tool • Leankit.com
- 70. Cool Visualization Resourcesand Websites • My blog – FocusedObject.com/blog • WindyTy.com – weather • NY Times • Tableau Public • Books • Tufty • Few
- 72. Source: JumpPlot.com (totalkudos to Tom VanBuskirk and Chris DeMartini )
- 73. Coaching professional teams •Is about team performance, not individual • If they don’t know it by now, they self improve it • http://www.landofbasketball.com/awards/nba_season_mvps_year.ht m • 23 championships + MVP / 60 = ~ 1/3 • http://www.nba.com/2011/news/features/04/08/race-to-the-mvp- final-rankings/index.html • http://national.suntimes.com/nba/7/72/1237030/lebron-james- stephen-curry-nba-finals-mvp
- 74. SDPI Dimensions • Productivity= throughput avg / team size • Predictability = variability of throughput / size • Responsiveness = time in process average • Quality = released defect density / throughput The Software Development Performance Index The SDPI framework includes a balanced set of outcome measures. These fall along the dimensions of Responsiveness, Quality, Productivity, Predictability, … Example, team over time - Source: Rally Dev. 76
- 75. Responsiveness • Average ormedian of the number of days between two dates for items closed within a period • Cycle time or Lead time of ??? • If reliable first touch date, use that • If just created date, then use P1 and P2 bug “If something urgent comes along, how fast can we turn that around” @t_magennis | Bit.Ly/SimResources 77
- 76. Completion Rate • Teamgoal is to maximize number of COMPLETED items, not started items • Count of items completed each period • Don’t celebrate bug throughput (as much) “What is holding us back on completing more. Lets discuss dependencies and blockers in the retrospective” @t_magennis | Bit.Ly/SimResources 78
- 77. Predictability • How muchvariation there is each week in throughput, normalized by “team size” in a rough way • Coefficient of Variation = Mean/SD “How consistently do we deliver value?” @t_magennis | Bit.Ly/SimResources 79
Editor's Notes
- #3 Interpreting data we capture for process improvement and coaching teams is hard. Sometimes you think you understand what you see, only to have that taken away from you. Sometimes, If it walks like a duck and quacks like a duck, it could still be a rabbit. Perspective and context matters.
- #4 Data isn’t evil, people are evil. Sure, sometimes data can highlight something we would rather not be true, but this doesn’t make it wrong to capture and analyze data in proper ways.
- #5 No matter how strong you are, being embarrassed is painful. When data is perceived as evil, its most commonly because it has been used as a blunt tool for coercing some behavior. Never do this, and never make data the tool to embarrass people. Not just because its wrong to that person, because it causes data to make better decisions to go underground. If you embarrass someone using data without context, then you are now just seeing a partial picture in the future – and your decisions will be sub-par. If that’s not enough, Deming puts the people aspect of process impact around the 10-15% range versus 85-90% for the system. A metric managing the system response is far more likely to return benefits.
- #6 A good way to assess how data is being used is to ask the simple question, is that data being used to make a difference? Or is it just being used to make a point, to say I’m better than you.
- #7 To make a difference, data should tell a story. The readers should be able to follow the argument and train of thinking, and see something they weren’t expecting. A story doesn’t just jump to the last page to see whether little red riding hood lives happily ever after, there is a buildup of the characters. Your data stories should do the same. They should take the reader on a journey and leave them with the moral to the story.
- #8 Lets see a couple of examples of taking boring data and turning it into a story. Obligatory cat photo to make sure this deck is tweeted.
- #9 Boring tabular data. The domain of Excel. Also the format NOAA makes weather data available to the public. Pretty boring stuff, even for weirdo data geeks like me. But the clever Ivo an avid kite surfer turned this data into this [click]
- #10 Beautiful. But it gets better, the wind speed and direction and temperature are animated [click]
- #11 You can feel the weather pattern in my hometown of Seattle. Color helps me quickly interpret areas of higher wind speed. I can change altitude and see how the jetstream will impede or help my flight to Atlanta. I can see what the wind and weather is like here in Atlanta. I can see clouds and rainfall, temperature or barometric pressure. I can see how this is expected to trend in the past or future. I can see the story of the atmosphere and weather better than reading it from an excel text table.
- #12 Impressive, but hard for someone of my skill to do. Lets see one that we all could have built using Excel. The Wall Street Journal did a feature piece on Vaccination. They took boring health data from the University of Pittsburgh and turned it into this [click]
- #13 Even without the annotation, was the Polio Vaccine effective? Introduced in 1955 you can see the story of how every state in the US saw a decline of infections for every 10,000 people. The picture tells the story of how within 5 years the disease infected fewer people as the number of immunized increases and thrarted a debilitating disease into the rear vision mirror. Source: Battling Infectious Diseases in the 20th Century: The Impact of Vaccines By Tynan DeBold and Dov Friedman Published Feb. 11, 2015 at 3:45 p.m. ET http://graphics.wsj.com/infectious-diseases-and-vaccines/ http://www.tycho.pitt.edu http://www.tycho.pitt.edu
- #14 Edward Tufte – author and researcher on visual communications. Every number we show or hear should be put into the context “Compared to What?” A single number is meaningless. Single numbers make us susceptible to drawing incorrect conclusions. We need to know if the value we just got is better, worse or comparable to something else to see a story.
- #15 In our prior example, the authors managed to pack four “Compared To’s” into one graphic. State versus state. Year versus year, Before and after an event, and occurrence rates versus others in color. Source: Battling Infectious Diseases in the 20th Century: The Impact of Vaccines By Tynan DeBold and Dov Friedman Published Feb. 11, 2015 at 3:45 p.m. ET http://graphics.wsj.com/infectious-diseases-and-vaccines/ http://www.tycho.pitt.edu Compared to event, compared to prior time, compared to other states
- #16 And to see it wasn’t just a fluke, here is the same chart for Measles. Gone in under 5 years after vaccination introduced. Source: Battling Infectious Diseases in the 20th Century: The Impact of Vaccines By Tynan DeBold and Dov Friedman Published Feb. 11, 2015 at 3:45 p.m. ET http://graphics.wsj.com/infectious-diseases-and-vaccines/ http://www.tycho.pitt.edu
- #17 Our last example shows census data. The age of the US population in a recent census. Steve Wexler took this boring data and put a clever spin on it to help create a personal story for the viewer. [click]
- #18 He let you put YOUR age in and see where you fell. Are you older than more than 50% of the population? Are you over the hill? I am. I really wish he had said YOUNGER than 37.3 of the male population. His point is that as long as its quiet and nobody else is around, you want to see how you compare to others like you. If for no other reason than to be “less over the hill” than your spouse or sibling. I’m both!
- #19 So that’s the theory. Lets look at how LeanKit took an existing report built-in to the application and applied these principles. Steve Wexler and myself works with the talented internal Visualization team to prototype and build the following visualization to replace an existing report as part of a training course.
- #20 Here is the current report. Pretty standard scatter plot of cycle times at the bottom, and a running average of cycle times for each type of work at the top. Design wise, these reports have served customers well. Being 46, the dots are a little small to hover over with my arthritic hands, and its pretty bland. It also doesn’t answer a lot of the questions i have around the time based aspects of software development.
- #21 So, I crowdsourced a set. I used Twitter to get feedback on what questions most resonated with my followers. Crude, but it quickly gave some weight to the possible questions we needed to answer. What we found was more comparison of similar work was key. And the ability to zoom in and out on date groupings, day, week, month for example. Leankit didn’t do wither of these very well in its current form.
- #22 So, taking input from the market, the following four questions were key. And we had three major “Compared to What” vectors. Card Type, Time Periods and My Cards. Now we had an understanding of the story we wanted to tell, it was time to do some research.
- #23 Edward Deming has published many books about his work in managing manufacturing processes, in his book “Out of the Crisis” he goes to good lengths to discuss that when looking at a process, your first job is to do no harm. If the process is “in-control” leave it alone! Touching it causes things to likely get worse. A graphic caught our eye. It is a run chart showing spring manufacturing data. Each dot represented a tested spring, and a downward trend can be seen even though the variation shown by the marginal histogram looks perfectly symetrical. In Deming’s words, you can assess a process unless you can see the change over time and the distribution of variation. The scatter plot alone isn’t enough. The histogram of frequency of result isn’t enough. You need both.
- #24 So we turned our attention to a paper prototype. And wondered what if we put marginal histograms like Deming on the right side and the top of a scatter plot. Would we tell a fuller story about the process and how “stable” it is. And that’s what the team did. [click]
- #25 Although this looks more aesthetically pleasing by not yelling in ALL CAPS using harsh colors. This visualizations packs a lot of information. Lets look at the layout. Across the top are the time period buckets. Users can choose day week or month, we default it to week. And they can choose to show all types of work or just selected ones, we default it to all types [click] Under that is a bar chart showing how many items were completed in that period. This is throughput. We can see by glancing at the bar height how completion rate is changing over time. [click] Then our scatter plot. To avoid all of the dots being overlapped and not accessible by cursor, these dots are randomly jittered in horizontal location. Hovering over them gives popul information about that specific piece of work.. This is technically a jitter plot. [click] The key story though is how the average reference lines in each period trend over time. We try and help the user follow the path from left to right giving them the average value of cycle times for items completed in that time period.[click] On the right hand side if the marginal histogram showing cycle-time distribution. This is an area of my research, how the cycle time histogram relates to Agile process factors, but most people will totally ignore it! [click]
- #26 And here is how I interpret the story of this team for this period. Demand is decreasing, and this lower demand is helping stabilize the cycle time of the items to a little less than 2 days on average. There are a few outliers to understand, in case we can solve the root cause of them. The cycle time distribution is following an expected shape. Of course, I’m the only one that knows that a dev-ops kanban team should expect an Exponential distribution, which this would have been if not for that clump of late january items all closing around 20 days.
- #30 And this is another direction I think we might be heading. Help the user feel the flow of work. This is a jump plot invented and mocked-up to show software process flow. The green jumps across the top is work flowing forward, underneath, work flowing back. The height of each arc represents average cycle time, and thickness represents count. In one plot, you get to see the story of cycle time, throughput and status steps. Amazing piece of story telling for understanding a process and its flow. Source: https://public.tableau.com/views/JumpLineExamples/SDLCDetailedJP?%3Aembed=y&%3Adisplay_count=yes&%3AshowTabs=y&%3AshowVizHome=no Amazing work by Tom VanBuskirk and Chris DeMartini. See JumpPlot.com
- #31 If you think its too hard or you don’t have enough data, your wrong.
- #32 I set out to prove this by setting myself a challenge to see what I could do with just completed and start dates. You can get the result for yourself by downloading the spreadsheet from bit.ly/Throughput with a capital T. Again, all these links will be tweeted at @t_magennis
- #33 Here is the input page. Completed date is mandatory. Start date is optional but well advised. And the type of work. Is it planned or un-planned work is needed to see how these different types stack up. This is optional. Most people don’t use it, I’ll leave that up to you.
- #34 The spreadsheet from just these three inputs creates 17 different charts so far. [click] Throughput rates and histograms. Cycle times values and histograms. Work in process rates, arrival and departure rates, cumulative flow diagrams. And some rather complex mathematical analysis of the cycle time histogram that fits the data to the Weibull probability distribution thanks to John Cook, someone who everyone in this industry should have heard of. It does all of this without macros. Everything is straight formula based. And all from 2 dates and one type column that’s optional.
- #35 Here are some examples. Here is the throughput chart. Intentionally kept light weight in design. Intentionally designed to draw your eye along the journey not focus on the values. This is the throughput trend week over week.
- #36 Here is a chart I use a lot to understand coaching opportunities. How much unplanned work is a team encountering. Helps to understand what external pressure is fighting for a teams attention and to help come up with a balance of managing planned versus unplanned work types. I can see this team is slowly driving down its un-planned percentage. Hopefully by using better triage practices and helping external parties get their work into the next sprint rather than interrupting this one.
- #37 The obligatory cycle time chart. Although now I showed you what LeanKit have planned, this is a let down. Still, not bad from a couple of dates. The percentile marker helps the team communicate an expected service level agreement. You don’t like 95%, change it to 85%. Its just a value.
- #38 And this is a new experiment. It tries to visualize the story of supply versus demand. Above zero, the team is completing more than its starting, below the line, starting more than completing and growing WIP. This is intended to tell the flow story, and help the team strive to balance starting and completion by staying close to zero in the center. I’m hoping this becomes more of a go-to chart than Cumulative Flow for teams who want to focus on consistent flow of value.
- #39 Moving onto talking about metrics and teams. We often think its about having the best and brightest superstars. To counter that thinking, what percentage of time do you think the league MVPs for baseball and basketball belong to a winning championship team? If it was just having the BEST player (whatever that means), it should be pretty high, above 50% at least. Well in Baseball, its 23% in the 1930 to 2012/13 timeframe. And not recently. 1984 and 1988 depending on the league. Basketball does better. 37%. Smaller team sizes? Who knows the cause, but still under 50% roughly 1 in 3 or 4. It takes a team to win championships, so lets focus some effort on metrics that help form that team.
- #40 Hope solo. Start Goalkeeper for Team USA. Would be high on that superstar player list. Formidable.
- #41 But even if she has a perfect match or season. The best can be 0-0 draws. And if her defenders are lacking. She has a higher chance of not being perfect. And without good attackers, they don’t stand a chance of scoring to win games. It take a team with a variety of expertise to win. The best hope can do is help avoid a loss. This is where coaches and managers fit. They need to balance the teams capabilities to cover all needs. By the time you are dealing with professional athletes and development teams, basic skills are assumed. Its how they play together in concert that decided who wins and loses. A coach help see the big picture.
- #43 I have a simple tool to help visualize skill capabilities. Its – you guessed it – a spreadsheet. You enter a list of required skills, and it helps produce a simple paper survey sheet. The team members assess their ability to teach others on a skill, to perform that skill, or to be willing to learn that skill. It takes this data and aggregates it into a heatmap. [click]
- #44 Along the continuum from green is “safe” to red is “at risk” for each skillset based on how many teachers, do-ers, and novices you have available. Being red isn’t bad. It just means you better hope demand doesn’t rise for that skillset. If it does, train up a novice or two. [click] The spreadsheet gives coaching advice based on what it sees, helping you as a coach or manager triage risky skill gaps and single points of failure. By proactively telling the story of skill capability of your teams, you position them to be resilient with changing demand.
- #45 And then you too will win the World Championship. Team USA.
- #46 And your team will thank you by causing severe back injuries. Luis Enrique – Barcelona coach….
- #47 One goal of mine when coaching teams using metrics is helping them make smart trades between competing forces. Maximizing performance in all facets of a process is stupid and likely impossible without gaming. I’m far more impressed by teams who trade something they are super good at for an incremental improvement in another area they are struggling. As a coach, we can help teams assess and make these trades.
- #49 Larry Maccherone spearheaded some research whilst working at Rally with collaboration with CMU/SEI. The Software Development Performance Index. The SDPI framework includes a balanced set of outcome measures. These fall along the dimensions of Responsiveness, Quality, Productivity, Predictability. Each of these are opposing forces. Its unlikely any team will excel at all of them. Increasing productivity beyond team capability will likely cause a decrease in quality. Responding faster likewise, could mean corners are cut on quality with fewer tests or less testing. By making sure you track data trends in each area will help the team creeping net positive.
- #50 Here is one dashboard produces using the SDPI principles. Ii’’ talk about the quality dimension in a second, but the others are Productivity, in this case throughput, dark green for story work, pale green for defect work. You can see the team traded some story work to rapidly burn down some defect debt nearing the end of a major release. I’d have been happier as a coach if they made that balance earlier. Responsiveness is defect cycle time resolution. I want the team to have a clear picture of how long it may take to burn-down defect debt. In this case, the team managed to move from 10 days down to 3 days once they moved through the most difficult ones. A great indicator as a coach that the remaining defects are small in nature and effort. Predictability is a little more complex a measure. If you just use standard deviation of the productivity number, bigger teams would be disadvantaged. The quickest way to control for team size is by dividing the standard deviation by the mean, a measure called the co-efficient of variance. This of it as the standard deviation of standard deviation. Credit for this Visualization goes to Isaac Obezo.
- #51 Quality is always difficult. I like to consider some measure of ongoing delivery ability. Often though it comes down to escaped defect counts. To make this axis useful to the team in a coaching context, I forecast how long to zero defects given THIS teams cycle time average. During planning the team can quickly see “if we did nothing but defects we would need x days as a team.” A great discussion to help focus on technical debt removal before taking on more story work. Key here: Make it personal about the team. THEIR cycle time, THEIR defect count. THEIR team size.
- #52 Team to team comparison is a dangerous pursuit. But lets try and do it safely anyway! This dashboard helps team compare their trend with similar teams in a company. By looking for patterns in the trends across the 4 different SDPI categories, contextual coaching advice can be displayed. But the data is noisy, so we quickly found removing the line cleared up the mess. [click]
- #53 Now it clear to see how “MY” teams trend in orange compares to the rest of the company shown in grey. No axis values, just trend. Every value is normalized as best as possible to fairly compare apples versus apples. Teams should focus on the steepest adversely trending category. They should trade something from the steepest favorable trending category. Smart trades, based on comparison against teams in the same context. Sure, it could be misused, but the way the categories work against each other its not possible for any one team to have a stable favorable trend in all categories. Or if they can, I haven’t seen it.
- #57 My name is Troy Magennis, I’ve been in software for 25 years now, from QA through to VP Architecture and Development for companies like Travelocity and Lastminute.com. Most recently I formed my own company building tools and running training on software development forecasting and risk management solutions. Feel free to take notes, but the slides and examples are available to you online. And as a special benefit for joining us today, you can download the software used throughout this session for free. Bit.ly/agilesim will take you to the right site. I wrote a book about these topics, “Forecasting and Simulating Software Development Projects” and I’d like to make sure you all got a free PDF copy of this book also. Just download it from the same location.
- #59 I set out to prove this by setting myself a challenge to see what I could do with just completed and start dates. You can get the result for yourself by downloading the spreadsheet from bit.ly/Throughput with a capital T. Again, all these links will be tweeted at @t_magennis
- #74 1930 to 2012 = 82 years 19 nat league, 19 am league Last time 1988, 1984
- #75 And this is another direction I think we might be heading. Help the user feel the flow of work. This is a jump plot invented and mocked-up to show software process flow. The blue jumps across the top is work flowing forward, underneath, work flowing back. The height of each arc represents average cycle time, and thickness represents count. In one plot, you get to see the story of cycle time, throughput and status steps. Amazing piece of story telling for understanding a process and its flow. Source: https://public.tableau.com/views/JumpLineExamples/SDLCDetailedJP?%3Aembed=y&%3Adisplay_count=yes&%3AshowTabs=y&%3AshowVizHome=no Amazing work by Tom VanBuskirk and Chris DeMartini. See JumpPlot.com