Real Agile example of how to deliver a project on-time with substantial adversity

1. Jim O’Keeffe, Jason Vogel
and Josh Matcuk
How to Play the Agile
Hand You’re Dealt

2. Who We Are

3. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
2
The Project Roller Coaster
• Problem: Projects often face adversity in:
• Scope
• Resourcing
• Deadlines
• Defects or Fire drills
• Solution: Better equip PM/Architects with
methods to effectively handle project adversity

4. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
3
The Hand We Were Dealt: People
Start of
2014:
8
people
August
2014:
27
people

5. The Hand We Were Dealt: Project
Scope

6. What Would You Do?
• Break into groups and
discuss:
• What would you do in this
situation?
• What are some practices
you have used to face
project adversity?
• Have at least 1 thought
to share at end of 2 mins

7. The next session will start in
5958575655545352515049484746454443424140393837363534333231302928272625242322212019181716151413121110090807060504030201000100

8. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
7
Presentation Objectives
Real Cerner example of how we utilized Agile
principles to best play the hand we were dealt
Demonstration of an effective Iteration 0
Explanation of Release Forecaster
Retrospective on Lessons Learned

9. Why You Should Care

10. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
9
Overview
1. Methods:
 Iteration 0
 Self-Organization
 Early and Continuous Integration
 Release Forecasting
2. Problem Solving:
 Managing Scope
 Handling the Fire Alarm
3. Conclusion:
 Retrospective

11. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
10
Iteration 0
Goals of Iteration 0:
1. Agree upon Initial Scope
2. Investigate Design Uncertainties
3. Document Design Notes
4. Create Sequence Diagrams
5. Finalize Initial Requirements with SD
6. Create JIRA Backlog and Story Point

12. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
11
Iteration 0 Best Practices
Get a room
Involve all
Stakeholders
Create JIRA
Backlog and
Story Point
Utilize uCern

13. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
12
Self-Organization
• Utilize Trello
• Scrum Team Set-Up
• Break planning up by scrum group
• Weekly Scrum of Scrums
• Shared Documents

14. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
13
Iteration Planning
For Focus: Plan with Scrum Group
For Communication: Have a Sprint Overview
For Clarity: Become a JIRA Filter Expert
For Accountability: Utilize Checkpoints

15. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
14
Early and Continuous Integration
Include App in Iteration 0
Encourage App team to Services scrum
Utilize Mock Services early
Test heavily in Dev and Staging
Demo Together
App/Services for outstanding questions sync

16. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
15
Why: Release Forecasting
First Iteration Metrics
• Number of Implementation Engineers
• Points Completed (Velocity)
• Average Points per Engineer

17. Iteration Metrics
Sprint Points Committed Points Completed Points Carryover Number of Engineers Points per Engineer Commitment Ratio
8/26-9/3 43 28 15 14.0 2.0 65%
9/9-9/23 96 71 25 14.0 5.1 74%
9/23-10/7 110 67 43 14.0 4.8 61%
10/7-10/21 117 108 9 16.0 6.8 92%
10/21-11/4 152 139 13 17.0 8.2 91%
11/5-11/18 119 71 48 14.0 5.1 60%
11/18-12/2 96 52 44 13.0 4.0 54%
12/2-12/16 83 109 -26 10.0 10.9 131%
STDEV +/- 29.6 33.8 15.3 1.8 14%
Low- 68.3 40.6 12.3 3.7 60%
Average 97.9 74.4 27.7 5.5 74%
High+ 127.5 108.2 43.0 7.3 88%

18. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
17
When and What: Release Forecasting
• When: End of Iteration 0
• What: Forecasted Release Date based on:
• Project backlog
• Historical Completion Rates
• Provide Forecast on:
• Initial Scope
• 20% Scope Increase
• 30% Scope Increase
• 50% Scope Increase

19. Who has kids?
My kid wants to ride
100 rides
Average kid rides 5
rides per hour
20 hours of it’s a
small world

20. Release Forecasting

21. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
20
Project Analysis
• What do these mean?
• Cumulative Points in a Release
• Net Points in Backlog
• Change in Forecasted Date (In Days)

22. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
21
Project Analysis
-100
0
100
200
300
400
500
600
9-Sep 16-Sep 23-Sep 30-Sep 7-Oct 14-Oct 21-Oct 28-Oct 4-Nov 11-Nov 18-Nov 25-Nov 2-Dec 9-Dec 16-Dec
Net Points in Backlog
Cumulative Points
Change in Date
Points Completed

23. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
22
Scope Measure
0
100
200
300
400
500
600
9-Sep 16-Sep 23-Sep 30-Sep 7-Oct 14-Oct 21-Oct 28-Oct 4-Nov 11-Nov 18-Nov 25-Nov 2-Dec 9-Dec 16-Dec
Cumulative Points

24. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
23
Release Burn Down Chart
0
50
100
150
200
250
300
350
9-Sep 16-Sep 23-Sep 30-Sep 7-Oct 14-Oct 21-Oct 28-Oct 4-Nov 11-Nov 18-Nov 25-Nov 2-Dec 9-Dec 16-Dec
Net Points in Backlog

25. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
24
Velocity
0
20
40
60
80
100
120
140
160
9-Sep 16-Sep 23-Sep 30-Sep 7-Oct 14-Oct 21-Oct 28-Oct 4-Nov 11-Nov 18-Nov 25-Nov 2-Dec 9-Dec 16-Dec
Points Completed

26. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
25
Dear Executive, We are Behind Schedule
-25
-20
-15
-10
-5
0
5
10
9-Sep 16-Sep 23-Sep 30-Sep 7-Oct 14-Oct 21-Oct 28-Oct 4-Nov 11-Nov 18-Nov 25-Nov 2-Dec 9-Dec 16-Dec
Release Forecaster 9-Sep 23-Sep 7-Oct 21-Oct 5-Nov 18-Nov 2-Dec 16-Dec
Forecasted End Date Low 11/27/2014 12/18/2014 12/8/2014 11/30/2014 12/1/2014 12/7/2014 12/13/2014 12/16/2014
Forecasted End Date Average 11/11/2014 12/1/2014 11/26/2014 11/22/2014 11/26/2014 12/3/2014 12/11/2014 12/16/2014
Forecasted End Date High 11/1/2014 11/19/2014 11/17/2014 11/16/2014 11/22/2014 12/1/2014 12/9/2014 12/16/2014
Change In Forecasted Date (In Days) 0 -19.60 5.17 3.89 -4.06 -7.37 -7.54 -4.74
Net Change From Original Forecasted Date -19.6 -14.43 -10.54 -14.60 -21.97 -29.51 -34.25

27. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
26
All Together Again
-100
0
100
200
300
400
500
600
9-Sep 16-Sep 23-Sep 30-Sep 7-Oct 14-Oct 21-Oct 28-Oct 4-Nov 11-Nov 18-Nov 25-Nov 2-Dec 9-Dec 16-Dec
Net Points in Backlog
Cumulative Points
Change in Date
Points Completed

28. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
27
Overview
1. Methods:
 Iteration 0
 Self-Organization
 Early and Continuous Integration
 Release Forecasting
2. Problem Solving:
 Managing Scope
 Handling the Fire Alarm
3. Conclusion:
 Retrospective

29. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
28
Problem 1: Managing Scope
-50
0
50
100
150
200
250
300
350
400
450
9-Sep
10-Sep
11-Sep
12-Sep
13-Sep
14-Sep
15-Sep
16-Sep
17-Sep
18-Sep
19-Sep
20-Sep
21-Sep
22-Sep
23-Sep
24-Sep
25-Sep
26-Sep
27-Sep
28-Sep
29-Sep
30-Sep
1-Oct
2-Oct
3-Oct
4-Oct
5-Oct
6-Oct
7-Oct
Net Points in Backlog
Cumulative Points
Change in Date
Points Completed

30. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
29
Problem 1: Managing Scope
Release Forecaster 9-Sep 23-Sep 7-Oct
Cumulative Points in Release 296 393 371
Net Points in Backlog 296 322 233
Total Scope Increase 33% 25%
Net Gain 0 -26 89
Points Completed 0 71 67
Points Added 0 97 44
Points Removed (decrease scope) 66
Forecasted End Date Low 11/27/2014 12/18/2014 12/8/2014
Forecasted End Date Average 11/11/2014 12/1/2014 11/26/2014
Forecasted End Date High 11/1/2014 11/19/2014 11/17/2014
Change In Forecasted Date (In Days) 0 -19.60 5.17
Net Change From Original Forecasted Date -19.6 -14.43

31. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
30
Problem 2: Handling the Fire Alarm
-100
0
100
200
300
400
500
600
9-Sep
11-Sep
13-Sep
15-Sep
17-Sep
19-Sep
21-Sep
23-Sep
25-Sep
27-Sep
29-Sep
1-Oct
3-Oct
5-Oct
7-Oct
9-Oct
11-Oct
13-Oct
15-Oct
17-Oct
19-Oct
21-Oct
23-Oct
25-Oct
27-Oct
29-Oct
31-Oct
2-Nov
4-Nov
Net Points in Backlog
Cumulative Points
Change in Date
Points Completed

32. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
31
Problem 2: Handling the Fire Alarm
Release Forecaster 9-Sep 23-Sep 7-Oct 21-Oct 5-Nov
Cumulative Points in Release 296 393 371 419 507
Net Points in Backlog 296 322 233 173 122
Total Scope Increase 33% 25% 42% 71%
Net Gain 0 -26 89 60 51
Points Completed 0 71 67 108 139
Points Added 0 97 44 48 88
Points Removed (decrease scope) 66
Forecasted End Date Low 11/27/2014 12/18/2014 12/8/2014 11/30/2014 12/1/2014
Forecasted End Date Average 11/11/2014 12/1/2014 11/26/2014 11/22/2014 11/26/2014
Forecasted End Date High 11/1/2014 11/19/2014 11/17/2014 11/16/2014 11/22/2014
Change In Forecasted Date (In Days) 0 -19.60 5.17 3.89 -4.06
Net Change From Original Forecasted Date -19.6 -14.43 -10.54 -14.60

33. We Passed
• 39 releases within 8 weekly prod deployments
• Went Live at 3 beta-test pilot clients on 1/5/15
• Live with 14+ other clients on 1/21/15

34. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
33
Retrospective: The Good
• Things that Worked Well:
• Breaking Scrum teams into functional
specific
• Subject Matter Expert involvement
• Less overtime for engineers than 2013

35. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
34
Retrospective: Lessons Learned
• Needs Improvement:
• Despite best effort, still big bang release/deploy
• Iteration 0: 3 weeks instead of 2 weeks
• Needed to involve Doc team earlier
• Ideas for next time:
• Better design/communication on Services internal
API
• Complete a piece all the way before starting on the
next one

36. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
35
For Your Next Roller Coaster
Iteration 0
Release
Forecasting
Scope
Management
Problem
Solving
Care

37. 1310231341_Standard - Cerner PPT template_v2.2 updated: 0115 © Cerner Corporation. All rights reserved.
This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner
36
Learn More
• uCern Space: Results to Endorse
• Trello
• Release Google Doc
• Release Forecaster Excel Template: To Be
Posted on uCern Meeting Post
• Monte-Carlo Analysis
• Blog Post: Cadence Commitment and Delivery
• Blog Post: The Team, The Team, The Team!

38. Feedback and Questions