Portugal
Problems of CMMI® Implementations
Recommendations to Avoid them
and Achieve High Maturity
Isabel Lopes Margarido
...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

agend...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

intro...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

intro...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

intro...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

typically organisations using CMMI improve performance
© 2013 CMMI Portugal ...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
introduction

method

problems

conclusions

© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes M...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

intro...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

quest...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

refer...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

acron...
© 2013 CMMI Portugal Conference Series – All Rights reserved.

Isabel Lopes Margarido, 18th of October 2013, Lisbon

image...
Upcoming SlideShare
Loading in …5
×

III Conferência CMMI Portugal, Presentation 5: Problems of CMMI® Implementations - Recommendations to Avoid them and Achieve High Maturity, Isabel Margarido, Faculdade de Engenharia da Universidade do Porto

670 views
444 views

Published on

Problems found in the implementation of CMMI, based on three case studies on organisations appraised at CMMI maturity level 5, a literature review and the analysis of the data of an SEI survey conducted to organisations being appraised for maturity levels 4 and 5. Proven recommendations on how to avoid those problems.

Published in: Travel
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
670
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

III Conferência CMMI Portugal, Presentation 5: Problems of CMMI® Implementations - Recommendations to Avoid them and Achieve High Maturity, Isabel Margarido, Faculdade de Engenharia da Universidade do Porto

  1. 1. Portugal Problems of CMMI® Implementations Recommendations to Avoid them and Achieve High Maturity Isabel Lopes Margarido PhD Student Researcher Faculdade de Engenharia da Universidade do Porto João Pascoal Faria FEUP/INESC Marco Vieira FCTUC Raul Moreira Vidal FEUP 2013-10-18, Lisboa
  2. 2. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon agenda introduction methodology problems and recommendations conclusions 2
  3. 3. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon introduction method problems conclusions 3
  4. 4. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon introduction method problems conclusions 4
  5. 5. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon introduction method problems conclusions objectives gain additional knowledge about CMMI 5
  6. 6. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon objectives understand problems and difficulties gain additional knowledge about CMMI 6
  7. 7. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon objectives learn solutions to avoid them understand problems and difficulties gain additional knowledge about CMMI 7
  8. 8. introduction method problems conclusions typically organisations using CMMI improve performance © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon motivation many programs failed in CMMI level 5 organisations SEI was concerned with high maturity 8
  9. 9. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon high maturity levels dependencies dependent on lower maturity levels maturity level (ML) 2 M&A   9
  10. 10. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon high maturity levels dependencies dependent on lower maturity levels maturity level (ML) 2 M&A building blocks for ML4: QPM, OPP (PPM and PPB)  10
  11. 11. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon high maturity levels dependencies dependent on lower maturity levels maturity level (ML) 2 M&A building blocks for ML4: QPM, OPP (PPM and PPB) knowledge base for quantitative continuous improvement at ML5 (CAR, OPM) 11
  12. 12. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon case study (published in QUATIC, Lopes Margarido et al., 2012) 3 organisations appraised at CMMI Dev ML5 – 2 organisations, 1 business unit real problems and difficulties from industry – documents, tools, interviews – how CMMI implementation was conducted and the processes were defined – what processes, metrics and tools were developed – how people were actually using them 12
  13. 13. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon analysis analysed an SEI report (McCurley and Goldenson, 2010) verified which problems were common to the case study, literature and SEI survey verified which of the recommendations were sustained by literature and SEI survey 13
  14. 14. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon problems (P) and recommendations (R) entry conditions P1: avoid underestimation R1: plan: maturing levels, analysing and understanding HML, building and maturing PPB and PPM P2: don’t begin the house by the roof R2: ML 2 and 3 need to mature P3: understand quantitative nature of level 4 R3: involve statistician R4: six sigma R5: review goals 14
  15. 15. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon process definition and implementation P4: copied processes R6: reflect organisation culture R7: involve experts and process users P5: multicultural environments R8: share processes, lessons learnt P6: impose processes R8 and R9: goals specific for business units related to organisation business goals R10: indicators at different report levels 15
  16. 16. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon process definition and implementation P7: dissemination problems R11: commitment from entire organisation R12: training contents, specialised training R13: coaching and monitoring P8: lack of institutionalisation R14: top management set goals for gradual institutionalisation, monitor and reward R13 and R15: give time for metrics and processes to mature 16
  17. 17. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon metrics definition P9: meaningless uncorrelated metrics P10: metrics definition (collect and analyse data) R16: use goal question metric or equivalent R17: unambiguous, repeatable, understandable R18: size metrics according with work product R19: interpret in context R20: variables normalisation 17
  18. 18. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon metrics definition P11: first data uncorrelated P12: metrics categorisation P13: baselines not applicable R21: several cycles R22: let PPM and PPB become stable R23: categorise data R24: aggregate normalised data 18
  19. 19. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon metrics usage P14: abusive elimination of outliers R25: quarantine R26: maintain data points that are unique but recurrent P15: not all projects are measurable R27: specific based measures normalised to get derived measures; R14 19
  20. 20. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon metrics usage P16: effort estimates R28: use expert judgment when needed R29: any related historical data R30: iterative planning, real time sampling P17: people behaviour R12, R13 and R31: personal data not used to evaluate people 20
  21. 21. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon tools setup P18: tools setup R32: give time for tool setup R33: do not use data collected when defects affect metrics P19: overhead P20: tools requirements R34: only collect necessary data R35: automated and unperceptive data collection R36: discipline people, change mentality 21
  22. 22. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon problems analysis 59% found in the case studies, literature and survey several problems shared between the studied organisations 37,5% found in literature were found in the case studies 53,3% found in the survey were found in the case studies  16,7% of those also found in literature 22
  23. 23. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon problems analysis 59% found in the case studies, literature and survey several problems shared between the studied organisations 37,5% found in literature were found in the case studies 53,3% found in the survey were found in the case studies  16,7% of those also found in literature “the percentage of problems shared in more than one organisation/source indicates that they can occur when implementing HML, so organisations should be aware of them.” (Lopes Margarido et al., 2013, submitted for publication to SQP) 23
  24. 24. introduction method problems conclusions © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon summary and future research wide variety of implementation methods variance of performance results SCAMPI: – evaluates a sample – does not evaluate performance future research – framework to evaluate quality of implementation of CMMI practices – definition published in PROFES 2012 (Lopes Margarido et al., 2012) 24
  25. 25. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon introduction method problems conclusions think about... 25
  26. 26. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon questions http://paginas.fe.up.pt/~pro09003 partially sponsored by 26
  27. 27. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon references • M. Campo. “Why Maturity Level 5?”, Crosstalk, January/February 2012. 15-18. • C. Hollenbach and D. Smith. (2002) A portrait of a CMMI level 4 effort Systems Engineering. 52-61. • P. Leeson, "Why the CMMI® does not work," presented at the SEPG Europe, Prague, Czech Republic, 2009. • R. Radice, "Statistical Process Control in Level 4 and Level 5 Software Organizations Worldwide," presented at the Software Technology Conference, 2000. • M. Schaeffer, "DoD Systems Engineering and CMMI," presented at the CMMI Technology Conference and User Group, 2004. 27
  28. 28. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon acronyms CMMI – Capability Maturity Model Integration M&A – Measurement and Analysis ML – Maturity Level OPM – Organisational Performance Management OPP – Organisational Process Performance P – Problem PPB – Process Performance Baselines PPM – Process Performance Models QPM – Quantitative Project Management R - Recommendation SEI – Software Engineering Institute SQP – Software Quality Professional 28
  29. 29. © 2013 CMMI Portugal Conference Series – All Rights reserved. Isabel Lopes Margarido, 18th of October 2013, Lisbon images http://api.ning.com/files/hpf*xOTebDs- F23o6FETZ3j*3sNiONWjfXjTJCzprPjU5bS1 WJoGgWBjMPIOiQkm3SbZ41ijncrJ4K2aT-6dM9QURwHK3led/Dissemination2.jpg -26-06-2010 http://blog.pmtech.com.br/wp-content/uploads/Square-Paradox.jpg – 29-04-2011 http://www.signsexpressshop.co.uk/prodpics/1103.gif – 29-04-2011 Benjamin Haas/Shutterstock, http://cynthiayildirim.posterous.com/how-can-we-measure-the-size-of-theuniverse – 29-04-2011 http://ryanstephensmarketing.com/blog/wp-content/uploads/2009/10/one_size_fits_all.JPG http://evolvingwe.com/wp-content/uploads/2010/11/image3.png – 29-04-2011 http://www.screenhog.com/sketch/LightbulbIdea.jpg – 21-04-2010 http://igraduatedwhatnow.files.wordpress.com/2009/11/thank_you_small.jpg – 02-05-2010 http://www.articulate.com/rapid-elearning/wp-content/uploads/2008/08/summary-objectives450.gif – adapted, 25-05-2011 http://www.braxtechconsulting.com/Portals/22771/images/cmmi_certificate.jpg.png - 10-04-2012 http://erikhatch.org/wp-content/uploads/2012/02/no_time_1_.jpg - 31-08-2012 http://blog.inktechnologies.com/wp-content/uploads/2012/06/Statistics.jpg - 31-08-2012 http://images.yourdictionary.com/images/definitions/lg/erase.jpg -31-08-2012 https://earlychildcare.files.wordpress.com/2009/09/child-misbehaving.jpg - 31-08-2012 http://www.thaiworldview.com/jpg/img024.jpg - 31-08-2012 http://image.shutterstock.com/display_pic_with_logo/5880/5880,1268065242,15/stock-photobackground-concept-wordcloud-illustration-of-scientific-method-research-glowing-light-48226942.jpg 02-09-2012 29

×