Your SlideShare is downloading. ×
A successful improvement process with  measurable results
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

A successful improvement process with measurable results

472
views

Published on

how to improve R&D process - presented during TACT tesing metrics convention

how to improve R&D process - presented during TACT tesing metrics convention

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
472
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. A successful improvement process with measurable results Yigal Cohen (Ex) R&D Manager Deputy - NDS March 2010 – ‫אדר תש"ע‬
  • 2. Chosen by the top TV operators – Global company – Complex – industry leading system – Changing matrix
  • 3. Improvement plan goals • What ? – Increase business results by improving project management, process & methodologies • How? – Provide a set of principles, tools, metrics, and improvement framework – Push Lines to lead and manage their actual improvement plan – Accompany and mentor real projects 3
  • 4. Continuous Improvement Framework 4 3 2 1 0 Criteria R&D Charter Toolkit 14 Ch.14 Every Build will have the following 5: NA artifacts:1. Build in Release management system2. Should have both production as well as Debug enabled image3. Release note with CQ list and component details 15 Ch.12 a Defects and issues are tracked 2 : somewhat 16 Ch.12 b Defects and issues undergo root-cause 2 : somewhat analysis 17 Communication plan is defined 3 : well into it 18 Ch.13 Artefacts are held within a configuration or 5: NA document management system 19 Ch.15 Project has end of milestone reviews 2 : somewhat 20 Communication about the completion of 2 : somewhat milestone Self Assessments Self Assessment Analysis Improvement Plan 4
  • 5. The Community 8 WW_QA + 440 35 SQAs 2200 R&D Management AI Holders Users 5
  • 6. Local toolkits R&D 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Division 2 4 7 10 11 12 14 Line 2 6 11 13 14 Project 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Every division and line is authorized and encouraged to prepare its own procedures for part or all of the R&D Charter. The most specific procedures will be used for each project. 6
  • 7. SDP Coding • Time & effort Estimation • Re-factpriong • Scheduling • Complexity analysis • WBS • Static Analysis • Design/Dev/Test done iteratively • Progress Monitoring Configuration • Burn Down • Branching Policy • TOC • Building • Risk Management • Automated Unit level tests Requirements Testing • Tests cover requirements • Common Logging • Automated regression tests Design • TPI • Defensive Design & coding • UML Integration • Pattern Design • Continuous Integration • Release • Release Criteria 7
  • 8. • Does it help ? • To Whom ? 8
  • 9. One line - improvement Done Successfully activities Infrastructure • Coding standards written for New Generation Smart Card, SimCAM QC Tools, and Non-Smart Card Products • Migration to SVN source management • Root cause analysis accepted part of standard quality procedure • Smart Card development utilizing Visual Studio environment and associated tools (pilot) Actual improvements • New Smartcard code base developed using: – Unit test on 80% of code – Continuous Build on 100% of code – ->New Smartcard ROM integration & testing in only 4 weeks! • SVN configuration management (Most of the line) • Root Cause Analysis being performed on all defects found in I&T and at the customer site • Average R&D Charter compliance of 89% 9
  • 10. Slide 9 AV4 Not sure where this fits in. Dr Alison Vincent, 02/07/2008
  • 11. Impacts on the development process (Saves time, better quality and increase efficiency) • Integration Plan – Planning the integration saves a lot of time by making pre- integrations and preparing tools in advance (Saves time) • Test Strategy – Testing process was improved especially in multi-site projects (Saves time and improve quality) • Architects started to increase their involvement along the development process (Quality and efficiency) • Spreading of Coding Standards, Build, Root Cause Analysis and Lessons learned across R&D with Common global language and transparency (All) • Contributed to increase profit by 25% in a year! 10
  • 12. Customer defects reported in CQ 16000 14000 12000 10000 8000 6000 4000 2000 0 6 6 6 6 7 7 7 7 8 8 8 8 9 '0 '0 '0 '0 '0 '0 '0 '0 '0 '0 '0 '0 '0 Customer Defects 1 2 3 4 1 2 3 4 1 2 3 4 1 Q Q Q Q Q Q Q Q Q Q Q Q Q 900 800 700 600 500 400 300 200 100 0 Qtr 1 Qtr 2 Qtr 3 Qtr 4 Qtr 1 Qtr 2 Qtr 3 Qtr 4 Qtr 1 Qtr 2 Qtr 3 Qtr 4 Qtr 1 '06 '06 '06 '06 '07 '07 '07 '07 '08 '08 '08 '08 '09 11
  • 13. R&D Charter NDS R&D Charter is the guideline for the process 1. For Planning, Tracking & Management projects will have: 1. Software Development Plan 2. Resource Allocation plan 3. Re-use strategy 4. Test Strategy and Integration Plan 2. Development phases 1. Requirements will be Documented/Reviewed/Managed 2. Designs will be documented and Reviewed 3. Coding standard 4. Code review 5. Automatic tools and ATP for every API 3. Infrastructure 1. Artefacts will be held within a configuration or document management system 2. Defects and Issues will be tracked and will undergo root cause analysis 3. Projects will have end of project reviews 4. Builds will be automated and repeatable 5. Released builds will be controlled 12
  • 14. What’s in the toolkit ? Templates Other Refs Checklists Guidelines Training Examples Support Tools Contacts ! 13
  • 15. R&D Charter Toolkit 14
  • 16. An Example Criteria for Self Assessment 4: TS updated throughout project 3: TS compliant to the checklist communicated and followed 2: TS reviewed and approved 1: Test Strategy is written 0: No Test Strategy 15 15
  • 17. Self Assessments - Example Ch.01 Projects will have a Software / 2: Documented or presented at 2 http://shareatnds.in.nds.com/sites/Intranet/Op Hardware Development Kick Off meeting enTV/master_ui/Shared%20Documents Plan that meets a defined /project_plan.aspx schedule according to the customer's needs Ch.02a Projects will have a Test 2: Approved Test Strategy 2 http://doogle.uk.nds.com/worksitemp/dispatch?obj strategy (SQSC-style) reviewed ectId=%21V3%21UKDMPRODUCTION1% and approved by all 21C%21PJ%2441860%21&returnUrl=1211 stakeholders 45625069359336&operation=zoom&dataso urce=project Ch.02b Projects will have an Integration 0: No Integration Plan 0 Include link to plan plan Ch.03 Projects will have clearly defined 4: All roles actively performed 4 http://shareatnds.in.nds.com/sites/Intranet/Op roles and responsibilities enTV/master_ui/default.aspx Ch.04 Projects will agree on the 1: Initial discussion on re-use 1 strategy for re-use at of previous assetts Project Kick-Off (requirements, design, test and code) Ch.05 Projects will follow the R&D 2: resource plan done 2 http://shareatnds.in.nds.com/sites/Intranet/Op Resource allocation plan (available) but not enTV/master_ui/Shared%20Documents reviewed against /project_plan.aspx?RootFolder=%2fsite delivery requirements s%2fIntranet%2fOpenTV%2fmaster%5f ui%2fProject%20Plan%2fWorkPlan&Fo lderCTID=&View=%7b69BD6617%2d6 CCB%2d46E0%2dA5CE%2dFCE21C8 85064%7d Ch.06 Requirements will be 2: requirements and change 2 http://doogle.uk.nds.com/worksitemp/dispatch?dat documented / managed / requests documented asource=documentFolder&operation=zoom reviewed (Document, ReqPro, &objectId=!V3!UKDMPRODUCTION1!C!FD ClearQuest). $44881!&returnUrl=12114569796644311 Ch.07 Designs will be documented & 3: design reviews done with 3 http://doogle.uk.nds.com/worksitemp/dispatch reviewed tangible outcomes, key ?datasource=documentFolder&operatio design choices n=zoom&objectId=!V3!UKDMPRODUC documented and major TION1!C!FD$44875!&returnUrl=12114 issues fixed.Traceability and effectiveness 5717958875383 16 checked in review
  • 18. Sample Charter Charter Item P1 Analysis P2 P3 P4 Average Item # Ch.01 Projects will have a Software / Hardware Development Plan that meets a defined schedule according to the customer's needs 1 2 2 2 1.8 Ch.02a Projects will have a Test strategy 3 1 2 0 1.5 Ch.02b Projects will have an Integration plan 4 3 3 3 3.3 Ch.03 Projects will have clearly defined roles and responsibilities 1 2 4 4 2.8 Ch.04 Projects will agree on the strategy for re-use at Project Kick-Off (requirements, design, test and code) 1 4 2 1 2.0 Ch.05 Projects will follow the R&D Resource allocation plan 4 3 4 1 3.0 Ch.06 Requirements will be documented / managed / reviewed 2 2 2 3 2.3 Ch.07 Designs will be documented & reviewed 3 1 4 2 2.5 Ch.08 Code will follow a coding standard 2 3 3 3 2.8 Ch.09 Code will be reviewed 2 2 1 1 1.5 Ch.10 Builds will be automated and repeatable 3 4 3 3 3.3 Ch.11 APIs will have a tool by which they can be tested 1 1 1 1 1.0 Ch.12a Defects and Issues will be tracked 3 3 3 3 3.0 Ch.12b Defects will undergo root cause analysis 2 2 2 1 1.8 Ch.13 Artefacts will be held within a configuration or a document management system 4 4 4 4 4.0 Ch.14 Released builds will be controlled 2 3 3 4 3.0 Ch.15 Projects will have end of project reviews 3 2 3 0 2.0 COLUMN AVERAGES 2.4 2.5 2.7 2.1 2.4 17
  • 19. Improvement Plan – 3 Examples • Faster and more reliable builder – Learn Hudson and build infrastructure. 6 weeks. Danny – Run a pilot. Next 8 weeks. Mike. – Evaluate Results. Next 2 weeks. Rachel • Deploy Scrum in all projects – Use Scrum in projects X, Y, Z. Q1. Danny, Shani, Rafi 1200 action items from IL/FR/IN/UK/USWeek 3 – 1 Day seminar for GL, including our lesson leaned. – Bi-Weekly Scrum Masters meeting. Moshe. 75% Write a Departemnet Scrum guide. Moshe. – of the were done – Run a full cycle of iteration 1 on a PC for evaluation All lines were reviewed by R&D Manager • Increase efficiency – AI: Inquire Why after using Agile the number of bug was not reduced. 6 weeks. Danny 18
  • 20. R&D Process Improvement Other Root Cause Lessons needs Analysis Learned 14 Ch.14 Every Build will have the following 5: NA artifacts:1. Build in Release management system2. Should have both production as well as Debug enabled image3. Release note QC Lead 3 4 4 4 4 4 4 4 3 3 3.70 with CQ list and component details Project has a software Development plan that defines the schedule according to the customer needs 3 2 3 3 3 4 4 4 4 4 3.40 Iteration plan exists 3 3 3 3 3 1 1 1 1 1 2.00 Component Development Plan exists along with component QC plan 2 2 3 2 1 1 1 1 1 1 1.50 15 Ch.12 a Defects and issues are tracked 2 : somewhat Configuration Management Plan exists 1 1 1 1 3 1 1 1 1 1 1.20 16 Ch.12 b Defects and issues undergo root-cause 2 : somewhat analysis Project has a Resource allocation plan 3 2 3 3 3 2 2 1 1 2 2.20 17 Communication plan is defined 3 : well into it Project has an Integration Plan 3 2 1 4 3 1 1 2 1 1 1.90 Project has a test strategy 3 2 2 3 3 3 3 1 3 3 2.60 18 Ch.13 Artefacts are held within a configuration or 5: NA Project has all the required documents document management system managed and reviewed 2 2 1 3 3 4 3 3 3 2.67 19 Ch.15 Project has end of milestone reviews 2 : somewhat Test cases are prepared, Mapped to 20 Communication about the completion of 2 : somewhat Requirements, reviewed and managed 3 3 2 2 3 3 4 4 3 3 3.00 milestone APIs have a tool by which they can be tested 2 2 2 2 1 1 1 4 1 1 1.70 Code follows a coding standard 3 1 3 2 2 3 3 1 3 3 2.40 code is reviewed 4 2 4 2 3 3 3 3 3 3 3.00 Self Assessments Self Assessment Analysis Improvement Plan 19
  • 21. Thank you ! WW_QA Working Group 20
  • 22. Metrics • Development progress – Defects trends – Code Coverage by tests – Test Progress – Requirements Coverage by tests – Team velocity, Gantt Progress • Quality prediction – Counting Escaped Bugs by phases 21
  • 23. Code Coverage Defect trends Test Progress Static Analysis 22