You say , I sayStories behind establishing Quality Benchmarks© 2011 Capgemini. All rights reserved. 1Colin CherryDirector,...
ContentThe Background StoryThe Story about National & Corporate CulturesThe Story about meeting the PlayersThe Story about...
1. A finely balanced project involving a Swedish Transport Authority & amajor Australian Software provider in a strained r...
The Story about National & Corporate Cultures© 2011 Capgemini. All rights reserved. 4
1. Meet the Vendor, the Client, the Consultants & Identify the Key Players2. Sort through the (last 7 years) information –...
The Story about Gathering the Evidence© 2011 Capgemini. All rights reserved. 6July 15Compare Vendor &Customer Metrics
The Story about Gathering the Evidence© 2011 Capgemini. All rights reserved. 7HPQC Structure
The Story about Analysing the Risk© 2011 Capgemini. All rights reserved. 8July 29Software Version 3.2.3
The Story about Root Cause Analysis© 2011 Capgemini. All rights reserved. 9
The Story about Root Cause Analysis© 2011 Capgemini. All rights reserved. 10TriageProcessInstallProcessVariance found &con...
The Story about Root Cause Analysis© 2011 Capgemini. All rights reserved. 11Trend: More Re-test Failures as we close in on...
The Story about a new Belief System© 2011 Capgemini. All rights reserved. 12Probability6 Frequent The risk is likely to oc...
The Story about achieving the Goal© 2011 Capgemini. All rights reserved. 13CodeFreezeProduction TodayRelease 3.2.3.1Produc...
The Story about achieving the Goal© 2011 Capgemini. All rights reserved. 14
The Story about Changed Perceptions© 2011 Capgemini. All rights reserved. 15We have been LIVE for almost 1 week, but are s...
The Story about a Changed Reality© 2011 Capgemini. All rights reserved. 161. We realised that the Vendor & the Client were...
The Story about Thanking my (new) Friends© 2011 Capgemini. All rights reserved. 171. The Consultants:• Magnus Wikholm, Jur...
Thank You…© 2011 Capgemini. All rights reserved. 18Colin CherryDirectorTesting ServicesCapgemini Australia Pty LtdLevel 2,...
Upcoming SlideShare
Loading in...5
×

'You Say "Kvalitet", I Say Quality' by Colin Cherry

142

Published on

At the heart of the matter was one word – QUALITY (or Kvalitet in Sweden). “Your software isn’t good enough” was the assertion from the customer. “You’re being unreasonable” was the retort from the vendor. “You are our last chance” was the message we received from the customer’s CEO. “You”, being a small specialist (SWOT) team comprising, a Senior Project Manager, a Systems Architect, a Technical Project Lead, a Senior Business Analyst and a Senior Testing Consultant. This is not an unfamiliar scenario for those of us who have seen the software landscape change from bespoke to prêt-à-porter solutions during the past 15 to 20 years. It is also not an unfamiliar outcome for lawyers to get rich in these situations and for the customer to lose credibility in the marketplace.



In May 2009 I was asked to assist with a Transport Ticketing project that had been struggling to Go Live for the previous 2 years (the overall project lifespan was 7 years). The solution was 98% ready - just 360 requirements of the original 16,000 required sign-off. This had been the situation for over 12 months, with no agreed end in sight. How could they be so close and yet so far away? How we achieved (what appeared impossible just seven short months earlier) is the story I will tell at EuroSTAR 2011. How we turned “your software isn’t good enough” into “we’re going live next Saturday” will never be a Nordic legend, but it will be remembered as the project that saved a software vendor from bankruptcy and another major city from a multi-million dollar failure.



At the centre of this story is how we re-focused everyone on QUALITY/KVALITET. What’s essential, what’s negotiable, what’s unacceptable? These questions were central to our success. Our team didn’t identify any new requirements or write any new software. We just analysed the available data, spoke to the right people, re-worked the Test Strategy and agreed on a new GO LIVE strategy with an agreed date – and then MADE IT HAPPEN.



An interesting Footnote from this Case Study is that the Customer hired/payed us (and expected us to deliver) – so we got little in the way of recognition. The vendor was over the moon, as we had saved them from an expensive lawsuit and (almost certain) bankruptcy. The vendor still calls us. The customer experienced NO significant outages – KVALITET!!

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
142
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • ONLY the current Production Code Base is used for SATONLY functionality relating to the current Implementation Step scope will be used as a basis for SAT If a Bug is detected using an earlier (or later) version of software than is currently in Production, it must be proven to exist within the Production version also, otherwise it will remain as an ObservationThe client will ONLY advise the Vendor of Bugs that must be fixed for the current Implementation Step; these Bugs will be categorised as either Critical(SAT is suspended due to this specific failure) or Major (unacceptable for customers or internal business operations once the )Bugs categorised as Minor will result in a (short to medium term) workaround within the client & will ONLY be sent to the Vendor if no Critical or Major defects are outstanding; otherwise they will be sent to the Vendor after the Code Freeze has been implementedBugs categorised as Trivial will be sent to the Vendor after the Code Freeze has been implementedAll Bugs begin life as Observations & are only categorised as Bugs once they have been replicated, prioritised & assessed via the Daily Defect Triage MeetingAll Bugs must be associated directly with a Test CaseAll Critical Bugs require a Patch to be scheduled within 24 hoursAll Major Bugs require a Patch to be scheduled within 3 working days (unless otherwise agreed by the client)
  • Current Production Code Base used as basis for SATONLY functionality relating to the current Implementation Step scope will be used as a basis for SAT If a Bug is detected using an earlier (or later) version of software than is currently in Production, it must be proven to exist within the Production version also, otherwise it will remain as an ObservationIf a Bug is identified by the Vendor, on a version of software other than that currently in use by the client, it will only be of interest to the client if it is proven to exist in the Code Base currently in use by SAT The client will ONLY advise the Vendor of Bugs that must be fixed for the current Implementation Step; these Bugs will be categorised as either Critical(SAT is suspended due to this specific failure) or Major (unacceptable for customers or internal business operations once the )Bugs categorised as Minor will result in a (short to medium term) workaround within the client & will ONLY be sent to the Vendor if no Critical or Major defects are outstanding; otherwise they will be sent to the Vendor after the Code Freeze has been implementedBugs categorised as Trivial will be sent to the Vendor after the Code Freeze has been implementedAll Bugs begin life as Observations & are only categorised as Bugs once they have been replicated, prioritised & assessed via the Daily Defect Triage MeetingAll Bugs must be associated directly with a SAT Test CaseAll Patches will be installed in the Acceptance Environment firstAll Patches rejected by the client will be removed from the Acceptance Environment before the next business day in StockholmAll rejected Patches will be escalated to the client’s Program Management team due to the delay to SAT & need for re-workAll Patches will be accumulated by the Vendor to create a Release that will be delivered to the client 2 weeks prior to the Code FreezeThe Vendor will conduct a Regression Test on the Accumulated Release prior to making it available to the client; The client will then conduct a Regression Test as the final SAT activity prior to Approving the Release for Production
  • 'You Say "Kvalitet", I Say Quality' by Colin Cherry

    1. 1. You say , I sayStories behind establishing Quality Benchmarks© 2011 Capgemini. All rights reserved. 1Colin CherryDirector, Testing ServicesCapgemini Australia
    2. 2. ContentThe Background StoryThe Story about National & Corporate CulturesThe Story about meeting the PlayersThe Story about Gathering the EvidenceThe Story about Analysing the RiskThe Story about Root Cause AnalysisThe Story about a new Belief SystemThe Story about achieving the GoalThe Story about Changed PerceptionsThe Story about a Changed RealityThe Story about thanking my (new) friends© 2011 Capgemini. All rights reserved. 2
    3. 3. 1. A finely balanced project involving a Swedish Transport Authority & amajor Australian Software provider in a strained relationship2. A Steering Committee running out of patience & a CEO who wantedone last chance to implement a new SMART transport ticketing system3. A press highlighting every new delay4. A project with more than 16,000 (fully documented) requirements thathad been running for over 7 years5. A solution that had almost gone live twice before6. A team of 5 (external) Senior Consultants from a leading SystemsIntegrator, given 6 months to get the ticketing solution LIVE7. A phone call from Sydney, a flight to Perth (followed by a slightly longerone to Stockholm)The Background Story© 2011 Capgemini. All rights reserved. 3
    4. 4. The Story about National & Corporate Cultures© 2011 Capgemini. All rights reserved. 4
    5. 5. 1. Meet the Vendor, the Client, the Consultants & Identify the Key Players2. Sort through the (last 7 years) information – what’s important & what’s not3. Perform a SWAT followed by a Root Cause Analysis4. Assess the gathered information5. Develop a Plan of AttackThe Story about meeting the Players© 2011 Capgemini. All rights reserved. 5
    6. 6. The Story about Gathering the Evidence© 2011 Capgemini. All rights reserved. 6July 15Compare Vendor &Customer Metrics
    7. 7. The Story about Gathering the Evidence© 2011 Capgemini. All rights reserved. 7HPQC Structure
    8. 8. The Story about Analysing the Risk© 2011 Capgemini. All rights reserved. 8July 29Software Version 3.2.3
    9. 9. The Story about Root Cause Analysis© 2011 Capgemini. All rights reserved. 9
    10. 10. The Story about Root Cause Analysis© 2011 Capgemini. All rights reserved. 10TriageProcessInstallProcessVariance found &confirmed by the clientVariance logged in HPQC;Type =Observation, Status =NewClient Vendor ClientHPQC automaticallygenerates EMAIL to VendorDEV MGR (with enoughdetail for Vendor assessment& fix)Vendor updates HPQCwith comments to accept orreject Observation; Status= Rejected or OpenIf client disagrees withrejection, Type = BugStatus = OpenIf Vendor has (existing) Bugwith samesymptoms, Vendor addscomments to HPQC BugrecordRejectedRejection overturnedAcceptedVendor Internal Bug /Fix ManagementProcessVendor fix testedOK, Status = VendorRetest Successful +HPQC generates EMAIL(PC) to client TestManagerSL & Vendor conductdaily Bug Statusreviews & updateHPQC accordinglyXX retest; if OK, Status =XX Retest Successful;else Status = RetestFailed (HPQC generatesEMAIL PC to Vendor DEVMGR)RetestFailedTriage Observation; assignCritical & Major Bugs toVendor DEV MGR; Minor +Trivial Bugs + Observationsstay at SLCritical MajorMinor & Trivial Bugsmonitored internally untilreleased to Vendor.Observations “watched”Minor&TrivialBugorObservationClient sets Status toClosedRejectionConfirmed
    11. 11. The Story about Root Cause Analysis© 2011 Capgemini. All rights reserved. 11Trend: More Re-test Failures as we close in on LIVE Date
    12. 12. The Story about a new Belief System© 2011 Capgemini. All rights reserved. 12Probability6 Frequent The risk is likely to occur frequently. The riskwill be experienced hourly or more often.5 Probable The risk will occur several times. The risk isexpected to occur at least once a week.4 Occasional The risk is likely to occur several times. Therisk is likely to occur once a month.3 Remote The risk is likely to occur sometime in thelifetime of the product; maybe once per 10years.2 Improbable The risk is unlikely to occur but is possible;maybe once per 100 years.1 Incredible The risk is extremely unlikely to occur; maybeonce in a 1,000 years.Consequence4 Significant Immobilizing failure that preventsa service being provided3 Major A service failure that must berectified to ensure a significantservice outage does not occur2 Minor A service failure that reduces theeffectiveness of a service1 Insignificant A failure that does not impact theprovision of a service
    13. 13. The Story about achieving the Goal© 2011 Capgemini. All rights reserved. 13CodeFreezeProduction TodayRelease 3.2.3.1Production Nov 15Release “3.2.3.X”STEP ZEROProduction Sept 1Release 3.2.3.2Production Sept 15Release 3.2.3.3Acceptance Sept 1Release 3.2.3.2+ PatchesAcceptance Sept 15Release “3.2.3.X-1”Acceptance Oct 1Release “3.2.3.X”Acceptance TodayRelease 3.2.3.2Acceptance Nov 15Release 3.2.3.XFinalSATRegressionTestConsolidate Patchesinto Single Release
    14. 14. The Story about achieving the Goal© 2011 Capgemini. All rights reserved. 14
    15. 15. The Story about Changed Perceptions© 2011 Capgemini. All rights reserved. 15We have been LIVE for almost 1 week, but are still finding new Bugs
    16. 16. The Story about a Changed Reality© 2011 Capgemini. All rights reserved. 161. We realised that the Vendor & the Client were not comparing Apples with Apples• The vendor was producing Release Notes, but not explaining to the customer how tounderstand & use them (JIRA was the vendor Release Management tool)• The customer wasn’t forceful enough in asking for the Release Notes to be provided ina way that was usable (HPQC was the customer Test Management tool)2. We improved visibility of what was really happening• We got the Vendor & the Customer to agree on a single source of truth – HPQC• We turned HPQC into a Release Management tool3. We got the Vendor to send Testers to the Customer’s site & the Customer’s TestManager to visit the Vendor (for the first time)4. We created a joint vision of what was required for Go Live, built a consolidatedPlan to achieve it & then executed the Plan5. We provided the GLUE
    17. 17. The Story about Thanking my (new) Friends© 2011 Capgemini. All rights reserved. 171. The Consultants:• Magnus Wikholm, Juraj Lesko, Martin Torebring, Maria Ander2. The Customer:• Lars Bromander, Anders Nillson, Thomas Enjin, Torkel Strahle, Milena Haykowska3. The Vendor:• Kim Fitzpatrick, Russell Ascott, Sonia Libao, Simon Morley, Brian Roberts, JericoUbalde
    18. 18. Thank You…© 2011 Capgemini. All rights reserved. 18Colin CherryDirectorTesting ServicesCapgemini Australia Pty LtdLevel 2, 477 Collins StreetMelbourne VIC 3000 AustraliaTel: +61 3 9613 3138Mob: +61 412 214 240colin.cherry@capgemini.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×