Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Push From Within - A Journey of Transformation at Walmart Labs by Claude Jones

99 views

Published on

Claude Jones, Sr. Director of Engineering at Walmart Labs, provides an inside look of what it took to push from within and help one of the world’s largest companies adopt a culture of quality. His presentation covers identifying the right project, building the right team structure, removing the fear of change, providing value with the right metrics, making testing fun and reinforcing the benefits. This case study will challenge you to re-think your approach or empower you to stay on your current path of adopting a culture of quality.

Published in: Software
  • Be the first to comment

  • Be the first to like this

The Push From Within - A Journey of Transformation at Walmart Labs by Claude Jones

  1. 1. THE PUSH FROM WITHIN: TRANSFORMATION A JOURNEY OF Sr. Director of Engineering Walmart Labs Claude Jones
  2. 2. WALMART LABS EXPERIENCE TOOLS MY PERSONAL JOURNEY Part 1 Part 2Adopting a Culture Quality WHAT I DID Maintaining a Culture of Quality WHAT I’M DOING
  3. 3. THE SCALE OF WALMART 270M WEEKLY CUSTOMERS 11K+ STORES WORLD WIDE 2.3M ASSOCIATES 100M+ ITEMS SOLD ONLINE
  4. 4. GROCERY STORE SYSTEMS ASSOCIATE TECH STORES GROWTH AND VOICE COMMERCE INTERNATIONAL MARKETS AND US SAMS PERSONALIZATION AND ADVERTISING WALMART.COM CE TECH AT JET.COM Responsible for all tech that power our omni-channel experiences
  5. 5. THE INVISIBLE WALL ANTI-PATTERN Lack of accountability between teams Poor product quality QA team is pressured to deliver under tight time constraints
  6. 6. AT WALMART SCALE PRODUCTION ISSUES HAVE A MASSIVE IMPACT TIMEFINANCIAL CUSTOMER How Do We Address The Problem?
  7. 7. PART 1: TRANSFORMATION JOURNEY adopting a culture of quality 2016
  8. 8. You have 6 months to automate 100% for your P1/P2 test cases THE EXECUTIVE PUSH Automate 2000+ Manual Test Train 600 Associates Onboard 25+ Apps on our testing platform Adopt Nodejs stack THE DETAILS THE MANDATE
  9. 9. ORGANIZATIONAL CHANGES DECENTRALIZE QA1 MAKE EVERYONE A SOFTWARE ENGINEER2 DENIAL RESISTANCE ACCEPTANCE THIS CREATED AN EMOTIONAL ROLLER COASTER ACROSS THE ORGANIZATION
  10. 10. THREE STEPS THAT ENABLED OUR SUCCESS Identify the right team1 Identify the right tools2 Provide Training & support3
  11. 11. IDENTIFY THE RIGHT TEAM
  12. 12. IDENTIFY THE RIGHT TOOLS FUNCTIONAL TESTING PERFORMANCE TESTING SYNTHETIC TESTING VISUAL TESTING ADA, ANALYTICS & SECURITY
  13. 13. IDENTIFY THE RIGHT TOOLS
  14. 14. PROVIDE TRAINING & SUPPORT
  15. 15. 100% P1/P2 Automated on Test Armada Daily deployments on the new FE stack Shared ownership of Quality Scalable test labs WHAT WAS DELIVEREDWHAT WAS ASKED FOR Automate 2000+ Manual Test Train 600 Associates On-board 25+ Apps on our testing platform Adopt Nodejs stack WHAT WE WERE ABLE TO ACCOMPLISH
  16. 16. TEST ARMADA BY THE NUMBERS 30K Test Cases Automated 5M Hours Saved 25M Test Executed API Mocked 360M 100 Projects Onboarded
  17. 17. OPERATIONAL EFFICIENCY AT SCALE 1,000+ Unique combinations 1,150 Available Browser/OS combinations Devices in a private cloud 81 5MAutomated Developer Hours Per Day 625KEngineers Working Per Day
  18. 18. THERE WAS A PROBLEM We were so focused on adoption that we were missing the bigger picture…
  19. 19. PART 2: TRANSFORMATION JOURNEY maintaining a culture of quality 2019
  20. 20. QUALITY TIME EXECUTIVE PUSH CULTURAL TRANSITION RESISTANT TO CHANGE VALUE ADD DECLINE IN USAGE REALITY HIT HARD 2016 2018 HIGH LOW
  21. 21. Our success of creating a culture of quality had negative side effects! WHAT WAS CAUSING THE DECLINE?
  22. 22. WHILE WE CELEBRATED OUR WINS… 0 20 40 60 80 100 120 23 24 25 27 28 30 31 32 33 34 36 37 38 40 41 42 43 44 45 46 47 48 49 50 Projects Created Cumulative Projects Created KPI • Reached 100 projects in the last 6 months KPI • Average 125k Per day (compared to previous estimate of 50k per day) SURGING ADOPTION INCREASING TEST EXECUTION
  23. 23. …WE LOST FOCUS ON THE BIGGER PICTURE CREATED VS. RESOLVED CHART • Peaked at 40 issues per week • Average 20 issues per week HIGH VOLUME OF SUPPORT CASES CHALLENGES CONCURRENT JOBS • Usage more than doubled over past year • Hitting max limit every day HITTING OUR RESOURCE LIMITS CHALLENGES
  24. 24. WE NEED TO GET BACK TO THE BASICS DEVELOPER EFFICIENCY THIS PROVIDED A POOR DEVELOPER EXPERIENCE PRODUCT DELIVERY PRODUCT QUALITY
  25. 25. DEVELOPER EFFICIENCY & PRODUCT DELIVERY WHAT THEY WANTED PR VERIFY 10 MINUTES MASTER VERIFY 1 HOUR OR LESS PROD VERIFY 1 HOUR OR LESS WHAT THEY GOT INSTEAD PR VERIFY 60 MINUTES MASTER VERIFY 3 - 4 HOURS PROD VERIFY 3 - 4 HOURS
  26. 26. WHAT ARE WE DOING ABOUT IT? Goal Approach Detail Streamline the utilization of our hardware Expand internal infrastructure Internal headless browser grid Speed up test execution times Smart test case execution Modular test grouping and execute long running test in parallel Test case development best practices Improve test automation design SauceLabs consulting to help optimize test case code
  27. 27. PRODUCT QUALITY APPLICATION QUALITY FUNCTIONAL SCORE Automation Score Escape Score Test Efficiency Score Code Coverage Score PERFORMANCE SCORE SYNTHETIC SCORE VISUAL SCORE ADA, ANALYTICS, & SECURITY OTHER SCORING TYPES
  28. 28. WHAT ARE WE DOING ABOUT IT?
  29. 29. OUR PROMISE TO THE CUSTOMER Holding the tools team accountable to a higher standard through measurable results Reduce tooling support incidents by 60% DEVELOPER EFFICIENCY Reduce cycle time to product by 80% PRODUCT DELIVERY Reduce customer facing issues by 50% PRODUCT QUALITY
  30. 30. QUALITY TIME INDIVIDUAL PUSH SENSE OF ACCOUNTABILITY INDIVIDUAL PUSH SENSE OF ACCOUNTABILITY WE ARE HERE HEALTHY QUALITY ECOSYSTEM EXECUTIVE PUSH CULTURAL TRANSITION RESISTANT TO CHANGE VALUE ADD 2016 2018 2019 HIGH LOW Helping the organization maintain a culture of quality.
  31. 31. TRANSFORMATIVE JOURNEY • Identify the right team • Identify the right tools • Provide Training & support 1 Adopting a Culture of Quality • Focus on Developer Efficiency • Focus on Product Delivery • Focus on Product Quality Maintaining a Culture of Quality2 CONTINUOUS DEPLOYMENTS Speed CONTINUOUS TESTING Automation CONTINUOUS MONITORING Quality
  32. 32. THANK YOU Sr. Director of Engineering Walmart Labs Claude Jones

×