Journey To Excellence


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Journey To Excellence

  1. 1. Another “Journey to Excellence” Walter Mamed, ASQ-CSQE (A Portfolio of Recent Work 2007-2009)
  2. 2. The Journey Starts Here <ul><li>Created automated test cases and test scripts incrementally for en Espanol (HEE). </li></ul><ul><li>Each successive build was tested quicker, thus allowing more time for additional scripting. </li></ul><ul><li>Examined sharing code base on (Hcom) </li></ul><ul><li>Utilized language specific constants. </li></ul><ul><li>Embedded all environment information in a config file. </li></ul><ul><li>Scripts run in either English or Espanol on a multitude of QA environments without code changes. </li></ul><ul><li>Automated regression initiated for weekly releases. </li></ul><ul><li>Shared my vision of extending success further. </li></ul>Walter Mamed
  3. 3. LODO <ul><li>“ Lights On, Doors Open” position created within QA. </li></ul><ul><li>Drive quality improvements across the organization. </li></ul><ul><li>Increase automated test coverage on Hcom. </li></ul><ul><li>Introduce and experiment with new testing methodologies. </li></ul><ul><li>Apply automation or test tools to other groups and product lines. </li></ul><ul><li>Facilitate and lead the Joint Test Council (JTC). </li></ul>Walter Mamed
  4. 4. Joint Test Council <ul><li>Council created to “move forward with a more engineered approach to testing across the organization.” </li></ul><ul><li>Committee composed of Platform Dev, App Dev, CM, QA, Managers and Directors. </li></ul><ul><li>Mission Statement: Quality Services through Innovation </li></ul>Walter Mamed
  5. 5. Joint Test Council in motion <ul><li>Facilitator and lead contributor on the Joint Test Council (JTC). </li></ul><ul><li>Determined test automation targets & priority. </li></ul><ul><li>Tasks were identified, prioritized, and tracked to closure. </li></ul><ul><li>Test code shared between Dev and QA. </li></ul><ul><li>Created and maintained a SharePoint site with Announcements, Events, and Documents. </li></ul>Walter Mamed
  6. 6. Determined test automation targets & priority Smoke Tests Booking Booking Scenarios High Automation Priority Low Balancing risk based testing and repetition of tasks Defect Escapes Business Adjustments Tools supporting manual testing Critical or High use of subsystem, function or feature High Test Coverage Low Searches Functional 1 Functional … Functional N Walter Mamed
  7. 7. Joint Test Council (Quality Services through Innovation) Walter Mamed
  8. 8. Methods coded in Ruby <ul><li>“ Pop” email that was generated from web site and verify delivery, format and content. </li></ul><ul><ul><li>Confirmation emails to customer. </li></ul></ul><ul><ul><li>Itinerary emails to customer. </li></ul></ul><ul><ul><li>Change password email, including extracting link, activating and validating password change on user accounts . </li></ul></ul><ul><ul><li>Increased test coverage beyond Web Application. </li></ul></ul><ul><ul><li>Tools: Ruby, NET::POP, stunnel </li></ul></ul><ul><li>Capture page image of UI when error detected. </li></ul><ul><ul><li>Used Ruby to communicate with SnagIt screen capture API. </li></ul></ul><ul><ul><li>Complete image of web page captured using scrolling window. </li></ul></ul><ul><ul><li>Provides visual information when documenting issues. </li></ul></ul><ul><ul><li>Tools: Ruby, SnagIt API </li></ul></ul>Walter Mamed
  9. 9. Example of Regression Test Report Walter Mamed
  10. 10. Issues Discovered and Tracked <ul><li>Tracked issues discovered during weekly regression, Preprod and Production testing. </li></ul><ul><li>Tracked issues discovered during project testing (incoming web enhancements and redesigns) </li></ul><ul><li>By keeping a record of issues discovered, the value of automated regression was easily recognized in both product streams. </li></ul>Walter Mamed
  11. 11. Issues discovered by WATiR automated regression tests Walter Mamed
  12. 12. Issues discovered by WATiR automated regression tests on Projects Walter Mamed
  13. 13. Joint Test Council Sharepoint Site Walter Mamed
  14. 14. Tracking page download times <ul><li>Measure actual web page download times (user experience). </li></ul><ul><li>Perform measurements during Smoke Testing. </li></ul><ul><li>Provides indication of application performance very early in the test cycle. </li></ul><ul><li>Record measurements from Release to Release for trending. </li></ul><ul><li>Eliminates subjective observations, “feels slower”. </li></ul><ul><li>Measurement scenarios include client side configurations (OS, Browser and Memory combinations). </li></ul><ul><li>Next four slides contain two test scenarios using XP & IE6 and Vista & IE7. </li></ul>Walter Mamed
  15. 15. Walter Mamed
  16. 16. Walter Mamed
  17. 17. Walter Mamed
  18. 18. Walter Mamed
  19. 19. Walter Mamed
  20. 20. Walter Mamed Page download analysis using Watir, HttpWatch and Firefox
  21. 21. Scraping remote server logs during automated testing <ul><li>Scrape Tomcat server logs for errors not presented on web UI. </li></ul><ul><li>Wrote a Ruby utility which extracts server log entries while Watir is driving web UI. </li></ul><ul><li>Gives more visibility to backend issues during regression testing. </li></ul><ul><li>Tools used: WATiR, Ruby, NET::SSH, & Unix tail </li></ul>Walter Mamed
  22. 22. tail remote server logs and view locally <ul><li>Proof of Concept Ruby code variant provides additional benefits (when using a file viewer on the desktop) </li></ul><ul><ul><li>Real-time file viewing (like PuTTY) </li></ul></ul><ul><ul><li>Monitoring and detecting serious log entries is easier. </li></ul></ul><ul><ul><li>Highlight text based on filters. </li></ul></ul><ul><ul><ul><li>Emphasize entries based on severity (red gives more pop to FATAL messages) </li></ul></ul></ul><ul><ul><ul><li>De-emphasize entries (light grey for INFO messages) </li></ul></ul></ul><ul><ul><li>See next slide for example. </li></ul></ul>Walter Mamed
  23. 23. Ruby code used to tail remote server logs and view locally Walter Mamed
  24. 24. ROI Calculation Summary Walter Mamed
  25. 25. Coding Accomplishments <ul><li>Descriptive programming style (like QTP) </li></ul><ul><li>All methods organized in a shared function library. </li></ul><ul><li>Scripts able to run on QA, Pre-Prod & Prod using an environment file. </li></ul><ul><li>Coded several Ruby utilities using gem extensions </li></ul><ul><li>Used behavioral naming of methods for clarity </li></ul><ul><li>Method level partitioning offered more precise and meaningful information when a test case fails. </li></ul><ul><li>Coded several scripts for QA staff usage (quick hits) </li></ul><ul><ul><li>XML SEO link validation </li></ul></ul><ul><ul><li>Missed destinations </li></ul></ul><ul><li>Utility pulls remote server logs (securely) for local script processing and viewing. </li></ul>Walter Mamed
  26. 26. Process Improvement Achievements <ul><li>Continuous integration supported (using automated Smoke tests) </li></ul><ul><li>Leveraged automation on core architecture/development projects (Terracotta JVM platform, transition 32 to 64 bit, ATG => Phoenix & Apache => Tomcat migration) </li></ul><ul><li>Ruby/Watir utilities eased repetitive manual testing for QA team. </li></ul><ul><li>Regression tests allowed QA team to do more exploratory testing (an agile concept) from build to build. </li></ul><ul><li>Comprehensive testing (high test coverage includes negative testing) </li></ul><ul><li>Gathering metrics from release to release for trend analysis. </li></ul><ul><li>Productivity achievements with “quick hits” </li></ul><ul><li>One hour regression institutionalized (required step per Mgmt) </li></ul><ul><li>Cross browser testing across multiple OS’s and memory configs </li></ul>Walter Mamed
  27. 27. Automation Achievements <ul><li>Full end-to-end regression automation of site. </li></ul><ul><li>Automated iPhone app testing (FireWatir & Firefox). </li></ul><ul><li>Introduced automated smoke testing to site. </li></ul><ul><li>Automation of Connector (Web Sales Entry) tool in progress. </li></ul><ul><li>Test scripts facilitated high volume testing (e.g. SEO XML validation) </li></ul><ul><li>Test scripts used to calculate “missed destinations” for Development </li></ul><ul><li>Validation of SEO tags and values embedded within html. </li></ul><ul><li>Many issues detected that were missed by manual testing alone. </li></ul><ul><li>Automated booking path of One H Connector for E2E testing. </li></ul>Walter Mamed
  28. 28. What’s Next? <ul><li>A new “Journey to Excellence” begins at, coming Oct ‘09 </li></ul><ul><li>Walter Mamed, ASQ-CSQE, ISTQB-CTFL </li></ul><ul><li> </li></ul>