Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

New Hexawise Features - Nov. 2012

8,731 views

Published on

An introduction to Hexawise's brand new features.

Hexawise is a software test design tool that improves the speed of software test design as well as the thoroughness and efficiency of software tests.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

New Hexawise Features - Nov. 2012

  1. 1. The Hexawise Test Design Tool Introduction to Powerful New Features November, 2012 1
  2. 2. Presenter Justin Hunter Founder and CEO of Hexawise justin.hunter@hexawise.com 2
  3. 3. Agenda• Intro and “Quickie Demo” - What is Hexawise? What doesn’t it do? - How does pairwise testing and related methods work? - Advantages and Limitations of the “Old” Hexawise• Hexawise “2.0” - The Most Powerful New Features We’ve Ever Released: - Requirements Traceability - More powerful Test Design Capabilities - Faster Test Script Documentation (including Expected Results) - Better Auto-Scripting• Demo of New Features• Wrap Up with Questions and Answers - Suggestions for Good Sources of Information 3
  4. 4. What is Hexawise?It’s a test design tool.Testers use it to do these two things better & faster. Define Select Test Document Requirements Conditions Test Scripts Export Test Execute Find Bugs Scripts into Test Test Scripts and Verify Mgmt. Tool BehaviorBut that’s definitely not all Hexawise does…. 4
  5. 5. What is Hexawise?… Hexawise also improves test execution efficiency andeffectiveness.The tests you generate with Hexawise are more powerful than thetests you would select by hand. Specifically, tests generated usingHexawise are optimized to: • Be as varied as possible, • Minimize wasteful repetition, and • Maximize test coverage in a manageable number of tests. Export Test Execute Find Bugs Scripts into Test Test Scripts and Verify Mgmt. Tool Behavior 5
  6. 6. What Doesn’t Hexawise Do?Hexawise is not a test management tool (like Quality Center).It’s also not a tool that automates test execution (like QTP). After testers generate Hexawise generates tests tests in Hexawise, they for teams that do both: usually export them to a - Manual testing, and tool like Quality Center. - Automated testing. Export Test Execute Scripts into Test Test Scripts Mgmt. Tool 6
  7. 7. User Poll Results: Most Popular Benefit? How? Hexawise automates1. You document Select Test Document many test design and test tests faster… Conditions Test Scripts documentation steps.2. You complete test Execute How? If you want to, you can execution faster… Test Scripts execute fewer, more powerful tests.3. You consistently Find Bugs How? Test generation algorithms and Verify find more bugs. Behavior maximize testing coverage.4. Your team makes more informed, How? From much more precise coverage analysis reporting. data-driven decisions. 7
  8. 8. Where Do Testers Use Hexawise? Everywhere. > 10,000 testers at >100 Fortune 500 firms On all kinds of testing projects 8
  9. 9. Is Hexawise Applicable to My System? Yes. “When testing anything: "No matter what kind of interface it has, you can always identify interesting things to vary." - Elisabeth Hendrickson – in her brand new book “Explore It!” 9
  10. 10. Agenda• Same Easy to Use Features as Always, to Increase Your… - Test selection speed - Test documentation speed - Testing thoroughness• “Quickie Demo” - Advantages and Limitations of the “Old” Hexawise• Plus The Most Powerful New Features We’ve Ever Released: - Requirements Traceability - More powerful Test Design Capabilities - Faster Documentation with Expected Results - Better Auto-Scripting• Demo of New Features 10
  11. 11. Charlie’s Chocolates – Release 7 The “interesting things to vary”… 11
  12. 12. Before, Select Test testers would (A) Enter test conditions,to Conditions and (B) click here. 12
  13. 13. How does Hexawise prioritize tests?Short version:1) Pairwise: In this example – using the default “pairwise” coverage strength - Hexawise will select 16 tests from 576 total possible ones. Those 16 tests include at least one test of every possible pair of Values.2) Rationale: The vast majority of your defects that sneak by undetected into production will be triggered if you tested for every possible pair of Values.3) Manually-Selected Tests Have Many Holes: If you started to test this way tomorrow, even with fewer tests, you would find many more defects than you currently do because you would be closing thousands of gaps that exist today.4) First Draft vs. Final: These 16 are a “first draft” set of powerful tests. Not a final set. You can and should modify them based on your knowledge.5) Turn the Coverage Dial Up or Down: You can easily generate however many tests you want depending upon your time vs. thoroughness preferences: a dozen?, a hundred?, all 576?For more information, data, and real-world case studies, see: pairwisetesting.com 13
  14. 14. Before, to Document Testers would transform powerfully optimized Test Scripts sets of test conditions like these… 14
  15. 15. …into “test scripts” like these using the Auto-Script feature. Note thatthese have both (A) detailed instructions and (B) test conditions. 15
  16. 16. Advantages & Limitations: “Old” HexawiseTest Script Documentation: While we were able to Quickly select and document new test scripts… And re-write the tester instructions “instantly” for each test… Those test scripts… Did not include Expected Results. 16
  17. 17. Advantages & Limitations: “Old” HexawiseRequirements / Ability to Include Specific Complex Combinations: While we were able to Quickly generate new sets of highly-varied test scripts… And those tests had maximum coverage possible with minimum repetition… We did not have the ability to… Include specific combinations of Values to test for our complex requirements. 17
  18. 18. So What’s New? 18
  19. 19. Charlie’s Chocolates – Release 8Let’s see how “Hexawise 2.0” handles changes for the next release:New Features to Test in Release 8 of Charlie’s Sales Platform: 1. Charlie now accepts MasterCard 2. Charlie now offers 2-day shippingAdditional “Trouble Area” to Test More Thoroughly in Release 8: 1. Single item orders are working well; when multiple items of an item are ordered, odd things have been happening in Release 7. 19
  20. 20. We need to add new Inputs (as usual…) 20
  21. 21. … which will lead to an entirely new set of“scrambled” tests which need new ERs. … tons of new Expected Results! 21
  22. 22. Good news: we can now generateExpected Results automatically! 22
  23. 23. Which means creating test scripts likethese is easy and fast:Once you generate your simple rules, hundreds (or thousands!) ofthese Expected Results will be generated with the push of abutton… without a single mistake. 23
  24. 24. So far so good, but how can we test aconvoluted warning message like this?!This“warning”feature canonly betested byspecifyingmanyspecificValues toappear in atest.This example (a result of both great customer service and good testing) was “seen in the wild” by one of my heroes:http://37signals.com/svn/posts/3239-a-great-customer-experience-nutscom-goes-the-extra-mile 24
  25. 25. Charlie’s Chocolates – Release 8And how does “Hexawise 2.0” handle these additional testing goals?Two Requirements to Test (from new Business Rules): 1. Fizzy Lifting Drinks ordered on November 11 should get an 11% discount. 2. “The complex conditional melted chocolate notification” • If customer selects 2-day delivery, • … and the type of food being shipped is made of chocolate, • … and it would actually take 3-days to deliver (b/c of a Sunday or Holiday), • … and it is predicted to be hot enough to melt chocolate outside, • …. Then notify the customer there is a risk the chocolate might melt. 25
  26. 26. Here’s how to think about enteringRequirements in Hexawise: First – Ask “what is the Expected Result of the requirement?” Second – Ask “What is the smallest number of test inputs you need to specify in order to see if the requirement is true or not?” Third – Enter those test inputs (and only those) into the Requirements screen. 26
  27. 27. Requirements “Fizzy Lifting Drinks ordered on November 11 should get an 11% discount.” 1. Expected Result =? “should get an 11% discount.” 2. Smallest Number of Values to trigger the discount = ? 2: Type of Candy must = Fizzy Lifting Drinks Date Ordered must = November 11th. 3. Now just enter those two Values (and only those two values!) into the Requirements screen and enter the ER! 27
  28. 28. Add the Values in Screen: 28
  29. 29. Requirements Feature (Cont.) To add a test for the second “complex melting chocolate notification” requirement, we follow the same process:What is the Expected Result we want to trigger? “Then notify the customer there is a risk the chocolate might melt.” How many specific Values do we need to trigger it? At least 4. 1. Customer selects 2-day delivery, and 2. The type of food being shipped is made of chocolate, and 3. It would actually take 3-days to deliver (b/c of a Sunday or Holiday), and 4. A weather prediction system predicts it will be hot. 29
  30. 30. Advantages of “New” HexawiseTest Script Documentation: As always, we are able to Quickly select and document new test scripts… And re-write the tester instructions “instantly” for each test… And now our test scripts also … Do include Expected Results. 30
  31. 31. Advantages & Limitations: “Old” HexawiseRequirements / Ability to Include Specific Complex Combinations: As before, we are able to Quickly generate new sets of highly-varied test scripts… And those tests had maximum coverage possible with minimum repetition… And now we also have the ability to… Include specific combinations of Values to test for our complex requirements. 31
  32. 32. “Traditional” Recommended SourcesExplanations of how Pairwise Testing (and similar, more sophisticated test design methods)consistently pack more testing coverage into a manageable set of tests than hand-selected tests • pairwisetesting.comQuestions about features (e.g., “How do I…?”) • help.hexawise.comTraining / understanding test design concepts • training.hexawise.comResponses from humans to questions, issues, and or feature suggestions • forum.hexawise.comAn introductory, fact-based article skeptical managers • An evidence-based article and explanation of benefits from a 10 project study that I co-wrote with 3 PhD’sDetailed case study showing more thorough coverage achieved with fewer than half as manytests at a large insurance firm across 8 diverse testing projects • Get to Market Faster with a Higher Quality Product (Available upon request by sending an email to justin.hunter@hexawise.com) 32
  33. 33. “Fun” Recommended Sources• A fun presentation on a powerful test design approach hexawise.com/?p=531• Nutty video announcement of our new “Hexawise 2.0” features http://www.youtube.com/watch?v=3F1eQag6XUg 33
  34. 34. Follow-ups, Questions and Answers 34
  35. 35. Appendix Slides 35
  36. 36. Create Your Test InputsHexawise packs as much coverage as possible into each test. For Hexawise to do so, you need toidentify the inputs that you want to test on the "Define Inputs" screen. 36
  37. 37. Generate Your TestsHexawise automatically generates a manageable number of unusually powerful tests so testers canfocus on testing the combinations of your test inputs that are most likely to trigger defects. 37
  38. 38. Analyze Your Testing CoverageHexawise reports exactly what percentage of in-scope combinations have been tested after eachHexawise-generated test. This helps you answer "How much testing is enough?" 38
  39. 39. Prevent Combinations from Appearing in TestsHexawise puts powerful test design features at your fingertips. Prevent "impossible to test for"combinations from appearing using the Paired Values features. 39
  40. 40. Test for Requirements and Easily Trace Them(Optional / Advanced) - Ensure your tests cover 100% of your requirements. Specify precisecombinations of Values to be included in your tests on the "Required" screen. 40
  41. 41. Generate Scripts w/ Detailed Tester Instructions(Optional / Advanced) - Add detailed tester instructions to all of your tests quickly and easily. Once youenter instructions on the Auto-Scripts screen, you can generate hundreds of tests like these below withthe push of a button. 41
  42. 42. Automatically Include Expected Results in Scripts(Optional / Advanced) - Even adding hundreds of Expected Results into your test scripts is easy andlightning fast. Once you enter instructions on the Expected Results screen, the Expected Results like thesewill automatically appear next to all of your test steps. 42
  43. 43. Help & Self-Guided Training Options: a Click Away(Optional) - Hexawise includes help files, a user forum to answer your questions and even test designtraining lessons to help you become an expert test designer. If you ever have questions, were here to help. 43
  44. 44. Become an Expert!You now know the basic layout and features of Hexawise. As you explore Hexawise and use features inHexawise, youll progress from a Novice user, to a Practitioner, and ultimately, an Expert. 44
  45. 45. Contact Info We’re here to help if you ever have any questions: Justin Hunter, CEO justin.hunter@hexawise.com +1 (919) 883-9388 45

×