Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intelligent Testing with mabl

18,246 views

Published on

With mabl the focus is on testing user journeys through simulation of user behavior - the underlying (and often changing) mechanics of xpaths, CSS selectors, element IDs, etc., are used as necessary but not the goal themselves and not something tester should have to think deeply about. Testing a user journey should verify that a user could easily complete the journey and provide insight into relevant behavior (and potential bugs and regressions) of the app.

To do this, mabl leverages machine intelligence (learning models of expected app behavior, states, and available user actions) and other machine strengths (run often, store and aggregate large amounts of information about each run, make multiple attempts). Check out the presentation or visit mabl.com to learn more.

Published in: Technology
  • Be the first to comment

Intelligent Testing with mabl

  1. 1. Intelligent Testing with mabl: Adaptive tests and automated visual insights with Sergei and John
  2. 2. What is mabl? Hi, I'm mabl ■ Founded in early 2017 ■ SaaS test automation - initial focus on web UI testing ■ Makes it easier to find and fix bugs quickly ■ Machine learning to drive robust testing and regression analysis (performance, errors, visual diffs)
  3. 3. ADAPTIVE USER-CENTRIC TESTS
  4. 4. Why adaptive testing ■ Scripting – brittle tests, high overhead ▲ Tightly coupled to changing UI elements ▲ Time writing scripts to minimize brittleness ▲ Time debugging brittle test failures ▲ Limited information for diagnosis ■ mabl approach ▲ Simulate user and test (end-to-end) journey ▲ Verification and insight into behavior of app ▲ Leverage machine intelligence and other machine strengths
  5. 5. Training a user journey ■ Capturing a user journey ▲ Domain-specific language to describe journey ▲ Evolving knowledge of how journey is achieved in app ■ What user interacts with ▲ Descriptive (element attributes) ▲ Context (ancestor element attributes, xpaths) ▲ Visual (image, bounding box, styling) ■ Expected behavior and states of app ▲ Descriptive (URL, title, page source, assets) ▲ Visual (screenshot, element bounding boxes) ▲ Timing (loading time, latencies)
  6. 6. Simulating the journey ■ Configuring and parameterizing ▲ Across environments, user types, app configurations ■ Identifying the right UI element ▲ Canonical identification ▲ Ranking partial/uncertain matches ■ Experimentation (multiple attempts) ▲ No need to craft best xpath/selector ▲ Robust to large UI revisions ▲ Broader coverage
  7. 7. Evaluating outcomes ■ Strong signals about action equivalence ▲ Assertions ▲ State transitions ▲ Comparison across runs and environments ■ State identification ▲ Visual appearance ▲ Descriptive properties (URL, title, assets loaded) ▲ Available actions
  8. 8. Evolving testing with the app ■ Results of adaptation ▲ Automatic updates ▲ Push-button (verify) update ▲ Retraining as a last resort ■ Continuous learning ▲ Incremental change ▲ A/B testing ▲ Transfer (across journeys, environments, users)
  9. 9. VISUAL MODELING OF APP STATES
  10. 10. can you SPOT THE DIFFERENCE?
  11. 11. benign changes
  12. 12. interesting changes
  13. 13. Roadmap: Near future: Learn dynamic areas ■ Learn areas of dynamic or periodic content ■ Model degree and types of expected change in dynamic areas ■ User feedback and control ▲ Should be added to the baseline (new feature, changed style) ▲ Static -> dynamic ▲ Dynamic -> static
  14. 14. Demo Time!

×