®          mVerify          A Million Users in a Box ®Test Objects -- They Just Work           September 8, 2006    Google...
Overview Motivation MTS Goals TTCN Influences X-Unit Influences MTS::TestObject Demo Q&A                © 2006 mVer...
The Mobile Testing Nightmare                                     Intense, high-stakes race to                            ...
MTS Background Advanced model-based, mobile-centric testing    Necessary for 21st century technology    It’s time for a...
MTS: Any App, Any PlatformConsole Host                                                               Agent Host     MTS Co...
Design Goals   Generate Test Objects from models   Generate models from IUT   Logical/Physical separation   Platform a...
About TTCN-3   Testing and Test Control Notation, international standard   Abstract specification   Structured/Procedur...
About TTCN-3 Point of Observation and Control   Abstract Interface Specification   Each user provides physical binding ...
Incremental Testing Framework                              «uses»TestRunner +Contains         TestLog               -Uses ...
TTCN and ITF -- ComplementaryMTS Design Goal                                                         TTCN       ITFAutomat...
TTCN and ITF -- Limitations TTCN   Not executable – must be compiled   L2P implementation out of scope   Modular, but ...
MTS::TestObject                                                           MTS::TestObject              MTS Framework      ...
MTS::TestObject Adapter: built-in remote host proxy; connection  management, logging, exception handling Control-composa...
Demo Adapter selection One-click Repeat Smart Progress Bar               © 2006 mVerify Corporation   A Million Users i...
Notes NeXTSTEP demo, Steve Jobs says “It just works --  they’re objects“  http://rixstep.com/1/1/20060814,00.shtml Mobil...
Q&A© 2006 mVerify Corporation   A Million Users in a Box ®   16
Upcoming SlideShare
Loading in …5
×

Test Objects -- They Just Work

333 views

Published on

Keynote: Google Test Automation Conference. September 8, 2006,London, England.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
333
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Test Objects -- They Just Work

  1. 1. ® mVerify A Million Users in a Box ®Test Objects -- They Just Work September 8, 2006 Google Test Automation Conference www.mverify.com
  2. 2. Overview Motivation MTS Goals TTCN Influences X-Unit Influences MTS::TestObject Demo Q&A © 2006 mVerify Corporation A Million Users in a Box ® 2
  3. 3. The Mobile Testing Nightmare  Intense, high-stakes race to market  Configurations (platforms x devices x airlinks) increase exponentially  More testing necessary for competitive quality, reliability, performance  Ad hoc manual testing is slow, costly, ineffective © 2006 mVerify Corporation A Million Users in a Box ® 3
  4. 4. MTS Background Advanced model-based, mobile-centric testing  Necessary for 21st century technology  It’s time for a change  Model-based, mobile-centric, reliability-maximizing Advanced test strategy useless without  End to end and embedded observation & control  Scalable, Robust  Distributed  Broadly usable © 2006 mVerify Corporation A Million Users in a Box ® 4
  5. 5. MTS: Any App, Any PlatformConsole Host Agent Host MTS Console MTS Test TEST RUN Agent REPORTSAgent Host MTS Remote Agent MTS Test Agent Client Under Test Client Host Under Test MTS Remote Agent MTS Remote Client Agent Under Test Host Under Test may be Server Client Host Under Test  Cell Phone Under Test  PDA  Desktop Server Host Under Test MTS Remote Agent  Server  Embedded Processor Server  Network Equipment Under Test  Access Point  Base Station Server Host Under Test © 2006 mVerify Corporation A Million Users in a Box ® 5
  6. 6. Design Goals Generate Test Objects from models Generate models from IUT Logical/Physical separation Platform agnostic/robust Channel agnostic/robust Minimize IUT footprint Distributed control Works out of the box/Agile Intuitive interaction Minimize brittleness Composable © 2006 mVerify Corporation A Million Users in a Box ® 6
  7. 7. About TTCN-3 Testing and Test Control Notation, international standard Abstract specification Structured/Procedural paradigm Proven support for complex scenarios Proven support for concurrency ASN.1 data (packed, maps to XML) Routine use in very high-reliability applications  Rigorous conformance testing of protocol implemenations, from single test spec © 2006 mVerify Corporation A Million Users in a Box ® 7
  8. 8. About TTCN-3 Point of Observation and Control  Abstract Interface Specification  Each user provides physical binding for logical interface definition  Same test suites can be used on many implementations Model-based Testing  Generate TTCN from MSCs, FSMs, etc.  Compile to implementation language (C++, Java)  Compile & run implementation © 2006 mVerify Corporation A Million Users in a Box ® 8
  9. 9. Incremental Testing Framework «uses»TestRunner +Contains TestLog -Uses TestObject 1 1 1 1 0..1 TestSuite -Contains TestCase -Contains 0..1 0..* 0..* MyTestSuite MyTestObject © 2006 mVerify Corporation A Million Users in a Box ® 9
  10. 10. TTCN and ITF -- ComplementaryMTS Design Goal TTCN ITFAutomate test object generation Yes NALogical/Physical separation Yes NoPlatform agnostic/robust Yes NoChannel agnostic/robust Yes NADistributed control Yes NoMinimize IUT footprint NA NoWorks out of the box/Agile No YesIntuitive interaction No YesComposable No YesMinimize brittleness No No © 2006 mVerify Corporation A Million Users in a Box ® 10
  11. 11. TTCN and ITF -- Limitations TTCN  Not executable – must be compiled  L2P implementation out of scope  Modular, but not composable ITF  TestSuite limited to static object list  Not control-composable  Tight coupling with IUT  Greedy  Language-centric  Developer-centric © 2006 mVerify Corporation A Million Users in a Box ® 11
  12. 12. MTS::TestObject MTS::TestObject MTS Framework +testObjectHeadline +getTestObjectClassName() +getTestObjectHeadline() +getTestObjectFileName() Adapter +getTestObjectDir() +getRemoteHostStatus()+connect() +getTestRunRepeatCount() YourTestObject+open() +getTestRunIteration() #require() MyTestObject+close()+test() #setup()+record() #test() +require() PPC 2002 #cleanup() +setup() WM5 #ensure() +test() #check() +cleanup() +getImage() #run() +ensure() #submit() WebService #userLogEntry() #userLogImage() +getServiceWSDL() #evaluateCompareFrame() +parseWSDL() #replayTestActionSequence() +queryForDictionary() +queryForXML() © 2006 mVerify Corporation A Million Users in a Box ® 12
  13. 13. MTS::TestObject Adapter: built-in remote host proxy; connection management, logging, exception handling Control-composable TestObject::run <testobject> Pre-conditions TestObject::require Post-conditions TestObject::ensure Built-in data-driven test from literal, variable, or function TestObject::submit <data_provider> Pass/fail any Tcl expression: TestObject::check <expr> XML test run log Incr Tcl, any Tcl package, library extension, or procedure Support stories, suites, cases, procedures … © 2006 mVerify Corporation A Million Users in a Box ® 13
  14. 14. Demo Adapter selection One-click Repeat Smart Progress Bar © 2006 mVerify Corporation A Million Users in a Box ® 14
  15. 15. Notes NeXTSTEP demo, Steve Jobs says “It just works -- they’re objects“ http://rixstep.com/1/1/20060814,00.shtml Mobile Testing Nightmare Poster Questions: Bob_Binder@mverify.com © 2006 mVerify Corporation A Million Users in a Box ® 15
  16. 16. Q&A© 2006 mVerify Corporation A Million Users in a Box ® 16

×