MDD and the Tautology Problem: Discussion Notes.

on

  • 343 views

MoDeVVA @ MODELS 2009, Denver. 10/5/2009. What are the dangers of using the same model to both generate a system and the test suite for that system.

MoDeVVA @ MODELS 2009, Denver. 10/5/2009. What are the dangers of using the same model to both generate a system and the test suite for that system.

Statistics

Views

Total Views
343
Views on SlideShare
343
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

MDD and the Tautology Problem: Discussion Notes. MDD and the Tautology Problem: Discussion Notes. Presentation Transcript

  • MDD and the Tautology Problem: Discussion NotesMoDeVVA @ MODELS 2009 October 5, 2009 Robert V. Binder
  • The Tautology Effect• Transformation of model into object code is like transformation of source code into object code• An antecedent and its transformation cannot provide additional information necessary to reveal many common faults• White-box testing has well know limitations• Do these apply to implementations produced with MDD?
  • White Box Testing Limitations• Cannot test missing code – can’t reveal omissions• Cannot completely infer requirements (intent) from code• Where do we get the oracle? (how to discern correct/incorrect outputs?)• Triggering of side-effects• Platform-specific behavior/interactions• Blind to interaction effects outside the scope of the unit under test• Doesn’t scale• Doesn’t support “non-functional” testing• Possible developer bias in test selection
  • Weuyker: Adequacy Axioms• In terms of the antecedent (source code), what must an adequate test suite exercise?• Must at least cover every statement – What is a statement in a model?• Anti-extensionality – The semantic equality of two programs is not sufficient to imply that they should necessarily be tested the same way• Anti-decomposition – Although a program has been adequately tested, it does not necessarily imply that each of its called modules has been adequately tested.• More …• Elaine J. Weyuker, “The Evaluation Of Program-based Software Test Data Adequacy Criteria,” Communications of the ACM, v 31 n 6, June 1988.
  • Manna: Limits of Verification• Verification = proof of correctness – We can never be sure that the specifications are correct – No verification system can verify every correct program – We can never be certain that a verification system is correct• Zohar Manna and Richard Waldinger, “The Logic of Computer Programming,” IEEE Transactions on Software Engineering, v SE-4, n 3, May 1978
  • Some Observations• Transforming model elements to an executable – Fewer constraints than code expression to machine instructions – Many assumptions baked-in with translator – More fault opportunities• Many functionally equivalent implementations possible, suggests test strategy – M to E1 with T1, M to E2 with T2 – Run same test suite on E1 and E2 – Expect identical results for deterministic behavior – Expect bounded results for non-deterministic behavior