SlideShare a Scribd company logo
MDD and the
Tautology Problem:
Discussion Notes
MoDeVVA @ MODELS 2009
October 5, 2009
Robert V. Binder
The Tautology Effect
• Transformation of model into object code
is like transformation of source code into
object code
• An antecedent and its transformation
cannot provide additional information
necessary to reveal many common faults
• White-box testing has well know limitations
• Do these apply to implementations
produced with MDD?
White Box Testing Limitations
• Cannot test missing code – can’t reveal omissions
• Cannot completely infer requirements (intent) from code
• Where do we get the oracle? (how to discern
correct/incorrect outputs?)
• Triggering of side-effects
• Platform-specific behavior/interactions
• Blind to interaction effects outside the scope of the unit
under test
• Doesn’t scale
• Doesn’t support “non-functional” testing
• Possible developer bias in test selection
Weuyker: Adequacy Axioms
• In terms of the antecedent (source code), what must an
adequate test suite exercise?
• Must at least cover every statement
– What is a statement in a model?
• Anti-extensionality
– The semantic equality of two programs is not sufficient to imply
that they should necessarily be tested the same way
• Anti-decomposition
– Although a program has been adequately tested, it does not
necessarily imply that each of its called modules has been
adequately tested.
• More …
• Elaine J. Weyuker, “The Evaluation Of Program-based
Software Test Data Adequacy Criteria,” Communications
of the ACM, v 31 n 6, June 1988.
Manna: Limits of Verification
• Verification = proof of correctness
– We can never be sure that the specifications are
correct
– No verification system can verify every correct
program
– We can never be certain that a verification system is
correct
• Zohar Manna and Richard Waldinger, “The
Logic of Computer Programming,” IEEE
Transactions on Software Engineering, v SE-4, n
3, May 1978
Some Observations
• Transforming model elements to an executable
– Fewer constraints than code expression to machine
instructions
– Many assumptions baked-in with translator
– More fault opportunities
• Many functionally equivalent implementations
possible, suggests test strategy
– M to E1 with T1, M to E2 with T2
– Run same test suite on E1 and E2
– Expect identical results for deterministic behavior
– Expect bounded results for non-deterministic behavior

More Related Content

What's hot

Test case writing
Test case writingTest case writing
Test case writing
Afsarah Jahin
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
Abdel Salam Sayyad
 
Software testing methods
Software testing methodsSoftware testing methods
Software testing methods
Donato Di Pierro
 
c
cc
On the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software TestingOn the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software Testing
jfrchicanog
 
All you need to know about regression testing | David Tzemach
All you need to know about regression testing | David TzemachAll you need to know about regression testing | David Tzemach
All you need to know about regression testing | David Tzemach
David Tzemach
 
Python: Design Patterns
Python: Design PatternsPython: Design Patterns
Python: Design Patterns
Damian T. Gordon
 
Lecture 1
Lecture 1Lecture 1
Lecture 1
RacingKings
 
Nimbuzz advanced vs novice testers
Nimbuzz   advanced vs novice testersNimbuzz   advanced vs novice testers
Nimbuzz advanced vs novice testers
Ravindran Antonysamy
 
Testability: Factors and Strategy
Testability: Factors and StrategyTestability: Factors and Strategy
Testability: Factors and Strategy
Bob Binder
 
Software testing fundamentals
Software testing fundamentalsSoftware testing fundamentals
Software testing fundamentals
Mona M. Abd El-Rahman
 
Measuring the Code Quality Using Software Metrics
Measuring the Code Quality Using Software MetricsMeasuring the Code Quality Using Software Metrics
Measuring the Code Quality Using Software Metrics
Geetha Anjali
 
FutureOfTesting2008
FutureOfTesting2008FutureOfTesting2008
FutureOfTesting2008
vipulkocher
 
Software Testing
Software TestingSoftware Testing
Software Testing
SKumar11384
 
Exploratory testing
Exploratory testingExploratory testing
Exploratory testing
Spoorthi Sham
 
Effective Project Execution
Effective Project ExecutionEffective Project Execution
Effective Project Execution
SenthilKumar Selvaraj
 
Finding Defects in C#: Coverity vs. FxCop
Finding Defects in C#: Coverity vs. FxCopFinding Defects in C#: Coverity vs. FxCop
Finding Defects in C#: Coverity vs. FxCop
Coverity
 
Regression testing
Regression testingRegression testing
Regression testing
mushfiqangshu
 
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Tim Menzies
 
The Myths and Facts Surrounding Software Testing
The Myths and Facts Surrounding Software TestingThe Myths and Facts Surrounding Software Testing
The Myths and Facts Surrounding Software Testing
Software Testing Solution
 

What's hot (20)

Test case writing
Test case writingTest case writing
Test case writing
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
 
Software testing methods
Software testing methodsSoftware testing methods
Software testing methods
 
c
cc
c
 
On the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software TestingOn the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software Testing
 
All you need to know about regression testing | David Tzemach
All you need to know about regression testing | David TzemachAll you need to know about regression testing | David Tzemach
All you need to know about regression testing | David Tzemach
 
Python: Design Patterns
Python: Design PatternsPython: Design Patterns
Python: Design Patterns
 
Lecture 1
Lecture 1Lecture 1
Lecture 1
 
Nimbuzz advanced vs novice testers
Nimbuzz   advanced vs novice testersNimbuzz   advanced vs novice testers
Nimbuzz advanced vs novice testers
 
Testability: Factors and Strategy
Testability: Factors and StrategyTestability: Factors and Strategy
Testability: Factors and Strategy
 
Software testing fundamentals
Software testing fundamentalsSoftware testing fundamentals
Software testing fundamentals
 
Measuring the Code Quality Using Software Metrics
Measuring the Code Quality Using Software MetricsMeasuring the Code Quality Using Software Metrics
Measuring the Code Quality Using Software Metrics
 
FutureOfTesting2008
FutureOfTesting2008FutureOfTesting2008
FutureOfTesting2008
 
Software Testing
Software TestingSoftware Testing
Software Testing
 
Exploratory testing
Exploratory testingExploratory testing
Exploratory testing
 
Effective Project Execution
Effective Project ExecutionEffective Project Execution
Effective Project Execution
 
Finding Defects in C#: Coverity vs. FxCop
Finding Defects in C#: Coverity vs. FxCopFinding Defects in C#: Coverity vs. FxCop
Finding Defects in C#: Coverity vs. FxCop
 
Regression testing
Regression testingRegression testing
Regression testing
 
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
 
The Myths and Facts Surrounding Software Testing
The Myths and Facts Surrounding Software TestingThe Myths and Facts Surrounding Software Testing
The Myths and Facts Surrounding Software Testing
 

Viewers also liked

Discrete Structures lecture 2
 Discrete Structures lecture 2 Discrete Structures lecture 2
Discrete Structures lecture 2
Ali Usman
 
Want to know about java
Want to know about javaWant to know about java
Want to know about java
adityamadgula
 
Discrete mathematics
Discrete mathematicsDiscrete mathematics
Discrete mathematics
University of Potsdam
 
Discrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكرو
Discrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكروDiscrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكرو
Discrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكرو
Dr. Khaled Bakro
 
Propositional logic
Propositional logicPropositional logic
Propositional logic
Janet Stemwedel
 
Introduction to parliamentary debate
Introduction to parliamentary debateIntroduction to parliamentary debate
Introduction to parliamentary debate
Abhinandan Ray
 
Logic (slides)
Logic (slides)Logic (slides)
Logic (slides)
IIUM
 
Truth table
Truth tableTruth table
Truth table
Abdur Rehman
 
Truth tables complete and p1 of short method
Truth tables complete and p1 of short methodTruth tables complete and p1 of short method
Truth tables complete and p1 of short method
Nat Karablina
 
Propositions - Discrete Structures
Propositions - Discrete Structures Propositions - Discrete Structures
Propositions - Discrete Structures
Drishti Bhalla
 
The logic
The logicThe logic
The logic
Nittaya Noinan
 
Mathematical Logic - Part 1
Mathematical Logic - Part 1Mathematical Logic - Part 1
Mathematical Logic - Part 1
blaircomp2003
 
Propositional logic
Propositional logicPropositional logic
Propositional logic
University of Potsdam
 
Lec 02 logical eq (Discrete Mathematics)
Lec 02   logical eq (Discrete Mathematics)Lec 02   logical eq (Discrete Mathematics)
Lec 02 logical eq (Discrete Mathematics)
Naosher Md. Zakariyar
 
Chapter 1 Logic of Compound Statements
Chapter 1 Logic of Compound StatementsChapter 1 Logic of Compound Statements
Chapter 1 Logic of Compound Statements
guestd166eb5
 
Discrete Mathematics - Propositional Logic
Discrete Mathematics - Propositional LogicDiscrete Mathematics - Propositional Logic
Discrete Mathematics - Propositional Logic
University of Potsdam
 
Propositional logic
Propositional logicPropositional logic
Propositional logic
ForwardBlog Enewzletter
 

Viewers also liked (17)

Discrete Structures lecture 2
 Discrete Structures lecture 2 Discrete Structures lecture 2
Discrete Structures lecture 2
 
Want to know about java
Want to know about javaWant to know about java
Want to know about java
 
Discrete mathematics
Discrete mathematicsDiscrete mathematics
Discrete mathematics
 
Discrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكرو
Discrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكروDiscrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكرو
Discrete mathematics Ch2 Propositional Logic_Dr.khaled.Bakro د. خالد بكرو
 
Propositional logic
Propositional logicPropositional logic
Propositional logic
 
Introduction to parliamentary debate
Introduction to parliamentary debateIntroduction to parliamentary debate
Introduction to parliamentary debate
 
Logic (slides)
Logic (slides)Logic (slides)
Logic (slides)
 
Truth table
Truth tableTruth table
Truth table
 
Truth tables complete and p1 of short method
Truth tables complete and p1 of short methodTruth tables complete and p1 of short method
Truth tables complete and p1 of short method
 
Propositions - Discrete Structures
Propositions - Discrete Structures Propositions - Discrete Structures
Propositions - Discrete Structures
 
The logic
The logicThe logic
The logic
 
Mathematical Logic - Part 1
Mathematical Logic - Part 1Mathematical Logic - Part 1
Mathematical Logic - Part 1
 
Propositional logic
Propositional logicPropositional logic
Propositional logic
 
Lec 02 logical eq (Discrete Mathematics)
Lec 02   logical eq (Discrete Mathematics)Lec 02   logical eq (Discrete Mathematics)
Lec 02 logical eq (Discrete Mathematics)
 
Chapter 1 Logic of Compound Statements
Chapter 1 Logic of Compound StatementsChapter 1 Logic of Compound Statements
Chapter 1 Logic of Compound Statements
 
Discrete Mathematics - Propositional Logic
Discrete Mathematics - Propositional LogicDiscrete Mathematics - Propositional Logic
Discrete Mathematics - Propositional Logic
 
Propositional logic
Propositional logicPropositional logic
Propositional logic
 

Similar to MDD and the Tautology Problem: Discussion Notes.

testing
testingtesting
testing
Rashmi Deoli
 
Software testing part
Software testing partSoftware testing part
Software testing part
Preeti Mishra
 
Type mock isolator
Type mock isolatorType mock isolator
Type mock isolator
MaslowB
 
Week 14 Unit Testing.pptx
Week 14  Unit Testing.pptxWeek 14  Unit Testing.pptx
Week 14 Unit Testing.pptx
mianshafa
 
ST UNIT-1.pptx
ST UNIT-1.pptxST UNIT-1.pptx
ST UNIT-1.pptx
GhanaVishnu
 
An introduction to Software Testing and Test Management
An introduction to Software Testing and Test ManagementAn introduction to Software Testing and Test Management
An introduction to Software Testing and Test Management
Anuraj S.L
 
What is software testing in software engineering?
What is software testing in software engineering?What is software testing in software engineering?
What is software testing in software engineering?
tommychauhan
 
What is Testing in Software Engineering?
What is Testing in Software Engineering?What is Testing in Software Engineering?
What is Testing in Software Engineering?
tommychauhan
 
How to Actually DO High-volume Automated Testing
How to Actually DO High-volume Automated TestingHow to Actually DO High-volume Automated Testing
How to Actually DO High-volume Automated Testing
TechWell
 
Software Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCESoftware Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCE
Dr Anuranjan Misra
 
testing.pdf
testing.pdftesting.pdf
testing.pdf
kumari36
 
IEEE 1633 Recommended Practices for Reliable Software
IEEE 1633 Recommended Practices for Reliable SoftwareIEEE 1633 Recommended Practices for Reliable Software
IEEE 1633 Recommended Practices for Reliable Software
Ann Marie Neufelder
 
Software Testing 1/5
Software Testing 1/5Software Testing 1/5
Software Testing 1/5
Damian T. Gordon
 
testing strategies and tactics
 testing strategies and tactics testing strategies and tactics
testing strategies and tactics
Preeti Mishra
 
Testing concepts [3] - Software Testing Techniques (CIS640)
Testing concepts [3] - Software Testing Techniques (CIS640)Testing concepts [3] - Software Testing Techniques (CIS640)
Testing concepts [3] - Software Testing Techniques (CIS640)
Venkatesh Prasad Ranganath
 
Software_Testing_Overview.pptx
Software_Testing_Overview.pptxSoftware_Testing_Overview.pptx
Software_Testing_Overview.pptx
JayPrakash255
 
A Software Testing Intro
A Software Testing IntroA Software Testing Intro
A Software Testing Intro
Evozon Test Lab
 
Utility of Test Coverage Metrics in TDD
Utility of Test Coverage Metrics in TDDUtility of Test Coverage Metrics in TDD
Utility of Test Coverage Metrics in TDD
XP Conference India
 
Chapter 13 software testing strategies
Chapter 13 software testing strategiesChapter 13 software testing strategies
Chapter 13 software testing strategies
SHREEHARI WADAWADAGI
 
Annotated Bibliography .Guidelines Annotated Bibliograph.docx
Annotated Bibliography  .Guidelines Annotated Bibliograph.docxAnnotated Bibliography  .Guidelines Annotated Bibliograph.docx
Annotated Bibliography .Guidelines Annotated Bibliograph.docx
justine1simpson78276
 

Similar to MDD and the Tautology Problem: Discussion Notes. (20)

testing
testingtesting
testing
 
Software testing part
Software testing partSoftware testing part
Software testing part
 
Type mock isolator
Type mock isolatorType mock isolator
Type mock isolator
 
Week 14 Unit Testing.pptx
Week 14  Unit Testing.pptxWeek 14  Unit Testing.pptx
Week 14 Unit Testing.pptx
 
ST UNIT-1.pptx
ST UNIT-1.pptxST UNIT-1.pptx
ST UNIT-1.pptx
 
An introduction to Software Testing and Test Management
An introduction to Software Testing and Test ManagementAn introduction to Software Testing and Test Management
An introduction to Software Testing and Test Management
 
What is software testing in software engineering?
What is software testing in software engineering?What is software testing in software engineering?
What is software testing in software engineering?
 
What is Testing in Software Engineering?
What is Testing in Software Engineering?What is Testing in Software Engineering?
What is Testing in Software Engineering?
 
How to Actually DO High-volume Automated Testing
How to Actually DO High-volume Automated TestingHow to Actually DO High-volume Automated Testing
How to Actually DO High-volume Automated Testing
 
Software Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCESoftware Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCE
 
testing.pdf
testing.pdftesting.pdf
testing.pdf
 
IEEE 1633 Recommended Practices for Reliable Software
IEEE 1633 Recommended Practices for Reliable SoftwareIEEE 1633 Recommended Practices for Reliable Software
IEEE 1633 Recommended Practices for Reliable Software
 
Software Testing 1/5
Software Testing 1/5Software Testing 1/5
Software Testing 1/5
 
testing strategies and tactics
 testing strategies and tactics testing strategies and tactics
testing strategies and tactics
 
Testing concepts [3] - Software Testing Techniques (CIS640)
Testing concepts [3] - Software Testing Techniques (CIS640)Testing concepts [3] - Software Testing Techniques (CIS640)
Testing concepts [3] - Software Testing Techniques (CIS640)
 
Software_Testing_Overview.pptx
Software_Testing_Overview.pptxSoftware_Testing_Overview.pptx
Software_Testing_Overview.pptx
 
A Software Testing Intro
A Software Testing IntroA Software Testing Intro
A Software Testing Intro
 
Utility of Test Coverage Metrics in TDD
Utility of Test Coverage Metrics in TDDUtility of Test Coverage Metrics in TDD
Utility of Test Coverage Metrics in TDD
 
Chapter 13 software testing strategies
Chapter 13 software testing strategiesChapter 13 software testing strategies
Chapter 13 software testing strategies
 
Annotated Bibliography .Guidelines Annotated Bibliograph.docx
Annotated Bibliography  .Guidelines Annotated Bibliograph.docxAnnotated Bibliography  .Guidelines Annotated Bibliograph.docx
Annotated Bibliography .Guidelines Annotated Bibliograph.docx
 

More from Bob Binder

How to Release Rock-solid RESTful APIs and Ice the Testing BackBlob
How to Release Rock-solid RESTful APIs and Ice the Testing BackBlobHow to Release Rock-solid RESTful APIs and Ice the Testing BackBlob
How to Release Rock-solid RESTful APIs and Ice the Testing BackBlob
Bob Binder
 
Lessons learned validating 60,000 pages of api documentation
Lessons learned validating 60,000 pages of api documentationLessons learned validating 60,000 pages of api documentation
Lessons learned validating 60,000 pages of api documentation
Bob Binder
 
Model-based Testing: Taking BDD/ATDD to the Next Level
Model-based Testing: Taking BDD/ATDD to the Next LevelModel-based Testing: Taking BDD/ATDD to the Next Level
Model-based Testing: Taking BDD/ATDD to the Next Level
Bob Binder
 
Model-based Testing: Today And Tomorrow
Model-based Testing: Today And TomorrowModel-based Testing: Today And Tomorrow
Model-based Testing: Today And Tomorrow
Bob Binder
 
Mobile App Assurance: Yesterday, Today, and Tomorrow.
Mobile App Assurance: Yesterday, Today, and Tomorrow.Mobile App Assurance: Yesterday, Today, and Tomorrow.
Mobile App Assurance: Yesterday, Today, and Tomorrow.
Bob Binder
 
Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?
Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?
Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?
Bob Binder
 
MTS: Controllable Test Objects
MTS: Controllable Test ObjectsMTS: Controllable Test Objects
MTS: Controllable Test Objects
Bob Binder
 
Achieving Very High Reliability for Ubiquitous Information Technology
Achieving Very High Reliability for Ubiquitous Information Technology Achieving Very High Reliability for Ubiquitous Information Technology
Achieving Very High Reliability for Ubiquitous Information Technology
Bob Binder
 
The Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision SupportThe Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision Support
Bob Binder
 
Performance Testing Mobile and Multi-Tier Applications
Performance Testing Mobile and Multi-Tier ApplicationsPerformance Testing Mobile and Multi-Tier Applications
Performance Testing Mobile and Multi-Tier Applications
Bob Binder
 
Testing Object-Oriented Systems: Lessons Learned
Testing Object-Oriented Systems: Lessons LearnedTesting Object-Oriented Systems: Lessons Learned
Testing Object-Oriented Systems: Lessons Learned
Bob Binder
 
mVerify Investor Overview
mVerify Investor OverviewmVerify Investor Overview
mVerify Investor Overview
Bob Binder
 
Model-Based Testing: Why, What, How
Model-Based Testing: Why, What, HowModel-Based Testing: Why, What, How
Model-Based Testing: Why, What, How
Bob Binder
 
Mobile Reliability Challenges
Mobile Reliability ChallengesMobile Reliability Challenges
Mobile Reliability Challenges
Bob Binder
 
Experience with a Profile-based Automated Testing Environment
Experience with a Profile-based Automated Testing EnvironmentExperience with a Profile-based Automated Testing Environment
Experience with a Profile-based Automated Testing Environment
Bob Binder
 
Testability: Factors and Strategy
Testability: Factors and StrategyTestability: Factors and Strategy
Testability: Factors and Strategy
Bob Binder
 
Test Objects -- They Just Work
Test Objects -- They Just WorkTest Objects -- They Just Work
Test Objects -- They Just Work
Bob Binder
 
A Million Users in a Box: The WTS Story
A Million Users in a Box: The WTS StoryA Million Users in a Box: The WTS Story
A Million Users in a Box: The WTS Story
Bob Binder
 
ISSRE 2008 Trip Report
ISSRE 2008 Trip ReportISSRE 2008 Trip Report
ISSRE 2008 Trip Report
Bob Binder
 
Software Test Patterns: Successes and Challenges
Software Test Patterns: Successes and ChallengesSoftware Test Patterns: Successes and Challenges
Software Test Patterns: Successes and Challenges
Bob Binder
 

More from Bob Binder (20)

How to Release Rock-solid RESTful APIs and Ice the Testing BackBlob
How to Release Rock-solid RESTful APIs and Ice the Testing BackBlobHow to Release Rock-solid RESTful APIs and Ice the Testing BackBlob
How to Release Rock-solid RESTful APIs and Ice the Testing BackBlob
 
Lessons learned validating 60,000 pages of api documentation
Lessons learned validating 60,000 pages of api documentationLessons learned validating 60,000 pages of api documentation
Lessons learned validating 60,000 pages of api documentation
 
Model-based Testing: Taking BDD/ATDD to the Next Level
Model-based Testing: Taking BDD/ATDD to the Next LevelModel-based Testing: Taking BDD/ATDD to the Next Level
Model-based Testing: Taking BDD/ATDD to the Next Level
 
Model-based Testing: Today And Tomorrow
Model-based Testing: Today And TomorrowModel-based Testing: Today And Tomorrow
Model-based Testing: Today And Tomorrow
 
Mobile App Assurance: Yesterday, Today, and Tomorrow.
Mobile App Assurance: Yesterday, Today, and Tomorrow.Mobile App Assurance: Yesterday, Today, and Tomorrow.
Mobile App Assurance: Yesterday, Today, and Tomorrow.
 
Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?
Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?
Popular Delusions, Crowds, and the Coming Deluge: end of the Oracle?
 
MTS: Controllable Test Objects
MTS: Controllable Test ObjectsMTS: Controllable Test Objects
MTS: Controllable Test Objects
 
Achieving Very High Reliability for Ubiquitous Information Technology
Achieving Very High Reliability for Ubiquitous Information Technology Achieving Very High Reliability for Ubiquitous Information Technology
Achieving Very High Reliability for Ubiquitous Information Technology
 
The Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision SupportThe Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision Support
 
Performance Testing Mobile and Multi-Tier Applications
Performance Testing Mobile and Multi-Tier ApplicationsPerformance Testing Mobile and Multi-Tier Applications
Performance Testing Mobile and Multi-Tier Applications
 
Testing Object-Oriented Systems: Lessons Learned
Testing Object-Oriented Systems: Lessons LearnedTesting Object-Oriented Systems: Lessons Learned
Testing Object-Oriented Systems: Lessons Learned
 
mVerify Investor Overview
mVerify Investor OverviewmVerify Investor Overview
mVerify Investor Overview
 
Model-Based Testing: Why, What, How
Model-Based Testing: Why, What, HowModel-Based Testing: Why, What, How
Model-Based Testing: Why, What, How
 
Mobile Reliability Challenges
Mobile Reliability ChallengesMobile Reliability Challenges
Mobile Reliability Challenges
 
Experience with a Profile-based Automated Testing Environment
Experience with a Profile-based Automated Testing EnvironmentExperience with a Profile-based Automated Testing Environment
Experience with a Profile-based Automated Testing Environment
 
Testability: Factors and Strategy
Testability: Factors and StrategyTestability: Factors and Strategy
Testability: Factors and Strategy
 
Test Objects -- They Just Work
Test Objects -- They Just WorkTest Objects -- They Just Work
Test Objects -- They Just Work
 
A Million Users in a Box: The WTS Story
A Million Users in a Box: The WTS StoryA Million Users in a Box: The WTS Story
A Million Users in a Box: The WTS Story
 
ISSRE 2008 Trip Report
ISSRE 2008 Trip ReportISSRE 2008 Trip Report
ISSRE 2008 Trip Report
 
Software Test Patterns: Successes and Challenges
Software Test Patterns: Successes and ChallengesSoftware Test Patterns: Successes and Challenges
Software Test Patterns: Successes and Challenges
 

Recently uploaded

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neo4j
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
TIPNGVN2
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Aggregage
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Vladimir Iglovikov, Ph.D.
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 

Recently uploaded (20)

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 

MDD and the Tautology Problem: Discussion Notes.

  • 1. MDD and the Tautology Problem: Discussion Notes MoDeVVA @ MODELS 2009 October 5, 2009 Robert V. Binder
  • 2. The Tautology Effect • Transformation of model into object code is like transformation of source code into object code • An antecedent and its transformation cannot provide additional information necessary to reveal many common faults • White-box testing has well know limitations • Do these apply to implementations produced with MDD?
  • 3. White Box Testing Limitations • Cannot test missing code – can’t reveal omissions • Cannot completely infer requirements (intent) from code • Where do we get the oracle? (how to discern correct/incorrect outputs?) • Triggering of side-effects • Platform-specific behavior/interactions • Blind to interaction effects outside the scope of the unit under test • Doesn’t scale • Doesn’t support “non-functional” testing • Possible developer bias in test selection
  • 4. Weuyker: Adequacy Axioms • In terms of the antecedent (source code), what must an adequate test suite exercise? • Must at least cover every statement – What is a statement in a model? • Anti-extensionality – The semantic equality of two programs is not sufficient to imply that they should necessarily be tested the same way • Anti-decomposition – Although a program has been adequately tested, it does not necessarily imply that each of its called modules has been adequately tested. • More … • Elaine J. Weyuker, “The Evaluation Of Program-based Software Test Data Adequacy Criteria,” Communications of the ACM, v 31 n 6, June 1988.
  • 5. Manna: Limits of Verification • Verification = proof of correctness – We can never be sure that the specifications are correct – No verification system can verify every correct program – We can never be certain that a verification system is correct • Zohar Manna and Richard Waldinger, “The Logic of Computer Programming,” IEEE Transactions on Software Engineering, v SE-4, n 3, May 1978
  • 6. Some Observations • Transforming model elements to an executable – Fewer constraints than code expression to machine instructions – Many assumptions baked-in with translator – More fault opportunities • Many functionally equivalent implementations possible, suggests test strategy – M to E1 with T1, M to E2 with T2 – Run same test suite on E1 and E2 – Expect identical results for deterministic behavior – Expect bounded results for non-deterministic behavior