SlideShare a Scribd company logo
Search-Based Testing of Relational Schema
Integrity Constraints Across Multiple
Database Management Systems
Gregory M. Kapfhammer1 & Phil McMinn2 & Chris J. Wright2
1Allegheny College, USA
2University of Sheffield, UK
Sixth IEEE International Conference on Software Testing,
Verification and Validation (ICST 2013)
Tuesday, March 19, 2013
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Relational
Database
Management
Systems
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Relational
Database
Management
Systems
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Deployment Locations for Databases
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Deployment Locations for Databases
Database
Application
Server
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Deployment Locations for Databases
Database
Application
Server
Mobile Phone
or Tablet
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Deployment Locations for Databases
Database
Application
Server
Mobile Phone
or Tablet
Office and
Productivity
Software
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Deployment Locations for Databases
Database
Application
Server
Mobile Phone
or Tablet
Office and
Productivity
Software
Government
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Motivation
Databases Are Everywhere!
Deployment Locations for Databases
Database
Application
Server
Mobile Phone
or Tablet
Office and
Productivity
Software
Government Astrophysics
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateSchema
Integrity Constraints
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateSchema
Integrity Constraints
PRIMARY KEY
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateSchema
Integrity Constraints
PRIMARY KEY FOREIGN KEY
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateSchema
Integrity Constraints
PRIMARY KEY FOREIGN KEY Arbitrary CHECK
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateState
Relational Components
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateState
Relational Components
Tables
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateState
Relational Components
Tables Rows
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema StateState
Relational Components
Tables Rows Columns
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working Correctly
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working Correctly
INSERT 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working Correctly
INSERT 
Schema
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working Correctly
INSERT 
Schema State
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working Correctly
INSERT 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working Correctly
Schema
INSERT 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working Correctly
Schema
INSERT 
State
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
INSERT 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
INSERT 
Schema
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
INSERT 
Schema State
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
INSERT 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
INSERT 
Schema
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
INSERT 
Schema State
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
Schema State
SELECT 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
Schema State
SELECT SELECT 
RESULT 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Relational Database Schema
Relational Database
Management System
E-commerce
Schema State
The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly
Schema State
SELECT SELECT 
RESULT 
Not working correctly!
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Need for Relational Schema Testing
The Data Warehouse Institute reports that
North American organizations experience a
$611 billion annual loss due to poor data quality
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Need for Relational Schema Testing
The Data Warehouse Institute reports that
North American organizations experience a
$611 billion annual loss due to poor data quality
Scott W. Ambler argues that the “virtual
absence” of database testing — the validation of
the contents, schema, and functionality of the
database — is the primary cause of this loss
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Challenges
Need for Relational Schema Testing
The Data Warehouse Institute reports that
North American organizations experience a
$611 billion annual loss due to poor data quality
Scott W. Ambler argues that the “virtual
absence” of database testing — the validation of
the contents, schema, and functionality of the
database — is the primary cause of this loss
This paper presents SchemaAnalyst, a
search-based system for testing the complex
integrity constraints in relational schemas
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
The highlighted integrity constraints determine what data is valid
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
The highlighted integrity constraints determine what data is valid
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
The highlighted integrity constraints determine what data is valid
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
The highlighted integrity constraints determine what data is valid
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
The highlighted integrity constraints determine what data is valid
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
The highlighted integrity constraints determine what data is valid
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
The highlighted integrity constraints determine what data is valid
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
The highlighted integrity constraints determine what data is valid
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
Defect: The schema does not contain the correct primary key!
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
Defect: The schema does not contain the correct primary key!
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
Defect: The schema does not contain the correct primary key!
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Question: What kind of INSERT(s) will reveal this defect?
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
Question: What kind of INSERT(s) will reveal this defect?
INSERT INTO Flights
VALUES(’UA20’, 1, ... ) 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
Question: What kind of INSERT(s) will reveal this defect?
INSERT INTO Flights
VALUES(’UA20’, 1, ... ) 
INSERT INTO Flights
VALUES(’UA20’, 2, ... ) 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
Question: What kind of INSERT(s) will reveal this defect?
INSERT INTO Flights
VALUES(’UA20’, 1, ... ) 
INSERT INTO Flights
VALUES(’UA20’, 2, ... ) 
Explanation: A flight with two different segments is no longer allowed!
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Defects in Relational Schemas
Question: What kind of INSERT(s) will reveal this defect?
INSERT INTO Flights
VALUES(’UA20’, 1, ... ) 
INSERT INTO Flights
VALUES(’UA20’, 2, ... ) 
Explanation: A flight with two different segments is no longer allowed!
SchemaAnalyst automatically generates these INSERTs and this data!
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Search-Based Testing with SchemaAnalyst
Schema
Representation
Generator
Test Suite
Generator
Mutation
Analysis
Test Suites
Mutants
and
Scores
Test suites
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Search-Based Testing with SchemaAnalyst
Schema
Representation
Generator
Test Suite
Generator
Mutation
Analysis
Test Suites
Mutants
and
Scores
Test suites
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Search-Based Testing with SchemaAnalyst
Schema
Representation
Generator
Test Suite
Generator
Mutation
Analysis
Test Suites
Mutants
and
Scores
Test suites
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Search-Based Testing with SchemaAnalyst
Schema
Representation
Generator
Test Suite
Generator
Mutation
Analysis
Test Suites
Mutants
and
Scores
Test suites
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
Goal of test data generation?
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
Goal of test data generation?
INSERT INTO T1 VALUES(1, Jan-08-99, ... ) 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
Goal of test data generation?
INSERT INTO T1 VALUES(1, Jan-08-99, ... ) 
INSERT INTO T1 VALUES(1, Jan-08-99, ... ) 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
Goal of test data generation?
INSERT INTO T1 VALUES(1, Jan-08-99, ... ) 
INSERT INTO T1 VALUES(1, Jan-08-99, ... ) 
INSERT INTO Tn VALUES(true, ’L-20’, ... ) 
INSERT INTO Tn VALUES(false, ’L-1’, ... ) 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Stage 1: Generate rows of data to satisfy the integrity constraints
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
Stage 1: Generate rows of data to satisfy the integrity constraints
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Stage 2: Generate rows of data to negate a constraint
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
Stage 2: Generate rows of data to negate a constraint
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
A fitness function computes a numeric value minimized by search
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Data’s fitness is closer to zero when nearer to a primary key value
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Types, primary and foreign keys, UNIQUE, NOT NULL, and CHECK
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Goals and Stages of Test Data Generation
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
See the paper for more details about the computation of fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use the defaults to form the initial values of the INSERT variables
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use exploratory moves to determine the correct direction for search
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use exploratory moves to determine the correct direction for search
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use exploratory moves to determine the correct direction for search
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use exploratory moves to determine the correct direction for search
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use exploratory moves to determine the correct direction for search
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use exploratory moves to determine the correct direction for search
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitness
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitnessAVM terminates when the fitness is zero or an exploration cycle fails
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Test Data Generation
Alternating Variable Method
Vi
Vj
Vk
Use pattern moves to accelerate the improvements in fitnessAVM terminates when the fitness is zero or an exploration cycle failsRestart AVM with random column values when an exploration cycle fails
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Use mutation analysis to assess the adequacy of INSERTs and values
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Primary Keys: Remove, replace, and add column operators
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
Primary Keys: Remove, replace, and add column operators
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
Primary Keys: Remove, replace, and add column operators
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
Primary Keys: Remove, replace, and add column operators
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(ORIGINAL AIRPORT, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
Primary Keys: Remove, replace, and add column operators
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
Primary Keys: Remove, replace, and add column operators
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER, DEST AIRPORT),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
UNIQUE: Handle in a fashion similar to the primary key operator
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
NOT NULL: Reverse the status for all non-primary key columns
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
NOT NULL: Reverse the status for all non-primary key columns
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
NOT NULL: Reverse the status for all non-primary key columns
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3) NOT NULL,
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
CHECK: Remove the constraint for each of the checked columns
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
CHECK: Remove the constraint for each of the checked columns
CREATE TABLE Flights(
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
ORIGINAL AIRPORT CHAR(3),
DEPART TIME TIME,
DEST AIRPORT CHAR(3),
ARRIVE TIME TIME,
MEAL CHAR(1),
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’))
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Foreign Keys: Remove each column from the key
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
Foreign Keys: Remove each column from the key
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Mutation Operators for Schemas
Foreign Keys: Remove each column from the key
CREATE TABLE FlightAvailable (
FLIGHT ID CHAR(6) NOT NULL,
SEGMENT NUMBER INT NOT NULL,
FLIGHT DATE DATE NOT NULL,
ECONOMY SEATS TAKEN INT,
BUSINESS SEATS TAKEN INT,
FIRSTCLASS SEATS TAKEN INT,
PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER),
FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER)
REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER)
);
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|

Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Relational Schema Mutation
Calculating the Mutation Score
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
MD =
|K ∪ Q|
|K ∪ N|
 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
BankAccount 2 9 0 1 5 2 0 8
BookTown 23 69 1 0 17 11 0 29
Cloc 2 10 0 0 0 0 0 0
CoffeeOrders 5 20 0 4 9 5 0 18
CustomerOrder 7 32 1 7 27 7 0 42
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
BankAccount 2 9 0 1 5 2 0 8
BookTown 23 69 1 0 17 11 0 29
Cloc 2 10 0 0 0 0 0 0
CoffeeOrders 5 20 0 4 9 5 0 18
CustomerOrder 7 32 1 7 27 7 0 42
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
BankAccount 2 9 0 1 5 2 0 8
BookTown 23 69 1 0 17 11 0 29
Cloc 2 10 0 0 0 0 0 0
CoffeeOrders 5 20 0 4 9 5 0 18
CustomerOrder 7 32 1 7 27 7 0 42
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
BankAccount 2 9 0 1 5 2 0 8
BookTown 23 69 1 0 17 11 0 29
Cloc 2 10 0 0 0 0 0 0
CoffeeOrders 5 20 0 4 9 5 0 18
CustomerOrder 7 32 1 7 27 7 0 42
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
BankAccount 2 9 0 1 5 2 0 8
BookTown 23 69 1 0 17 11 0 29
Cloc 2 10 0 0 0 0 0 0
CoffeeOrders 5 20 0 4 9 5 0 18
CustomerOrder 7 32 1 7 27 7 0 42
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
BankAccount 2 9 0 1 5 2 0 8
BookTown 23 69 1 0 17 11 0 29
Cloc 2 10 0 0 0 0 0 0
CoffeeOrders 5 20 0 4 9 5 0 18
CustomerOrder 7 32 1 7 27 7 0 42
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
BankAccount 2 9 0 1 5 2 0 8
BookTown 23 69 1 0 17 11 0 29
Cloc 2 10 0 0 0 0 0 0
CoffeeOrders 5 20 0 4 9 5 0 18
CustomerOrder 7 32 1 7 27 7 0 42
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
DellStore 8 52 0 0 36 0 0 36
Employee 1 7 3 0 0 1 0 4
Examination 2 21 6 1 0 2 0 9
Flights 2 13 1 1 6 2 0 10
FrenchTowns 3 14 0 2 13 0 8 23
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
DellStore 8 52 0 0 36 0 0 36
Employee 1 7 3 0 0 1 0 4
Examination 2 21 6 1 0 2 0 9
Flights 2 13 1 1 6 2 0 10
FrenchTowns 3 14 0 2 13 0 8 23
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
Inventory 1 4 0 0 0 1 1 2
Iso3166 1 3 0 0 2 1 0 3
JWhoisServer 6 49 0 0 44 6 0 50
NistDML181 2 7 0 1 0 1 0 2
NistDML182 2 32 0 1 0 1 0 2
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
Inventory 1 4 0 0 0 1 1 2
Iso3166 1 3 0 0 2 1 0 3
JWhoisServer 6 49 0 0 44 6 0 50
NistDML181 2 7 0 1 0 1 0 2
NistDML182 2 32 0 1 0 1 0 2
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
Inventory 1 4 0 0 0 1 1 2
Iso3166 1 3 0 0 2 1 0 3
JWhoisServer 6 49 0 0 44 6 0 50
NistDML181 2 7 0 1 0 1 0 2
NistDML182 2 32 0 1 0 1 0 2
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
NistDML183 2 6 0 1 0 0 1 2
NistWeather 2 9 5 0 2 2 0 9
NistXTS748 1 3 1 0 1 0 1 3
NistXTS749 2 7 1 1 3 2 0 7
Person 1 5 1 0 5 1 0 7
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
Products 3 9 4 2 5 3 0 14
Residence 2 6 3 1 2 2 0 8
RiskIt 13 56 0 10 15 11 0 36
UnixUsage 8 32 0 7 9 7 0 23
Usda 10 67 0 0 30 0 0 30
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
Products 3 9 4 2 5 3 0 14
Residence 2 6 3 1 2 2 0 8
RiskIt 13 56 0 10 15 11 0 36
UnixUsage 8 32 0 7 9 7 0 23
Usda 10 67 0 0 30 0 0 30
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Schema
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
Products 3 9 4 2 5 3 0 14
Residence 2 6 3 1 2 2 0 8
RiskIt 13 56 0 10 15 11 0 36
UnixUsage 8 32 0 7 9 7 0 23
Usda 10 67 0 0 30 0 0 30
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Case Study Schemas
Tables
Columns
Checks
Foreignkeys
NotNulls
Primarykeys
Uniques
TotalConstraints
Totals 111 542 27 40 231 68 11 377
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Data Generation Techniques
DBMonster
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Data Generation Techniques
DBMonster SchemaAnalyst
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Data Generation Techniques
DBMonster SchemaAnalyst
HSQLDB 
SQLite 
Postgres 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Data Generation Techniques
DBMonster SchemaAnalyst
HSQLDB 
SQLite 
Postgres 
HSQLDB 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Data Generation Techniques
DBMonster SchemaAnalyst
HSQLDB 
SQLite 
Postgres 
HSQLDB 
SQLite 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Configuration
Data Generation Techniques
DBMonster SchemaAnalyst
HSQLDB 
SQLite 
Postgres 
HSQLDB 
SQLite 
Postgres 
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
Flights 100.0 70.0
FrenchTowns 100.0 70.0
Inventory 100.0 75.0
Iso3166 100.0 50.0
JWhoisServer 100.0 50.0
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
Flights 100.0 70.0
FrenchTowns 100.0 70.0
Inventory 100.0 75.0
Iso3166 100.0 50.0
JWhoisServer 100.0 50.0
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
NistDML181 100.0 75.0
NistDML182 100.0 50.0
NistDML183 100.0 100.0
NistXTS748 100.0 72.2
NistXTS749 100.0 21.4
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
NistDML181 100.0 75.0
NistDML182 100.0 50.0
NistDML183 100.0 100.0
NistXTS748 100.0 72.2
NistXTS749 100.0 21.4
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
NistDML181 100.0 75.0
NistDML182 100.0 50.0
NistDML183 100.0 100.0
NistXTS748 100.0 72.2
NistXTS749 100.0 21.4
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
Residence 100.0 62.5
RiskIt 100.0 4.1
Products 96.4 59.3
UnixUsage 97.8 59.3
Usda 100.0 50.0
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
Residence 100.0 62.5
RiskIt 100.0 4.1
Products 96.4 59.3
UnixUsage 97.8 59.3
Usda 100.0 50.0
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
Residence 100.0 62.5
RiskIt 100.0 4.1
Products 96.4 59.3
UnixUsage 97.8 59.3
Usda 100.0 50.0
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Constraint Coverage Results
Schema AVM (%) DBMonster (%)
Residence 100.0 62.5
RiskIt 100.0 4.1
Products 96.4 59.3
UnixUsage 97.8 59.3
Usda 100.0 50.0
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Quasi-Mutant Results
Number of Mutants
Hsqldb
Postgres
SQLite
0 20 40 60 80 100 120
CustomerOrder
Non−Quasi Quasi
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Quasi-Mutant Results
Number of Mutants
Hsqldb
Postgres
SQLite
0 50 100 150
JWhoisServer
Non−Quasi Quasi
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Quasi-Mutant Results
Number of Mutants
Hsqldb
Postgres
SQLite
0 50 100 150
DellStore
Non−Quasi Quasi
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Summary: Quasi-Mutant Results
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Summary: Quasi-Mutant Results
None SomeSome
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Summary: Quasi-Mutant Results
None SomeSome
Few quasi-mutants means that the mutation
scores are good effectiveness indicators
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
JWhoisServer DBI=62, MD = 0.7DBI=300, MD = 0.2
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
JWhoisServer DBI=62, MD = 0.7DBI=300, MD = 0.2
NistDML181 DBI=7, MD = 0.6DBI=13,650, MD = 0.5
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
(0.29, 0.59, 0.65, 0.70, 0.89)
(0.0, 0.11, 0.41, 0.52, 0.68)
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
(0.29, 0.59, 0.65, 0.70, 0.89)
(0.0, 0.11, 0.41, 0.52, 0.68)
DBMonster crashes
for six schemas!
CustomerOrder
Flights
NistDML182
NistXTS748
Person
RiskIt
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
(0.29, 0.59, 0.65, 0.70, 0.89)
(0.0, 0.11, 0.41, 0.52, 0.68)
DBMonster crashes
for six schemas!
CustomerOrder
Flights
NistDML182
NistXTS748
Person
RiskIt
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
(0.29, 0.59, 0.65, 0.70, 0.89)
(0.0, 0.11, 0.41, 0.52, 0.68)
DBMonster crashes
for six schemas!
CustomerOrder
Flights
NistDML182
NistXTS748
Person
RiskIt
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
(0.29, 0.59, 0.65, 0.70, 0.89)
(0.0, 0.11, 0.41, 0.52, 0.68)
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
(0.29, 0.59, 0.65, 0.70, 0.89)
(0.0, 0.11, 0.41, 0.52, 0.68)
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Mutation Score Results
DBMonster SchemaAnalyst
SchemaAnalyst’s
mutation score is
higher than DB-
Monster’s for 96%
of the schemas
(0.29, 0.59, 0.65, 0.70, 0.89)
(0.0, 0.11, 0.41, 0.52, 0.68)
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Efficiency Results
DBMonster SchemaAnalyst
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Efficiency Results
DBMonster SchemaAnalyst
(0.41, 1.09, 1.90, 5.07, 36.52)
(1.50, 3.01, 5.21, 16.79, 639.93)
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Efficiency Results
DBMonster SchemaAnalyst
(0.41, 1.09, 1.90, 5.07, 36.52)
(1.50, 3.01, 5.21, 16.79, 639.93)
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Results Analysis
Efficiency Results
DBMonster SchemaAnalyst
(0.41, 1.09, 1.90, 5.07, 36.52)
(1.50, 3.01, 5.21, 16.79, 639.93)
SchemaAnalyst
exhibits competi-
tive data genera-
tion times that are
less variable
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Summary
Important Contributions
This paper presents SchemaAnalyst, a
search-based system for testing the complex
integrity constraints in relational schemas
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Summary
Important Contributions
This paper presents SchemaAnalyst, a
search-based system for testing the complex
integrity constraints in relational schemas
The empirical study demonstrates that Schema-
Analyst’s efficiency is competitive with DBMonster’s
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Summary
Important Contributions
This paper presents SchemaAnalyst, a
search-based system for testing the complex
integrity constraints in relational schemas
The empirical study demonstrates that Schema-
Analyst’s efficiency is competitive with DBMonster’s
SchemaAnalyst almost always covers 100% of the
constraints in the 25 chosen relational schemas
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Summary
Important Contributions
This paper presents SchemaAnalyst, a
search-based system for testing the complex
integrity constraints in relational schemas
The empirical study demonstrates that Schema-
Analyst’s efficiency is competitive with DBMonster’s
SchemaAnalyst almost always covers 100% of the
constraints in the 25 chosen relational schemas
SchemaAnalyst’s mutation score is higher
than DBMonster’s for 96% of the schemas
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
Introduction Testing Technique Empirical Study Conclusion
Summary
Important Contributions
This paper presents SchemaAnalyst, a
search-based system for testing the complex
integrity constraints in relational schemas
The empirical study demonstrates that Schema-
Analyst’s efficiency is competitive with DBMonster’s
SchemaAnalyst almost always covers 100% of the
constraints in the 25 chosen relational schemas
SchemaAnalyst’s mutation score is higher
than DBMonster’s for 96% of the schemas
http://www.schemaanalyst.org
Kapfhammer, McMinn, and Wright March 19, 2013
Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems

More Related Content

Viewers also liked

NoSQL
NoSQLNoSQL
Normalization
NormalizationNormalization
Sql Injection Myths and Fallacies
Sql Injection Myths and FallaciesSql Injection Myths and Fallacies
Sql Injection Myths and Fallacies
Karwin Software Solutions LLC
 
Database Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NF
Database Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NFDatabase Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NF
Database Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NF
Oum Saokosal
 
ER model to Relational model mapping
ER model to Relational model mappingER model to Relational model mapping
ER model to Relational model mapping
Shubham Saini
 
ER DIAGRAM TO RELATIONAL SCHEMA MAPPING
ER DIAGRAM TO RELATIONAL SCHEMA MAPPING ER DIAGRAM TO RELATIONAL SCHEMA MAPPING
ER DIAGRAM TO RELATIONAL SCHEMA MAPPING
ARADHYAYANA
 
Erd examples
Erd examplesErd examples
Erd examples
Pramod Redekar
 

Viewers also liked (7)

NoSQL
NoSQLNoSQL
NoSQL
 
Normalization
NormalizationNormalization
Normalization
 
Sql Injection Myths and Fallacies
Sql Injection Myths and FallaciesSql Injection Myths and Fallacies
Sql Injection Myths and Fallacies
 
Database Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NF
Database Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NFDatabase Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NF
Database Normalization 1NF, 2NF, 3NF, BCNF, 4NF, 5NF
 
ER model to Relational model mapping
ER model to Relational model mappingER model to Relational model mapping
ER model to Relational model mapping
 
ER DIAGRAM TO RELATIONAL SCHEMA MAPPING
ER DIAGRAM TO RELATIONAL SCHEMA MAPPING ER DIAGRAM TO RELATIONAL SCHEMA MAPPING
ER DIAGRAM TO RELATIONAL SCHEMA MAPPING
 
Erd examples
Erd examplesErd examples
Erd examples
 

Similar to Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems

ppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvy
ppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvyppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvy
ppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvy
vk5985399
 
Zachman Framework vs Data Architecture
Zachman Framework vs Data ArchitectureZachman Framework vs Data Architecture
Zachman Framework vs Data Architecture
Carol Harstad
 
Improving the search mechanism for unstructured peer to-peer networks using t...
Improving the search mechanism for unstructured peer to-peer networks using t...Improving the search mechanism for unstructured peer to-peer networks using t...
Improving the search mechanism for unstructured peer to-peer networks using t...
Aditya Kumar
 
00.00 fundamentals of database management syllabus
00.00 fundamentals of database management syllabus00.00 fundamentals of database management syllabus
00.00 fundamentals of database management syllabus
Bishal Ghimire
 
M phil-computer-science-data-mining-projects
M phil-computer-science-data-mining-projectsM phil-computer-science-data-mining-projects
M phil-computer-science-data-mining-projects
Vijay Karan
 
M.Phil Computer Science Data Mining Projects
M.Phil Computer Science Data Mining ProjectsM.Phil Computer Science Data Mining Projects
M.Phil Computer Science Data Mining Projects
Vijay Karan
 
t2_4-architecting-data-for-integration-and-longevity
t2_4-architecting-data-for-integration-and-longevityt2_4-architecting-data-for-integration-and-longevity
t2_4-architecting-data-for-integration-and-longevity
Jonathan Hamilton Solórzano
 
Tg03
Tg03Tg03
Tg03
kelasapa
 
Introduction to question answering for linked data & big data
Introduction to question answering for linked data & big dataIntroduction to question answering for linked data & big data
Introduction to question answering for linked data & big data
Andre Freitas
 
M.E Computer Science Data Mining Projects
M.E Computer Science Data Mining ProjectsM.E Computer Science Data Mining Projects
M.E Computer Science Data Mining Projects
Vijay Karan
 
IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...
IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...
IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...
IEEEFINALYEARSTUDENTPROJECTS
 
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
IEEEMEMTECHSTUDENTSPROJECTS
 
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
IEEEFINALYEARSTUDENTPROJECT
 
Patel
PatelPatel
QUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASES
QUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASESQUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASES
QUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASES
Nexgen Technology
 
Data models
Data modelsData models
Data models
Usman Tariq
 
Sub1582
Sub1582Sub1582
-linkedin
-linkedin-linkedin
Spe165 t
Spe165 tSpe165 t
Spe165 t
Rajesh War
 
GCCOM\_DART: Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...
GCCOM\_DART:  Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...GCCOM\_DART:  Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...
GCCOM\_DART: Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...
Mariangel (Angie) Garcia, Ph.D
 

Similar to Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems (20)

ppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvy
ppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvyppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvy
ppt_rdbms.pdfuvuguvuvugycycyctcucuvyvvuvuvy
 
Zachman Framework vs Data Architecture
Zachman Framework vs Data ArchitectureZachman Framework vs Data Architecture
Zachman Framework vs Data Architecture
 
Improving the search mechanism for unstructured peer to-peer networks using t...
Improving the search mechanism for unstructured peer to-peer networks using t...Improving the search mechanism for unstructured peer to-peer networks using t...
Improving the search mechanism for unstructured peer to-peer networks using t...
 
00.00 fundamentals of database management syllabus
00.00 fundamentals of database management syllabus00.00 fundamentals of database management syllabus
00.00 fundamentals of database management syllabus
 
M phil-computer-science-data-mining-projects
M phil-computer-science-data-mining-projectsM phil-computer-science-data-mining-projects
M phil-computer-science-data-mining-projects
 
M.Phil Computer Science Data Mining Projects
M.Phil Computer Science Data Mining ProjectsM.Phil Computer Science Data Mining Projects
M.Phil Computer Science Data Mining Projects
 
t2_4-architecting-data-for-integration-and-longevity
t2_4-architecting-data-for-integration-and-longevityt2_4-architecting-data-for-integration-and-longevity
t2_4-architecting-data-for-integration-and-longevity
 
Tg03
Tg03Tg03
Tg03
 
Introduction to question answering for linked data & big data
Introduction to question answering for linked data & big dataIntroduction to question answering for linked data & big data
Introduction to question answering for linked data & big data
 
M.E Computer Science Data Mining Projects
M.E Computer Science Data Mining ProjectsM.E Computer Science Data Mining Projects
M.E Computer Science Data Mining Projects
 
IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...
IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...
IEEE 2014 JAVA DATA MINING PROJECTS Mining weakly labeled web facial images f...
 
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
 
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
2014 IEEE JAVA DATA MINING PROJECT Mining weakly labeled web facial images fo...
 
Patel
PatelPatel
Patel
 
QUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASES
QUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASESQUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASES
QUALITY-AWARE SUBGRAPH MATCHING OVER INCONSISTENT PROBABILISTIC GRAPH DATABASES
 
Data models
Data modelsData models
Data models
 
Sub1582
Sub1582Sub1582
Sub1582
 
-linkedin
-linkedin-linkedin
-linkedin
 
Spe165 t
Spe165 tSpe165 t
Spe165 t
 
GCCOM\_DART: Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...
GCCOM\_DART:  Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...GCCOM\_DART:  Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...
GCCOM\_DART: Ensemble Data Assimilation Analysis System for Sub-mesoscale Pr...
 

Recently uploaded

Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
TheSMSPoint
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
timtebeek1
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Aftab Hussain
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Remote DBA Services
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain SecurityopenEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
Shane Coughlan
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
XfilesPro
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
Quickdice ERP
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Neo4j
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
Łukasz Chruściel
 
SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
GohKiangHock
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
Deuglo Infosystem Pvt Ltd
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
Remote DBA Services
 

Recently uploaded (20)

Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain SecurityopenEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
 
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
 
SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
 

Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems

  • 1. Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems Gregory M. Kapfhammer1 & Phil McMinn2 & Chris J. Wright2 1Allegheny College, USA 2University of Sheffield, UK Sixth IEEE International Conference on Software Testing, Verification and Validation (ICST 2013) Tuesday, March 19, 2013
  • 2. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Relational Database Management Systems Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 3. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Relational Database Management Systems Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 4. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Deployment Locations for Databases Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 5. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Deployment Locations for Databases Database Application Server Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 6. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Deployment Locations for Databases Database Application Server Mobile Phone or Tablet Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 7. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Deployment Locations for Databases Database Application Server Mobile Phone or Tablet Office and Productivity Software Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 8. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Deployment Locations for Databases Database Application Server Mobile Phone or Tablet Office and Productivity Software Government Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 9. Introduction Testing Technique Empirical Study Conclusion Motivation Databases Are Everywhere! Deployment Locations for Databases Database Application Server Mobile Phone or Tablet Office and Productivity Software Government Astrophysics Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 10. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 11. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 12. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 13. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 14. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateSchema Integrity Constraints Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 15. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateSchema Integrity Constraints PRIMARY KEY Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 16. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateSchema Integrity Constraints PRIMARY KEY FOREIGN KEY Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 17. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateSchema Integrity Constraints PRIMARY KEY FOREIGN KEY Arbitrary CHECK Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 18. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateState Relational Components Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 19. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateState Relational Components Tables Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 20. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateState Relational Components Tables Rows Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 21. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema StateState Relational Components Tables Rows Columns Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 22. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working Correctly Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 23. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working Correctly INSERT Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 24. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working Correctly INSERT Schema Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 25. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working Correctly INSERT Schema State Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 26. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working Correctly INSERT Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 27. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working Correctly Schema INSERT Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 28. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working Correctly Schema INSERT State Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 29. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 30. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly INSERT Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 31. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly INSERT Schema Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 32. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly INSERT Schema State Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 33. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly INSERT Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 34. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly INSERT Schema Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 35. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly INSERT Schema State Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 36. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly Schema State SELECT Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 37. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly Schema State SELECT SELECT RESULT Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 38. Introduction Testing Technique Empirical Study Conclusion Challenges Relational Database Schema Relational Database Management System E-commerce Schema State The Relational Schema is Working CorrectlyThe Relational Schema is Not Working Correctly Schema State SELECT SELECT RESULT Not working correctly! Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 39. Introduction Testing Technique Empirical Study Conclusion Challenges Need for Relational Schema Testing The Data Warehouse Institute reports that North American organizations experience a $611 billion annual loss due to poor data quality Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 40. Introduction Testing Technique Empirical Study Conclusion Challenges Need for Relational Schema Testing The Data Warehouse Institute reports that North American organizations experience a $611 billion annual loss due to poor data quality Scott W. Ambler argues that the “virtual absence” of database testing — the validation of the contents, schema, and functionality of the database — is the primary cause of this loss Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 41. Introduction Testing Technique Empirical Study Conclusion Challenges Need for Relational Schema Testing The Data Warehouse Institute reports that North American organizations experience a $611 billion annual loss due to poor data quality Scott W. Ambler argues that the “virtual absence” of database testing — the validation of the contents, schema, and functionality of the database — is the primary cause of this loss This paper presents SchemaAnalyst, a search-based system for testing the complex integrity constraints in relational schemas Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 42. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 43. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); The highlighted integrity constraints determine what data is valid Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 44. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas The highlighted integrity constraints determine what data is valid CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 45. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas The highlighted integrity constraints determine what data is valid CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 46. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas The highlighted integrity constraints determine what data is valid CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 47. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas The highlighted integrity constraints determine what data is valid CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 48. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas The highlighted integrity constraints determine what data is valid CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 49. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas The highlighted integrity constraints determine what data is valid CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 50. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas The highlighted integrity constraints determine what data is valid CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 51. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas Defect: The schema does not contain the correct primary key! Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 52. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas Defect: The schema does not contain the correct primary key! CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 53. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas Defect: The schema does not contain the correct primary key! CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 54. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Question: What kind of INSERT(s) will reveal this defect? Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 55. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas Question: What kind of INSERT(s) will reveal this defect? INSERT INTO Flights VALUES(’UA20’, 1, ... ) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 56. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas Question: What kind of INSERT(s) will reveal this defect? INSERT INTO Flights VALUES(’UA20’, 1, ... ) INSERT INTO Flights VALUES(’UA20’, 2, ... ) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 57. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas Question: What kind of INSERT(s) will reveal this defect? INSERT INTO Flights VALUES(’UA20’, 1, ... ) INSERT INTO Flights VALUES(’UA20’, 2, ... ) Explanation: A flight with two different segments is no longer allowed! Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 58. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Defects in Relational Schemas Question: What kind of INSERT(s) will reveal this defect? INSERT INTO Flights VALUES(’UA20’, 1, ... ) INSERT INTO Flights VALUES(’UA20’, 2, ... ) Explanation: A flight with two different segments is no longer allowed! SchemaAnalyst automatically generates these INSERTs and this data! Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 59. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Search-Based Testing with SchemaAnalyst Schema Representation Generator Test Suite Generator Mutation Analysis Test Suites Mutants and Scores Test suites Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 60. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Search-Based Testing with SchemaAnalyst Schema Representation Generator Test Suite Generator Mutation Analysis Test Suites Mutants and Scores Test suites Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 61. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Search-Based Testing with SchemaAnalyst Schema Representation Generator Test Suite Generator Mutation Analysis Test Suites Mutants and Scores Test suites Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 62. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Search-Based Testing with SchemaAnalyst Schema Representation Generator Test Suite Generator Mutation Analysis Test Suites Mutants and Scores Test suites Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 63. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation Goal of test data generation? Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 64. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation Goal of test data generation? INSERT INTO T1 VALUES(1, Jan-08-99, ... ) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 65. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation Goal of test data generation? INSERT INTO T1 VALUES(1, Jan-08-99, ... ) INSERT INTO T1 VALUES(1, Jan-08-99, ... ) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 66. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation Goal of test data generation? INSERT INTO T1 VALUES(1, Jan-08-99, ... ) INSERT INTO T1 VALUES(1, Jan-08-99, ... ) INSERT INTO Tn VALUES(true, ’L-20’, ... ) INSERT INTO Tn VALUES(false, ’L-1’, ... ) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 67. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 68. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Stage 1: Generate rows of data to satisfy the integrity constraints Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 69. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation Stage 1: Generate rows of data to satisfy the integrity constraints CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 70. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 71. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Stage 2: Generate rows of data to negate a constraint Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 72. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation Stage 2: Generate rows of data to negate a constraint CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 73. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); A fitness function computes a numeric value minimized by search Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 74. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Data’s fitness is closer to zero when nearer to a primary key value Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 75. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Types, primary and foreign keys, UNIQUE, NOT NULL, and CHECK Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 76. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Goals and Stages of Test Data Generation CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); See the paper for more details about the computation of fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 77. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 78. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 79. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 80. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use the defaults to form the initial values of the INSERT variables Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 81. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use exploratory moves to determine the correct direction for search Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 82. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use exploratory moves to determine the correct direction for search Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 83. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use exploratory moves to determine the correct direction for search Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 84. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use exploratory moves to determine the correct direction for search Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 85. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use exploratory moves to determine the correct direction for search Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 86. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use exploratory moves to determine the correct direction for search Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 87. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 88. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 89. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 90. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 91. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 92. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 93. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 94. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitness Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 95. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitnessAVM terminates when the fitness is zero or an exploration cycle fails Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 96. Introduction Testing Technique Empirical Study Conclusion Test Data Generation Alternating Variable Method Vi Vj Vk Use pattern moves to accelerate the improvements in fitnessAVM terminates when the fitness is zero or an exploration cycle failsRestart AVM with random column values when an exploration cycle fails Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 97. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 98. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Use mutation analysis to assess the adequacy of INSERTs and values Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 99. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Primary Keys: Remove, replace, and add column operators Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 100. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas Primary Keys: Remove, replace, and add column operators CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 101. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas Primary Keys: Remove, replace, and add column operators CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 102. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas Primary Keys: Remove, replace, and add column operators CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(ORIGINAL AIRPORT, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 103. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas Primary Keys: Remove, replace, and add column operators CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 104. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas Primary Keys: Remove, replace, and add column operators CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER, DEST AIRPORT), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 105. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas UNIQUE: Handle in a fashion similar to the primary key operator CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 106. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); NOT NULL: Reverse the status for all non-primary key columns Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 107. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas NOT NULL: Reverse the status for all non-primary key columns CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 108. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas NOT NULL: Reverse the status for all non-primary key columns CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3) NOT NULL, DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 109. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas CHECK: Remove the constraint for each of the checked columns CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 110. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas CHECK: Remove the constraint for each of the checked columns CREATE TABLE Flights( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, ORIGINAL AIRPORT CHAR(3), DEPART TIME TIME, DEST AIRPORT CHAR(3), ARRIVE TIME TIME, MEAL CHAR(1), PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), CHECK(MEAL IN (’B’, ’L’, ’D’, ’S’)) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 111. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Foreign Keys: Remove each column from the key Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 112. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas Foreign Keys: Remove each column from the key CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 113. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Mutation Operators for Schemas Foreign Keys: Remove each column from the key CREATE TABLE FlightAvailable ( FLIGHT ID CHAR(6) NOT NULL, SEGMENT NUMBER INT NOT NULL, FLIGHT DATE DATE NOT NULL, ECONOMY SEATS TAKEN INT, BUSINESS SEATS TAKEN INT, FIRSTCLASS SEATS TAKEN INT, PRIMARY KEY(FLIGHT ID, SEGMENT NUMBER), FOREIGN KEY(FLIGHT ID, SEGMENT NUMBER) REFERENCES Flights(FLIGHT ID, SEGMENT NUMBER) ); Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 114. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 115. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 116. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 117. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 118. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 119. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 120. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 121. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 122. Introduction Testing Technique Empirical Study Conclusion Relational Schema Mutation Calculating the Mutation Score MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| MD = |K ∪ Q| |K ∪ N| Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 123. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints BankAccount 2 9 0 1 5 2 0 8 BookTown 23 69 1 0 17 11 0 29 Cloc 2 10 0 0 0 0 0 0 CoffeeOrders 5 20 0 4 9 5 0 18 CustomerOrder 7 32 1 7 27 7 0 42 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 124. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints BankAccount 2 9 0 1 5 2 0 8 BookTown 23 69 1 0 17 11 0 29 Cloc 2 10 0 0 0 0 0 0 CoffeeOrders 5 20 0 4 9 5 0 18 CustomerOrder 7 32 1 7 27 7 0 42 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 125. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints BankAccount 2 9 0 1 5 2 0 8 BookTown 23 69 1 0 17 11 0 29 Cloc 2 10 0 0 0 0 0 0 CoffeeOrders 5 20 0 4 9 5 0 18 CustomerOrder 7 32 1 7 27 7 0 42 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 126. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints BankAccount 2 9 0 1 5 2 0 8 BookTown 23 69 1 0 17 11 0 29 Cloc 2 10 0 0 0 0 0 0 CoffeeOrders 5 20 0 4 9 5 0 18 CustomerOrder 7 32 1 7 27 7 0 42 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 127. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints BankAccount 2 9 0 1 5 2 0 8 BookTown 23 69 1 0 17 11 0 29 Cloc 2 10 0 0 0 0 0 0 CoffeeOrders 5 20 0 4 9 5 0 18 CustomerOrder 7 32 1 7 27 7 0 42 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 128. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints BankAccount 2 9 0 1 5 2 0 8 BookTown 23 69 1 0 17 11 0 29 Cloc 2 10 0 0 0 0 0 0 CoffeeOrders 5 20 0 4 9 5 0 18 CustomerOrder 7 32 1 7 27 7 0 42 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 129. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints BankAccount 2 9 0 1 5 2 0 8 BookTown 23 69 1 0 17 11 0 29 Cloc 2 10 0 0 0 0 0 0 CoffeeOrders 5 20 0 4 9 5 0 18 CustomerOrder 7 32 1 7 27 7 0 42 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 130. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints DellStore 8 52 0 0 36 0 0 36 Employee 1 7 3 0 0 1 0 4 Examination 2 21 6 1 0 2 0 9 Flights 2 13 1 1 6 2 0 10 FrenchTowns 3 14 0 2 13 0 8 23 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 131. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints DellStore 8 52 0 0 36 0 0 36 Employee 1 7 3 0 0 1 0 4 Examination 2 21 6 1 0 2 0 9 Flights 2 13 1 1 6 2 0 10 FrenchTowns 3 14 0 2 13 0 8 23 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 132. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints Inventory 1 4 0 0 0 1 1 2 Iso3166 1 3 0 0 2 1 0 3 JWhoisServer 6 49 0 0 44 6 0 50 NistDML181 2 7 0 1 0 1 0 2 NistDML182 2 32 0 1 0 1 0 2 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 133. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints Inventory 1 4 0 0 0 1 1 2 Iso3166 1 3 0 0 2 1 0 3 JWhoisServer 6 49 0 0 44 6 0 50 NistDML181 2 7 0 1 0 1 0 2 NistDML182 2 32 0 1 0 1 0 2 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 134. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints Inventory 1 4 0 0 0 1 1 2 Iso3166 1 3 0 0 2 1 0 3 JWhoisServer 6 49 0 0 44 6 0 50 NistDML181 2 7 0 1 0 1 0 2 NistDML182 2 32 0 1 0 1 0 2 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 135. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints NistDML183 2 6 0 1 0 0 1 2 NistWeather 2 9 5 0 2 2 0 9 NistXTS748 1 3 1 0 1 0 1 3 NistXTS749 2 7 1 1 3 2 0 7 Person 1 5 1 0 5 1 0 7 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 136. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints Products 3 9 4 2 5 3 0 14 Residence 2 6 3 1 2 2 0 8 RiskIt 13 56 0 10 15 11 0 36 UnixUsage 8 32 0 7 9 7 0 23 Usda 10 67 0 0 30 0 0 30 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 137. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints Products 3 9 4 2 5 3 0 14 Residence 2 6 3 1 2 2 0 8 RiskIt 13 56 0 10 15 11 0 36 UnixUsage 8 32 0 7 9 7 0 23 Usda 10 67 0 0 30 0 0 30 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 138. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Schema Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints Products 3 9 4 2 5 3 0 14 Residence 2 6 3 1 2 2 0 8 RiskIt 13 56 0 10 15 11 0 36 UnixUsage 8 32 0 7 9 7 0 23 Usda 10 67 0 0 30 0 0 30 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 139. Introduction Testing Technique Empirical Study Conclusion Configuration Case Study Schemas Tables Columns Checks Foreignkeys NotNulls Primarykeys Uniques TotalConstraints Totals 111 542 27 40 231 68 11 377 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 140. Introduction Testing Technique Empirical Study Conclusion Configuration Data Generation Techniques DBMonster Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 141. Introduction Testing Technique Empirical Study Conclusion Configuration Data Generation Techniques DBMonster SchemaAnalyst Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 142. Introduction Testing Technique Empirical Study Conclusion Configuration Data Generation Techniques DBMonster SchemaAnalyst HSQLDB SQLite Postgres Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 143. Introduction Testing Technique Empirical Study Conclusion Configuration Data Generation Techniques DBMonster SchemaAnalyst HSQLDB SQLite Postgres HSQLDB Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 144. Introduction Testing Technique Empirical Study Conclusion Configuration Data Generation Techniques DBMonster SchemaAnalyst HSQLDB SQLite Postgres HSQLDB SQLite Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 145. Introduction Testing Technique Empirical Study Conclusion Configuration Data Generation Techniques DBMonster SchemaAnalyst HSQLDB SQLite Postgres HSQLDB SQLite Postgres Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 146. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) Flights 100.0 70.0 FrenchTowns 100.0 70.0 Inventory 100.0 75.0 Iso3166 100.0 50.0 JWhoisServer 100.0 50.0 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 147. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) Flights 100.0 70.0 FrenchTowns 100.0 70.0 Inventory 100.0 75.0 Iso3166 100.0 50.0 JWhoisServer 100.0 50.0 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 148. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) NistDML181 100.0 75.0 NistDML182 100.0 50.0 NistDML183 100.0 100.0 NistXTS748 100.0 72.2 NistXTS749 100.0 21.4 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 149. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) NistDML181 100.0 75.0 NistDML182 100.0 50.0 NistDML183 100.0 100.0 NistXTS748 100.0 72.2 NistXTS749 100.0 21.4 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 150. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) NistDML181 100.0 75.0 NistDML182 100.0 50.0 NistDML183 100.0 100.0 NistXTS748 100.0 72.2 NistXTS749 100.0 21.4 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 151. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) Residence 100.0 62.5 RiskIt 100.0 4.1 Products 96.4 59.3 UnixUsage 97.8 59.3 Usda 100.0 50.0 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 152. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) Residence 100.0 62.5 RiskIt 100.0 4.1 Products 96.4 59.3 UnixUsage 97.8 59.3 Usda 100.0 50.0 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 153. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) Residence 100.0 62.5 RiskIt 100.0 4.1 Products 96.4 59.3 UnixUsage 97.8 59.3 Usda 100.0 50.0 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 154. Introduction Testing Technique Empirical Study Conclusion Results Analysis Constraint Coverage Results Schema AVM (%) DBMonster (%) Residence 100.0 62.5 RiskIt 100.0 4.1 Products 96.4 59.3 UnixUsage 97.8 59.3 Usda 100.0 50.0 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 155. Introduction Testing Technique Empirical Study Conclusion Results Analysis Quasi-Mutant Results Number of Mutants Hsqldb Postgres SQLite 0 20 40 60 80 100 120 CustomerOrder Non−Quasi Quasi Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 156. Introduction Testing Technique Empirical Study Conclusion Results Analysis Quasi-Mutant Results Number of Mutants Hsqldb Postgres SQLite 0 50 100 150 JWhoisServer Non−Quasi Quasi Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 157. Introduction Testing Technique Empirical Study Conclusion Results Analysis Quasi-Mutant Results Number of Mutants Hsqldb Postgres SQLite 0 50 100 150 DellStore Non−Quasi Quasi Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 158. Introduction Testing Technique Empirical Study Conclusion Results Analysis Summary: Quasi-Mutant Results Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 159. Introduction Testing Technique Empirical Study Conclusion Results Analysis Summary: Quasi-Mutant Results None SomeSome Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 160. Introduction Testing Technique Empirical Study Conclusion Results Analysis Summary: Quasi-Mutant Results None SomeSome Few quasi-mutants means that the mutation scores are good effectiveness indicators Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 161. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 162. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst JWhoisServer DBI=62, MD = 0.7DBI=300, MD = 0.2 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 163. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst JWhoisServer DBI=62, MD = 0.7DBI=300, MD = 0.2 NistDML181 DBI=7, MD = 0.6DBI=13,650, MD = 0.5 Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 164. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst (0.29, 0.59, 0.65, 0.70, 0.89) (0.0, 0.11, 0.41, 0.52, 0.68) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 165. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst (0.29, 0.59, 0.65, 0.70, 0.89) (0.0, 0.11, 0.41, 0.52, 0.68) DBMonster crashes for six schemas! CustomerOrder Flights NistDML182 NistXTS748 Person RiskIt Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 166. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst (0.29, 0.59, 0.65, 0.70, 0.89) (0.0, 0.11, 0.41, 0.52, 0.68) DBMonster crashes for six schemas! CustomerOrder Flights NistDML182 NistXTS748 Person RiskIt Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 167. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst (0.29, 0.59, 0.65, 0.70, 0.89) (0.0, 0.11, 0.41, 0.52, 0.68) DBMonster crashes for six schemas! CustomerOrder Flights NistDML182 NistXTS748 Person RiskIt Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 168. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst (0.29, 0.59, 0.65, 0.70, 0.89) (0.0, 0.11, 0.41, 0.52, 0.68) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 169. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst (0.29, 0.59, 0.65, 0.70, 0.89) (0.0, 0.11, 0.41, 0.52, 0.68) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 170. Introduction Testing Technique Empirical Study Conclusion Results Analysis Mutation Score Results DBMonster SchemaAnalyst SchemaAnalyst’s mutation score is higher than DB- Monster’s for 96% of the schemas (0.29, 0.59, 0.65, 0.70, 0.89) (0.0, 0.11, 0.41, 0.52, 0.68) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 171. Introduction Testing Technique Empirical Study Conclusion Results Analysis Efficiency Results DBMonster SchemaAnalyst Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 172. Introduction Testing Technique Empirical Study Conclusion Results Analysis Efficiency Results DBMonster SchemaAnalyst (0.41, 1.09, 1.90, 5.07, 36.52) (1.50, 3.01, 5.21, 16.79, 639.93) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 173. Introduction Testing Technique Empirical Study Conclusion Results Analysis Efficiency Results DBMonster SchemaAnalyst (0.41, 1.09, 1.90, 5.07, 36.52) (1.50, 3.01, 5.21, 16.79, 639.93) Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 174. Introduction Testing Technique Empirical Study Conclusion Results Analysis Efficiency Results DBMonster SchemaAnalyst (0.41, 1.09, 1.90, 5.07, 36.52) (1.50, 3.01, 5.21, 16.79, 639.93) SchemaAnalyst exhibits competi- tive data genera- tion times that are less variable Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 175. Introduction Testing Technique Empirical Study Conclusion Summary Important Contributions This paper presents SchemaAnalyst, a search-based system for testing the complex integrity constraints in relational schemas Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 176. Introduction Testing Technique Empirical Study Conclusion Summary Important Contributions This paper presents SchemaAnalyst, a search-based system for testing the complex integrity constraints in relational schemas The empirical study demonstrates that Schema- Analyst’s efficiency is competitive with DBMonster’s Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 177. Introduction Testing Technique Empirical Study Conclusion Summary Important Contributions This paper presents SchemaAnalyst, a search-based system for testing the complex integrity constraints in relational schemas The empirical study demonstrates that Schema- Analyst’s efficiency is competitive with DBMonster’s SchemaAnalyst almost always covers 100% of the constraints in the 25 chosen relational schemas Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 178. Introduction Testing Technique Empirical Study Conclusion Summary Important Contributions This paper presents SchemaAnalyst, a search-based system for testing the complex integrity constraints in relational schemas The empirical study demonstrates that Schema- Analyst’s efficiency is competitive with DBMonster’s SchemaAnalyst almost always covers 100% of the constraints in the 25 chosen relational schemas SchemaAnalyst’s mutation score is higher than DBMonster’s for 96% of the schemas Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems
  • 179. Introduction Testing Technique Empirical Study Conclusion Summary Important Contributions This paper presents SchemaAnalyst, a search-based system for testing the complex integrity constraints in relational schemas The empirical study demonstrates that Schema- Analyst’s efficiency is competitive with DBMonster’s SchemaAnalyst almost always covers 100% of the constraints in the 25 chosen relational schemas SchemaAnalyst’s mutation score is higher than DBMonster’s for 96% of the schemas http://www.schemaanalyst.org Kapfhammer, McMinn, and Wright March 19, 2013 Search-Based Testing of Relational Schema Integrity Constraints Across Multiple Database Management Systems