SlideShare a Scribd company logo
Automatic Non-functional Testing of Code Generators
Families
Mohamed Boussaa, Olivier Barais, Gerson Sunye and Benoit Baudry
INRIA Rennes, France
Mohamed BOUSSAA
INRIA Rennes, France
Presentation date: GPCE 2016 - Tue 1 Nov: 16:40 at Z1
Email: mohamed.boussaa@inria.fr
Personal webpage: mboussaa@wordpress.com
Tool webpage: testingcodegenerators.wordpress.com
Phone: +33626492436
Contact
Code	generators	are	used	everywhere	
They	automa7cally	transform	high-level system specications (Models,	DSLs,	
GUIs,	etc.)	into	general-purpose	languages	
Target	diverse	and	heterogeneous soNware	plakorms	(JAVA,	C++,	C#,	etc.)	
	
	
	
	
	
	
	
	
	
				
	
	The	generated	soNware	ar7facts	work	but…	
	
How	about	the	non-funcDonal	properDes	(quality)		
of	generated	code	?	
Context
¤ Definition (Code generator family): We define a code generator family
as a set of code generators that takes as input the same language/
model and generate code for different target platforms
¤ Non-functional testing of code generators: The classical way
•  Analyze the non-functional properties of generated code using platform-
specific tools, profilers, etc.
•  Report inconsistencies, bugs, performance issues, etc.
Lack of tools for automatic non-functional testing of code generators
15th International Conference on Generative Programming: Concepts & Experiences (GPCE 2016)
Amsterdam, Netherlands, October 31, 2016
Footprint C
Footprint A
DSL
(Model)
SUT
SUT
SUT
Design
Generate
Generate
Generate
Code
Generator A
Code
Generator B
Code
Generator C
Execute
Execute
Execute
C++
Platform C
Platform B
Platform A
JAVA
C#
Profiler A
Profiler B
Profiler C
Bugs
Finding
Report
Report
Report
Footprint B
Code Generation Non-functional TestingCode ExecutionSoftware Design
Software Platform
Diversity
Software Design Automatic Code Generation
Software
Designer
DSL
Model
GPL
Specs
GUI
Code Generator
creators/maintainers
Code generators
Generated code
Container C
Container B
Container A
DSL
(Model)
SUT
SUT
SUT
Design
Generate
Generate
Generate
Code
Generator A
Code
Generator B
Code
Generator C
Code Generation Runtime monitoring engineCode ExecutionSoftware Design
Container A’
C#
Container B’
Container C’
Monitoring
Container
Back-end
Data Base
Container
Front-end
Visualization
Container
JAVA
C++
Footprint C’
Footprint A’
REST
Calls
Footprint B’Request
Bugs Finding
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
Testing Infrastructure
GPCE’16
Research Team
diverse.irisa.fr
Research Lab
inria.fr/centre/rennes
EU Funding Project
heads-project.eu
Running…
Haxe library
+ Test suites
Component
Under Test
Back-end
Database
Component
Cgroup file systems
Running…
Monitoring records
Front-end:
Visualization
Component
Time-series database
HTTP Requests
CPU
Memory
…
8086:
Back-end:
Monitoring
ComponentTarget
language …
Code
Generation
Software
Tester
Motivation
We propose:
•  A black-box approach to automatically check the potential inefficient
code generators
•  A runtime monitoring infrastructure, based on system containers
(Docker) as execution platforms, that allow code-generator developers to
evaluate the non-functional properties of generated code
Differential Testing: Compare equivalent implementations of the same
program written in different languages
Standard deviation (std_dev): Quantify the amount of variation among
the execution traces in terms of memory usage and execution time
Testing Method
Approach Overview
Running…
¤  Test suites with the highest variation in terms of execution time (k=60)
¤ Test suites with the highest variation in terms of memory usage (k=400)
We can identify a singular behavior of the PHP code regarding the
memory usage and exectution time
Validation…
Test suites with Std_dev > threshold value are interpreted as code
generator inconsistencies
…
Memory usage Memory usage Memory usage
Compare
Std_dev > kStd_dev < k
BugNo Bug

More Related Content

What's hot

Remix Your Language Tooling (JSConf.eu 2012)
Remix Your Language Tooling (JSConf.eu 2012)Remix Your Language Tooling (JSConf.eu 2012)
Remix Your Language Tooling (JSConf.eu 2012)
lennartkats
 
Integrated Language Definition Testing: Enabling Test-Driven Language Develop...
Integrated Language Definition Testing: Enabling Test-Driven Language Develop...Integrated Language Definition Testing: Enabling Test-Driven Language Develop...
Integrated Language Definition Testing: Enabling Test-Driven Language Develop...
lennartkats
 
The Spoofax Language Workbench (SPLASH 2010)
The Spoofax Language Workbench (SPLASH 2010)The Spoofax Language Workbench (SPLASH 2010)
The Spoofax Language Workbench (SPLASH 2010)
lennartkats
 
Java Source Code Analysis using SonarQube
Java Source Code Analysis using SonarQubeJava Source Code Analysis using SonarQube
Java Source Code Analysis using SonarQube
Angelin R
 
ctchou-resume
ctchou-resumectchou-resume
ctchou-resume
Ching-Tsun Chou
 
C-Sharp 6.0 ver2
C-Sharp 6.0 ver2C-Sharp 6.0 ver2
C-Sharp 6.0 ver2
Tome Tomovski
 
ctchou-resume
ctchou-resumectchou-resume
ctchou-resume
Ching-Tsun Chou
 
Resume LinkedIn
Resume LinkedInResume LinkedIn
Resume LinkedIn
Drew Williamson
 
Python for Swift
Python for SwiftPython for Swift
Python for Swift
LINE Corporation
 
torque - Automation Testing Tool for C-C++ on Linux
torque -  Automation Testing Tool for C-C++ on Linuxtorque -  Automation Testing Tool for C-C++ on Linux
torque - Automation Testing Tool for C-C++ on Linux
JITENDRA LENKA
 
ctchou-resume
ctchou-resumectchou-resume
ctchou-resume
Ching-Tsun Chou
 
Benefits of Extensions
Benefits of ExtensionsBenefits of Extensions
Benefits of Extensions
Alexandro Colorado
 
บทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรม
บทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรมบทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรม
บทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรม
Ung Titaya
 
Introduction to Robot Framework
Introduction to Robot FrameworkIntroduction to Robot Framework
Introduction to Robot Framework
Carl Su
 
Megha_Smriti_resume
Megha_Smriti_resumeMegha_Smriti_resume
Megha_Smriti_resume
megha smriti
 
Sonar
SonarSonar
An Overview of LLVM Link Time Optimization
An Overview of LLVM Link Time Optimization An Overview of LLVM Link Time Optimization
An Overview of LLVM Link Time Optimization
Vivek Pandya
 
A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0
A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0
A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0
Antonio Chagoury
 
Electron
ElectronElectron
balaraju
balarajubalaraju

What's hot (20)

Remix Your Language Tooling (JSConf.eu 2012)
Remix Your Language Tooling (JSConf.eu 2012)Remix Your Language Tooling (JSConf.eu 2012)
Remix Your Language Tooling (JSConf.eu 2012)
 
Integrated Language Definition Testing: Enabling Test-Driven Language Develop...
Integrated Language Definition Testing: Enabling Test-Driven Language Develop...Integrated Language Definition Testing: Enabling Test-Driven Language Develop...
Integrated Language Definition Testing: Enabling Test-Driven Language Develop...
 
The Spoofax Language Workbench (SPLASH 2010)
The Spoofax Language Workbench (SPLASH 2010)The Spoofax Language Workbench (SPLASH 2010)
The Spoofax Language Workbench (SPLASH 2010)
 
Java Source Code Analysis using SonarQube
Java Source Code Analysis using SonarQubeJava Source Code Analysis using SonarQube
Java Source Code Analysis using SonarQube
 
ctchou-resume
ctchou-resumectchou-resume
ctchou-resume
 
C-Sharp 6.0 ver2
C-Sharp 6.0 ver2C-Sharp 6.0 ver2
C-Sharp 6.0 ver2
 
ctchou-resume
ctchou-resumectchou-resume
ctchou-resume
 
Resume LinkedIn
Resume LinkedInResume LinkedIn
Resume LinkedIn
 
Python for Swift
Python for SwiftPython for Swift
Python for Swift
 
torque - Automation Testing Tool for C-C++ on Linux
torque -  Automation Testing Tool for C-C++ on Linuxtorque -  Automation Testing Tool for C-C++ on Linux
torque - Automation Testing Tool for C-C++ on Linux
 
ctchou-resume
ctchou-resumectchou-resume
ctchou-resume
 
Benefits of Extensions
Benefits of ExtensionsBenefits of Extensions
Benefits of Extensions
 
บทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรม
บทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรมบทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรม
บทที่ 1ภาษาคอมพิวเตอร์และการโปรแกรม
 
Introduction to Robot Framework
Introduction to Robot FrameworkIntroduction to Robot Framework
Introduction to Robot Framework
 
Megha_Smriti_resume
Megha_Smriti_resumeMegha_Smriti_resume
Megha_Smriti_resume
 
Sonar
SonarSonar
Sonar
 
An Overview of LLVM Link Time Optimization
An Overview of LLVM Link Time Optimization An Overview of LLVM Link Time Optimization
An Overview of LLVM Link Time Optimization
 
A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0
A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0
A Sneak Peek At Visual Studio 2010 And .Net Framework 4.0
 
Electron
ElectronElectron
Electron
 
balaraju
balarajubalaraju
balaraju
 

Similar to GPCE16 Poster: Automatic Non-functional Testing of Code Generators Families

Build Great Networked APIs with Swift, OpenAPI, and gRPC
Build Great Networked APIs with Swift, OpenAPI, and gRPCBuild Great Networked APIs with Swift, OpenAPI, and gRPC
Build Great Networked APIs with Swift, OpenAPI, and gRPC
Tim Burks
 
UnDeveloper Studio
UnDeveloper StudioUnDeveloper Studio
UnDeveloper Studio
Christien Rioux
 
Continuous Inspection of Code Quality: SonarQube
Continuous Inspection of Code Quality: SonarQubeContinuous Inspection of Code Quality: SonarQube
Continuous Inspection of Code Quality: SonarQube
Emre Dündar
 
Intro to Reverse Engineering
Intro to Reverse EngineeringIntro to Reverse Engineering
Intro to Reverse Engineering
Null Bhubaneswar
 
The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...
The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...
The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...
ITCamp
 
Maria Tverdostup Resume 2017
Maria Tverdostup Resume 2017Maria Tverdostup Resume 2017
Maria Tverdostup Resume 2017
Maria Tverdostup
 
System software
System softwareSystem software
System software
Senthil Kanth
 
TypeScript - Javascript done right
TypeScript - Javascript done rightTypeScript - Javascript done right
TypeScript - Javascript done right
Wekoslav Stefanovski
 
An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...
An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...
An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...
Jean Vanderdonckt
 
The story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps EngineerThe story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps Engineer
Manu Pk
 
Kunal bhatia resume mass
Kunal bhatia   resume massKunal bhatia   resume mass
Kunal bhatia resume mass
Kunal Bhatia, MBA Candidate, BSc.
 
Typhoon Managed Execution Toolkit
Typhoon Managed Execution ToolkitTyphoon Managed Execution Toolkit
Typhoon Managed Execution Toolkit
Dimitry Snezhkov
 
.Net overviewrajnish
.Net overviewrajnish.Net overviewrajnish
.Net overviewrajnish
Rajnish Kalla
 
High Productivity Web Development Workflow
High Productivity Web Development WorkflowHigh Productivity Web Development Workflow
High Productivity Web Development Workflow
Vũ Nguyễn
 
High productivity web development workflow - JavaScript Meetup Saigon 2014
High productivity web development workflow - JavaScript Meetup Saigon 2014High productivity web development workflow - JavaScript Meetup Saigon 2014
High productivity web development workflow - JavaScript Meetup Saigon 2014
Oliver N
 
Continuous Integration and development environment approach
Continuous Integration and development environment approachContinuous Integration and development environment approach
Continuous Integration and development environment approach
Aleksandr Tsertkov
 
Java Code Quality Tools
Java Code Quality ToolsJava Code Quality Tools
Java Code Quality Tools
Anju ML
 
RESUME_SW
RESUME_SWRESUME_SW
RESUME_SW
PRADEEP C PRASAD
 
.Net Debugging Techniques
.Net Debugging Techniques.Net Debugging Techniques
.Net Debugging Techniques
Bala Subra
 
.NET Debugging Tips and Techniques
.NET Debugging Tips and Techniques.NET Debugging Tips and Techniques
.NET Debugging Tips and Techniques
Bala Subra
 

Similar to GPCE16 Poster: Automatic Non-functional Testing of Code Generators Families (20)

Build Great Networked APIs with Swift, OpenAPI, and gRPC
Build Great Networked APIs with Swift, OpenAPI, and gRPCBuild Great Networked APIs with Swift, OpenAPI, and gRPC
Build Great Networked APIs with Swift, OpenAPI, and gRPC
 
UnDeveloper Studio
UnDeveloper StudioUnDeveloper Studio
UnDeveloper Studio
 
Continuous Inspection of Code Quality: SonarQube
Continuous Inspection of Code Quality: SonarQubeContinuous Inspection of Code Quality: SonarQube
Continuous Inspection of Code Quality: SonarQube
 
Intro to Reverse Engineering
Intro to Reverse EngineeringIntro to Reverse Engineering
Intro to Reverse Engineering
 
The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...
The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...
The Roslyn Compiler: Look at Your Code from a Different Perspective (Raffaele...
 
Maria Tverdostup Resume 2017
Maria Tverdostup Resume 2017Maria Tverdostup Resume 2017
Maria Tverdostup Resume 2017
 
System software
System softwareSystem software
System software
 
TypeScript - Javascript done right
TypeScript - Javascript done rightTypeScript - Javascript done right
TypeScript - Javascript done right
 
An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...
An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...
An Open Source Workbench for Prototyping Multimodal Interactions Based on Off...
 
The story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps EngineerThe story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps Engineer
 
Kunal bhatia resume mass
Kunal bhatia   resume massKunal bhatia   resume mass
Kunal bhatia resume mass
 
Typhoon Managed Execution Toolkit
Typhoon Managed Execution ToolkitTyphoon Managed Execution Toolkit
Typhoon Managed Execution Toolkit
 
.Net overviewrajnish
.Net overviewrajnish.Net overviewrajnish
.Net overviewrajnish
 
High Productivity Web Development Workflow
High Productivity Web Development WorkflowHigh Productivity Web Development Workflow
High Productivity Web Development Workflow
 
High productivity web development workflow - JavaScript Meetup Saigon 2014
High productivity web development workflow - JavaScript Meetup Saigon 2014High productivity web development workflow - JavaScript Meetup Saigon 2014
High productivity web development workflow - JavaScript Meetup Saigon 2014
 
Continuous Integration and development environment approach
Continuous Integration and development environment approachContinuous Integration and development environment approach
Continuous Integration and development environment approach
 
Java Code Quality Tools
Java Code Quality ToolsJava Code Quality Tools
Java Code Quality Tools
 
RESUME_SW
RESUME_SWRESUME_SW
RESUME_SW
 
.Net Debugging Techniques
.Net Debugging Techniques.Net Debugging Techniques
.Net Debugging Techniques
 
.NET Debugging Tips and Techniques
.NET Debugging Tips and Techniques.NET Debugging Tips and Techniques
.NET Debugging Tips and Techniques
 

Recently uploaded

Height and depth gauge linear metrology.pdf
Height and depth gauge linear metrology.pdfHeight and depth gauge linear metrology.pdf
Height and depth gauge linear metrology.pdf
q30122000
 
NATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENT
NATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENTNATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENT
NATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENT
Addu25809
 
SENTIMENT ANALYSIS ON PPT AND Project template_.pptx
SENTIMENT ANALYSIS ON PPT AND Project template_.pptxSENTIMENT ANALYSIS ON PPT AND Project template_.pptx
SENTIMENT ANALYSIS ON PPT AND Project template_.pptx
b0754201
 
AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...
AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...
AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...
Paris Salesforce Developer Group
 
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODELDEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
ijaia
 
Null Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAMNull Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAM
Divyanshu
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Sinan KOZAK
 
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
ecqow
 
Object Oriented Analysis and Design - OOAD
Object Oriented Analysis and Design - OOADObject Oriented Analysis and Design - OOAD
Object Oriented Analysis and Design - OOAD
PreethaV16
 
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
PriyankaKilaniya
 
Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...
Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...
Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...
shadow0702a
 
1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf
1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf
1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf
MadhavJungKarki
 
Software Engineering and Project Management - Software Testing + Agile Method...
Software Engineering and Project Management - Software Testing + Agile Method...Software Engineering and Project Management - Software Testing + Agile Method...
Software Engineering and Project Management - Software Testing + Agile Method...
Prakhyath Rai
 
Mechatronics material . Mechanical engineering
Mechatronics material . Mechanical engineeringMechatronics material . Mechanical engineering
Mechatronics material . Mechanical engineering
sachin chaurasia
 
一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理
一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理
一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理
upoux
 
Generative AI Use cases applications solutions and implementation.pdf
Generative AI Use cases applications solutions and implementation.pdfGenerative AI Use cases applications solutions and implementation.pdf
Generative AI Use cases applications solutions and implementation.pdf
mahaffeycheryld
 
Supermarket Management System Project Report.pdf
Supermarket Management System Project Report.pdfSupermarket Management System Project Report.pdf
Supermarket Management System Project Report.pdf
Kamal Acharya
 
Applications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdfApplications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdf
Atif Razi
 
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
nedcocy
 
TIME TABLE MANAGEMENT SYSTEM testing.pptx
TIME TABLE MANAGEMENT SYSTEM testing.pptxTIME TABLE MANAGEMENT SYSTEM testing.pptx
TIME TABLE MANAGEMENT SYSTEM testing.pptx
CVCSOfficial
 

Recently uploaded (20)

Height and depth gauge linear metrology.pdf
Height and depth gauge linear metrology.pdfHeight and depth gauge linear metrology.pdf
Height and depth gauge linear metrology.pdf
 
NATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENT
NATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENTNATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENT
NATURAL DEEP EUTECTIC SOLVENTS AS ANTI-FREEZING AGENT
 
SENTIMENT ANALYSIS ON PPT AND Project template_.pptx
SENTIMENT ANALYSIS ON PPT AND Project template_.pptxSENTIMENT ANALYSIS ON PPT AND Project template_.pptx
SENTIMENT ANALYSIS ON PPT AND Project template_.pptx
 
AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...
AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...
AI + Data Community Tour - Build the Next Generation of Apps with the Einstei...
 
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODELDEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
 
Null Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAMNull Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAM
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
 
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
 
Object Oriented Analysis and Design - OOAD
Object Oriented Analysis and Design - OOADObject Oriented Analysis and Design - OOAD
Object Oriented Analysis and Design - OOAD
 
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
Prediction of Electrical Energy Efficiency Using Information on Consumer's Ac...
 
Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...
Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...
Use PyCharm for remote debugging of WSL on a Windo cf5c162d672e4e58b4dde5d797...
 
1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf
1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf
1FIDIC-CONSTRUCTION-CONTRACT-2ND-ED-2017-RED-BOOK.pdf
 
Software Engineering and Project Management - Software Testing + Agile Method...
Software Engineering and Project Management - Software Testing + Agile Method...Software Engineering and Project Management - Software Testing + Agile Method...
Software Engineering and Project Management - Software Testing + Agile Method...
 
Mechatronics material . Mechanical engineering
Mechatronics material . Mechanical engineeringMechatronics material . Mechanical engineering
Mechatronics material . Mechanical engineering
 
一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理
一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理
一比一原版(uofo毕业证书)美国俄勒冈大学毕业证如何办理
 
Generative AI Use cases applications solutions and implementation.pdf
Generative AI Use cases applications solutions and implementation.pdfGenerative AI Use cases applications solutions and implementation.pdf
Generative AI Use cases applications solutions and implementation.pdf
 
Supermarket Management System Project Report.pdf
Supermarket Management System Project Report.pdfSupermarket Management System Project Report.pdf
Supermarket Management System Project Report.pdf
 
Applications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdfApplications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdf
 
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
 
TIME TABLE MANAGEMENT SYSTEM testing.pptx
TIME TABLE MANAGEMENT SYSTEM testing.pptxTIME TABLE MANAGEMENT SYSTEM testing.pptx
TIME TABLE MANAGEMENT SYSTEM testing.pptx
 

GPCE16 Poster: Automatic Non-functional Testing of Code Generators Families

  • 1. Automatic Non-functional Testing of Code Generators Families Mohamed Boussaa, Olivier Barais, Gerson Sunye and Benoit Baudry INRIA Rennes, France Mohamed BOUSSAA INRIA Rennes, France Presentation date: GPCE 2016 - Tue 1 Nov: 16:40 at Z1 Email: mohamed.boussaa@inria.fr Personal webpage: mboussaa@wordpress.com Tool webpage: testingcodegenerators.wordpress.com Phone: +33626492436 Contact Code generators are used everywhere They automa7cally transform high-level system specications (Models, DSLs, GUIs, etc.) into general-purpose languages Target diverse and heterogeneous soNware plakorms (JAVA, C++, C#, etc.) The generated soNware ar7facts work but… How about the non-funcDonal properDes (quality) of generated code ? Context ¤ Definition (Code generator family): We define a code generator family as a set of code generators that takes as input the same language/ model and generate code for different target platforms ¤ Non-functional testing of code generators: The classical way •  Analyze the non-functional properties of generated code using platform- specific tools, profilers, etc. •  Report inconsistencies, bugs, performance issues, etc. Lack of tools for automatic non-functional testing of code generators 15th International Conference on Generative Programming: Concepts & Experiences (GPCE 2016) Amsterdam, Netherlands, October 31, 2016 Footprint C Footprint A DSL (Model) SUT SUT SUT Design Generate Generate Generate Code Generator A Code Generator B Code Generator C Execute Execute Execute C++ Platform C Platform B Platform A JAVA C# Profiler A Profiler B Profiler C Bugs Finding Report Report Report Footprint B Code Generation Non-functional TestingCode ExecutionSoftware Design Software Platform Diversity Software Design Automatic Code Generation Software Designer DSL Model GPL Specs GUI Code Generator creators/maintainers Code generators Generated code Container C Container B Container A DSL (Model) SUT SUT SUT Design Generate Generate Generate Code Generator A Code Generator B Code Generator C Code Generation Runtime monitoring engineCode ExecutionSoftware Design Container A’ C# Container B’ Container C’ Monitoring Container Back-end Data Base Container Front-end Visualization Container JAVA C++ Footprint C’ Footprint A’ REST Calls Footprint B’Request Bugs Finding Testing Infrastructure GPCE’16 Research Team diverse.irisa.fr Research Lab inria.fr/centre/rennes EU Funding Project heads-project.eu Running… Haxe library + Test suites Component Under Test Back-end Database Component Cgroup file systems Running… Monitoring records Front-end: Visualization Component Time-series database HTTP Requests CPU Memory … 8086: Back-end: Monitoring ComponentTarget language … Code Generation Software Tester Motivation We propose: •  A black-box approach to automatically check the potential inefficient code generators •  A runtime monitoring infrastructure, based on system containers (Docker) as execution platforms, that allow code-generator developers to evaluate the non-functional properties of generated code Differential Testing: Compare equivalent implementations of the same program written in different languages Standard deviation (std_dev): Quantify the amount of variation among the execution traces in terms of memory usage and execution time Testing Method Approach Overview Running… ¤  Test suites with the highest variation in terms of execution time (k=60) ¤ Test suites with the highest variation in terms of memory usage (k=400) We can identify a singular behavior of the PHP code regarding the memory usage and exectution time Validation… Test suites with Std_dev > threshold value are interpreted as code generator inconsistencies … Memory usage Memory usage Memory usage Compare Std_dev > kStd_dev < k BugNo Bug