SlideShare a Scribd company logo
THE SEARCH FOR
THE PERFECT PROGRAM
NATALLIE BAIKEVICH
@LU_A_JALLA
A ‘PERFECT
PROGRAM’ IS …
- Maintainable
- Efficient
- Correct
A ‘PERFECT
PROGRAM’ IS …
- Maintainable
- Efficient
- Correct
Does anything else really matter
without correctness?
THE PLAN
- Assertions
- Random
Testing
- Symbolic
Execution
- Types
THE PLAN
- Assertions
- Random
Testing
- Symbolic
Execution
- Types
1. Do
something
2. Check it
3. Go to 1.
THE FAST TRACK
Write code -> Release
WELL, NOT THAT FAST
Dear <…>
…
We see XYZException!!!.............!
where n = enough for you to feel miserable
n times
OK, LET’S TEST IT
1. Choose your favourite library
(good chances it’s .*Unit)
2. Write some tests
aand
3. XYZException has gone
THE PROBLEM, PART 1
But…
- they only check what you think a
program should do, not what it
actually should do
- ... on what you think is all possible
inputs
RANDOMIZE IT
Throw the thousands of inputs into
your program with
- QuickCheck (Haskell)
- FsCheck (F#)
- ScalaCheck (Scala)
- or something else
DEMO
RANDOMIZE IT
FSCHECK
- Expressive and
easy to use
- Checks your spec
- … in a large
number
- … of randomly
generated cases
“500 checks can’t be
wrong!”
© don’t remember who said that 
THE PROBLEM, PART 2
- Expressive and
easy to use
- Checks your spec
- … in a large
number
- … of randomly
generated cases
“500 checks can’t be
wrong!”
© don’t remember who said that 
- Writing good tests
might be tricky
- Checks your spec
- … in a large
number
- … of randomly
generated cases
500 checks can easily
be wrong!
But:
DEMO
RANDOMIZE IT – WHAT’S MISSING?
I WILL BUILD MY OWN
TEST GENERATOR
let f x y =
if x < 10 then x
else if x = 42 then
failwith “42”
else 1 / y
x < 10 or x >= 10
x = 42 or x <> 42
y = 0 or y <> 0
DEMO
QUOTATIONS
THE PROBLEM, PART 3
But it’s too complicated to do by
hands:
- too many cases,
- lots of overlapping paths,
- difficult to solve when the number
of variables grows,
- not everything is a quotation…
IDEA: USE A
CONSTRAINT SOLVER
- Check out Z3 homepage
- Try Z3 in your browser
- LINQ to Z3 (ch9 video and a post by
Bart de Smet)
Specially for fsharpers:
- Z3Fs on github
THERE’S A TOOL FOR
THAT – MEET PEX
- Dynamic symbolic execution
- Analyses .NET instructions
- Uses constraint solver to find the
inputs
PEX REFERENCES
- Project homepage
- Pex for fun in your browser
- Code Digger, addin for VS
- Code Hunt website
- Documentation, videos
and more
DEMO
PEX
TEST GENERATION
Static:
- Conditional
statements
- Check the
formulas
satisfiability
Dynamic:
- Collect the
information during
the program
execution
- Unknown
environments
- Enhanced values
generation
- Better Performance
DYNAMIC EXECUTION
EXAMPLE
int obscure(int x, int y) {
if (x == hash(y)) return -1; // error
return 0;
}
“Compositional Dynamic Test Generation”,
Patrice Godefroid (paper)
THE PROBLEM, PART 4
Limitations:
- Nondeterministic cases (e.g. native
code)
- Concurrency
- Constraint solver limitations
MORE AND LESS
TYPES WITH F*
F* - an ML-like verification-oriented
language
- F* project homepage
- GitHub repo
- Try F* in your browser
DEMO
TRYING F*
THE PROBLEM, PART 5
- currently under development
- issues on mono
- in more complex cases, the errors
become quite cryptic
- you still need to come up with a
way to define the requirements, at
the type level
CONCLUSION
If you don’t know where you’re going,
any road’ll take you there.
Lewis Caroll
QUESTIONS?

More Related Content

What's hot

Mutiny on the JVM: Taming Complexity in the Heart of Reactive Java
Mutiny on the JVM: Taming Complexity in the Heart of Reactive JavaMutiny on the JVM: Taming Complexity in the Heart of Reactive Java
Mutiny on the JVM: Taming Complexity in the Heart of Reactive Java
Jeremy Davis
 
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet
 
When Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting PloneWhen Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
David Glick
 
Making it Work Offline: Current & Future Offline APIs for Web Apps
Making it Work Offline: Current & Future Offline APIs for Web AppsMaking it Work Offline: Current & Future Offline APIs for Web Apps
Making it Work Offline: Current & Future Offline APIs for Web Apps
Natasha Rooney
 
Async await...oh wait!
Async await...oh wait!Async await...oh wait!
Async await...oh wait!
Thomas Pierrain
 
Welcome to Swift (CocoaCoder 6/12/14)
Welcome to Swift (CocoaCoder 6/12/14)Welcome to Swift (CocoaCoder 6/12/14)
Welcome to Swift (CocoaCoder 6/12/14)
Carl Brown
 
Brighton SEO 2021 - A Deep Dive into the Depths of DevTools
Brighton SEO 2021 - A Deep Dive into the Depths of DevToolsBrighton SEO 2021 - A Deep Dive into the Depths of DevTools
Brighton SEO 2021 - A Deep Dive into the Depths of DevTools
ChrisJohnson792
 
Your first Elm program
Your first Elm programYour first Elm program
Your first Elm program
Per Lundholm
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
Sylvain Zimmer
 
Docker in Continuous Integration
Docker in Continuous IntegrationDocker in Continuous Integration
Docker in Continuous Integration
Alexander Akbashev
 
Upstate CSCI 450 PHP Chapters 5, 12, 13
Upstate CSCI 450 PHP Chapters 5, 12, 13Upstate CSCI 450 PHP Chapters 5, 12, 13
Upstate CSCI 450 PHP Chapters 5, 12, 13
DanWooster1
 
Bugs found in GCC with the help of PVS-Studio
Bugs found in GCC with the help of PVS-StudioBugs found in GCC with the help of PVS-Studio
Bugs found in GCC with the help of PVS-Studio
PVS-Studio
 
Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014
Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014
Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014
Andrea Telatin
 
Jakob Holderbaum - Managing Shared secrets using basic Unix tools
Jakob Holderbaum - Managing Shared secrets using basic Unix toolsJakob Holderbaum - Managing Shared secrets using basic Unix tools
Jakob Holderbaum - Managing Shared secrets using basic Unix tools
DevSecCon
 
Contribute to rails
Contribute to railsContribute to rails
Contribute to rails
martinsvalin
 
Introducing Ghost Inspector
Introducing Ghost InspectorIntroducing Ghost Inspector
Introducing Ghost Inspector
Neil Mansilla
 
X page developer
X page developerX page developer
X page developer
Jakob Majkilde
 
Padre user experience
Padre user experiencePadre user experience
Padre user experience
Charlie Gonzalez
 
Continuous integration with Git & CI Joe
Continuous integration with Git & CI JoeContinuous integration with Git & CI Joe
Continuous integration with Git & CI JoeShawn Price
 

What's hot (20)

Mutiny on the JVM: Taming Complexity in the Heart of Reactive Java
Mutiny on the JVM: Taming Complexity in the Heart of Reactive JavaMutiny on the JVM: Taming Complexity in the Heart of Reactive Java
Mutiny on the JVM: Taming Complexity in the Heart of Reactive Java
 
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
 
When Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting PloneWhen Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
When Good Code Goes Bad: Tools and Techniques for Troubleshooting Plone
 
Making it Work Offline: Current & Future Offline APIs for Web Apps
Making it Work Offline: Current & Future Offline APIs for Web AppsMaking it Work Offline: Current & Future Offline APIs for Web Apps
Making it Work Offline: Current & Future Offline APIs for Web Apps
 
Async await...oh wait!
Async await...oh wait!Async await...oh wait!
Async await...oh wait!
 
Welcome to Swift (CocoaCoder 6/12/14)
Welcome to Swift (CocoaCoder 6/12/14)Welcome to Swift (CocoaCoder 6/12/14)
Welcome to Swift (CocoaCoder 6/12/14)
 
Brighton SEO 2021 - A Deep Dive into the Depths of DevTools
Brighton SEO 2021 - A Deep Dive into the Depths of DevToolsBrighton SEO 2021 - A Deep Dive into the Depths of DevTools
Brighton SEO 2021 - A Deep Dive into the Depths of DevTools
 
Your first Elm program
Your first Elm programYour first Elm program
Your first Elm program
 
Developer-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing oneDeveloper-friendly taskqueues: What you should ask yourself before choosing one
Developer-friendly taskqueues: What you should ask yourself before choosing one
 
Drupal 7 ci and testing
Drupal 7 ci and testingDrupal 7 ci and testing
Drupal 7 ci and testing
 
Docker in Continuous Integration
Docker in Continuous IntegrationDocker in Continuous Integration
Docker in Continuous Integration
 
Upstate CSCI 450 PHP Chapters 5, 12, 13
Upstate CSCI 450 PHP Chapters 5, 12, 13Upstate CSCI 450 PHP Chapters 5, 12, 13
Upstate CSCI 450 PHP Chapters 5, 12, 13
 
Bugs found in GCC with the help of PVS-Studio
Bugs found in GCC with the help of PVS-StudioBugs found in GCC with the help of PVS-Studio
Bugs found in GCC with the help of PVS-Studio
 
Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014
Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014
Bioinformatica: introduzione (BMR Genomics) - Lezione 25 luglio 2014
 
Jakob Holderbaum - Managing Shared secrets using basic Unix tools
Jakob Holderbaum - Managing Shared secrets using basic Unix toolsJakob Holderbaum - Managing Shared secrets using basic Unix tools
Jakob Holderbaum - Managing Shared secrets using basic Unix tools
 
Contribute to rails
Contribute to railsContribute to rails
Contribute to rails
 
Introducing Ghost Inspector
Introducing Ghost InspectorIntroducing Ghost Inspector
Introducing Ghost Inspector
 
X page developer
X page developerX page developer
X page developer
 
Padre user experience
Padre user experiencePadre user experience
Padre user experience
 
Continuous integration with Git & CI Joe
Continuous integration with Git & CI JoeContinuous integration with Git & CI Joe
Continuous integration with Git & CI Joe
 

Viewers also liked

Monoids, Store, and Dependency Injection - Abstractions for Spark Streaming Jobs
Monoids, Store, and Dependency Injection - Abstractions for Spark Streaming JobsMonoids, Store, and Dependency Injection - Abstractions for Spark Streaming Jobs
Monoids, Store, and Dependency Injection - Abstractions for Spark Streaming Jobs
Ryan Weald
 
Time for Functions
Time for FunctionsTime for Functions
Time for Functions
simontcousins
 
Apache kafka
Apache kafkaApache kafka
Apache kafka
Daan Gerits
 
[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화
NAVER D2
 
Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)
Scott Wlaschin
 

Viewers also liked (6)

Connascence
ConnascenceConnascence
Connascence
 
Monoids, Store, and Dependency Injection - Abstractions for Spark Streaming Jobs
Monoids, Store, and Dependency Injection - Abstractions for Spark Streaming JobsMonoids, Store, and Dependency Injection - Abstractions for Spark Streaming Jobs
Monoids, Store, and Dependency Injection - Abstractions for Spark Streaming Jobs
 
Time for Functions
Time for FunctionsTime for Functions
Time for Functions
 
Apache kafka
Apache kafkaApache kafka
Apache kafka
 
[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화
 
Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)
 

Similar to The Search for the Perfect Program

Effective Techniques for Support Teams.pptx
Effective Techniques for Support Teams.pptxEffective Techniques for Support Teams.pptx
Effective Techniques for Support Teams.pptx
Vikas Prabhu
 
Sensepost assessment automation
Sensepost assessment automationSensepost assessment automation
Sensepost assessment automation
SensePost
 
The 3 Things Every New Development Team Needs (and what to avoid like the pla...
The 3 Things Every New Development Team Needs (and what to avoid like the pla...The 3 Things Every New Development Team Needs (and what to avoid like the pla...
The 3 Things Every New Development Team Needs (and what to avoid like the pla...
Atlassian
 
The Mysteries Of JavaScript-Fu (@media Europe Edition)
The Mysteries Of JavaScript-Fu (@media Europe Edition)The Mysteries Of JavaScript-Fu (@media Europe Edition)
The Mysteries Of JavaScript-Fu (@media Europe Edition)
danwrong
 
Debugging
DebuggingDebugging
Debugging
Olivier Teytaud
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdmins
Puppet
 
The Mysteries Of JavaScript-Fu (@media SF Edition)
The Mysteries Of JavaScript-Fu (@media SF Edition)The Mysteries Of JavaScript-Fu (@media SF Edition)
The Mysteries Of JavaScript-Fu (@media SF Edition)
danwrong
 
Ekoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's MethodologyEkoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's Methodology
bugcrowd
 
Fuzzing: The New Unit Testing
Fuzzing: The New Unit TestingFuzzing: The New Unit Testing
Fuzzing: The New Unit Testing
Dmitry Vyukov
 
Archeology for Entertainment, or Checking Microsoft Word 1.1a with PVS-Studio
Archeology for Entertainment, or Checking Microsoft Word 1.1a with PVS-StudioArcheology for Entertainment, or Checking Microsoft Word 1.1a with PVS-Studio
Archeology for Entertainment, or Checking Microsoft Word 1.1a with PVS-Studio
Andrey Karpov
 
Write tests, please
Write tests, pleaseWrite tests, please
Write tests, please
Joan López de la Franca
 
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan RomanDevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys Admins
Puppet
 
It's all about behaviour, also in php - phpspec
It's all about behaviour, also in php - phpspecIt's all about behaviour, also in php - phpspec
It's all about behaviour, also in php - phpspec
Giulio De Donato
 
Testing: ¿what, how, why?
Testing: ¿what, how, why?Testing: ¿what, how, why?
Testing: ¿what, how, why?
David Rodenas
 
DevDay.lk - Bare Knuckle Web Development
DevDay.lk - Bare Knuckle Web DevelopmentDevDay.lk - Bare Knuckle Web Development
DevDay.lk - Bare Knuckle Web Development
Johannes Brodwall
 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
Chris Gates
 
Stammtisch plus ui5 lessons learned
Stammtisch plus ui5 lessons learnedStammtisch plus ui5 lessons learned
Stammtisch plus ui5 lessons learned
Robert Euler
 
Leveling Up at JavaScript
Leveling Up at JavaScriptLeveling Up at JavaScript
Leveling Up at JavaScript
Raymond Camden
 

Similar to The Search for the Perfect Program (20)

Effective Techniques for Support Teams.pptx
Effective Techniques for Support Teams.pptxEffective Techniques for Support Teams.pptx
Effective Techniques for Support Teams.pptx
 
Sensepost assessment automation
Sensepost assessment automationSensepost assessment automation
Sensepost assessment automation
 
The 3 Things Every New Development Team Needs (and what to avoid like the pla...
The 3 Things Every New Development Team Needs (and what to avoid like the pla...The 3 Things Every New Development Team Needs (and what to avoid like the pla...
The 3 Things Every New Development Team Needs (and what to avoid like the pla...
 
The Mysteries Of JavaScript-Fu (@media Europe Edition)
The Mysteries Of JavaScript-Fu (@media Europe Edition)The Mysteries Of JavaScript-Fu (@media Europe Edition)
The Mysteries Of JavaScript-Fu (@media Europe Edition)
 
Debugging
DebuggingDebugging
Debugging
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdmins
 
The Mysteries Of JavaScript-Fu (@media SF Edition)
The Mysteries Of JavaScript-Fu (@media SF Edition)The Mysteries Of JavaScript-Fu (@media SF Edition)
The Mysteries Of JavaScript-Fu (@media SF Edition)
 
Ekoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's MethodologyEkoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's Methodology
 
Fuzzing: The New Unit Testing
Fuzzing: The New Unit TestingFuzzing: The New Unit Testing
Fuzzing: The New Unit Testing
 
Archeology for Entertainment, or Checking Microsoft Word 1.1a with PVS-Studio
Archeology for Entertainment, or Checking Microsoft Word 1.1a with PVS-StudioArcheology for Entertainment, or Checking Microsoft Word 1.1a with PVS-Studio
Archeology for Entertainment, or Checking Microsoft Word 1.1a with PVS-Studio
 
Write tests, please
Write tests, pleaseWrite tests, please
Write tests, please
 
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan RomanDevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys Admins
 
It's all about behaviour, also in php - phpspec
It's all about behaviour, also in php - phpspecIt's all about behaviour, also in php - phpspec
It's all about behaviour, also in php - phpspec
 
Testing: ¿what, how, why?
Testing: ¿what, how, why?Testing: ¿what, how, why?
Testing: ¿what, how, why?
 
DevDay.lk - Bare Knuckle Web Development
DevDay.lk - Bare Knuckle Web DevelopmentDevDay.lk - Bare Knuckle Web Development
DevDay.lk - Bare Knuckle Web Development
 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
 
Stammtisch plus ui5 lessons learned
Stammtisch plus ui5 lessons learnedStammtisch plus ui5 lessons learned
Stammtisch plus ui5 lessons learned
 
Leveling Up at JavaScript
Leveling Up at JavaScriptLeveling Up at JavaScript
Leveling Up at JavaScript
 
All of Javascript
All of JavascriptAll of Javascript
All of Javascript
 

Recently uploaded

Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 

Recently uploaded (20)

Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 

The Search for the Perfect Program

  • 1. THE SEARCH FOR THE PERFECT PROGRAM NATALLIE BAIKEVICH @LU_A_JALLA
  • 2. A ‘PERFECT PROGRAM’ IS … - Maintainable - Efficient - Correct
  • 3. A ‘PERFECT PROGRAM’ IS … - Maintainable - Efficient - Correct Does anything else really matter without correctness?
  • 4. THE PLAN - Assertions - Random Testing - Symbolic Execution - Types
  • 5. THE PLAN - Assertions - Random Testing - Symbolic Execution - Types 1. Do something 2. Check it 3. Go to 1.
  • 6. THE FAST TRACK Write code -> Release
  • 7. WELL, NOT THAT FAST Dear <…> … We see XYZException!!!.............! where n = enough for you to feel miserable n times
  • 8. OK, LET’S TEST IT 1. Choose your favourite library (good chances it’s .*Unit) 2. Write some tests aand 3. XYZException has gone
  • 9. THE PROBLEM, PART 1 But… - they only check what you think a program should do, not what it actually should do - ... on what you think is all possible inputs
  • 10. RANDOMIZE IT Throw the thousands of inputs into your program with - QuickCheck (Haskell) - FsCheck (F#) - ScalaCheck (Scala) - or something else
  • 12. FSCHECK - Expressive and easy to use - Checks your spec - … in a large number - … of randomly generated cases “500 checks can’t be wrong!” © don’t remember who said that 
  • 13. THE PROBLEM, PART 2 - Expressive and easy to use - Checks your spec - … in a large number - … of randomly generated cases “500 checks can’t be wrong!” © don’t remember who said that  - Writing good tests might be tricky - Checks your spec - … in a large number - … of randomly generated cases 500 checks can easily be wrong! But:
  • 14. DEMO RANDOMIZE IT – WHAT’S MISSING?
  • 15. I WILL BUILD MY OWN TEST GENERATOR let f x y = if x < 10 then x else if x = 42 then failwith “42” else 1 / y x < 10 or x >= 10 x = 42 or x <> 42 y = 0 or y <> 0
  • 17. THE PROBLEM, PART 3 But it’s too complicated to do by hands: - too many cases, - lots of overlapping paths, - difficult to solve when the number of variables grows, - not everything is a quotation…
  • 18. IDEA: USE A CONSTRAINT SOLVER - Check out Z3 homepage - Try Z3 in your browser - LINQ to Z3 (ch9 video and a post by Bart de Smet) Specially for fsharpers: - Z3Fs on github
  • 19. THERE’S A TOOL FOR THAT – MEET PEX - Dynamic symbolic execution - Analyses .NET instructions - Uses constraint solver to find the inputs
  • 20. PEX REFERENCES - Project homepage - Pex for fun in your browser - Code Digger, addin for VS - Code Hunt website - Documentation, videos and more
  • 22. TEST GENERATION Static: - Conditional statements - Check the formulas satisfiability Dynamic: - Collect the information during the program execution - Unknown environments - Enhanced values generation - Better Performance
  • 23. DYNAMIC EXECUTION EXAMPLE int obscure(int x, int y) { if (x == hash(y)) return -1; // error return 0; } “Compositional Dynamic Test Generation”, Patrice Godefroid (paper)
  • 24. THE PROBLEM, PART 4 Limitations: - Nondeterministic cases (e.g. native code) - Concurrency - Constraint solver limitations
  • 25. MORE AND LESS TYPES WITH F* F* - an ML-like verification-oriented language - F* project homepage - GitHub repo - Try F* in your browser
  • 27. THE PROBLEM, PART 5 - currently under development - issues on mono - in more complex cases, the errors become quite cryptic - you still need to come up with a way to define the requirements, at the type level
  • 28. CONCLUSION If you don’t know where you’re going, any road’ll take you there. Lewis Caroll