Felix Dobslaw, profile picture
Uploaded byFelix Dobslaw
PDF, PPTX88 views

Sast 2021

The document discusses the exploration of software boundaries, focusing on how to automatically identify relevant areas for testing and how to support testers for improved efficiency. It outlines boundary value testing and analysis, emphasizing the significance of correct boundaries in ensuring software reliability. The presentation is based on collaboration with Robert Feldt and Francisco Gomes de Oliveira Neto, and discusses methodologies and examples related to boundary mining in software testing.

Embed presentation

Download as PDF, PPTX
1/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Exploring the Boundaries of our Software Assist. Prof. Felix Dobslaw 25/5/2021 10:50-11:40 CEST https://www.flickr.com/photos/a_siegel/2356136219
2/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Exploring the Boundaries of our Software Assist. Prof. Felix Dobslaw 25/5/2021 10:50-11:40 CEST G https://www.flickr.com/photos/a_siegel/2356136219 Based on collaboration with Robert Feldt, Francisco Gomes de Oliveira Neto
3/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Outline ● Why Boundaries, why exploration? ● Boundary Value…. Testing/Analysis/Exploration – Terminology – Process ● Outlook
4/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Research Questions
5/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Research Questions ● How can we automatically find areas of relevance for testing in our software? – “Boundaries”
6/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Research Questions ● How can we automatically find areas of relevance for testing in our software? – “Boundaries” ● How can we practically support testers in their working routine? – Better fault finding and efficiency
7/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) What do we mean by test automation?
8/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) What do we mean by test automation? Not classical CI/CD running human created test cases automatically
9/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) What do we mean by test automation? Not classical CI/CD running human created test cases automatically But running machine created test cases automatically
https://time.com/3785942/blue-marble/
https://time.com/3785942/blue-marble/ Defining Boundaries is hard
https://time.com/3785942/blue-marble/ Defining Boundaries is hard
https://time.com/3785942/blue-marble/ Defining Boundaries is hard Test Automation Augmentation
https://time.com/3785942/blue-marble/ Defining Boundaries is hard Test Automation Augmentation Diversity
15/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) inputs outputs requirements constraints
16/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification: inputs outputs requirements constraints Entirely implicit Greatly detailed
17/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification: inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
18/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
19/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
20/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
21/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
22/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
23/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
24/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
25/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
26/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System Correct software depends on correct boundaries. Boundary Value Exploration reveals discrepancies between desired and actual behavior.
27/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se)
28/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Testing
29/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Testing BVT: “Execution of specific input pairs in order to ensure that an actual boundary is also expected.”
30/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing BVT: “Execution of specific input pairs in order to ensure that an actual boundary is also expected.”
31/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) BVA: “Analysis of artifacts of the software development process to clarify the expected and actual boundaries of a software.” Boundary Value Analysis Boundary Value Testing BVT: “Execution of specific input pairs in order to ensure that an actual boundary is also expected.”
32/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates
33/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates
34/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates BVE: “A collection of techniques that select or help select inputs to detect and identify boundary candidates.”
35/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates BVE: “A collection of techniques that select or help select inputs to detect and identify boundary candidates.”
36/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count
37/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count bytecount(int) → human readable string
38/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count bytecount(int) → human readable string
39/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count bytecount(int) → human readable string
40/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints BVE Process Example
41/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints
42/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining
43/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB”
44/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB”
45/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB”
46/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
47/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
48/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification Pull Request Update: Bc(1_000_000...000) → last valid Bc(1_000_000...001) → ArgumentError(“out of bounce”) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” testing → quality assurance not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
49/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification Pull Request Update: Bc(1_000_000...000) → last valid Bc(1_000_000...001) → ArgumentError(“out of bounce”) program Specification: tests inputs outputs requirements constraints boundary implementation boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” testing → quality assurance not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
50/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification Pull Request Update: Bc(1_000_000...000) → last valid Bc(1_000_000...001) → ArgumentError(“out of bounce”) program Specification: tests inputs outputs requirements constraints boundary implementation boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” testing → quality assurance not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
51/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) How to automatically mine boundaries? ● Without oracle, or only partial information, only having the SUT at hand.
Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
SUT x y Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
SUT x y Assert y == yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
SUT x y Assert y == yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
SUT x y x1 x2 y1 y2 SUT SUT Assert y == yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
SUT x y x1 x2 y1 y2 ? SUT SUT Assert y == yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
SUT x y x1 x2 y1 y2 ? SUT SUT Assert y == yexp Absolute vs. Relative Assessment Distance Metric Examples: Jaccard, Hamming, Euclidean… Very basic one: StringLength R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
SUT x y x1 x2 y1 y2 ? SUT SUT Assert y == yexp Absolute vs. Relative Assessment Distance Metric Examples: Jaccard, Hamming, Euclidean… Very basic one: StringLength Example: 30 and 31 are neighbors in the input space for field month: Date(2021, 30, 4) → “30/4/2021” StringLength is 9 Date(2021, 31, 4) → “ERROR, month field out of bounds.” StringLength is 33 Big difference, this seems to be a boundary candidate! But is it an interesting one?... R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
60/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) F. Dobslaw, F. G. de Oliveira Neto and R. Feldt, "Boundary Value Exploration for Software Analysis," 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2020, pp. 346-353, doi: 10.1109/ICSTW50294.2020.00062.
61/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Interactive Boundary Exploration F. Dobslaw, F. G. de Oliveira Neto and R. Feldt, "Boundary Value Exploration for Software Analysis," 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2020, pp. 346-353, doi: 10.1109/ICSTW50294.2020.00062.
62/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) - Go to www.menti.com - Enter the code 6546 248 - Answer 1 question. Thanks! Questions, Comments?
63/63 --- Exploring the Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Please partake in an eye-tracking study! https://studenter.miun.se/~evth1400/EyeGazeStudy/?group=3 Link

More Related Content

PDF
SAST vs. DAST: What’s the Best Method For Application Security Testing?
byCigital
 
PDF
8 Patterns For Continuous Code Security by Veracode CTO Chris Wysopal
byThreat Stack
 
PPTX
What Good is this Tool? A Guide to Choosing the Right Application Security Te...
byKevin Fealey
 
PPTX
DAST, SAST, Hybrid, Hybrid 2.0 & IAST - Methodology & Limitations
byiAppSecure Solutions
 
PPTX
Managing Open Source in Application Security and Software Development Lifecycle
byBlack Duck by Synopsys
 
PPTX
Open Source Libraries - Managing Risk in Cloud
bySuman Sourav
 
PPTX
Application Security at DevOps Speed and Portfolio Scale
byJeff Williams
 
PDF
Devops security-An Insight into Secure-SDLC
bySuman Sourav
 
SAST vs. DAST: What’s the Best Method For Application Security Testing?
byCigital
 
8 Patterns For Continuous Code Security by Veracode CTO Chris Wysopal
byThreat Stack
 
What Good is this Tool? A Guide to Choosing the Right Application Security Te...
byKevin Fealey
 
DAST, SAST, Hybrid, Hybrid 2.0 & IAST - Methodology & Limitations
byiAppSecure Solutions
 
Managing Open Source in Application Security and Software Development Lifecycle
byBlack Duck by Synopsys
 
Open Source Libraries - Managing Risk in Cloud
bySuman Sourav
 
Application Security at DevOps Speed and Portfolio Scale
byJeff Williams
 
Devops security-An Insight into Secure-SDLC
bySuman Sourav
 

What's hot

PPTX
Secure Code review - Veracode SaaS Platform - Saudi Green Method
bySalil Kumar Subramony
 
PPTX
Continuous and Visible Security Testing with BDD-Security
byStephen de Vries
 
PPTX
DevSecOps: Securing Applications with DevOps
byWouter de Kort
 
PDF
Security Development Lifecycle Tools
byn|u - The Open Security Community
 
PDF
we45 - Web Application Security Testing Case Study
bywe45
 
PPTX
Veracode - Inglês
byDeServ - Tecnologia e Servços
 
PPTX
Secure Software Development Life Cycle
byMaurice Dawson
 
PDF
Software Security Assurance for DevOps
byBlack Duck by Synopsys
 
PPTX
BUSTED! How to Find Security Bugs Fast!
byParasoft
 
PDF
A Secure DevOps Journey
byVeracode
 
PPTX
Static Application Security Testing Strategies for Automation and Continuous ...
byKevin Fealey
 
PPTX
Unit testing : what are you missing for security
bySuman Sourav
 
PDF
Automation of Security scanning easy or cheese
byKatherine Golovinova
 
PPTX
Automation of Security scanning easy or cheese?
byDmitriy Gumeniuk
 
PPTX
September 13, 2016: Security in the Age of Open Source:
byBlack Duck by Synopsys
 
PDF
Pactera - App Security Assessment - Mobile, Web App, IoT - v2
byKyle Lai
 
PDF
Manual Code Review
byn|u - The Open Security Community
 
PDF
Bringing Security Testing to Development: How to Enable Developers to Act as ...
byAchim D. Brucker
 
PPTX
24may 1200 valday eric anklesaria 'secure sdlc – core banking'
byPositive Hack Days
 
PDF
A Successful SAST Tool Implementation
byCheckmarx
 
Secure Code review - Veracode SaaS Platform - Saudi Green Method
bySalil Kumar Subramony
 
Continuous and Visible Security Testing with BDD-Security
byStephen de Vries
 
DevSecOps: Securing Applications with DevOps
byWouter de Kort
 
Security Development Lifecycle Tools
byn|u - The Open Security Community
 
we45 - Web Application Security Testing Case Study
bywe45
 
Veracode - Inglês
byDeServ - Tecnologia e Servços
 
Secure Software Development Life Cycle
byMaurice Dawson
 
Software Security Assurance for DevOps
byBlack Duck by Synopsys
 
BUSTED! How to Find Security Bugs Fast!
byParasoft
 
A Secure DevOps Journey
byVeracode
 
Static Application Security Testing Strategies for Automation and Continuous ...
byKevin Fealey
 
Unit testing : what are you missing for security
bySuman Sourav
 
Automation of Security scanning easy or cheese
byKatherine Golovinova
 
Automation of Security scanning easy or cheese?
byDmitriy Gumeniuk
 
September 13, 2016: Security in the Age of Open Source:
byBlack Duck by Synopsys
 
Pactera - App Security Assessment - Mobile, Web App, IoT - v2
byKyle Lai
 
Manual Code Review
byn|u - The Open Security Community
 
Bringing Security Testing to Development: How to Enable Developers to Act as ...
byAchim D. Brucker
 
24may 1200 valday eric anklesaria 'secure sdlc – core banking'
byPositive Hack Days
 
A Successful SAST Tool Implementation
byCheckmarx
 

Similar to Sast 2021

PDF
Boundary Value Exploration for Software Analysis
byFelix Dobslaw
 
PDF
Using Diversity for Automated Boundary Value Testing
byFelix Dobslaw
 
PDF
A Future where we don’t write tests
byFelix Dobslaw
 
PDF
Past and Future of Software Testing and Analysis
byLionel Briand
 
PPTX
UNIT-IV SE_engineering_software_ipu.pptx
bykchitkara24
 
ODP
Black boxtesting
byfuntest06
 
PDF
wasp_2023.pdf
byFelix Dobslaw
 
PPT
lec-11 Testing.ppt
bydebjani12
 
PPT
AutoTest for software engineering for automated testing
byVishnuVardhan909561
 
PPT
Automation testing basics and tools presentation
byareebjafriv
 
PPT
AutoTest4444444444444444444444444444.ppt
bysaraswatatul491
 
PPT
AutoTest.ppt
byPrashanthJanakiraman
 
PPT
AutoTest.ppt
byRohit846825
 
PPT
AutoTest.ppt
byCHANDUKAYALA
 
PDF
Leaping over the Boundaries of Boundary Value Analysis
byTechWell
 
PPT
Testing foundations
byNeha Singh
 
PPTX
Testing strategies -2
byDivya Tiwari
 
PPTX
Test Case Design Techniques – Practical Examples & Best Practices in Software...
byMuhammad Fahad Bashir
 
PDF
TRANSFORMING SOFTWARE REQUIREMENTS INTO TEST CASES VIA MODEL TRANSFORMATION
byijseajournal
 
PPT
SWTesting this has impacted testing components
bysabarishsg16
 
Boundary Value Exploration for Software Analysis
byFelix Dobslaw
 
Using Diversity for Automated Boundary Value Testing
byFelix Dobslaw
 
A Future where we don’t write tests
byFelix Dobslaw
 
Past and Future of Software Testing and Analysis
byLionel Briand
 
UNIT-IV SE_engineering_software_ipu.pptx
bykchitkara24
 
Black boxtesting
byfuntest06
 
wasp_2023.pdf
byFelix Dobslaw
 
lec-11 Testing.ppt
bydebjani12
 
AutoTest for software engineering for automated testing
byVishnuVardhan909561
 
Automation testing basics and tools presentation
byareebjafriv
 
AutoTest4444444444444444444444444444.ppt
bysaraswatatul491
 
AutoTest.ppt
byPrashanthJanakiraman
 
AutoTest.ppt
byRohit846825
 
AutoTest.ppt
byCHANDUKAYALA
 
Leaping over the Boundaries of Boundary Value Analysis
byTechWell
 
Testing foundations
byNeha Singh
 
Testing strategies -2
byDivya Tiwari
 
Test Case Design Techniques – Practical Examples & Best Practices in Software...
byMuhammad Fahad Bashir
 
TRANSFORMING SOFTWARE REQUIREMENTS INTO TEST CASES VIA MODEL TRANSFORMATION
byijseajournal
 
SWTesting this has impacted testing components
bysabarishsg16
 

Recently uploaded

PDF
Software Development Company in India.pdf
byitswssoftware
 
PPTX
Best Application Development Consulting Company in Houston
byDesss Applying Technologies
 
PDF
apidays Paris 2025 - What If GraphQL Knew Accessibility.pdf
byapidays
 
PPTX
The Triple Bottom Line of Software: Uses, Misuses, and Strategic Risks
byrawaisah124
 
PDF
The Hidden Cost of “Quick” Web App Development
byiProgrammer Solutions Private Limited
 
PDF
Job Interview for Global Federal Government MD Panel Presentation
byafnupe09
 
PPTX
Aviation Fleet Management Software for Airlines & Operators
bySISGAIN
 
PPTX
Odoo vs SAP – Which ERP Is Better for Growing Businesses?
bySatishKumar2651
 
PDF
DSD-INT 2025 MODFLOW 6 and iMOD User Day - Introduction - Kroon
byDeltares
 
PPTX
Testing Strategies for Agile Teams in 2026
byMatt Calder
 
PPTX
Build Smarter with Google: A Workshop on MediaPipe, Firebase, Gemini and Anti...
byvanshikaprakash05
 
PDF
DSD-INT 2025 The National Hydrological Model of Denmark and its enhancements ...
byDeltares
 
PDF
Mitr Sarthi: A Smart ERP for Modern Manufacturers
byGamavis Software Solutions
 
PDF
DSD-INT 2025 Groundwatermodel Flanders coastal zone - Application of parallel...
byDeltares
 
PDF
DSD-INT 2025 Modelling cold water infiltration with the Groundwater Energy (G...
byDeltares
 
PDF
Oracle AI Database 26ai _ AI-Native Database for Enterprises.pdf
byAstuteBusiness
 
PDF
DSD-INT 2025 Flexible Reservoir Modelling with Ribasim - From Simple to Compl...
byDeltares
 
PDF
DSD-INT 2025 An Introduction to MODFLOW 6 Solver Settings (Without the Agoniz...
byDeltares
 
PDF
IT Asset Lifecycle Management Solution | Infraon
byEverestIMS Technologies Pvt. Ltd
 
PDF
Cloud Engineering Services | Infysion Technologies
bysolutioninginfysion
 
Software Development Company in India.pdf
byitswssoftware
 
Best Application Development Consulting Company in Houston
byDesss Applying Technologies
 
apidays Paris 2025 - What If GraphQL Knew Accessibility.pdf
byapidays
 
The Triple Bottom Line of Software: Uses, Misuses, and Strategic Risks
byrawaisah124
 
The Hidden Cost of “Quick” Web App Development
byiProgrammer Solutions Private Limited
 
Job Interview for Global Federal Government MD Panel Presentation
byafnupe09
 
Aviation Fleet Management Software for Airlines & Operators
bySISGAIN
 
Odoo vs SAP – Which ERP Is Better for Growing Businesses?
bySatishKumar2651
 
DSD-INT 2025 MODFLOW 6 and iMOD User Day - Introduction - Kroon
byDeltares
 
Testing Strategies for Agile Teams in 2026
byMatt Calder
 
Build Smarter with Google: A Workshop on MediaPipe, Firebase, Gemini and Anti...
byvanshikaprakash05
 
DSD-INT 2025 The National Hydrological Model of Denmark and its enhancements ...
byDeltares
 
Mitr Sarthi: A Smart ERP for Modern Manufacturers
byGamavis Software Solutions
 
DSD-INT 2025 Groundwatermodel Flanders coastal zone - Application of parallel...
byDeltares
 
DSD-INT 2025 Modelling cold water infiltration with the Groundwater Energy (G...
byDeltares
 
Oracle AI Database 26ai _ AI-Native Database for Enterprises.pdf
byAstuteBusiness
 
DSD-INT 2025 Flexible Reservoir Modelling with Ribasim - From Simple to Compl...
byDeltares
 
DSD-INT 2025 An Introduction to MODFLOW 6 Solver Settings (Without the Agoniz...
byDeltares
 
IT Asset Lifecycle Management Solution | Infraon
byEverestIMS Technologies Pvt. Ltd
 
Cloud Engineering Services | Infysion Technologies
bysolutioninginfysion
 

Sast 2021

  • 1.
    1/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Exploring the Boundaries of our Software Assist. Prof. Felix Dobslaw 25/5/2021 10:50-11:40 CEST https://www.flickr.com/photos/a_siegel/2356136219
  • 2.
    2/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Exploring the Boundaries of our Software Assist. Prof. Felix Dobslaw 25/5/2021 10:50-11:40 CEST G https://www.flickr.com/photos/a_siegel/2356136219 Based on collaboration with Robert Feldt, Francisco Gomes de Oliveira Neto
  • 3.
    3/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Outline ● Why Boundaries, why exploration? ● Boundary Value…. Testing/Analysis/Exploration – Terminology – Process ● Outlook
  • 4.
    4/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Research Questions
  • 5.
    5/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Research Questions ● How can we automatically find areas of relevance for testing in our software? – “Boundaries”
  • 6.
    6/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Research Questions ● How can we automatically find areas of relevance for testing in our software? – “Boundaries” ● How can we practically support testers in their working routine? – Better fault finding and efficiency
  • 7.
    7/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) What do we mean by test automation?
  • 8.
    8/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) What do we mean by test automation? Not classical CI/CD running human created test cases automatically
  • 9.
    9/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) What do we mean by test automation? Not classical CI/CD running human created test cases automatically But running machine created test cases automatically
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
    https://time.com/3785942/blue-marble/ Defining Boundaries ishard Test Automation Augmentation Diversity
  • 15.
    15/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) inputs outputs requirements constraints
  • 16.
    16/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification: inputs outputs requirements constraints Entirely implicit Greatly detailed
  • 17.
    17/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification: inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
  • 18.
    18/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
  • 19.
    19/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
  • 20.
    20/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
  • 21.
    21/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model
  • 22.
    22/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
  • 23.
    23/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
  • 24.
    24/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
  • 25.
    25/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System
  • 26.
    26/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests validate Specification inputs outputs requirements constraints Entirely implicit Greatly detailed Waterfall Model Goal vs. System Correct software depends on correct boundaries. Boundary Value Exploration reveals discrepancies between desired and actual behavior.
  • 27.
    27/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se)
  • 28.
    28/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Testing
  • 29.
    29/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Testing BVT: “Execution of specific input pairs in order to ensure that an actual boundary is also expected.”
  • 30.
    30/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing BVT: “Execution of specific input pairs in order to ensure that an actual boundary is also expected.”
  • 31.
    31/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) BVA: “Analysis of artifacts of the software development process to clarify the expected and actual boundaries of a software.” Boundary Value Analysis Boundary Value Testing BVT: “Execution of specific input pairs in order to ensure that an actual boundary is also expected.”
  • 32.
    32/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates
  • 33.
    33/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates
  • 34.
    34/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates BVE: “A collection of techniques that select or help select inputs to detect and identify boundary candidates.”
  • 35.
    35/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Boundary Value Analysis Boundary Value Testing Boundary Value Exploration Candidates BVE: “A collection of techniques that select or help select inputs to detect and identify boundary candidates.”
  • 36.
    36/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count
  • 37.
    37/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count bytecount(int) → human readable string
  • 38.
    38/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count bytecount(int) → human readable string
  • 39.
    39/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) byte-count bytecount(int) → human readable string
  • 40.
    40/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints BVE Process Example
  • 41.
    41/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints
  • 42.
    42/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining
  • 43.
    43/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB”
  • 44.
    44/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB”
  • 45.
    45/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB”
  • 46.
    46/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
  • 47.
    47/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
  • 48.
    48/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification Pull Request Update: Bc(1_000_000...000) → last valid Bc(1_000_000...001) → ArgumentError(“out of bounce”) program Specification: tests inputs outputs requirements constraints boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” testing → quality assurance not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
  • 49.
    49/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification Pull Request Update: Bc(1_000_000...000) → last valid Bc(1_000_000...001) → ArgumentError(“out of bounce”) program Specification: tests inputs outputs requirements constraints boundary implementation boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” testing → quality assurance not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
  • 50.
    50/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Specification Pull Request Update: Bc(1_000_000...000) → last valid Bc(1_000_000...001) → ArgumentError(“out of bounce”) program Specification: tests inputs outputs requirements constraints boundary implementation boundary mining bc(99999…...16) → BoundsError(“kMGTPE”, 9) bc(99999…...17) → BoundsError(“kMGTPE”, 10) Standardized Specification Syntax: bc(999949) → “999.9 kB” bc(999950) → “1.0 MB” “clear case” testing → quality assurance not so clear case... bc(999999) → “1000.0 kB” bc(1000000) → “1.0 MB” Specification Pull Request:
  • 51.
    51/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) How to automatically mine boundaries? ● Without oracle, or only partial information, only having the SUT at hand.
  • 52.
    Absolute vs. RelativeAssessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 53.
    SUT x y Absolute vs.Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 54.
    SUT x y Assert y== yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 55.
    SUT x y Assert y== yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 56.
    SUT x y x1 x2 y1y2 SUT SUT Assert y == yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 57.
    SUT x y x1 x2 y1y2 ? SUT SUT Assert y == yexp Absolute vs. Relative Assessment R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 58.
    SUT x y x1 x2 y1y2 ? SUT SUT Assert y == yexp Absolute vs. Relative Assessment Distance Metric Examples: Jaccard, Hamming, Euclidean… Very basic one: StringLength R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 59.
    SUT x y x1 x2 y1y2 ? SUT SUT Assert y == yexp Absolute vs. Relative Assessment Distance Metric Examples: Jaccard, Hamming, Euclidean… Very basic one: StringLength Example: 30 and 31 are neighbors in the input space for field month: Date(2021, 30, 4) → “30/4/2021” StringLength is 9 Date(2021, 31, 4) → “ERROR, month field out of bounds.” StringLength is 33 Big difference, this seems to be a boundary candidate! But is it an interesting one?... R. Feldt and F. Dobslaw. "Towards automated boundary value testing with program derivatives and search." International Symposium on Search Based Software Engineering. Springer, Cham, 2019.
  • 60.
    60/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) F. Dobslaw, F. G. de Oliveira Neto and R. Feldt, "Boundary Value Exploration for Software Analysis," 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2020, pp. 346-353, doi: 10.1109/ICSTW50294.2020.00062.
  • 61.
    61/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Interactive Boundary Exploration F. Dobslaw, F. G. de Oliveira Neto and R. Feldt, "Boundary Value Exploration for Software Analysis," 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2020, pp. 346-353, doi: 10.1109/ICSTW50294.2020.00062.
  • 62.
    62/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) - Go to www.menti.com - Enter the code 6546 248 - Answer 1 question. Thanks! Questions, Comments?
  • 63.
    63/63 --- Exploringthe Boundaries of our Software, SAST 2021, Felix Dobslaw (felix.dobslaw@miun.se) Please partake in an eye-tracking study! https://studenter.miun.se/~evth1400/EyeGazeStudy/?group=3 Link