SlideShare a Scribd company logo

Agile testing for embedded software development

A large part of the success of agile adoptions is due to the automated testing approach used in agile projects. Because many of these techniques were pioneered in the development of web applications it can be hard to see how these techniques can be leveraged for a project where the software being built is for an embedded application. Discover ways to leverage agile testing techniques for embedded systems. Whether you are building a medical device, embedded controller, or Internet of Things device learn how to leverage these testing practices to create fully automated tests that fit into a DevOps build pipeline and help your team create higher quality, more reliable software. Test automation is the best way to maintain and execute a comprehensive suite of regression tests that allows you to take back control of your testing process while increasing test coverage. Learn how to be in control of your test process by stepping up your test automation to the next level. Embedded development and Internet of Things development is often done on platforms that lack modern software development and test automation tools. The more esoteric or the smaller the target audience, the less likely tool vendors are to create products that directly support the deployment environment. This can make getting started with test automation using older tools that are not as actively supported by vendors can be a challenge that has to be overcome by a team that wants to move toward a Continuous Deployment process. This session is aimed at people that are trying to adopt agile and continuous delivery with embedded technology, but might be worried that it can’t work in their particular environment due to their industry, technology stack, culture, or regulatory environment.

1 of 29
Download to read offline
© COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 1
Agility. Security. Delivered.
Agile Testing for Embedded and
IoT Software Development
© COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 2
About Coveros
• Coveros builds security-critical applications using
agile methods.
• Coveros Services
• Agile transformations
• Agile development and testing
• DevOps and continuous integration
• Application security analysis
• Agile & Security training
• Government qualifications
• DCAA approved rates and accounting
• TS facility clearance
Areas of Expertise
© COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 3
What is embedded software
• Embedded software is software that controls a device
• A device is a combination of software, firmware and hardware that
make up a system
• The system has hardware components that are either sensors or
manipulators
• The software and firmware control the hardware components, using
the sensors to discover and the manipulators to carry out actions
• Examples include:
• Medical devices
• Self driving cars
• Fitness wearables
• HVAC systems
© COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 4
What is agile testing
• Agile testing is the the set of practices used to build quality in and
make sure that software created by the team is of high quality
• The whole team is responsible for quality
• Test automation is integrated into the Continuous Integration and
Continuous Delivery pipeline
• Testing of new functionality is conducted during the same sprint or
iteration in which the functionality is developed
• Unit testing is used as part of the quality assurance process
• Acceptance testing is conducted using acceptance criteria created
directly from the requirements
© COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 5
User Acceptance Testing (UAT)
• UAT is the process of getting stakeholders to write down what results
they want to see from the software in order to determine it works as
the requirements intended
• Acceptance criteria are written before development of the software
begins
• Acceptance criteria are a form of requirements that are used to let
the team know what key aspects of the requirements need to be met
in order for the stakeholders to feel that they can accept the software
increment
• Testing of acceptance criteria happens at the end of an increment and
gives the team feedback on how well they have created software that
the stakeholders expected.
© COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 6
Behavior Driven Development (BDD)
• BDD is a test automation process for testing software based on the
requirements
• The tests are written before the code is written or at least
independently of the code
• For some teams BDD is used to automate UAT tests
• Many BDD tools exist such as Cucumber and Behave
• Most tools use the Gherkin language, which uses the following
format:
• Given initial condition
• When some action or trigger
• Then resulting action

Recommended

Flight East 2018 Presentation–Continuous Integration––An Overview
Flight East 2018 Presentation–Continuous Integration––An OverviewFlight East 2018 Presentation–Continuous Integration––An Overview
Flight East 2018 Presentation–Continuous Integration––An OverviewSynopsys Software Integrity Group
 
Software testing-and-analysis
Software testing-and-analysisSoftware testing-and-analysis
Software testing-and-analysisWBUTTUTORIALS
 
Game Development Best Practices
Game Development Best PracticesGame Development Best Practices
Game Development Best PracticesPerforce
 
Improving Quality through Continuous Integration - A case study of CollabNet
Improving Quality through Continuous Integration - A case study of CollabNetImproving Quality through Continuous Integration - A case study of CollabNet
Improving Quality through Continuous Integration - A case study of CollabNetVenkat Janardhanam, MS, MBA
 
8 Reasons Your Team Should Connect to the Rocket ALM Hub
8 Reasons Your Team Should Connect to the Rocket ALM Hub8 Reasons Your Team Should Connect to the Rocket ALM Hub
8 Reasons Your Team Should Connect to the Rocket ALM HubRocket Software ALM
 
Presentation on 3 Pillars of DevOps - Kovair DevOps
Presentation on 3 Pillars of DevOps - Kovair DevOpsPresentation on 3 Pillars of DevOps - Kovair DevOps
Presentation on 3 Pillars of DevOps - Kovair DevOpsKovair
 
Verification at scale: Fitting static code analysis into continuous integration
Verification at scale: Fitting static code analysis into continuous integrationVerification at scale: Fitting static code analysis into continuous integration
Verification at scale: Fitting static code analysis into continuous integrationRogue Wave Software
 

More Related Content

What's hot

DevOps with Microsoft Stack
DevOps with Microsoft StackDevOps with Microsoft Stack
DevOps with Microsoft StackDeepti Jain
 
Design Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesDesign Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesInductive Automation
 
Overview of Kovair Omnibus Integration Platform
Overview of Kovair Omnibus Integration PlatformOverview of Kovair Omnibus Integration Platform
Overview of Kovair Omnibus Integration PlatformKovair
 
Integrated Test Management
Integrated Test ManagementIntegrated Test Management
Integrated Test ManagementKovair
 
Architecting for the cloud storage build test
Architecting for the cloud storage build testArchitecting for the cloud storage build test
Architecting for the cloud storage build testLen Bass
 
The Best of Both Worlds: Agile Development and Fast Compliance
The Best of Both Worlds: Agile Development and Fast ComplianceThe Best of Both Worlds: Agile Development and Fast Compliance
The Best of Both Worlds: Agile Development and Fast CompliancePerforce
 
Kovair Migration Solution for ALM Tools
Kovair Migration Solution for ALM ToolsKovair Migration Solution for ALM Tools
Kovair Migration Solution for ALM ToolsKovair
 
Experience in teaching devops
Experience in teaching devopsExperience in teaching devops
Experience in teaching devopsLen Bass
 
Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter Hima Bindu Kosuru
 
Kovair at STeP-IN Summit 2014 Conference
Kovair at STeP-IN Summit 2014 ConferenceKovair at STeP-IN Summit 2014 Conference
Kovair at STeP-IN Summit 2014 ConferenceKovair
 
The Continuous delivery value - Funaro
The Continuous delivery value - FunaroThe Continuous delivery value - Funaro
The Continuous delivery value - FunaroCodemotion
 
SCM Transformation Challenges and How to Overcome Them
SCM Transformation Challenges and How to Overcome ThemSCM Transformation Challenges and How to Overcome Them
SCM Transformation Challenges and How to Overcome ThemCompuware
 
Performance Testing Cloud-Based Systems
Performance Testing Cloud-Based SystemsPerformance Testing Cloud-Based Systems
Performance Testing Cloud-Based SystemsTechWell
 
Performance testing
Performance testingPerformance testing
Performance testingJyoti Babbar
 
What's new in visual studio 2013
What's new in visual studio 2013What's new in visual studio 2013
What's new in visual studio 2013Keith Lopez
 

What's hot (19)

Software testing and analysis
Software testing and analysisSoftware testing and analysis
Software testing and analysis
 
DevOps with Microsoft Stack
DevOps with Microsoft StackDevOps with Microsoft Stack
DevOps with Microsoft Stack
 
Design Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best PracticesDesign Like a Pro: Scripting Best Practices
Design Like a Pro: Scripting Best Practices
 
Overview of Kovair Omnibus Integration Platform
Overview of Kovair Omnibus Integration PlatformOverview of Kovair Omnibus Integration Platform
Overview of Kovair Omnibus Integration Platform
 
Integrated Test Management
Integrated Test ManagementIntegrated Test Management
Integrated Test Management
 
Architecting for the cloud storage build test
Architecting for the cloud storage build testArchitecting for the cloud storage build test
Architecting for the cloud storage build test
 
QualiTest
QualiTestQualiTest
QualiTest
 
The Best of Both Worlds: Agile Development and Fast Compliance
The Best of Both Worlds: Agile Development and Fast ComplianceThe Best of Both Worlds: Agile Development and Fast Compliance
The Best of Both Worlds: Agile Development and Fast Compliance
 
Kovair Migration Solution for ALM Tools
Kovair Migration Solution for ALM ToolsKovair Migration Solution for ALM Tools
Kovair Migration Solution for ALM Tools
 
Experience in teaching devops
Experience in teaching devopsExperience in teaching devops
Experience in teaching devops
 
Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter Software Testing includes Performance testing with Load Runner and the JMeter
Software Testing includes Performance testing with Load Runner and the JMeter
 
Kovair at STeP-IN Summit 2014 Conference
Kovair at STeP-IN Summit 2014 ConferenceKovair at STeP-IN Summit 2014 Conference
Kovair at STeP-IN Summit 2014 Conference
 
ProSET Brochure
ProSET BrochureProSET Brochure
ProSET Brochure
 
The Continuous delivery value - Funaro
The Continuous delivery value - FunaroThe Continuous delivery value - Funaro
The Continuous delivery value - Funaro
 
Chapter6
Chapter6Chapter6
Chapter6
 
SCM Transformation Challenges and How to Overcome Them
SCM Transformation Challenges and How to Overcome ThemSCM Transformation Challenges and How to Overcome Them
SCM Transformation Challenges and How to Overcome Them
 
Performance Testing Cloud-Based Systems
Performance Testing Cloud-Based SystemsPerformance Testing Cloud-Based Systems
Performance Testing Cloud-Based Systems
 
Performance testing
Performance testingPerformance testing
Performance testing
 
What's new in visual studio 2013
What's new in visual studio 2013What's new in visual studio 2013
What's new in visual studio 2013
 

Similar to Agile testing for embedded software development

Testing in the new age of DevOps
Testing in the new age of DevOpsTesting in the new age of DevOps
Testing in the new age of DevOpsMoataz Mahmoud
 
How to go from waterfall app dev to secure agile development in 2 weeks
How to go from waterfall app dev to secure agile development in 2 weeks How to go from waterfall app dev to secure agile development in 2 weeks
How to go from waterfall app dev to secure agile development in 2 weeks Ulf Mattsson
 
software testing
 software testing software testing
software testingSara shall
 
DevOps Overview in my own words
DevOps Overview in my own wordsDevOps Overview in my own words
DevOps Overview in my own wordsSUBHENDU KARMAKAR
 
DevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the AutomationDevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the AutomationKeith Pleas
 
DevOps_service.pptx
DevOps_service.pptxDevOps_service.pptx
DevOps_service.pptxphamvinhcntt
 
Automation Tools Overview
Automation Tools OverviewAutomation Tools Overview
Automation Tools OverviewMurageppa-QA
 
SQA_Lec#01-1.ppt
SQA_Lec#01-1.pptSQA_Lec#01-1.ppt
SQA_Lec#01-1.pptAhmad Abbas
 
Automation Tool Overview
Automation Tool OverviewAutomation Tool Overview
Automation Tool OverviewANKUR-BA
 
Automation Tools Overview
Automation Tools OverviewAutomation Tools Overview
Automation Tools OverviewSachin-QA
 
SENG202-v-and-v-modeling_121810.pptx
SENG202-v-and-v-modeling_121810.pptxSENG202-v-and-v-modeling_121810.pptx
SENG202-v-and-v-modeling_121810.pptxMinsasWorld
 
26.1a.Introduction to DEVOPS_v2.pptx
26.1a.Introduction to DEVOPS_v2.pptx26.1a.Introduction to DEVOPS_v2.pptx
26.1a.Introduction to DEVOPS_v2.pptxPanos Fitsilis
 
Software Testing - Software Quality
Software Testing - Software QualitySoftware Testing - Software Quality
Software Testing - Software QualityAjeng Savitri
 
Experiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD ProjectExperiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD ProjectGene Gotimer
 
verification and validation
verification and validationverification and validation
verification and validationDinesh Pasi
 
When Medical Device Software Fails Due to Improper Verification & Validation ...
When Medical Device Software Fails Due to Improper Verification & Validation ...When Medical Device Software Fails Due to Improper Verification & Validation ...
When Medical Device Software Fails Due to Improper Verification & Validation ...Sterling Medical Devices
 
Create code confidence for better application security
Create code confidence for better application security Create code confidence for better application security
Create code confidence for better application security Rogue Wave Software
 
_VoicePPT_QA_Testing_Training_4_Days_Schedule.ppt
_VoicePPT_QA_Testing_Training_4_Days_Schedule.ppt_VoicePPT_QA_Testing_Training_4_Days_Schedule.ppt
_VoicePPT_QA_Testing_Training_4_Days_Schedule.pptAnilKumarARS
 
Software testing methods, levels and types
Software testing methods, levels and typesSoftware testing methods, levels and types
Software testing methods, levels and typesConfiz
 

Similar to Agile testing for embedded software development (20)

Testing in the new age of DevOps
Testing in the new age of DevOpsTesting in the new age of DevOps
Testing in the new age of DevOps
 
How to go from waterfall app dev to secure agile development in 2 weeks
How to go from waterfall app dev to secure agile development in 2 weeks How to go from waterfall app dev to secure agile development in 2 weeks
How to go from waterfall app dev to secure agile development in 2 weeks
 
software testing
 software testing software testing
software testing
 
DevOps Overview in my own words
DevOps Overview in my own wordsDevOps Overview in my own words
DevOps Overview in my own words
 
DevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the AutomationDevOps-as-a-Service: Towards Automating the Automation
DevOps-as-a-Service: Towards Automating the Automation
 
DevOps_service.pptx
DevOps_service.pptxDevOps_service.pptx
DevOps_service.pptx
 
Automation Tools Overview
Automation Tools OverviewAutomation Tools Overview
Automation Tools Overview
 
SQA_Lec#01-1.ppt
SQA_Lec#01-1.pptSQA_Lec#01-1.ppt
SQA_Lec#01-1.ppt
 
Software Development
Software DevelopmentSoftware Development
Software Development
 
Automation Tool Overview
Automation Tool OverviewAutomation Tool Overview
Automation Tool Overview
 
Automation Tools Overview
Automation Tools OverviewAutomation Tools Overview
Automation Tools Overview
 
SENG202-v-and-v-modeling_121810.pptx
SENG202-v-and-v-modeling_121810.pptxSENG202-v-and-v-modeling_121810.pptx
SENG202-v-and-v-modeling_121810.pptx
 
26.1a.Introduction to DEVOPS_v2.pptx
26.1a.Introduction to DEVOPS_v2.pptx26.1a.Introduction to DEVOPS_v2.pptx
26.1a.Introduction to DEVOPS_v2.pptx
 
Software Testing - Software Quality
Software Testing - Software QualitySoftware Testing - Software Quality
Software Testing - Software Quality
 
Experiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD ProjectExperiences Bringing CD to a DoD Project
Experiences Bringing CD to a DoD Project
 
verification and validation
verification and validationverification and validation
verification and validation
 
When Medical Device Software Fails Due to Improper Verification & Validation ...
When Medical Device Software Fails Due to Improper Verification & Validation ...When Medical Device Software Fails Due to Improper Verification & Validation ...
When Medical Device Software Fails Due to Improper Verification & Validation ...
 
Create code confidence for better application security
Create code confidence for better application security Create code confidence for better application security
Create code confidence for better application security
 
_VoicePPT_QA_Testing_Training_4_Days_Schedule.ppt
_VoicePPT_QA_Testing_Training_4_Days_Schedule.ppt_VoicePPT_QA_Testing_Training_4_Days_Schedule.ppt
_VoicePPT_QA_Testing_Training_4_Days_Schedule.ppt
 
Software testing methods, levels and types
Software testing methods, levels and typesSoftware testing methods, levels and types
Software testing methods, levels and types
 

More from Tom Stiehm

Shifting security all day dev ops
Shifting security all day dev opsShifting security all day dev ops
Shifting security all day dev opsTom Stiehm
 
Shifting Security Left - The Innovation of DevSecOps - ValleyTechCon
Shifting Security Left - The Innovation of DevSecOps - ValleyTechConShifting Security Left - The Innovation of DevSecOps - ValleyTechCon
Shifting Security Left - The Innovation of DevSecOps - ValleyTechConTom Stiehm
 
Shifting Security Left - The Innovation of DevSecOps - AgileDC
Shifting Security Left - The Innovation of DevSecOps - AgileDCShifting Security Left - The Innovation of DevSecOps - AgileDC
Shifting Security Left - The Innovation of DevSecOps - AgileDCTom Stiehm
 
Shifting Security Left from the Lean+Agile 2019 Conference
Shifting Security Left from the Lean+Agile 2019 ConferenceShifting Security Left from the Lean+Agile 2019 Conference
Shifting Security Left from the Lean+Agile 2019 ConferenceTom Stiehm
 
Failure is inevitable but it isn't permanent
Failure is inevitable but it isn't permanentFailure is inevitable but it isn't permanent
Failure is inevitable but it isn't permanentTom Stiehm
 
Overcoming problems implementing cloud based dev ops for distributed agile pr...
Overcoming problems implementing cloud based dev ops for distributed agile pr...Overcoming problems implementing cloud based dev ops for distributed agile pr...
Overcoming problems implementing cloud based dev ops for distributed agile pr...Tom Stiehm
 
Implementing cloud based devops for distributed agile projects
Implementing cloud based devops for distributed agile projectsImplementing cloud based devops for distributed agile projects
Implementing cloud based devops for distributed agile projectsTom Stiehm
 
Integrating security into Continuous Delivery
Integrating security into Continuous DeliveryIntegrating security into Continuous Delivery
Integrating security into Continuous DeliveryTom Stiehm
 

More from Tom Stiehm (8)

Shifting security all day dev ops
Shifting security all day dev opsShifting security all day dev ops
Shifting security all day dev ops
 
Shifting Security Left - The Innovation of DevSecOps - ValleyTechCon
Shifting Security Left - The Innovation of DevSecOps - ValleyTechConShifting Security Left - The Innovation of DevSecOps - ValleyTechCon
Shifting Security Left - The Innovation of DevSecOps - ValleyTechCon
 
Shifting Security Left - The Innovation of DevSecOps - AgileDC
Shifting Security Left - The Innovation of DevSecOps - AgileDCShifting Security Left - The Innovation of DevSecOps - AgileDC
Shifting Security Left - The Innovation of DevSecOps - AgileDC
 
Shifting Security Left from the Lean+Agile 2019 Conference
Shifting Security Left from the Lean+Agile 2019 ConferenceShifting Security Left from the Lean+Agile 2019 Conference
Shifting Security Left from the Lean+Agile 2019 Conference
 
Failure is inevitable but it isn't permanent
Failure is inevitable but it isn't permanentFailure is inevitable but it isn't permanent
Failure is inevitable but it isn't permanent
 
Overcoming problems implementing cloud based dev ops for distributed agile pr...
Overcoming problems implementing cloud based dev ops for distributed agile pr...Overcoming problems implementing cloud based dev ops for distributed agile pr...
Overcoming problems implementing cloud based dev ops for distributed agile pr...
 
Implementing cloud based devops for distributed agile projects
Implementing cloud based devops for distributed agile projectsImplementing cloud based devops for distributed agile projects
Implementing cloud based devops for distributed agile projects
 
Integrating security into Continuous Delivery
Integrating security into Continuous DeliveryIntegrating security into Continuous Delivery
Integrating security into Continuous Delivery
 

Recently uploaded

Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellencePrecisely
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfIsidro Navarro
 
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro KozhevinFwdays
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanDatabarracks
 
Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?MENGSAYLOEM1
 
Importance of magazines in education ppt
Importance of magazines in education pptImportance of magazines in education ppt
Importance of magazines in education pptsafnarafeek2002
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaISPMAIndia
 
Dynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringDynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringMassimo Talia
 
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner,  Challenge Like a VC by former CPO, TripadvisorAct Like an Owner,  Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner, Challenge Like a VC by former CPO, TripadvisorProduct School
 
"How we created an SRE team in Temabit as a part of FOZZY Group in conditions...
"How we created an SRE team in Temabit as a part of FOZZY Group in conditions..."How we created an SRE team in Temabit as a part of FOZZY Group in conditions...
"How we created an SRE team in Temabit as a part of FOZZY Group in conditions...Fwdays
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...DianaGray10
 
"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura RochniakFwdays
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stackSummit
 
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes", Volodymyr TsapFwdays
 
Apex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptxApex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptxmohayyudin7826
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...shaiyuvasv
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build PolandGDSC PJATK
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, GoogleISPMAIndia
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERNRonnelBaroc
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVARobert McDermott
 

Recently uploaded (20)

Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center Excellence
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdf
 
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response Plan
 
Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?
 
Importance of magazines in education ppt
Importance of magazines in education pptImportance of magazines in education ppt
Importance of magazines in education ppt
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
 
Dynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineeringDynamical systems simulation in Python for science and engineering
Dynamical systems simulation in Python for science and engineering
 
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner,  Challenge Like a VC by former CPO, TripadvisorAct Like an Owner,  Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
 
"How we created an SRE team in Temabit as a part of FOZZY Group in conditions...
"How we created an SRE team in Temabit as a part of FOZZY Group in conditions..."How we created an SRE team in Temabit as a part of FOZZY Group in conditions...
"How we created an SRE team in Temabit as a part of FOZZY Group in conditions...
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
 
"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stack
 
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
 
Apex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptxApex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptx
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build Poland
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
 
Introduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVAIntroduction to Multimodal LLMs with LLaVA
Introduction to Multimodal LLMs with LLaVA
 

Agile testing for embedded software development

  • 1. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 1 Agility. Security. Delivered. Agile Testing for Embedded and IoT Software Development
  • 2. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 2 About Coveros • Coveros builds security-critical applications using agile methods. • Coveros Services • Agile transformations • Agile development and testing • DevOps and continuous integration • Application security analysis • Agile & Security training • Government qualifications • DCAA approved rates and accounting • TS facility clearance Areas of Expertise
  • 3. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 3 What is embedded software • Embedded software is software that controls a device • A device is a combination of software, firmware and hardware that make up a system • The system has hardware components that are either sensors or manipulators • The software and firmware control the hardware components, using the sensors to discover and the manipulators to carry out actions • Examples include: • Medical devices • Self driving cars • Fitness wearables • HVAC systems
  • 4. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 4 What is agile testing • Agile testing is the the set of practices used to build quality in and make sure that software created by the team is of high quality • The whole team is responsible for quality • Test automation is integrated into the Continuous Integration and Continuous Delivery pipeline • Testing of new functionality is conducted during the same sprint or iteration in which the functionality is developed • Unit testing is used as part of the quality assurance process • Acceptance testing is conducted using acceptance criteria created directly from the requirements
  • 5. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 5 User Acceptance Testing (UAT) • UAT is the process of getting stakeholders to write down what results they want to see from the software in order to determine it works as the requirements intended • Acceptance criteria are written before development of the software begins • Acceptance criteria are a form of requirements that are used to let the team know what key aspects of the requirements need to be met in order for the stakeholders to feel that they can accept the software increment • Testing of acceptance criteria happens at the end of an increment and gives the team feedback on how well they have created software that the stakeholders expected.
  • 6. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 6 Behavior Driven Development (BDD) • BDD is a test automation process for testing software based on the requirements • The tests are written before the code is written or at least independently of the code • For some teams BDD is used to automate UAT tests • Many BDD tools exist such as Cucumber and Behave • Most tools use the Gherkin language, which uses the following format: • Given initial condition • When some action or trigger • Then resulting action
  • 7. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 7 Test Driven Development (TDD) • TDD, also called Test First Development, is the process of using unit tests to help design the code for an application • The developer will write a test of what they intend the code to do and then write the code to make the test pass • Red, Green, Refactor cycle • This has a number of positive results: • Makes the code more testable • Makes the code more modular or self contained • Makes the code more maintainable • Gives the code a regression test suite • Makes the intent of the code easier to understand
  • 8. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 8 Continuous Integration (CI) • CI is the process of integrating and testing code upon check in • In CI, integration is the process of integrating a developer’s code with the code already in the repository • The goal of CI is to make sure the code can build and pass an initial set of regression tests before other testing and quality assurance is applied to a specific build • CI uses a build server such as Jenkins to do a clean build of the code and run a suite of unit tests • If the build breaks, the people that checked in since the last successful build are informed and they are expected to get the build back to a working state
  • 9. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 9 Continuous Delivery (CD) • CD is the process of taking a build through a build pipeline that consists of a CI process followed by several quality gates that verify the code through various quality checks including: • Automated functional tests • Static code analysis • Automated non-functional tests like performance and security • Automated deployment into progressively higher environments • It could include on demand deployment to a manual test environment • On demand deployment to a UAT environment • On demand deployment to a demo environment • Continuous Deployment is automatically progressing to production including automatically deploying into production
  • 10. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 10 Value Stream Mapping • Value stream mapping is the process of determining how a product is created in your environment from inception to deployment and use in production • By creating a value stream map you can figure out how many steps you have in your delivery process and how long each step takes • Once you have a map of your process you can start to optimize and automate the different steps to streamline your process • The goal is to reduce the time it takes to move requirements through your process and get new features into production
  • 11. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 11 Build Pipeline • The build pipeline is the part of your value stream that takes checked in code and makes it into high quality, tested software that is either in production or ready to be distributed to customers • CI/CD are a subset of your build pipeline • Automation, including test automation, is the only way to make a build pipeline work in a fast and efficient manner • Many of the quality gates in a build pipeline are testing focused and require strong testing acumen in order to help deliver high quality software
  • 12. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 12 Version Control • Version control is the process of managing the code over time by allowing a project to keep track of the state of the code as it progresses • All code, including testing code and test automation code needs to be checked into version control • Application code and testing code should be versioned together since the test code acts on the application and changes over time to keep in sync with changes to the application code • Version control tools include: • Subversion • Git • Others
  • 13. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 13 Configuration Management • Configuration Management are the tools and process used to determine how software is setup in any given environment and how to account for all of the different files needed to make the software work • This includes binaries, configuration files and environment settings • Most build pipelines have a component that manages configuration elements for each deployment so that the team can know exactly what is deployed and tested for every environment
  • 14. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 14 Testing Software vs. Hardware vs. System • With embedded systems you are testing three things: • Software – The code that manages the system • Hardware – The mechanical and/or electrical parts of the system • System – The combination of hardware and software • Sometimes firmware is tested like software, sometimes it is tested like hardware (depending on how firm it is) • Isolating what you are testing can help you create test automation that covers a much of the product as possible • Some amount of manual testing might be unavoidable
  • 15. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 15 Software Testing • During software testing you are testing only the software part of your system • This means that all input (sensor data) should be supplied by a known source, if possible by recording the data and playing it back, or simulating it • Output can be ignored or simulated • Record and playback, or simulation can be less expensive and easier to setup • You should be focused on testing the business logic of the software, i.e. the calculations and determinations that software makes • You can also focus on testing the user interface or interactive elements of the system
  • 16. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 16 Hardware Testing • Hardware testing can be agile as well • Hardware can have a different cadence than software • Agile hardware testing is about automating as much of the testing process as possible • Software tools, both homegrown and third party, can be used to automate hardware testing • Using the system software the controls that hardware to test the hardware is acceptable but keep the focus on the hardware
  • 17. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 17 System Testing • System testing is the combination of the software and hardware working together • Depending on the nature of your embedded system, this can be hard to automated or automate completely • Depending on the nature of your embedded system there may be a good deal of tooling that can help you test the system or there may be nothing at all • Testing the system is critical because you don’t know how the software and hardware will interact until you actually have them interact
  • 18. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 18 Test Automation vs. Manual Testing • Agile testing focuses on test automation and in some types of applications it is possible to automate nearly all of the testing • With the focus on test automaton it is easy to think that manually testing has no place in agile but that isn’t true • Manual testing is often a precursor to functional test automation • Manual testing can be cheaper or more effective for some features of an embedded system • Exploratory testing can exercise the system in a way that test automation will not and find deeper issues
  • 19. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 19 How much can you automate? • Look at this from a number of points of view: • What can you afford • What do you have the skills or abilities to automate • What is possible • What is the most beneficial • Finding a balance between test automation and manual testing will be a challenge that all projects face • The answer will change over time as you learn and as your code improves
  • 20. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 20 Coding Standards (for tests too) • Using coding standards means: • Making the coding standards • Enforcing coding standards using code reviews • It doesn’t matter what specific coding standards you use so much as you have them and the team agrees to them • If possible, use an existing coding standard for your technology • Make coding standards compliance part of your code review • Better yet, use a code formatter to apply the standard during code commit • Apply coding standards to your tests as well
  • 21. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 21 Code Inspections • Code inspections include a number of things: • Code reviews • Static code analysis • Secure code reviews • Format and style reviews • Automate what you can, format and style are easy to automate • Static code analysis can be a good input into code reviews • Code review tools can help, things like Crucible and ReviewBoard • A secure code review is different than a code review and is a different skill set, you should master both
  • 22. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 22 Unit Testing • Unit tests are developer tests that are used to make sure that the code that they write behaves as the developer intended • Unit tests are written in the language and technology platform of the application code and often use a framework like Unity (see http://www.throwtheswitch.org/unity/) • Unit tests might be run on the dev platform or the target platform, this often depends on how the tooling works • Sample code: void test_FunctionWhichReturnsLocalVariable_ShouldReturnTheCurrentCounterValueAgain(void) { TEST_ASSERT_EQUAL(0, FindFunction_WhichIsBroken(78)); TEST_ASSERT_EQUAL_HEX(0x5a5a, FunctionWhichReturnsLocalVariable()); }
  • 23. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 23 BDD Sample Code Scenario: Subtract two numbers Given I have a calculator on When I enter ”25" into the calculator And I enter ”10" into the calculator And I press subtract Then the result should be "15" on the screen Scenario: Add two numbers Given I have a calculator on When I enter ”25" into the calculator And I enter ”45" into the calculator And I press add Then the result should be ”70" on the screen
  • 24. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 24 Mock Objects • Mock objects are replacements for the dependencies of the code under test • They are used to make it easier to test the code by responding the same way all the time • There are a number of good mock object frameworks including CMock (see http://www.throwtheswitch.org/cmock/)
  • 25. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 25 Test Fixtures • Test fixtures are the code that creates a fixed state that is used to baseline tests so that the test environment is the same each time the tests are run • Test frameworks do this by exposing methods or functions that allow for the configuration of dependent objects or for the creation of mock objects • For a data related test a test fixture might load a specific set of data into the database • Most unit testing frameworks and many test automation tools have test fixtures build into them
  • 26. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 26 Working with Simulators • A simulator is software or hardware used for testing that has the same interface as the application or system it is replacing • Simulators provide a consistent interface for testing, much like test fixtures or mock objects • Simulators are more full featured and can react to input, in more varied ways that other testing tools • The idea is that a simulator will react more like the real component in the real world • Simulators can be expensive to build and maintain but there may be no other option
  • 27. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 27 Testing services for embedded software • Many Internet of Things (IoT) devices use services • Services are Internet accessible bits of functionality that do things like: • Collect data • Calculate results • Coordinate resources • Testing these services can be easier to automate then the devices themselves • Scale can be an issue for services • Network availability or reliability can be an issue for the devices • Performance testing is often important for testing services
  • 28. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 28 What to take away from the presentation • Test automation is the key to agile testing • Do test automation planning while you do test planning • Create test suites that test your software separately from your hardware and another suite that tests the system as a whole
  • 29. © COPYRIGHT 2016 COVEROS, INC. ALL RIGHTS RESERVED. 29 Questions? • Tom Stiehm • tom.stiehm@coveros.com • @thomasstiehm

Editor's Notes

  1. http://www.throwtheswitch.org/cmock/
  2. http://www.throwtheswitch.org/cmock/
  3. bench testing