Model-based Testing (MBT) of web and mobile applications: A practical tutorialVahid Garousi
A tutorial presented in the User Conference on Advanced Automated Testing (UCAAT)
October 19-21, 2021
By:
Vahid Garousi
Bahar Software Engineering Consulting Corporation
Queen’s University Belfast, UK
www.vgarousi.com
@vgarousi
Alper Buğra Keleş, Yunus Balaman, Zeynep Özdemir Güler
Testinium A.Ş., Istanbul, Turkey
Application of TMMi to improve test approaches and processes: Experience from...Vahid Garousi
By: Vahid Garousi
Bahar Software Engineering Consulting Corporation, UK
Queen’s University Belfast, UK
www.vgarousi.com
v.garousi@qub.ac.uk
Alper Buğra Keleş
Testinium A.Ş., Istanbul, Turkey
alper.keles@testinium.com
www.testinium.com
An invited talk for:
TMMi Hungarian Local Chapter
May 26, 2021
Talk by Dr. Vahid Garousi, in the Turkey-UK Research Partnerships Event (Feb ...Vahid Garousi
Talk by Dr. Vahid Garousi, in the Turkey-UK Research Partnerships Event (Feb 18, 2021)
https://www.vgarousi.com
https://uktechweekturkey.kongrelive.com/
Can we do it better? Improving the maturity of software testing in test teamsVahid Garousi
An invited talk, given in the "tesena-fest" software testing summit, Prague, Czech Republic, Oct. 5, 2018
https://www.tesena.com/can-we-do-it-better-improving-the-maturity-of-software-testing-in-test-teams-en/
Experience and best practices from automated software testing in the finance ...Vahid Garousi
Invited talk for the NatWest Bank Group (National Westminster Bank)
October 20, 2021
By:
Vahid Garousi
Bahar Software Engineering Consulting Corporation
Queen’s University Belfast, UK
www.vgarousi.com
@vgarousi
Software Testing Certification Courses: https://www.edureka.co/software-testing-certification-courses **
This Edureka PPT on Top 50 Software Testing Interview Question and Answers will help you to prepare yourself for Software Testing Interviews. It covers questions for beginners, intermediate and experienced professionals.
Below topics are covered in this PPT:
Market Trends in Software Testing
What is Software testing
Top 50 interview questions
1. Beginners level questions
2. Intermediate level questions
3. Advance level questions
Selenium playlist: https://goo.gl/NmuzXE
Selenium Blog playlist: http://bit.ly/2B7C3QR
Software Testing Blog playlist: http://bit.ly/2UXwdJm
Follow us to never miss an update in the future.
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Model-based Testing (MBT) of web and mobile applications: A practical tutorialVahid Garousi
A tutorial presented in the User Conference on Advanced Automated Testing (UCAAT)
October 19-21, 2021
By:
Vahid Garousi
Bahar Software Engineering Consulting Corporation
Queen’s University Belfast, UK
www.vgarousi.com
@vgarousi
Alper Buğra Keleş, Yunus Balaman, Zeynep Özdemir Güler
Testinium A.Ş., Istanbul, Turkey
Application of TMMi to improve test approaches and processes: Experience from...Vahid Garousi
By: Vahid Garousi
Bahar Software Engineering Consulting Corporation, UK
Queen’s University Belfast, UK
www.vgarousi.com
v.garousi@qub.ac.uk
Alper Buğra Keleş
Testinium A.Ş., Istanbul, Turkey
alper.keles@testinium.com
www.testinium.com
An invited talk for:
TMMi Hungarian Local Chapter
May 26, 2021
Talk by Dr. Vahid Garousi, in the Turkey-UK Research Partnerships Event (Feb ...Vahid Garousi
Talk by Dr. Vahid Garousi, in the Turkey-UK Research Partnerships Event (Feb 18, 2021)
https://www.vgarousi.com
https://uktechweekturkey.kongrelive.com/
Can we do it better? Improving the maturity of software testing in test teamsVahid Garousi
An invited talk, given in the "tesena-fest" software testing summit, Prague, Czech Republic, Oct. 5, 2018
https://www.tesena.com/can-we-do-it-better-improving-the-maturity-of-software-testing-in-test-teams-en/
Experience and best practices from automated software testing in the finance ...Vahid Garousi
Invited talk for the NatWest Bank Group (National Westminster Bank)
October 20, 2021
By:
Vahid Garousi
Bahar Software Engineering Consulting Corporation
Queen’s University Belfast, UK
www.vgarousi.com
@vgarousi
Software Testing Certification Courses: https://www.edureka.co/software-testing-certification-courses **
This Edureka PPT on Top 50 Software Testing Interview Question and Answers will help you to prepare yourself for Software Testing Interviews. It covers questions for beginners, intermediate and experienced professionals.
Below topics are covered in this PPT:
Market Trends in Software Testing
What is Software testing
Top 50 interview questions
1. Beginners level questions
2. Intermediate level questions
3. Advance level questions
Selenium playlist: https://goo.gl/NmuzXE
Selenium Blog playlist: http://bit.ly/2B7C3QR
Software Testing Blog playlist: http://bit.ly/2UXwdJm
Follow us to never miss an update in the future.
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
An Open Modern Software Testing Laboratory Courseware: An Experience ReportVahid Garousi
Vahid Garousi, An Open Modern Software Testing Laboratory Courseware: An Experience Report, Proceedings of the 23rd IEEE Conference on Software Engineering Education and Training, Pittsburgh, USA, March 2010
YouTube Link: https://youtu.be/Zkeqvl8cxGc
** Test Automation Masters Program: https://www.edureka.co/masters-program/automation-testing-engineer-training**
This Edureka PPT on "Manual Testing Interview Questions and Answers" will help you to prepare yourself for Software Testing Interviews based on manual testing. It covers manual testing interview questions for beginners, intermediate and experienced professionals. Below topics are covered in this PPT:
Top 50 manual testing interview questions
1. Beginners level questions
2. Intermediate level questions
3. Advance level questions
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Castbox: https://castbox.fm/networks/505?country=in
Being a fresher, it's challenging to get a job so here I'm documenting question bank which I've been asked during my job searching ( struggle ) time, this list of questions will help freshers candidates in interview of software testing jobs.
Getting value from your test automation is fundamental for fast feedback, risk reduction, and return on investment from your testing activities. Once developing the test scenarios, teams cannot stop monitoring and ensuring that their tests continuously bring value, are not flaky, and can support the latest functionalities in your web and mobile apps. Teams often “forget” about their tests once they have been developed and integrated into the CI pipeline — regardless of the value they bring. In this session, learn how to optimize your Appium and Selenium test suites so you can get more value from them.
We provide our view at ICCC21 on how Automatization will be a key factor in the Common Criteria evolution for the whole community (developers, evaluators and certifiers.)
Looking to acquire a job as automation tester? You might be skilled however, you need to impress the interviewer first. Go through this obvious interview questions and make it possible
Doron Reuveni - The Mobile App Quality Challenge - EuroSTAR 2010TEST Huddle
EuroSTAR Software Testing Conference 2010 presentation on The Mobile App Quality Challenge by Doron Reuveni. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
An Open Modern Software Testing Laboratory Courseware: An Experience ReportVahid Garousi
Vahid Garousi, An Open Modern Software Testing Laboratory Courseware: An Experience Report, Proceedings of the 23rd IEEE Conference on Software Engineering Education and Training, Pittsburgh, USA, March 2010
YouTube Link: https://youtu.be/Zkeqvl8cxGc
** Test Automation Masters Program: https://www.edureka.co/masters-program/automation-testing-engineer-training**
This Edureka PPT on "Manual Testing Interview Questions and Answers" will help you to prepare yourself for Software Testing Interviews based on manual testing. It covers manual testing interview questions for beginners, intermediate and experienced professionals. Below topics are covered in this PPT:
Top 50 manual testing interview questions
1. Beginners level questions
2. Intermediate level questions
3. Advance level questions
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Castbox: https://castbox.fm/networks/505?country=in
Being a fresher, it's challenging to get a job so here I'm documenting question bank which I've been asked during my job searching ( struggle ) time, this list of questions will help freshers candidates in interview of software testing jobs.
Getting value from your test automation is fundamental for fast feedback, risk reduction, and return on investment from your testing activities. Once developing the test scenarios, teams cannot stop monitoring and ensuring that their tests continuously bring value, are not flaky, and can support the latest functionalities in your web and mobile apps. Teams often “forget” about their tests once they have been developed and integrated into the CI pipeline — regardless of the value they bring. In this session, learn how to optimize your Appium and Selenium test suites so you can get more value from them.
We provide our view at ICCC21 on how Automatization will be a key factor in the Common Criteria evolution for the whole community (developers, evaluators and certifiers.)
Looking to acquire a job as automation tester? You might be skilled however, you need to impress the interviewer first. Go through this obvious interview questions and make it possible
Doron Reuveni - The Mobile App Quality Challenge - EuroSTAR 2010TEST Huddle
EuroSTAR Software Testing Conference 2010 presentation on The Mobile App Quality Challenge by Doron Reuveni. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
The article observes some questions related to testing the 64-bit software. Some difficulties which a developer of resource-intensive 64-bit applications may face and the ways to overcome them are described.
Hakan Fredriksson - Experiences With MBT and Qtronic TEST Huddle
EuroSTAR Software Testing Conference 2009 presentation on Experiences With MBT and Qtronic by Hakan Fredriksson. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
Click Here---> https://bit.ly/3W4457l <---Get complete detail on CT-MBT exam guide to crack CT-MBT 2015. You can collect all information on CT-MBT tutorial, practice test, books, study material, exam questions, and syllabus. Firm your knowledge on CT-MBT 2015 and get ready to crack CT-MBT certification. Explore all information on CT-MBT exam with number of questions, passing percentage and time duration to complete test.
Elise Greveraars - Tester Needed? No Thanks, We Use MBT!TEST Huddle
EuroSTAR Software Testing Conference 2008 presentation on Tester Needed? No Thanks, We Use MBT! by Elise Greveraars. See more at conferences.eurostarsoftwaretesting.com/past-presentations/
The Importance of Performance Testing Theory and Practice - QueBIT Consulting...QueBIT Consulting
Why is good testing so hard to do? Not Enough Time. Not Enough Testers. Inconsistent or Incomplete Test Scripts. Lack of Performance Metrics. Difficult to Summarize Results
With the rise of agile development and the adoption of continuous integration, the software industry has seen an increasing interest in test automation. Many organizations invest in test automation but fail to reap the expected benefits, most likely due to a lack of test-automation maturity. In this talk, we present the results of a test automation maturity survey collecting responses of 151 practitioners coming from 101 organizations in 25 countries. We make observations regarding the state of the practice and provide a benchmark for assessing the maturity of an agile team. The benchmark resulted in a self-assessment tool for practitioners to be released under an open source license. An alfa version is presented herein. The research underpinning the survey has been conducted through the TESTOMAT project, a European project with 34 partners coming from 6 different countries.
(Presentation delivered at the Test Automation Days and the Testnet Autumn Event; October 2020)
Thomas Kauders - Agile Test Design And Automation of a Life-Critical Medical ...TEST Huddle
EuroSTAR Software Testing Conference 2010 presentation on Agile Test Design And Automation of a Life-Critical Medical Device by Thomas Kauders. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
Similar to Next level of test automation with Model-based Testing (MBT): Experience and best practices (20)
Pragmatic look at education and training of software test engineers: Further ...Vahid Garousi
Presented in: The Third International Software Testing Education Workshop (TestEd 2024), co-located with the 17th IEEE International Conference on Software Testing, Verification and Validation (ICST 2024),
May 28, 2024
Toronto, Canada
Coverage measurement in model-based testing of web applications: Tool suppor...Vahid Garousi
Presented in: The 20th Workshop on Advances in Model-Based Testing (A-MOST 2024), co-located with the 17th IEEE International Conference on Software Testing, Verification and Validation (ICST 2024),
May 27, 2024
Toronto, Canada
Education and Training of Highly-skills Software Test EngineersVahid Garousi
Education and Training of Highly-skills Software Test Engineers (insights from 20+ years of experience)
By:
Vahid Garousi, PhD, PEng
Queen’s University Belfast, UK
Bahar Software Engineering Consulting Corporation, UK
www.vgarousi.com
Event:
BCS Software Testing specialist group (SIGIST)
https://www.bcs.org/events/2020/november/webinar-testing-apprentice/
Date:
November 3, 2020
Benefitting from the Grey Literature in Software Engineering ResearchVahid Garousi
A short presentation about our book chapter in the book: "Contemporary Empirical Methods in Software Engineering"
Video: https://youtu.be/T75KyQqXhgA
Book chapter DOI and PDF: www.doi.org/10.1007/978-3-030-32489-6_14
Free PDF: https://arxiv.org/abs/1911.12038
Vahid Garousi, Austen Rainer
Queen’s University Belfast
Belfast, Northern Ireland, UK
https://www.vgarousi.com/home
https://twitter.com/vgarousi
https://twitter.com/unfinitude
Michael Felderer
University of Innsbruck, Innsbruck, Austria & Blekinge Institute of Technology, Sweden
https://twitter.com/mfelderer
Mika V. Mäntylä
University of Oulu, Oulu, Finland
https://twitter.com/mmantyla
Slides for for JSS "Happy Hour": Aligning software engineering education with...Vahid Garousi
Short video for JSS "Happy Hour"
To take place on: May 29, 2020
Paper:
Aligning software engineering education with industrial needs: A meta-analysis
Full paper details:
Vahid Garousi, Görkem Giray, Eray Tüzün, Cagatay Catal, Michael Felderer, "Aligning software engineering education with industrial needs: a meta-analysis", Journal of Systems and Software, vol. 156, pp. 65-83, October 2019
Full-text paper link: https://www.doi.org/10.1016/j.jss.2019.06.044
Narrated video: https://youtu.be/DxLQFYouDAQ
Experience in conducting 19 secondary (survey) studies in Software EngineeringVahid Garousi
Experience in conducting 19 secondary (survey) studies in Software Engineering
By: Dr. Vahid Garousi
Invited (online) talk for the University of Adelaide, Australia
May 8, 2020
Sustainability of Scientific Software: Experience from Several ProjectsVahid Garousi
Presentation by: Dr. Vahid Garousi, Dr. David Cutting
The Software Sustainability Institute’s Collaborations Workshop 2020 (CW20)
https://www.software.ac.uk/cw20
April 1, 2020
Engineering software systems for improving the operational efficiency of oil ...Vahid Garousi
Corresponding to the following paper:
Vahid Garousi, Ehsan Abbasi, Roshanak Farhoodi, Richard Bauer, Allan Shea, “Engineering software systems for improving the operational efficiency of oil pipeline networks and decreasing their carbon footprint”, A video presentation for the Second International Workshop on Software Research and Climate Change, Cape Town, South Africa, May 3, 2010, part of the ICSE 2010
Introducing automated GUI testing and observing its benefits: an industrial c...Vahid Garousi
Presented in: https://www.testomatproject.eu/nexta2018/
Vahid Garousi, Erdem Yıldırım, "Introducing automated GUI testing and observing its benefits: an industrial case study in the context of law-practice management software", Proceedings of IEEE Workshop on NEXt level of Test Automation (NEXTA), co-located with the IEEE International Conference on Software Testing, Verification and Validation (ICST), 2018
Working with Students to Co-design and Enhance Realistic (Modern) Exercises f...Vahid Garousi
Vahid Garousi, Working with Students to Co-design and Enhance Realistic (Modern) Exercises for Software Testing Labs, Software Testing Education Workshop (STEW), co-located with the IEEE International Conference on Software Testing, Verification and Validation (ICST), Denver, Colorado, USA, April 1, 2009
Choosing the Right Testing Tools and Systems Under Test (SUTs) for Practical ...Vahid Garousi
Vahid Garousi, Choosing the Right Testing Tools and Systems Under Test (SUTs) for Practical Exercises in Testing Education, 8th Workshop on Teaching Software Testing (WTST), Melbourne, Florida, January 30-Feb. 1, 2009
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Monitoring Java Application Security with JDK Tools and JFR Events
Next level of test automation with Model-based Testing (MBT): Experience and best practices
1. 1
Next level of test automation with
Model-based Testing (MBT):
Experience and best practices
Vahid Garousi
Bahar Software Engineering Consulting Corporation, UK
Queen’s University Belfast, UK
v.garousi@qub.ac.uk
www.vgarousi.com
@vgarousi
Alper Buğra Keleş, Yunus Balaman, Zeynep Özdemir Güler
Testinium A.Ş., Istanbul, Turkey
{alper.keles, yunus.balaman, zeynep.ozdemir}@testinium.com
www.testinium.com
A presentation for:
Northern Ireland Developer Conference
October 2020
2. 2
Outline
A review of our industry context
The needs for model-based testing (MBT)
Our approach (Solution): Applying MBT
Assessing the benefits of MBT
Current/next steps
Take-away messages
3. 3
A review of our industry context
Testinium A.Ş. is a provider of software testing
services and solutions in Turkey, UK and across
Europe
Founded in year 2010
Has provided automated testing to 60+ clients in several
countries so far
As of August 2020, has over 190 employees
Almost all test engineers have the ISTQB certificates
In addition to the test service offered to clients, the
company has developed and offers two flagship test
automation tools:
Testinium (testinium.com): A web-based test management
system with many features, e.g., a web interface to Selenium
and Appium
Loadium (loadium.com): A web-based load test management
system
And many other
clients….
4. 4
One of the flagship test automation tool:
Testinium (testinium.com)
5. 5
Test automation approaches and tools, in use in
the company since 2010
Takes Behaviour-driven Development
(BDD) to a higher levelgauge.org
selenium.dev
And many other test automation approaches and tools
testinium.com
• For test-case design, we use established black-box testing approaches,
such as category partitioning and boundary-value testing
• But, as also reported by other testers, making sure that all the black-box
test-case design approaches are followed systematically by all test
engineers, is not always easy
• Thus, we have seen the need for “systematic” test-case design, thus the
need for this project! (more next)
6. 6
Outline
A review of our industry context
The needs for model-based testing (MBT)
Our approach (Solution): Applying MBT
Assessing the benefits of MBT
Current/next steps
Take-away messages
7. 7
The needs (motivations) for MBT
While test automation tools are effective for automated execution of
tests, tools alone are not enough for a successful test automation
outcome
If test cases are not designed properly, a test tool will only
automatically run bad-quality (read: useless) tests!
We need “systematic” test-case design
Imagine: We can have two testers:
Tester A designs 10 test case which would detect 20 bugs (defects)
Tester B designs 100 test case which would detect 2 bugs (defects)
Which one is a more effective test engineer?
Model-based testing (MBT) is a suitable approach for this need!
Question: How many of
you think this statement
is true?
8. 8
Outline
A review of our industry context
The needs for model-based testing (MBT)
Our approach (Solution): Applying MBT
Assessing the benefits of MBT
Current/next steps
Take-away messages
9. 9
What is Model-based Testing (MBT)?
Model-based testing:
Designing test “models”: GUI flow-diagrams of a given Software Under Test (SUT)
Nodes, or vertices. Edges or transitions
A small example:
Benefits:
Automating both the test-case design and test-case execution
Deriving almost all possible test paths is a very efficient way (would be very effort intensive to do in
manual testing, or even when developing automated test scripts manually)
Click
Note: MBT is popular…
11. 11
Choosing the “right” MBT tool
There are literally hundreds of MBT tools
Which MBT tool is the “right” choice for a given
context?
Type of SUT
Testing needs
Other factors
A non-trivial question
After a systematic comparison of several tools, we
selected GraphWalker (graphwalker.github.io)
It’s free and open-source and has a wide user base
12. 12
Time for a quick demo!
Several online videos from our MBT projects are on
YouTube: bit.ly/MBTVideosTestinium
The entire test suite with all the code is open-source:
github.com/vgarousi/MBTofTestinium
13. 13
An overview of the MBT approach
The test architecture:
Web browser
Front-end of the
web application
System Under
Test (SUT)
Call Return
Exercise
(call)
Return
Test
outputs
Return
Anlayzes
GraphWalker
Test models
Designs
Test enigneer
Run using...
Java Selenium
methods
Implementaion of
nodes/edges behavior
in Java using the
Selenium framework
Call
Return
Back-end of the
web application
SUT
Develops
Requirements
(Use-case
diagram)
Uses
14. 14
Development of nodes/edges’ behavior in Java using the
Selenium framework
public void e_valid_login() {
WebElement userNameElement =
methodsPage.findElement(By.id("username"));
userNameElement.clear();
userNameElement.sendKeys(email);
WebElement passwordElement =
methodsPage.findElement(By.id("password"));
passwordElement.clear();
passwordElement.sendKeys(password);
methodsPage.findElement(By.cssSelector(
"input[class$="login-page__submit-btn"][value="Sign In"]")).click();
}
15. 15
Report from one MBT project
SUT: Our own test automation tool: Testinium
(testinium.com)
Testing a testing tool!
Size measures (metrics) from the MBT project:
# of test models = 18
Total # of nodes=170
Total # of edges=260
Size of the Selenium Java code for nodes/edges’ behavior = 4,699
LOC
Each test run takes about: 6 hours, going through ~28,000 test
steps! (nodes and edges, stopping condition=%100 edge coverage)
Reminder: The entire test suite with all the code is open-
source: github.com/vgarousi/MBTofTestinium
16. 16
Outline
A review of our industry context
The needs for model-based testing (MBT)
Our approach (Solution): Applying MBT
Assessing the benefits of MBT
Current/next steps
Take-away messages
17. 17
Assessing the benefits of MBT
Tangible benefits of MBT
Improved test-case design practices, due to MBT
In the old approach (manual test-case design), many deficiencies were identified, e.g.,
duplications among different test suites (could lead to test integrity problems, etc.), and many
missing test paths and test cases
Detecting real fault during regression testing
Several defects were detected, via “Endurance testing” using MBT (soak testing, longevity
testing)
Measuring code coverage:
Both in client side (JS) and server side (Java)
Intangible benefits of MBT
According to the feedbacks we have received: MBT has made the
work of test engineers “more interesting” and more organized
Many in the company have told us that, thanks to MBT models, they
can now see the “big picture” much more easily with having the test
models in front of them, and the model being directly executable.
18. 18
Current / next work directions
Our MBT project has showed us several important ongoing work
directions, e.g.:
Test visualization (showing the number of times each edge and node has been
covered)
We are in the process of improving the testing tool to incorporate fault tolerance
(when an assertion fails)
Assessing effectiveness of MBT in detection of injected faults (by mutation
testing).
Developing a code coverage tool for clients side (JS ) and server side
19. 19
Take-away messages
In retrospective, using MBT, we have observed various
benefits, compared to the “before state” of test automation
practices (in which test-cases were designed in ad-hoc
manner)
MBT has helped us:
By providing clarity and road-map (what to do next) for test engineers
Resulting in higher-quality test scripts in term of modularity with no test-
code duplication
Taking test automation to a higher level of maturity
Pre-MBT MBT
State of test
automation in the
company:
20. 20
This work is a part of a large international EU project
The TESTOMAT project: The Next Level of Test Automation
www.testomatproject.eu
Among different topics of the project:
Model-based testing
Test prioritization and selection, etc.
Project is funded by the European ITEA3 (Information Technology for
European Advancement) program and the government of the six
countries involved in the project
“ITEA is a transnational and industry-driven R&D&I programme in the
domain of software innovation [in Europe]”
An R&D collaboration among 34 entities (software companies
and universities) across Europe
21. 21
END of the talk
Questions /
Answers
Vahid Garousi
Queen’s University Belfast, UK
Bahar Software Engineering Consulting Corporation
v.garousi@qub.ac.uk
www.vgarousi.com
@vgarousi
Alper Buğra Keleş, Yunus Balaman, Zeynep Özdemir Güler
Testinium A.Ş., Istanbul, Turkey
{alper.keles, yunus.balaman, zeynep.ozdemir}@testinium.com
www.testinium.com