This document summarizes a presentation on metamorphic testing for web system security given by Nazanin Bayati on September 13, 2023. Metamorphic testing uses relations between the outputs of multiple test executions to test systems when specifying expected outputs is difficult. It was applied to web systems by generating follow-up inputs based on transformations of valid interactions and checking that output relations held. The approach detected over 60% of vulnerabilities in tested systems and addressed more vulnerability types than static and dynamic analysis tools. It provides an effective and automated way to test for security issues in web systems.
Cyber attacks are a real and growing threat to businesses and an increasing number of attacks take place at application layer. The best defence against is to develop applications where security is incorporated as part of the software development lifecycle.
The OWASP Top 10 Proactive Controls project is designed to integrate security in the software development lifecycle. In this special presentation for PHPNW, based on v2.0 released this year, you will learn how to incorporate security into your software projects.
Recommended to all developers who want to learn the security techniques that can help them build more secure applications.
Link to Youtube video: https://youtu.be/OJMqMWnxlT8
You can contact me at abhimanyu.bhogwan@gmail.com
My linkdin id : https://www.linkedin.com/in/abhimanyu-bhogwan-cissp-ctprp-98978437/
Threat Modeling(system+ enterprise)
What is Threat Modeling?
Why do we need Threat Modeling?
6 Most Common Threat Modeling Misconceptions
Threat Modelling Overview
6 important components of a DevSecOps approach
DevSecOps Security Best Practices
Threat Modeling Approaches
Threat Modeling Methodologies for IT Purposes
STRIDE
Threat Modelling Detailed Flow
System Characterization
Create an Architecture Overview
Decomposing your Application
Decomposing DFD’s and Threat-Element Relationship
Identify possible attack scenarios mapped to S.T.R.I.D.E. model
Identifying Security Controls
Identify possible threats
Report to Developers and Security team
DREAD Scoring
My Opinion on implementing Threat Modeling at enterprise level
Mining software vulns in SCCM / NIST's NVDLoren Gordon
Patch management for 3rd-party software can be a significant challenge. The raw data for effective vulnerability management is available in MS’ SCCM (software inventory) and NIST’s NVD (vulnerability database). However extracting the relevant information from complex, sometimes undocumented data structures poses significant challenges.
The stage is set with a brief overview of SCCM / NVD data structures as well as a look at a (non-typical but interesting!) production environment. Then we’ll take a quick dive into data wrangling / Machine Learning fundamentals applied to this problem: feature extraction, choice of approach, algorithm choice and turning.
Once the technical challenges are resolved, the path to “Data Nirvana” can still be strewn with significant non-technical hurdles to overcome as well. We will discuss some practical “been there, done that” examples.
Cyber attacks are a real and growing threat to businesses and an increasing number of attacks take place at application layer. The best defence against is to develop applications where security is incorporated as part of the software development lifecycle.
The OWASP Top 10 Proactive Controls project is designed to integrate security in the software development lifecycle. In this special presentation for PHPNW, based on v2.0 released this year, you will learn how to incorporate security into your software projects.
Recommended to all developers who want to learn the security techniques that can help them build more secure applications.
Link to Youtube video: https://youtu.be/OJMqMWnxlT8
You can contact me at abhimanyu.bhogwan@gmail.com
My linkdin id : https://www.linkedin.com/in/abhimanyu-bhogwan-cissp-ctprp-98978437/
Threat Modeling(system+ enterprise)
What is Threat Modeling?
Why do we need Threat Modeling?
6 Most Common Threat Modeling Misconceptions
Threat Modelling Overview
6 important components of a DevSecOps approach
DevSecOps Security Best Practices
Threat Modeling Approaches
Threat Modeling Methodologies for IT Purposes
STRIDE
Threat Modelling Detailed Flow
System Characterization
Create an Architecture Overview
Decomposing your Application
Decomposing DFD’s and Threat-Element Relationship
Identify possible attack scenarios mapped to S.T.R.I.D.E. model
Identifying Security Controls
Identify possible threats
Report to Developers and Security team
DREAD Scoring
My Opinion on implementing Threat Modeling at enterprise level
Mining software vulns in SCCM / NIST's NVDLoren Gordon
Patch management for 3rd-party software can be a significant challenge. The raw data for effective vulnerability management is available in MS’ SCCM (software inventory) and NIST’s NVD (vulnerability database). However extracting the relevant information from complex, sometimes undocumented data structures poses significant challenges.
The stage is set with a brief overview of SCCM / NVD data structures as well as a look at a (non-typical but interesting!) production environment. Then we’ll take a quick dive into data wrangling / Machine Learning fundamentals applied to this problem: feature extraction, choice of approach, algorithm choice and turning.
Once the technical challenges are resolved, the path to “Data Nirvana” can still be strewn with significant non-technical hurdles to overcome as well. We will discuss some practical “been there, done that” examples.
Advanced SQL injection to operating system full control (whitepaper)Bernardo Damele A. G.
Over ten years have passed since a famous hacker coined the term "SQL injection" and it is still considered one of the major web application threats, affecting over 70% of web application on the Net. A lot has been said on this specific vulnerability, but not all of the aspects and implications have been uncovered, yet.
It's time to explore new ways to get complete control over the database management system's underlying operating system through a SQL injection vulnerability in those over-looked and theoretically not exploitable scenarios: From the command execution on MySQL and PostgreSQL to a stored procedure's buffer overflow exploitation on Microsoft SQL Server. These and much more will be unveiled and demonstrated with my own tool's new version that I will release at the Conference (http://www.blackhat.com/html/bh-europe-09/bh-eu-09-speakers.html#Damele).
Artificial Intelligence Large Language Models (LLM) and Machine Learning (ML) Application Security Threats and Defenses. OWASP Top Tens for LLM and ML along with software development attack preventative best practices.
The slide consists of:
An explanation for SQL injections.
First order and second order SQL injections.
Methods: Normal and Blind SQL injections with examples.
Examples: Injection using true/false, drop table and update table commands.
Prevention using dynamic embedded SQL queries.
Conclusion and References.
AI & ML in Cyber Security - Why Algorithms are DangerousRaffael Marty
Link to the video of the presentation: https://www.youtube.com/watch?v=WG1k-Xh1TqM
Every single security company is talking in some way or another about how they are applying machine learning. Companies go out of their way to make sure they mention machine learning and not statistics when they explain how they work. Recently, that's not enough anymore either. As a security company you have to claim artificial intelligence to be even part of the conversation.
Guess what. It's all baloney. We have entered a state in cyber security that is, in fact, dangerous. We are blindly relying on algorithms to do the right thing. We are letting deep learning algorithms detect anomalies in our data without having a clue what that algorithm just did. In academia, they call this the lack of explainability and verifiability. But rather than building systems with actual security knowledge, companies are using algorithms that nobody understands and in turn discover wrong insights.
In this talk, I will show the limitations of machine learning, outline the issues of explainability, and show where deep learning should never be applied. I will show examples of how the blind application of algorithms (including deep learning) actually leads to wrong results. Algorithms are dangerous. We need to revert back to experts and invest in systems that learn from, and absorb the knowledge, of experts.
Cyber Security Layers - Defense in Depth
7P's, 2D's & 1 N
People
Process
Perimeter
Physical
Points (End)
Network
Platform
Programs (Apps)
Database
Data
Splunk in 60 Minutes | Splunk Tutorial For Beginners | Splunk Training | Splu...Edureka!
***** Splunk Training: https://www.edureka.co/splunk *****
This Edureka Live session on Splunk will help you understand the fundamentals for Splunk with a demo on Log Collection & Analysis. Below are the topics that will be discussed in this session:
1. Why Splunk? – Problems With Log Data
2. What Is Splunk? – Ultimate Soln. For Log Processing
3. How Does It Work? – Splunk Components
4. Hands-On:- Log Collection & Analysis
XPDDS17: Introduction to Intel SGX and SGX Virtualization - Kai Huang, IntelThe Linux Foundation
In era of cloud computing, security is becoming more and more critical for customers. In existing HW/SW architecture hypervisor does not protect tenants against the cloud provider and thus the supplied operating system and hardware. Intel Software Guard Extension (SGX) provides a mechanism that addresses this scenario. It aims at protecting user-level software from attacks from other processes, the operating system, and even physical attackers. Intel SGX makes such protection possible through the use of enclave, which is a protected area in userspace application where the code/data cannot be accessed directly by any software from outside. This presentation intends to give you an introduction of Intel SGX technology, including what it is, how it works, and the existing SW stack to enable SGX for customers, followed by introduction of our work to support SGX virtualization in Xen hypervisor, including the high-level design, current status and future plan.
This presentation covers the Cross site scripting attacks and defences in web applications, this talk was delivered as part of OWASP Hyderabad Chapter meet. Comments and suggestions are welcome.
Vulnerabilities in modern web applicationsNiyas Nazar
Microsoft powerpoint presentation for BTech academic seminar.This seminar discuses about penetration testing, penetration testing tools, web application vulnerabilities, impact of vulnerabilities and security recommendations.
Advanced SQL injection to operating system full control (whitepaper)Bernardo Damele A. G.
Over ten years have passed since a famous hacker coined the term "SQL injection" and it is still considered one of the major web application threats, affecting over 70% of web application on the Net. A lot has been said on this specific vulnerability, but not all of the aspects and implications have been uncovered, yet.
It's time to explore new ways to get complete control over the database management system's underlying operating system through a SQL injection vulnerability in those over-looked and theoretically not exploitable scenarios: From the command execution on MySQL and PostgreSQL to a stored procedure's buffer overflow exploitation on Microsoft SQL Server. These and much more will be unveiled and demonstrated with my own tool's new version that I will release at the Conference (http://www.blackhat.com/html/bh-europe-09/bh-eu-09-speakers.html#Damele).
Artificial Intelligence Large Language Models (LLM) and Machine Learning (ML) Application Security Threats and Defenses. OWASP Top Tens for LLM and ML along with software development attack preventative best practices.
The slide consists of:
An explanation for SQL injections.
First order and second order SQL injections.
Methods: Normal and Blind SQL injections with examples.
Examples: Injection using true/false, drop table and update table commands.
Prevention using dynamic embedded SQL queries.
Conclusion and References.
AI & ML in Cyber Security - Why Algorithms are DangerousRaffael Marty
Link to the video of the presentation: https://www.youtube.com/watch?v=WG1k-Xh1TqM
Every single security company is talking in some way or another about how they are applying machine learning. Companies go out of their way to make sure they mention machine learning and not statistics when they explain how they work. Recently, that's not enough anymore either. As a security company you have to claim artificial intelligence to be even part of the conversation.
Guess what. It's all baloney. We have entered a state in cyber security that is, in fact, dangerous. We are blindly relying on algorithms to do the right thing. We are letting deep learning algorithms detect anomalies in our data without having a clue what that algorithm just did. In academia, they call this the lack of explainability and verifiability. But rather than building systems with actual security knowledge, companies are using algorithms that nobody understands and in turn discover wrong insights.
In this talk, I will show the limitations of machine learning, outline the issues of explainability, and show where deep learning should never be applied. I will show examples of how the blind application of algorithms (including deep learning) actually leads to wrong results. Algorithms are dangerous. We need to revert back to experts and invest in systems that learn from, and absorb the knowledge, of experts.
Cyber Security Layers - Defense in Depth
7P's, 2D's & 1 N
People
Process
Perimeter
Physical
Points (End)
Network
Platform
Programs (Apps)
Database
Data
Splunk in 60 Minutes | Splunk Tutorial For Beginners | Splunk Training | Splu...Edureka!
***** Splunk Training: https://www.edureka.co/splunk *****
This Edureka Live session on Splunk will help you understand the fundamentals for Splunk with a demo on Log Collection & Analysis. Below are the topics that will be discussed in this session:
1. Why Splunk? – Problems With Log Data
2. What Is Splunk? – Ultimate Soln. For Log Processing
3. How Does It Work? – Splunk Components
4. Hands-On:- Log Collection & Analysis
XPDDS17: Introduction to Intel SGX and SGX Virtualization - Kai Huang, IntelThe Linux Foundation
In era of cloud computing, security is becoming more and more critical for customers. In existing HW/SW architecture hypervisor does not protect tenants against the cloud provider and thus the supplied operating system and hardware. Intel Software Guard Extension (SGX) provides a mechanism that addresses this scenario. It aims at protecting user-level software from attacks from other processes, the operating system, and even physical attackers. Intel SGX makes such protection possible through the use of enclave, which is a protected area in userspace application where the code/data cannot be accessed directly by any software from outside. This presentation intends to give you an introduction of Intel SGX technology, including what it is, how it works, and the existing SW stack to enable SGX for customers, followed by introduction of our work to support SGX virtualization in Xen hypervisor, including the high-level design, current status and future plan.
This presentation covers the Cross site scripting attacks and defences in web applications, this talk was delivered as part of OWASP Hyderabad Chapter meet. Comments and suggestions are welcome.
Vulnerabilities in modern web applicationsNiyas Nazar
Microsoft powerpoint presentation for BTech academic seminar.This seminar discuses about penetration testing, penetration testing tools, web application vulnerabilities, impact of vulnerabilities and security recommendations.
SecurityGen's telecom security monitoring services are a game-changer for the industry. As cyber threats continue to grow in complexity and sophistication, having a dedicated partner like SecurityGen can make all the difference. Their state-of-the-art monitoring systems employ advanced algorithms and AI-driven analytics to identify suspicious activities and potential vulnerabilities in telecom networks. This proactive approach allows telecom providers to stay one step ahead of cybercriminals, providing a robust defense against data breaches and service disruptions.
Vulnerability assessment & Penetration testing Basics Mohammed Adam
In these days of widespread Internet usage, security is of prime importance. The almost universal use of mobile and Web applications makes systems vulnerable to cyber attacks. Vulnerability assessment can help identify the loopholes in a system while penetration testing is a proof-of-concept approach to actually explore and exploit a vulnerability.
Analyze and Detect Packet Loss for Data Transmission in WSNIJERA Editor
An emerging technology is Wireless Sensor Network where sensors are deployed at extreme geographical
locations where human intervention is not possible. The data transferred through the sensor nodes are majorly
used in crucial decision making process. Since WSN is a wireless infrastructure it tempts the attackers to
tamper/misuse the data. Privacy-preserving routing is important for some ad hoc networks that require stronger
privacy protection. Hence a routing protocol to achieve total unobservability by anonymous key establishment
using secret session keys and group signature is used. The unobservable routing protocol is divided into two
main phases. First phases define an anonymous key establishment process to construct secret session keys.
Second phase consist of unobservable route discovery process to find appropriate as well as secure route to the
destination. A node establishes a key with its direct neighbour and uses the same key to encrypt the packet
before transferring.
A network security detection and prevention
scheme using a combination of network taps
and aggregation devices can improve visibility
and redundancy, reduce system complexity
and diminish initial and continuing costs for
implementation.
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Advanced Flow Concepts Every Developer Should KnowPeter Caitens
Tim Combridge from Sensible Giraffe and Salesforce Ben presents some important tips that all developers should know when dealing with Flows in Salesforce.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
Worried about document security while sharing them in Salesforce? Fret no more! Here are the top-notch security standards XfilesPro upholds to ensure strong security for your Salesforce documents while sharing with internal or external people.
To learn more, read the blog: https://www.xfilespro.com/how-does-xfilespro-make-document-sharing-secure-and-seamless-in-salesforce/
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
Metamorphic Testing for Web System Security
1. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
1
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
Journal First – IEEE Transaction on Software Engineering
Presented by: Nazanin Bayati
13 September 2023
University of Ottawa University of Luxembourg
Nazanin Bayati
University of Ottawa
Fabrizio Pastore
University of Luxembourg
Lionel Briand
University of Ottawa
University of Luxembourg
Arda Goknil
SINTEF Digital, Norway
Metamorphic Testing for Web System Security
3. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
3
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
Metamorphic Testing alleviates the Oracle Problem
• Metamorphic Testing (MT) is based on the idea that
• it may be simpler to reason about relations between outputs of multiple test executions,
called Metamorphic Relations (MRs), than to specify the output of the system for a given
input
• In MT, system properties are captured as MRs that
• specify how to automatically transform an initial set of test inputs (source inputs) into
follow-up test inputs
• specify the relation between the outputs obtained from source and follow-up inputs
• A failure is observed when such relations are violated.
4. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
4
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
Metamorphic Security Testing
• Source input: a sequence of valid interactions with the system
{login(Admin), RequestURL(settings_page)}
• Follow-up input: generated by altering valid interactions as an attacker would do
{login(User1), RequestURL(settings_page)}
• Relations: capture properties that hold when the system is not vulnerable
if the user in the follow-up input cannot access the URL from her GUI then the output of the
source and follow-up inputs should be different
5. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
5
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi: Metamorphic Security Testing for Web Interfaces
Web System
Execute the Data
Collection Framework
Catalog of 76
Metamorphic Relations
Select or Specify the
Metamorphic Relations
Execute the
Metamorphic Testing
Framework
Test results
Translate Metamorphic
Relations to Java
List of
Metamorphic Relations
Executable
Metamorphic
Relations in Java
Source Inputs
1 2
3 4
Submit
form
logout
Log in
logout
Log in
6. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
6
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – MR Example
• Security issue: Bypass Authorization Schema
7. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
7
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – MR Example
• Security issue: Bypass Authorization Schema
8. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
8
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – MR Example
• Security issue: Bypass Authorization Schema
9. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
9
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – MR Example
• Security issue: Bypass Authorization Schema
10. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
10
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – MR Example
• Security issue: Bypass Authorization Schema
11. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
11
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – MR Example
• Security issue: Bypass Authorization Schema
12. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
12
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – MR Example
• Security issue: Bypass Authorization Schema
Our metamorphic testing algorithm executes
each MR multiple times, to ensure that every
possible combination of source and follow-up
inputs is exercised
13. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
13
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – Research Questions
• RQ1. What testing activities can be automated thanks to oracle automation provided by MST-wi?
• RQ2. What vulnerability types can MST-wi detect?
• RQ3. What testability guidelines can we define to enable effective test automation with MST-wi?
• RQ4. How does MST-wi compare to state-of-the-art SAST and DAST tools?
• RQ5. Can we identify patterns for writing MST-wi relations?
• RQ6. Is MST-wi effective?
• RQ7. Is MST-wi efficient?
14. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
14
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – What vulnerability types can MST-wi detect?
• We investigated the feasibility of implementing MRs that discover the vulnerability types described in the
MITRE Common Weakness Enumeration (CWE) database
• Considered three subsets:
• CWE view for common security architectural tactics
• CWE Top 25 most dangerous software errors
• OWASP Top 10 Web security risks
• To implement an MR, for each weakness, we first inspect its description, its demonstrative examples, the
description of concrete vulnerabilities (CVE) and common attack patterns (CAPEC) associated with the
weakness.
• This process led to a catalog of 76 MRs.
15. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
15
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – What vulnerability types can MST-wi detect?
Security Design Principle Vulnerability types Addressed by MST-wi Rank
Audit 6 1(16%) 10th
Authenticate Actors 28 12 (43%) 4th
Authorize Actors 60 34 (57%) 3rd
Cross Cutting 9 3 (33%) 6th
Encrypt Data 38 8 (21%) 8th
Identify Actors 12 3 (25%) 7th
Limit Access 8 3 (38%) 5th
Limit Exposure 6 0 (0%) 11th
Lock Computer 1 0 (0%) 11th
Manage User Session 6 4 (67%) 2nd
Validate Inputs 39 31 (79%) 1st
Verify Message Integrity 19 2 (20%) 9th
Total 223 101 (45%)
Summary of the CWE architectural security design principles and weaknesses
addressed by MST-wi.
16. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
16
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – How does MST-wi compare to state-of-the-art SAST and DAST tools?
• We compared the vulnerability types detected by MST-wi, with the vulnerability types detected by state-
of-the-art SAST and DAST tool reported in a recent empirical study
17. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
17
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – How does MST-wi compare to state-of-the-art SAST and DAST tools?
Security Design
Principle
Weaknesses Addresses by Weaknesses Addressed by MST but not
by
Weaknesses bot addresses by MST but
addresses by
MST Zap DA2 Sonar SA2 Zap DA2 Sonar SA2 Zap DA2 Sonar SA2
Audit 1 0 0 0 3 1 1 1 0 0 0 0 2
Authenticate Actors 12 0 2 1 9 12 11 11 7 0 1 0 4
Authorize Actors 34 2 0 1 13 32 34 34 25 0 0 1 4
Cross Cutting 3 0 0 2 0 3 3 2 3 0 0 1 0
Encrypt Data 8 2 5 8 10 8 8 7 4 2 5 7 6
Identify Actors 3 1 1 1 7 3 3 3 1 1 1 1 5
Limit Access 3 0 1 1 5 3 3 2 0 0 1 0 2
Limit Exposure 0 1 0 0 1 0 0 0 0 1 0 0 1
Lock Computer 0 0 0 0 0 0 0 0 0 0 0 0 0
Manage User Session 4 0 0 0 2 4 4 4 2 0 0 0 0
Validate Inputs 31 10 7 2 14 24 25 30 19 3 1 1 2
Verify Message Integrity 2 1 0 0 3 2 2 2 1 1 0 0 2
Total 101 17 16 16 67 92 94 96 62 8 9 11 28
84
The set of weaknesses targeted by MST-wi
is larger than what can be targeted by applying
all four competing approaches together.
18. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
18
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – Is MST-wi effective?
Applied MST-wi to test well-known Web systems:
• Jenkins v 2.121
• Joomla v. 3.8.7.
Assessed MST-wi capability to detect known vulnerabilities:
• 11 for Jenkins, 3 for Joomla.
• One of them discovered by MST-wi (CVE-2018-17857)
Considered two setups:
• Derive source inputs with crawler only
• Consider additional manually implemented functional test cases
Metrics:
• Sensitivity: proportion of vulnerabilities identified
• Specificity: proportion of inputs not leading to false alarms
19. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
19
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – Is MST-wi effective?
• The high specificity indicates that only a negligible fraction of follow-up inputs leads to false alarms
• Since sensitivity reflects the fault detection rate (i.e., the proportion of vulnerabilities discovered),
we conclude that our approach is highly effective
• We can discover more than 60% of vulnerabilities in a completely automated manner, using only the crawler
• And up to 85% using both crawler and manual inputs
20. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
20
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
https://github.com/MetamorphicSecurityTesting/MST
21. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
21
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
Metamorphic Testing for Web
System Security
Presented by: Nazanin Bayati
13 September 2023
N. Bayati Chaleshtari, F. Pastore, A. Goknil, and L. Briand, "Metamorphic Testing for Web System Security",
IEEE Transactions on Software Engineering, 2023, https://ieeexplore.ieee.org/document/10089522
n.bayati@uottawa.ca
University of Ottawa University of Luxembourg
22. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
23
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – How does MST-wi compare to state-of-the-art SAST and DAST tools?
Security Design
Principle
Weaknesses Addresses by Weaknesses Addressed by MST but not
by
Weaknesses bot addresses by MST but
addresses by
MST Zap DA2 Sonar SA2 Zap DA2 Sonar SA2 Zap DA2 Sonar SA2
Audit 1 0 0 0 3 1 1 1 0 0 0 0 2
Authenticate Actors 12 0 2 1 9 12 11 11 7 0 1 0 4
Authorize Actors 34 2 0 1 13 32 34 34 25 0 0 1 4
Cross Cutting 3 0 0 2 0 3 3 2 3 0 0 1 0
Encrypt Data 8 2 5 8 10 8 8 7 4 2 5 7 6
Identify Actors 3 1 1 1 7 3 3 3 1 1 1 1 5
Limit Access 3 0 1 1 5 3 3 2 0 0 1 0 2
Limit Exposure 0 1 0 0 1 0 0 0 0 1 0 0 1
Lock Computer 0 0 0 0 0 0 0 0 0 0 0 0 0
Manage User Session 4 0 0 0 2 4 4 4 2 0 0 0 0
Validate Inputs 31 10 7 2 14 24 25 30 19 3 1 1 2
Verify Message Integrity 2 1 0 0 3 2 2 2 1 1 0 0 2
Total 101 17 16 16 67 92 94 96 62 8 9 11 28
56
MST can detect 56 weaknesses that any other approach cannot address.
23. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
24
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – How does MST-wi compare to state-of-the-art SAST and DAST tools?
Security Design
Principle
Weaknesses Addresses by Weaknesses Addressed by MST but not
by
Weaknesses not addresses by MST but
addresses by
MST Zap DA2 Sonar SA2 Zap DA2 Sonar SA2 Zap DA2 Sonar SA2
Audit 1 0 0 0 3 1 1 1 0 0 0 0 2
Authenticate Actors 12 0 2 1 9 12 11 11 7 0 1 0 4
Authorize Actors 34 2 0 1 13 32 34 34 25 0 0 1 4
Cross Cutting 3 0 0 2 0 3 3 2 3 0 0 1 0
Encrypt Data 8 2 5 8 10 8 8 7 4 2 5 7 6
Identify Actors 3 1 1 1 7 3 3 3 1 1 1 1 5
Limit Access 3 0 1 1 5 3 3 2 0 0 1 0 2
Limit Exposure 0 1 0 0 1 0 0 0 0 1 0 0 1
Lock Computer 0 0 0 0 0 0 0 0 0 0 0 0 0
Manage User Session 4 0 0 0 2 4 4 4 2 0 0 0 0
Validate Inputs 31 10 7 2 14 24 25 30 19 3 1 1 2
Verify Message Integrity 2 1 0 0 3 2 2 2 1 1 0 0 2
Total 101 17 16 16 67 92 94 96 62 8 9 11 28
39
Combined together, DAST/SAST approaches can address only 39 weaknesses that
MST-wi does not address.
24. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
25
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – How does MST-wi compare to state-of-the-art SAST and DAST tools?
Security Design
Principle
Weaknesses Addresses by Weaknesses Addressed by MST but not
by
Weaknesses not addresses by MST but
addresses by
MST Zap DA2 Sonar SA2 Zap DA2 Sonar SA2 Zap DA2 Sonar SA2
Audit 1 0 0 0 3 1 1 1 0 0 0 0 2
Authenticate Actors 12 0 2 1 9 12 11 11 7 0 1 0 4
Authorize Actors 34 2 0 1 13 32 34 34 25 0 0 1 4
Cross Cutting 3 0 0 2 0 3 3 2 3 0 0 1 0
Encrypt Data 8 2 5 8 10 8 8 7 4 2 5 7 6
Identify Actors 3 1 1 1 7 3 3 3 1 1 1 1 5
Limit Access 3 0 1 1 5 3 3 2 0 0 1 0 2
Limit Exposure 0 1 0 0 1 0 0 0 0 1 0 0 1
Lock Computer 0 0 0 0 0 0 0 0 0 0 0 0 0
Manage User Session 4 0 0 0 2 4 4 4 2 0 0 0 0
Validate Inputs 31 10 7 2 14 24 25 30 19 3 1 1 2
Verify Message Integrity 2 1 0 0 3 2 2 2 1 1 0 0 2
Total 101 17 16 16 67 92 94 96 62 8 9 11 28
39
Combined together, DAST/SAST approaches can address only 39 weaknesses that
MST-wi does not address.
• The weaknesses that MST-wi cannot address are mostly those
(i) that can only be discovered using program analysis,
(ii) that are not related to user-system interactions, or
(iii) that concern non-Web-based systems.
25. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
26
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – How does MST-wi compare to state-of-the-art SAST and DAST tools?
Security Design
Principle
Weaknesses Addresses by Weaknesses Addressed by MST but not
by
Weaknesses bot addresses by MST but
addresses by
MST Zap DA2 Sonar SA2 Zap DA2 Sonar SA2 Zap DA2 Sonar SA2
Audit 1 0 0 0 3 1 1 1 0 0 0 0 2
Authenticate Actors 12 0 2 1 9 12 11 11 7 0 1 0 4
Authorize Actors 34 2 0 1 13 32 34 34 25 0 0 1 4
Cross Cutting 3 0 0 2 0 3 3 2 3 0 0 1 0
Encrypt Data 8 2 5 8 10 8 8 7 4 2 5 7 6
Identify Actors 3 1 1 1 7 3 3 3 1 1 1 1 5
Limit Access 3 0 1 1 5 3 3 2 0 0 1 0 2
Limit Exposure 0 1 0 0 1 0 0 0 0 1 0 0 1
Lock Computer 0 0 0 0 0 0 0 0 0 0 0 0 0
Manage User Session 4 0 0 0 2 4 4 4 2 0 0 0 0
Validate Inputs 31 10 7 2 14 24 25 30 19 3 1 1 2
Verify Message Integrity 2 1 0 0 3 2 2 2 1 1 0 0 2
Total 101 17 16 16 67 92 94 96 62 8 9 11 28
Combining MST-wi with SA2 seems to be a particularly effective
combination as it enables detecting 129 weaknesses (i.e., 101 + 28),
which is 92% of the 140 weaknesses that can be detected by any approach.
26. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
29
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi: Metamorphic Security Testing for Web Interfaces
Web System
Execute the Data
Collection
Framework
List of Predefined
Metamorphic Relations
Select and Specify
the MRs
Execute the
Metamorphic Testing
Framework
Test results
Transform MRs
to Java
List of MRs
Executable
MRs
S(x,y)
Source Inputs
27. University of Ottawa | University of Luxembourg
University of Ottawa | University of Luxembourg
30
Nazanin Bayati - Metamorphic Testing for Web System Security
13 September 2023
MST-wi – Is MST-wi effective?
• The high specificity indicates that only a negligible fraction of follow-up inputs leads to false alarms
• Since sensitivity reflects the fault detection rate (i.e., the proportion of vulnerabilities discovered),
we conclude that our approach is highly effective
• We can discover more than 60% of vulnerabilities in a completely automated manner, using only the crawler
• And up to 85% using both crawler and manual inputs