The document discusses mobile application testing and provides examples of issues that can arise. It describes current problems like management not allowing enough testing time or prioritizing speed over quality. This can lead to bugs being missed and poor user experiences. The document also introduces taxonomies to help categorize different types of bugs, like those related to timing, to aid in more effective testing. Overall, it advocates for improved mobile app testing practices to avoid common pitfalls and ensure high quality user experiences.
Use Combinatorial Testing for Mobile Device FragmentationJosiah Renaudin
A common problem in mobile systems testing is the number of hardware, operational, and software configurations that need to be tested. For example, the so-called Android fragmentation problem might lead a test team to test hundreds of device and software configurations, yielding thousands or even tens-of-thousands of tests. A branch of mathematics, called combinatorics, and associated tools exist that allow teams to minimize the number of test cases required, while assuring high error finding percentages. Jon Hagar defines the fragmentation problem and then examines test patterns supported by tools that can help improve testing success. Jon outlines how combinatorial test patterns can be applied to other testing situations. To solve real-world fragmentation problems, he identifies specific tools, which you can take back to your project for quick use. Reference work and data are provided to help your team justify adding combinatorial testing to your mobile test activities.
Inconvenient Truth(s) - On Application Security (from 2007)Dinis Cruz
This document discusses inconvenient truths about software security. It notes that there are no standardized security metrics, making it difficult for customers to assess security. It also draws parallels between global warming and the growing impact of insecure software. The document argues that secure software does not currently make business sense for vendors due to a lack of incentives. It warns that society's heavy dependence on software leaves it vulnerable if attacker business models evolve to more effectively monetize exploiting insecure systems at scale. Overall, the document presents several inconvenient realities about the current state of software security.
Testing Is How You Avoid Looking StupidSteve Branam
Presented at With The Best IOT online conference, Oct 14 2017: As IOT products become more pervasive, they have an increasing ability to adversely affect the lives of their users and those around them. Testing is the due diligence that closes the engineering loop to verify proper behavior. This presents an introductory overview to testing for IOT products, covering the IOT triad: embedded IOT devices, backend servers, and frontend apps. I talk about the consequences of inadequate testing for companies and individual contributors, and levels and types of testing.
The security of seven popular fitness trackers and the Apple Watch was tested. Some trackers had issues with Bluetooth visibility, authentication, and data tampering. Pebble Time, Microsoft Band 2, and Basis Peak were among the most secure, while Striiv Fusion, Xiaomi MiBand, and Runtastic Moment Elite had the most security risks due to inconsistencies with authentication, tampering protection, and encrypted data transmission. The Apple Watch was also found to be highly secure, though some encrypted data could be accessed with additional steps.
Why Usability Works For It Audit Iacis 2010Lek Voraphan
1) The document discusses using usability testing techniques in IT audits to evaluate user interface problems instead of just typical computer-assisted audit techniques.
2) A usability test of a university's fund management system found high rates of unsuccessful tasks, inefficiency, and low user acceptance, indicating risks of data inaccuracy.
3) Usability testing provides ways to measure user behavior and attitudes that typical audits overlook, but does not find direct evidence of data problems on its own. It helps identify risk areas to then investigate further.
Vulnerability Assessment and Penetration Testing Report Rishabh Upadhyay
This document is Rishabh Upadhyay's bachelor's project on ethical hacking and penetration testing. It includes an acknowledgements section thanking those who provided guidance. The project aims to penetration test the local area network of the University of Allahabad, map the network, identify important hosts and services, and demonstrate some attacks. It also includes developing a simple network scanner program. The document is divided into multiple parts covering introductions to topics like hackers vs ethical hackers and penetration testing methodology, as well as a vulnerability assessment report from testing the university's network.
IBM AppScan - the total software security solution, Content:
- Introduction to security
- Best Practices for Application Security
- IBM AppScan security solution
- DEMO
This document provides an overview of a bug tracking system final year project. It discusses what a bug is, types of bugs, why bug tracking systems are necessary, components of an effective system, and examples of bugs that had extreme effects. It also outlines the proposed software's functionalities, development environment, hardware requirements, timeline, and ER diagram. The document aims to plan and design a bug tracking software application.
Use Combinatorial Testing for Mobile Device FragmentationJosiah Renaudin
A common problem in mobile systems testing is the number of hardware, operational, and software configurations that need to be tested. For example, the so-called Android fragmentation problem might lead a test team to test hundreds of device and software configurations, yielding thousands or even tens-of-thousands of tests. A branch of mathematics, called combinatorics, and associated tools exist that allow teams to minimize the number of test cases required, while assuring high error finding percentages. Jon Hagar defines the fragmentation problem and then examines test patterns supported by tools that can help improve testing success. Jon outlines how combinatorial test patterns can be applied to other testing situations. To solve real-world fragmentation problems, he identifies specific tools, which you can take back to your project for quick use. Reference work and data are provided to help your team justify adding combinatorial testing to your mobile test activities.
Inconvenient Truth(s) - On Application Security (from 2007)Dinis Cruz
This document discusses inconvenient truths about software security. It notes that there are no standardized security metrics, making it difficult for customers to assess security. It also draws parallels between global warming and the growing impact of insecure software. The document argues that secure software does not currently make business sense for vendors due to a lack of incentives. It warns that society's heavy dependence on software leaves it vulnerable if attacker business models evolve to more effectively monetize exploiting insecure systems at scale. Overall, the document presents several inconvenient realities about the current state of software security.
Testing Is How You Avoid Looking StupidSteve Branam
Presented at With The Best IOT online conference, Oct 14 2017: As IOT products become more pervasive, they have an increasing ability to adversely affect the lives of their users and those around them. Testing is the due diligence that closes the engineering loop to verify proper behavior. This presents an introductory overview to testing for IOT products, covering the IOT triad: embedded IOT devices, backend servers, and frontend apps. I talk about the consequences of inadequate testing for companies and individual contributors, and levels and types of testing.
The security of seven popular fitness trackers and the Apple Watch was tested. Some trackers had issues with Bluetooth visibility, authentication, and data tampering. Pebble Time, Microsoft Band 2, and Basis Peak were among the most secure, while Striiv Fusion, Xiaomi MiBand, and Runtastic Moment Elite had the most security risks due to inconsistencies with authentication, tampering protection, and encrypted data transmission. The Apple Watch was also found to be highly secure, though some encrypted data could be accessed with additional steps.
Why Usability Works For It Audit Iacis 2010Lek Voraphan
1) The document discusses using usability testing techniques in IT audits to evaluate user interface problems instead of just typical computer-assisted audit techniques.
2) A usability test of a university's fund management system found high rates of unsuccessful tasks, inefficiency, and low user acceptance, indicating risks of data inaccuracy.
3) Usability testing provides ways to measure user behavior and attitudes that typical audits overlook, but does not find direct evidence of data problems on its own. It helps identify risk areas to then investigate further.
Vulnerability Assessment and Penetration Testing Report Rishabh Upadhyay
This document is Rishabh Upadhyay's bachelor's project on ethical hacking and penetration testing. It includes an acknowledgements section thanking those who provided guidance. The project aims to penetration test the local area network of the University of Allahabad, map the network, identify important hosts and services, and demonstrate some attacks. It also includes developing a simple network scanner program. The document is divided into multiple parts covering introductions to topics like hackers vs ethical hackers and penetration testing methodology, as well as a vulnerability assessment report from testing the university's network.
IBM AppScan - the total software security solution, Content:
- Introduction to security
- Best Practices for Application Security
- IBM AppScan security solution
- DEMO
This document provides an overview of a bug tracking system final year project. It discusses what a bug is, types of bugs, why bug tracking systems are necessary, components of an effective system, and examples of bugs that had extreme effects. It also outlines the proposed software's functionalities, development environment, hardware requirements, timeline, and ER diagram. The document aims to plan and design a bug tracking software application.
IBM AppScan Source is a static application security testing (SAST) tool that scans source code to identify vulnerabilities like SQL injection and cross-site scripting. It has components for analysis, development, remediation, and automation. It can be deployed as a standard desktop tool, in a small workgroup, or in an enterprise environment integrated with other tools. AppScan Source features include importing apps, configuring scans, viewing results, and generating reports. It aims to help security analysts, developers, and organizations identify and fix issues to prevent data breaches and other security problems.
Some security experts would tell you that security testing is very different from functional or non-functional software testing. They are wrong. Having worked on both sides, Paco gives 3 specific recommendations for how testers can make significant contributions to the security of their software and applications by making small changes to the way they do their software testing. The first technique has to do with selecting points in the user journey that are ripe for security testing. The second is to leverage some common free tools that enable security tests. The final technique is adjusting old school boundary value testing and equivalence class partitioning to incorporate security tests. The result is a lot of security testing done and issues fixed long before any security specialists arrive.
Key Takeaways:
-Great places in the user journey to inject security tests
- Ways to augment existing test approaches to cover security concerns
- Typical security tools that are free, cheap, and easy for software testers
What Good is this Tool? A Guide to Choosing the Right Application Security Te...Kevin Fealey
Abstract:
Choosing the right Application Security Testing (AST) tool can be challenging for any security program, and after rolling it out, discovering the real security value it brings can be downright discouraging. No single tool can solve all of all of your security problems, but unfortunately, that is exactly how many of them are marketed. This is compounded by sales teams who convince executive leadership that security programs should be built around their tools, rather than fitting each tool within a well-planned security program. The primary takeaways from this talk are:
• An understanding the real value of each type of AST tool (SAST, DAST, IAST);
• How to leverage your tools for better security visibility and process efficiency;
• Steps to find the right tool for your security program;
• Keys to finding the best stage of the SDLC to implement each tool type within your security program;
• How to integrate new tools with your existing DevOps or Agile environments and processes
Additional Takeaways:
• Examine the strengths and limitations of SAST, DAST, and IAST tools
• Learn how to choose the right tools for your security program
• Discover how to seamlessly integrate your tools into existing DevOps and Agile environments and processes
• Provide security visibility to developers, managers, and executives by enhancing your existing technology
• Learn to use your tools to improve the efficiency of security tasks that are currently manual
This document is a penetration testing report for a customer. It contains details of the testing conducted between specified dates, including vulnerabilities found organized by risk level and category. High risk vulnerabilities were discovered in web applications that could seriously harm the company's reputation. The report provides statistics on vulnerabilities found, methodology used in testing, details of vulnerabilities by system tested, and recommendations for remediation.
IBM Rational App Scan Tester Edition and Quality ManagerАлександр Шамрай
Rational AppScan Tester Edition for Rational Quality Manager allows QA teams to manage security testing alongside other testing types. It seamlessly integrates with Rational Quality Manager to automate security scan execution and defect tracking. The demo showed how to create a web application security scan, run it, analyze results, report a defect, and see the impact on the dashboard. This enables organizations to scale security testing within their existing development and testing processes.
The document summarizes the creation of a penetration testing laboratory by Thomas Butler for his master's degree project. It describes setting up three virtual machines - an "attack machine" running Backtrack5R3, and two "victim machines", one running Metasploitable and another running Badstore.net. Appendices cover the penetration testing methodology, reconnaissance, scanning, exploitation with Metasploit, and post-exploitation activities. The goal was to create a hands-on environment for practicing penetration testing skills.
Issue tracking allows organizations to manage changes and bugs in an ordered way. An issue can represent any query, occurrence, or task that may impact a project, such as a software bug, request for change, or action item from a meeting. When an issue is reported, it goes through a typical workflow of being assigned, accepted, worked on, and eventually closed or reopened. Popular open source bug trackers include Bugzilla, Trac, and JIRA, with Trac being a good example of a lightweight yet powerful option that allows cross-referencing with wikis and version control systems.
This document discusses important factors to consider when investing in penetration testing services, including hiring an experienced testing team, clearly defining the scope of the test, understanding the benefits of blackbox vs whitebox testing, setting goals and objectives for the test, ensuring recommendations are provided in the report, and properly scheduling test events.
Automated identification of load testing problems analyzes execution logs to detect anomalies during load tests. It decomposes and abstracts logs to identify dominant behavior patterns. It then detects deviations to find potential application bugs, environment issues, or load generation problems. Case studies on a DVD store app found specific application errors and environment problems not detectable with typical crash/performance checks. While false positives are possible, this technique reduces the labor of manually analyzing large log volumes.
Black Search Engine Optimisation (SEO), often referred as negative SEO, is a term that covers sabotage techniques aiming to reduce a web site's ranking in search engine results. Black SEO techniques are typically used in business and socio-political contexts, such as information warfare.
The presentation will focus on the use of these techniques to discredit a web site by making it vanish from the major search engine result pages. The discussion will also cover how to exploit common web application vulnerabilities such as Cross Site Scripting, SQL injection and other popular exploitation methods to leverage black SEO attacks. Examples will be included to demonstrate each method of exploitation, and how the vulnerabilities can be used to impact revenues and the reputation of business and political targets.
Black SEO attacks represent a unique class of threats and from a security perspective, any threat which can incur a potential loss should be considered a risk. So far, some of these techniques have only existed as a discussion topic in the SEO industry. Consequently, the intent of my presentation is to bring this complex topic to light to the security community.
Techniques, Tips & Tools For Mobile App TestingSOASTA
Today, mobile app testing expertise is in high demand and offers an exciting career path in test/QA. However, the recent Future of Testing study, sponsored by TechWell, noted that the biggest challenge in mobile―just behind having enough time to test―is expertise. Brad Johnson shares how companies from banking to retail use data from real production users, continuous integration frameworks, cloud-based testing platforms, and real mobile devices to help ensure every user experiences top-rated performance—all the time. Brad shares insight about what to test for mobile, when to first automate, and a metric that will drive real change. Explore how organizations are communicating across teams and improving developer-to-tester collaboration with new approaches. Testers need to develop new skills ranging from software coding requirements to data science. Takeaway tips and ideas to impact your company, enhance your skill set, and propel your career with exciting options and new challenges.
Whittaker How To Break Software Security - SoftTest IrelandDavid O'Dowd
The document discusses different approaches to software testing, specifically functional testing versus security testing. It notes that security testing requires thinking about what the software should not do rather than just what it should do. It provides examples of security bugs related to external dependencies, unanticipated user input, vulnerable design, and vulnerable implementation. It advocates using specific security testing techniques to identify these types of vulnerabilities, such as exploring how applications interact with their environment and inputs they may not anticipate. The key takeaways are to consider what should not happen with a program, understand its environment, identify worst-case scenarios, and use attacks and tools commonly used by hackers to test for security issues.
Static Analysis Tools and Frameworks: Overcoming a Dangerous Blind SpotCigital
More and more organizations are using static analysis tools to find security bugs and other quality issues in software long before the code is tested and released. This is a good thing, and despite their well-known frustrations like high false positive rates and relatively slow speeds, these tools are helping improve the overall security of software.
Unfortunately, these known frustrations may also introduce a dangerous blind spot in these tools which do not know modern frameworks as well as they know the base languages. Learn how organizations are often left feeling secure when they’re not.
New Era of Software with modern Application Security v1.0Dinis Cruz
(as presented at Codemotion Rome 2016)
This presentation will start with an overview of the current state of Application Insecurity (with practical examples). This will make the attendees think twice about what is about to happen to their applications. The solution is to leverage a new generation of application security thinking such as: TDD, Docker, Test Automation, Static Analysis, cleaver Fuzzing, JIRA Risk workflows, Kanban, micro web services visualization, and ELK. These practices will not only make applications/software more secure/resilient, but it allow them to be developed in a much more efficient, cheaper and productive
The Power of an Individual Tester: The HealthCare.gov ExperienceTechWell
Like millions of other Americans, Ben Simo visited HealthCare.gov in search of health insurance and found a frustratingly buggy website that was failing to fulfill its purpose―to educate people on the new health insurance law and help them purchase health insurance. After failing to create an account, Ben put on his tester hat and turned on his web developer tools. In addition to many functional and performance issues, Ben soon discovered a chain of security vulnerabilities that exposed users to unnecessary risk. Finding HealthCare.gov customer service unequipped to receive reports of security vulnerabilities, he blogged his discoveries, spawning a storm of public attention which hailed Ben as a “web expert,” “methodical IT guru," “folk hero”—and “not too bright.” His reports even came up in congressional hearings, where the Secretary of Health and Human Services referred to Ben as “a sort of skilled hacker.” Ben’s reports helped bring attention to problems that suggested a systematic lack of care and understanding of system design and information security. Join Ben as he shares his experience, the issues he found, and lessons testers can learn from HealthCare.gov.
Many projects implicitly use some kind of risk-based approach for prioritizing testing activities. However, critical testing decisions should be based on a product risk assessment process using key business drivers as its foundation. For agile projects, this assessment should be both thorough and lightweight. Erik van Veenendaal discusses PRISMA (PRoduct RISk MAnagement), a highly practical method for performing systematic product risk assessments. Learn how to employ PRISMA techniques in agile projects using Risk Poker. Carry out risk identification and analysis, see how to use the outcome to select the best test approach, and learn how to transform the result into an agile one-page sprint test plan. Erik shares practical experiences and results achieved by employing product risk assessments. Learn how to optimize your test effort by including product risk assessment in your agile testing practices.
Les Honniball leads quality assurance testing for Walt Disney's websites globally. Testing Disney's websites presents unique challenges as the sites must accommodate high volumes of users from around the world on different devices and browsers. This requires testing pages, URLs, analytics, various languages and localizations, and ensuring a quality user experience. Honniball's team uses an agile development process and tools like JIRA, ALM, and Google Docs to manage testing across multiple studios. They employ a variety of testing approaches including automation, and test websites on desktop and mobile browsers and devices to ensure a seamless experience for all users.
Implement an Enterprise Performance Test ProcessTechWell
Suddenly, application performance is important to your business, and you have been given the budget to improve it. You’re in a hurry because customers are complaining or because you expect jumps in transaction volume and your application needs to scale quickly. Do you know where to start? Join Ryan Riehle as he shares his experiences developing enterprise performance testing programs. Ryan covers the key techniques and heuristics that lead to an effective performance improvement effort. He discusses patterns teams use to effectively collaborate to achieve performance requirements, how to configure and organize test environments, considerations for application deployment and release cycles, appropriate metrics to use and how to report them, and strategies and techniques for data movement that support reproducible test results. But measuring alone does not solve the performance problem. So Ryan discusses how teams can act on testing results to improve and verify the impact of application and infrastructure changes.
IBM AppScan Source is a static application security testing (SAST) tool that scans source code to identify vulnerabilities like SQL injection and cross-site scripting. It has components for analysis, development, remediation, and automation. It can be deployed as a standard desktop tool, in a small workgroup, or in an enterprise environment integrated with other tools. AppScan Source features include importing apps, configuring scans, viewing results, and generating reports. It aims to help security analysts, developers, and organizations identify and fix issues to prevent data breaches and other security problems.
Some security experts would tell you that security testing is very different from functional or non-functional software testing. They are wrong. Having worked on both sides, Paco gives 3 specific recommendations for how testers can make significant contributions to the security of their software and applications by making small changes to the way they do their software testing. The first technique has to do with selecting points in the user journey that are ripe for security testing. The second is to leverage some common free tools that enable security tests. The final technique is adjusting old school boundary value testing and equivalence class partitioning to incorporate security tests. The result is a lot of security testing done and issues fixed long before any security specialists arrive.
Key Takeaways:
-Great places in the user journey to inject security tests
- Ways to augment existing test approaches to cover security concerns
- Typical security tools that are free, cheap, and easy for software testers
What Good is this Tool? A Guide to Choosing the Right Application Security Te...Kevin Fealey
Abstract:
Choosing the right Application Security Testing (AST) tool can be challenging for any security program, and after rolling it out, discovering the real security value it brings can be downright discouraging. No single tool can solve all of all of your security problems, but unfortunately, that is exactly how many of them are marketed. This is compounded by sales teams who convince executive leadership that security programs should be built around their tools, rather than fitting each tool within a well-planned security program. The primary takeaways from this talk are:
• An understanding the real value of each type of AST tool (SAST, DAST, IAST);
• How to leverage your tools for better security visibility and process efficiency;
• Steps to find the right tool for your security program;
• Keys to finding the best stage of the SDLC to implement each tool type within your security program;
• How to integrate new tools with your existing DevOps or Agile environments and processes
Additional Takeaways:
• Examine the strengths and limitations of SAST, DAST, and IAST tools
• Learn how to choose the right tools for your security program
• Discover how to seamlessly integrate your tools into existing DevOps and Agile environments and processes
• Provide security visibility to developers, managers, and executives by enhancing your existing technology
• Learn to use your tools to improve the efficiency of security tasks that are currently manual
This document is a penetration testing report for a customer. It contains details of the testing conducted between specified dates, including vulnerabilities found organized by risk level and category. High risk vulnerabilities were discovered in web applications that could seriously harm the company's reputation. The report provides statistics on vulnerabilities found, methodology used in testing, details of vulnerabilities by system tested, and recommendations for remediation.
IBM Rational App Scan Tester Edition and Quality ManagerАлександр Шамрай
Rational AppScan Tester Edition for Rational Quality Manager allows QA teams to manage security testing alongside other testing types. It seamlessly integrates with Rational Quality Manager to automate security scan execution and defect tracking. The demo showed how to create a web application security scan, run it, analyze results, report a defect, and see the impact on the dashboard. This enables organizations to scale security testing within their existing development and testing processes.
The document summarizes the creation of a penetration testing laboratory by Thomas Butler for his master's degree project. It describes setting up three virtual machines - an "attack machine" running Backtrack5R3, and two "victim machines", one running Metasploitable and another running Badstore.net. Appendices cover the penetration testing methodology, reconnaissance, scanning, exploitation with Metasploit, and post-exploitation activities. The goal was to create a hands-on environment for practicing penetration testing skills.
Issue tracking allows organizations to manage changes and bugs in an ordered way. An issue can represent any query, occurrence, or task that may impact a project, such as a software bug, request for change, or action item from a meeting. When an issue is reported, it goes through a typical workflow of being assigned, accepted, worked on, and eventually closed or reopened. Popular open source bug trackers include Bugzilla, Trac, and JIRA, with Trac being a good example of a lightweight yet powerful option that allows cross-referencing with wikis and version control systems.
This document discusses important factors to consider when investing in penetration testing services, including hiring an experienced testing team, clearly defining the scope of the test, understanding the benefits of blackbox vs whitebox testing, setting goals and objectives for the test, ensuring recommendations are provided in the report, and properly scheduling test events.
Automated identification of load testing problems analyzes execution logs to detect anomalies during load tests. It decomposes and abstracts logs to identify dominant behavior patterns. It then detects deviations to find potential application bugs, environment issues, or load generation problems. Case studies on a DVD store app found specific application errors and environment problems not detectable with typical crash/performance checks. While false positives are possible, this technique reduces the labor of manually analyzing large log volumes.
Black Search Engine Optimisation (SEO), often referred as negative SEO, is a term that covers sabotage techniques aiming to reduce a web site's ranking in search engine results. Black SEO techniques are typically used in business and socio-political contexts, such as information warfare.
The presentation will focus on the use of these techniques to discredit a web site by making it vanish from the major search engine result pages. The discussion will also cover how to exploit common web application vulnerabilities such as Cross Site Scripting, SQL injection and other popular exploitation methods to leverage black SEO attacks. Examples will be included to demonstrate each method of exploitation, and how the vulnerabilities can be used to impact revenues and the reputation of business and political targets.
Black SEO attacks represent a unique class of threats and from a security perspective, any threat which can incur a potential loss should be considered a risk. So far, some of these techniques have only existed as a discussion topic in the SEO industry. Consequently, the intent of my presentation is to bring this complex topic to light to the security community.
Techniques, Tips & Tools For Mobile App TestingSOASTA
Today, mobile app testing expertise is in high demand and offers an exciting career path in test/QA. However, the recent Future of Testing study, sponsored by TechWell, noted that the biggest challenge in mobile―just behind having enough time to test―is expertise. Brad Johnson shares how companies from banking to retail use data from real production users, continuous integration frameworks, cloud-based testing platforms, and real mobile devices to help ensure every user experiences top-rated performance—all the time. Brad shares insight about what to test for mobile, when to first automate, and a metric that will drive real change. Explore how organizations are communicating across teams and improving developer-to-tester collaboration with new approaches. Testers need to develop new skills ranging from software coding requirements to data science. Takeaway tips and ideas to impact your company, enhance your skill set, and propel your career with exciting options and new challenges.
Whittaker How To Break Software Security - SoftTest IrelandDavid O'Dowd
The document discusses different approaches to software testing, specifically functional testing versus security testing. It notes that security testing requires thinking about what the software should not do rather than just what it should do. It provides examples of security bugs related to external dependencies, unanticipated user input, vulnerable design, and vulnerable implementation. It advocates using specific security testing techniques to identify these types of vulnerabilities, such as exploring how applications interact with their environment and inputs they may not anticipate. The key takeaways are to consider what should not happen with a program, understand its environment, identify worst-case scenarios, and use attacks and tools commonly used by hackers to test for security issues.
Static Analysis Tools and Frameworks: Overcoming a Dangerous Blind SpotCigital
More and more organizations are using static analysis tools to find security bugs and other quality issues in software long before the code is tested and released. This is a good thing, and despite their well-known frustrations like high false positive rates and relatively slow speeds, these tools are helping improve the overall security of software.
Unfortunately, these known frustrations may also introduce a dangerous blind spot in these tools which do not know modern frameworks as well as they know the base languages. Learn how organizations are often left feeling secure when they’re not.
New Era of Software with modern Application Security v1.0Dinis Cruz
(as presented at Codemotion Rome 2016)
This presentation will start with an overview of the current state of Application Insecurity (with practical examples). This will make the attendees think twice about what is about to happen to their applications. The solution is to leverage a new generation of application security thinking such as: TDD, Docker, Test Automation, Static Analysis, cleaver Fuzzing, JIRA Risk workflows, Kanban, micro web services visualization, and ELK. These practices will not only make applications/software more secure/resilient, but it allow them to be developed in a much more efficient, cheaper and productive
The Power of an Individual Tester: The HealthCare.gov ExperienceTechWell
Like millions of other Americans, Ben Simo visited HealthCare.gov in search of health insurance and found a frustratingly buggy website that was failing to fulfill its purpose―to educate people on the new health insurance law and help them purchase health insurance. After failing to create an account, Ben put on his tester hat and turned on his web developer tools. In addition to many functional and performance issues, Ben soon discovered a chain of security vulnerabilities that exposed users to unnecessary risk. Finding HealthCare.gov customer service unequipped to receive reports of security vulnerabilities, he blogged his discoveries, spawning a storm of public attention which hailed Ben as a “web expert,” “methodical IT guru," “folk hero”—and “not too bright.” His reports even came up in congressional hearings, where the Secretary of Health and Human Services referred to Ben as “a sort of skilled hacker.” Ben’s reports helped bring attention to problems that suggested a systematic lack of care and understanding of system design and information security. Join Ben as he shares his experience, the issues he found, and lessons testers can learn from HealthCare.gov.
Many projects implicitly use some kind of risk-based approach for prioritizing testing activities. However, critical testing decisions should be based on a product risk assessment process using key business drivers as its foundation. For agile projects, this assessment should be both thorough and lightweight. Erik van Veenendaal discusses PRISMA (PRoduct RISk MAnagement), a highly practical method for performing systematic product risk assessments. Learn how to employ PRISMA techniques in agile projects using Risk Poker. Carry out risk identification and analysis, see how to use the outcome to select the best test approach, and learn how to transform the result into an agile one-page sprint test plan. Erik shares practical experiences and results achieved by employing product risk assessments. Learn how to optimize your test effort by including product risk assessment in your agile testing practices.
Les Honniball leads quality assurance testing for Walt Disney's websites globally. Testing Disney's websites presents unique challenges as the sites must accommodate high volumes of users from around the world on different devices and browsers. This requires testing pages, URLs, analytics, various languages and localizations, and ensuring a quality user experience. Honniball's team uses an agile development process and tools like JIRA, ALM, and Google Docs to manage testing across multiple studios. They employ a variety of testing approaches including automation, and test websites on desktop and mobile browsers and devices to ensure a seamless experience for all users.
Implement an Enterprise Performance Test ProcessTechWell
Suddenly, application performance is important to your business, and you have been given the budget to improve it. You’re in a hurry because customers are complaining or because you expect jumps in transaction volume and your application needs to scale quickly. Do you know where to start? Join Ryan Riehle as he shares his experiences developing enterprise performance testing programs. Ryan covers the key techniques and heuristics that lead to an effective performance improvement effort. He discusses patterns teams use to effectively collaborate to achieve performance requirements, how to configure and organize test environments, considerations for application deployment and release cycles, appropriate metrics to use and how to report them, and strategies and techniques for data movement that support reproducible test results. But measuring alone does not solve the performance problem. So Ryan discusses how teams can act on testing results to improve and verify the impact of application and infrastructure changes.
Innovation for Existing Software Product: An R&D ApproachTechWell
In the world of software, innovating an existing product often makes the difference between continued success and rapid irrelevance and failure. Although innovation can come from many different sources, it can be difficult to develop breakthrough innovations while simultaneously trying to maintain an existing piece of software. Aaron Barrett says that a stand-alone R&D team, freed from the constraints of production software, is a great answer to this dilemma. Join Aaron as he shares some simple guidelines to facilitate the process of integrating R&D efforts into an existing software product while avoiding R&D that does not lead to production-ready systems. Learn how and when to get company buy-in, actively engage your developers, and develop with your go-to-market strategy in mind to reap the innovation benefits of a dedicated R&D team.
Rick Craig, a consultant with over 30 years of experience in testing and test management, presented a training on essential test management and planning. The presentation covered topics such as test levels, test methodologies, test planning, and test documentation like the master test plan. It emphasized treating testing as a lifecycle process integrated throughout development.
Jim McKeeth of Embarcadero Technologies gave a presentation on the Internet of Things (IoT) and development considerations. He discussed how everyday devices are increasingly connecting to the internet and how the value of the IoT network increases exponentially as more things connect. McKeeth covered opportunities in various industries, common device types and connectivity standards used in IoT. He emphasized avoiding lock-in to specific networks or platforms and addressed privacy, security and legal issues for developers to consider when creating IoT solutions.
Building on Existing Infrastructure for Mobile ApplicationsTechWell
In 2013 Farm Credit Services of America (FCSAmerica) wanted to enter the mobile application arena so their customers could manage their FCSAmerica lending accounts. Anthony Carlson explains that in the previous thirteen years, FCSAmerica had built an SOA infrastructure for internal applications, including services for customer authentication, lending accounts, and remote check depositing. However, mobility had not been considered when the services were created, and these services were internally protected by a firewall inside their DMZ. If your company has concerns of exposing services to a mobile app, yet wants to reuse what already exists in the enterprise, then the concept of designing services through an API Gateway may be your answer. API Gateways are part of an API Management solution to deal with issues of integration and security. Anthony shares the benefits, challenges, and results of designing a system with an API Management solution to expose services to a mobile application.
Mindmaps: Lightweight Documentation for TestingTechWell
Quality starts with requirements. In small to mid-size companies, it is not uncommon for the communication chain to be broken. Florin Ursu shares ways to avoid miscommunication through a streamlined process in which requirements are communicated to both developers and testers simultaneously; then developers write code while testers document what will be tested. Florin explores what mindmaps are; what they can be used for, both in general and applied to software development; and then dives deeper into how mindmaps can be used for testing. He describes how his teams use mindmaps to brainstorm, organize testing scenarios, prioritize work, review test scenarios, present results to stakeholders highlighting what was tested and (just as important) what was not tested, issues found, and risks. Using example mindmaps, Florin highlights important details captured in day to day work, including tips regarding format, communication style, and how to “sell” the idea of mindmaps to your stakeholders.
Survival Guide: Taming the Data Quality BeastTechWell
As companies scramble to adjust to the demands of an increasingly data-driven world, testers are told “go test data quality” without any guidance as to what that entails or how to go about it. The fact that the data is often a living, flowing ecosystem, rather than just a single object, requires the use of different strategies to gain meaningful insights. Shauna Ayers and Catherine Cruz Agosto guide you through the challenges of data quality and apply a structured approach to analyze, measure, test, and monitor living data sets, and gauge the business impact of data quality issues. Shauna and Catherine define data quality, describe the five goals of data quality management, provide the four pillars of data quality assurance, and show how data flow, scale, and properties interact to build the data quality landscape. Learn how to tame the data quality beast, determine what and how to test, overcome technical obstacles—and emerge with a usable plan of attack.
Why Agile Fails in Large Enterprises—and What to Do about ItTechWell
Agile works. We get it. You don’t have to sell people on the underlying principles anymore. Even so, many large-scale agile transformations are struggling. Some have failed. Others can’t figure out why things aren't working after multiple attempts. It’s easy to blame the people, the process, and the culture. And it’s especially easy to blame management. However, the underlying problem is that most large organizations weren’t built to be agile. You need a way to safely and pragmatically refactor your company into an organization that can adopt agile and sustain the transformation. Mike Cottmeyer introduces a framework for understanding the type of company in which you work, its delivery constraints, and likely challenges you’ll face in your agile transformation. Mike shares a strategy for establishing an end-state vision and operational model to guide your transformation. Finally, he defines an approach for incrementally introducing change, measuring outcomes, and sustaining those changes.
Successful Test Automation: A Manager’s ViewTechWell
Many organizations invest substantial time and effort in test automation but do not achieve the significant returns they expected. Some blame the tool they used; others conclude test automation just doesn't work in their situation. The truth, however, is often very different. These organizations are typically doing many of the right things but they are not addressing key issues that are vital to long term test automation success. Describing the most important issues that you must address, Mark Fewster helps you understand and choose the best approaches for your organization—no matter which automation tools you use. We’ll discuss both management issues—responsibilities, automation objectives, and return on investment—and technical issues—testware architecture, pre- and post-processing, and automated comparison techniques. If you are involved with managing test automation and need to understand the key issues in making test automation successful, join Mark for this enlightening tutorial.
Crafting Smaller User Stories: Examples and ExercisesTechWell
Agile development techniques generally emphasize frequent iterations. But even after adopting agile values, methods, and ceremonies, many organizations struggle to make such iterations work in practice. These organizations inevitably wrestle with agile rhythms until they learn to break up their work into small user stories that will fit within short iterations and allow for fast feedback. Stephen Frein discusses the importance of small user stories and how crucial they are to finishing the stories within the iteration and avoiding a mini-waterfall inside an iteration. After reviewing the characteristics of a good user story, Stephen introduces various techniques for identifying stories that could be decomposed into several other stories, along with accompanying practice exercises to help you get a good feel for the practical aspects of breaking up large stories. Join Stephen if you are having trouble finishing stories within their planned iterations or if your work seems to double in the last days of an iteration.
Metrics Program Implementation: Pitfalls and SuccessesTechWell
When we talk about product quality, test team efficiency, and productivity, we always talk numbers. However, very few companies implement metrics programs in a way that supports solid decision making. Many have tried and failed, leaving a negative impression of metrics. Kris Kosyk explains what metrics like Defect Removal Efficiency tell us and how it is impacted by Test Coverage and Defect Backlog Change Rate. Moving up a level, Kris explains how to use operational testing metrics to understand the development lifecycle process. Though it’s a common belief that a successful metrics program depends on the metrics selected, that is really only half the battle. The other half is a well-designed implementation of the metrics program and effective ongoing governance. Kris addresses these issues and other related questions, and shares a case study on her successes and mistakes while implementing a company-wide test metrics program for more than 200 projects.
Quality Index: A Composite Metric for the Voice of TestingTechWell
It is quite possible that you are spending a considerable amount of your time as a QA manager making sense of the multitude of metrics reported by your teams, connecting the facts, understanding the underlying reality, and articulating it to your peers and leadership. Still, others in the organization may not interpret the message correctly, rendering most of your efforts futile. Nirav Patel and Sutharson Veeravalli share insights to help you resolve this challenge through a composite measure called Quality Index. By aligning metrics to business outcomes and using Quality Index as a tool of articulation, disparate interpretation of data can be eliminated and a cohesive message delivered to stakeholders. Learn how QA can acquire a voice across the senior forums by articulating succinct, contextual, and actionable information to speed up executive decisions in the course of programs and projects.
Software Attacks for Embedded, Mobile, and Internet of ThingsTechWell
In the world of embedded systems, mission-critical mobile apps, and the Internet of Things (IoT), developers and testers must do more than just look for feature bugs. To find potential failures and serious security errors, their arsenal should include attack-based exploratory testing. In the tradition of James Whittaker’s How to Break Software books, Jon Hagar applies the “attack” concept to embedded, mobile, and IoT software. Jon examines common industry patterns of product failures and shares a set of his favorite software test attacks for native, web-based, and hybrid apps. He explains when and how to conduct the attacks, including the pros and cons of some attacks. Take back an arsenal of at least three basic tester attacks, three developer attacks, and three security attacks that you can employ on your current or next project.
Mobile App Testing: Design Automation Patterns You Should UseTechWell
In mobile app development, better test design is important to project velocity and user satisfaction. Jon Hagar explores underused or poorly practiced test design automation approaches that you should employ in development and testing. Jon begins by defining the domain of mobile app software and examines common industry patterns of product failures. He then shares three approaches you can use to speed development and improve quality for native, web-based, and hybrid apps. The methods examined—each supported with detailed checklists—are combinatorial testing, model-based testing, and user experience testing. Jon explains when, where, and how each testing approach can be used to support improved testing and to benefit the whole team. In addition to mobile apps, you and your team can use these same three approaches in other software environments to reduce technical debt during development.
Exploratory testing and the mobile tester : A presentation by Jon HagarGallop Solutions
The document discusses various testing attacks that can be performed on mobile and embedded systems. It begins by providing context on mobile testing and defines key terms. It then outlines 33 different attacks that target common issues like static code analysis bugs, data computation errors, hardware/software interface problems, security vulnerabilities, and usability defects. Several attacks are described in more detail, including developer attacks, basic tester attacks, and dangerous security attacks. The document aims to educate testers on effective exploratory testing techniques for breaking mobile and IoT devices.
In the tradition of James Whittaker’s book series How to Break … Software, Jon Hagar applies the testing “attack” concept to the domain of embedded software systems. Jon defines the sub-domain of embedded software and examines the issues of product failure caused by defects in that software. Next, he shares a set of attacks against embedded software based on common modes of failure that testers can direct against their own software. For specific attacks, Jon explains when and how to conduct the attack, as well as why the attack works to find bugs. In addition to learning these testing skills, attendees get to practice the attacks on a device—a robot that Jon will bring to the tutorial—containing embedded software. Specific attack methods considered include data issues, computation and control structures, hardware-software interfaces, and communications.
Mobile Testing Methodologies: Trends, Successes, and PitfallsTechWell
In today's dynamic mobile marketplace—where new handsets and mobile operating systems are released every day—your ability to deal with these changes which impact your mobile product is vital. The mobile application lifecycle today must be short; must be of great quality; cover a myriad of handsets with different sizes, layouts, and enhanced capabilities; and, of course, cover as many operating systems as possible. This lifecycle requires a new methodology and approach. Eran Kinsbruner describes the mobile project challenges and provides real life examples of ways to overcome them. Take back the main mobile market trends and forecasts together with the key automation tools available for your use today. Learn the differences between the various mobile cloud and automation tools to help you select the right tool for your project. See how you can ramp up a successful mobile project, avoid the common pitfalls, and shorten the time to market—all while delivering a top-notch quality product.
The burgeoning use of mobile devices has created enormous opportunities for organizations to leverage mobile to increase sales, advertise products, and collaborate with internal and external resources. However, with increasing usage, the need to perform testing on these devices is increasing significantly. This is not an easy task considering the number of devices, device operating systems, and operating system versions. To manage the number of variations, organizations rely on mobile testing tools to support their testing efforts. David Dang shares his experiences analyzing numerous mobile testing tool platforms for a prominent shopping network. Learn how identifying the "right" mobile testing tool depends on multiple factors such as supported devices, level of testing, resources, and required integration with other tools. Take back to share with your team a review of common tools on the market and the pros and cons of each.
Implement Combinatorial Test Patterns for Better Mobile and IoT TestingJosiah Renaudin
A common problem in mobile and IoT systems is the large number and combinations of hardware, operational, and software configurations that need to be tested. For example, the so-called Android fragmentation problem might lead a test team to test hundreds of device and several software configurations, potentially yielding thousands or even tens of thousands of tests. Combinatorial testing, a technique involving mathematics and specific tooling, allows teams to reduce the number of test cases, while still assuring good error finding capabilities. Jon Hagar examines test combinatorial patterns supported by tools that will help you speed up testing these many configurations and use for other test tasks, too. During this session Jon will identify and demonstrate specific tools to solve real-world mobile and IoT testing problems. Take back reference materials and data to help your team justify adding combinatorial testing to its toolkit and regular testing activities.
IoT Software Testing Challenges: The IoT World Is Really DifferentTechWell
The Internet of Things (IoT) is poised to become the next growth area—and biggest challenge—for software development and testing. Although many traditional test techniques and strategies remain viable, IoT testing includes working with huge amounts of data, multiple communication channels, device protocols, resource limitations (battery or memory), sensors, controllers, cloud-hardware-device integration, and security concerns. Jon Hagar says that successful IoT testers must develop new knowledge and skills and apply them based on real data and proven test design methods. Testing analytics should include raw test data, data relationships across software integration boundaries, and social media inputs—as well as a keen understanding of sociological and psychological factors. Jon shares insights into math-based testing, model-based testing, attack-based and exploratory testing for IoT applications and systems. Take back a new holistic view for your IoT testing which considers the world environment, connected systems, local systems, and the IoT device itself.
Experitest & Capgemini held a co-webinar on the topic: A Secure Mobile Testing Cloud Resource - Accessible Anytime Anywhere. This interactive webinar will help you to learn more about a Cloud solution that offers features, functions and benefits for each member of the mobile-applications development team:
* For developers an easy reproduction of bugs and the ability to reserve a time slot;
* For QA managers, how to plan in advance device usage and control applications under test;
* For device lab managers a 24x7 continuous testing environment;
* For manual QA testers learn to emulate keyboards for devices and script automation;
* For automation engineers, automate tests via desktop, reserve devices, & generate reports;
* For the security team, resource access from the cloud within your company firewall;
* For executives – reduce user-device procurement costs, speed time-to-market, and improve user/customer experiences.
Security is a high priority when developing and testing mobile apps for companies that are sensitive to security. Access to a provider’s Cloud-hosted testing resources may be convenient and cost-effective, but the security of that environment falls under their control. What was originally a solution becomes a risk to your institution’s data and information.
Among other challenges faced by developers and testers using generic Cloud testing resources:
* Securing a location for devices.
* Concentrating a central pool of devices accessed by offsite development and testing teams;
* Distribution of work across geographically isolated teams;
* Specific device identification and availability;
This document provides an overview of mobile application testing. It discusses why mobile testing is important, the types of mobile applications and testing, challenges in mobile testing like device fragmentation, and methods for testing mobile apps. Key aspects covered include testing installation, networks, user interfaces, different mobile platforms, and automation tools. The document emphasizes the need for thorough testing across various devices and environments to ensure apps work as intended and are bug-free before release.
IoT Software Testing Challenges: The IoT World Is Really DifferentTechWell
With billions of devices containing new software connected to the Internet, the Internet of Things (IoT) is poised to become the next growth area for software development and testing. Although many traditional test techniques and strategies remain viable, challenges in IoT testing include huge amounts of data, multiple communication channels, device protocols, resource limitations (battery or memory), addressing sensors and controllers, cloud-hardware-device integration, and security concerns. Jon Hagar says that for IoT testers to be successful, they must develop new knowledge and skills, and apply them based on real data and proven test design methods. Testing analytics should include raw test data, data relationships across software integration boundaries, and social media inputs, as well as a keen understanding of sociological and psychological factors. Jon shares insights into math-based testing, model-based testing, attack-based exploratory testing, and appropriate types of standards as basics of IoT testing. Take back a new holistic view for your IoT testing which considers the world environment, connected systems, local systems, and the IoT device itself.
This document provides an agenda for mobile app security testing. It discusses topics like mobile OS versions, the mobile app SDLC, testing techniques, vulnerabilities, and security tools. Testing approaches include black box testing, code review, penetration testing and security assessments. Real devices are preferred over emulators due to limitations like missing features and network behavior issues. Common vulnerabilities discussed are cross-site scripting, SQL injection, and client-side injection. Popular security tools mentioned are ZAP, IBM AppScan, HP Fortify, and VeraCode. A three-tiered approach of testing the client, network and server layers is recommended for building secure mobile apps.
Running Head: LAB 5 1
LAB 5 7
Lab 5
Gretchen Greene
Nathan Stewart, PhD
May 8, 2017
Executive Summary
As with any new technology, risks can arise in e-commerce that is not common to those traditional “brick-and-mortar” stores. A huge concern for e-commerce applications is credit/debit card use. Major damage can be done to an organization if the credit/debit card transactions are not secured in terms of financial fraud, loss of consumer confidence, identity theft, or legal regulations.
Online Goodies provides custom promotional gifts to corporate customers and is an Internet-based company. Some of their products include mugs, computer accessories, t-shirts, and office décor. The majority of its income comes from online credit card purchase. They give their repeat customers a discount based on their annual purchase amount.
This report is to create a test plan for Online Goodies based on the OWASP standards. The report includes an overview and rationale of all of the tests performed including a brute force test, an authentication test, privilege escalation test, code injection test, and web application fingerprint test.
Table of Contents
Executive Summary……………………………………………………………………………….2
Table of Contents………………………………………………………………………………….3
Types of Test Being Performed…………………………………………………………………...4
Test Plan for Online Goodies Site According to OWASP Standards……………………………..4
Rationale for Testing Used………………………………………………………………………..4
References…………………………………………………………………………………………7
Types of Tests Performed
The least expensive way to reduce costs and risks and improve software quality is to catch deficiencies as early as possible. To understand the guidelines for testing the OWASP Testing Guide was used. The tests used in this plan are: Usability Testing, Unit Testing, Interface Testing, Integration Testing, Functionality Testing, Performance Testing, Security Testing, Authentication and Authorization Testing, Privilege Escalation Testing, and Web Application Fingerprint Testing.
Test Plan for Online Goodies Site
The purpose of his test plan is to ensure the Goodies site meets all of its business, functional, and technical requirements. The test plan describes the schedule of test activities, test plan strategy, activities, resources, and scope. This document will identify the features on the site to be tested, the testing tasks, the user assigned to each task, each testing environment, techniques, explanation of options, and risks.
Before actually testing the site, you have to create test cases. This is the sample data which will be used to go through the system. These can be created as soon as the requirements are received. Additional test cases should be created to test other aspects of the system due to its complexity.
Explanation of Testing
Usability testing is one of the most important aspects of building a website. Users are not going to take the time to try to use a website that is poorly designed. We are used.
Choosing the Right Testing Strategy to Scale up Mobile App Testing.pdfpCloudy
The document discusses the importance of developing a robust mobile app testing strategy to handle the challenges of mobile app testing at scale. It outlines 14 key elements that should be considered when creating a testing strategy, including device selection, deciding between automated and manual testing, network connectivity testing, performance testing, and security testing. The document stresses the need for a balanced approach that blends automated and manual testing techniques to effectively test mobile apps.
Mobile App Test Attacks to Efficiently Explore SoftwareTEST Huddle
In the tradition of James Whittaker’s book series How to Break … Software, Jon Hagar applies the testing “attack” concept to the domain of mobile app software. Jon defines the sub-domain of mobile software and examines industry product failure caused by defects in that software. Next, Jon summarizes a set of attacks against mobile software based on these common modes of failure that testers can direct against their own app software to quickly find bugs. Specific attack methods identified include developer based cases, computation and control structures for batteries and sensor hubs, hardware-software interfaces, and communications. This session is based on the book: “Software Test Attacks to Break Mobile and Embedded Devices” CRC press, 2013
Key Takeaways:
- Breaking Mobile App Software to find bugs
- Embedded risk-based exploratory testing concepts
- Attack based testing specific to mobile devices
Chapter 3 - Common Test Types and Test Process for Mobile ApplicationsNeeraj Kumar Singh
This is chapter 3 of ISTQB Specialist Mobile Application Tester certification. This presentation helps aspirants understand and prepare the content of the certification.
Softwere Testing Aplication Specific Techniquesmaharajdey
This document discusses various types of software testing techniques including:
1. Unit testing focuses on testing individual units of code to check if they are fit for use.
2. Integration testing tests software modules when integrated together to expose defects.
3. Acceptance testing determines if the software meets acceptance criteria and is performed by end users before moving to production.
It also provides examples of why software testing is important to avoid costly failures and ensure quality, security, and customer satisfaction. A variety of testing types are described from functional to load testing.
Mobile application security testing is important to identify vulnerabilities and protect sensitive user data. The key concepts of mobile app security testing include authentication, authorization, availability, confidentiality, integrity and non-repudiation. Common mobile security threats include malware, spyware, privacy threats and vulnerable applications. Effective security testing employs strategies like strong authentication, encryption, access control and session management. The testing methodology involves profiling the app, analyzing threats, planning tests, executing tests, and providing daily status reports. Deliverables include management reports, technical vulnerability reports, and best practices documents.
Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...Curiosity Software Ireland
The document discusses approaches for determining when to stop testing based on three dimensions of test coverage: system logic, device mix, and system tiers. It advocates targeting tests based on changes across the software development lifecycle to focus testing on at-risk areas. Maintaining a single source of truth and using coverage algorithms can help generate targeted automated tests across user interfaces, APIs and backends in a way that avoids wasteful over-testing or risky under-testing. The presentation concludes with a question and answer section and a call to action to learn more about test modeling and automation approaches.
Similar to Mobile App Testing: The Good, the Bad, and the Ugly (20)
Isabel Evans stopped drawing and painting after being told she was not very good at it, which led to a loss of confidence in her creative and professional abilities. However, she realized that attempting creative activities is important for cognitive and emotional development, and that making mistakes and learning from failures allows for growth. By reengaging with failure through art and with support from others, Isabel was able to regain confidence in her abilities and reboot her career. The document discusses different perspectives on failure and the importance of learning from mistakes.
Instill a DevOps Testing Culture in Your Team and Organization TechWell
The DevOps movement is here. Companies across many industries are breaking down siloed IT departments and federating them into product development teams. Testing and its practices are at the heart of these changes. Traditionally, IT organizations have been staffed with mostly manual testers and a limited number of automation and performance engineers. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay relevant and add greater value to the business. DevOps really starts with testing. Join Adam Auerbach as he explains what DevOps is and how it relates to testing. He describes how testing must change from top to bottom and how to access your own environment to identify improvement opportunities. Adam dives into practices like service virtualization, test data management, and continuous testing so you can understand where you are now and identify steps needed to instill a DevOps testing culture in your team and organization.
Test Design for Fully Automated Build ArchitectureTechWell
This document summarizes a half-day tutorial on test design for fully automated build architectures presented by Melissa Benua of mParticle at STAREAST 2018. The tutorial covered guiding principles for test design including prioritizing important and reliable tests, structuring automated pipelines around components, packages, and releases, and monitoring test results through code coverage, flaky test handling, and logging versus counters. It also included exercises mapping test cases to functional boundaries and categories of tests to pipeline stages.
System-Level Test Automation: Ensuring a Good StartTechWell
Many organizations invest a lot of effort in test automation at the system level but then have serious problems later on. As a leader, how can you ensure that your new automation efforts will get off to a good start? What can you do to ensure that your automation work provides continuing value? This tutorial covers both “theory” and “practice”. Dot Graham explains the critical issues for getting a good start, and Chris Loder describes his experiences in getting good automation started at a number of companies. The tutorial covers the most important management issues you must address for test automation success, particularly when you are new to automation, and how to choose the best approaches for your organization—no matter which automation tools you use. Focusing on system level testing, Dot and Chris explain how automation affects staffing, who should be responsible for which automation tasks, how managers can best support automation efforts to promote success, what you can realistically expect in benefits and how to report them. They explain—for non-techies—the key technical issues that can make or break your automation effort. Come away with your own clarified automation objectives, and a draft test automation strategy to use to plan your own system-level test automation.
Build Your Mobile App Quality and Test StrategyTechWell
Let’s build a mobile app quality and testing strategy together. Whether you have a web, hybrid, or native app, building a quality and testing strategy means (1) knowing what data and tools you have available to make agile decisions, (2) understanding your customers and your competitors, and (3) testing your app under real-world conditions. Jason Arbon guides you through the latest techniques, data, and tools to ensure the awesomeness of your mobile app quality and testing strategy. Leave this interactive session with a strategy for your very own app—or one you pretend to own. The information Jason shares is based on data from Appdiff’s next-gen mobile app testing platform, lessons from Applause/uTest’s crowd, text mining hundreds of millions of app store reviews, and in-depth discussions with top mobile app development teams.
Testing Transformation: The Art and Science for SuccessTechWell
Technologies, testing processes, and the role of the tester have evolved significantly in the past few years with the advent of agile, DevOps, and other new technologies. It is critical that we testing professionals evaluate ourselves and continue to add tangible value to our organizations. In your work, are you focused on the trivial or on real game changers? Jennifer Bonine describes critical elements that help you artfully blend people, process, and technology to create a synergistic relationship that adds value. Jennifer shares ideas on mastering politics, maneuvering core vs. context, and innovating your technology strategies and processes. She explores how new processes can be introduced in an organization, what the role of organizational culture is in determining the success of a project, and how you can know what tools will add value vs. simply adding overhead and complexity. Jennifer reviews critically needed tester skills and discusses a continual learning model to evolve your skills and stay relevant. This discussion can lead you to technologies, processes, and skills you can stake your career on.
We’ve all been there. We work incredibly hard to develop a feature and design tests based on written requirements. We build a detailed test plan that aligns the tests with the software and the documented business needs. And when we put the tests to the software, it all falls apart because the requirements were changed without informing everyone. Mary Thorn says help is at hand. Enter behavior-driven development (BDD), and Cucumber and SpecFlow, tools for running automated acceptance tests and facilitating BDD. Mary explores the nuances of Cucumber and SpecFlow, and shows you how to implement BDD and agile acceptance testing. By fostering collaboration for implementing active requirements via a common language and format, Cucumber and SpecFlow bridge the communication gap between business stakeholders and implementation teams. In this workshop, practice writing feature files with the best practices Mary has discovered over numerous implementations. If you experience developers not coding to requirements, testers not getting requirements updates, or customers who feel out of the loop and don’t get what they ask for, Mary has answers for you.
Develop WebDriver Automated Tests—and Keep Your SanityTechWell
Many teams go crazy because of brittle, high-maintenance automated test suites. Jim Holmes helps you understand how to create a flexible, maintainable, high-value suite of functional tests using Selenium WebDriver. Learn the basics of what to test, what not to test, and how to avoid overlapping with other types of testing. Jim includes both philosophical concepts and hands-on coding. Testers who haven't written code should not be intimidated! We'll pair you up to make sure you're successful. Learn to create practical tests dealing with advanced situations such as input validation, AJAX delays, and working with file downloads. Additionally, discover when you need to work together with developers to create a system that's more easily testable. This tutorial focuses primarily on automating web tests, but many of the same concepts can be applied to other UI environments. Demos and labs will be in C# and Java using WebDriver. Leave this tutorial having learned how to write high-value WebDriver tests—and stay sane while doing so.
DevOps is a cultural shift aimed at streamlining intergroup communication and improving operational efficiency for development and operations groups. Over time, inclusion of other IT groups under the DevOps umbrella has become the norm for many organizations. But even broadening the boundaries of DevOps, the conversation has been largely devoid of the business units’ place at the table. A common mistake organizations make while going through the DevOps transformation is drawing a line at the IT boundary. If that occurs, a larger, more inclusive silo within the organization is created, operating in an informational vacuum and causing operational inefficiency and goal misalignment. Sharing his experiences working on both sides of the fence, Leon Fayer describes the importance of including business units in order to align technology decisions with business goals. Leon discusses inclusion of business units in existing agile processes, benefits of cross-departmental monitoring, and a business-first approach to technology decisions.
Eliminate Cloud Waste with a Holistic DevOps StrategyTechWell
Chris Parlette maintains that renting infrastructure on demand is the most disruptive trend in IT in decades. In 2016, enterprises spent $23B on public cloud IaaS services. By 2020, that figure is expected to reach $65B. The public cloud is now used like a utility, and like any utility, there is waste. Who's responsible for optimizing the infrastructure and reducing wasted expenses? It’s DevOps. The excess expense, known as cloud waste, comprises several interrelated problems: services running when they don't need to be, improperly sized infrastructure, orphaned resources, and shadow IT. There are a few core tenets of DevOps—holistic thinking, no silos, rapid useful feedback, and automation—that can be applied to reducing your cloud waste. Join Chris to learn why you should include continuous cost optimization in your DevOps processes. Automate cost control, reduce your cloud expenses, and make your life easier.
Transform Test Organizations for the New World of DevOpsTechWell
With the recent emergence of DevOps across the industry, testing organizations are being challenged to transform themselves significantly within a short period of time to stay meaningful within their organizations. It’s not easy to plan and approach these changes considering the way testing organizations have remained structured for ages. These challenges start from foundational organizational structures and can cut across leadership influence, competencies, tools strategy, infrastructure, and other dimensions. Sumit Kumar shares his experience assisting various organizations to overcome these challenges using an organized DevOps enablement framework. The framework includes radical restructuring, turning the tools strategy upside down, a multidimensional workforce enablement supported by infrastructure changes, redeveloped collaborations models, and more. From his real world experiences Sumit shares tips for approaching this journey and explains the roadmap for testing organizations to transform themselves to lead the quality in DevOps.
The Fourth Constraint in Project Delivery—LeadershipTechWell
All too often, the triple constraints—time, cost, and quality—are bandied about as if they are the be-all, end-all. While they are important, leadership—the fourth and larger underpinning constraint—influences the first three. Statistics on project success and failure abound, and these measurements are usually taken against the triple constraints. According to the Project Management Institute, only 53 percent of projects are completed within budget, and only 49 percent are completed on time. If so many projects overrun budget and are late, we can’t really say, “Good, fast, or cheap—pick two.” Rob Burkett talks about leadership at every level of a team. He shares his insights and stories gleaned from his years of IT and project management experience. Rob speaks to some of the glaring difficulties in the workplace in general and some specifically related to IT delivery and project management. Leave with a clearer understanding of how to communicate with teams and team members, and gain a better understanding of how you can be a leader—up and down your organization.
Resolve the Contradiction of Specialists within Agile TeamsTechWell
As teams grow, organizations often draw a distinction between feature teams, which deliver the visible business value to the user, and component teams, which manage shared work. Steve Berczuk says that this distinction can help organizations be more productive and scale effectively, but he recognizes that not all shared work fits into this model. Some work is best handled by “specialists,” that is people with unique skills. Although teams composed entirely of T-shaped people is ideal, certain skills are hard to come by and are used irregularly across an organization. Since these specialists often need to work closely with teams, rather than working from their own backlog, they don’t fit into the component team model. The use of shared resources presents challenges to the agile planning model. Steve Berczuk shares how teams such as those providing infrastructure services and specialists can fit into a feature+component team model, and how variations such as embedding specialists in a scrum team can both present process challenges and add significant value to both the team and the larger organization.
Pin the Tail on the Metric: A Field-Tested Agile GameTechWell
Metrics don’t have to be a necessary evil. If done right, metrics can help guide us to make better forward-looking decisions, rather than being used for simply managing or monitoring. They can help us identify trade-offs between options for what to do next versus punitive or worse, purely managerial measures. Steve Martin won’t be giving the Top Ten List of field-tested metrics you should use. Instead, in this interactive mini-workshop, he leads you through the critical thinking necessary for you to determine what is right for you to measure. First, Steve explores why you want to measure something—whether it’s for a team, a portfolio, or even an agile transformation. Next, he provides multiple real-life metrics examples to help drive home concepts behind characteristics of good and bad metrics. Finally, Steve shows how to run his field-tested agile game—Pin the Tail on the Metric. Take back this activity to help you guide metrics conversations at your organization.
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsTechWell
A hierarchy is an organizational network that has a top and a bottom, and where position is determined by rank, importance, and value. A holarchy is a network that has no top or bottom and where each person’s value derives from his ability, rather than position. As more companies seek the benefits of agile, leaders need to build and sustain delivery capability while scaling agile without introducing unnecessary process and overhead. The Agile Performance Holarchy (APH) is an empirical model for scaling and sustaining agility while continuing to deliver great products. Jeff Dalton designed the APH by drawing from lessons learned observing and assessing hundreds of agile companies and teams. The APH helps implement a holarchy—a system composed of interacting organizational units called holons—centered on a series of performance circles that embody the behaviors of high performing agile organizations. Jeff describes how APH provides guidelines in the areas of leadership, values, teaming, visioning, governing, building, supporting, and engaging within an all-agile organization. Join Jeff to see what the APH is all about and how you can use it in your team and organization.
A Business-First Approach to DevOps ImplementationTechWell
DevOps is a cultural shift aimed at streamlining intergroup communication and improving operational efficiency for development and operations groups. Over time, inclusion of other IT groups under the DevOps umbrella has become the norm for many organizations. But even broadening the boundaries of DevOps, the conversation has been largely devoid of the business units’ place at the table. A common mistake organizations make while going through the DevOps transformation is drawing a line at the IT boundary. If that occurs, a larger, more inclusive silo within the organization is created, operating in an informational vacuum and causing operational inefficiency and goal misalignment. Sharing his experiences working on both sides of the fence, Leon Fayer describes the importance of including business units in order to align technology decisions with business goals. Leon discusses inclusion of business units in existing agile processes, benefits of cross-departmental monitoring, and a business-first approach to technology decisions.
Databases in a Continuous Integration/Delivery ProcessTechWell
The document summarizes a presentation about including databases in a continuous integration/delivery process. It discusses treating database code like application code by placing it under version control and integrating databases into the DevOps software development pipeline. This allows databases to be built, tested, and released like other software through continuous integration, delivery, and deployment.
Mobile Testing: What—and What Not—to AutomateTechWell
Organizations are moving rapidly into mobile technology, which has significantly increased the demand for testing of mobile applications. David Dangs says testers naturally are turning to automation to help ease the workload, increase potential test coverage, and improve testing efficiency. But should you try to automate all things mobile? Unfortunately, the answer is not always clear. Mobile has its own set of complications, compounded by a wide variety of devices and OS platforms. Join David to learn what mobile testing activities are ripe for automation—and those items best left to manual efforts. He describes the various considerations for automating each type of mobile application: mobile web, native app, and hybrid applications. David also covers device-level testing, types of testing, available automation tools, and recommendations for automation effectiveness. Finally, based on his years of mobile testing experience, David provides some tips and tricks to approach mobile automation. Leave with a clear plan for automating your mobile applications.
Cultural Intelligence: A Key Skill for SuccessTechWell
Diversity is becoming the norm in everyday life. However, introducing global delivery models without a proper understanding of intercultural differences can lead to difficulty, frustration, and reduced productivity. Priyanka Sharma and Thena Barry say that in our diverse world, we need teams with people who can cross these boundaries, communicate effectively, and build the diverse networks necessary to avoid problems. We need to learn about cultural intelligence (CI) and cultural quotient (CQ). CI is the ability to relate and work effectively across cultures. CQ is the cognitive, motivational, and behavioral capacity to understand and respond to beliefs, values, attitudes, and behaviors of individuals and groups. Together, CI and CQ can help us build behavioral capacities that aid motivation, behavior, and productivity in teams as well as individuals. Priyanka and Thena show how to build a more culturally intelligent place with tools and techniques from Leading with Cultural Intelligence, as well as content from the Hofstede cultural model. In addition, they illustrate the model with real-life experiences and demonstrate how they adapted in similar circumstances.
Turn the Lights On: A Power Utility Company's Agile TransformationTechWell
Why would a century-old utility with no direct competitors take on the challenge of transforming its entire IT application organization to an agile methodology? In an increasingly interconnected world, the expectations of customers continue to evolve. From smart meters to smart phones, IoT is creating a crisis point for industries not accustomed to rapid change. Glen Morris explains that pizzas can be tracked by the minute and packages at every stop, and customers now expect this same customer service model should exist for all industries—including power. Glen examines how to create momentum and transform non-IT-focused industries to an agile model. If you are struggling with gaining traction in your pursuit of agile within your business, Glen gives you concrete, practical experiences to leverage in your pursuit. Finally, he communicates how to gain buy-in from business partners who have no idea or concern about agile or its methodologies. If your business partners look at you with amusement when you mention the need for a dedicated Product Owner, join Glen as he walks you through the approaches to overcoming agile skepticism.
When it is all about ERP solutions, companies typically meet their needs with common ERP solutions like SAP, Oracle, and Microsoft Dynamics. These big players have demonstrated that ERP systems can be either simple or highly comprehensive. This remains true today, but there are new factors to consider, including a promising new contender in the market that’s Odoo. This blog compares Odoo ERP with traditional ERP systems and explains why many companies now see Odoo ERP as the best choice.
What are ERP Systems?
An ERP, or Enterprise Resource Planning, system provides your company with valuable information to help you make better decisions and boost your ROI. You should choose an ERP system based on your company’s specific needs. For instance, if you run a manufacturing or retail business, you will need an ERP system that efficiently manages inventory. A consulting firm, on the other hand, would benefit from an ERP system that enhances daily operations. Similarly, eCommerce stores would select an ERP system tailored to their needs.
Because different businesses have different requirements, ERP system functionalities can vary. Among the various ERP systems available, Odoo ERP is considered one of the best in the ERp market with more than 12 million global users today.
Odoo is an open-source ERP system initially designed for small to medium-sized businesses but now suitable for a wide range of companies. Odoo offers a scalable and configurable point-of-sale management solution and allows you to create customised modules for specific industries. Odoo is gaining more popularity because it is built in a way that allows easy customisation, has a user-friendly interface, and is affordable. Here, you will cover the main differences and get to know why Odoo is gaining attention despite the many other ERP systems available in the market.
8 Best Automated Android App Testing Tool and Framework in 2024.pdfkalichargn70th171
Regarding mobile operating systems, two major players dominate our thoughts: Android and iPhone. With Android leading the market, software development companies are focused on delivering apps compatible with this OS. Ensuring an app's functionality across various Android devices, OS versions, and hardware specifications is critical, making Android app testing essential.
Neo4j - Product Vision and Knowledge Graphs - GraphSummit ParisNeo4j
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfVALiNTRY360
Salesforce Healthcare CRM, implemented by VALiNTRY360, revolutionizes patient management by enhancing patient engagement, streamlining administrative processes, and improving care coordination. Its advanced analytics, robust security, and seamless integration with telehealth services ensure that healthcare providers can deliver personalized, efficient, and secure patient care. By automating routine tasks and providing actionable insights, Salesforce Healthcare CRM enables healthcare providers to focus on delivering high-quality care, leading to better patient outcomes and higher satisfaction. VALiNTRY360's expertise ensures a tailored solution that meets the unique needs of any healthcare practice, from small clinics to large hospital systems.
For more info visit us https://valintry360.com/solutions/health-life-sciences
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemPeter Muessig
Learn about the latest innovations in and around OpenUI5/SAPUI5: UI5 Tooling, UI5 linter, UI5 Web Components, Web Components Integration, UI5 2.x, UI5 GenAI.
Recording:
https://www.youtube.com/live/MSdGLG2zLy8?si=INxBHTqkwHhxV5Ta&t=0
SOCRadar's Aviation Industry Q1 Incident Report is out now!
The aviation industry has always been a prime target for cybercriminals due to its critical infrastructure and high stakes. In the first quarter of 2024, the sector faced an alarming surge in cybersecurity threats, revealing its vulnerabilities and the relentless sophistication of cyber attackers.
SOCRadar’s Aviation Industry, Quarterly Incident Report, provides an in-depth analysis of these threats, detected and examined through our extensive monitoring of hacker forums, Telegram channels, and dark web platforms.
UI5con 2024 - Bring Your Own Design SystemPeter Muessig
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
Mobile App Development Company In Noida | Drona InfotechDrona Infotech
Drona Infotech is a premier mobile app development company in Noida, providing cutting-edge solutions for businesses.
Visit Us For : https://www.dronainfotech.com/mobile-application-development/
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesQuickdice ERP
Explore the seamless transition to e-invoicing with this comprehensive guide tailored for Saudi Arabian businesses. Navigate the process effortlessly with step-by-step instructions designed to streamline implementation and enhance efficiency.
Hand Rolled Applicative User ValidationCode KataPhilip Schwarz
Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>?
The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory.
Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away.
The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.
Unveiling the Advantages of Agile Software Development.pdfbrainerhub1
Learn about Agile Software Development's advantages. Simplify your workflow to spur quicker innovation. Jump right in! We have also discussed the advantages.
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
Microservice Teams - How the cloud changes the way we workSven Peters
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
WWDC 2024 Keynote Review: For CocoaCoders AustinPatrick Weigel
Overview of WWDC 2024 Keynote Address.
Covers: Apple Intelligence, iOS18, macOS Sequoia, iPadOS, watchOS, visionOS, and Apple TV+.
Understandable dialogue on Apple TV+
On-device app controlling AI.
Access to ChatGPT with a guest appearance by Chief Data Thief Sam Altman!
App Locking! iPhone Mirroring! And a Calculator!!
Mobile App Testing: The Good, the Bad, and the Ugly
1. 4/23/15
1
Mobile
App
Testing:
The
Good,
The
Bad,
and
The
Ugly
Jon
D.
Hagar,
Consultant,
Grand
Software
Testing
embedded@ecentral.com
Author:
Software
Test
Attacks
to
Break
Mobile
and
Embedded
Devices
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
1
* Gaming
Testing
Story
* It
only
takes
a
few
minutes
using
an
App
before
users
like
or
hate
it
* Worse
than
that.
.
.
* Many
users
will
post
a
social
media
review
of
the
app
* You
don’t
want
to
be
a
BAD
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
2
The
Mobile
Opportunity
2. 4/23/15
2
* Depth
* Passion
* Speed
What
Does
it
Take
to
be
a
Great
Mobile
App
Tester?
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
Software Test Attacks to Break Mobile and Embedded Devices
3
* As
the
names
imply,
these
are
devices—small,
held
in
the
hand,
connected
to
communication
networks,
including
* Cell
and
smart
phones
–
apps
* Tablets
* Medical
devices
* Typically
have:
* Many
of
the
problems
of
classic
embedded
systems
* The
power
of
PCs/IT
* More
user
interface
(UI)
than
classic
embedded
systems
* Fast
and
frequent
updates
* However,
mobile
devices
are
“evolving”
with
more
power,
resources,
apps,
etc.
* Mobile
is
the
“hot”
area
of
computers/software
* Testing
rules
and
concepts
are
still
evolving
* Now
starting
to
include
IoT
You
know
what
they
are
right?
Mobile
and
Handheld?
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
3. 4/23/15
3
* Requirements
verification
checking
* Necessary
but
not
sufficient
* Risk–based
testing
* Tried
and
true
in
many
contexts
including
mobile,
but
we
need
more
Here
comes
the
Good,
Bad
and
Ugly
We
Need
Better
App
Testing
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
The
Bad
You
are
between
a
Management
Rock
and
a
Hard
App
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
6
4. 4/23/15
4
* Management
directed
“No
testing”
* Dev-‐ops
without
enough
“thinking”
of
context
and
risk
to
find
the
big
BUGS
* Stupid
requirements
verification
checking
without
GOOD
test
activities
* Testing
without
thinking
of
* cost
* schedule
* users
Con:
Current
Badness
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
7
* Are
you
part
of
the
problem?
* Do
you
help
management
“SEE”
the
info
they
need?
* Are
you
Agile?
* Are
you
using
your
testing
skills
daily?
* Bug
are
out
there
(and
always
will
be)…………..
Pro:
In
the
Bad
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
Software Test Attacks to Break Mobile and Embedded Devices
8
5. 4/23/15
5
* From
Wikipedia:
Taxonomy
is
the
practice
and
science
of
classification.
The
word
finds
its
roots
in
the
Greek
τάξις,
taxis
(meaning
'order',
'arrangement')
and
νόμος,
nomos
('law'
or
'science').
Taxonomy
uses
taxonomic
units,
known
as
taxa
(singular
taxon).
In
addition,
the
word
is
also
used
as
a
count
noun:
a
taxonomy,
or
taxonomic
scheme,
is
a
particular
classification
("the
taxonomy
of
..."),
arranged
in
a
hierarchical
structure.
* Helping
to
“understand
and
know”
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
9
A
Bad
Situation
-‐
Lets
look
for
bugs,
but
where?
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
10
Pro:
Taxonomy
(researched)
Super
Category
Aero-‐Space
Med
sys
Mobile
General
Time
3
2
3
Interrupted
-‐
Satura>on
(over
>me)
5.5
Time
Boundary
–
failure
resul>ng
from
incompa>ble
system
>me
formats
or
values
0.5
1
Time
-‐
Race
Condi>ons
3
1
Time
-‐
Long
run
usages
4
1
20
Interrupt
-‐
>ming
or
priority
inversions
0.7
3
Date(s)
wrong/cause
problem
0.5
1
Clocks
4
2
Computa>on
-‐
Flow
6
23
19
Computa>on
-‐
on
data
4
1
3
1
6. 4/23/15
6
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
11
Taxonomy
part
2
Super
Category
Aero-‐Space
Med
sys
Mobile
General
Data
(wrong
data
loaded
or
used)
4
5.00
2
Ini>aliza>on
6
2.00
3
5
Pointers
8
2.00
18
10
Logic
and/or
control
law
ordering
8
43
3
30
Loop
control
–Recursion
1
Decision
point
(if
test
structure)
0.5
1
1
Logically
Impossible
&
dead
code
0.7
Opera>ng
system
–
(Lack
of
Fault
tolerance
,
interface
to
OS,
other)
1.5
2
6
Software - Hardware interfaces
16
13
So9ware
-‐
Software Interface
5
2.00
3
So9ware
-‐
Bad command- problem
on server
3
5
UI
-‐
User/
operator
interface
4
5.00
20
10
UI
-‐
Bad
Alarm
0.5
3
UI
-‐
Training
–
system
fault
resul>ng
from
improper
training
3
Other
10.6
9.00
5
5
Note:
one
report
on
C/C++
indicated
70%
of
errors
found
involved
pointers
* How
many
of
you
have
a
Mobile
App
taxonomy
that
you
use?
Question
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
12
7. 4/23/15
7
The
Ugly
We need Wisdom, Tooling, and Security
13
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* Some
of
you
lack
mobile
tester
skills
* Many
of
you
suffer
from
group
think
and
lack
wisdom
* We
listen
to
the
loudest
voices
* Testers
do
not
use
available
ideas
to
aide
their
skill
base
* Attacks,
techniques,
tools,
concepts,
standards,
etc.
Con:
Mobile
can
have
an
Ugly
Face
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
14
8. 4/23/15
8
* Danger
of
group
think
in
Agile
Mobile
Teams
* Amplification
* Snowballing
effect
* Polarization
* Ignoring
critical
minority
opinions
Pro:
You
Need
Test
Wisdom
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
15
* Stop
talking
and
LISTEN
to
all
sides,
particularly
the
ones
you
may
not
agree
with
* Question
beliefs
* Be
passionate
and
follow
your
bliss
about
testing
* Try
to
remain
open
minded
* Do
not
submit
to
the
negatives
of
group
think
* Consider
the
context
of
the
testing
and
believe
that
context
matters
* Seek
the
council
of
people
you
believe
to
be
wise
* Reward
your
test
team
for
being
open
and
providing
other
views
without
fear
* Try
to
take
a
role
of
“devil’s
advocate”
in
your
test
team
* Fight
the
“me
too”
syndrome
and
everyone
falling
in
line
to
the
loudest
voice
* Work
to
be
a
knowledgeable
and
skilled
tester
(they
are
different)
* Be
the
voice
of
loyal
opposition
in
the
team
and
think
outside
of
the
group
“box”
* Don’t
paint
a
viewpoint
as
totally
invalid,
when
a
few
ideas
of
the
viewpoint
conflict
with
local
ideals
Seeking
Test
Wisdom
(Pro:
try
these
tricks)
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
16
9. 4/23/15
9
Categories
of
Automation
Tooling
(Open
Source
and
Commercial)
*
Capture
Playback
-‐
Actual
devices
(cabinet
vs
a
pile)
vs
Emulator
-‐
API
vs
GUI/UI
*
Planning
and
lifecycle
support
*
Modeling
-‐
Risks
-‐
Mind-‐mapping
-‐
Formal
models
(UTP)
-‐
Test
Techniques
Pro/Con?
-‐
Mobile/Handheld
Test
Tools
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
17
* To
Automate
or
Not?
* When
testing
configurations
of
hw/sw
(good
idea)
* When
testing
combinations
(combinatorial
test
tools)
* When
dealing
with
testing
qualities
* Security
(very
good
idea)
* Reliability
(necessary)
* Configuration
management
(can
not
be
done
without)
* Usability
(important
but
a
hard
one
and
questionable
tools)
* When
supporting
Development
* Structural
testing
(measures
coverage)
* Static
code
analysis
(finds
hard
to
test
bugs)
* Dev-‐Ops,
Continuous
Integration
and
Agile
(really
good)
More
on
Test
Tools
–
Now
in
Mobile
Support
has
Improved
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
Software Test Attacks to Break Mobile and Embedded Devices
18
10. 4/23/15
10
* Your
app
gets
on
the
nightly
news
* Your
team
sees
security
as
someone
else’s
problem
Real
Ugly:
Security
and
Privacy
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
19
* Mobile–
IoT
systems
are
highly
integrated
hardware–
software–system
solutions
which:
* Must
be
highly
trustworthy
since
they
handle
sensitive
data
* Often
perform
critical
tasks
* Security
holes
and
problems
abound
* Coverity
Scan
2010
Open
Source
Integrity
Report
-‐
Android
* Static
analysis
test
attack
found
0.47
defects
per
1,000
SLOC
* 359
defects
in
total,
88
of
which
were
considered
“high
risk”
in
the
security
domain
* OS
hole
Android
with
Angry
Birds
* Researchers
Jon
Oberheide
and
Zach
Lanier
* Robots
and
Drones
rumored
to
be
attacked
* Cars
and
medical
devices
being
hacked
The
Current
Security
Situation
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
11. 4/23/15
11
* Fraud
–
Identity
* Worms,
virus,
etc.
* Fault
injection
* Processing
on
the
run
* Hacks
impact
* Power
* Memory
* CPU
usage
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
Con:
Mobile
Security
Bugs
(taxonomy)
• Eavesdropping
–
“yes
everyone
can
hear
you”
• Hijacking
• Click-‐jacking
• Voice/Screen
• Physical
Hacks
• File
snooping
• Lost
phone
* A
pattern
(of
testing)
based
on
a
common
mode
of
failure
seen
over
and
over
* Part
of
Exploratory
Testing
* May
be
seen
as
a
negative,
when
it
really
is
a
positive
* Goes
after
the
“bugs”
that
may
be
in
the
software
* May
include
or
use
classic
test
techniques
and
test
concepts
* Lee
Copeland’s
book
on
test
design
* Many
other
good
books
* A
Pattern
(more
than
a
process)
which
must
be
modified
for
the
context
at
hand
to
do
the
testing
* Testers
learn
mental
attack
patterns
working
over
the
years
in
a
specific
domain
Pro:
Apply
Attack-‐based
Testing
What
is
an
attack?
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
12. 4/23/15
12
* Apply
when
the
device
is
mobile
and
has
* Account
numbers
* User-‐ids
and
passwords
* Location
tags
* Restricted
data
*
Current
authentication
approaches
in
use
on
mobile
devices
* Server-‐based
* Registry
(user/password)
* Location
or
device-‐based
* Profile-‐based
Security
Attacks
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* Attack
28
Penetration
Attack
Test
* Attack
28.1
Penetration
Sub–Attacks:
Authentication
—
Password
* Attack
28.2
Sub–Attack
Fuzz
Test
* Attack
29:
Information
Theft—Stealing
Device
Data
* Attack
29.1
Sub
Attack
–Identity
Social
Engineering
* Attack
30:
Spoofing
Attacks
* Attack
30.1
Location
and/or
User
Profile
Spoof
Sub–Attack
* Attack
30.2
GPS
Spoof
Sub–Attack
Security
Attacks
(Con:
only
a
starting
point,
a
checklist
of
things
to
start
with)
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
13. 4/23/15
13
* What
kind
of
App
software
do
you
work
on?
* Security
concerns?
* Privacy
concerns?
What
is
missing?
Exercise
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
25
§ Security
attacks
must
be
done
with
the
knowledge
and
approval
of
owners
of
the
system
and
software
§ Severe
legal
implications
exist
in
this
area
§ Many
of
these
attacks
must
be
done
in
a
lab
(sandbox)
§ In
these
attacks,
I
tell
you
conceptually
how
to
“drive
a
car
very
fast
(150
miles
an
hour)
but
there
are
places
to
do
this
with
a
car
legally
(a
race
track)
and
places
where
you
will
get
a
ticket
(most
public
streets)”
§ Be
forewarned
-‐
Do
not
attack
you
favorite
app
on
your
phone
or
any
connected
server
without
the
right
permissions
due
to
legal
implications
Warnings
When
Conducting
Security
Attacks
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
14. 4/23/15
14
Finally,
The
Good
–
Functional
and
Non-‐functional
Experiments
and
Attacks
(Exploratory
testing)
Skills
App
testers
should
have
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–”So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
27
Attacks
(from
Software
Test
Attacks
to
Break
Mobile
and
Embedded
Devices)
* Attack
1:
Static
Code
Analysis
* Attack
2:
Finding
White–Box
Data
Computation
Bugs
* Attack
3:
White–Box
Structural
Logic
Flow
Coverage
* Attack
4:
Finding
Hardware–System
Unhandled
Uses
in
Software
* Attack
5:
Hw-‐Sw
and
Sw-‐Hw
signal
Interface
Bugs
* Attack
6:
Long
Duration
Control
Attack
Runs
* Attack
7:
Breaking
Software
Logic
and/or
Control
Laws
* Attack
8:
Forcing
the
Unusual
Bug
Cases
* Attack
9
Breaking
Software
with
Hardware
and
System
Operations
* 9.1
Sub–Attack:
Breaking
Battery
Power
* Attack
10:
Finding
Bugs
in
Hardware–Software
Communications
* Attack
11:
Breaking
Software
Error
Recovery
* Attack
12:
Interface
and
Integration
Testing
* 12.1
Sub–Attack:
Configuration
Integration
Evaluation
* Attack
13:
Finding
Problems
in
Software–System
Fault
Tolerance
* Attack
14:
Breaking
Digital
Software
Communications
* Attack
15:
Finding
Bugs
in
the
Data
* Attack
16:
Bugs
in
System–Software
Computation
* Attack
17:
Using
Simulation
and
Stimulation
to
Drive
Software
Attacks
* Attack
18:
Bugs
in
Timing
Interrupts
and
Priority
Inversion
* Attack
19:
Finding
Time
Related
Bugs
* Attack
20:
Time
Related
Scenarios,
Stories
and
Tours
* Attack
21:
Performance
Testing
Introduction
* Attack
22:
Finding
Supporting
(User)
Documentation
Problems
* Sub–Attack
22.1:
Confirming
Install–ability
* Attack
23:
Finding
Missing
or
Wrong
Alarms
* Attack
24:
Finding
Bugs
in
Help
Files
* Attack
25:
Finding
Bugs
in
Apps
* Attack
26:
Testing
Mobile
and
Embedded
Games
* Attack
27:
Attacking
App–Cloud
Dependencies
* Attack
28
Penetration
Attack
Test
* Attack
28.1
Penetration
Sub–Attacks:
Authentication
—
Password
Attack
* Attack
28.2
Sub–Attack
Fuzz
Test
* Attack
29:
Information
Theft—Stealing
Device
Data
* Attack
29.1
Sub
Attack
–Identity
Social
Engineering
* Attack
30:
Spoofing
Attacks
* Attack
30.1
Location
and/or
User
Profile
Spoof
Sub–Attack
* Attack
30.2
GPS
Spoof
Sub–Attack
* Attack
31:
Attacking
Viruses
on
the
Run
in
Factories
or
PLCs
* Attack
32:
Using
Combinatorial
Tests
* Attack
33:
Attacking
Functional
Bugs
Copyright
2015,
Jon
D.
Hagar
Mobile-‐Embedded
Taxonomies
from
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
15. 4/23/15
15
Attack
1:
Static
Code
Analysis
(testing)
* When
to
apply
this
attack?
*
After/during
coding
* What
faults
make
this
attack
successful?
* Many
* Example:
Issues
with
pointers
* Who
conducts
this
attack?
*
Developer,
tester,
independent
party
* Where
is
this
attack
conducted?
* Tool/test
lab
* How
to
determine
if
the
attack
exposes
failures?
* Review
warning
messages
and
find
true
bugs
* How
to
conduct
this
attack
* Obtain
and
run
tool
* Find
and
eliminate
false
positive
* Identify
and
address
real
bugs
* Repeat
as
code
evolves
* Single
unit/object
* Class/Group
* Component
* Full
system
29
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–”So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
Attack
2:
Finding
White–Box
Data
Computation
Bugs
* When
to
apply
this
attack?
*
After/during
coding
* What
faults
make
this
attack
successful?
* Mistakes
associated
with
data
* Example:
Wrong
value
of
Pi
* Who
conducts
this
attack?
*
Developer,
tester,
independent
party
* Where
is
this
attack
conducted?
* Development
Tool/test
lab
* How
to
determine
if
the
attack
exposes
failures?
* Structural-‐data
test
success
criteria
not
met
* How
to
conduct
this
attack
* Obtain
tool
* Determine
criteria
and
coverage
* Create
test
automation
with
specific
values
(really
a
programing
problem)
* NOT
NICE
NUMBERS
* Run
automated
test
cases
* Resolve
failures
* Peer
check
test
cases
* Repeat
as
code
evolves
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–”So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
16. 4/23/15
16
* When
to
apply
this
attack?
…when
your
app/device
has
a
user
* What
faults
make
this
attack
successful?
…devices
are
increasingly
complex
* Who
conducts
this
attack?
…see
chart
on
Roles
* Where
is
this
attack
conducted?
…throughout
lifecycle
and
in
user’s
environments
* How
to
determine
if
the
attack
exposes
failures?
* Unhappy
“users”
* Bugs
found
* See
sample
checklist
Attack
:
Testing
Usability
Mobile IoT Usability Tends to be “Poor”
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* Refine
checklist
to
context
scope
* Define
a
role
* Watch
what
is
happening
with
this
role
* Define
a
usage
(many
different
user
roles)
* Guided
explorations
or
ad
hoc
* Stress,
unusual
cases,
explore
options
* Capture
understanding,
risk,
observations,
etc.
* Checklist
(watch
for
confusion
of
the
tester)
* Run
Exploratory
Attack
(s)
* Learn
* Re-‐plan-‐design
* Watch
for
Bias
* Switch
testers
* Repeat
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
Usability
Attack
Pattern
17. 4/23/15
17
The
Good,
Bad,
and
Ugly
of
Mobile
App
Testing
Lots
of
room
for
Growth
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices”
33
How
to
be
Better
after
This
Section
Pick
One
or
Two
to
work
On
Cons:
Bad
and
Ugly
* Taxonomy
help
only
if
you
use
them
* Skill
improvement
* Knowledge
and
Skill
* Security
Testing
* Attack,
Attack,
Attack
Pro:
The
Good
* Better
and
Faster
* Functional
testing
* Test
strategy
and
planning
* Test
Attacks
* Tools
and
technique
maturing
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC –
“Software Test Attacks to Break Mobile and Embedded Devices
“
34 After Mobile
comes IoT
18. 4/23/15
18
* There
will
always
be
Good,
Bad,
and
Ugly
* Work
with
the
Good
* Work
to
over
come
the
Bad
* Change
the
Ugly
into
good
* Understanding
your
local
context
and
error
patterns
is
important
(one
size
does
NOT
fit
all)
* Attacks
are
patterns…you
must
still
THINK
and
tailor
Wrap
Up
of
this
Session
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
* James
Whittaker
(attacks)
* Elisabeth
Hendrickson
(simulations)
* Lee
Copeland
(techniques)
* Brian
Merrick
(testing)
* James
Bach
(exploratory
and
tours)
* Cem
Kaner
(test
thinking)
* Jean
Ann
Harrison
(her
thinking
and
help)
* Many
teachers
* Generations
past
and
future
* Books,
references,
and
so
on
Notes:
Thank
You
(ideas
used
from)
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
19. 4/23/15
19
* “Software
Test
Attacks
to
Break
Mobile
and
Embedded
Devices”
–
Jon
Hagar
* “How
to
Break
Software”
James
Whittaker,
2003
* And
his
other
“How
To
Break…”
books
* “A
Practitioner’s
Guide
to
Software
Test
Design”
Copeland,
2004
* “A
Practitioner’s
Handbook
for
Real-‐Time
Analysis”
Klein
et.
al.,
1993
* “Computer
Related
Risks”,
Neumann,
1995
* “Safeware:
System
Safety
and
Computers”,
Leveson,
1995
* Honorable
mentions:
* “Systems
Testing
with
an
Attitude”
Petschenik
2005
* “Software
System
Testing
and
Quality
Assurance”
Beizer,
1987
* “Testing
Computer
Software”
Kaner
et.
al.,
1988
* “Systematic
Software
Testing”
Craig
&
Jaskiel,
2001
* “Managing
the
Testing
Process”
Black,
2002
Book/Notes
List
(my
favorites)
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
“So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices”
• www.stickyminds.com
–
Collection
of
test
info
• www.embedded.com
–
info
on
attacks
* www.sqaforums.com
-‐
Mobile
Devices,
Mobile
Apps
-‐
Embedded
Systems
Testing
forum
• Association
of
Software
Testing
– BBST
Classes
http://www.testingeducation.org/BBST/
• Your
favorite
search
engine
More
Resources
Copyright
2015,
Jon
D.
Hagar
Grand
So9ware
Tes>ng,
LLC
–
So9ware
Test
ACacks
to
Break
Mobile
and
Embedded
Devices