Darshan Desai - Virtual Test Labs,The Next Frontier - EuroSTAR 2010TEST Huddle
EuroSTAR Software Testing Conference 2010 presentation on Virtual Test Labs,The Next Frontier by Darshan Desai.
See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
An example of the integration of Standard Program Management Workflows with Agile Methodology using MicroSoft's Visual Studios. (Along with some personal observations)
Darshan Desai - Virtual Test Labs,The Next Frontier - EuroSTAR 2010TEST Huddle
EuroSTAR Software Testing Conference 2010 presentation on Virtual Test Labs,The Next Frontier by Darshan Desai.
See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
An example of the integration of Standard Program Management Workflows with Agile Methodology using MicroSoft's Visual Studios. (Along with some personal observations)
A quick paced introduction to "Test Driven Development" (TDD) in an agile environment. The TDD philosophy states that you should develop your tests and then write code to make your tests pass and satisfy user requirements.
DevOps Tactical Adoption Theory tries to make the transition process as smooth as possible. It hypothesis each step towards DevOps maturity should bring a visible business value empowering management and team commitment for the next step. The innovative idea here, it is not required to add the tools/processes to stack from sequential beginning to end, but seeking benefit.
The reason behind the theory is to encourage practitioners to apply each step one-by-one and then having the benefits in projects. Consequently, each step is tested in terms of utility and proved method validity for the further steps. In contrast to previous adoption models, our model indicates concrete activities rather than general statements.
Theory built on the claim that many DevOps transition projects considered problematic, impractical or even unsuccessful causing concept to become a goal more than a technique. Basically, theory consists of different areas of interest describing various actions on a schema.
In the session, it is planned to demonstrate “DevOps Tactical Adoption Theory” with focus on Delivery Pipeline/Testing Practices sectioned "Continuous Testing in DevOps".
The Leaders Guide to Getting Started with Automated TestingJames Briers
Conventional testing is yesterday’s news, is required but needs the same overhaul that has happened in development. It needs to be a slicker operation that really identifies the risk associated with release and protects the business from serious system failure. The only way to achieve this is to remove the humans, they are prone to error, take a long time, cost a lot of money and don’t always do what they are told.
Automation needs to be adopted as a total process, not a bit part player. Historically automation has focussed on the User Interface, which can be a start, but is often woefully lacking. Implementing an Automation Eco-System, sees automation drive through to the interface or service layer, enabling far higher reuse of automated scripts, encompasses the environment and the test data within it’s strategy, providing a robust, repeatable and reusable asset.
Don’t just automate the obvious. Automation is not a black box testing technique. Rather it is mirroring the development and building an exercise schedule for the code. Take your testing to the next level and realise the real benefits of a modern Automation Eco-system.
Slides from "Taking an Holistic Approach to Product Quality"Peter Marshall
This is the base material used during a half day workshop at expoQA 17 June 2019. Peter Marshall runs over the necessary technical, organisational, and improvement practices required to deliver high quality software. Deep dives into Continuous delivery, devops, organisational structures, agile and digital transformation.
What skills can software testers provide the best testing services?NexSoftsys
Although software testers have the versatility, software testers must have both technical and non-technical skills to provide the best testing services as this skill helps the tester to fight against any circumstances.
Common Challenges & Best Practices for TDD on iOSDerek Lee Boire
Presented at the June 2016 Tokyo iOS Meetup on 6/11/2016, in this presentation I give an overview of test driven development and how it can be used when developing for iOS / Swift.
TDD is an enormous topic so this really just scratches the surface to give an expansive glimpse into the journey that TDD truly is. Those unfamiliar with TDD will hopefully have a desire to learn more about it while those familiar will hopefully learn something new.
Its a simple presentation show you new concepts of software development like waterfall, agile and scrum also in testing like test driven development "TDD"
A quick paced introduction to "Test Driven Development" (TDD) in an agile environment. The TDD philosophy states that you should develop your tests and then write code to make your tests pass and satisfy user requirements.
DevOps Tactical Adoption Theory tries to make the transition process as smooth as possible. It hypothesis each step towards DevOps maturity should bring a visible business value empowering management and team commitment for the next step. The innovative idea here, it is not required to add the tools/processes to stack from sequential beginning to end, but seeking benefit.
The reason behind the theory is to encourage practitioners to apply each step one-by-one and then having the benefits in projects. Consequently, each step is tested in terms of utility and proved method validity for the further steps. In contrast to previous adoption models, our model indicates concrete activities rather than general statements.
Theory built on the claim that many DevOps transition projects considered problematic, impractical or even unsuccessful causing concept to become a goal more than a technique. Basically, theory consists of different areas of interest describing various actions on a schema.
In the session, it is planned to demonstrate “DevOps Tactical Adoption Theory” with focus on Delivery Pipeline/Testing Practices sectioned "Continuous Testing in DevOps".
The Leaders Guide to Getting Started with Automated TestingJames Briers
Conventional testing is yesterday’s news, is required but needs the same overhaul that has happened in development. It needs to be a slicker operation that really identifies the risk associated with release and protects the business from serious system failure. The only way to achieve this is to remove the humans, they are prone to error, take a long time, cost a lot of money and don’t always do what they are told.
Automation needs to be adopted as a total process, not a bit part player. Historically automation has focussed on the User Interface, which can be a start, but is often woefully lacking. Implementing an Automation Eco-System, sees automation drive through to the interface or service layer, enabling far higher reuse of automated scripts, encompasses the environment and the test data within it’s strategy, providing a robust, repeatable and reusable asset.
Don’t just automate the obvious. Automation is not a black box testing technique. Rather it is mirroring the development and building an exercise schedule for the code. Take your testing to the next level and realise the real benefits of a modern Automation Eco-system.
Slides from "Taking an Holistic Approach to Product Quality"Peter Marshall
This is the base material used during a half day workshop at expoQA 17 June 2019. Peter Marshall runs over the necessary technical, organisational, and improvement practices required to deliver high quality software. Deep dives into Continuous delivery, devops, organisational structures, agile and digital transformation.
What skills can software testers provide the best testing services?NexSoftsys
Although software testers have the versatility, software testers must have both technical and non-technical skills to provide the best testing services as this skill helps the tester to fight against any circumstances.
Common Challenges & Best Practices for TDD on iOSDerek Lee Boire
Presented at the June 2016 Tokyo iOS Meetup on 6/11/2016, in this presentation I give an overview of test driven development and how it can be used when developing for iOS / Swift.
TDD is an enormous topic so this really just scratches the surface to give an expansive glimpse into the journey that TDD truly is. Those unfamiliar with TDD will hopefully have a desire to learn more about it while those familiar will hopefully learn something new.
Its a simple presentation show you new concepts of software development like waterfall, agile and scrum also in testing like test driven development "TDD"
Most of the people might say that software test engineers do not write code. Testers normally need completely different skill set which could be a mix of Java, C, Ruby, and Python.
That is not all you require to be a successful tester. A tester requires having a good knowledge of the software manuals and automation tools.
Depending on the complexity of a project, a software testing engineer may write more complicated code than the developer.
DevOps - Continuous Integration, Continuous Delivery - let's talkD Z
Brief but detailed insight about what to expect and what not from DevOps engineer if an organization is willing to hire one.
At the same time detailed insight about someone who is willing to dive into DevOps as a career option.
When system gradually become bigger or expands to match with business needs or policy, a good developer can’t rely only on manual testing because it will consume more of our resources. Therefor developer need to ensure that our code is good in terms of quality. So, it expected use different kinds of automated tests such as unit test and integration test.
Here we want to share how Blibli implement integration test in UI side, deep down on visual regression, and any usecase that we encounter thus far.
What will you learn?
How Integration Test in UI Works
The Technologies Behind Integration UI
Best practice and strategies!
Continuous Integration testing based on Selenium and HudsonZbyszek Mockun
Open source tools in continuous integration environment, article describe who to use Selenium and Hudson to achieve CI testing.
Article has been written for Testing Experience magazine
Open Source tools in Continuous Integration environment (case study for agil...suwalki24.pl
Article wrote for Testing Experience magazine, publicated in December 2010.
The aim of this article is to share our experience in building and
managing Continuous Integration environments on the basis of
open-source tools like Hudson and Selenium. In this article we
will concentrate on testing purposes, suggest just few improvements
and describe our experience with using open-source tools.
The main idea is to present how to use automated tests reasonably
by minimizing the time spent on them while optimizing the
benefits that automated tests give us.
WinOps Conf 2016 - Matteo Emili - Development and QA Dilemmas in DevOpsWinOps Conf
The quick rise of Continuous Delivery in the enterprise means that common problems are often approached the other way round. Concepts like Feature Flags and Testing In Production caused several headaches to developers and QA engineers, especially where they have a wealth of experience about traditional development.
There are some challenges and approaches which are very common, and they still scare newcomers. Let's have a look at a few of these, with the most common solutions.
CdCon + GitOpsCon 2023 in Vancouver Canada. Slidedeck for the talk on Scaling Software Delivery: A framework for developer enablement through devRel and outreach.
This presentation talks about the concepts of continuous Integration with TFS as an example platform on whihc you can implement this concept but it can apply to open source platforms as well
This is a talk I gave on patterns and antipatterns of SOA, based on my understandings and practices and inspired by Ron Jacobs famous webcast by the same name.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
2. Software Developers, Development Leads 1 2 3 4 5 6 7 WHAT ARE THEY LIKE? Software developers are problem solvers, tinkerers, artists and engineers all rolled up in one. They are continuously building something for the first time. No matter what they are building there is some element of it—or all of it—that is new, that they have never done before. Developers are analytical and believe, to a certain extent, that there is an art form to software development. WHY ARE THEY HERE? Developers attend this session because they want to learn how to make their work life better. They have to work with testers every day, and anything that improves how they receive a bug, and how they can fix it and never see it again is an improvement. WHAT KEEPS THEM UP AT NIGHT? Bugs. Software developers are a proud people. They don’t like to find out that they introduced a bug into the code. The goal is bug-free code. Obviously this is impossible, because some bugs are the result of bad requirements or changes in scope. If a bug is found it can be massively disruptive to what they believe is a more creative process—writing new code. No software developer wants to spend time fixing bugs; time spent on bugs is like paying taxes. HOW CAN YOU SOLVE THEIR PROBLEM? Since the majority of time spent fixing bugs is in (a) reproducing them and (b) finding the root cause, and not in actually coding the fix, you need to give them tools that enable them to do these two things—reproduce the bug and find the root cause faster. WHAT DO YOU WANT THEM TO DO? Install Visual Studio 2010 Ultimate and start using IntelliTrace in their local debugging sessions. Work with their QA or test team to get them to begin using Visual Studio Test Professional 2010 so that bugs that are filed are more actionable. HOW MIGHT THEY RESIST? If the new tools and capabilities appear to have a lot of setup overhead they may quickly reject it as something they don’t have time for. Additionally, the need to get another team to use a new tool—the QA team—may appear to be a non-starter. WHAT’S THE BEST WAY TO REACH THEM? Demonstration. Show them what IntelliTrace does and how it can help them. Show them the entire workflow—from discovering a defect and filing a bug, to quickly locating the cause, to automating the functional test to identify if the bug ever returns. Target Audience
3. S2S: Situation to Solution 1 2 3 4 SITUATION Developers and testers frequently work in separate but connected teams. Each role is managed separately, perhaps only connecting at the Director of Development or Vice President of IT level. The development team is responsible for implementing a set of requirements, while the QA team is responsible for validating the requirements were met; however there is often little direct interaction between the developers and the testers and often a skill and pay gap that divides the two roles. In 90% of IT organizations it’s obvious the developers are more highly valued and usually more highly skilled. PROBLEM Since the developers and testers are managed separately, they are frequently given different direction. There is nothing forcing the two roles to interact, and likely the division of reporting structure creates a greater boundary between the two roles. The level of collaboration is very low, and too often the level of blame is too high. IMPLICATION There is erratic code quality, dependent on the individual developer with a high escaped bug rate, frequent regressions and performance problems. As a result there is a high-level of mistrust between testers & developers. When bugs are found, they are often difficult to reproduce due to poor bug documentation. In addition, the high complexity of the typical enterprise environment means developers assume the testers incorrectly configured the software or made some other mistake that explains away the bug. SOLUTION Visual Studio 2010 introduces a new set of tools and capabilities that enable testers to easily create actionable bugs when a defect is found. The level of data collected and attached to the bug enables developers to quickly take action and locate the root cause of the bug. IntelliTrace™ enables the developer to step through the code execution from the test environment as if it was currently running. The data collectors can also capture environment and configuration settings, and developers can see the actual execution of the program using Lab Management 2010. Once the developer fixes the bug they are able to create a Coded UI Test which can be added to an automated regression test suite, ensuring that if the bug is ever reintroduced, it is quickly discovered. Target Audience
4. What are your most common defect resolutions? What If… Today Fixed … Won’t Fix Not a Bug Cannot Reproduce Cannot Reproduce Not a Bug Won’t Fix … Fixed
5. Why Can’t a Bug Be Fixed? Poorly documentedsteps to reproduce Lack of visibility into the tester’s actions Environment differences
19. Microsoft Approach in VSTS 2010 70% of testing happens here majority of test tools target here Black Box Testing White Box Testing API Testing
20. Microsoft Test Capabilities Load Test Web Test Coded UI Test Microsoft Test Runner Unit Testing Test Case Management Lab Management Data Collectors (Historical Debugging, Test Impact, Metrics) Team Foundation Server Reporting
21. VSTS 2010 Test Elements Test Case Management Compose test cases, plans and suites. Integration with TFS
22. Manual Test Runner Standalone tool Enables execution of test cases Collects system data and event log Captures still images and video Enables filing a new bug
28. “…the realization came over me with full force that a good part of the remainder of my life was going to be spent in finding errors in my own programs.” - Maurice Wilkes,Memoirs of a Computer Pioneer, MIT Press
36. Createricher actionable bugs. Environment Snapshots. Quickly debug issues in the same environment. Connect to the test environment snapshot with network isolation. Automate end-to-end workflow Out of the box build-deploy-test automation workflow
37. *For complete list visit Microsoft Visual Studio gallery Visual Studio Test Partners
38. What will be your most common defect resolution? What If… Today Fixed … Won’t Fix Not a Bug Cannot Reproduce Cannot Reproduce Not a Bug Won’t Fix … Fixed
39. 3 Create actionable bugs Quickly discover theroot cause of any bug Automate the discovery of a regression
Walk-in slide. Advance to the next slide to begin the presentation.
HIDDEN SLIDE FOR SPEAKER PREP ONLYAudience Definition:Software Developers, Development LeadsWhat are they like? Software developers are problem solvers, tinkerers, artists and engineers all rolled up in one. They are continuously building something for the first time. No matter what they are building there is some element of it—or all of it—that is new, that they have never done before. Developers are analytical and believe, to a certain extent, that there is an art form to software development.Why are they here?Developers attend this session because they want to learn how to make their work life better. They have to work with testers every day, and anything that improves how they receive a bug, and how they can fix it and never see it again is an improvement.What keeps them up at night? Bugs. Software developers are a proud people. They don’t like to find out that they introduced a bug into the code. The goal is bug-free code. Obviously this is impossible, because some bugs are the result of bad requirements or changes in scope. If a bug is found it can be massively disruptive to what they believe is a more creative process—writing new code. No software developer wants to spend time fixing bugs; time spent on bugs is like paying taxes.How can you solve their problem?Since the majority of time spent fixing bugs is in (a) reproducing them and (b) finding the root cause, and not in actually coding the fix, you need to give them tools that enable them to do these two things—reproduce the bug and find the root cause faster. What do you want them to do? Install Visual Studio 2010 Ultimate and start using IntelliTrace in their local debugging sessions. Work with their QA or test team to get them to begin using Visual Studio Test Professional 2010 so that bugs that are filed are more actionable.How can you best reach them? Demonstration. Show them what IntelliTrace does and how it can help them. Show them the entire workflow—from discovering a defect and filing a bug, to quickly locating the cause, to automating the functional test to identify if the bug ever returns.How might they resist?If the new tools and capabilities appear to have a lot of setup overhead they may quickly reject it as something they don’t have time for. Additionally, the need to get another team to use a new tool—the QA team—may appear to be a non-starter.
HIDDEN SLIDE FOR SPEAKER PREP ONLYSituationDevelopers and testers frequently work in separate but connected teams. Each role is managed separately, perhaps only connecting at the Director of Development or Vice President of IT level. The development team is responsible for implementing a set of requirements, while the QA team is responsible for validating the requirements were met; however there is often little direct interaction between the developers and the testers and often a skill and pay gap that divides the two roles. In 90% of IT organizations it’s obvious the developers are more highly valued and usually more highly skilled.ProblemSince the developers and testers are managed separately, they are frequently given different direction. There is nothing forcing the two roles to interact, and likely the division of reporting structure creates a greater boundary between the two roles. The level of collaboration is very low, and too often the level of blame is too high.ImplicationThere is erratic code quality, dependent on the individual developer with a high escaped bug rate, frequent regressions and performance problems. As a result there is a high-level of mistrust between testers & developers. When bugs are found, they are often difficult to reproduce due to poor bug documentation. In some environments, coding mistakes are caught by the developers before handing off to testing, but consistency still depends on individual discipline. Since developers don’t understand the testing role there’s often a lack of trust. In addition, the high complexity of the typical enterprise environment means developers assume the testers incorrectly configured the software or made some other mistake that explains away the bug.SolutionVisual Studio 2010 introduces a new set of tools and capabilities that enable testers to easily create rich, actionable bugs when a defect is found. The level of data collected and attached to the bug enables developers to quickly take action and locate the root cause of the bug. IntelliTrace™ enables the developer to step through the code execution from the test environment as if it was currently running—similar to a digital video recorder (DVR). The data collectors can also capture environment and configuration settings, and developers can see the actual execution of the program using Lab Management 2010. Once the developer resolves the bug they are able to create a Coded UI Test which can be added to an automated regression test suite, ensuring that if the bug is ever reintroduced, it is quickly discovered.
What is the most common bug resolution used right now by your development team? Is it ‘Cannot Reproduce,’ or ‘Not a Bug,’ or ‘Won’t Fix?’ If you stack ranked all of the bug resolution by how commonly they were used, where would ‘Fixed’ appear? Imagine if you could move ‘Fixed’ to the top of the list? How would your software development process change if resolutions like “Not a Bug” and “Won’t Fix” became rarities? What if “Cannot Reproduce” became obsolete?Welcome. My name is [SPEAKERNAME] and I am a [SPEAKER TITLE].
The first step in resolving a bug is reproducing it reliably. Once the bug is reproduced, a developer can use tools like the Visual Studio debugger to step through the code execution, watching variable states and execution flow to identify the cause of the bug. Unfortunately, all too often bugs are difficult to reproduce. If you cannot reproduce a bug, how can you be expected to fix the root cause? The reasons for not being able to reproduce the bug could be varied. Some bugs are triggered by inputs that are difficult to recreate, or poorly documented in the bug report; often a tester will perform actions or steps that weren’t explicit in the test case. Without knowing what the tester did it becomes very difficult to reproduce the bug. In some cases, bugs are the result of the speed at which the tester performed the actions—a reproduction that is often very hard to identify until you see the tester in action. In some cases the bug is the result of environmental differences between the development environment and the test environment—which could include different operating systems, service packs, different versions of required runtimes, and even the version of the software being tested.
All of these things lead to a lot of wasted effort on your team, and create a divide—a chasm—between the developers and the testers. The reality is that you share a common goal. Both developers and testers want to ship high-quality software that meets the expectations of the customers and stakeholders. How can you reduce the waste of bug Ping-Pong, and increase the level of collaboration between developers and testers. Sure you can engage in some crazy team building outing where you have the developers have to fall backwards and the testers have to catch them…but is that really going to help?
Visual Studio 2010 is going to help. Today you are going to see three (3) ways that developers and testers can use the tools and capabilities of Visual Studio 2010 to:Create rich, actionable bugs that enable any developer to learn enough about the bug to take immediate action.Replay the code execution from a test run to reproduce a bug and quickly find the root cause, even if the development environment is different than the test environment.Create an automated functional test that will quickly discover the bug if it reemerges.
In order to move ‘Fixed’ to the top of the resolution list you have to start by looking at how software is tested and how defects are reported. In most environments today the test or QA team uses a set of tools that enable them to do their job, but which don’t integrate into the tools used by the development team. This is the first fracture in the system and a contributor to costly bug ping-pong. In an ideal world the development team and the test team would each use tools that integrated with one another and enables easily passing information back and forth.
Microsoft Test Manager 2010 is for testers what Visual Studio is for developers. That is to say, where Visual Studio is an IDE – an integrated development environment, Test Manager is an ITE – an integrated test environment. This is the interface that a tester will use to create test cases, organize test plans, track test results, and file bugs when defects are found. Test Manager is integrated with Team Foundation Server, and is designed to improve the productivity of testers. While I am not going to do a deep-dive of all that Test Manager can do, it is important to understand how it integrates with the Visual Studio Agents to make the most of test case execution, and ensure that when a tester files a bug, it is actionable with very little work on the tester’s part.
Visual Studio Agents 2010 are set of technology—included with Visual Studio 2010—that enable tasks to be performed on behalf of one or more users. In the context of developer-tester collaboration, the most important Agent is the Test Agent, which includes a set of diagnostic data adapters, or DDA’s. There are several DDA’s which each server a different purpose. During test execution DDA’s can be used to:Collect a video capture of the tester’s machine.Collect the steps performed during the test execution.Collect system information from any or all of the machines involved in the test execution.Collect an action recording—a recording of the keyboard strokes and mouse clicks performed by the tester.Collect an IntelliTrace file which can be later used to replay the events that occurred during the test run.DDA’s can be used to impact the system under test (e.g. network emulation), and are extensible as well; so if there’s a piece of information which you find would be helpful for your developers to have from test runs, you can easily write your own DDA which collects that data for inclusion with the actionable bug.While this is not an exhaustive list of the DDA’s and their capabilities, these are the ones that will provide the most value when filing a bug; these are the DDA’s that will automate the collection of information that will turn a normal, somewhat helpful bug into a rich, actionable bug.
To define an actionable bug I’d like to examine a typical bug. When a tester files a bug they record information they think is important or will help whomever the bug is assigned to understand it better. The problem starts right here. You might say that Developers are from Mars, testers are from Venus. That is to say, the roles of these individuals, and what is important to them and the language they speak is so different that they may as well be from different planets. How can anyone expect a tester to accurately provide the information that a developer needs to do their job? As a result a typical bug fails to contain the necessary detail for a developer to understand the bug, find the root cause and fix it.
An actionable bug, on the other hand, is a bug that is full of data that will aid the developer in understanding the bug, and enable them to take immediate action toward fixing the bug. An Actionable bug contains enough information that whomever it is assigned to is not blocked from taking action to fix the bug.Let me demonstrate how to use Microsoft Test Manager 2010 to execute a test, find a defect and file an actionable bug.
See DemoMate file: CreateActionableBug.demo 10-mins
We have to understand the types of testers and who we are targeting with these tools.We’ve had specialist testing tools in the past and have improved on them.However the big leap with Test Elements is the support for the generalist tester.
+In General, Manual Testing is supported by the Microsoft Test and Lab Manager that is included in Visual Studio Test Elements 2010 and Specialist testing is supported by Visual Studio.
As a developer, the most important thing you need to see in a bug is information, and the more the better. An actionable bug is tremendously helpful. No longer do you have to hope that the tester properly documented the steps to reproduce the bug, or the system information—those things are collected automatically and added to the bug. No longer do you have to walk down the hall and ask the tester to run the test case again so that you can watch and see what happened with your own eyes—the screen capture and video capture do that for you. But that still may not be enough to do the most important thing—find the root cause of the bug quickly.
To quote - Maurice Wilkes, the builder and designer of the EDSAC, the first computer with an internally stored program,“…the realization came over me with full force that a good part of the remainder of my life was going to be spent in finding errors in my own programs.”Even with all the information provided by an actionable bug, unearthing that nugget of information—the root cause of a defect—may be the most difficult thing to do. Imagine if you could go back in time and watch the code execution happen as the bug was discovered. Think of how much time you would save if you could rewind and fast forward through the test execution.
IntelliTrace is a new capability in Visual Studio 2010 Ultimate that enables developers and testers to collaborate by sharing a historical perspective. When a test case is executed using Test Manager 2010, you have the option of collecting an IntelliTrace file using one of the DDAs. The IntelliTrace DDA collects information about the events that executed during the test run. The level of event data collection is adjustable via DDA configuration, and may be fairly coarse grain collection, or much more fine grain collection. When a tester files a new bug, the IntelliTrace file is automatically attached and available to the developer when they open the bug in Visual Studio. This enables the developer to see the exact code execution that occurred during the test run.
You could think of this as being like a DVR—a digital video recorder—for developers. When you attach the Visual Studio debugger to an IntelliTrace file you are able to move through the code execution similarly to a DVR—move forward, backward, pause, stop, rewind, etc. Similar to a DVR, you can’t change the code execution that occurred—after all, it already happened, and you can’t change the past—but you are able to examine the code execution more easily. If you miss something, simply “rewind” back to the event to see what happened. This enables you to more easily and more quickly identify the issues that caused the bug.
Demo: IntelliTrace.demo 10-mins
Functional testing refers to the testing of a specific action of function of the code. A function test is typically designed to identify if a particular requirements has been fulfilled, if a feature works, or if a user story can be completed without error.
Currently about 70% of functional testing is done manually—that is, a software tester follows a script to execute a series of steps to verify the outcome of a test. Microsoft Test Manager 2010 is an example of a tool that someone doing manual testing would use to be more productive. Imagine if you could still get the benefits of functional testing without the overhead and resource cost of doing in manually.Consider a regression test—a test designed to identify if a bug fix begins to fail. With a regression test (and even other functional tests) you are repeatedly testing something that was known to work at one point. These too are typically done manually. This is a massive resource hit. Software testers spend countless hours testing functionality that works, solely to ensure it still works. Imagine how much more productive they could be if they could focus their efforts on creating and running new test cases that covered parts of the system not currently tested instead of spending their time testing the same thing, over and over again.
Comp Photo: http://www.istockphoto.com/stock-photo-1629680-the-cyber-robot-hand.phpA new capability in Visual Studio 2010 Premium is the ability to create a Coded UI Test—a test that automates the testing of the user interface of an application. Coded UI Tests work similarly to unit tests, in that they are written in a programming language—such as Visual Basic or C#—perform some action or series of actions, validate one or more assertions, and are easily repeatable with predictable results. The difference from a unit test is that, while a unit test is intended to test an individual unit of code, such as a single method call, the Coded UI Test is intended for automating a functional test, including the validation of the outcomes within the test. Visual Studio makes it easy for you to create Coded UI Tests be enabling you to use an action recorder to capture your actions—keyboard strokes and mouse clicks—or to simply import an existing action recording created with Test Manager 2010 as part of a test case execution. In either case, the actions recorded are converted into code—Visual Basic or C#. This creates a great collaboration point for developers and testers. Consider the following workflow:A developer writes code to fulfill a requirement.A tester uses Test Manager 2010 to test the implementation and discovers a defect.The tester files an actionable bug.The developer fixes the bug.The tester verifies the fix by successfully completing the test case.The developer (or a test automation engineer) imports the Action Recording from the last test run and creates a Coded UI Test.Every time a new build is deployed to the test environment, all of the Coded UI Tests are executed to ensure there are no regressions.By automating functional tests, including regression tests, you are able to free up testing resources to focus on creating and executing new tests to ensure complete test coverage of your software. This is like adding an automated tester to you team. One that is never late for work, never complains, works at incredible speed and who will take on additional testing work as fast as you can create the Coded UI Tests.
Demo CodedUITest.demo 10-mins
Visual Studio Lab Management 2010 takes developer and tester collaboration to the next level. It offers an out-of-the-box solution fully integrated with Visual Studio. It leverages virtualization for application lifecycle management and enables you to Create even richer bugs. With a single click, you can take an environment snapshot and attach a link to it in the bug. Developers can quickly debug issues on real testing environment by connecting to environment snapshot. In case the QA team is still holding on to the test environment, network isolation allows you to create multiple copies of environments without having to reconfigure machines or your application.Once you have created automated test cases, use lab management to automate the end-to-end workflow that builds your app, provisions an environment for application deployment in minutes instead of hours or days, deploys your application through a distributed workflow based on Team Foundation Build and runs automated test suites. The test results are stored in TFS for further analysis and reporting.Furthermore, teams spend a lot of time on tasks that don’t contribute to the over all quality of the application or project success. Areas where teams maybe wasting precious cycles is around setting up and tearing down of environments. Our customers tell us that they spend up to 40% of the test cycle time setting up machines. Lab Management empowers the team members to quickly create complex multi-machine environments. Since Lab Management leverages virtualization, you get all the benefits of virtualization, such as, hardware consolidation which lowers your total cost of ownership.
At the beginning of this session I asked what your most common defect resolution is currently. More importantly I asked you to consider what the impact to your business and your software projects would be if you could move “Fixed” to the top of the list, and resolutions like ”Cannot Reproduce” to the bottom of the list.
Today you saw three (3) ways that developers and testers can collaborate to deliver high quality software more easily, and ensure “Fixed” is far and away the most common defect resolution by using the capabilities of Visual Studio 2010.Visual Studio 2010 enables you to:Create rich, actionable bugs that enable any developer to learn enough about the bug to take immediate action.Replay the code execution from a test run to reproduce a bug and quickly find the root cause, even if the development environment is different than the test environment.Create an automated functional test that will quickly discover the bug if it reemerges.