This document discusses applying automated application lifecycle management (ALM) practices to Azure cloud development. It outlines 5 scenarios with increasing levels of automation: 1) developers only, 2) adding manual testing, 3) adding automated deployment to a staging environment during builds, 4) adding automated testing during builds, and 5) fully automated testing, building, deployment, and acceptance testing integrated with operations. The document demonstrates configuring automated deployment and testing with Microsoft tools like Visual Studio, Test Manager, and PowerShell for Azure. While increasing automation brings benefits, it also requires more complex build workflows and management of certificates and configurations.
Agile Open Source Performance Test Workshop for Developers, Testers, IT OpsClever Moe
Training For Selenium, soapUI, Sahi, TestMaker Performance Testing. Slide deck from the free Webinar titled "Technical Training On The Agile Open Source Way To Load Test, Scalability Test, and Stress Test." Learn the Agile Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
Infinum Android Talks #17 - Testing your Android applications by Ivan KustInfinum
van will show you how to setup unit tests using Robolectric, how to test user interactions using Espresso and provide some tips and tricks for stress-free testing.
Drupal & Continous Integration - SF State Study CaseEmanuele Quinto
HigherEd Drupal Summit @ BADCamp 2011 (http://2011.badcamp.net/higher-education-drupal-summit)
Cal State San Francisco will talk about how they implemented their drupal development cycle process based on continuous integration and QuickBuild.
Agile Open Source Performance Test Workshop for Developers, Testers, IT OpsClever Moe
Training For Selenium, soapUI, Sahi, TestMaker Performance Testing. Slide deck from the free Webinar titled "Technical Training On The Agile Open Source Way To Load Test, Scalability Test, and Stress Test." Learn the Agile Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
Infinum Android Talks #17 - Testing your Android applications by Ivan KustInfinum
van will show you how to setup unit tests using Robolectric, how to test user interactions using Espresso and provide some tips and tricks for stress-free testing.
Drupal & Continous Integration - SF State Study CaseEmanuele Quinto
HigherEd Drupal Summit @ BADCamp 2011 (http://2011.badcamp.net/higher-education-drupal-summit)
Cal State San Francisco will talk about how they implemented their drupal development cycle process based on continuous integration and QuickBuild.
slide deck from my tccc10 presentation. please use the URL's and references for the source code and other technologies which are discussed but not covered.
Agile Open Source Performance Testing Workshop for Business ManagersClever Moe
Use Selenium, soapUI, Sahi, TestMaker Performance Testing In Your Organization. These are slides from a free Webinar titled "The Agile Open Source Way To Load Test, Scalability Test, and Stress Test In Your Organization". Your organization may have adopted Agile Software Development Methodology and forgot about load and performance testing! Learn the Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
This is a draft of a presentation for a course on Visual Studio 2010 Unit Testing, I've uploaded mainly because I tried to create a Metro Style presentation, so if everyone like it, he can use as base for own presentation.
Turbocharge Your Automation Framework to Shorten Regression Execution TimeJosiah Renaudin
Asad Faruqui and Moni Mau say that their old regression automation used to take three-to-four days of execution time as they ran against different browser versions, locales, and currencies. They wanted to make the automation framework more efficient so features could go to market faster. Symantec explored Selenium Grid which could reduce the run time by offering some parallelism, but Asad and Moni wanted even more efficiency. They explain how they created both a framework in which they write modular test classes and an engine that can read test cases from a test repository (XML, database, or XLS sheet) and distribute the execution of these tests on virtualized images based on user requirements. If they don't have enough virtual machines, this creates a queue of outstanding jobs. The engine keeps track of results from execution of these classes, reports to the user when complete, and provides substantial error handling. Asad and Moni present this framework so you can learn from their approach and apply this framework to shorten execution times on regression automation.
AD208 - End to End Quality Processes for Top Notch XPages Appsbeglee
When it comes to getting XPages apps ready for production there's a lot to know. Do you have a web test automation suite in place to drive your apps through the browser and guarantee quality? Do you have a JUnit test framework to exercise your XPages components? Is your application accessible to the latest international standards? This session, direct from the XPages engineering team, shows how to automate your app dev processes using the very latest tools and standards so that your products are ahead of the game.
Coded UI - Test automation Practices from the FieldClemens Reijnen
CodedUI tests within Visual Studio makes it easy for developers together with tester to create, fully-automated, functional user interface tests. These tests alert the team in an, easy to execute, automated way about regressions. CodedUI tests are easy to create for different UI technologies. But, all kinds of test automation needs an investment. To get a good return on this test automation investment you need to create CodedUI tests in a robust manner which can sustain changes to your application over time.
In this session you will see how maintainable CodedUI tests can be created and how the test infrastructure needs to be configured for efficient execution.
slide deck from my tccc10 presentation. please use the URL's and references for the source code and other technologies which are discussed but not covered.
Agile Open Source Performance Testing Workshop for Business ManagersClever Moe
Use Selenium, soapUI, Sahi, TestMaker Performance Testing In Your Organization. These are slides from a free Webinar titled "The Agile Open Source Way To Load Test, Scalability Test, and Stress Test In Your Organization". Your organization may have adopted Agile Software Development Methodology and forgot about load and performance testing! Learn the Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
This is a draft of a presentation for a course on Visual Studio 2010 Unit Testing, I've uploaded mainly because I tried to create a Metro Style presentation, so if everyone like it, he can use as base for own presentation.
Turbocharge Your Automation Framework to Shorten Regression Execution TimeJosiah Renaudin
Asad Faruqui and Moni Mau say that their old regression automation used to take three-to-four days of execution time as they ran against different browser versions, locales, and currencies. They wanted to make the automation framework more efficient so features could go to market faster. Symantec explored Selenium Grid which could reduce the run time by offering some parallelism, but Asad and Moni wanted even more efficiency. They explain how they created both a framework in which they write modular test classes and an engine that can read test cases from a test repository (XML, database, or XLS sheet) and distribute the execution of these tests on virtualized images based on user requirements. If they don't have enough virtual machines, this creates a queue of outstanding jobs. The engine keeps track of results from execution of these classes, reports to the user when complete, and provides substantial error handling. Asad and Moni present this framework so you can learn from their approach and apply this framework to shorten execution times on regression automation.
AD208 - End to End Quality Processes for Top Notch XPages Appsbeglee
When it comes to getting XPages apps ready for production there's a lot to know. Do you have a web test automation suite in place to drive your apps through the browser and guarantee quality? Do you have a JUnit test framework to exercise your XPages components? Is your application accessible to the latest international standards? This session, direct from the XPages engineering team, shows how to automate your app dev processes using the very latest tools and standards so that your products are ahead of the game.
Coded UI - Test automation Practices from the FieldClemens Reijnen
CodedUI tests within Visual Studio makes it easy for developers together with tester to create, fully-automated, functional user interface tests. These tests alert the team in an, easy to execute, automated way about regressions. CodedUI tests are easy to create for different UI technologies. But, all kinds of test automation needs an investment. To get a good return on this test automation investment you need to create CodedUI tests in a robust manner which can sustain changes to your application over time.
In this session you will see how maintainable CodedUI tests can be created and how the test infrastructure needs to be configured for efficient execution.
Writing code is fun, but deploying to production is not. Production releases are scary events that last all weekend, and you find yourself worrying about how it will go. Did we miss a configuration file? Is the database schema the same as the one in the test environment? Does the last minute hot fix we just applied break any other features? Did I forget to include an installation instruction for the system administrators?
Continuous Delivery is a collection of principles and practices aimed at addressing the problems teams typically face when releasing changes to production. By applying rigorous automation, testing and configuration management, teams are able to confidently and consistently deploy changes from version control to production without fear.
In this talk, Mike McGarr will provide listeners with an introduction into the world of Continuous Delivery. After an introduction into the concepts and principles of Continuous Delivery, he will discuss many of the techniques for implementing Continuous Delivery and recommend some tools that can be used on your development project.
Watch the recorded version of this Webinar here:
Curious about Continuous Integration? Tune in!
Continuous Integration (CI), which is a big part of continuous delivery, is the concept of continuously building and testing software using an automated process. We have learned that utilizing CI could help us catch bugs earlier, enable better visibility, reduce repetitive processes, enable the development team to produce deployable products at a moment's notice, and reduce risk overall.
These slides will identify the various levels of continuous integration and delivery with regards to a release maturity of the development team or parent organization.
Coding Naked – Practical steps to moving towards & embracing TDD. We'll overview the normal roadblocks that people typically run in to, and practical ways to overcome those road blocks on your way to embracing Test Driven Development - make coding without tests as uncomfortable as coding naked!
Writing code is fun, but deploying to production is not. Production releases are scary events that last all weekend, and you find yourself worrying about how it will go. Did we miss a configuration file? Is the database schema the same as the one in the test environment? Does the last minute hot fix we just applied break any other features? Did I forget to include an installation instruction for the system administrators?
Continuous Delivery is a collection of principles and practices aimed at addressing the problems teams typically face when releasing changes to production. By applying rigorous automation, testing and configuration management, teams are able to confidently and consistently deploy changes from version control to production without fear.
In this talk, Mike McGarr will provide listeners with an introduction into the world of Continuous Delivery. After an introduction into the concepts and principles of Continuous Delivery, he will discuss many of the techniques for implementing Continuous Delivery and recommend some tools that can be used on your development project.
If you are building a commercial Force.com app with a team of developers, this session is for you. Join us to learn best practices for setting up your Force.com IDE, managing source code, creating automated builds, deploying to test environments, and more. Hear from a panel of seasoned ISVs who are employing key team development principles. This session is primarily for product managers, architects, and developers (isvpartners).
In 2010, Microsoft released a bold new featureset to support management of virtual test environments. "Lab Management" provided the ability to easily spin up test environments, perform automated build and deployments, run automated tests, and collect diagnostic data. Unfortunately, many teams were discouraged by the infrastructure requirements. Now, with Visual Studio 2012 and standard environments, even small teams or groups that can't use Microsoft's Hyper-V can still benefit from lab management. This session will demonstrate how to configure your existing environments for many of the same compelling features formally available only with Hyper-V.
Full Testing Experience - Visual Studio and TFS 2010Ed Blankenship
This presentation goes through the full testing experience of Visual Studio 2010 and Team Foundation Server 2010 including using the new Lab Management features in the full testing process.
Leveraging Azure for Performance TestingTarun Arora
Working with various clients in the industry I have realized that the biggest barrier in Load Testing & Performance Testing adoption is the high infrastructure and administration cost that comes with this phase of testing.
I will present an approach using Visual Studio and Windows Azure to effectively abstract the administration cost of infrastructure management and lower the total cost of Load & Performance Testing. This should hopefully help you leverage Windows Azure for Performance Testing your applications.
Learn why you should put your blackbox (or system/integration) tests into Docker Containers.
Brief (remedial) overview of Docker for software testers who don't know docker, and only need to know the basics to wrap their regression tests inside of a container.
Teams need to move fast, every action which results in wait time must be minimized to zero. Teams need to move flexible, context changes must be easy adoptable by the team and the system they realize. Using Azure for their Environment and ALM needs helps them fulfill this need.
Agile teams find it hard to get the testing effort in sync with the other development activities. Not only development tests are executed during sprints, all other kind of testing activities are part of done. This session will give guidance how Microsoft Visual Studio ALM tools can support agile teams. How to run sprints and get testing done in a sprint.
This guide is an extract from the two and three day course provided by me. It spans the complete testing lifecycle and the tool usages. It will look at the infrastructure implications and testing practices in formal and in agile teams. But, the main focus stays on the usages of the Microsoft Visual Studio testing tools, the knowledge you need to get starting with it, the practices you must have to work with it in real live and how you can bend the tools, with extensibility and normal use to your team needs.
The course and this guide is work in progress. It is not a testing training (I expect you already have testing knowledge), if you need that test process information I refer to the TMap website from Sogeti where you can find tons of information. This training guide follows the TMap testing lifecycle.
BETA work in progress, I add every training new material and tune current material.
Talk Through for the VS2010 ALM MTLM usages patterns for Windows Azure hosted application development persentation http://www.slideshare.net/clemensreijnen/alm-4-azure-7425724
TMap® meets Visual Studio®, München - http://sogeti.de/536.html
The demo's can be found on Youtube:
01: http://www.youtube.com/watch?v=mVaRJes_Qaw
02: http://www.youtube.com/watch?v=3KR5Sxile14
03: http://www.youtube.com/watch?v=39xVnl02tHQ
04: http://www.youtube.com/watch?v=TkdVoXJI8KU
05: http://www.youtube.com/watch?v=V1RiN3EDcw4
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
5. ALM - Agile | Lean
Application Lifecycle Management
Collaborate over Processes and Products with tools.
Agile, Lean and also TMap, DyA, ITIL
Methodologies, principles customized, adopted
and merged from all ALM roles.
optimized environments for supporting
the ALM team goals
5
11. ALM 4 Azure
Cloud System
Goals: Collaborative
High Quality
Flexible
Automated
Repeatable
Efficient
11
12. Scenario 1: Dev only
Scenario 2: Dev and Test
Scenario 3: Dev, Test and Build, Deploy
Scenario 4: Dev, Auto Tests and Build, Deploy
Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test
and Operations
12
13. Scenario 1: Dev only
Scenario 2: Dev and Test
Scenario 3: Dev, Test and Build, Deploy
Scenario 4: Dev, Auto Tests and Build, Deploy
Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test
and Operations
13
14. Scenario 1:
Developer only
1 Source Production
Repository Staging
C# 3
TEAM (implementatie + unittests) (TFS)
Developers implementeren het systeem
Local unittesting
‘SprintReview’Build
Compile
Run Unit Tests
2
14
17. Scenario 1:
Developer only
Pro: Con:
Easy installation and configuration No collaboration
Single click deployment from VS2010 Easy deployment errors (configuration)
What about test and ops
17
18. Scenario 1: Dev only
Scenario 2: Dev and Test
Scenario 3: Dev, Test and Build, Deploy
Scenario 4: Dev, Auto Tests and Build, Deploy
Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test
and Operations
18
19. Scenario 2:
Developer with
manual tester
Source Production
Repository Staging
C# 3
TEAM (implementatie + unittests) (TFS)
1 Developers implementeren het systeem
Local unittesting
Testers specificeren testcases ‘SprintReview’Build 4
“Dry” run van tests tegen Development Fabric Compile
Run Unit Tests
2
19
22. Source Production
Repository Staging
TEAM (TFS) 3
1
4
2
1 2 4
Functional, system, Build Verification Platform accepatance
manual tests executed Tests (unit tests) and tests executed on
in compute emulator other quality gates, Azure stating with
with MTM. executed in the build. MTM.
27. Capabilities
• Link test cases to requirements
• Fast Forward
• Test data Iterations
• Test case management
• Test configurations
Interesting capabilities 4 Azure
• Shared steps ( 4 VISP Swap )
• Diagnostic adapters ( Rich bug reports )
Not working capabilities in Azure
• Test Impact 4 manual tests
• Intelli Trace 4 manual tests (although possible custom
adapter)
27
28. Scenario 2:
Developer with
manual tester
Pro: Con:
Easy installation and configuration Easy deployment errors (configuration)
Single click deployment from VS2010 Time consuming (deploy and test)
Testers connected, same heartbeat as dev Not repeatable (annoyed testers)
Proven quality Testers connected
28
29. Scenario 1: Dev only
Scenario 2: Dev and Test
Scenario 3: Dev, Test and Build, Deploy
Scenario 4: Dev, Auto Tests and Build, Deploy
Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test
and Operations
29
30. Scenario 3:
Developer with manual tester
and deployment build
3
Source Production
Repository Staging
C#
TEAM (implementatie + unittests) (TFS)
1 Developers implementeren het systeem
Local unittesting
Testers specificeren testcases ‘SprintReview’Build 4
Dry run van tests tegen Development Fabric Compile
Run Unit Tests
2
Package (CSRun.exe, CSPack,exe, CSManage.exe)
Deploy 2 Staging (CMdLets, Powershell)
30
32. Deploy during build
1. Change .ccproj project
• New build target which uses the built-in CorePublish target
2. Create PowerShell script
• Called by build target and uses Azure CMDLets
3. Edit Build Config
• Add the necessary parameters
4. Do something with certificates
• Put the Azure certificate on the build server
32
34. Scenario 3:
Developer with manual tester
and deployment build
Pro: Con:
Easy installation and configuration Time consuming testing
No click deployment from build Application can contain ‘annoying’ bugs
Repeatable ‘proven’ deployments* Build workflow knowledge necessary
Testers connected, same heartbeat as dev Powershell, ccproj tweaks, target files,
Proven quality certificates
*demo needs some additional environment tweaks for different csfg files
per environment
34
35. Scenario 1: Dev only
Scenario 2: Dev and Test
Scenario 3: Dev, Test and Build, Deploy
Scenario 4: Dev, Auto Tests and Build, Deploy
Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test
and Operations
35
36. Scenario 4:
Developer with automated regresion
tests, manual tests and deployment build
3
Source Production
Repository Staging
C#
TEAM (implementatie + unittests) (TFS)
Test cases
(automated UI tests)
1 Developers implementeren het systeem
Local unittesting
4
Testers specificeren testcases ‘SprintReview’Build
Dry run van tests tegen Development Fabric Compile
Automation van testcases 5 Run Unit Tests
(moeten gedraaide testcases zijn voor CodedUI) 2
UI automation tests run tegen Development Fabric Package (CSRun.exe, CSPack,exe, CSManage.exe)
en associëren met test plan Deploy 2 Staging (CMdLets, Powershell)
36
43. Where to run the auto test?
a) At the Build server?
b) In the Development environment?
c) In the Test environment?
d) In the Azure environment?
43
44. Visual Studio Team
Foundation
Server
Test Manager
Build Server Test Agent
Test Agent
Test Agent
44
45. Configurations for automated test execution:
On physical environments (no lab)
TFS 2010
VS 2010 Build Controller
Build Agent
MTM 2010 Test Controller
Test Agent
45
47. Configurations for automated test execution:
A
Flavor A: Execution from VS2010…
Real developer test / test automation dry run
(Triggered by: right mouse click – run tests, in test project)
http://msdn.microsoft.com/en-us/library/dd286580.aspx
No additional configuration needed*
* Data Diagnostic Settings in the Local.Testsettings file are configurable.
Pro: Con:
Easy setup No collection of test results in TFS
Debug-able test automation Run on developer environment 47
48. Configurations for automated test execution:
B
Flavor B: Execution during build with Build controller…
not recommended, strange to run UI tests on build
Part of Build Verification Tests
(triggered by: build)
Set the Build Service to run in interactive mode
http://blogs.msdn.com/b/mathew_aniyan/archive/2009/05/26/coded-ui-test-in-a-team-build.aspx
Configure the build to run the tests
http://msdn.microsoft.com/en-us/library/ms182465.aspx#CreateBuildType
* Data Diagnostic Settings in the *.Testsettings file are configurable.
Pro: Con:
Easy setup No collection of test results in TFS
Test results in build result Build Controller needs to run in interactive mode
Tests are executed on build environment 48
Run on build environment
49. Configurations for automated test execution:
C
Flavor C: Execution during build with Test controller… B
Preferred configuration above flavor
Part of Build Verification Tests on multiple test agents
(triggered by: build)
Configure Test Controller (don’t register it with a project collection )
http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestControllers
Configure Test Agents on clients (interactive mode)
http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestAgents
Configure *.Testsettings file in solution to use Test Controller and Test Agents
Configure the build to run the tests (see B)
Pro: Con:
Test run on test environments No collection of test results in TFS
Tests run on multiple environments Harder to configure
Test Results in Build result Need for specific test client environments 49
Test Settings from VS
50. Configurations for automated test execution:
D
Flavor D: Execution from Microsoft Test Manager…than BVT
Other type of test
Part of Regression Tests
(triggered by: MTM user, right mouse click on test case, run)
Configure Test Controller (register it with a project collection )
Configure Test Agents on clients (interactive mode, can be the same as MTM)
Configure Lab Center in MTM to use test controller and create test ‘agent’ environment.
http://msdn.microsoft.com/en-us/library/ee390842.aspx
http://msdn.microsoft.com/en-us/library/dd293551.aspx
Associate CodedUI test with WI Test Case from VS.
http://www.richard-banks.org/2010/11/how-to-use-codedui-tests-watin-and-mtm.html
Pro: Con:
Test run on test environments Manually triggered by Tester
Tests run on multiple environments Hard to configure
Test Result in MTM and TFS Need for specific test client (same as MTM?)
50
Test Settings from MTM
51. Configurations for automated test execution:
E
Flavor E: Execution from MTM duringPreferred configuration above flavor C
Build…
Part of BVT Flavor D and E can be configured together
(triggered by: Build)
Configure Test Controller (register it with a project collection )
Configure Test Agents on clients (interactive mode, can be the same as MTM)
Configure Lab Center in MTM to use test controller and create test ‘agent’ environment.
Associate CodedUI test with WI Test Case from VS.
Create Build task to run MTC or MSTEST task for Test Plan
http://blogs.microsoft.co.il/blogs/shair/archive/2010/10/30/how-to-run-coded-ui-tests-from-command-line.aspx
Pro: Con:
Test run on test environments Hard to configure
Tests run on multiple environments Need for specific test client (same as MTM?)
Test Result in MTM and TFS
Triggered by build 51
53. Scenario 4:
Developer with automated regresion
tests, manual tests and deployment build
Pro: Con:
No click deployment from build Build workflow knowledge necessary
Repeatable ‘proven’ deployments* Powershell, ccproj tweaks, target files,
Testers connected, same heartbeat as dev Certificates
Proven quality Test Infrastructure knowledge necessary
Automated BVT on different Environments A balanced thinking of test automation
Comfortable Acc Testing
Done Done
*demo needs some additional environment tweaks for different csfg files
per environment 53
54. Scenario 1: Dev only
Scenario 2: Dev and Test
Scenario 3: Dev, Test and Build, Deploy
Scenario 4: Dev, Auto Tests and Build, Deploy
Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test
and Operations
54
55. Automated ALM for Azure Cloud development
Efficient, repeatable with proven quality fast
flexible delivering of useful functionality … 4
Acceptation
Making optimal use of the characteristics of Azure:
- Identical environments (Guest OS versioning).
- Separate in depended staging production environments. ‘Release ’Drop
- Easy up and downgrade of applications. Package (same as test env
- Unambiguous deployment mechanism 3 package)
- rich diagnostic capabilities Acc testing met MTM en
- Simplified Application Deployment & Management Diagnostic Data Collectors
- …
BUILD Production
(TFS) Staging
C#
TEAM (implementation + unittests)
Test cases
(automated UI tests)
Developers implement the system
Local unit testing
Testers specify test cases ‘SprintReview’Build
Dry run tests cases against Development Fabric Compile
Automation of test cases Run Unit Tests
1
2 Package (CSRun.exe, CSPack,exe, CSManage.exe)
UI automation tests execute against Development Deploy 2 Staging (CMdLets, Powershell)
Fabric and associate with a test case Run automated Test Suites (MTC.exe) with different
environments (test controller – agents infrastructure)
Operations determine instrumentation data Collect and publish test results
DFO, SLA “Build Verification Tests” green -> deployment 2 ‘test
Usages Metrics, payment 'production 55
56. Scenario 1: Dev only
Scenario 2: Dev and Test
Scenario 3: Dev, Test and Build, Deploy
Scenario 4: Dev, Auto Tests and Build, Deploy
Scenario 5: Dev, Auto Tests, Build, Deploy, Acc test
and Operations
Scenario 6: TFS on Azure
Scenario 7: Other Infrastructure on Azure
56