Alm 4 Azure

3,539 views
3,404 views

Published on

VS2010 ALM MTLM usages patterns for Windows Azure hosted application development

Published in: Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total views
3,539
On SlideShare
0
From Embeds
0
Number of Embeds
1,334
Actions
Shares
0
Downloads
56
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Alm 4 Azure

  1. 1. ALM 4 AzureAutomated ALM for Azure Cloud Development 1
  2. 2. AGENDA• ALM• Azure• ALM 4 Azure 2
  3. 3. Clemens Reijnen• Sogeti | Management Consultant www.ClemensReijnen.nl @ClemensReijnen Clemens.Reijnen@Sogeti.nl 3
  4. 4. application lifecycle management 4http://www.clemensreijnen.nl/post/2008/02/18/ALM-Definitions.aspx
  5. 5. ALM - Agile | LeanApplication Lifecycle ManagementCollaborate over Processes and Products with tools.Agile, Lean and also TMap, DyA, ITILMethodologies, principles customized, adoptedand merged from all ALM roles. optimized environments for supporting the ALM team goals 5
  6. 6. work together__with work packages__on alm artifacts 6
  7. 7. 7http://msdn.microsoft.com/en-us/library/bb385832.aspx
  8. 8. Guest OS Versioningidentical – independent environments 8
  9. 9. Development & Deployment SDK *.cspkg Config *.cscfg 9
  10. 10. Start 10
  11. 11. ALM 4 Azure Cloud SystemGoals: Collaborative High Quality Flexible Automated Repeatable Efficient 11
  12. 12. Scenario 1: Dev onlyScenario 2: Dev and TestScenario 3: Dev, Test and Build, DeployScenario 4: Dev, Auto Tests and Build, DeployScenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations 12
  13. 13. Scenario 1: Dev onlyScenario 2: Dev and TestScenario 3: Dev, Test and Build, DeployScenario 4: Dev, Auto Tests and Build, DeployScenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations 13
  14. 14. Scenario 1:Developer only 1 Source Production Repository Staging C# 3 TEAM (implementatie + unittests) (TFS) Developers implementeren het systeem  Local unittesting ‘SprintReview’Build  Compile  Run Unit Tests 2 14
  15. 15. Stages of Service Deployment Taken from:
  16. 16. DEMODeploying aCloud Servicefrom VisualStudio 16
  17. 17. Scenario 1:Developer only Pro: Con: Easy installation and configuration No collaboration Single click deployment from VS2010 Easy deployment errors (configuration) What about test and ops 17
  18. 18. Scenario 1: Dev onlyScenario 2: Dev and TestScenario 3: Dev, Test and Build, DeployScenario 4: Dev, Auto Tests and Build, DeployScenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations 18
  19. 19. Scenario 2:Developer withmanual tester Source Production Repository Staging C# 3 TEAM (implementatie + unittests) (TFS) 1 Developers implementeren het systeem  Local unittesting Testers specificeren testcases ‘SprintReview’Build 4  “Dry” run van tests tegen Development Fabric  Compile  Run Unit Tests 2 19
  20. 20. Microsoft TestManager 21
  21. 21. Source Production Repository Staging TEAM (TFS) 3 1 4 2 1 2 4 Functional, system, Build Verification Platform accepatancemanual tests executed Tests (unit tests) and tests executed onin compute emulator other quality gates, Azure stating with with MTM. executed in the build. MTM.
  22. 22. 1 Get the CSX folder and CSFG file234
  23. 23. DEMOTest a CloudService withMicrosoft TestManager 24
  24. 24. Diagnostic adapters
  25. 25. DEMOAzure LogsDiagnostic DataAdapter 26
  26. 26. Capabilities• Link test cases to requirements• Fast Forward• Test data Iterations• Test case management• Test configurationsInteresting capabilities 4 Azure• Shared steps ( 4 VISP Swap )• Diagnostic adapters ( Rich bug reports )Not working capabilities in Azure• Test Impact 4 manual tests• Intelli Trace 4 manual tests (although possible custom adapter) 27
  27. 27. Scenario 2:Developer withmanual tester Pro: Con: Easy installation and configuration Easy deployment errors (configuration) Single click deployment from VS2010 Time consuming (deploy and test) Testers connected, same heartbeat as dev Not repeatable (annoyed testers) Proven quality Testers connected  28
  28. 28. Scenario 1: Dev onlyScenario 2: Dev and TestScenario 3: Dev, Test and Build, DeployScenario 4: Dev, Auto Tests and Build, DeployScenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations 29
  29. 29. Scenario 3:Developer with manual testerand deployment build 3 Source Production Repository Staging C# TEAM (implementatie + unittests) (TFS) 1 Developers implementeren het systeem  Local unittesting Testers specificeren testcases ‘SprintReview’Build 4  Dry run van tests tegen Development Fabric  Compile  Run Unit Tests 2  Package (CSRun.exe, CSPack,exe, CSManage.exe)  Deploy 2 Staging (CMdLets, Powershell) 30
  30. 30. Automatic Deployment Management API CSPack CSRun CSManage PowerShell CmdLets Build Targetshttp://scottdensmore.typepad.com/blog/2010/03/azure-deployment-for-your-build-server.htmlhttp://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx 31
  31. 31. Deploy during build 1. Change .ccproj project • New build target which uses the built-in CorePublish target 2. Create PowerShell script • Called by build target and uses Azure CMDLets 3. Edit Build Config • Add the necessary parameters 4. Do something with certificates • Put the Azure certificate on the build server 32
  32. 32. DEMODeploy a CloudSolution withinthe Build 33
  33. 33. Scenario 3:Developer with manual testerand deployment build Pro: Con: Easy installation and configuration Time consuming testing No click deployment from build Application can contain ‘annoying’ bugs Repeatable ‘proven’ deployments* Build workflow knowledge necessary Testers connected, same heartbeat as dev Powershell, ccproj tweaks, target files, Proven quality certificates *demo needs some additional environment tweaks for different csfg files per environment 34
  34. 34. Scenario 1: Dev onlyScenario 2: Dev and TestScenario 3: Dev, Test and Build, DeployScenario 4: Dev, Auto Tests and Build, DeployScenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations 35
  35. 35. Scenario 4:Developer with automated regresiontests, manual tests and deployment build 3 Source Production Repository Staging C# TEAM (implementatie + unittests) (TFS) Test cases (automated UI tests) 1 Developers implementeren het systeem  Local unittesting 4 Testers specificeren testcases ‘SprintReview’Build  Dry run van tests tegen Development Fabric  Compile  Automation van testcases 5  Run Unit Tests (moeten gedraaide testcases zijn voor CodedUI) 2  UI automation tests run tegen Development Fabric  Package (CSRun.exe, CSPack,exe, CSManage.exe) en associëren met test plan  Deploy 2 Staging (CMdLets, Powershell) 36
  36. 36. CodedUI
  37. 37. Where? When? How? Who? Why? What?
  38. 38. Where? When? How? Who? Why? What?
  39. 39. Different URL’s 1 2 40
  40. 40. UIMap Editor 41
  41. 41. DEMOCreate automaticTests for the CloudApplication withCodedUI 42
  42. 42. Where to run the auto test? a) At the Build server? b) In the Development environment? c) In the Test environment? d) In the Azure environment? 43
  43. 43. Visual Studio Team Foundation ServerTest Manager Build Server Test Agent Test Agent Test Agent 44
  44. 44. Configurations for automated test execution: On physical environments (no lab) TFS 2010 VS 2010 Build Controller Build Agent MTM 2010 Test Controller Test Agent 45
  45. 45. Automated Test Settings 46
  46. 46. Configurations for automated test execution: AFlavor A: Execution from VS2010… Real developer test / test automation dry run (Triggered by: right mouse click – run tests, in test project) http://msdn.microsoft.com/en-us/library/dd286580.aspx No additional configuration needed* * Data Diagnostic Settings in the Local.Testsettings file are configurable. Pro: Con: Easy setup No collection of test results in TFS Debug-able test automation Run on developer environment 47
  47. 47. Configurations for automated test execution: BFlavor B: Execution during build with Build controller… not recommended, strange to run UI tests on build Part of Build Verification Tests (triggered by: build) Set the Build Service to run in interactive mode http://blogs.msdn.com/b/mathew_aniyan/archive/2009/05/26/coded-ui-test-in-a-team-build.aspx Configure the build to run the tests http://msdn.microsoft.com/en-us/library/ms182465.aspx#CreateBuildType * Data Diagnostic Settings in the *.Testsettings file are configurable. Pro: Con: Easy setup No collection of test results in TFS Test results in build result Build Controller needs to run in interactive mode Tests are executed on build environment 48 Run on build environment
  48. 48. Configurations for automated test execution: CFlavor C: Execution during build with Test controller… B Preferred configuration above flavor Part of Build Verification Tests on multiple test agents (triggered by: build) Configure Test Controller (don’t register it with a project collection ) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestControllers Configure Test Agents on clients (interactive mode) http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestAgents Configure *.Testsettings file in solution to use Test Controller and Test Agents Configure the build to run the tests (see B) Pro: Con: Test run on test environments No collection of test results in TFS Tests run on multiple environments Harder to configure Test Results in Build result Need for specific test client environments 49 Test Settings from VS
  49. 49. Configurations for automated test execution: DFlavor D: Execution from Microsoft Test Manager…than BVT Other type of test Part of Regression Tests (triggered by: MTM user, right mouse click on test case, run) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. http://msdn.microsoft.com/en-us/library/ee390842.aspx http://msdn.microsoft.com/en-us/library/dd293551.aspx Associate CodedUI test with WI Test Case from VS. http://www.richard-banks.org/2010/11/how-to-use-codedui-tests-watin-and-mtm.html Pro: Con: Test run on test environments Manually triggered by Tester Tests run on multiple environments Hard to configure Test Result in MTM and TFS Need for specific test client (same as MTM?) 50 Test Settings from MTM
  50. 50. Configurations for automated test execution: EFlavor E: Execution from MTM duringPreferred configuration above flavor C Build… Part of BVT Flavor D and E can be configured together (triggered by: Build) Configure Test Controller (register it with a project collection ) Configure Test Agents on clients (interactive mode, can be the same as MTM) Configure Lab Center in MTM to use test controller and create test ‘agent’ environment. Associate CodedUI test with WI Test Case from VS. Create Build task to run MTC or MSTEST task for Test Plan http://blogs.microsoft.co.il/blogs/shair/archive/2010/10/30/how-to-run-coded-ui-tests-from-command-line.aspx Pro: Con: Test run on test environments Hard to configure Tests run on multiple environments Need for specific test client (same as MTM?) Test Result in MTM and TFS Triggered by build 51
  51. 51. DEMOAutomatic test aCloud Service withMicrosoft Test andLab Managerinfrastructure 52
  52. 52. Scenario 4:Developer with automated regresiontests, manual tests and deployment build Pro: Con: No click deployment from build Build workflow knowledge necessary Repeatable ‘proven’ deployments* Powershell, ccproj tweaks, target files, Testers connected, same heartbeat as dev Certificates Proven quality Test Infrastructure knowledge necessary Automated BVT on different Environments A balanced thinking of test automation Comfortable Acc Testing Done Done *demo needs some additional environment tweaks for different csfg files per environment 53
  53. 53. Scenario 1: Dev onlyScenario 2: Dev and TestScenario 3: Dev, Test and Build, DeployScenario 4: Dev, Auto Tests and Build, DeployScenario 5: Dev, Auto Tests, Build, Deploy, Acc test and Operations 54
  54. 54. Automated ALM for Azure Cloud developmentEfficient, repeatable with proven quality fastflexible delivering of useful functionality … 4 AcceptationMaking optimal use of the characteristics of Azure:- Identical environments (Guest OS versioning).- Separate in depended staging production environments. ‘Release ’Drop- Easy up and downgrade of applications.  Package (same as test env- Unambiguous deployment mechanism 3 package)- rich diagnostic capabilities  Acc testing met MTM en- Simplified Application Deployment & Management Diagnostic Data Collectors- … BUILD Production (TFS) Staging C# TEAM (implementation + unittests) Test cases (automated UI tests) Developers implement the system  Local unit testing Testers specify test cases ‘SprintReview’Build  Dry run tests cases against Development Fabric  Compile  Automation of test cases  Run Unit Tests 1  2 Package (CSRun.exe, CSPack,exe, CSManage.exe)  UI automation tests execute against Development  Deploy 2 Staging (CMdLets, Powershell) Fabric and associate with a test case  Run automated Test Suites (MTC.exe) with different environments (test controller – agents infrastructure) Operations determine instrumentation data  Collect and publish test results  DFO, SLA  “Build Verification Tests” green -> deployment 2 ‘test  Usages Metrics, payment production 55
  55. 55. Scenario 1: Dev onlyScenario 2: Dev and TestScenario 3: Dev, Test and Build, DeployScenario 4: Dev, Auto Tests and Build, DeployScenario 5: Dev, Auto Tests, Build, Deploy, Acc test and OperationsScenario 6: TFS on AzureScenario 7: Other Infrastructure on Azure 56
  56. 56. 57

×