Enterprise Build and Test
in the Cloud

Carlos Sanchez
G2iX
About me

>   Sr. Solutions
    Architect at G2iX
>   Member of Apache
    Maven PMC,
    Continuum, Archiva
>   Eclipse I...
Index

1.   The tools
2.   Maven Integration Tests
3.   Maven & Selenium
4.   Adding Continuum to the Mix
5.   Testing in ...
1. The tools




               4
Build: Apache Maven




                      5
Build: Apache Maven

>   Build tool and more




                          5
Build: Apache Maven

>   Build tool and more
>   Dependency management




                            5
Build: Apache Maven

>   Build tool and more
>   Dependency management
>   Execution of unit/integration tests




       ...
Build: Apache Maven

>   Build tool and more
>   Dependency management
>   Execution of unit/integration tests
>   Start/s...
Build: Apache Maven

>   Build tool and more
>   Dependency management
>   Execution of unit/integration tests
>   Start/s...
Test Cases: TestNG




                     6
Test Cases: TestNG

>   Unit/integration tests




                             6
Test Cases: TestNG

>   Unit/integration tests
>   parameterized tests




                             6
Test Cases: TestNG

>   Unit/integration tests
>   parameterized tests
>   parallel testing




                          ...
Integration Tests: Selenium
Integration Tests: Selenium

>   UI and Integration testing
Integration Tests: Selenium

>   UI and Integration testing
>   Tests run in the browser
    • support for multiple browse...
Integration Tests: Selenium

>   UI and Integration testing
>   Tests run in the browser
    • support for multiple browse...
Continuous Integration: Apache Continuum




                                           8
Continuous Integration: Apache Continuum

>   Tight integration with Apache Maven
    • dependency handling




          ...
Continuous Integration: Apache Continuum

>   Tight integration with Apache Maven
    • dependency handling
>   Trigger bu...
Cloud Computing: Amazon Web Services




                                       9
Cloud Computing: Amazon Web Services

>   Amazon S3
    • storage




                                       9
Cloud Computing: Amazon Web Services

>   Amazon S3
    • storage
>   Amazon SQS
    • queue service




                 ...
Cloud Computing: Amazon Web Services

>   Amazon S3
    • storage
>   Amazon SQS
    • queue service
>   Amazon Mechanical...
Cloud Computing: Amazon Web Services

>   Amazon S3
    • storage
>   Amazon SQS
    • queue service
>   Amazon Mechanical...
2. Maven integration tests




                             10
The webapp to test

>   appfuse-struts




                     11
A new Maven project

>   appfuse-selenium



                 <groupId>org.appfuse</groupId>
                 <artifactId>...
that depends on

>   appfuse-struts



                     <dependency>
                      <groupId>org.appfuse</group...
using Cargo to run it

>   automatically




             <plugin>
               <groupId>org.codehaus.cargo</groupId>
  ...
Record Selenium tests

>   SeleniumIDE




                        15
as Java




          16
test cases
                         public void setUp() throws Exception {
>   using Selenium API       setUp("http://loca...
breakpoint

>   Selenium tests in Maven IT project




                                         18
3. Maven & Selenium




                      19
Selenium Architecture




                        Selenium
                         server




                           ...
Maven & Selenium

>   Start Selenium server locally
>   Run integration tests
>   Stop selenium server




               ...
Selenium Maven plugin

>   http://mojo.codehaus.org/selenium-maven-plugin

>   selenium:start-server
>   selenium:stop-ser...
Selenium Maven plugin
       <plugin>
         <groupId>org.codehaus.mojo</groupId>
         <artifactId>selenium-maven-pl...
24
-------------------------------------------------------
 T E S T S
-------------------------------------------------------...
breakpoint

>   Selenium tests in Maven IT project
>   running locally in the browser




                                ...
4. Adding Continuum to the Mix




                                 27
Continuum




            28
Continuum

>   Build the webapp if




                          28
Continuum

>   Build the webapp if
    • the webapp changes




                           28
Continuum

>   Build the webapp if
    • the webapp changes
    • the dependencies change




                            ...
Continuum

>   Build the webapp if
    • the webapp changes
    • the dependencies change
>   Run the integration tests


...
Continuum

>   Build the webapp if
    • the webapp changes
    • the dependencies change
>   Run the integration tests
  ...
Continuum

>   Build the webapp if
    • the webapp changes
    • the dependencies change
>   Run the integration tests
  ...
Continuum

>   Add appfuse-struts
>   Add appfuse-selenium




                           29
Selenium headless

>   Most likely needed

>   Use *nix X Virtual Frame Buffer

>   In Maven
    • selenium:xvfb



      ...
breakpoint

>   Selenium tests in Maven IT project
>   running locally in the browser
>   triggered by Continuum when ther...
5. Testing in different environments




                                       32
Multi environment




                    33
Multi environment
               Multiple Browsers




                                   33
Multi environment
               Multiple Browsers




           Multiple Operating Systems




                         ...
Multi environment
               Multiple Browsers




           Multiple Operating Systems




                         ...
Parallel

>   Run tests in parallel
    • in each browser
    • several browsers per test




                            ...
Parallel

>   Faster testing
>   Load testing




           Credits: XKCD
                           35
Selenium Grid




                36
37
38
TestNG properties

>   Allows to run the same tests in different
    environments

>   and in parallel




               ...
TestNG properties

   <suite name="Example" parallel="tests" thread-count="3">
     <test name="iexplore">
       <paramet...
TestNG properties

   <suite name="Example" parallel="tests" thread-count="3">
     <test name="iexplore">
       <paramet...
breakpoint

>   Selenium tests in Maven IT project
>   running locally in several browsers




                           ...
6. and now in the Cloud




                          42
$$$




      43
$$$




>   Machines cost money




                          43
$$$




>   Machines cost money
>   Bandwidth costs money




                            43
$$$




>   Machines cost money
>   Bandwidth costs money
>   Electricity costs money



                              43
$$$




>   Machines cost money
>   Bandwidth costs money
>   Electricity costs money
>   Server administration costs mone...
Resource Optimization

>   test servers working 10-20% of the time




                                              44
Predictions cost money
    Infrastructure
        Cost $




                                      time
                  ...
Predictions cost money
    Infrastructure
        Cost $




                                      Predicted
             ...
Predictions cost money
    Infrastructure
        Cost $




                                      Predicted
             ...
Predictions cost money
    Infrastructure
        Cost $



           Large
           Capital
         Expenditure


   ...
Predictions cost money
    Infrastructure
        Cost $




                                      Predicted
             ...
Predictions cost money
    Infrastructure
        Cost $




                                      Predicted
             ...
Predictions cost money
    Infrastructure
        Cost $




                                                Predicted
   ...
Predictions cost money
    Infrastructure
        Cost $



                                                You just lost
...
Predictions cost money
    Infrastructure
        Cost $




                                      Predicted
             ...
Availability

>   Can you get 3000 machines in 3 days?




                                           46
Amazon EC2

>   Elastic Compute Cloud

>   Machines on demand




                            47
Amazon EC2

>   pay per hour
    • same price for
     •   10 machines / 1 hour
     •   1 machine / 10 hours
>   unlimite...
AMIs




       49
AMIs

       Amazon Machine Images




                               49
AMIs

       Amazon Machine Images




          Bundle your own


                               49
Countless possibilities

>   Internet Explorer
>   Firefox 2
>   Opera
>   Safari
>   Firefox 3
>   Firefox 2
>   etc...
>...
Amazon AMIs




              51
Amazon AMIs

>   Create AMIs for different environments to test




                                                     51
Amazon AMIs

>   Create AMIs for different environments to test
>   Read user data to customize the images at runtime
    ...
Maven EC2 plugin

>   in progress

>   http://mojo.codehaus.org/maven-ec2-plugin
    • start Amazon AMIs
    • stop at the...
Workflow




          53
Starting point




                 Amazon EC2
Starting point
Build is triggered
Start Images
Start Images
Register with Hub
Tests call Selenium
Distribute Tests
Run tests against webapp
Return results to Hub
Return results to builder
Kill Images
64
Questions?




             65
Carlos Sanchez
csanchez@g2ix,com
carlos@apache.org
http://www.carlossanchez.eu

Twitter: csanchez
Upcoming SlideShare
Loading in...5
×

Enterprise Build And Test In The Cloud

2,650

Published on

Enterprise Build And Test In The Cloud
JavaOne 2009 San Francisco
http://www.carlossanchez.eu

Building and testing software can be a time- and resource-consuming task. Cloud computing/on-demand services such as Amazon EC2 provide a cost-effective way to scale applications and, for building and testing software, can reduce the time needed to find and correct problems, meaning a cost reduction as well.

Properly configuring your build tools (Maven, Ant,...), continuous integration servers (Continuum, Cruise Control,...), and testing tools (TestNG, Selenium,...) can enable you to run all the building/testing process in a cloud environment, simulating high-load environments, distributing long-running tests to reduce their execution time, using different environments for client or server applications, and so on -- and in the case of on-demand services such as Amazon EC2, pay only for the time you use it.

In this presentation we will introduce a development process and architecture using popular open source tools for the build and test process such as Apache Maven or Ant for building, Apache Continuum as continuous integration server, TestNG and Selenium for testing, and how to configure them to achieve the best results and performance in several typical use cases (long running testing processes, different client platforms,...) by using he Amazon Elastic Computing Cloud EC2, and therefore reducing time and costs compared to other solutions.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,650
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Enterprise Build And Test In The Cloud

  1. 1. Enterprise Build and Test in the Cloud Carlos Sanchez G2iX
  2. 2. About me > Sr. Solutions Architect at G2iX > Member of Apache Maven PMC, Continuum, Archiva > Eclipse IAM co-lead > etc... 2
  3. 3. Index 1. The tools 2. Maven Integration Tests 3. Maven & Selenium 4. Adding Continuum to the Mix 5. Testing in Different Environments 6. And Now in the Cloud 3
  4. 4. 1. The tools 4
  5. 5. Build: Apache Maven 5
  6. 6. Build: Apache Maven > Build tool and more 5
  7. 7. Build: Apache Maven > Build tool and more > Dependency management 5
  8. 8. Build: Apache Maven > Build tool and more > Dependency management > Execution of unit/integration tests 5
  9. 9. Build: Apache Maven > Build tool and more > Dependency management > Execution of unit/integration tests > Start/stop application server 5
  10. 10. Build: Apache Maven > Build tool and more > Dependency management > Execution of unit/integration tests > Start/stop application server > Automatic application deployment 5
  11. 11. Test Cases: TestNG 6
  12. 12. Test Cases: TestNG > Unit/integration tests 6
  13. 13. Test Cases: TestNG > Unit/integration tests > parameterized tests 6
  14. 14. Test Cases: TestNG > Unit/integration tests > parameterized tests > parallel testing 6
  15. 15. Integration Tests: Selenium
  16. 16. Integration Tests: Selenium > UI and Integration testing
  17. 17. Integration Tests: Selenium > UI and Integration testing > Tests run in the browser • support for multiple browsers
  18. 18. Integration Tests: Selenium > UI and Integration testing > Tests run in the browser • support for multiple browsers > Tests can be recorded • no need for developers writing tests • Selenium IDE Firefox plugin
  19. 19. Continuous Integration: Apache Continuum 8
  20. 20. Continuous Integration: Apache Continuum > Tight integration with Apache Maven • dependency handling 8
  21. 21. Continuous Integration: Apache Continuum > Tight integration with Apache Maven • dependency handling > Trigger builds • when integration tests change • when the webapp changes • when the webapp dependencies change 8
  22. 22. Cloud Computing: Amazon Web Services 9
  23. 23. Cloud Computing: Amazon Web Services > Amazon S3 • storage 9
  24. 24. Cloud Computing: Amazon Web Services > Amazon S3 • storage > Amazon SQS • queue service 9
  25. 25. Cloud Computing: Amazon Web Services > Amazon S3 • storage > Amazon SQS • queue service > Amazon Mechanical Turk • manpower 9
  26. 26. Cloud Computing: Amazon Web Services > Amazon S3 • storage > Amazon SQS • queue service > Amazon Mechanical Turk • manpower > Amazon EC2 • computation 9
  27. 27. 2. Maven integration tests 10
  28. 28. The webapp to test > appfuse-struts 11
  29. 29. A new Maven project > appfuse-selenium <groupId>org.appfuse</groupId> <artifactId>appfuse-selenium</artifactId> <version>2.0.2-SNAPSHOT</version> <packaging>war</packaging> <name>AppFuse Selenium testing</name> 12
  30. 30. that depends on > appfuse-struts <dependency> <groupId>org.appfuse</groupId> <artifactId>appfuse-struts</artifactId> <version>2.0.2</version> <type>war</type> </dependency> 13
  31. 31. using Cargo to run it > automatically <plugin> <groupId>org.codehaus.cargo</groupId> <artifactId>cargo-maven2-plugin</artifactId> 14
  32. 32. Record Selenium tests > SeleniumIDE 15
  33. 33. as Java 16
  34. 34. test cases public void setUp() throws Exception { > using Selenium API setUp("http://localhost:18880/", "*chrome"); } public void testNew() throws Exception { selenium.open("/mainMenu.html"); selenium.click("link=Administration"); selenium.waitForPageToLoad("30000"); selenium.type("j_password", "admin"); selenium.click("rememberMe"); selenium.click("//input[@name='login']"); selenium.waitForPageToLoad("30000"); selenium.click("link=Edit Profile"); selenium.waitForPageToLoad("30000"); selenium.click("link=Main Menu"); selenium.waitForPageToLoad("30000"); selenium.click("link=Edit Profile"); selenium.waitForPageToLoad("30000"); selenium.click("document.forms[0].elements[24]"); selenium.waitForPageToLoad("30000"); } 17
  35. 35. breakpoint > Selenium tests in Maven IT project 18
  36. 36. 3. Maven & Selenium 19
  37. 37. Selenium Architecture Selenium server 20
  38. 38. Maven & Selenium > Start Selenium server locally > Run integration tests > Stop selenium server 21
  39. 39. Selenium Maven plugin > http://mojo.codehaus.org/selenium-maven-plugin > selenium:start-server > selenium:stop-server 22
  40. 40. Selenium Maven plugin <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>selenium-maven-plugin</artifactId> <version>1.0-rc-1</version> <executions> <execution> <id>start-selenium</id> <phase>pre-integration-test</phase> <goals> <goal>start-server</goal> </goals> </execution> </executions> <configuration> <background>${selenium.background}</background> </configuration> </plugin> 23
  41. 41. 24
  42. 42. ------------------------------------------------------- T E S T S ------------------------------------------------------- Running TestSuite log4j:WARN No appenders could be found for logger (com.thoughtworks.selenium.grid.tools.ThreadSafeSeleniumSessionStorage). log4j:WARN Please initialize the log4j system properly. DEBUG [btpool0-5] UserAction.edit(117) | checking for remember me login... DEBUG [btpool0-8] LookupDaoHibernate.getRoles(20) | Retrieving all role names... DEBUG [btpool0-8] StartupListener.setupContext(102) | Drop-down initialization complete [OK] INFO [btpool0-8] ReloadAction.execute(34) | reload complete, reloading user back to: http://localhost: 18880/admin/activeUsers.html DEBUG [btpool0-3] UserAction.edit(117) | checking for remember me login... DEBUG [btpool0-3] UserAction.edit(117) | checking for remember me login... Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 89.703 sec Results : Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 [INFO] [cargo:stop {execution: stop-container}] 2009-03-23 16:41:42.176:/appfuse-selenium-2.0.2-SNAPSHOT:INFO: Closing Spring root WebApplicationContext [INFO] [install:install] [INFO] Installing /Users/csanchez/dev/maestro/appfuse-selenium/target/appfuse-selenium-2.0.2- SNAPSHOT.war to /Users/csanchez/.m2/repository/org/appfuse/appfuse-selenium/2.0.2-SNAPSHOT/appfuse- selenium-2.0.2-SNAPSHOT.war [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2 minutes 20 seconds [INFO] Finished at: Mon Mar 23 16:41:45 CET 2009 [INFO] Final Memory: 43M/63M [INFO] ------------------------------------------------------------------------ 2009-03-23 16:41:46.168::INFO: Shutdown hook executing 2009-03-23 16:41:46.169::INFO: Shutdown hook complete 25
  43. 43. breakpoint > Selenium tests in Maven IT project > running locally in the browser 26
  44. 44. 4. Adding Continuum to the Mix 27
  45. 45. Continuum 28
  46. 46. Continuum > Build the webapp if 28
  47. 47. Continuum > Build the webapp if • the webapp changes 28
  48. 48. Continuum > Build the webapp if • the webapp changes • the dependencies change 28
  49. 49. Continuum > Build the webapp if • the webapp changes • the dependencies change > Run the integration tests 28
  50. 50. Continuum > Build the webapp if • the webapp changes • the dependencies change > Run the integration tests • if the ITs change 28
  51. 51. Continuum > Build the webapp if • the webapp changes • the dependencies change > Run the integration tests • if the ITs change • if the webapp changes 28
  52. 52. Continuum > Add appfuse-struts > Add appfuse-selenium 29
  53. 53. Selenium headless > Most likely needed > Use *nix X Virtual Frame Buffer > In Maven • selenium:xvfb 30
  54. 54. breakpoint > Selenium tests in Maven IT project > running locally in the browser > triggered by Continuum when there is any change to webapp, tests or dependencies 31
  55. 55. 5. Testing in different environments 32
  56. 56. Multi environment 33
  57. 57. Multi environment Multiple Browsers 33
  58. 58. Multi environment Multiple Browsers Multiple Operating Systems 33
  59. 59. Multi environment Multiple Browsers Multiple Operating Systems 33
  60. 60. Parallel > Run tests in parallel • in each browser • several browsers per test 34
  61. 61. Parallel > Faster testing > Load testing Credits: XKCD 35
  62. 62. Selenium Grid 36
  63. 63. 37
  64. 64. 38
  65. 65. TestNG properties > Allows to run the same tests in different environments > and in parallel 39
  66. 66. TestNG properties <suite name="Example" parallel="tests" thread-count="3"> <test name="iexplore"> <parameter name="browser" value="*iexplore"/> <packages> <package name="org.appfuse.webapp.selenium" /> </packages> </test> <test name="firefox2"> <parameter name="browser" value="*firefox2"/> <packages> <package name="org.appfuse.webapp.selenium" /> </packages> </test> <test name="opera"> <parameter name="browser" value="*opera"/> <packages> <package name="org.appfuse.webapp.selenium" /> </packages> </test> </suite> 40
  67. 67. TestNG properties <suite name="Example" parallel="tests" thread-count="3"> <test name="iexplore"> <parameter name="browser" value="*iexplore"/> <packages> <package name="org.appfuse.webapp.selenium" /> </packages> </test> <test name="firefox2"> <parameter name="browser" value="*firefox2"/> <packages> <package name="org.appfuse.webapp.selenium" /> </packages> </test> <test name="opera"> <parameter name="browser" value="*opera"/> <packages> <package name="org.appfuse.webapp.selenium" /> </packages> </test> </suite> 40
  68. 68. breakpoint > Selenium tests in Maven IT project > running locally in several browsers 41
  69. 69. 6. and now in the Cloud 42
  70. 70. $$$ 43
  71. 71. $$$ > Machines cost money 43
  72. 72. $$$ > Machines cost money > Bandwidth costs money 43
  73. 73. $$$ > Machines cost money > Bandwidth costs money > Electricity costs money 43
  74. 74. $$$ > Machines cost money > Bandwidth costs money > Electricity costs money > Server administration costs money 43
  75. 75. Resource Optimization > test servers working 10-20% of the time 44
  76. 76. Predictions cost money Infrastructure Cost $ time Source: Amazon
  77. 77. Predictions cost money Infrastructure Cost $ Predicted Demand time Source: Amazon
  78. 78. Predictions cost money Infrastructure Cost $ Predicted Demand Traditional Hardware time Source: Amazon
  79. 79. Predictions cost money Infrastructure Cost $ Large Capital Expenditure Predicted Demand Traditional Hardware time Source: Amazon
  80. 80. Predictions cost money Infrastructure Cost $ Predicted Demand Traditional Hardware time Source: Amazon
  81. 81. Predictions cost money Infrastructure Cost $ Predicted Demand Traditional Hardware Actual Demand time Source: Amazon
  82. 82. Predictions cost money Infrastructure Cost $ Predicted Demand Opportunity Traditional Cost Hardware Actual Demand time Source: Amazon
  83. 83. Predictions cost money Infrastructure Cost $ You just lost customers Predicted Demand Opportunity Traditional Cost Hardware Actual Demand time Source: Amazon
  84. 84. Predictions cost money Infrastructure Cost $ Predicted Demand Traditional Hardware Actual Demand Automated Virtualization time Source: Amazon
  85. 85. Availability > Can you get 3000 machines in 3 days? 46
  86. 86. Amazon EC2 > Elastic Compute Cloud > Machines on demand 47
  87. 87. Amazon EC2 > pay per hour • same price for • 10 machines / 1 hour • 1 machine / 10 hours > unlimited number of machines > start in couple minutes 48
  88. 88. AMIs 49
  89. 89. AMIs Amazon Machine Images 49
  90. 90. AMIs Amazon Machine Images Bundle your own 49
  91. 91. Countless possibilities > Internet Explorer > Firefox 2 > Opera > Safari > Firefox 3 > Firefox 2 > etc... > no OS X :( 50
  92. 92. Amazon AMIs 51
  93. 93. Amazon AMIs > Create AMIs for different environments to test 51
  94. 94. Amazon AMIs > Create AMIs for different environments to test > Read user data to customize the images at runtime • where is the Selenium Hub 51
  95. 95. Maven EC2 plugin > in progress > http://mojo.codehaus.org/maven-ec2-plugin • start Amazon AMIs • stop at the end of the build • pass on user data 52
  96. 96. Workflow 53
  97. 97. Starting point Amazon EC2
  98. 98. Starting point
  99. 99. Build is triggered
  100. 100. Start Images
  101. 101. Start Images
  102. 102. Register with Hub
  103. 103. Tests call Selenium
  104. 104. Distribute Tests
  105. 105. Run tests against webapp
  106. 106. Return results to Hub
  107. 107. Return results to builder
  108. 108. Kill Images
  109. 109. 64
  110. 110. Questions? 65
  111. 111. Carlos Sanchez csanchez@g2ix,com carlos@apache.org http://www.carlossanchez.eu Twitter: csanchez

×