The Usual Suspects - Red Hat Developer Day 2012-11-01
Upcoming SlideShare
Loading in...5
×
 

The Usual Suspects - Red Hat Developer Day 2012-11-01

on

  • 161 views

Slides for my talk at Red Hat Developer Day 2012-11-01. ...

Slides for my talk at Red Hat Developer Day 2012-11-01.
Abstract: OpenShift has already established itself as a great platform to run Java applications (plus many other technologies) in the Cloud. But OpenShift also offers the foundation for a development environment, including source control with Git, build automation with Maven and continuous integration with Jenkins. In this session, we will learn how to extend the basic development environment and create a more powerful environment on OpenShift featuring usual suspects’ such as Sonar for continuous quality assurance, Cargo/Selenium/Arquillian for integration testing or JMeter for performance/load testing. The session includes a live demo built on OpenShift Origin.

Statistics

Views

Total Views
161
Views on SlideShare
161
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The Usual Suspects - Red Hat Developer Day 2012-11-01 The Usual Suspects - Red Hat Developer Day 2012-11-01 Presentation Transcript

  • The Usual Suspects – Jorge Hidalgo1 The Usual Suspects Creating a Cloud Development Environment with Sonar, Selenium and JMeter on OpenShift Origin Jorge Hidalgo Senior Technology Architect Accenture Delivery Centre in Spain – Java Lead
  • The Usual Suspects – Jorge Hidalgo2 Creating a Cloud Development Environment with Sonar, Selenium and JMeter on OpenShift Origin Abstract: OpenShift has already established itself as a great platform to run Java applications (plus many other technologies) in the Cloud. But OpenShift also offers the foundation for a development environment, including source control with Git, build automation with Maven and continuous integration with Jenkins. In this session, we will learn how to extend the basic development environment and create a more powerful environment on OpenShift featuring “usual suspects” such as Sonar for continuous quality assurance, Selenium for functional testing, JMeter for performance/load testing as well as Arquillian for in-container testing. The session includes a live demo built on OpenShift Origin. Speaker: Jorge Hidalgo – Senior Technology Architect Accenture Delivery Centre in Spain – Java Lead Author of dr. macphail’s trance blog: http://deors.wordpress.com The Usual Suspects
  • The Usual Suspects – Jorge Hidalgo3 What is a Development Environment? The Usual Suspects
  • The Usual Suspects – Jorge Hidalgo4 How Cloud Platforms can be of help? The Usual Suspects
  • The Usual Suspects – Jorge Hidalgo5 • Agenda – Understanding the Basics – Git and Apache Maven – Continuous Integration with Jenkins – Continuous Quality Assurance with Sonar – Functional, Cross-browser Tests with Selenium WebDriver – Performance/Load Tests with Apache JMeter – In-Container Tests with Arquillian – Q & A The Usual Suspects
  • The Usual Suspects – Jorge Hidalgo6 00: Understanding the Basics – Git and Apache Maven
  • The Usual Suspects – Jorge Hidalgo7 Understanding the Basics • Each application in OpenShift comes with a Git repository. – Stores sources, configuration, binaries and scripts. • Each push action to the repository triggers a new build and deploy process. – Details on what means “build and deploy” depends on the type of application, and can be customised with action hooks. git push origin master action hooks for a DIY app
  • The Usual Suspects – Jorge Hidalgo8 package integration-test • For Java applications (jbossas-7 type) builds are managed with Apache Maven. – Isolated configuration. – Isolated local repository. • Lifecycle standardisation. Understanding the Basics action hooks for a jbossas-7 app validate compile test verify install deploy
  • The Usual Suspects – Jorge Hidalgo9 • We are using Spring Pet Clinic reference application as test subject. – Resource redirection fixed to work with Tomcat 7 and JBoss 7 – Added some Selenium and Apache JMeter test scripts Preparing Pet Clinic
  • The Usual Suspects – Jorge Hidalgo10 Preparing Pet Clinic • Create a new JBoss application: >rhc app create -a petclinic -t jbossas-7 UUID: a74be67eaafb4ced969a279ce3082e03 Git URL: ssh://a74be67eaafb4ced969a279ce3082e03@petclinic- demo.example.com/~/git/petclinic.git/ Public URL: http://petclinic-demo.example.com/ Command line
  • The Usual Suspects – Jorge Hidalgo11 Preparing Pet Clinic • Add MySQL cartridge: >rhc app cartridge add -a petclinic -c mysql-5.1 Root User: admin Root Password: Uvvp2ZZ_nBzn Database Name: petclinic Connection URL: mysql://127.0.250.129:3306/ Command line
  • The Usual Suspects – Jorge Hidalgo12 • Merge Pet Clinic sources and configuration. • Load schema and initial data: – Connect your favourite database client to the MySQL cartridge: • OpenShift: redirect ports with command rhc port-forward. • Origin: use the client directly in the Origin box. – Run the scripts at: src/main/resources/db/mysql • Alternatively: – Run scripts on app start and disable them next time here: src/main/resources/applicationContext-dataSource.xml • Commit and push. Preparing Pet Clinic
  • The Usual Suspects – Jorge Hidalgo13 Preparing Pet Clinic
  • The Usual Suspects – Jorge Hidalgo14 01: Continuous Integration with Jenkins
  • The Usual Suspects – Jorge Hidalgo15 Continuous Integration • OpenShift offers easy configuration of Jenkins, the popular Continuous Integration engine. – Jenkins dashboard runs as another application within our domain. Jenkins consumes one gear. – For each application managed with Jenkins a especial type of Java application is created: the builder. Builders consume gears. • Why add Jenkins? – Centralized build reports: status, error causes, which version is deployed, change log, etc. – Better control to build process, e.g. add code profiling, add integration tests.
  • The Usual Suspects – Jorge Hidalgo16 Jenkins Installation and Configuration • Create a new Jenkins application: >rhc app create -a jenkins -t jenkins-1.4 UUID: 810197679f614bee8414b0fb701f50fc Git URL: ssh://810197679f614bee8414b0fb701f50fc@jenkins- demo.example.com/~/git/jenkins.git/ Public URL: http://jenkins-demo.example.com/ Command line
  • The Usual Suspects – Jorge Hidalgo17 Jenkins Installation and Configuration • Add build job (and builder app) for Pet Clinic: >rhc app cartridge add -a petclinic –c jenkins-client-1.4 Job URL: https://jenkins-demo.example.com/job/petclinic- build/ Command line
  • The Usual Suspects – Jorge Hidalgo18 Jenkins Installation and Configuration
  • The Usual Suspects – Jorge Hidalgo19 02: Continuous Quality Assurance with Sonar
  • The Usual Suspects – Jorge Hidalgo20 Continuous Quality Assurance • Sonar is a popular code profiler and dashboard that excels when used along a Continuous Integration engine: – Seamless integration with Maven. – Leverages best-of-breed tools as Checkstyle, PMD or FindBugs. – Configurable quality profiles. – Re-execution of tests and test code coverage (UT, IT). – Design Structure Matrix analysis. – Flexible and highly customisable dashboard. – Actions plans / peer reviews. – Historic views / run charts. – Can be used with Java, .Net, C/C++, Groovy, PHP,...
  • The Usual Suspects – Jorge Hidalgo21 Sonar Installation and Configuration • Create a new DIY application: >rhc app create -a sonar -t diy-0.1 UUID: 4cd9eacb71b34401b81ea712352ffccb Git URL: ssh://4cd9eacb71b34401b81ea712352ffccb@sonar- demo.example.com/~/git/sonar.git/ Public URL: http://sonar-demo.example.com/ Command line
  • The Usual Suspects – Jorge Hidalgo22 Sonar Installation and Configuration • Add MySQL cartridge: >rhc app cartridge add -a sonar -c mysql-5.1 Root User: admin Root Password: VyikSHzlLvjx Database Name: sonar Connection URL: mysql://127.0.251.129:3306/ Command line
  • The Usual Suspects – Jorge Hidalgo23 • Unzip Sonar distribution to app root folder. • Update conf/sonar.properties with settings for MySQL cartridge and bind to OpenShift internal IP and port: sonar.web.host: 127.0.251.129 sonar.web.port: 8080 sonar.web.context: / sonar.jdbc.username: admin sonar.jdbc.password: VyikSHzlLvjx sonar.jdbc.driver: com.mysql.jdbc.Driver sonar.jdbc.url: jdbc:mysql://127.0.251.129:3306/sonar ?useUnicode=true&characterEncoding=utf8 &rewriteBatchedStatements=true Sonar Installation and Configuration File: /sonar/sonar-3.1.1/conf/sonar.properties
  • The Usual Suspects – Jorge Hidalgo24 • Update action hooks with the appropriate commands to start and stop Sonar: #!/bin/bash cd $OPENSHIFT_REPO_DIR/sonar-3.1.1/bin/linux-x86-64 chmod 755 wrapper sh ./sonar.sh start #!/bin/bash cd $OPENSHIFT_REPO_DIR/sonar-3.1.1/bin/linux-x86-64 sh ./sonar.sh stop Sonar Installation and Configuration File: /sonar/.openshift/action_hooks/start File: /sonar/.openshift/action_hooks/stop
  • The Usual Suspects – Jorge Hidalgo25 • Add this to Pet Clinic build job to trigger the Sonar analysis before packaging and deploying the application: mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean test export SONAR_JDBC="-Dsonar.jdbc.url=jdbc:mysql://127.0.251.129:3306/sonar ?useUnicode=true&characterEncoding=utf8" export SONAR_DRIVER=-Dsonar.jdbc.driver=com.mysql.jdbc.Driver export SONAR_DBUSER=-Dsonar.jdbc.username=admin export SONAR_DBPASS=-Dsonar.jdbc.password=VyikSHzlLvjx export SONAR_URL=-Dsonar.host.url=http://sonar-demo.example.com export SONAR_OPTS="$SONAR_JDBC $SONAR_DRIVER $SONAR_DBUSER $SONAR_DBPASS $SONAR_URL" mvn --global-settings $OPENSHIFT_MAVEN_MIRROR sonar:sonar $SONAR_OPTS Sonar Installation and Configuration Jenkins job: petclinic-build
  • The Usual Suspects – Jorge Hidalgo26 Sonar Installation and Configuration
  • The Usual Suspects – Jorge Hidalgo27 03: Functional, Cross-browser Tests with Selenium WebDriver
  • The Usual Suspects – Jorge Hidalgo28 Functional Tests • Selenium is a popular framework / tool that allows, for web based applications, to record, re-execute and distribute tests across multiple browsers and computers. – Because Java-only tests may not be enough. – Can run with HtmlUnit to simulate a real browser (useful to gather code coverage metrics). – Can run with local browsers or distributed across computers in a grid. – The grid can be hosted in OpenShift, but browsers cannot – need dedicated machines with a desktop session that will act as test nodes. – You can use your Origin box as both hub and test node.
  • The Usual Suspects – Jorge Hidalgo29 Selenium Grid Hub Installation and Configuration • Create a new DIY application: >rhc app create -a seleniumhub -t diy-0.1 UUID: ddd8d0a2182b4b6799ec7c7ce8ae62b8 Git URL: ssh://ddd8d0a2182b4b6799ec7c7ce8ae62b8@ seleniumhub-demo.example.com/~/git/seleniumhub.git/ Public URL: http://seleniumhub-demo.example.com Command line
  • The Usual Suspects – Jorge Hidalgo30 File: /seleniumhub/.openshift/action_hooks/start File: /seleniumhub/.openshift/action_hooks/stop Selenium Grid Hub Installation and Configuration • Copy selenium-server-standalone-2.25.0.jar (or other version) to folder selenium-2.25.0. • Update action hooks with appropriate commands to start and stop the hub: #!/bin/bash nohup java -jar $OPENSHIFT_REPO_DIR/selenium-2.25.0/ selenium-server-standalone-2.25.0.jar -role hub -host $OPENSHIFT_INTERNAL_IP -port $OPENSHIFT_INTERNAL_PORT >>$OPENSHIFT_LOG_DIR/seleniumhub.log 2>&1 & #!/bin/bash kill `ps -ef | grep selenium | grep -v grep | awk '{ print $2 }'` > /dev/null 2>&1 exit 0
  • The Usual Suspects – Jorge Hidalgo31 • Update Maven target in Pet Clinic build job to launch Selenium tests with HtmlUnit to gather code coverage: mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean test ... Sonar settings ... mvn --global-settings $OPENSHIFT_MAVEN_MIRROR sonar:sonar $SONAR_OPTS mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean verify –Pcargo-tomcat,selenium-tests ... Sonar settings ... mvn --global-settings $OPENSHIFT_MAVEN_MIRROR sonar:sonar $SONAR_OPTS Selenium Grid Hub Installation and Configuration Jenkins job: petclinic-build BEFORE Jenkins job: petclinic-build AFTER
  • The Usual Suspects – Jorge Hidalgo32 • Add this at the end of Pet Clinic build job to launch Selenium tests with real browsers once the app is deployed: export SELENIUM_BROWSERS="-Dtest.run.htmlunit=false -Dtest.run.firefox=true -Dtest.run.chrome=true" export SELENIUM_HUB="-Dtest.selenium.hub.url= http://seleniumhub-demo.example.com/wd/hub" export SELENIUM_TARGET="-Dtest.target.server.url= http://petclinic-demo.example.com" export SELENIUM_OPTS="$SELENIUM_BROWSERS $SELENIUM_HUB $SELENIUM_TARGET" mvn --global-settings $OPENSHIFT_MAVEN_MIRROR failsafe:integration-test –Pselenium-tests $SELENIUM_OPTS Selenium Grid Hub Installation and Configuration Jenkins job: petclinic-build
  • The Usual Suspects – Jorge Hidalgo33 • What is the cargo-tomcat Maven profile doing? – Provisions a Tomcat 7 container before ITs are executed. – Tomcat runs with JaCoCo agent to gather coverage metrics. – Stops the Tomcat 7 container after ITs are executed. – Instead of Tomcat, we could use Glassfish, JBoss, Jetty... Pet Clinic Advanced Settings
  • The Usual Suspects – Jorge Hidalgo34 • What is the selenium-tests Maven profile doing? – Configures Maven’s Failsafe plug-in to execute Selenium integration tests during Maven’s integration-test phase. • Tell Sonar where to find JaCoCo report. Pet Clinic Advanced Settings
  • The Usual Suspects – Jorge Hidalgo35 • How are we dealing with different configuration for local and cloud execution? – By default, assumes local execution (so developers can test within their IDEs). Local settings also work in CI server. – Leverage openshift profile to copy configuration files over default ones when packaging the app for OpenShift. Pet Clinic Advanced Settings
  • The Usual Suspects – Jorge Hidalgo36 Selenium in Action
  • The Usual Suspects – Jorge Hidalgo37 Integration Tests Coverage as Seen by Sonar
  • The Usual Suspects – Jorge Hidalgo38 04: Performance/Load Tests with Apache JMeter
  • The Usual Suspects – Jorge Hidalgo39 Performance/Load Tests • Performance/Load Tests purpose is to verify application performance under stress: – Capable of handling multiple requests at the same time. – Well designed and implemented for concurrency. • Adding Load Tests to continuous integration allows for early detection of concurrency and performance issues. • Engineers/Architects may define thresholds to prevent deployments of non performant builds.
  • The Usual Suspects – Jorge Hidalgo40 Performance/Load Tests • Apache JMeter is a veteran framework / tool that allows to record, define and re-execute a wide variety of integration tests: – HTTP requests. – Web Services (SOAP, JAX-RPC) requests. – And many more. • JMeter is capable of running tests concurrently using multiple threads: – Locally, limited due to the amount of available resources. – Remotely, using JMeter Server on multiple test nodes (load injectors).
  • The Usual Suspects – Jorge Hidalgo41 • Update Maven target in Pet Clinic build job to launch JMeter tests and save them in a known place for Sonar: mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean verify -Pcargo-tomcat,selenium-tests mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean verify -Pcargo-tomcat,selenium-tests,jmeter-tests Adding JMeter Tests to Jenkins and Sonar Jenkins job: petclinic-build BEFORE Jenkins job: petclinic-build AFTER
  • The Usual Suspects – Jorge Hidalgo42 • What is the jmeter-tests Maven profile doing? – Configures Maven’s JMeter plug-in to execute JMeter tests during Maven’s integration-test phase. • Tell Sonar where to find JMeter report. Pet Clinic Advanced Settings
  • The Usual Suspects – Jorge Hidalgo43 JMeter Execution Report as Seen by Sonar
  • The Usual Suspects – Jorge Hidalgo44 05: In-Container Tests with Arquillian
  • The Usual Suspects – Jorge Hidalgo45 In-Container Tests • There is a gap between ‘strict’ unit tests (method or class level, in isolation) and functional integration tests (focused on user interactions). • Another type of tests, called unit-integration tests or in- container tests, bridge that gap and help testing application components: – Within a container, using the services the container provides. – Key to properly test features like: AOP transactions, AOP error handling, CDI bean injection, etc.
  • The Usual Suspects – Jorge Hidalgo46 In-Container Tests • Arquillian is a popular in-container test framework, recently born but with great traction in the community. • Key features of Arquillian include: – It’s open and modular – very easy to add extensions and new features. – Allows for cross-container tests with the help of Maven profiles – very convenient if you want to test your components in multiple containers (Weld, JBoss, Glassfish...). – Makes all the plumbing for you – prepares deployable units and puts them into the container of choice, either embedded (within the same JVM) or not (local / remote containers).
  • The Usual Suspects – Jorge Hidalgo47 Preparing Arquillian Test Application • Create a new JBoss application: >rhc app create -a arqtest -t jbossas-7 UUID: 0690fabf2e3549abb746a911d8087227 Git URL: ssh://0690fabf2e3549abb746a911d8087227@arqtest- demo.example.com/~/git/arqtest.git/ Public URL: http://arqtest-demo.example.com/ • Merge sources and configuration from test application. • Using some examples from Arquillian web site. Command line
  • The Usual Suspects – Jorge Hidalgo48 Preparing Arquillian Test Application • Add build job (and builder app) for Arqtest: >rhc app cartridge add -a arqtest -c jenkins- client-1.4 Job URL: https://jenkins-demo.example.com/job/arqtest- build/ Command line
  • The Usual Suspects – Jorge Hidalgo49 • Add this to Arqtest build job to launch Arquillian tests: mvn --global-settings $OPENSHIFT_MAVEN_MIRROR --version mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean package –Popenshift -DskipTests mvn --global-settings $OPENSHIFT_MAVEN_MIRROR --version mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean test -Parquillian-glassfish-embedded mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean package –Popenshift -DskipTests • Commit and push. Preparing Arquillian Test Application Jenkins job: arqtest-build BEFORE Jenkins job: arqtest-build AFTER
  • The Usual Suspects – Jorge Hidalgo50 Running Arqtest Build Job • Arqtest build job will execute tests with a Glassfish embedded container:
  • The Usual Suspects – Jorge Hidalgo51 The Usual Suspects – Wrap up • Platforms like OpenShift enable development teams to build and use powerful, best-of-breed development tools. • Easy access to the best software engineering practices. • With Origin, you get all the tools you need in the same box. Deploy it locally or in your in-house infrastructure. • Excellent for individuals and small development teams... • ...But if you work in a larger team and want more power, why not deploy the same in OpenShift?
  • The Usual Suspects – Jorge Hidalgo52 Q & A