Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Developing in the Cloud

836 views

Published on

Configuring and maintaining a continuous integration environment is quite a bit of work. It requires ongoing resources both in terms of manpower and hardware infrastructure. As an application evolves so does the number of ongoing projects. The challenge is creating a scalable continuous integration environment which does not impede development and can handle the complexities of Java EE testing. This session covers how to setup and configure a cloud-based continuous integration environment for Java EE applications.

The presentation will focus on demonstrating how to use Atlassian Bamboo running on AWS to build and test a Maven/Gradle Java EE project that uses Arquillian for testing. Topics that will be covered include creating a custom AWS VM for use with Bamboo, creating an Amazon VPC (Virtual Private Cloud) along with test database using Amazon RDS. The presentation will delve into the specifics of testing EJBs, WebSocket endpoints, RESTful web services, as well as performing load testing in this environment. Security, cost control, and build monitoring will be covered as well.

Published in: Software
  • Be the first to comment

Developing in the Cloud

  1. 1. DEVELOPING IN THE CLOUD Ryan Cuprak
  2. 2. Developing in the Cloud Agenda Continuous Integration in Cloud Java EE Testing
  3. 3. Why Develop in the Cloud? Topics Covered Today Version Control Continuous Integration Issue Tracking/Proces s Integration Testing Java EE
  4. 4. Why Develop in the Cloud? • Processes are audited by customers (GMP) • All code changes are tracked • Rigorous issue life-cycle • Dashboard for tracking development progress • Continuous integration setup: • Unit tests/Integration tests/system tests • Database setup • Automated process for testing deployment Work Development Processes I want this for home!
  5. 5. Why Develop in the Cloud? Continuous Integration Challenges Hardware Software Maintenance/Sec urity Configuration Networking Cost
  6. 6. Why Develop in the Cloud? • Constructing a robust development environment is: • Expensive • Hard • Time consuming • Cloud services simplifies CI environment: • Easy to setup… • Relatively cheap… • Low maintenance Cloud Benefits
  7. 7. Developing in the Cloud • Self-Hosted in cloud • Create VMs and install developer tools (version control/bug trackers/etc.) • Manage all aspects of configuration/security • Costs: data storage and usage • Vendor Hosted in cloud • Vendor provides hosted solution – fully configured/managed. • Little maintenance or configuration • Costs: monthly fees, storage and usage • Hybrid • Some tools are hosted locally (example: code repository) Cloud Models
  8. 8. Why Develop in the Cloud? • Minimize time spent configuring/managing tools. • Scale projects as resource needs change. • Easily support distributed development. • Use pre-configured integrated tool stack. • Build/test for other platforms requiring special hardware/tools. • iOS • Windows mobile • Scalability testing • Easily create a ‘clean environment’ Benefits of Cloud Development
  9. 9. Developing in the Cloud Pieces of the Puzzle Version Control Continuous Integration Issue Management Test Environment(s)
  10. 10. Developing in the Cloud Vendor Price (monthly) github $0-$50 BitBucket $0-$200 Java.net Free Sourceforge Free Beanstalk $15-$200 CodePlex Free Version Control * github and bitbucket are the leading vendors.
  11. 11. Developing in the Cloud Vendor Opensource Paid (monthly) Atlassian N/A $10-$1000 CloudBees N/A $60-$200 Travis CI Free $129-$489 AppVeyor None $19-$99 Shippable Free $12-Custom CodeShip Free $49-$1999 Semaphore Free $0-$199 drone.io Free $25-$49 Snap CI Free $0-$180 Continuous Integration Cloud Vendors
  12. 12. Developing in the Cloud Vendor Monthly Price JIRA $10-$1000 YouTrack Free-$500 BugHost $29-$159 Bontq $9-$149 FogBugz $18-$540 CloudForge $2-$10 (per user) devZing (Bugzilla hosting) $15 Issue Trackers Many more…
  13. 13. Developing in the Cloud • Old Java standbys: • java.net • sourceforge.net • Project type: open source code • Features: • Wikis • Blogging • Issue tracking • No continuous integration support Old Services
  14. 14. Developing in the Cloud Considerations Issue Tracking Continuous Integration Version Control How do these integrate?
  15. 15. Developing in the Cloud • Open source or private? • Paid or free account? • Use integrated issue system or separate? • Integration with tools. • Ability to customize environment • Operating systems • Databases • Test installation scripts • Data migrations Additional Considerations
  16. 16. Developing in the Cloud • Private projects • Ability to switch from private to public and vice versa • Version control: git • Continuous integration Amazon • Projects: • Maven/Gradle builds • Junit/Jasmine for unit testing • NetBeans • IDE • IntelliJ • NetBeans • SourceTree • xCode Requirements
  17. 17. Developing in the Cloud Cloud services: • Github – version control ($7 / month) • Unlimited public repositories • 5 private repositories • Bamboo – Continuous Integration ($10 / month) • 10 jobs / 1 agent • Builds are executing on AWS EC2 instances. • JIRA – issue tracking ($10 / month) • 10 users • Amazon Web Services • VMs/databases Total cost: $27 / month (excluding AWS usage). What cloud services did I choose?
  18. 18. Developing in the Cloud • Jenkins • More popular than Bamboo. • Easier to configure than Bamboo. • Free • Large pool of plugins (1000+) • Bamboo • Integrated with JIRA. • Automated merging (git/mercurial) • CI & CD pipelines • Build agent • Requires JIRA for cloud account Bamboo versus Jenkins
  19. 19. Developing in the Cloud Puzzle Pieces
  20. 20. Developing in the Cloud Getting Started Create github account Create JIRA/Bamboo account Create AWS Account http://tinyurl.com/leo7rop http://aws.amazon.com http://github.com
  21. 21. Developing in the Cloud Amazon Web Services (AWS)
  22. 22. Developing in the Cloud Amazon Web Services EC2 RDS VPC
  23. 23. Developing in the Cloud • EC2 – Elastic Cloud Computing • VMs == AMI (Amazon Machine Image) AWS Core Concepts – EC2 Prices Sampling Name vCPU ECU Memory (GiB) Storage Hourly Fee t2.micro 1 Variable 1 EBS Only $0.013 t2.small 1 Variable 2 EBS Only $0.026 t2.medium 1 Variable 4 EBS Only $0.052 m3.medium 3 3.75 1 x 4 SSD $0.070 m3.large 2 6.5 7.5 1 x 32 SSD $0.140 m3.xlarge 4 13 15 2 x 40 SSD $0.280 m2.2xlarge 8 26 30 2 x 80 $0.560
  24. 24. Developing in the Cloud • Amazon provides a large library of AMIs. • Bamboo has pre-configured AMIs with Bamboo agent installed. • Instances are accessed via ssh (download keys) • Pick data center when launching an instance. AWS Core Concepts – EC2
  25. 25. Developing in the Cloud • VPC = Virtual Private Cloud • Complete control over virtual networking environment: • IP address range • Subnets • Routing tables • Network gateways • VPC can be connected to a corporate data center • AMI instances can be launched into a VPC. • Databases to a VPC. AWS Core Concepts - VPC
  26. 26. Developing in the Cloud AWS Core Concepts – VPC Wizard
  27. 27. Developing in the Cloud AWS Core Concepts – VPC Wizard
  28. 28. Developing in the Cloud AWS Core Concepts – Wizard VPC
  29. 29. Developing in the Cloud AWS Core Concepts – Wizard VPC
  30. 30. Developing in the Cloud AWS Core Concepts – Wizard VPC
  31. 31. Developing in the Cloud • Amazon RDS – Amazon Relational Database Service • Supported databases: • MySQL • Oracle • Microsoft SQL Server • PostgreSQL • Provisioned using SSD, IOPS Storage, Magnetic. • Automated failover, backups, multiple-zones, mirroring • Can attach to a VPC. Amazon RDS
  32. 32. Developing in the Cloud Amazon RDS
  33. 33. Developing in the Cloud Amazon RDS
  34. 34. Developing in the Cloud Amazon RDS
  35. 35. Developing in the Cloud Amazon RDS
  36. 36. Developing in the Cloud • After database starts-up, log into a VM. • Run: “sudo yum postgresql” to install PostgreSQL client. Amazon RDS
  37. 37. Developing in the Cloud Term Definition Project Provides reporting across all plans in a project. Plan Specifies default repository, how build is triggered, dependencies between other plans, notifications, permissions, definition of plan variables. Stage Processes its jobs in parallel on multiple agents. Can produce artifacts for another stage. Job Processes a series of one or more tasks that are run sequentially on the same agent. Controls which tasks are performed. Defines artifacts that the build will produce. Task Small discrete unit of work (code checkout, execute Maven goal, etc.). Is run sequentially within a job on a Bamboo working directory. Bamboo Core Concepts
  38. 38. Developing in the Cloud Bamboo Concepts Plan Stage Stage Stage Job Task Task Job Task Job Task Task Task Job Task Job Task
  39. 39. Developing in the Cloud Bamboo
  40. 40. Developing in the Cloud Bamboo
  41. 41. Developing in the Cloud Bamboo – Configure Plan
  42. 42. Developing in the Cloud Bamboo – Configure Plan
  43. 43. Developing in the Cloud Bamboo – Configure Tasks
  44. 44. Developing in the Cloud Bamboo – Plan Configuration These environment variables are passed to the JVM.
  45. 45. Developing in the Cloud Bamboo – test-persistence.xml – using environment properties
  46. 46. Developing in the Cloud Testing locally /.m2/settings.xml
  47. 47. Developing in the Cloud Bamboo – Configure Tasks
  48. 48. Developing in the Cloud • Bamboo is central management server which schedules/coordinates all work. With cloud, Bamboo management server is hosted at Atlassian. • Bamboo Agent – a service that can run build jobs. • Elastic Agent – a remote Bamboo agent that runs on Amazon EC2. • Capability – feature of an agent, such as JDK installed, version of Maven available, etc. Bamboo Overview
  49. 49. Developing in the Cloud • Bamboo builds Bamboo - Building Running Bamboo Agent git pull Run job Build and test
  50. 50. Developing in the Cloud AMI
  51. 51. Developing in the Cloud Customizing Bamboo AMI Find Bambo AMI Launch instance Customize instance Take Snapshot Launch customized instance
  52. 52. Developing in the Cloud • Plugins available for IDEs (NetBeans/IntelliJ/Eclipse) • SourceTree integration • Benefits of integration • git pushes trigger issue state transitions. JIRA Integration
  53. 53. Developing in the Cloud JIRA Integration - Configuration
  54. 54. Developing in the Cloud JIRA Integration – Configuration github
  55. 55. Developing in the Cloud JIRA Integration – Configuration github
  56. 56. Developing in the Cloud JIRA Integration – Configuration
  57. 57. Developing in the Cloud JIRA Integration - NetBeans
  58. 58. Developing in the Cloud Testing challenges: • RESTful webservices • WebSocket endpoints • Security • Scalability • JavaScript code • JavaScript code using AJAX (with server) • Real data, bad data, etc. Testing Java EE
  59. 59. Developing in the Cloud Example Test Environment (VPC) App Server A App Server B App Server C Load Balacner Test Client A Test Client B Test Client C AWS
  60. 60. Developing in the Cloud • Bamboo + AWS = full testing in the cloud. • Create custom environments for simulating a cluster. • Test load balancing setup. • Stress test server under heavy load. • Testing: unit testing, integration testing, load testing. Java EE + Bamboo + AWS
  61. 61. Developing in the Cloud • Project from JBoss. • Unit testing in the container (GlassFish/WebLogic/etc.) What is Arquillian? Container Java EE app server Servlet Container Weld SE ArquillianTest Case Unit testing framework (Junit/TestNG) ShrinkWrap
  62. 62. Developing in the Cloud Java EE Application - ctjava ctcore ctweb migrate ctadmin Java EE 7 Application – Maven Based ctmobile https://github.com/rcuprak/ctjava.git
  63. 63. Developing in the Cloud Project Overview • Java EE 7 web application • GlassFish 4.1 • PostgreSQL • Maven based build • Arquillian unit testing Setup • Modularized architecture • Data model and DAOs in ctcore • Unit and integration testing Java EE Application - ctjava
  64. 64. Developing in the Cloud Testing ctcore - Overview • Contains data model and DAOs. • No EJBs • @Dao and @DefaultDatabase are CDI Qualifiers.
  65. 65. Developing in the Cloud Testing ctcore – Setup for DAO testing
  66. 66. Developing in the Cloud Testing ctcore – Configuring Weld
  67. 67. Developing in the Cloud Testing ctcore – Test Implementation
  68. 68. Developing in the Cloud Testing ctcore – pom.xml dependencies Dependencies: • Weld • EclipseLink • PostgreSQL Scope: Test
  69. 69. Developing in the Cloud Testing ctweb – Integration Testing with EJBs Test runs in embedded GlassFish container.
  70. 70. Developing in the Cloud Testing ctweb – Integration Testing with EJBs
  71. 71. Developing in the Cloud • Testing RESTful webservices and WebSockets easier with Arquillian. • Best practices: • Test API first using Java • Test using AJAX using Jasmine + Arquillian + REST & WebSockets
  72. 72. Developing in the Cloud • Java API for WebSockets (JSR-224) • High level declarative API for WebSocket • API similar to JavaScript WebSocket API • Both client and server-side • Important pieces: • Annotations for annotated endpoints: @ServerEndpoint, @OnOpen, @OnClose, @OnMessage, @OnError • Session object – used to send messages. • RemoteEndpoint object – used to sent messages to a client • MessageHandler – interface used by programmatic endpoints. • Pluggable and extensible Encoders, decoders, sub-protocols JAX-WS WebSockets Review
  73. 73. Developing in the Cloud • JAX-RS 2.0 is the REST development API for Java • Major upgrade with Java EE 7. (JSR-339) • Client API, Aysnc, Validation, Filters/Handlers, Interceptors, and Content Negotiation. • Server and client • Integrates with JAXB and Bean Validation • Annotation based, declarative • @Path, @GET, @POST, @PUT, @DELETE, • Pluggable and extensible. JAX-RS- RS Review
  74. 74. Developing in the Cloud Testing ctweb – Registration Service Test Setup
  75. 75. Developing in the Cloud Testing ctweb – Registration Service Test -Java
  76. 76. Developing in the Cloud Testing ctweb – Registration Service Test (Setup) - JavaScript
  77. 77. Developing in the Cloud Testing ctweb – Registration Service Test - JavaScript
  78. 78. Developing in the Cloud Testing ctweb – Registration Service Test (pom) - JavaScript
  79. 79. Developing in the Cloud Testing ctweb – WebSockets
  80. 80. Developing in the Cloud Testing ctweb – WebSockets
  81. 81. Developing in the Cloud Cloud tools + Java EE = Garage Startup! Conclusion
  82. 82. Q&A Twitter: ctjava Email: ryan.cuprak@3ds.com Blog: http://cuprak.info Contact

×