Developing in the Cloud

639 views

Published on

Configuring and maintaining a continuous integration environment is quite a bit of work. It requires ongoing resources both in terms of manpower and hardware infrastructure. As an application evolves so does the number of ongoing projects. The challenge is creating a scalable continuous integration environment which does not impede development and can handle the complexities of Java EE testing. This session covers how to setup and configure a cloud-based continuous integration environment for Java EE applications.

The presentation will focus on demonstrating how to use Atlassian Bamboo running on AWS to build and test a Maven/Gradle Java EE project that uses Arquillian for testing. Topics that will be covered include creating a custom AWS VM for use with Bamboo, creating an Amazon VPC (Virtual Private Cloud) along with test database using Amazon RDS. The presentation will delve into the specifics of testing EJBs, WebSocket endpoints, RESTful web services, as well as performing load testing in this environment. Security, cost control, and build monitoring will be covered as well.

Published in: Software
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
639
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
18
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • So what cloud hosting services did I choose?
  • Developing in the Cloud

    1. 1. DEVELOPING IN THE CLOUD Ryan Cuprak
    2. 2. Developing in the Cloud Agenda Continuous Integration in Cloud Java EE Testing
    3. 3. Why Develop in the Cloud? Topics Covered Today Version Control Continuous Integration Issue Tracking/Proces s Integration Testing Java EE
    4. 4. Why Develop in the Cloud? • Processes are audited by customers (GMP) • All code changes are tracked • Rigorous issue life-cycle • Dashboard for tracking development progress • Continuous integration setup: • Unit tests/Integration tests/system tests • Database setup • Automated process for testing deployment Work Development Processes I want this for home!
    5. 5. Why Develop in the Cloud? Continuous Integration Challenges Hardware Software Maintenance/Sec urity Configuration Networking Cost
    6. 6. Why Develop in the Cloud? • Constructing a robust development environment is: • Expensive • Hard • Time consuming • Cloud services simplifies CI environment: • Easy to setup… • Relatively cheap… • Low maintenance Cloud Benefits
    7. 7. Developing in the Cloud • Self-Hosted in cloud • Create VMs and install developer tools (version control/bug trackers/etc.) • Manage all aspects of configuration/security • Costs: data storage and usage • Vendor Hosted in cloud • Vendor provides hosted solution – fully configured/managed. • Little maintenance or configuration • Costs: monthly fees, storage and usage • Hybrid • Some tools are hosted locally (example: code repository) Cloud Models
    8. 8. Why Develop in the Cloud? • Minimize time spent configuring/managing tools. • Scale projects as resource needs change. • Easily support distributed development. • Use pre-configured integrated tool stack. • Build/test for other platforms requiring special hardware/tools. • iOS • Windows mobile • Scalability testing • Easily create a ‘clean environment’ Benefits of Cloud Development
    9. 9. Developing in the Cloud Pieces of the Puzzle Version Control Continuous Integration Issue Management Test Environment(s)
    10. 10. Developing in the Cloud Vendor Price (monthly) github $0-$50 BitBucket $0-$200 Java.net Free Sourceforge Free Beanstalk $15-$200 CodePlex Free Version Control * github and bitbucket are the leading vendors.
    11. 11. Developing in the Cloud Vendor Opensource Paid (monthly) Atlassian N/A $10-$1000 CloudBees N/A $60-$200 Travis CI Free $129-$489 AppVeyor None $19-$99 Shippable Free $12-Custom CodeShip Free $49-$1999 Semaphore Free $0-$199 drone.io Free $25-$49 Snap CI Free $0-$180 Continuous Integration Cloud Vendors
    12. 12. Developing in the Cloud Vendor Monthly Price JIRA $10-$1000 YouTrack Free-$500 BugHost $29-$159 Bontq $9-$149 FogBugz $18-$540 CloudForge $2-$10 (per user) devZing (Bugzilla hosting) $15 Issue Trackers Many more…
    13. 13. Developing in the Cloud • Old Java standbys: • java.net • sourceforge.net • Project type: open source code • Features: • Wikis • Blogging • Issue tracking • No continuous integration support Old Services
    14. 14. Developing in the Cloud Considerations Issue Tracking Continuous Integration Version Control How do these integrate?
    15. 15. Developing in the Cloud • Open source or private? • Paid or free account? • Use integrated issue system or separate? • Integration with tools. • Ability to customize environment • Operating systems • Databases • Test installation scripts • Data migrations Additional Considerations
    16. 16. Developing in the Cloud • Private projects • Ability to switch from private to public and vice versa • Version control: git • Continuous integration Amazon • Projects: • Maven/Gradle builds • Junit/Jasmine for unit testing • NetBeans • IDE • IntelliJ • NetBeans • SourceTree • xCode Requirements
    17. 17. Developing in the Cloud Cloud services: • Github – version control ($7 / month) • Unlimited public repositories • 5 private repositories • Bamboo – Continuous Integration ($10 / month) • 10 jobs / 1 agent • Builds are executing on AWS EC2 instances. • JIRA – issue tracking ($10 / month) • 10 users • Amazon Web Services • VMs/databases Total cost: $27 / month (excluding AWS usage). What cloud services did I choose?
    18. 18. Developing in the Cloud • Jenkins • More popular than Bamboo. • Easier to configure than Bamboo. • Free • Large pool of plugins (1000+) • Bamboo • Integrated with JIRA. • Automated merging (git/mercurial) • CI & CD pipelines • Build agent • Requires JIRA for cloud account Bamboo versus Jenkins
    19. 19. Developing in the Cloud Puzzle Pieces
    20. 20. Developing in the Cloud Getting Started Create github account Create JIRA/Bamboo account Create AWS Account http://tinyurl.com/leo7rop http://aws.amazon.com http://github.com
    21. 21. Developing in the Cloud Amazon Web Services (AWS)
    22. 22. Developing in the Cloud Amazon Web Services EC2 RDS VPC
    23. 23. Developing in the Cloud • EC2 – Elastic Cloud Computing • VMs == AMI (Amazon Machine Image) AWS Core Concepts – EC2 Prices Sampling Name vCPU ECU Memory (GiB) Storage Hourly Fee t2.micro 1 Variable 1 EBS Only $0.013 t2.small 1 Variable 2 EBS Only $0.026 t2.medium 1 Variable 4 EBS Only $0.052 m3.medium 3 3.75 1 x 4 SSD $0.070 m3.large 2 6.5 7.5 1 x 32 SSD $0.140 m3.xlarge 4 13 15 2 x 40 SSD $0.280 m2.2xlarge 8 26 30 2 x 80 $0.560
    24. 24. Developing in the Cloud • Amazon provides a large library of AMIs. • Bamboo has pre-configured AMIs with Bamboo agent installed. • Instances are accessed via ssh (download keys) • Pick data center when launching an instance. AWS Core Concepts – EC2
    25. 25. Developing in the Cloud • VPC = Virtual Private Cloud • Complete control over virtual networking environment: • IP address range • Subnets • Routing tables • Network gateways • VPC can be connected to a corporate data center • AMI instances can be launched into a VPC. • Databases to a VPC. AWS Core Concepts - VPC
    26. 26. Developing in the Cloud AWS Core Concepts – VPC Wizard
    27. 27. Developing in the Cloud AWS Core Concepts – VPC Wizard
    28. 28. Developing in the Cloud AWS Core Concepts – Wizard VPC
    29. 29. Developing in the Cloud AWS Core Concepts – Wizard VPC
    30. 30. Developing in the Cloud AWS Core Concepts – Wizard VPC
    31. 31. Developing in the Cloud • Amazon RDS – Amazon Relational Database Service • Supported databases: • MySQL • Oracle • Microsoft SQL Server • PostgreSQL • Provisioned using SSD, IOPS Storage, Magnetic. • Automated failover, backups, multiple-zones, mirroring • Can attach to a VPC. Amazon RDS
    32. 32. Developing in the Cloud Amazon RDS
    33. 33. Developing in the Cloud Amazon RDS
    34. 34. Developing in the Cloud Amazon RDS
    35. 35. Developing in the Cloud Amazon RDS
    36. 36. Developing in the Cloud • After database starts-up, log into a VM. • Run: “sudo yum postgresql” to install PostgreSQL client. Amazon RDS
    37. 37. Developing in the Cloud Term Definition Project Provides reporting across all plans in a project. Plan Specifies default repository, how build is triggered, dependencies between other plans, notifications, permissions, definition of plan variables. Stage Processes its jobs in parallel on multiple agents. Can produce artifacts for another stage. Job Processes a series of one or more tasks that are run sequentially on the same agent. Controls which tasks are performed. Defines artifacts that the build will produce. Task Small discrete unit of work (code checkout, execute Maven goal, etc.). Is run sequentially within a job on a Bamboo working directory. Bamboo Core Concepts
    38. 38. Developing in the Cloud Bamboo Concepts Plan Stage Stage Stage Job Task Task Job Task Job Task Task Task Job Task Job Task
    39. 39. Developing in the Cloud Bamboo
    40. 40. Developing in the Cloud Bamboo
    41. 41. Developing in the Cloud Bamboo – Configure Plan
    42. 42. Developing in the Cloud Bamboo – Configure Plan
    43. 43. Developing in the Cloud Bamboo – Configure Tasks
    44. 44. Developing in the Cloud Bamboo – Plan Configuration These environment variables are passed to the JVM.
    45. 45. Developing in the Cloud Bamboo – test-persistence.xml – using environment properties
    46. 46. Developing in the Cloud Testing locally /.m2/settings.xml
    47. 47. Developing in the Cloud Bamboo – Configure Tasks
    48. 48. Developing in the Cloud • Bamboo is central management server which schedules/coordinates all work. With cloud, Bamboo management server is hosted at Atlassian. • Bamboo Agent – a service that can run build jobs. • Elastic Agent – a remote Bamboo agent that runs on Amazon EC2. • Capability – feature of an agent, such as JDK installed, version of Maven available, etc. Bamboo Overview
    49. 49. Developing in the Cloud • Bamboo builds Bamboo - Building Running Bamboo Agent git pull Run job Build and test
    50. 50. Developing in the Cloud AMI
    51. 51. Developing in the Cloud Customizing Bamboo AMI Find Bambo AMI Launch instance Customize instance Take Snapshot Launch customized instance
    52. 52. Developing in the Cloud • Plugins available for IDEs (NetBeans/IntelliJ/Eclipse) • SourceTree integration • Benefits of integration • git pushes trigger issue state transitions. JIRA Integration
    53. 53. Developing in the Cloud JIRA Integration - Configuration
    54. 54. Developing in the Cloud JIRA Integration – Configuration github
    55. 55. Developing in the Cloud JIRA Integration – Configuration github
    56. 56. Developing in the Cloud JIRA Integration – Configuration
    57. 57. Developing in the Cloud JIRA Integration - NetBeans
    58. 58. Developing in the Cloud Testing challenges: • RESTful webservices • WebSocket endpoints • Security • Scalability • JavaScript code • JavaScript code using AJAX (with server) • Real data, bad data, etc. Testing Java EE
    59. 59. Developing in the Cloud Example Test Environment (VPC) App Server A App Server B App Server C Load Balacner Test Client A Test Client B Test Client C AWS
    60. 60. Developing in the Cloud • Bamboo + AWS = full testing in the cloud. • Create custom environments for simulating a cluster. • Test load balancing setup. • Stress test server under heavy load. • Testing: unit testing, integration testing, load testing. Java EE + Bamboo + AWS
    61. 61. Developing in the Cloud • Project from JBoss. • Unit testing in the container (GlassFish/WebLogic/etc.) What is Arquillian? Container Java EE app server Servlet Container Weld SE ArquillianTest Case Unit testing framework (Junit/TestNG) ShrinkWrap
    62. 62. Developing in the Cloud Java EE Application - ctjava ctcore ctweb migrate ctadmin Java EE 7 Application – Maven Based ctmobile https://github.com/rcuprak/ctjava.git
    63. 63. Developing in the Cloud Project Overview • Java EE 7 web application • GlassFish 4.1 • PostgreSQL • Maven based build • Arquillian unit testing Setup • Modularized architecture • Data model and DAOs in ctcore • Unit and integration testing Java EE Application - ctjava
    64. 64. Developing in the Cloud Testing ctcore - Overview • Contains data model and DAOs. • No EJBs • @Dao and @DefaultDatabase are CDI Qualifiers.
    65. 65. Developing in the Cloud Testing ctcore – Setup for DAO testing
    66. 66. Developing in the Cloud Testing ctcore – Configuring Weld
    67. 67. Developing in the Cloud Testing ctcore – Test Implementation
    68. 68. Developing in the Cloud Testing ctcore – pom.xml dependencies Dependencies: • Weld • EclipseLink • PostgreSQL Scope: Test
    69. 69. Developing in the Cloud Testing ctweb – Integration Testing with EJBs Test runs in embedded GlassFish container.
    70. 70. Developing in the Cloud Testing ctweb – Integration Testing with EJBs
    71. 71. Developing in the Cloud • Testing RESTful webservices and WebSockets easier with Arquillian. • Best practices: • Test API first using Java • Test using AJAX using Jasmine + Arquillian + REST & WebSockets
    72. 72. Developing in the Cloud • Java API for WebSockets (JSR-224) • High level declarative API for WebSocket • API similar to JavaScript WebSocket API • Both client and server-side • Important pieces: • Annotations for annotated endpoints: @ServerEndpoint, @OnOpen, @OnClose, @OnMessage, @OnError • Session object – used to send messages. • RemoteEndpoint object – used to sent messages to a client • MessageHandler – interface used by programmatic endpoints. • Pluggable and extensible Encoders, decoders, sub-protocols JAX-WS WebSockets Review
    73. 73. Developing in the Cloud • JAX-RS 2.0 is the REST development API for Java • Major upgrade with Java EE 7. (JSR-339) • Client API, Aysnc, Validation, Filters/Handlers, Interceptors, and Content Negotiation. • Server and client • Integrates with JAXB and Bean Validation • Annotation based, declarative • @Path, @GET, @POST, @PUT, @DELETE, • Pluggable and extensible. JAX-RS- RS Review
    74. 74. Developing in the Cloud Testing ctweb – Registration Service Test Setup
    75. 75. Developing in the Cloud Testing ctweb – Registration Service Test -Java
    76. 76. Developing in the Cloud Testing ctweb – Registration Service Test (Setup) - JavaScript
    77. 77. Developing in the Cloud Testing ctweb – Registration Service Test - JavaScript
    78. 78. Developing in the Cloud Testing ctweb – Registration Service Test (pom) - JavaScript
    79. 79. Developing in the Cloud Testing ctweb – WebSockets
    80. 80. Developing in the Cloud Testing ctweb – WebSockets
    81. 81. Developing in the Cloud Cloud tools + Java EE = Garage Startup! Conclusion
    82. 82. Q&A Twitter: ctjava Email: ryan.cuprak@3ds.com Blog: http://cuprak.info Contact

    ×