Taming the Cloud Database with Apache jclouds
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Taming the Cloud Database with Apache jclouds

on

  • 347 views

 

Statistics

Views

Total Views
347
Views on SlideShare
344
Embed Views
3

Actions

Likes
0
Downloads
5
Comments
0

2 Embeds 3

http://www.slideee.com 2
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Taming the Cloud Database with Apache jclouds Presentation Transcript

  • 1. 1 Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14
  • 2. Introductions Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 2  Zack Shoylev  irc: zacksh twitter: @zackshoylev freenode #jclouds Also email!
  • 3. Introductions Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 3 Developer Relations Group sdk-support@rackspace.com
  • 4. Goals Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 4  cloud  jclouds  Create a database in the cloud  Abstractions and how to contribute
  • 5. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 5
  • 6. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 6  Buzzword  Networked and distributed computing  Variety of services  Compute (servers)  Storage (files)  Databases  Email  …
  • 7. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 7
  • 8. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 8  Public cloud (external provider)  Private cloud (internal deployment)  Hybrid cloud  Public + Private Rackspace example
  • 9. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 9 Your System S D K Cloud A P I VM VM VM  This can be an application server  Or your home machine  Or belong to one of your end users  Or a cloud VM  Or a smartphone jclouds
  • 10. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 10  Advantages  Metered pricing (pay as you go)  Project scalability (unlimited)  Safer (offsite/redundant)  Economies of scale  Expertise  Support
  • 11. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 11  Disadvantages  Less hardware control  Provider-controlled downtime  Virtualization (efficiency)  Provider lock-in  Provider-specific apis/sdks/features  Expensive to switch clouds or deploy locally  jclouds minimizes this disadvantage
  • 12. OpenStack Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 12  Cloud OS  Open-sourced by Rackspace and NASA (2010)  Free  Supported  AT&T, HP, IBM, Red Hat, Rackspace, Dell, Cisco, Intel, VMware, …
  • 13. Databases Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 13
  • 14. OpenStack Trove / Cloud Databases Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 14  Database specific advantages  Optimized by provider  Settings  Container virtualization  Automated software updates  Redundant data storage  Backup  Migration  Support
  • 15. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 15  Cloud SDK  Apache project  Java (and Clojure)  Easy  Portable  Cloud-agnostic  Community  Open source
  • 16. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 16  HTTP requests, responses, retries  Authentication and re-authentication  Pagination  Polling  Rate limits  Retries  Abstractions  Logging  Less code
  • 17. The Cloud Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 17  Can’t someone else do it?
  • 18. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 18
  • 19. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 19  Services  Storage  Compute  VM Images  Load Balancers  DNS  Databases  …  Providers  Openstack  Rackspace  HP  Amazon  Azure  …
  • 20. jclouds Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 20  Showcase database code  Best practices  Gotchas and workarounds  Abstractions
  • 21. Requirements Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 21  Maven 3  Java 6+  jclouds  Windows or Linux [etc..] (thanks Java!)
  • 22. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 22 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  • 23. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 23  Maven Dependencies  Selective subset  Versioning  jclouds-labs
  • 24. POM Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 24 https://github.com/jclouds/jclouds-examples/blob/master/rackspace/pom.xml <dependency> <groupId>org.apache.jclouds.provider</groupId> <artifactId>rackspace-clouddatabases-us</artifactId> <version>${jclouds.version}</version> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.25</version> </dependency>
  • 25. Logging Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 25 // This module is responsible for enabling logging Iterable<Module> modules = ImmutableSet.<Module> of(new SLF4JLoggingModule()); ComputeServiceContext context = ContextBuilder.newBuilder(provider) .credentials(username, apiKey) .modules(modules) // don't forget to add the modules to your context! .buildView(ComputeServiceContext.class); logback.xml <configuration scan="false"> … <appender name="WIREFILE" class="ch.qos.logback.core.FileAppender"> <file>target/test-data/jclouds-wire.log</file> <encoder> <Pattern>%d %-5p [%c] [%thread] %m%n</Pattern> </encoder> </appender>
  • 26. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 26 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  • 27. Initialize Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 27 private final TroveApi troveApi; troveApi = ContextBuilder .newBuilder("rackspace-clouddatabases- us") .credentials(username, apiKey) .buildApi(TroveApi.class);
  • 28. Initialize Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 28 private final FlavorApi flavorApi; flavorApi = troveApi.getFlavorApiForZone(ZONE);
  • 29. APIs Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 29  TroveApi  FlavorApi  InstanceApi  DatabaseApi  UserApi  Utils
  • 30. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 30 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  • 31. Create Instance Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 31 Flavor flavor = Iterables.getFirst(flavorApi.list(), null); … Instance instance = instanceApi.create(flavorId, size, name); flavor.getId() Volume size in GB
  • 32. Create Instance Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 32 Instance updatedInstance = awaitAvailable(instance, instanceApi);
  • 33. Create Instance Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 33 Instance updatedInstance = awaitAvailable(instance, instanceApi); Polls status, waits until ready. Will not retry by itself!
  • 34. Actual Code Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 34 InstanceApi instanceApi = api.getInstanceApiForZone(zone); for (int retries = 0; retries < 10; retries++) { Instance instance = null; try { instance = instanceApi.create(flavorId, size, name); } catch (Exception e) { Uninterruptibles.sleepUninterruptibly(15, TimeUnit.SECONDS); logger.error(Arrays.toString(e.getStackTrace())); continue; } Instance updatedInstance = awaitAvailable(instance, instanceApi); if (updatedInstance != null) { return updatedInstance; } instanceApi.delete(instance.getId()); InstancePredicates.awaitDeleted(instanceApi).apply(instance); } return null;
  • 35. Actual Code Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 35 Utils. getWorkingInstance( String zone, String name, String flavorId, int size)
  • 36. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 36 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  • 37. Create Database Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 37 troveApi = ContextBuilder.newBuilder(PROVIDER) .credentials(username, apiKey) .buildApi(TroveApi.class); instanceApi = troveApi.getInstanceApiForZone(ZONE); databaseApi = troveApi .getDatabaseApiForZoneAndInstance(ZONE,getInstance().getId()); getSomeApiForXandYandZ – fairly common in jclouds
  • 38. Create Database Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 38 boolean result = databaseApi.create(NAME);
  • 39. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 39 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  • 40. Create User Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 40 userApi = troveApi .getUserApiForZoneAndInstance(ZONE, instance.getId()); boolean result = userApi.create(UNAME, PASSWORD, DBNAME); User name User password Database name
  • 41. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 41 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  • 42. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 42 Load balancer needed! CreateLoadBalancer createLB = CreateLoadBalancer.builder() .name(NAME) .protocol("MYSQL") .port(3306) .algorithm(LoadBalancer.Algorithm.RANDOM) .nodes(addNodes) .virtualIPType(VirtualIP.Type.PUBLIC) .build();
  • 43. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 43 loadBalancer = lbApi.create(createLB);
  • 44. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 44 StringBuilder connString = new StringBuilder(); connString.append("jdbc:mysql://"); connString .append(getVirtualIPv4(getLb().getVirtualIPs())); connString.append("/"); connString.append(DBNAME); connString.append("?user="); connString.append(UNAME); connString.append("&password="); connString.append(PASSWORD);
  • 45. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 45 Connection conn = DriverManager.getConnection(connString.toString());
  • 46. Test Connection Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 46 Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("SELECT 3+5"); rs.first(); System.out.format(" 3+5 is %s%n", rs.getInt(1));
  • 47. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 47  jclouds examples  https://github.com/jclouds/jclouds-examples  More docs  http://jclouds.incubator.apache.org/documentation/  http://javadocs.jclouds.cloudbees.net/  Maven alternatives?  http://jclouds.incubator.apache.org/documentation/use rguide/installation-guide/  Contribute!  https://wiki.apache.org/jclouds/How%20to%20Contrib ute
  • 48. Java Project Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 48  Compile  javac -classpath "lib/*:src/main/java/:src/main/resources/" src/main/java/org/jclouds/examples/rackspace/*.java  Run  java -classpath "lib/*:src/main/java/:src/main/resources/" org.jclouds.examples.rackspace.clouddatabases.CreateInstance username apikey
  • 49. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 49  TroveApi  Backup Extension  Settings Extension  Abstraction layer
  • 50. Architecture Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 50 Cloud Databases Trove jclouds-database Increasing level of abstraction Provider settings: Endpoint, etc. Implementation code Abstractions
  • 51. Architecture Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 51 Cloud Databases Trove jclouds-database Increasing level of abstraction Provider settings: Endpoint, etc. Implementation code Abstractions Not implemented
  • 52. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 52  TroveApi  Backup Extension  Settings Extension  Abstraction layer Rackspace Microsoft Amazon CouchDB Salesforce
  • 53. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 53  TroveApi  Backup Extension  Settings Extension  Abstraction layer Rackspace Microsoft Amazon CouchDB Salesforce
  • 54. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 54
  • 55. Going forward Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 55 Collaborate! http://jclouds.apache.org/ http://jclouds.apache.org/community/
  • 56. Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14 56 Thank you! http://developer.rackspace.com sdk-support@rackspace.com Zack Shoylev Software Developer zacksh #jclouds @zackshoylev