0
1
Taming the Cloud Database
with
Apache jclouds
http://j.mp/acjdb14
Introductions
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
2
➢ Zack Shoylev
➢ irc: zacksh twitter: @...
Introductions
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
3
Developer Relations Group
https://devel...
Goals
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
4
➢ jclouds
➢ Create a database in the cloud
➢ Ab...
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
5
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
6
➢ Buzzword
➢ Networked and distributed comp...
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
7
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
8
➢ Public cloud (external provider)
➢ Privat...
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
9
Your
System
S
D
K
Cloud
AP
I
VM
VM
VM
➢ Thi...
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
10
➢ Advantages
➢ Metered pricing (pay as you...
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
11
➢ Disadvantages
➢ Less hardware control
➢ ...
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
12
➢ Can’t someone else do it?
The Cloud
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
13
➢ Database specific advantages
➢ Optimized...
OpenStack
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
14
➢ Cloud OS
➢ Open-sourced by Rackspace and...
jclouds
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
15
➢ Cloud SDK
➢ Apache project
➢ Java (and Clo...
jclouds
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
16
➢ HTTP requests, responses, retries
➢ Authen...
jclouds
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
17
jclouds
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
18
➢ Services
➢ Storage
➢ Compute
➢ VM Images
➢...
jclouds
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
19
AWS aws-ec2 US-VA,US-CA,IE,SG
Bluelock bluel...
jclouds
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
20
➢ Showcase database code
➢ Best practices
➢ ...
Requirements
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
21
➢ Maven 3
➢ Java 6+
➢ jclouds
➢ Windows...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
22
Skeleton
Project
Create User
Create Ins...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
23
➢ Maven Dependencies
➢ Selective subset...
POM
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
24
https://github.com/jclouds/jclouds-examples/blob...
Logging
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
25
// This module is responsible for enabling l...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
26
Skeleton
Project
Create User
Create Ins...
Initialize
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
27
private final TroveApi troveApi;
troveApi...
Architecture
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
28
Cloud Databases
Trove
jclouds-database
...
Architecture
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
29
Cloud Databases
Trove
jclouds-database
...
Initialize
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
30
private final FlavorApi flavorApi;
flavor...
APIs
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
31
➢ TroveApi
➢ FlavorApi
➢ InstanceApi
➢ Database...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
32
Skeleton
Project
Create User
Create Ins...
Create Instance
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
33
Flavor flavor = Iterables.getFirst(f...
Create Instance
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
34
Instance updatedInstance = awaitAvai...
Create Instance
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
35
Instance updatedInstance = awaitAvai...
Actual Code
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
36
InstanceApi instanceApi = api.getInstanc...
Actual Code
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
37
Utils.
getWorkingInstance(
String zone,
...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
38
Skeleton
Project
Create User
Create Ins...
Create Database
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
39
troveApi = ContextBuilder.newBuilder...
Create Database
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
40
boolean result = databaseApi.create(...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
41
Skeleton
Project
Create User
Create Ins...
Create User
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
42
userApi = troveApi
.getUserApiForZoneAnd...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
43
Skeleton
Project
Create User
Create Ins...
Test Connection
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
44
Load balancer needed!
CreateLoadBala...
Test Connection
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
45
loadBalancer = lbApi.create(createLB...
Test Connection
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
46
StringBuilder connString = new Strin...
Test Connection
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
47
Connection conn =
DriverManager.getC...
Test Connection
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
48
Statement stmt = conn.createStatemen...
Going forward
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
49
➢ jclouds examples
➢ https://github.co...
Java Project
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
50
➢ Compile
➢ javac -classpath "lib/*:src...
Going forward
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
51
➢ TroveApi
➢ Backup Extension
➢ Settin...
Going forward
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
52
➢ TroveApi
➢ Backup Extension
➢ Settin...
Going forward
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
53
➢ TroveApi
➢ Backup Extension
➢ Settin...
Going forward
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
54
Going forward
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
55
Collaborate!
http://jclouds.apache.org...
Taming the Cloud Database with Apache jclouds http://j.
mp/acjdb14
56
Thank you!
http://developer.rackspace.com
sdk-suppor...
Upcoming SlideShare
Loading in...5
×

Taming the Cloud Database with Apache jclouds

373

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
373
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Taming the Cloud Database with Apache jclouds"

  1. 1. 1 Taming the Cloud Database with Apache jclouds http://j.mp/acjdb14
  2. 2. Introductions Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 2 ➢ Zack Shoylev ➢ irc: zacksh twitter: @zackshoylev freenode #jclouds Also email!
  3. 3. Introductions Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 3 Developer Relations Group https://developer.rackspace.com/support/
  4. 4. Goals Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 4 ➢ jclouds ➢ Create a database in the cloud ➢ Abstractions and how to contribute
  5. 5. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 5
  6. 6. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 6 ➢ Buzzword ➢ Networked and distributed computing ➢ Variety of services ➢ Compute (servers) ➢ Storage (files) ➢ Databases ➢ Email ➢ …
  7. 7. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 7
  8. 8. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 8 ➢ Public cloud (external provider) ➢ Private cloud (internal deployment) ➢ Hybrid cloud ➢ Public + Private Rackspace example
  9. 9. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 9 Your System S D K Cloud AP I VM VM VM ➢ This can be an application server ➢ Or your home machine ➢ Or belong to one of your end users ➢ Or a cloud VM ➢ Or a smartphone jclouds
  10. 10. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 10 ➢ Advantages ➢ Metered pricing (pay as you go) ➢ Project scalability (unlimited) ➢ Safer (offsite/redundant) ➢ Economies of scale ➢ Expertise ➢ Support
  11. 11. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 11 ➢ Disadvantages ➢ Less hardware control ➢ Provider-controlled downtime ➢ Virtualization (efficiency) ➢ Provider lock-in ➢ Provider-specific apis/sdks/features ➢ Expensive to switch clouds or deploy locally ➢ jclouds minimizes this disadvantage
  12. 12. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 12 ➢ Can’t someone else do it?
  13. 13. The Cloud Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 13 ➢ Database specific advantages ➢ Optimized by provider ➢ Settings ➢ Container virtualization ➢ Automated software updates ➢ Redundant data storage ➢ Backup ➢ Migration ➢ Support
  14. 14. OpenStack Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 14 ➢ Cloud OS ➢ Open-sourced by Rackspace and NASA (2010) ➢ Free ➢ Supported ➢ AT&T, HP, IBM, Red Hat, Rackspace, Dell, Cisco, Intel, VMware, …
  15. 15. jclouds Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 15 ➢ Cloud SDK ➢ Apache project ➢ Java (and Clojure) ➢ Easy ➢ Portable ➢ Cloud-agnostic ➢ Community ➢ Open source
  16. 16. jclouds Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 16 ➢ HTTP requests, responses, retries ➢ Authentication and re-authentication ➢ Pagination ➢ Polling ➢ Rate limits ➢ Retries ➢ Abstractions ➢ Logging ➢ Less code
  17. 17. jclouds Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 17
  18. 18. jclouds Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 18 ➢ Services ➢ Storage ➢ Compute ➢ VM Images ➢ Load Balancers ➢ DNS ➢ Databases ➢ … ➢ Providers ➢ Openstack ➢ Rackspace ➢ HP ➢ Amazon ➢ Azure ➢ …
  19. 19. jclouds Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 19 AWS aws-ec2 US-VA,US-CA,IE,SG Bluelock bluelock-vcloud-zone01 US-IN CloudSigma US cloudsigma-lvs US-NV CloudSigma CH cloudsigma-zrh CH-ZH CloudSigma DC cloudsigma-wdc US-DC DigitalOcean digitalocean ElasticHosts GB elastichosts-lon-b GB-LND ElasticHosts GB elastichosts-lon-p GB-LND ElasticHosts US elastichosts-sat-p US-TX Go2Cloud go2cloud-jhb1 ZA-GP GoGrid gogrid US-CA,US-VA Green House Data greenhousedata-element- vcloud US-WY HP hpcloud-compute US-NV Ninefold ninefold-compute AU-NSW OpenHosting openhosting-east1 US-VA Rackspace UK (First Gen) cloudservers-uk GB-SLG Rackspace US (First Gen) cloudservers-us US-IL,US-TX Rackspace UK (Next Gen) rackspace-cloudservers-uk GB-SLG Rackspace US (Next Gen) rackspace-cloudservers-us US-IL,US-TX SeverLove serverlove-z1-man GB-MAN SkaliCloud skalicloud-sdg-my MY-10 SoftLayer softlayer Compute Providers
  20. 20. jclouds Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 20 ➢ Showcase database code ➢ Best practices ➢ Compare with compute code ➢ Gotchas and workarounds ➢ Abstractions
  21. 21. Requirements Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 21 ➢ Maven 3 ➢ Java 6+ ➢ jclouds ➢ Windows or Linux [etc..] (thanks Java!)
  22. 22. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 22 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  23. 23. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 23 ➢ Maven Dependencies ➢ Selective subset ➢ Versioning ➢ jclouds-labs
  24. 24. POM Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 24 https://github.com/jclouds/jclouds-examples/blob/master/rackspace/pom.xml <dependency> <groupId>org.apache.jclouds.provider</groupId> <artifactId>rackspace-clouddatabases-us</artifactId> <version>${jclouds.version}</version> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.25</version> </dependency>
  25. 25. Logging Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 25 // This module is responsible for enabling logging Iterable<Module> modules = ImmutableSet.<Module> of(new SLF4JLoggingModule()); ComputeServiceContext context = ContextBuilder.newBuilder(provider) .credentials(username, apiKey) .modules(modules) // don't forget to add the modules to your context! .buildView(ComputeServiceContext.class); logback.xml <configuration scan="false"> … <appender name="WIREFILE" class="ch.qos.logback.core.FileAppender"> <file>target/test-data/jclouds-wire.log</file> <encoder> <Pattern>%d %-5p [%c] [%thread] %m%n</Pattern> </encoder> </appender>
  26. 26. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 26 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  27. 27. Initialize Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 27 private final TroveApi troveApi; troveApi = ContextBuilder .newBuilder("rackspace-clouddatabases- us") .credentials(username, apiKey) .buildApi(TroveApi.class);
  28. 28. Architecture Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 28 Cloud Databases Trove jclouds-database Increasing level of abstraction Provider settings: Endpoint, etc. Implementation code Abstraction s
  29. 29. Architecture Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 29 Cloud Databases Trove jclouds-database Increasing level of abstraction Provider settings: Endpoint, etc. Implementation code Abstraction s Not implemented
  30. 30. Initialize Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 30 private final FlavorApi flavorApi; flavorApi = troveApi.getFlavorApiForZone(ZONE);
  31. 31. APIs Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 31 ➢ TroveApi ➢ FlavorApi ➢ InstanceApi ➢ DatabaseApi ➢ UserApi ➢ Utils
  32. 32. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 32 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  33. 33. Create Instance Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 33 Flavor flavor = Iterables.getFirst(flavorApi.list(), null); … Instance instance = instanceApi.create(flavorId, size, name); flavor.getId() Volume size in GB
  34. 34. Create Instance Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 34 Instance updatedInstance = awaitAvailable(instance, instanceApi);
  35. 35. Create Instance Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 35 Instance updatedInstance = awaitAvailable(instance, instanceApi); Polls status, waits until ready. Will not retry by itself!
  36. 36. Actual Code Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 36 InstanceApi instanceApi = api.getInstanceApiForZone(zone); for (int retries = 0; retries < 10; retries++) { Instance instance = null; try { instance = instanceApi.create(flavorId, size, name); } catch (Exception e) { Uninterruptibles.sleepUninterruptibly(15, TimeUnit.SECONDS); logger.error(Arrays.toString(e.getStackTrace())); continue; } Instance updatedInstance = awaitAvailable(instance, instanceApi); if (updatedInstance != null) { return updatedInstance; } instanceApi.delete(instance.getId()); InstancePredicates.awaitDeleted(instanceApi).apply(instance); } return null;
  37. 37. Actual Code Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 37 Utils. getWorkingInstance( String zone, String name, String flavorId, int size)
  38. 38. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 38 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  39. 39. Create Database Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 39 troveApi = ContextBuilder.newBuilder(PROVIDER) .credentials(username, apiKey) .buildApi(TroveApi.class); instanceApi = troveApi.getInstanceApiForZone(ZONE); databaseApi = troveApi .getDatabaseApiForZoneAndInstance(ZONE,getInstance().getId()); getSomeApiForXandYandZ – fairly common in jclouds
  40. 40. Create Database Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 40 boolean result = databaseApi.create(NAME);
  41. 41. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 41 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  42. 42. Create User Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 42 userApi = troveApi .getUserApiForZoneAndInstance(ZONE, instance.getId()); boolean result = userApi.create(UNAME, PASSWORD, DBNAME); User name User password Database name
  43. 43. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 43 Skeleton Project Create User Create Instance Initialize API Create Database Test Connection Cleanup
  44. 44. Test Connection Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 44 Load balancer needed! CreateLoadBalancer createLB = CreateLoadBalancer. builder() .name(NAME) .protocol("MYSQL") .port(3306) .algorithm(LoadBalancer.Algorithm.RANDOM) .nodes(addNodes) .virtualIPType(VirtualIP.Type.PUBLIC) .build();
  45. 45. Test Connection Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 45 loadBalancer = lbApi.create(createLB);
  46. 46. Test Connection Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 46 StringBuilder connString = new StringBuilder(); connString.append("jdbc:mysql://"); connString .append(getVirtualIPv4(getLb().getVirtualIPs())); connString.append("/"); connString.append(DBNAME); connString.append("? user="); connString.append(UNAME); connString.append("&password="); connString.append(PASSWORD);
  47. 47. Test Connection Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 47 Connection conn = DriverManager.getConnection(connString.toString());
  48. 48. Test Connection Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 48 Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("SELECT 3+5"); rs.first(); System.out.format(" 3+5 is %s%n", rs.getInt(1));
  49. 49. Going forward Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 49 ➢ jclouds examples ➢ https://github.com/jclouds/jclouds-examples ➢ More docs ➢ http://jclouds.incubator.apache.org/documentation/ ➢ http://javadocs.jclouds.cloudbees.net/ ➢ Maven alternatives? ➢ http://jclouds.incubator.apache. org/documentation/userguide/installation-guide/ ➢ Contribute! ➢ https://wiki.apache.org/jclouds/How%20to% 20Contribute
  50. 50. Java Project Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 50 ➢ Compile ➢ javac -classpath "lib/*:src/main/java/:src/main/resources/" src/main/java/org/jclouds/examples/rackspace/*.java ➢ Run ➢ java -classpath "lib/*:src/main/java/:src/main/resources/" org.jclouds.examples. rackspace.clouddatabases.CreateInstance username apikey
  51. 51. Going forward Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 51 ➢ TroveApi ➢ Backup Extension ➢ Settings Extension ➢ Abstraction layer
  52. 52. Going forward Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 52 ➢ TroveApi ➢ Backup Extension ➢ Settings Extension ➢ Abstraction layer Rackspac e Microsof t Amazo n CouchDB Salesforc e
  53. 53. Going forward Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 53 ➢ TroveApi ➢ Backup Extension ➢ Settings Extension ➢ Abstraction layer Rackspac e Microsof t Amazo n CouchDB Salesforc e
  54. 54. Going forward Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 54
  55. 55. Going forward Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 55 Collaborate! http://jclouds.apache.org/ http://jclouds.apache.org/community/
  56. 56. Taming the Cloud Database with Apache jclouds http://j. mp/acjdb14 56 Thank you! http://developer.rackspace.com sdk-support@rackspace.com Zack Shoylev Software Developer zacksh #jclouds @zackshoylev
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×