MongoLA - Cloud Foundry

1,770 views
1,626 views

Published on

Spring and Ruby MongoDB Cloud Foundry Deep dive

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,770
On SlideShare
0
From Embeds
0
Number of Embeds
132
Actions
Shares
0
Downloads
38
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MongoLA - Cloud Foundry

  1. 1. MongoDB App Duet Ruby and Java on Cloud Foundry By Monica Wilkinson and Josh Long Cloud Foundry, Developer Relations © 2012 VMware, Inc. All rights reserved MongoLA 2012Thursday, January 19, 12
  2. 2. About Monica Wilkinson Loves the web and data portability. Developer Advocate @ Cloud Foundry 12 years development experience. Last 5 years in Social Web Open Web Standards Advocate @ciberch mwilkinson@vmware.com CONFIDENTIAL 2Thursday, January 19, 12
  3. 3. About Josh Long Spring Developer Advocate @starbuxman josh.long@springsource.com th si is im po rta nt ! Free Book! SpringSource.org/roo CONFIDENTIAL 3Thursday, January 19, 12
  4. 4. Cloud Foundry CONFIDENTIAL 4Thursday, January 19, 12
  5. 5. About Cloud Foundry The first Open PaaS Multi(n) Languages, Frameworks, Services & Clouds Open Source CONFIDENTIAL 5Thursday, January 19, 12
  6. 6. STEP 1 - Get a Cloud Foundry account  https://my.cloudfoundry.com/signup/MongoLA CONFIDENTIAL 6Thursday, January 19, 12
  7. 7. STEP 2 - Download VMC •sudo gem install vmc •vmc login <username> CONFIDENTIAL 7Thursday, January 19, 12
  8. 8. STEP 3: Push your app to Cloud Foundry vmc push --runtime=ruby19 CONFIDENTIAL 8Thursday, January 19, 12
  9. 9. You are done Service gets created and bound Code gets pushed App is Live immediately at the url you requested © 2012 VMware, Inc. All rights reservedThursday, January 19, 12
  10. 10. Every day development  Debugging and accessing the data locally • Caldecott --> Service tunneling. Access your Cloud Foundry service as if it was local. CONFIDENTIAL 10Thursday, January 19, 12
  11. 11. Tunneling gem install caldecott vmc tunnel <mongodb> CONFIDENTIAL 11Thursday, January 19, 12
  12. 12. Using your favorite tools CONFIDENTIAL 12Thursday, January 19, 12
  13. 13. CONFIDENTIAL 13Thursday, January 19, 12
  14. 14. Why MongoDB? 14 © 2012 VMware, Inc. All rights reservedThursday, January 19, 12
  15. 15. Data Access Challenge #1: Scale Horizontally CONFIDENTIAL 15Thursday, January 19, 12
  16. 16. Data Access Challenge #2: Heterogeneous CONFIDENTIAL 16Thursday, January 19, 12
  17. 17. New demands on data access •... until we needed inexpensive horizontal scaling for some large web based applications ... • ... and we needed to deploy our apps in the cloud ... CONFIDENTIAL * image courtesy of Bitcurrent 17Thursday, January 19, 12
  18. 18. NoSQL offers several data store categoriesKey-Value Column Document Graph Redis, Cassandra, MongoDB Neo4J Riak HBase CONFIDENTIAL 18Thursday, January 19, 12
  19. 19. About Mongo DB 1. No-SQL database development : { 2. Stores JSON-style tools: many, documents language_support: superb, = 3. Horizontally scalable agility: high }, 4. Full Indexing Support production: { 5. Open Source speed: fast fault_tolerance: true 6. Great Community(You!) scalability: high CONFIDENTIAL 19Thursday, January 19, 12
  20. 20. NoSQL offers several data store categoriesKey-Value Column Document Graph MongoDB (who cares about the rest?) CONFIDENTIAL 20Thursday, January 19, 12
  21. 21. Mongo and Ruby Deep Dive CONFIDENTIAL 21Thursday, January 19, 12
  22. 22. Integration with Box(.net) CONFIDENTIAL 22Thursday, January 19, 12
  23. 23. Box Sample App Creation Wizard • Requirement: Clone sample Box app for developers into their own Cloud Foundry account so they can learn how to use the BOX Api. CONFIDENTIAL 23Thursday, January 19, 12
  24. 24. Challenges • No command to download all the files for an app or copy it to another account. •Solution: The Cloud •Get the code for an app from: GitHub •Use vmc gem to upload the contents to your desired cloud CONFIDENTIAL 24Thursday, January 19, 12
  25. 25. VMC as a gem  Can be used to authenticate users and exchange their credentials for an API token  Edit Applications and Services from your code • Files • Environment Variables  Start and Stop Apps CONFIDENTIAL 25Thursday, January 19, 12
  26. 26. How we used VMC on this project def create()       begin         @vmcclient.create_app(@manifest["name"], @manifest)       rescue RuntimeError => ex         if (ex.message =~ /Error 701/)           new_candidate = @generator.next           unless new_candidate.nil?             change_name! new_candidate             create           else             # Format is "Error #{parsed_body[:code]}: #{desc}"             raise "App Url: #{@uri} is already taken"           end         else           raise ex         end       end     end     def delete()       @vmcclient.delete_app(@manifest["name"])     end     def copy_code()       @vmcclient.upload_app(@manifest["name"], @app_meta.build!)     end CONFIDENTIAL 26Thursday, January 19, 12
  27. 27. More Challenges  External Models we didn’t control • Cloud Foundry Apps: Files, Runtime, Services, Env Vars, etc • GitHub Repositories: Files, Commits, branches, tags • Solution • Use an Object Document Mapper. I chose Mongoid • Easy to add new objects and properties • No Migrations ! CONFIDENTIAL 27Thursday, January 19, 12
  28. 28. CONFIDENTIAL 28Thursday, January 19, 12
  29. 29. Things you should know about Mongoid  Project Page - http://mongoid.org.  Features: • Uses Active Model (Dirty Tracking, Mass Assignment Security, Validations, Callbacks, JSON/XML Serialization) • Indexing • Supports identity map, replica sets and sharding • Custom field serialization • A Range in Ruby 1...3 could be stored in MongoDB as a Hash { :min => 1, :max => 3 } • Localization Support:(Translated Fields, i18n Fallbacks) • Inheritance CONFIDENTIAL 29Thursday, January 19, 12
  30. 30. How we used Mongoid module CloudFoundry   class AppInfo     include TmpZip     include Mongoid::Document     embeds_many :app_clone_requests     belongs_to :repo, :class_name => "GitHub::RepositorySnapshot"     field :display_name, :type => String     field :description, :type => String     field :instances, :type => Integer, :default => 1     field :memory, :type => Integer, :default => 128     field :runtime, :type => String     field :framework, :type => String     field :env_vars, :type => Hash     field :thumb_url, :type => String     field :browseable, :type => Boolean     field :cloneable, :type => Boolean     field :starting_url, :type => String     index :display_name, :unique => true     validates_presence_of :display_name, :runtime, :framework     validates_presence_of :repo, :if => :cloneable CONFIDENTIAL 30Thursday, January 19, 12
  31. 31. How we used Mongoid module GitHub   class RepositorySnapshot     include TmpZip     include Mongoid::Document     field :url, :type => String     field :name, :type => String     field :parent, :type => String     field :branch, :type => String, :default => master     field :tag, :type => String     field :commit, :type => String     index :url, :unique => true     validates_presence_of :url, :name, :parent, :commit, :branch     def url=(value)       parts = value.gsub(/https://github.com//, ).split(/)       if (parts.length == 2)         write_attribute(:parent, parts[0])         write_attribute(:name, parts[1])       else         write_attribute(:parent, nil)         write_attribute(:name, nil)       end       write_attribute(:url, value)     end CONFIDENTIAL 31Thursday, January 19, 12
  32. 32. Getting your ruby app to use the proper module CloudFoundry   class Mongo       def self.config         Mongoid.configure do |config|           conn_info = nil           dbname = db           if ENV[VCAP_SERVICES]             services = JSON.parse ENV[VCAP_SERVICES]             services.each do |service_version, bindings|               mongo_binding = bindings.find {|binding| binding[label] =~ /mongo/i}               conn_info = mongo_binding[credentials] if mongo_binding             end             raise "ERROR - Could not find connection info for mongo" unless conn_info           else             conn_info = {hostname => localhost, port => 27017}             dbname = "gallery_db"           end CONFIDENTIAL           @@cnx = Mongo::Connection.new conn_info[hostname], conn_info[port], :pool_size => 32 5, :timeout => 5Thursday, January 19, 12
  33. 33. Demo CONFIDENTIAL 33Thursday, January 19, 12
  34. 34. Durran, creator of Mongoid detailed for us: Full atomic update support out of the box for *all* MongoDB atomic operations, either explicitly or handled under the covers by Mongoid itself with support for embedded n levels deep. ie: model = Model.find(id) model.field = “value” model.relations.build(field: “value”) model.save #=> Does { field: { “$set” : “value” }, relations: { “$push” : { field: “value” }}} for you. *or things like* model.add_to_set(field: “value”) CONFIDENTIAL 34Thursday, January 19, 12
  35. 35. Durran, creator of Mongoid detailed for us “Smart memory management during iteration and working with relations: Mongoid never loads everything into memory unless you specifically want it to. It can handle working with millions of documents without putting stress on the server/RAM.” CONFIDENTIAL 35Thursday, January 19, 12
  36. 36. More Challenges  Make app fast. • Downloading and copying files from app to app is not cheap • Solution • Avoid downloading the build if it has not changed • Local cache using Mongo GridFS CONFIDENTIAL 36Thursday, January 19, 12
  37. 37. Working with GridFS  Saving and Reading files Once you have a Grid object, you can start saving data to it. @db = Mongo::Connection.new.db(social_site) @grid = Grid.new(@db) # On this Cloud Foundry App @grid = Mongo::GridFileSystem.new(Mongoid.database) # Saving IO data and including the optional filename image = File.open("me.jpg") id = @grid.put(image, :filename => "me.jpg") Grid#put returns an object id, which you can use to retrieve the file: # Get the file we saved image = @grid.get(id)  Deleting files Deleting a file is as simple as providing the id: @grid.delete(id2) CONFIDENTIAL 37Thursday, January 19, 12
  38. 38. Spring Data on Cloud Foundry Deep Dive CONFIDENTIAL 38Thursday, January 19, 12
  39. 39. Spring Framework built-in data access support •Transaction abstractions •Common data access exception hierarchy •JDBC - JdbcTemplate •ORM - Hibernate, JPA support •OXM - Object to XML mapping •Serializer/Deserializer strategies (Spring 3.0) •Cache support (Spring 3.1) CONFIDENTIAL 39Thursday, January 19, 12
  40. 40. Spring Data Building Blocks •Low level data access APIs ✓MongoTemplate, RedisTemplate ... •Object Mapping (Java and GORM) •Cross Store Persistence Programming model •Generic Repository support •Productivity support in Roo and Grails CONFIDENTIAL 40Thursday, January 19, 12
  41. 41. Spring Data Document Mongo •MongoTemplate •MongoConverter interface for mapping Mongo documents •SimpleMongoConverter for basic POJO mapping support •Leverage Spring 3.0 TypeConverters and SpEL •Exception translation •Advanced Mapping •Annotation based (@Document, @Id, @DbRef) •MongoRepository •Built on Hades support for JPA Repositories CONFIDENTIAL 41Thursday, January 19, 12
  42. 42. Simple Domain Class CONFIDENTIAL 42Thursday, January 19, 12
  43. 43. Mongo Template Direct Usage of the Mongo Template: CONFIDENTIAL 43Thursday, January 19, 12
  44. 44. Mongo Template Direct Usage of the Mongo Template: Insert into “Person” Collection CONFIDENTIAL 43Thursday, January 19, 12
  45. 45. Mongo Template Direct Usage of the Mongo Template: findOne using query: { "name" : "Joe"} in db.collection: database.Person CONFIDENTIAL 43Thursday, January 19, 12
  46. 46. Mongo Template Direct Usage of the Mongo Template: Dropped collection [database.person] CONFIDENTIAL 43Thursday, January 19, 12
  47. 47. Generic Repository Interface for generic CRUD operations on a repository for a specific type CONFIDENTIAL 44Thursday, January 19, 12
  48. 48. Paging and Sorting Repository Paging and Sorting Repository: Extends “CrudRepository” CONFIDENTIAL 45Thursday, January 19, 12
  49. 49. Paging and Sorting Repository Paging and Sorting Repository: Extends “CrudRepository” Usage: CONFIDENTIAL 45Thursday, January 19, 12
  50. 50. Custom Repository Custom Repository: CONFIDENTIAL 46Thursday, January 19, 12
  51. 51. Custom Repository Custom Repository: Keywords : Keyword Sample Logical result GreaterThan findByAgeGreaterThan(int age) {"age" : {"$gt" : age}} LessThan findByAgeLessThan(int age) {"age" : {"$lt" : age}} Between findByAgeBetween(int from, int to) {"age" : {"$gt" : from, "$lt" : to}} NotNull findByFirstnameNotNull() {”firstname" : {"$ne" : null}} Null findByFirstnameNull() {”firstname" : null} Like findByFirstnameLike(String name) CONFIDENTIAL "firstname" : firstname} (regex) 46Thursday, January 19, 12
  52. 52. JPA and MongoDB JPA “Customer” with a “SurveyInfo” Document CONFIDENTIAL 47Thursday, January 19, 12
  53. 53. Using a Cross-Store Saving a Customer with a SurveryInfo CONFIDENTIAL 48Thursday, January 19, 12
  54. 54. Using a Cross-Store Saving a Customer with a SurveryInfo Create Customer CONFIDENTIAL 48Thursday, January 19, 12
  55. 55. Using a Cross-Store Saving a Customer with a SurveryInfo Create SurveyInfo CONFIDENTIAL 48Thursday, January 19, 12
  56. 56. Using a Cross-Store Saving a Customer with a SurveryInfo Assign Survey to Customer CONFIDENTIAL 48Thursday, January 19, 12
  57. 57. Using a Cross-Store Saving a Customer with a SurveryInfo Save CONFIDENTIAL 48Thursday, January 19, 12
  58. 58. Using a Cross-Store Saving a Customer with a SurveryInfo Save Mongo Document: CONFIDENTIAL 48Thursday, January 19, 12
  59. 59. Accessing Services Bound to Cloud Foundry private String mongoDatabaseServiceName = "survey-mongo"; @Bean public CloudEnvironment cloudEnvironment() { return new CloudEnvironment(); } @Bean public MongoServiceInfo mongoServiceInfo() { return cloudEnvironment().getServiceInfo( mongoDatabaseServiceName, MongoServiceInfo.class); } @Bean public MongoDbFactory mongoDbFactory() { MongoServiceCreator msc = new MongoServiceCreator(); MongoDbFactory db = msc.createService(mongoServiceInfo()); } CONFIDENTIAL 49Thursday, January 19, 12
  60. 60. Accessing Services Bound to Cloud Foundry @Inject private Mongo mongo ; @Inject private MongoTemplate mongoTemplate; CONFIDENTIAL 50Thursday, January 19, 12
  61. 61. Demo 2 CONFIDENTIAL 51Thursday, January 19, 12
  62. 62. Thank You CloudFoundry Spring Source Samples: http://bit.ly/cloudfoundry-samples http://cloudfoundry.com Questions: @cloudfoundry •Josh -> @starbuxman or josh.long@springsource.com •Monica -> @ciberch or mwilkinson@cloudfoundry.com © 2012 VMware, Inc. All rights reservedThursday, January 19, 12

×