MongoLA - Cloud Foundry

About Monica Wilkinson Loves the web and data portability. Developer Advocate @ Cloud Foundry 12 years development experience. Last 5 years in Social Web Open Web Standards Advocate @ciberch mwilkinson@vmware.com CONFIDENTIAL 2Thursday, January 19, 12

About Josh Long Spring Developer Advocate @starbuxman josh.long@springsource.com th si is im po rta nt ! Free Book! SpringSource.org/roo CONFIDENTIAL 3Thursday, January 19, 12

Cloud Foundry CONFIDENTIAL 4Thursday, January 19, 12

About Cloud Foundry The first Open PaaS Multi(n) Languages, Frameworks, Services & Clouds Open Source CONFIDENTIAL 5Thursday, January 19, 12

STEP 1 - Get a Cloud Foundry account  https://my.cloudfoundry.com/signup/MongoLA CONFIDENTIAL 6Thursday, January 19, 12

STEP 2 - Download VMC •sudo gem install vmc •vmc login <username> CONFIDENTIAL 7Thursday, January 19, 12

STEP 3: Push your app to Cloud Foundry vmc push --runtime=ruby19 CONFIDENTIAL 8Thursday, January 19, 12

Every day development  Debugging and accessing the data locally • Caldecott --> Service tunneling. Access your Cloud Foundry service as if it was local. CONFIDENTIAL 10Thursday, January 19, 12

Tunneling gem install caldecott vmc tunnel <mongodb> CONFIDENTIAL 11Thursday, January 19, 12

Using your favorite tools CONFIDENTIAL 12Thursday, January 19, 12

CONFIDENTIAL 13Thursday, January 19, 12

Data Access Challenge #1: Scale Horizontally CONFIDENTIAL 15Thursday, January 19, 12

Data Access Challenge #2: Heterogeneous CONFIDENTIAL 16Thursday, January 19, 12

New demands on data access •... until we needed inexpensive horizontal scaling for some large web based applications ... • ... and we needed to deploy our apps in the cloud ... CONFIDENTIAL * image courtesy of Bitcurrent 17Thursday, January 19, 12

NoSQL offers several data store categoriesKey-Value Column Document Graph Redis, Cassandra, MongoDB Neo4J Riak HBase CONFIDENTIAL 18Thursday, January 19, 12

About Mongo DB 1. No-SQL database development : { 2. Stores JSON-style tools: many, documents language_support: superb, = 3. Horizontally scalable agility: high }, 4. Full Indexing Support production: { 5. Open Source speed: fast fault_tolerance: true 6. Great Community(You!) scalability: high CONFIDENTIAL 19Thursday, January 19, 12

NoSQL offers several data store categoriesKey-Value Column Document Graph MongoDB (who cares about the rest?) CONFIDENTIAL 20Thursday, January 19, 12

Mongo and Ruby Deep Dive CONFIDENTIAL 21Thursday, January 19, 12

Integration with Box(.net) CONFIDENTIAL 22Thursday, January 19, 12

Box Sample App Creation Wizard • Requirement: Clone sample Box app for developers into their own Cloud Foundry account so they can learn how to use the BOX Api. CONFIDENTIAL 23Thursday, January 19, 12

Challenges • No command to download all the files for an app or copy it to another account. •Solution: The Cloud •Get the code for an app from: GitHub •Use vmc gem to upload the contents to your desired cloud CONFIDENTIAL 24Thursday, January 19, 12

VMC as a gem  Can be used to authenticate users and exchange their credentials for an API token  Edit Applications and Services from your code • Files • Environment Variables  Start and Stop Apps CONFIDENTIAL 25Thursday, January 19, 12

How we used VMC on this project def create() begin @vmcclient.create_app(@manifest["name"], @manifest) rescue RuntimeError => ex if (ex.message =~ /Error 701/) new_candidate = @generator.next unless new_candidate.nil? change_name! new_candidate create else # Format is "Error #{parsed_body[:code]}: #{desc}" raise "App Url: #{@uri} is already taken" end else raise ex end end end def delete() @vmcclient.delete_app(@manifest["name"]) end def copy_code() @vmcclient.upload_app(@manifest["name"], @app_meta.build!) end CONFIDENTIAL 26Thursday, January 19, 12

More Challenges  External Models we didn’t control • Cloud Foundry Apps: Files, Runtime, Services, Env Vars, etc • GitHub Repositories: Files, Commits, branches, tags • Solution • Use an Object Document Mapper. I chose Mongoid • Easy to add new objects and properties • No Migrations ! CONFIDENTIAL 27Thursday, January 19, 12

CONFIDENTIAL 28Thursday, January 19, 12

Things you should know about Mongoid  Project Page - http://mongoid.org.  Features: • Uses Active Model (Dirty Tracking, Mass Assignment Security, Validations, Callbacks, JSON/XML Serialization) • Indexing • Supports identity map, replica sets and sharding • Custom field serialization • A Range in Ruby 1...3 could be stored in MongoDB as a Hash { :min => 1, :max => 3 } • Localization Support:(Translated Fields, i18n Fallbacks) • Inheritance CONFIDENTIAL 29Thursday, January 19, 12

How we used Mongoid module CloudFoundry class AppInfo include TmpZip include Mongoid::Document embeds_many :app_clone_requests belongs_to :repo, :class_name => "GitHub::RepositorySnapshot" field :display_name, :type => String field :description, :type => String field :instances, :type => Integer, :default => 1 field :memory, :type => Integer, :default => 128 field :runtime, :type => String field :framework, :type => String field :env_vars, :type => Hash field :thumb_url, :type => String field :browseable, :type => Boolean field :cloneable, :type => Boolean field :starting_url, :type => String index :display_name, :unique => true validates_presence_of :display_name, :runtime, :framework validates_presence_of :repo, :if => :cloneable CONFIDENTIAL 30Thursday, January 19, 12

How we used Mongoid module GitHub class RepositorySnapshot include TmpZip include Mongoid::Document field :url, :type => String field :name, :type => String field :parent, :type => String field :branch, :type => String, :default => master field :tag, :type => String field :commit, :type => String index :url, :unique => true validates_presence_of :url, :name, :parent, :commit, :branch def url=(value) parts = value.gsub(/https://github.com//, ).split(/) if (parts.length == 2) write_attribute(:parent, parts[0]) write_attribute(:name, parts[1]) else write_attribute(:parent, nil) write_attribute(:name, nil) end write_attribute(:url, value) end CONFIDENTIAL 31Thursday, January 19, 12

Getting your ruby app to use the proper module CloudFoundry class Mongo def self.config Mongoid.configure do |config| conn_info = nil dbname = db if ENV[VCAP_SERVICES] services = JSON.parse ENV[VCAP_SERVICES] services.each do |service_version, bindings| mongo_binding = bindings.find {|binding| binding[label] =~ /mongo/i} conn_info = mongo_binding[credentials] if mongo_binding end raise "ERROR - Could not find connection info for mongo" unless conn_info else conn_info = {hostname => localhost, port => 27017} dbname = "gallery_db" end CONFIDENTIAL @@cnx = Mongo::Connection.new conn_info[hostname], conn_info[port], :pool_size => 32 5, :timeout => 5Thursday, January 19, 12

Demo CONFIDENTIAL 33Thursday, January 19, 12

Durran, creator of Mongoid detailed for us: Full atomic update support out of the box for *all* MongoDB atomic operations, either explicitly or handled under the covers by Mongoid itself with support for embedded n levels deep. ie: model = Model.find(id) model.field = “value” model.relations.build(field: “value”) model.save #=> Does { field: { “$set” : “value” }, relations: { “$push” : { field: “value” }}} for you. *or things like* model.add_to_set(field: “value”) CONFIDENTIAL 34Thursday, January 19, 12

Durran, creator of Mongoid detailed for us “Smart memory management during iteration and working with relations: Mongoid never loads everything into memory unless you specifically want it to. It can handle working with millions of documents without putting stress on the server/RAM.” CONFIDENTIAL 35Thursday, January 19, 12

More Challenges  Make app fast. • Downloading and copying files from app to app is not cheap • Solution • Avoid downloading the build if it has not changed • Local cache using Mongo GridFS CONFIDENTIAL 36Thursday, January 19, 12

Working with GridFS  Saving and Reading files Once you have a Grid object, you can start saving data to it. @db = Mongo::Connection.new.db(social_site) @grid = Grid.new(@db) # On this Cloud Foundry App @grid = Mongo::GridFileSystem.new(Mongoid.database) # Saving IO data and including the optional filename image = File.open("me.jpg") id = @grid.put(image, :filename => "me.jpg") Grid#put returns an object id, which you can use to retrieve the file: # Get the file we saved image = @grid.get(id)  Deleting files Deleting a file is as simple as providing the id: @grid.delete(id2) CONFIDENTIAL 37Thursday, January 19, 12

Spring Data on Cloud Foundry Deep Dive CONFIDENTIAL 38Thursday, January 19, 12

Spring Framework built-in data access support •Transaction abstractions •Common data access exception hierarchy •JDBC - JdbcTemplate •ORM - Hibernate, JPA support •OXM - Object to XML mapping •Serializer/Deserializer strategies (Spring 3.0) •Cache support (Spring 3.1) CONFIDENTIAL 39Thursday, January 19, 12

Spring Data Building Blocks •Low level data access APIs ✓MongoTemplate, RedisTemplate ... •Object Mapping (Java and GORM) •Cross Store Persistence Programming model •Generic Repository support •Productivity support in Roo and Grails CONFIDENTIAL 40Thursday, January 19, 12

Spring Data Document Mongo •MongoTemplate •MongoConverter interface for mapping Mongo documents •SimpleMongoConverter for basic POJO mapping support •Leverage Spring 3.0 TypeConverters and SpEL •Exception translation •Advanced Mapping •Annotation based (@Document, @Id, @DbRef) •MongoRepository •Built on Hades support for JPA Repositories CONFIDENTIAL 41Thursday, January 19, 12

Simple Domain Class CONFIDENTIAL 42Thursday, January 19, 12

Mongo Template Direct Usage of the Mongo Template: CONFIDENTIAL 43Thursday, January 19, 12

Mongo Template Direct Usage of the Mongo Template: Insert into “Person” Collection CONFIDENTIAL 43Thursday, January 19, 12

Mongo Template Direct Usage of the Mongo Template: findOne using query: { "name" : "Joe"} in db.collection: database.Person CONFIDENTIAL 43Thursday, January 19, 12

Mongo Template Direct Usage of the Mongo Template: Dropped collection [database.person] CONFIDENTIAL 43Thursday, January 19, 12

Generic Repository Interface for generic CRUD operations on a repository for a specific type CONFIDENTIAL 44Thursday, January 19, 12

Paging and Sorting Repository Paging and Sorting Repository: Extends “CrudRepository” CONFIDENTIAL 45Thursday, January 19, 12

Paging and Sorting Repository Paging and Sorting Repository: Extends “CrudRepository” Usage: CONFIDENTIAL 45Thursday, January 19, 12

Custom Repository Custom Repository: CONFIDENTIAL 46Thursday, January 19, 12

Custom Repository Custom Repository: Keywords : Keyword Sample Logical result GreaterThan findByAgeGreaterThan(int age) {"age" : {"$gt" : age}} LessThan findByAgeLessThan(int age) {"age" : {"$lt" : age}} Between findByAgeBetween(int from, int to) {"age" : {"$gt" : from, "$lt" : to}} NotNull findByFirstnameNotNull() {”firstname" : {"$ne" : null}} Null findByFirstnameNull() {”firstname" : null} Like findByFirstnameLike(String name) CONFIDENTIAL "firstname" : firstname} (regex) 46Thursday, January 19, 12

JPA and MongoDB JPA “Customer” with a “SurveyInfo” Document CONFIDENTIAL 47Thursday, January 19, 12

Using a Cross-Store Saving a Customer with a SurveryInfo CONFIDENTIAL 48Thursday, January 19, 12

Using a Cross-Store Saving a Customer with a SurveryInfo Create Customer CONFIDENTIAL 48Thursday, January 19, 12

Using a Cross-Store Saving a Customer with a SurveryInfo Create SurveyInfo CONFIDENTIAL 48Thursday, January 19, 12

Using a Cross-Store Saving a Customer with a SurveryInfo Assign Survey to Customer CONFIDENTIAL 48Thursday, January 19, 12

Using a Cross-Store Saving a Customer with a SurveryInfo Save CONFIDENTIAL 48Thursday, January 19, 12

Using a Cross-Store Saving a Customer with a SurveryInfo Save Mongo Document: CONFIDENTIAL 48Thursday, January 19, 12

Accessing Services Bound to Cloud Foundry private String mongoDatabaseServiceName = "survey-mongo"; @Bean public CloudEnvironment cloudEnvironment() { return new CloudEnvironment(); } @Bean public MongoServiceInfo mongoServiceInfo() { return cloudEnvironment().getServiceInfo( mongoDatabaseServiceName, MongoServiceInfo.class); } @Bean public MongoDbFactory mongoDbFactory() { MongoServiceCreator msc = new MongoServiceCreator(); MongoDbFactory db = msc.createService(mongoServiceInfo()); } CONFIDENTIAL 49Thursday, January 19, 12

Accessing Services Bound to Cloud Foundry @Inject private Mongo mongo ; @Inject private MongoTemplate mongoTemplate; CONFIDENTIAL 50Thursday, January 19, 12

Demo 2 CONFIDENTIAL 51Thursday, January 19, 12

Thank You CloudFoundry Spring Source Samples: http://bit.ly/cloudfoundry-samples http://cloudfoundry.com Questions: @cloudfoundry •Josh -> @starbuxman or josh.long@springsource.com •Monica -> @ciberch or mwilkinson@cloudfoundry.com © 2012 VMware, Inc. All rights reservedThursday, January 19, 12

http://www.10gen.com	45
http://www.hanrss.com	3
http://fromwww.mongodb.org	1
http://w.mongodb.org	1
http://archimedesjira.mongodb.org	1
http://download.mongodb.org	1

MongoLA - Cloud Foundry

by Monica Wilkinson on Jan 25, 2012

Accessibility

Categories

Tags

Upload Details

Usage Rights

6 Embeds 52

Statistics

MongoLA - Cloud Foundry — Presentation Transcript