Spring in-the-cloud

Spring into the Cloud Josh Long Chris Richardson @starbuxman @crichardson josh.long@springsource.com chris.richardson@springsource.comFriday, July 13, 12

About Josh Long Spring Developer Advocate twitter: @starbuxman josh.long@springsource.com 2Friday, July 13, 12

About Josh Long Spring Developer Advocate @starbuxman josh.long@springsource.com th s i is im po rta nt ! Free Book! SpringSource.org/roo CONFIDENTIALFriday, July 13, 12 3

getting started with Cloud Foundry... CONFIDENTIALFriday, July 13, 12 4

Register http://cloudfoundry.com/signup Use promo code CFOpenTour2012 CONFIDENTIALFriday, July 13, 12 5

Welcome email CONFIDENTIALFriday, July 13, 12 6

Ways to use Cloud Foundry CONFIDENTIALFriday, July 13, 12 7

Installing vmc on Windows CONFIDENTIALFriday, July 13, 12 8

Installing vmc on Windows Install the vmc gem CONFIDENTIALFriday, July 13, 12 8

Installing vmc on Mac OSX Use Ruby Version Manager http://beginrescueend.com/ $ bash -s stable <<(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer) $ rvm install 1.9.2 $ rvm use 1.9.2 $ gem install vmc CONFIDENTIALFriday, July 13, 12 9

what does Cloud Foundry offer? CONFIDENTIALFriday, July 13, 12 10

Cloud Foundry: Choice of Runtimes CONFIDENTIALFriday, July 13, 12 11

Frameworks and Runtimes Supported • Out of the Box • Java (.WAR files, on Tomcat. Spring’s an ideal choice here, of course..) • Scala (Lift, Play!) • Ruby (Rails, Sinatra, etc.) • Node.js • Ecosystem Partners • .NET (Uhuru, Tier3) • Python (Stackato) • PHP (AppFog) • Haskell (1) • Erlang (2) 1) http://www.cakesolutions.net/teamblogs/2011/11/25/haskell-happstack-on-cloudfoundry/ 2) https://github.com/cloudfoundry/vcap/pull/20 CONFIDENTIALFriday, July 13, 12

--- Manifests applications: target: name: html5expenses url: ${name}.${target-base} framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: expenses-mysql: type: :mysql CONFIDENTIALFriday, July 13, 12 13

Cloud Foundry: Choice of Clouds CONFIDENTIALFriday, July 13, 12 14

Main Risk: Lock In Welcome to the hotel california Such a lovely place Such a lovely face Plenty of room at the hotel california Any time of year, you can ﬁnd it here Last thing I remember, I was Running for the door I had to ﬁnd the passage back To the place I was before ’relax,’ said the night man, We are programmed to receive. You can checkout any time you like, But you can never leave! -the Eagles CONFIDENTIALFriday, July 13, 12 15

Open Source Advantage CONFIDENTIALFriday, July 13, 12 16

Open Source Advantage CONFIDENTIALFriday, July 13, 12

Cloud Foundry.org CONFIDENTIALFriday, July 13, 12 18

Cloud Foundry: Clouds § AppFog.com • community lead for PHP • PaaS for PHP § Joyent • community lead for Node.js § ActiveState • community lead for Python, Perl • Providers of Stackato private PaaS CONFIDENTIALFriday, July 13, 12 19

Micro Cloud Foundry (beta) CONFIDENTIALFriday, July 13, 12 20

Example vmc commands $ vmc target <any cloud> $ vmc login <credentials> $ vmc push <my-app> ! > bind services? Yes $ vmc update <my-app> $ vmc instances <my-app> +100 CONFIDENTIALFriday, July 13, 12 21

Scale the application $ vmc instances cf1 +2 Scaling Application instances up to 3: OK $ vmc stats cf1 +----------+-------------+----------------+--------------+--------------+ | Instance | CPU (Cores) | Memory (limit) | Disk (limit) | Uptime | +----------+-------------+----------------+--------------+--------------+ | 0 | 1.2% (4) | 90.6M (512M) | 8.5M (2G) | 0d:0h:3m:3s | | 1 | 9.5% (4) | 60.9M (512M) | 8.4M (2G) | 0d:0h:0m:17s | | 2 | 9.4% (4) | 55.5M (512M) | 8.4M (2G) | 0d:0h:0m:17s | +----------+-------------+----------------+--------------+--------------+ CONFIDENTIALFriday, July 13, 12

Demo: logging in, asking questions, & deploying from the CLI CONFIDENTIALFriday, July 13, 12 23

Using Cloud Foundry from Eclipse & STS http://springsource.com/developer/sts CONFIDENTIALFriday, July 13, 12 24

Demo: developing with Eclipse & STS CONFIDENTIALFriday, July 13, 12 25

Cloud Foundry: Services CONFIDENTIALFriday, July 13, 12 26

Cloud Foundry: Services § Services are one of the extensibility planes in Cloud Foundry • there are more services being contributed by the community daily! § MySQL, Redis, MongoDB, RabbitMQ, PostgreSQL § Services may be shared across applications § Cloud Foundry abstracts the provisioning aspect of services through a uniform API hosted in the cloud controller § It’s very easy to take an app and add a service to the app in a uniform way • Cassandra? COBOL / CICS, Oracle CONFIDENTIALFriday, July 13, 12 27

Cloud Foundry: Services § Take Advantage of Services • they cost nothing to setup • they deliver value § They Encourage Better Architectures • Need a fast read-write cache? Redis is ready to go! • Need to store long-tail documents? Give MongoDB a try • Need to decouple what applications do from when they do it? Use messaging and RabbitMQ CONFIDENTIALFriday, July 13, 12 28

Accessing Your Services § Debugging and accessing the data locally • Caldecott --> Service tunneling. Access your Cloud Foundry service as if it was local. CONFIDENTIALFriday, July 13, 12 31

Demo: Using the tunnel to talk to your services CONFIDENTIALFriday, July 13, 12 32

Cloud Foundry architecture CONFIDENTIALFriday, July 13, 12 33

Application Deployment Flow VMC Cloud Controller DEA DEA Application Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow vmc push VMC Cloud Controller DEA DEA Application Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow vmc push VMC Cloud Controller DEA DEA Application Framework detection Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow vmc push create app VMC Cloud Controller DEA DEA Application Framework detection Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow vmc push create app VMC Cloud Controller DEA DEA Application Framework detection Framework speciﬁc staging plugin Creates Droplet Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow ﬁnd DEA for app vmc push create app VMC Cloud Controller DEA DEA Application Framework detection Framework speciﬁc staging plugin Creates Droplet Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow ﬁnd DEA for app vmc push I’m available create app VMC Cloud Controller DEA DEA Application Framework detection Framework speciﬁc staging plugin Creates Droplet Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow ﬁnd DEA for app vmc push I’m available create app VMC Cloud Controller DEA DEA Application deploy droplet Framework detection Framework speciﬁc staging plugin Creates Droplet Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow ﬁnd DEA for app vmc push I’m available create app start VMC Cloud Controller DEA DEA Application deploy droplet Framework detection Framework speciﬁc staging plugin Creates Droplet Router Router CONFIDENTIALFriday, July 13, 12 34

Application Deployment Flow ﬁnd DEA for app vmc push I’m available create app start VMC Cloud Controller DEA DEA Application deploy droplet Framework detection Framework speciﬁc staging plugin update routes Creates Droplet Router Router CONFIDENTIALFriday, July 13, 12 34

About the staging plugin § Framework specific § Creates a zip file = black box • Contains everything necessary for DEA to run the application • Two scripts: start.sh, stop.sh CONFIDENTIALFriday, July 13, 12 35

Staging plugin for the Spring framework § Adds CloudAutoStagingBeanFactoryPostProcessor bean: • Modifies bean definitions • Implements auto-reconfiguration § Defines CloudApplicationContextInitializer in web.xml • Activates cloud profile • Defines PropertySource for CloudEnvironment properties CONFIDENTIALFriday, July 13, 12 36

Request handling flow HTTP Request HTTP Request HTTP Request Load Balancer Router Application Load Balancer Router DEA CONFIDENTIALFriday, July 13, 12 37

Self Healing Infrastructure §If an application instance crashes •DEA detects unexpected exit => DEA broadcasts message •Routers remove instance from routing •Health manager notifies Cloud Controller •Cloud Controller re-launches instance §If an DEA VM crashes •Application instances become unavailable •Health Manager notices the missing instances and notifies the Cloud Controller •Cloud Controller requests application instances to be started •Existing DEA will reply and start the applications CONFIDENTIALFriday, July 13, 12

But where are the VMs?!? § Cloud Foundry has a layered architecture • Cloud Controller, DEA, … layer = processes • VM management layer § Benefits: decoupling, simplicity, …. § CloudFoundry.com creates vSphere VMs running different Cloud Foundry components CONFIDENTIALFriday, July 13, 12

using Micro Cloud Foundry CONFIDENTIALFriday, July 13, 12 40

What is in Micro Cloud Foundry? App Instances Services Open source Platform as a Service project 10.04 A cloud packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS CONFIDENTIALFriday, July 13, 12 41

Pre-requisites for using Micro Cloud Foundry Minimum 1 GB Minimum 8 GB Internet Connectivity RAM Disk (w/DHCP is ideal) VMC STS Command line GUI CONFIDENTIALFriday, July 13, 12 42

Register and login to CloudFoundry.com CONFIDENTIALFriday, July 13, 12 43

Select a domain name *.cloudfoundry.me => IP address of your machine where Micro Cloud Foundry is running CONFIDENTIALFriday, July 13, 12 44

Remember the configuration token CONFIDENTIALFriday, July 13, 12 45

Download, unzip and open micro.vmx CONFIDENTIALFriday, July 13, 12 46

Configure Micro Cloud Foundry CONFIDENTIALFriday, July 13, 12 47

Configure Micro Cloud Foundry Pick password CONFIDENTIALFriday, July 13, 12 47

Configure Micro Cloud Foundry Pick password Enter token CONFIDENTIALFriday, July 13, 12 47

Logging into Micro Cloud Foundry CONFIDENTIALFriday, July 13, 12 48

Logging into Micro Cloud Foundry $ vmc target api.cer-micro1.cloudfoundry.me Successfully targeted to [http://api.cer-micro1.cloudfoundry.me] CONFIDENTIALFriday, July 13, 12 48

Logging into Micro Cloud Foundry $ vmc target api.cer-micro1.cloudfoundry.me Successfully targeted to [http://api.cer-micro1.cloudfoundry.me] $ vmc register Email: crichardson@vmware.com Password: ******** Verify Password: ******** Creating New User: OK Attempting login to [http://api.cer-micro1.cloudfoundry.me] Successfully logged into [http://api.cer-micro1.cloudfoundry.me] CONFIDENTIALFriday, July 13, 12 48

Agenda § Why Cloud? Why PaaS? § Introducing Cloud Foundry § Cloud Foundry for Spring developers § Developing NoSQL applications for Cloud Foundry • Why NoSQL? • Overview of NoSQL databases • Introduction to Spring Data • Using Spring Data for Redis • Using Spring Data for Mongo • Deploying on Cloud Foundry § Application integration with RabbitMQ and Spring AMQP § Wrap up 49Friday, July 13, 12

Cloud Foundry provides NoSQL-aaS But what’s a NoSQL database? Why would you want to use it? How do you use it? 50Friday, July 13, 12

Relational databases are great... § SQL • High-level • Sorting • Aggregation § ACID semantics § Well supported • JDBC • Hibernate/JPA • Spring § Well understood • Developers • Operators 51Friday, July 13, 12

... but they have limitations § Object/relational impedance mismatch § Complicated to map rich domain model to relational schema § Difficult to handle semi-structured data, e.g. varying attributes § Schema changes § Extremely difficult/impossible to scale § Poor performance for some use cases 52Friday, July 13, 12

Solution: Spend Money OR http://upload.wikimedia.org/wikipedia/commons/e/e5/Rising_Sun_Yacht.JPG http://www.trekbikes.com/us/en/bikes/road/race_performance/madone_5_series/madone_5_2/# 53Friday, July 13, 12

Solution: Use NoSQL Benefits • Higher performance • Higher scalability • Richer data-model • Schema-less Drawbacks • Limited transactions • Relaxed consistency • Unconstrained data 54Friday, July 13, 12

Growing in popularity… 55Friday, July 13, 12

Future = multi-paradigm data storage for enterprise applications e.g. Netflix • RDBMS • SimpleDB • Cassandra • Hadoop/Hbase IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg 56Friday, July 13, 12

Agenda § Why Cloud? Why PaaS? § Introducing Cloud Foundry § Cloud Foundry for Spring developers § Building Java applications on Cloud Foundry § Moving Spring applications to the Cloud § Developing NoSQL applications for Cloud Foundry • Why NoSQL? • Overview of NoSQL databases • Introduction to Spring Data • Using Spring Data for Redis • Using Spring Data for Mongo • Deploying on Cloud Foundry § Application integration with RabbitMQ and Spring AMQP § Wrap up 57Friday, July 13, 12

Redis § Advanced key-value store § Very fast § Optional persistence § Transactions with optimistic locking § Master-slave replication § Sharding using client-side consistent hashing 58Friday, July 13, 12

Redis § Advanced key-value store § Very fast § Optional persistence K1 V1 § Transactions with optimistic locking K2 V2 § Master-slave replication K3 V2 § Sharding using client-side consistent hashing 58Friday, July 13, 12

Adding members to a sorted set Redis Server Value Score Key zadd myset 5.0 a 59Friday, July 13, 12

Adding members to a sorted set Redis Server Value Score Key a zadd myset 5.0 a myset 5.0 59Friday, July 13, 12

Adding members to a sorted set Redis Server a b zadd myset 10.0 b myset 5.0 10. 60Friday, July 13, 12

Adding members to a sorted set Redis Server c a b zadd myset 1.0 c myset 1.0 5.0 10. 61Friday, July 13, 12

Retrieving members by index range Start Stop Index Index Key Redis Server zrange myset 0 1 c a b myset 1.0 5.0 10. 62Friday, July 13, 12

Retrieving members by index range Start Stop Index Index Key Redis Server zrange myset 0 1 c a b myset 1.0 5.0 10. c a 62Friday, July 13, 12

Retrieving members by score Key Min. Max. Score Score Redis Server zrangebyscore myset 1 6 c a b myset 1.0 5.0 10. 63Friday, July 13, 12

Retrieving members by score Key Min. Max. Score Score Redis Server zrangebyscore myset 1 6 c a b myset 1.0 5.0 10. c a 63Friday, July 13, 12

Redis use cases § Drop-in replacement for Memcached • Session state • Cache of data retrieved from SOR § Replica of SOR for queries needing high-performance § Handling tasks that overload an RDBMS • Hit counts - INCR • Most recent N items - LPUSH and LTRIM • Randomly selecting an item – SRANDMEMBER • Queuing – Lists with LPOP, RPUSH, …. • High score tables – Sorted sets and ZINCRBY •… § Notable users: github, guardian.co.uk, …. 64Friday, July 13, 12

MongoDB § Document-oriented database • JSON-style documents: Lists, Maps, primitives • Schema-less § Transaction = update of a single document § Rich query language for dynamic queries § Very fast § Writes are asynchronous! § Highly scalable and available 65Friday, July 13, 12

Data model = Binary JSON documents { "name" : "Sahn Maru", "type" : ”Korean", "serviceArea" : [ "94619", "94618" ], "openingHours" : [ { Sequence of "dayOfWeek" : "Wednesday", "open" : 1730, bytes on "close" : 2230 disk è fast } ], i/o "_id" : ObjectId("4bddc2f49d1505567c6220a0") } 66Friday, July 13, 12

Data model = Binary JSON documents Collection: Restaurants { "name" : "Sahn Maru", "type" : ”Korean", "serviceArea" : [ "94619", "94618" ], "openingHours" : [ { Sequence of "dayOfWeek" : "Wednesday", "open" : 1730, bytes on "close" : 2230 disk è fast } ], i/o "_id" : ObjectId("4bddc2f49d1505567c6220a0") } 66Friday, July 13, 12

Data model = Binary JSON documents Database: Food To Go Collection: Restaurants { "name" : "Sahn Maru", "type" : ”Korean", "serviceArea" : [ "94619", "94618" ], "openingHours" : [ { Sequence of "dayOfWeek" : "Wednesday", "open" : 1730, bytes on "close" : 2230 disk è fast } ], i/o "_id" : ObjectId("4bddc2f49d1505567c6220a0") } 66Friday, July 13, 12

Data model = Binary JSON documents Server Database: Food To Go Collection: Restaurants { "name" : "Sahn Maru", "type" : ”Korean", "serviceArea" : [ "94619", "94618" ], "openingHours" : [ { Sequence of "dayOfWeek" : "Wednesday", "open" : 1730, bytes on "close" : 2230 disk è fast } ], i/o "_id" : ObjectId("4bddc2f49d1505567c6220a0") } 66Friday, July 13, 12

MongoDB CLI > r = {name: Ajanta} > db.restaurants.save(r) > r { "_id" : ObjectId("4e555dd9646e338dca11710c"), "name" : "Ajanta" } > r = db.restaurants.findOne({name:"Ajanta"}) { "_id" : ObjectId("4e555dd9646e338dca11710c"), "name" : "Ajanta" } > r.type= "Indian” > db.restaurants.save(r) > db.restaurants.update({name:"Ajanta"}, {$set: {name:"Ajanta Restaurant"}, $push: { menuItems: {name: "Chicken Vindaloo"}}}) > db.restaurants.find() { "_id" : ObjectId("4e555dd9646e338dca11710c"), "menuItems" : [ { "name" : "Chicken Vindaloo" } ], "name" : "Ajanta Restaurant", "type" : "Indian" } > db.restaurants.remove(r.id) 67Friday, July 13, 12

MongoDB query by example { Find a restaurant serviceArea:"94619", that serves the openingHours: { 94619 zip code and is open at 6pm on a $elemMatch : { Monday "dayOfWeek" : "Monday", "open": {$lte: 1800}, "close": {$gte: 1800} } } } DBCursor cursor = collection.ﬁnd(qbeObject); while (cursor.hasNext()) { DBObject o = cursor.next(); … } 68Friday, July 13, 12

MongoDB use cases § Use cases • High volume writes • Complex data • Semi-structured data § Who is using it? • Shutterfly, Foursquare • Bit.ly Intuit • SourceForge, NY Times • GILT Groupe, Evite, • SugarCRM 69Friday, July 13, 12

Other NoSQL databases Type Examples Extensible columns/Column-oriented Hbase SimpleDB DynamoDB Graph Neo4j Key-value Membase Voldemort Document CouchDb http://nosql-database.org/ lists 122+ NoSQL databases 70Friday, July 13, 12

Other NoSQL databases Type Examples Extensible columns/Column-oriented Hbase ri te vo SimpleDB DynamoDB fa Graph Neo4j yo ur o ut Key-value left Membase I if Voldemort or ry Document S CouchDb http://nosql-database.org/ lists 122+ NoSQL databases 70Friday, July 13, 12

Spring Data is here to help For NoSQL databases http://www.springsource.org/spring-data 72Friday, July 13, 12

Spring Data sub-projects § SQL: Spring Data JPA, JDBC extensions § Commons: Polyglot persistence § Key-Value: Redis, Riak § Document: MongoDB § Graph: Neo4j § GORM for NoSQL 73Friday, July 13, 12

What you get § Template classes that hide the boilerplate code § Auto-generated (generic) repositories § Java NoSQL mapping § Cross Store Persistence § Support in Roo and Grails 74Friday, July 13, 12

Redis challenges § Connection management • Need to get and close connections § Data mapping • Redis = binary/strings • Application = objects § Multiple client libraries • Gratuitously different APIs 76Friday, July 13, 12

Spring Data for Redis § Low-level - RedisConnection(Factory) • Supports Jedis, Jredis and Rjc • Insulates client code from underlying library § High-level - RedisTemplate • Builds on RedisConnection(Factory) • Connection management • Pluggable Java binary conversion § Support classes: • Collections-backed by RedisTemplate • Atomic Counters 77Friday, July 13, 12

Low-level API = RedisConnection(Factory) 78Friday, July 13, 12

Using RedisConnectionFactory public class LowLevelRedisTest { @Autowired private RedisConnectionFactory redisConnectionFactory; @Test public void testLowLevel() { Library independent code J RedisConnection con = null; try { con = redisConnectionFactory.getConnection(); Ugly byte arrays L byte[] key = "foo".getBytes(); byte[] value = "bar".getBytes(); con.set(key, value); byte[] retrievedValue = con.get(key); Assert.assertArrayEquals(value, retrievedValue); } ﬁnally { if (con != null) { con.close(); } Need to clean up L } } 79Friday, July 13, 12

Configuring RedisConnectionFactory @Conﬁguration public class RedisConﬁguration { @Value("${databaseHostName}") protected String databaseHostName; @Bean public RedisConnectionFactory jedisConnectionFactory() { JedisConnectionFactory factory = new JedisConnectionFactory(); factory.setHostName(databaseHostName); factory.setPort(6379); factory.setUsePool(true); return factory; } } 80Friday, July 13, 12

High-level API = RedisTemplate § Builds on RedisConnection(Factory) § Analogous to JdbcTemplate § Parameterized type • K - Key type • V – Value type § Handles Java Key/Value Redis byte[] § Maps Redis exceptions DataAccessException § StringRedisTemplate • Extends RedisTemplate<String, String> • Keys and values are Strings 81Friday, July 13, 12

Using StringRedisTemplate public class RedisTemplateTest { @Autowired private StringRedisTemplate stringRedisTemplate; @Test Returns KV type speciﬁc interface public void testGetAndSet() { stringRedisTemplate.opsForValue().set("foo", "bar"); assertEquals("bar", stringRedisTemplate.opsForValue().get("foo")); } @Test Converts between Strings and byte[] public void testHashOps() { stringRedisTemplate.opsForHash().put("myHash", "myKey", "value"); assertEquals("value", stringRedisTemplate.opsForHash().get("myHash", "myKey")); assertEquals(Collections.singleton("myKey"), stringRedisTemplate.opsForHash().keys("myHash")); assertEquals(Collections.singletonMap("myKey", "value"), stringRedisTemplate.opsForHash().entries("myHash")); } 82Friday, July 13, 12

Configuring StringRedisTemplate @Conﬁguration public class RedisConﬁguration { @Bean public RedisConnectionFactory jedisConnectionFactory() { … } @Bean public StringRedisTemplate stringRedisTemplate(RedisConnectionFactory factory) { StringRedisTemplate template = new StringRedisTemplate(); template.setConnectionFactory(factory); return template; } } 83Friday, July 13, 12

RedisTemplate: Java objects binary data § RedisTemplate has multiple Serializers: • DefaultSerializer - defaults to JdkSerializationRedisSerializer • KeySerializer • ValueSerializer • HashKeySerializer • HashValueSerializer 84Friday, July 13, 12

StringRedisTemplate uses StringRedisSerializer 85Friday, July 13, 12

Register serializers to override the default behavior Converted to JSON by RedisTemplate 86Friday, July 13, 12

Redis caching support KVs = <preﬁx + K,V> Template needs to (de)serialize K and V Sorted set of all keys for clear() 87Friday, July 13, 12

Other Spring data for Redis features § Redis-backed collections § Atomic counters § Support for Redis Pub/sub 88Friday, July 13, 12

MongoDB API usage patterns § Create and store Mongo singleton § Externalized server host, port etc. § Inserts/Updates • Map application POJO DBObject • mongo.getDatabase(…).getCollection(…) • Partial document updates • Configure asynchronous vs. synchronous writes § Queries • Construct query object • mongo.getDatabase(…).getCollection(…) • Iterate through Cursor • Map DBObject application POJO Higher-level than JDBC but still repetitive, … 90Friday, July 13, 12

Spring Data - MongoDB § MongoTemplate § Generic repositories § Querydsl integration § Cross-store persistence 91Friday, July 13, 12

MongoTemplate Simplifies data access MongoTemplate POJO ó DBObject Translates exceptions databaseName mapping userId Password defaultCollectionName writeConcern writeResultChecking save() <<interface>> insert() MongoConvertor remove() updateFirst() write(Object, DBObject) findOne() read(Class, DBObject) find() … uses Mongo MongoMapping (Java Driver class) Converter 92Friday, July 13, 12

Example entity public class Restaurant { private String id; private String name; private List<MenuItem> menuItems; public Restaurant() { } public class MenuItem { public Restaurant(String name) { private String name; this.name = name; private double price; … } public MenuItem() { } public String getName() { return name; } public MenuItem(String name, public void setName(String name) { double price) { this.name = name; this.name = name; } this.price = price; } …getters and setters… …getters and setters… 93Friday, July 13, 12

Example data access code @Repository public class RestaurantRepository { @Autowired private MongoTemplate mongoTemplate; public static final String RESTAURANTS_COLLECTION = "restaurants"; public void add(Restaurant restaurant) { mongoTemplate.save(RESTAURANTS_COLLECTION, restaurant); } public List<Restaurant> findRestaurantsByName(String restaurantName) { return mongoTemplate.find(RESTAURANTS_COLLECTION, new Query(where("name").is(restaurantName)), Restaurant.class); } 94Friday, July 13, 12

Mongo document { "_id" : ObjectId("4d977f55d3fe3119c904e026"), "menuItems" : [ { "name" : "Tandoori Portobello Mushrooms", "price" : 5.5 }, { "name" : "Duck Curry Kerala", "price" : 15 } ], "name" : "Ajanta" } 95Friday, July 13, 12

Spring MongoDB Example - Config 1 @Configuration public class MongoDbExampleConfig { private @Value("#{mongoDbProperties.databaseName}") String mongoDbDatabase; private @Value("#{mongoDbProperties.host}") String mongoDbHost; @Bean public Mongo mongo() throws Exception { return new Mongo(mongoDbHost); } @Bean public MongoTemplate mongoTemplate(Mongo mongo) { MongoTemplate mongoTemplate = new MongoTemplate(mongo, mongoDbDatabase); mongoTemplate.setWriteConcern(WriteConcern.SAFE); mongoTemplate.setWriteResultChecking(WriteResultChecking.EXCEPTION); return mongoTemplate; } … <beans> <context:annotation-config/> External Config <context:component-scan base-package="net.chrisrichardson.mongodb.example"/> <util:properties id="mongoDbProperties" mongodb.properties: location="mongodb.properties"/> </beans> databaseName=demo1 host=192.168.253.150 96Friday, July 13, 12

Spring MongoDB Example - Config 2 <bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> <constructor-arg ref="mongoFactory"/> </bean> <mongo:db-factory id="mongoFactory" host= "#{mongoDbProperties.host}" dbname="#{mongoDbProperties.databaseName}" /> <util:properties id="mongoDbProperties" location="mongodb.properties"/> 97Friday, July 13, 12

Summarize other features § In-place updates § Callbacks § Generic repositories § Annotation-driven mapping § Support for QueryDSL § Cross-store persistence 98Friday, July 13, 12

Using Mongo and Redis with Cloud Foundry § Create a service - Mongo or Redis § Bind it to your application § Use <cloud:*/> namespace to access the bound service • when cloud profile is active 100Friday, July 13, 12

Creating a Redis Server 101Friday, July 13, 12

Deploying a Redis application 102Friday, July 13, 12

Redis bean definitions 103Friday, July 13, 12

Using the application 104Friday, July 13, 12

About <cloud:redis-connection-factory/> <cloud:redis-connection-factory id="redisConnectionFactory" Use when multiple [ service-name="redis1" ] services are bound /> 105Friday, July 13, 12

Deploying a Mongo application 106Friday, July 13, 12

MongoDB bean definitions 107Friday, July 13, 12

Using the Mongo Application 108Friday, July 13, 12

About <cloud:mongo-db-factory/> Use when multiple services are bound <cloud:mongo-db-factory id="mongoFactory" [ service-name="mongo1" ] [ write-concern="SAFE" ] Whether to wait > [ <cloud:mongo-options for writes to complete connections-per-host="..." max-wait-time="..." /> ] </cloud:mongo-db-factory> 109Friday, July 13, 12

Cross store persistence example 110Friday, July 13, 12

Uses MySQL and MongoDB @Entity public class Customer { Stored in MySQL @Id @GeneratedValue(strategy = GenerationType.IDENTITY) private Long id; private String ﬁrstName; private String lastName; Stored in Mongo @RelatedDocument private SurveyInfo surveyInfo; public class SurveyInfo { public class Survey { private List<Survey> questionsAndAnswers = new ArrayList<Survey>(); String question; public List<Survey> getQuestionsAndAnswers() { String answer; return questionsAndAnswers; } 111Friday, July 13, 12

Cross store configuration @Conﬁguration @ComponentScan(basePackageClasses = CrossStoreCustomerRepository.class) @EnableTransactionManagement(mode = AdviceMode.ASPECTJ) public class ServicesConﬁguration { private String mongoDatabaseServiceName = "survey-mongo"; private String mysqlDatabaseServiceName = "survey-mysql"; @Bean public CloudEnvironment cloudEnvironment() { return new CloudEnvironment(); } @Bean public MongoServiceInfo mongoServiceInfo() { return cloudEnvironment().getServiceInfo(mongoDatabaseServiceName, MongoServiceInfo.class); } @Bean public MongoDbFactory mongoDbFactory() { MongoServiceCreator mongoServiceCreator = new MongoServiceCreator(); return mongoServiceCreator.createService(mongoServiceInfo()); } @Bean public DataSource dataSource() { RdbmsServiceInfo rdbmsServiceInfo = cloudEnvironment().getServiceInfo(mysqlDatabaseServiceName, RdbmsServiceInfo.class); RdbmsServiceCreator rdbmsServiceCreator = new RdbmsServiceCreator(); DataSource dataSource = rdbmsServiceCreator.createService(rdbmsServiceInfo); return dataSource; } ... 112Friday, July 13, 12

Manifest for Cloud Foundry deployment --- applications: target: name: xs-survey url: ${name}.${target-base} framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: survey-mongo: type: :mongodb survey-mysql: type: :mysql Chris-Richardsons-Mac-Pro:cross-store cer$ vmc apps +-------------+----+---------+-------------------------------------------------------+----------------------------+ | Application | # | Health | URLS | Services | +-------------+----+---------+-------------------------------------------------------+----------------------------+ | xs-survey | 1 | RUNNING | xs-survey.cloudfoundry.com | survey-mysql, survey-mongo | +-------------+----+---------+-------------------------------------------------------+----------------------------+ 113Friday, July 13, 12

NoSQL and Caldecott § Caldecott let’s you tunnel to a NoSQL service § Use Redis CLI • redis-cli • Explore database, adhoc operations • ... § Use Mongo CLI etc • Explore database, adhoc operations • Mongo dump/restore • ... 114Friday, July 13, 12

NoSQL wrap up § Cloud Foundry supports Mongo and Redis § For some use cases, NoSQL databases offer some combination of: • Higher scalability • Higher performance • Richer data models • Schema less § Spring Data simplifies the development of NoSQL applications Cloud Foundry Easy development = and + deployment of Spring Data NoSQL applications 115Friday, July 13, 12

Agenda § Why Cloud? Why PaaS? § Introducing Cloud Foundry § Cloud Foundry for Spring developers § Building Java applications on Cloud Foundry § Moving Spring applications to the Cloud § Developing NoSQL applications for Cloud Foundry § Application integration with RabbitMQ and Spring AMQP •Why messaging? •Messaging with RabbitMQ and AMQP •Using Spring Integration •Cloud Foundry and RabbitMQ § Wrap up 116Friday, July 13, 12

Cloud Foundry provides RabbitMQ - aaS 118Friday, July 13, 12

But why messaging? Why RabbitMQ? Application A Application B RabbitMQ Traditional application integration 119Friday, July 13, 12

But why messaging? Why RabbitMQ? Application Service A Service B Service ... RabbitMQ • Essential component of our new scalable and fault tolerant architecture • Integration mechanism for the services • Enables services to discover each other 120Friday, July 13, 12

Let’s imagine you are building an e- commerce application 121Friday, July 13, 12

wgrus-monolithic.war @Controller StoreFront Widget Gadget Accounting Shipping InventoryService InventoryService Service Service RDBMS 122Friday, July 13, 12

It’s simple to develop but .... § Lack of scalability • Scale through replication • Non-replicable component => nothing can be replicated • Can’t scale different parts of the application differently § Lack of deployability • Deploy it all in one go • Increased risk of something breaking § Applications are brittle • Store can’t accept orders unless all services are available • Failure (e.g. memory leak) in one component can take down every other § Monolingual • Can’t use non-JVM server-side technologies: NodeJS, Rails, 123Friday, July 13, 12

Decompose application into services By noun or by verbs 124Friday, July 13, 12

wgrus-billing.war Accounting Service wgrus-shipping.war wgrus-store.war Shipping Synchronous Service Spring StoreFront Remoting MySQL Spring Web Services wgrus-inventory.war Widget InventoryService wgrus-inventory.war Gadget InventoryService 125Friday, July 13, 12

Benefits and Drawbacks § Benefits: • Scale each service independently • Deploy each service independently • Mix JVM and non-JVM languages § Drawbacks • Application is still brittle • Store can’t accept orders unless all services are available • Failure (e.g. memory leak) in one component can take down every other 126Friday, July 13, 12

Solution: Asynchronous Architecture 127Friday, July 13, 12

wgrus-billing.war Accounting Service wgrus-inventory.war wgrus-store.war Widget InventoryService Message StoreFront Broker MySQL wgrus-inventory.war Gadget InventoryService wgrus-shipping.war Shipping Service 128Friday, July 13, 12

Benefits and Drawbacks § Benefits: • Scale each service independently • Deploy each service independently • Mix JVM and non-JVM languages • Improved availability • Front-end keeps working even when backend services are down • Messaging broker can buffer traffic and smooth out spikes § Drawbacks • Yet another moving part • Sometimes synchronous RPC is a better fit 129Friday, July 13, 12

RabbitMQ – Messaging that Just Works Robust High-performance Easy to use AMQP LEADERFriday, July 13, 12

Why AMQP? A Protocol, not an API •A defined set of messaging capabilities called the AMQ model •A network wire-level protocol, AMQP On commodity hardware •10-‐25 thousand messages per second is rou>ne * •The NIC is usually the boDleneck * Non-‐persistent messages 17Friday, July 13, 12

AMQP Architecture café deliveries queue café NA deliveries queue café WW deliveries queue 20Friday, July 13, 12

AMQP Architecture café deliveries M1 M2 M3 queue café NA deliveries M1 M2 M3 queue café WW deliveries M1 M2 M3 queue 20Friday, July 13, 12

AMQP Architecture café deliveries queue café NA deliveries queue café WW deliveries queue 20Friday, July 13, 12

AMQP Architecture 19Friday, July 13, 12

AMQP Architecture new.order queue 19Friday, July 13, 12

AMQP Architecture new.order new.order binding queue 19Friday, July 13, 12

AMQP Architecture new.order new.order new.order routing key binding queue 19Friday, July 13, 12

AMQP Architecture 21Friday, July 13, 12

AMQP Architecture all_drinks queue 21Friday, July 13, 12

AMQP Architecture all_drinks queue cold_drinks queue 21Friday, July 13, 12

AMQP Architecture all_drinks queue cold_drinks queue hot_drinks queue 21Friday, July 13, 12

AMQP Architecture all_drinks queue k.* drin cold_drinks queue hot_drinks queue 21Friday, July 13, 12

AMQP Architecture all_drinks queue k.* drin drink.cold cold_drinks queue hot_drinks queue 21Friday, July 13, 12

AMQP Architecture all_drinks queue k.* drin drink.cold cold_drinks queue dri nk. ho hot_drinks t queue 21Friday, July 13, 12

AMQP Architecture all_drinks queue k.* drin drink.cold cold_drinks queue dri nk. ho hot_drinks t queue Message Routing Keys: 1.drink.hot 2.drink.cold 3.drink.warm 21Friday, July 13, 12

AMQP Architecture all_drinks 1 2 3 queue k.* drin drink.cold cold_drinks 2 queue dri nk. ho hot_drinks t 1 queue Message Routing Keys: 1.drink.hot 2.drink.cold 3.drink.warm 21Friday, July 13, 12

Spring AMQP § Encapsulates low-level details § Simplifies sending and receiving Producer Consumer of messages Amqp Listener Template Container Spring AMQP AMQPFriday, July 13, 12

Sending AMQP messages @Component public class MessageSender { @Autowired private volatile AmqpTemplate amqpTemplate; public void send(String message) { this.amqpTemplate.convertAndSend( "myExchange", "some.routing.key", message); } } 137Friday, July 13, 12

Receiving AMQP messages public class MyComponent { @Autowired private AmqpTemplate amqpTemplate; public void read() throws Exception { ... String value = amqpTemplate.receiveAndConvert("myQueueName"); ... } } 138Friday, July 13, 12

Spring AMQP: SimpleMessageListenerContainer l Asynchronous message receiver l POJO handlers l Handles re-connection and listener failure (rollback, redelivery) l Message conversion and error handling strategies <listener-container connection-factory="rabbitConnectionFactory"> <listener ref="handler" method="handle" queue-names="my.queue"> </listener-container> 139Friday, July 13, 12

Spring configuration <rabbit:template id="rabbitTemplate" connection-factory="rabbitConnectionFactory"/> <rabbit:connection-factory id="rabbitConnectionFactory"/> 140Friday, July 13, 12

Spring AMQP is flexible and dynamic BUT It’s very low level 141Friday, July 13, 12

Spring Integration § Builds on Spring framework § High-level of abstraction for building message based applications § Implements EAI patterns § Provides plumbing for exchanging messages between application components § Promotes loosely coupled components § Integrates with external messaging infrastructure: JMS, AMQP, HTTP, Email, File transfer 143Friday, July 13, 12

Spring Integration concepts § Message channel • Virtual pipe connecting producer and consumer § Message endpoints • The filter of a pipes-and-filter architecture • Read from and/or write to channel § Endpoint types: • Transformer • Filter • Router • Splitter • Aggregator • ServiceActivator • Inbound channel adapter - read from external source, writes to channel • Outbound channel adapter - read from channel write to external destination 144Friday, July 13, 12

Example of reconfigurability - local @Service public class OrderServiceImpl { @Service public class ShippingServiceImpl { @Autowired private ShippingService shippingService; public void shipOrder(String orderId) { System.out.println("shipped order: " + public void placeOrder() { orderId); String orderId = generateOrderId(); } … shippingService.shipOrder(orderId); } } } Order Shipping Service service Messaging Gateway Channel Service Activator 145Friday, July 13, 12

Example of reconfigurability - distributed Code unchanged in new deployment Order Shipping Service service Messaging Gateway RabbitMQ Channel AMQP AMQP Channel Service Activator 146Friday, July 13, 12

Using Spring Integration with the web store application 147Friday, July 13, 12

wgrus-store.war wgrus-shipping.war StoreFront Shipping Service wgrus-billing.war wgrus-inventory.war Credit Service Spring Integration Logic Widget InventoryService Gadget InventoryService 148Friday, July 13, 12

Store front flow StoreUI object to Message Endpoint orderChannel amqpOut AMQP JSON 149Friday, July 13, 12

Inventory flow Json to credit check inventory inventory Object to AMQP inventory AMQP Object enricher encricher AMQP Out JSON inventory router credit check credit check Content Based service Router activator Credit Service widget gadget inventory inventory service service 150Friday, July 13, 12

Shipping flow shipping service Json to AMQP shipping Service Object order Activator channel 151Friday, July 13, 12

Rabbit on Cloud Foundry 153Friday, July 13, 12

Configuring a ConnectionFactory <rabbit:template id="rabbitTemplate" connection-factory="rabbitConnectionFactory"/> <beans proﬁle="default"> ... <rabbit:connection-factory id="rabbitConnectionFactory"/> </beans> <beans proﬁle="cloud"> ... <cloud:rabbit-connection-factory id="rabbitConnectionFactory"/> </beans> 154Friday, July 13, 12

Using Caldecott with RabbitMQ § Use for JUnit/Integration tests § Run RabbitMQ tools xxxxx xxxxx xxxxx 155Friday, July 13, 12

Summary § Modern applications need to have message-based architecture § Spring Integration abstracts away the low-level aspects of messaging § Cloud Foundry simplifies the development and deployment of RabbitMQ-based applications 156Friday, July 13, 12

Agenda § Why Cloud? Why PaaS? § Introducing Cloud Foundry § Cloud Foundry for Spring developers § Developing NoSQL applications for Cloud Foundry § Application integration with RabbitMQ and Spring AMQP § Wrap Up 157Friday, July 13, 12

Josh Long | josh.long@springsource.com | @starbuxman Questions? Signup @ www.cloudfoundry.com using promo code: CFOpenTour2012 CONFIDENTIALFriday, July 13, 12

Spring in-the-cloud

by Joshua Long on Jul 14, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics