MongoUK 2012

Andy Piper social bridgebuilder, techie Developer Advocate @ Cloud Foundry 15 years in enterprise middleware & messaging social web enthusiast OSS supporter / contributor excited by “what’s next”, Internet of Things, etc. @andypiper apiper@vmware.com CONFIDENTIAL 2Wednesday, June 20, 12

About Monica Wilkinson Loves the web and data portability. Developer Advocate @ Cloud Foundry 12 years development experience. Last 5 years in Social Web Open Web Standards Advocate Contact Me: @ciberch mwilkinson@vmware.com CONFIDENTIAL 3Wednesday, June 20, 12

Agenda 1.Overview of Cloud Foundry 2.What is Activity Streams ? 3.Building an Activity Stream Engine CONFIDENTIAL 4Wednesday, June 20, 12

About Cloud Foundry The first Open PaaS Multi(n) Languages, Frameworks, Services & Clouds Open Source CONFIDENTIAL 5Wednesday, June 20, 12

Traditional web app architecture ! Browser client Web Application ! Monolithic WAR / other deployment artefact StoreFront ! RDBMS AccountingService MySQL Desktop Database Apache Browser InventoryService “easy” to... ShippingService ! develop ! test Container (e.g. Tomcat) ! deploy ! scale ...apart from: provisioning, setup, need to replicate, shard, scale to massive numbers of mobile clients, provide polyglot support, run multiple dev teams, collaborate... CONFIDENTIAL 6Wednesday, June 20, 12

Need to handle massive loads and the data explosion Scalable architectures! ! Application tier: •Replicated/clustered servers •Modular so that components can be scaled differently •Asynchronous architecture - communication via a message broker ! Database tier: •Replication •Sharding •Polyglot persistence: Relational, NoSQL, NewSQL databases CONFIDENTIAL 7Wednesday, June 20, 12

Modern application architecture billing web applicationDesktop Browser Accounting Service StoreUI Native Mobile NodeJS NodeJS inventory web application Application front-end MySQL StoreUI application StoreUI InventoryService HTML5 mobile application StoreUI Redis shipping web application RabbitMQ Mongo ShippingService CONFIDENTIAL 8Wednesday, June 20, 12

Platform-as-a-Service is the solution Deployment Services Easy deployment SQL databases Application management NoSQL databases + Easy scaling up and down Message Broker CONFIDENTIAL 9Wednesday, June 20, 12

The Open Platform as a Service “Deploy and scale applications in seconds, without locking yourself into a single cloud or vendor” , xible n, Fle Ope e ple, l Si m Sc alab CONFIDENTIAL 10Wednesday, June 20, 12

Cloud Foundry open PaaS - Choice of clouds OSS community Private( Ap I) Data Services CP Clouds( pli vFabric ( ca ce Postgres tio r fa nS e Int er Public v ic r ide eI Msg Services ov Clouds nt e Pr r fa vFabric RabbitMQTM d ou ce Cl Micro Other Services Clouds to h: d eploy c) ! bos et N EW PIs (AWS n at e C Additional partner services … alter CONFIDENTIALWednesday, June 20, 12

Multi-cloud flexibility is critical to long-term success ! Make use of both public and private clouds without rewriting your applications ! Protect against vendor lock-in ! Meet different compliance and geographical needs ! Accommodate peak loads while optimizing costs ! Manage your growth and changing needs over time CONFIDENTIAL 12Wednesday, June 20, 12

Getting Started on Cloud Foundry CONFIDENTIAL 13Wednesday, June 20, 12

Register! http://cloudfoundry.com/signup optional promo CONFIDENTIAL 14Wednesday, June 20, 12

Ways to use Cloud Foundry command line... IDE plugin... toolchain plugin... Cloud IDEs CONFIDENTIAL 15Wednesday, June 20, 12

Example vmc commands $"gem"install"vmc $"vmc"target"<any"cloud> $"vmc"login"<credentials> $"vmc"push"<my7app> " "">"bind"services?"Yes $"vmc"update"<my7app> $"vmc"instances"<my7app>"+100" CONFIDENTIAL 16Wednesday, June 20, 12

Logging into CloudFoundry.com$"vmc"target"api.cloudfoundry.comSuccessfully"targeted"to"""""""""[http://api.cloudfoundry.com]$"vmc"loginAttempting"login"to"""[http://api.cloudfoundry.com]Email:"apiper@vmware.comPassword:"********Successfully"logged"into"""[http://api.cloudfoundry.com] CONFIDENTIAL 17Wednesday, June 20, 12

Cloud Foundry Integration plugin for Eclipse CONFIDENTIAL 18Wednesday, June 20, 12

What is Activity Streams ? CONFIDENTIAL 19Wednesday, June 20, 12

http://activityStrea.ms ! Activity Streams is a simple open specification used to describe social actions around the web. ! The goal of Activity Streams is data portability ! Some publishers of Activity Streams: Socialcast, MySpace, Facebook and G+ ! The default format is JSON but Atom is also supported. The JSON format was added after we saw some initial limitations with JSON ! Core concepts are: Actor, Verb, Object and Target CONFIDENTIAL 20Wednesday, June 20, 12

Actor -> Monica Verb -> Pinned(Bookmarked) Object -> Rosa Parks’ Bio Page Target -> Female Leaders Board Source -> Pinterest Published -> MLK Day 2012 © 2012 VMware, Inc. All rights reservedWednesday, June 20, 12

With Timeline Actions Posting with a richer protocol like Activity Streams allows you to do: - Richer messages and links - Aggregate - Assign relevance © 2012 VMware, Inc. All rights reservedWednesday, June 20, 12

Example in JSON CONFIDENTIAL 23Wednesday, June 20, 12

What is an Activity Stream Engine ? ! An activity stream engine allows you to publish events and subscribe to events. ! Many activity stream engines support aggregation of events via Streams. ! Many activity stream engines support fanning out activities to subscribers. CONFIDENTIAL 24Wednesday, June 20, 12

So how did we build this Activity Stream Engine on Cloud Foundry ? CONFIDENTIAL 25Wednesday, June 20, 12

Building an Activity Streams Engine CONFIDENTIAL 26Wednesday, June 20, 12

What should we use to store Activities ?Key-Value Column Document Graph Redis, Cassandra, MongoDB Neo4J Riak HBase CONFIDENTIAL 27Wednesday, June 20, 12

Mongo DB schema-less documents are perfect for ActivitiesKey-Value Column Document Graph MongoDB (so easy) CONFIDENTIAL 28Wednesday, June 20, 12

About MongoDB 1. No-SQL database development : { 2. Stores JSON-style tools: many, documents with language_support: superb, embedded documents = agility: high 3. Horizontally scalable }, production: { 4. Full Indexing Support speed: fast 5. Open Source so great fault_tolerance: true ORMs and drivers. scalability: high CONFIDENTIAL 29Wednesday, June 20, 12

Step 1- Build the persistence Model var MediaLinkHash = { duration: Number, height: Number, width: Number, url: String }; var ActivityObjectHash = { id: {type: String}, image: MediaLinkHash, displayName: {type: String}, summary: {type: String}, content: {type: String}, url: {type:String}, author: {type: ObjectId, ref: activityObject}, published: {type: Date, default: Date.now}, updated: {type: Date, default: Date.now}, objectType: {type: String}, attachments: [{type: ObjectId, ref: activityObject}], upstreamDuplicates: [String], downstreamDuplicates: [String] }; var ActivityObjectSchema = new Schema(ActivityObjectHash) this.ActivityObject = mongoose.model(activityObject, ActivityObjectSchema); CONFIDENTIAL 30Wednesday, June 20, 12

Step 2 - Expose helpers for the queries ! The most important is to be able to list the activities in descending order by published time. ! It is also important to hydrate any objects for which we have references ! With Mongoose you can do: ! this.getActivityStream = function(n, fx) { Activity.find().sort(published, descending).limit(n).populate(target).run (fx); } ! https://github.com/ciberch/activity-streams-mongoose CONFIDENTIAL 32Wednesday, June 20, 12

How do we make our engine faster and more scalable ? ! Add support for PubSub ! In a PubSub model as soon as an event is published it gets sent to all the subscribers. ! The Publisher doesn’t know who the subscribers are, it simply knows where to publish. ! Redis has nice support for Pub Sub http://redis.io/topics/ pubsub CONFIDENTIAL 33Wednesday, June 20, 12

Using Socket.io ! Server Side: • Subscribe to a Stream • When there is a new event send it to the client as JSON ! Client Side • Add support to publish activities to the Stream • Change the client rendering to use ActivityStrea.ms ! Benefits • Richer messaging • Activity Syndication CONFIDENTIAL 34Wednesday, June 20, 12

Demo Part 1: App 1 and walkthrough http://blog.cloudfoundry.com/ 2012/05/17/node-activity-streams-app-1/ Part 2: App 2 and walkthrough http://blog.cloudfoundry.com/ 2012/06/05/node-activity-streams-app-2/ Part 3: http://asms.cloudfoundry.com © 2012 VMware, Inc. All rights reservedWednesday, June 20, 12

Thank You - Questions ? Cloud Foundry Activity Streams Libraries are at: https://github.com/organizations/cloudfoundry-samples http://cloudfoundry.com Questions: @cloudfoundry, @ciberch or @andypiper © 2012 VMware, Inc. All rights reservedWednesday, June 20, 12

Resources Primary Site: cloudfoundry.com Open Source Site: cloudfoundry.org Twitter: @cloudfoundry or #cloudfoundry Blog: blog.cloudfoundry.com FB: facebook.com/cloudfoundry Documentation: docs.cloudfoundry.com Full “bootcamp” presentation: slidesha.re/cf-bootcamp © 2012 VMware, Inc. All rights reservedWednesday, June 20, 12

http://nodeslide.herokuapp.com	5
http://www.linkedin.com	4
http://leapf.org	2
https://si0.twimg.com	1

MongoUK 2012

by Monica Wilkinson on Jun 20, 2012

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 12

Statistics

MongoUK 2012 Presentation Transcript