MongoUK 2012

Activity Streams on Cloud Foundry with
MongoDB and Node.js
MongoUK - 2012

By Monica Wilkinson and Andy Piper

© 2012 VMware, Inc. All rights reserved

Wednesday, June 20, 12

Andy Piper social bridgebuilder, techie

Developer Advocate @ Cloud Foundry
15 years in enterprise middleware & messaging
social web enthusiast
OSS supporter / contributor
excited by “what’s next”, Internet of Things, etc.

@andypiper
apiper@vmware.com

CONFIDENTIAL
2


About Monica Wilkinson Loves the web and data portability.

Developer Advocate @ Cloud Foundry
12 years development experience.
Last 5 years in Social Web
Open Web Standards Advocate
Contact Me: @ciberch
mwilkinson@vmware.com

CONFIDENTIAL
3


Agenda
1.Overview of Cloud Foundry
2.What is Activity Streams ?
3.Building an Activity Stream Engine

CONFIDENTIAL
4


About Cloud Foundry The first Open PaaS

Multi(n) Languages, Frameworks,
Services & Clouds
Open Source

CONFIDENTIAL
5


Traditional web app architecture
! Browser client
Web Application
! Monolithic WAR / other
deployment artefact StoreFront

! RDBMS
AccountingService
MySQL
Desktop Database
Apache
Browser
InventoryService

“easy” to...
ShippingService
! develop
! test
Container (e.g. Tomcat)
! deploy
! scale ...apart from: provisioning, setup, need to replicate, shard, scale to massive numbers of
mobile clients, provide polyglot support, run multiple dev teams, collaborate...
CONFIDENTIAL
6


Need to handle massive loads and the data explosion

Scalable architectures!
! Application tier:
•Replicated/clustered servers
•Modular so that components can be scaled differently
•Asynchronous architecture - communication via a message broker
! Database tier:
•Replication
•Sharding
•Polyglot persistence: Relational, NoSQL, NewSQL databases

CONFIDENTIAL
7


Modern application architecture
billing web application

Desktop Browser Accounting
Service
StoreUI

Native Mobile NodeJS
NodeJS inventory web application
Application
front-end MySQL
StoreUI
application
StoreUI InventoryService

HTML5 mobile
application

StoreUI Redis shipping web application
RabbitMQ
Mongo
ShippingService

CONFIDENTIAL
8


Platform-as-a-Service is the solution

Deployment Services

Easy deployment SQL databases

Application management NoSQL databases
+
Easy scaling up and down Message Broker

CONFIDENTIAL
9


The Open
Platform as a Service

“Deploy and scale applications in
seconds, without locking yourself into a
single cloud or vendor”
,
xible
n, Fle
Ope e
ple, l
Si m Sc alab

CONFIDENTIAL
10


Cloud Foundry open PaaS - Choice of clouds
OSS community

Private(
Ap

I)
Data Services

CP
Clouds(
pli

vFabric

(
ca

ce
Postgres
tio

r fa
nS

e
Int
er

Public
v ic

r
ide
eI

Msg Services

ov
Clouds
nt e

Pr
r fa

vFabric RabbitMQTM
d
ou
ce

Cl

Micro
Other Services
Clouds to
h: d eploy c)
! bos et
N EW PIs (AWS
n at e C
Additional partner services … alter
CONFIDENTIAL


Multi-cloud flexibility is critical to long-term success

! Make use of both public and private clouds without rewriting your applications

! Protect against vendor lock-in

! Meet different compliance and geographical needs

! Accommodate peak loads while optimizing costs

! Manage your growth and changing needs over time

CONFIDENTIAL
12


Getting Started on Cloud Foundry

CONFIDENTIAL
13


Register!

http://cloudfoundry.com/signup

optional promo

CONFIDENTIAL
14


Ways to use Cloud Foundry
command line...

IDE plugin...

toolchain plugin...

Cloud IDEs

CONFIDENTIAL
15


Example vmc commands

$"gem"install"vmc

$"vmc"target"<any"cloud>
$"vmc"login"<credentials>
$"vmc"push"<my7app>
" "">"bind"services?"Yes

$"vmc"update"<my7app>
$"vmc"instances"<my7app>"+100"

CONFIDENTIAL
16


Logging into CloudFoundry.com

$"vmc"target"api.cloudfoundry.com
Successfully"targeted"to"
""""""""[http://api.cloudfoundry.com]

$"vmc"login
Attempting"login"to"
""[http://api.cloudfoundry.com]
Email:"apiper@vmware.com
Password:"********
Successfully"logged"into"
""[http://api.cloudfoundry.com]

CONFIDENTIAL
17


Cloud Foundry Integration plugin for Eclipse

CONFIDENTIAL
18


What is Activity Streams ?

CONFIDENTIAL
19


http://activityStrea.ms
! Activity Streams is a simple open specification used to describe social
actions around the web.
! The goal of Activity Streams is data portability
! Some publishers of Activity Streams: Socialcast, MySpace, Facebook
and G+
! The default format is JSON but Atom is also supported. The JSON
format was added after we saw some initial limitations with JSON
! Core concepts are: Actor, Verb, Object and Target

CONFIDENTIAL
20


Actor -> Monica
Verb -> Pinned(Bookmarked)
Object -> Rosa Parks’ Bio Page
Target -> Female Leaders Board
Source -> Pinterest
Published -> MLK Day 2012



With Timeline Actions
Posting with a richer protocol like
Activity Streams allows you to
do:
- Richer messages and links
- Aggregate
- Assign relevance



Example in JSON

CONFIDENTIAL
23


What is an Activity Stream Engine ?
! An activity stream engine allows you to publish events and
subscribe to events.
! Many activity stream engines support aggregation of events
via Streams.
! Many activity stream engines support fanning out activities
to subscribers.

CONFIDENTIAL
24


So how did we build this Activity Stream
Engine on Cloud Foundry ?

CONFIDENTIAL
25


Building an Activity Streams Engine

CONFIDENTIAL
26


What should we use to store Activities ?

Key-Value Column Document Graph

Redis, Cassandra, MongoDB Neo4J
Riak HBase

CONFIDENTIAL
27


Mongo DB schema-less documents are perfect
for Activities

Key-Value Column Document Graph

MongoDB
(so easy)

CONFIDENTIAL
28


About MongoDB

1. No-SQL database development : {

2. Stores JSON-style tools: many,
documents with language_support: superb,
embedded documents

=
agility: high
3. Horizontally scalable },
production: {
4. Full Indexing Support
speed: fast
5. Open Source so great fault_tolerance: true
ORMs and drivers.
scalability: high

CONFIDENTIAL
29


Step 1- Build the persistence Model
var MediaLinkHash = {
        duration: Number,
        height: Number,
        width: Number,
        url: String
    };

    var ActivityObjectHash = {
       id: {type: String},
       image: MediaLinkHash,
       displayName: {type: String},
       summary: {type: String},
       content: {type: String},
       url: {type:String},
       author: {type: ObjectId, ref: 'activityObject'},
       published: {type: Date, default: Date.now},
       updated: {type: Date, default: Date.now},
       objectType: {type: String},
       attachments: [{type: ObjectId, ref: 'activityObject'}],
       upstreamDuplicates: [String],
       downstreamDuplicates: [String]
    };

    var ActivityObjectSchema = new Schema(ActivityObjectHash)
    this.ActivityObject = mongoose.model('activityObject', ActivityObjectSchema);

CONFIDENTIAL
30


Step 2 - Expose helpers for the queries
! The most important is to be able to list the activities in
descending order by published time.
! It is also important to hydrate any objects for which we
have references

! With Mongoose you can do:
! this.getActivityStream = function(n, fx)
{ Activity.find().sort('published',
'descending').limit(n).populate('target').run
(fx);
}
! https://github.com/ciberch/activity-streams-mongoose

CONFIDENTIAL
32


How do we make our engine faster and more
scalable ?
! Add support for PubSub
! In a PubSub model as soon as an event is published it
gets sent to all the subscribers.
! The Publisher doesn’t know who the subscribers are, it
simply knows where to publish.
! Redis has nice support for Pub Sub http://redis.io/topics/
pubsub

CONFIDENTIAL
33


Using Socket.io

! Server Side:
• Subscribe to a Stream
• When there is a new event send it to the client as JSON

! Client Side
• Add support to publish activities to the Stream
• Change the client rendering to use ActivityStrea.ms

! Benefits
• Richer messaging
• Activity Syndication

CONFIDENTIAL
34


Demo
Part 1: App 1 and walkthrough http://blog.cloudfoundry.com/
2012/05/17/node-activity-streams-app-1/
Part 2: App 2 and walkthrough http://blog.cloudfoundry.com/
2012/06/05/node-activity-streams-app-2/
Part 3: http://asms.cloudfoundry.com



Thank You - Questions ?
Cloud Foundry Activity Streams Libraries are at:
https://github.com/organizations/cloudfoundry-samples

http://cloudfoundry.com
Questions: @cloudfoundry, @ciberch or @andypiper



Resources
Primary Site: cloudfoundry.com
Open Source Site: cloudfoundry.org
Twitter: @cloudfoundry or #cloudfoundry
Blog: blog.cloudfoundry.com
FB: facebook.com/cloudfoundry
Documentation: docs.cloudfoundry.com
Full “bootcamp” presentation: slidesha.re/cf-bootcamp



MongoUK 2012

More Related Content

What's hot

Viewers also liked

Similar to MongoUK 2012

Recently uploaded

MongoUK 2012