The document discusses the cloud architecture of Presence Insights, a service that provides analytics for physical locations. Some key points: - Presence Insights migrated from a traditional on-premises JEE architecture to a cloud-native microservices architecture on Bluemix using 29 microservices and 317 Node.js instances. - The new architecture utilizes various technologies like Node.js, MQLight for messaging, Redis for caching and real-time eventing, and Cloudant for persistence. - Lessons learned include deciding how to break services into actors, testing complex cloud architectures, optimizing for different scaling needs, and choosing the right data store for read/write patterns. - The evolution

1. Closer Look at Cloud Centric
Architectures
Todd Kaplinger
STSM, IBM MobileFirst Cloud Platform Architect

2. Presence Insights

5. • ”Web Analytics” for the physical world
• Available in the Bluemix Catalog as part of the larger
Mobile set of capabilities.
• May 28, 2015 - US Release Date
• June 1, 2015 - London Release Date
• November 11, 2015 - Sydney Release Date
• POCs are in the process of moving to the new cloud
architecture
• Large pipeline of opportunities
• Initial focus is Retail but will extend to multiple industries in
2H2015
• Get Started Today!!!
https://www.ng.bluemix.net/docs/#services/
presenceinsights/index.html
• Traditional JEE
• Delivery model was focused on-
premises deliveries
• WebSphere Application Sever
(Web-Tier)
• DB2 ( Persistence)
• Cognos (Reporting)
• Docs (Knowledge Center)
• Single Tenant
• 29+ Micro Services
• 317+ Node.js Instances
• 3 Bluemix Regions
• SaaS Delivery Model on Bluemix
• Node.js (Web Tier)
• MQLight (Messaging)
• Redis (Caching / Real Time Eventing)
• Cloudant (Persistence)
• Elastic Search (Analytics)
• D3 (Charting)
• Docs (Markdown)
• Multi-tenant
Presence Zones Presence Insights
Presence Insights Cloud Architecture

6. Presence Insights Technical Cloud Architecture

7. Industry Trends

8. Evolution of Disruption
…..CloudMobileSOA RESTWeb

9. Cloud Centric Architecture
• Standards Based Web Protocols
• HTTP
• Provides largest breadth of environments today
• WebSockets | MQTT | Push Notifications
• Next generation protocols
• Standards Based Data Models
• JSON (primary)
• Standards based security models
• OAUTH |SAML | OpenID Connect
• Social Integration
• Facebook | Twitter | Linkedin
• API Economy and Marketplace
• Elastically Scale (Grow|Shrink) based on load

10. • Event Driven Flows
• Fire and Forget
• Loosely coupled interactions
• Messaging Backbone
− Publish and Subscribe
• Data Persistence Tier
− Unstructured Data
• Structured Data
• Microservices Architecture
− Expect failure
− Be resilient
− Limit Dependencies
− Stateless
− Elastic Scale
Cloud Centric Patterns

11. Presence Insights Embracing Tech Trends

12. MQ Light
Technology Adoption

13. Runtime
Packages
express
lodash
bluebird
cuid
request
rbush
cloudant/nano
elasticsearch
ioredis
Development
Packages
mocha
supertest
eslint
chai
grunt
shelljs
• Akka Model
• Event Driven - IoT sensor events (Fire &
Forget)
• Scalable - Autoscale instances to satisfy
demand
• Resilient - Messaging Infrastructure
• Responsive - Farm of Redis caches
• Actor Approach - Each Node app serves
one role
• NPM Adoption
• Best of breed open source packages
• express - REST
• ioredis - caching
• bluebird - promises
• Deployed custom NPM Private Repository
• Shared node modules for our micro-
services
• Extended to development
• Grunt builds with shelljs
• ESLint to enforce coding standards
• Mocha and Chai for testing
• NPM CLI extensions (example npm test)
Node Embracement

14. Presence Insights Lessons Learned

15. Why Private Node Package Manager (NPM)?
• Simplify deployment process
• Private Node modules need to be packaged in app
• Significant increase in build time
(need to download and package each module for each app)
• Larger deployment artifacts causes deployment timeouts
• Not feasible for micro services architectures with many
modules
• CF push resolves node dependencies during upload process
• CF push defaults to public NPM repo
• Bluemix lacking service to host private modules
• Follows industry best practices
• Encourages use of small reusable modules across micro
services
• Provides built in model for versioning of modules
• Built in security model
• Private modules are not intended to be consumed externally

16. Presence Insights Node Evolution
• Actor Approach
• Decision on what qualifies as an actor (micro-service too small or too large)
• Testing Approach
• Cloud Native Applications introduce new complexities
• Multi-tenant architectures stress the system in different ways
• Scale for both large and small tenant configurations in the same system
• Scaling characteristics
• Large number of node apps with limited memory
• Small number of node apps with increased memory
• Large number of small sensor event payloads (limited batching /more
frequent)
• Small number of large sensor event payloads (batching / less frequent)
• Memory Management Lifecycles around Garbage Collection and Compaction
• Identification of bottlenecks
• Micro-services approach introduces many variables in the flow to troubleshoot
• Scaling on “actor” can result in performance degradations in other points in the
architecture
• Instrumentation
• Actionable insights leveraging “runtime agents” to get insights into memory and
CPU consumption
• ELK (Elastic Search / Logstash / Kibana)
• Alerts and Pager Duty

17. Presence Insights Data Evolution
• Persistence Tier
• Architecture led us to a NoSQL architecture
• Choices abound (Cloudant/MongoDB/Cassandra/Spark/Elastic Search)
• Right tool for the job (Battle over importance of read versus write)
• No single answer is correct
• Real Time Eventing
• Cloudant
• Track users (entry/exit/dwell) via Change Feed Listener
• Scalability concerns w/ many Node.js instances long lived connections to
Cloudant
• MQLight
• Track users (entry/exit/dwell) via MQ topics
• MQLight lacked ability to be notified when topic key expired
• Redis
• Track users (entry/exit/dwell) via publish subscribe
• Warlock (node-redis-warlock) required for distributed locking (not inherent)
• Actionable Insights
• Leveraging PI to make business decisions
• Requires ability to dissect data and view from multiple vantage points
• Need highly available and distributed architecture to support large data sets
• Need ability to summarize data with well defined Time to Live (TTLs)

18. Take Aways
• Develop A Point of View on Cloud Centric Architectures -
Guarantee this is a constant moving target and many ways to
achieve success.
Important to choose right tool for the right job!
• Document the Journey of Moving to SaaS - A Journey Takes
Time.
• Adopt A Best Of Breed Technology - Open Source solutions
are great accelerators
• Create a Vibrant Technical Team - Retaining top technical
talent is priority #1.
Be the team that other developers want to join!
• Share Lessons Learned - (Articles, Presentations and
Conferences) - CTO Talks!