Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
lightblue: 
Enterprise Data Services in the 
Cloud with MongoDB 
Naveen Malik 
Principal Software Applications Engineer, R...
Where can I get this presentation? 
Also in footer... 
Naveen Malik http://goo.gl/f7S4PA
Introduction 
● Goals: 
○ Share project lightblue. 
○ Answer your questions. 
○ Listen to your ideas. 
● Please interrupt ...
What is lightblue? Cloud focused data services with dynamic 
querying, versioned schemas, and robust 
security. 
Naveen Ma...
Why Open Source? 
● Code Quality 
○ More eyes, more ideas, better code. 
● Better Architecture 
○ More mindful of separati...
Why was lightblue created? 
● Provide clients standard API for any CRUD operations. 
● Enable deployment anywhere with rob...
How does lightblue benefit developers? 
● Applications have a robust lightblue API 
● Versioned metadata allows changes wi...
How does lightblue benefit operations? 
● Designed to be deployed anywhere 
○ AWS 
○ Rackspace 
○ OpenShift 
○ Embedded as...
How does lightblue benefit SOA teams? 
Naveen Malik http://goo.gl/f7S4PA
What about support and development tools? 
Naveen Malik http://goo.gl/f7S4PA
Why MongoDB? 
● Lightweight and flexible 
● Scale horizontally by adding nodes 
● Replication out of the box 
● Ability to...
How does 
it work? 
● JSON in 
● JSON to BSON 
● BSON to JSON 
● JSON out 
Naveen Malik http://goo.gl/f7S4PA
Metadata 
● Entity Info 
○ not versioned 
○ persistence information 
○ indexes, enumerations 
● Schema 
○ versioned 
○ fie...
User Metadata - Entity Info 
"entityInfo" : { 
"name": "user", 
"enums" : [{ 
"name": "site_type_enum", 
"values": [ "bill...
User Metadata - Schema 
"schema" : { 
"name" : "user", 
"version": { "value": "1.0.0", "changelog": "Test version" }, 
"st...
User Metadata - Fields and Objects 
"fields": { 
"object_type": { 
"type": "string" 
}, 
"login": { 
"type": "string", 
"c...
Should we risk a live demo? 
Setup 
● OpenShift Online - https://www.openshift.com/ 
● Small jbosseap-6 gear with mongodb-...
How powerful is lightblue’s query API? 
● Queries: 
○ “field”=”value”, “field1”=”field2” 
○ $and, $or, $in, $nin 
○ Regula...
How does lightblue keep my data safe? 
● Recommend encryption in transit and at rest 
○ Container can terminate SSL from c...
What about MongoDB security? 
● Restrict access to database! 
○ ACL, Security Group, iptables, firewall, etc. 
● Require S...
Logical Reference Architecture 
Naveen Malik http://goo.gl/f7S4PA
Provisioning 
● CloudFormation - template for each node type 
● Cumulus - orchestration of node creation 
● Puppet - confi...
Monitoring MongoDB 
● Mongo Management Service 
○ https://mms.mongodb.com/ 
○ Agent connects to mongo database 
○ Can be s...
Lessons Learned 
Nothing is perfect… 
● JSON 
○ Date isn’t done well in JSON, we decided on a standard. 
○ BigDecimal and ...
What’s next for lightblue? 
● Associations (joins) between entities 
○ 1-1, 1-n, n-n associations defined in metadata 
○ D...
Can I move existing data to lightblue? 
YES! 
Presentation giving migration overview: 
● www.slideshare.net/derek63/lightb...
Want more information? 
● Source 
○ https://github.com/lightblue-platform/ 
● Documentation 
○ http://lightblue.io/ 
● IRC...
What questions can I answer? 
● Thoughts you would like to share? 
Naveen Malik http://goo.gl/f7S4PA
Upcoming SlideShare
Loading in …5
×

lightblue: Enterprise Data Services in the Cloud with MongoDB

Overview of project lightblue, an open source project for enabling deployment of enterprise data services (SOA) to the cloud. Lightblue has a pluggable software architecture and supports MongoDB persistence at this time with more support coming soon (such as RDBMS).

lightblue on github: https://github.com/lightblue-platform/

Documentation:
General - http://lightblue.io/
User Guide - http://docs.lightblue.io/
Developer Manual - http://dev.docs.lightblue.io/

  • Login to see the comments

lightblue: Enterprise Data Services in the Cloud with MongoDB

  1. 1. lightblue: Enterprise Data Services in the Cloud with MongoDB Naveen Malik Principal Software Applications Engineer, Red Hat
  2. 2. Where can I get this presentation? Also in footer... Naveen Malik http://goo.gl/f7S4PA
  3. 3. Introduction ● Goals: ○ Share project lightblue. ○ Answer your questions. ○ Listen to your ideas. ● Please interrupt for questions, comments, ideas, etc. Naveen Malik http://goo.gl/f7S4PA
  4. 4. What is lightblue? Cloud focused data services with dynamic querying, versioned schemas, and robust security. Naveen Malik http://goo.gl/f7S4PA
  5. 5. Why Open Source? ● Code Quality ○ More eyes, more ideas, better code. ● Better Architecture ○ More mindful of separation of concerns ● Giving back! ○ Not a unique problem, others can benefit and contribute. ○ Contributors bring ideas, testers, developers, and more! Naveen Malik http://goo.gl/f7S4PA
  6. 6. Why was lightblue created? ● Provide clients standard API for any CRUD operations. ● Enable deployment anywhere with robust security. ● Rapid response to changing business needs. ● Stability for existing consumers. Naveen Malik http://goo.gl/f7S4PA
  7. 7. How does lightblue benefit developers? ● Applications have a robust lightblue API ● Versioned metadata allows changes without breaking existing applications ○ clients see their own version of data ○ different clients operate on same object with different versions ○ backwards compatibility of minor changes guaranteed Naveen Malik http://goo.gl/f7S4PA
  8. 8. How does lightblue benefit operations? ● Designed to be deployed anywhere ○ AWS ○ Rackspace ○ OpenShift ○ Embedded as a library in other applications ● Flexible component architecture ○ Deploy and scale components independently ● Latency and fault tolerance ● Robust security Naveen Malik http://goo.gl/f7S4PA
  9. 9. How does lightblue benefit SOA teams? Naveen Malik http://goo.gl/f7S4PA
  10. 10. What about support and development tools? Naveen Malik http://goo.gl/f7S4PA
  11. 11. Why MongoDB? ● Lightweight and flexible ● Scale horizontally by adding nodes ● Replication out of the box ● Ability to shard (partition) out of the box First controller implementation! RDBMS late 2014. Naveen Malik http://goo.gl/f7S4PA
  12. 12. How does it work? ● JSON in ● JSON to BSON ● BSON to JSON ● JSON out Naveen Malik http://goo.gl/f7S4PA
  13. 13. Metadata ● Entity Info ○ not versioned ○ persistence information ○ indexes, enumerations ● Schema ○ versioned ○ fields ○ constraints ○ access rights Naveen Malik http://goo.gl/f7S4PA
  14. 14. User Metadata - Entity Info "entityInfo" : { "name": "user", "enums" : [{ "name": "site_type_enum", "values": [ "billing", "marketing", "service", "shipping" ] }], "datastore": { "backend":"mongo", "datasource": "mongo", "collection": "user" } } } Naveen Malik http://goo.gl/f7S4PA
  15. 15. User Metadata - Schema "schema" : { "name" : "user", "version": { "value": "1.0.0", "changelog": "Test version" }, "status": {"value": "active"}, "access" : { "insert": ["anyone"], "find":["anyone"], "update":["anyone"], "delete":["anyone"] }, "fields": {...} } Naveen Malik http://goo.gl/f7S4PA
  16. 16. User Metadata - Fields and Objects "fields": { "object_type": { "type": "string" }, "login": { "type": "string", "constraints": { "maxLength": 64, "minLength": 1, "required": true } }, ... Naveen Malik http://goo.gl/f7S4PA
  17. 17. Should we risk a live demo? Setup ● OpenShift Online - https://www.openshift.com/ ● Small jbosseap-6 gear with mongodb-2.4 cartridge ● Setup following https://github.com/lightblue-platform/openshift-lightblue-all List all entities: ● curl http://services-lightblue.rhcloud.com/rest/metadata/ All versions for ‘country’: ● curl http://services-lightblue.rhcloud.com/rest/metadata/country/ Details of country with iso2Code of ‘US’: ● curl http://services-lightblue.rhcloud.com/rest/data/find/country/1.0.0?Q=iso2Code:US Naveen Malik http://goo.gl/f7S4PA
  18. 18. How powerful is lightblue’s query API? ● Queries: ○ “field”=”value”, “field1”=”field2” ○ $and, $or, $in, $nin ○ Regular expressions ○ Array element searches ● Projections: Ability to get what you want ○ Include some fields ○ Exclude some fields ○ Array ranges ○ Return matching array elements ● Batch: Supported for all operations! Naveen Malik http://goo.gl/f7S4PA
  19. 19. How does lightblue keep my data safe? ● Recommend encryption in transit and at rest ○ Container can terminate SSL from client ○ Support SSL communication with MongoDB ● Secure by design ○ Ability to control access to data at entity and field level ○ Data access controlled down to individual fields ● Authentication & authorization as plugin ○ JBoss EAP 6 with PicketLink and Login Modules ○ Authentication with SAML 2.0 (people) and Certificates (systems) ○ Authorization with LDAP Naveen Malik http://goo.gl/f7S4PA
  20. 20. What about MongoDB security? ● Restrict access to database! ○ ACL, Security Group, iptables, firewall, etc. ● Require SSL! ○ SSL is enabled in community edition 2.6.0+ ○ Require certificate from client. ○ http://docs.mongodb.org/manual/tutorial/configure-ssl/ Naveen Malik http://goo.gl/f7S4PA
  21. 21. Logical Reference Architecture Naveen Malik http://goo.gl/f7S4PA
  22. 22. Provisioning ● CloudFormation - template for each node type ● Cumulus - orchestration of node creation ● Puppet - configuration of nodes after creation ○ Our module is on github! ○ https://github.com/lightblue-platform/lightblue-puppet Naveen Malik http://goo.gl/f7S4PA
  23. 23. Monitoring MongoDB ● Mongo Management Service ○ https://mms.mongodb.com/ ○ Agent connects to mongo database ○ Can be shared across environments ○ Supports secure connections over SSL ● Further reading ○ http://docs.mongodb.org/manual/administration/monitoring/ Naveen Malik http://goo.gl/f7S4PA
  24. 24. Lessons Learned Nothing is perfect… ● JSON ○ Date isn’t done well in JSON, we decided on a standard. ○ BigDecimal and BigInteger support. ○ Solution: solve with standards and in JSON conversion ● MongoDB ○ No transaction support. ○ Ad hoc queries against large data sets ○ Solution: implement other controllers: RDBMS Naveen Malik http://goo.gl/f7S4PA
  25. 25. What’s next for lightblue? ● Associations (joins) between entities ○ 1-1, 1-n, n-n associations defined in metadata ○ Different cascading options for insert/update/delete ○ Ad-hoc associations in queries ● Asynchronous processing ○ Call, receive a handle, don’t wait for response to complete ○ Check handle to get status Naveen Malik http://goo.gl/f7S4PA
  26. 26. Can I move existing data to lightblue? YES! Presentation giving migration overview: ● www.slideshare.net/derek63/lightblue-migration Coming soon: ● Publishing detailed pattern for migration. ● Utilities to make migrations easier. Naveen Malik http://goo.gl/f7S4PA
  27. 27. Want more information? ● Source ○ https://github.com/lightblue-platform/ ● Documentation ○ http://lightblue.io/ ● IRC: Freenode / #lightblue ● Forums: ○ http://forum.lightblue.io/ ○ http://dev.forum.lightblue.io/ Naveen Malik http://goo.gl/f7S4PA
  28. 28. What questions can I answer? ● Thoughts you would like to share? Naveen Malik http://goo.gl/f7S4PA

×