Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MongoDB at Visibiz

Why and How We’re Using MongoDB In Our Application



                  Mike Brocious
                ...
Why We’re Here


•   Discuss why we chose MongoDB at Visibiz

•   Show how we’re using it

•   Made mistakes Learned along...
About Us
•   Startup

•   Founded April, 2010

•   8 employees

•   Located just outside of Philadelphia, PA

•   Social C...
Physical
                    Apache
Components

                     Tomcat
                        -
                Grai...
How Did We Get To MongoDB?
Application Requirements


•   Application requirements for extensibility

      •   customer extensible objects

      • ...
Extensible Objects
Extensible Objects

•   We provide core objects

      •   person, company, prospect, relationship, etc.
Extensible Objects

•   We provide core objects

        •   person, company, prospect, relationship, etc.

•   Customer c...
CUSTOMER #1                                CUSTOMER #2
•   Person                                          •   Person
    ...
Customer Definable Objects



•   Give customers ability to define their own objects

      •   collection of attribute name...
Scalability

•   Will eventually have large amount of data

    •   social net works, blogs, articles, etc.

    •   email...
What We Liked About MongoDB
What We Liked About MongoDB


•   Document datastore

      •   easy to understand, visualize objects

      •   cmd shell...
What We Liked About MongoDB


•   Document datastore

      •   easy to understand, visualize objects

      •   cmd shell...
Schema Flexibility Comparison



•   Relational vs. document-based



•   Support extensible ‘person’ object
Relational Database Example

                                                                               Date of
      ...
Document Datastore Example
{
    "_id" : ObjectId("4d87a4d32739a23b3c834b67"),
    "name" : "Mike Brocious",
    "address"...
Document Datastore Example
{
     "_id" : ObjectId("4d87a4d32739a23b3c834b67"),
     "name" : "Mike Brocious",
     "addre...
More MongoDB Goodness
More MongoDB Goodness

•   Scalability

    •   replication - easy to setup

    •   sharding
More MongoDB Goodness

•   Scalability

    •   replication - easy to setup

    •   sharding

•   Uses JSON

    •   “the...
JSON

•   JSON * (MongoDB + Groovy + Grails + JavaScript) == LOVE



•   MongoDB == BSON/JSON

•   Groovy == Map

•   Grai...
JSON-eze

•   Every layer of the application understands common format


                                                 ...
What Else We Liked About MongoDB

•   Active product development

•   Community support

    •   plugins, drivers, viewers...
Be Aware Of...
Be Aware Of...

•   No transactions

    •   mitigation: schema design + atomic document updates



•   No really complex ...
•   OK, so that’s WHY



•   Let’s get into the HOW
Application
    Layers                      Browser



                               Controller
                      App...
Our Primary Collections



•   Things



•   Rels
Things



•   Our main collection: “things”



•   Domain objects (person, company, note, event, etc.)

•   Customer-define...
‘Person’ Thing
{   "_id" : ObjectId("4d10f60f39fe153be3316478"),
    "thingType" : "person",
    "name" : {
         "firs...
‘Company’ Thing
{
    "_id" : ObjectId("4d10ffe939fe153b193b6478"),
    "thingType" : "company",
    "name" : "We Be Coder...
‘Company’ Thing
    {
        "_id" : ObjectId("4d10ffe939fe153b193b6478"),
        "thingType" : "company",
        "name...
Relationships - Act One

•   Connections bet ween things

    •   employment history

    •   co-workers, friends
Relationships - Act One

•   Connections bet ween things

    •   employment history

    •   co-workers, friends




•   ...
Relationships - Act One


A             rel             B
Relationships - Act One


           A                rel              B



•   Single atomic insert (good! no transaction...
Relationships - Act One

Find all things I’m related to
me = ObjectId(“4d10f60e39fe153b20316478”)

// Get all the relation...
Relationships - Act Two

•   Moved relationships to nested document inside thing

    •   natural approach for document-ba...
Relationships - Act Two

•   Queries easy and fast - awesome!
    // Get all the things I’m related to
    relatedThings =...
Relationships - Act Two

•   Queries easy and fast - awesome!
    // Get all the things I’m related to
    relatedThings =...
Relationships - Act Three (final?)
•   Best of both worlds
Relationships - Act Three (final?)
•   Best of both worlds

•   Separate “rels” collection
                                ...
Relationships - Act Three (final?)
•       Best of both worlds

•       Separate “rels” collection
                        ...
Relationships - Act Three (final?)
•       Best of both worlds                                 insert / delete

•       Sep...
Relationships - Act Three (final?)
•       Best of both worlds                                 insert / delete

•       Sep...
Other MongoDB Collections


•   Workflow status log

•   Query log

•   Staging area for imported data

•   Users

•   Dupl...
REST

•   RESTful interface on top of ser vices

•   Expose services for internal and
    external development (API)
REST

    •       RESTful interface on top of ser vices

    •       Expose services for internal and
            external...
person = [
                          Person Schema
    name: "person",
    type: "object",
    extends: "contact",

   pro...
Thing Schemas

•   Separate “schemas” MongoDB collection

•   Every “thing” passes through validation before being stored
...
Why MongoDB Works For Us

•   Schema-free

•   Document-oriented

•   JSON

•   Scalable

•   Active product
             ...
Questions?
Upcoming SlideShare
Loading in …5
×

of

mongoDB at Visibiz Slide 1 mongoDB at Visibiz Slide 2 mongoDB at Visibiz Slide 3 mongoDB at Visibiz Slide 4 mongoDB at Visibiz Slide 5 mongoDB at Visibiz Slide 6 mongoDB at Visibiz Slide 7 mongoDB at Visibiz Slide 8 mongoDB at Visibiz Slide 9 mongoDB at Visibiz Slide 10 mongoDB at Visibiz Slide 11 mongoDB at Visibiz Slide 12 mongoDB at Visibiz Slide 13 mongoDB at Visibiz Slide 14 mongoDB at Visibiz Slide 15 mongoDB at Visibiz Slide 16 mongoDB at Visibiz Slide 17 mongoDB at Visibiz Slide 18 mongoDB at Visibiz Slide 19 mongoDB at Visibiz Slide 20 mongoDB at Visibiz Slide 21 mongoDB at Visibiz Slide 22 mongoDB at Visibiz Slide 23 mongoDB at Visibiz Slide 24 mongoDB at Visibiz Slide 25 mongoDB at Visibiz Slide 26 mongoDB at Visibiz Slide 27 mongoDB at Visibiz Slide 28 mongoDB at Visibiz Slide 29 mongoDB at Visibiz Slide 30 mongoDB at Visibiz Slide 31 mongoDB at Visibiz Slide 32 mongoDB at Visibiz Slide 33 mongoDB at Visibiz Slide 34 mongoDB at Visibiz Slide 35 mongoDB at Visibiz Slide 36 mongoDB at Visibiz Slide 37 mongoDB at Visibiz Slide 38 mongoDB at Visibiz Slide 39 mongoDB at Visibiz Slide 40 mongoDB at Visibiz Slide 41 mongoDB at Visibiz Slide 42 mongoDB at Visibiz Slide 43 mongoDB at Visibiz Slide 44 mongoDB at Visibiz Slide 45 mongoDB at Visibiz Slide 46 mongoDB at Visibiz Slide 47 mongoDB at Visibiz Slide 48 mongoDB at Visibiz Slide 49 mongoDB at Visibiz Slide 50 mongoDB at Visibiz Slide 51 mongoDB at Visibiz Slide 52 mongoDB at Visibiz Slide 53 mongoDB at Visibiz Slide 54
Upcoming SlideShare
Django for IoT: From hackathon to production (DjangoCon US)
Next
Download to read offline and view in fullscreen.

8 Likes

Share

Download to read offline

mongoDB at Visibiz

Download to read offline

Presentation describing why and how mongoDB is used at Visibiz, Inc.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

mongoDB at Visibiz

  1. 1. MongoDB at Visibiz Why and How We’re Using MongoDB In Our Application Mike Brocious Tech Lead Visibiz, Inc.
  2. 2. Why We’re Here • Discuss why we chose MongoDB at Visibiz • Show how we’re using it • Made mistakes Learned along the way • Collections • Schema
  3. 3. About Us • Startup • Founded April, 2010 • 8 employees • Located just outside of Philadelphia, PA • Social CRM • ‘know your net work...sell better’ • Currently in limited beta • Sign up at www.visibiz.com
  4. 4. Physical Apache Components Tomcat - Grails application Elastic MongoDB Search Unix Amazon EC2
  5. 5. How Did We Get To MongoDB?
  6. 6. Application Requirements • Application requirements for extensibility • customer extensible objects • customer definable objects • Scalability
  7. 7. Extensible Objects
  8. 8. Extensible Objects • We provide core objects • person, company, prospect, relationship, etc.
  9. 9. Extensible Objects • We provide core objects • person, company, prospect, relationship, etc. • Customer can add their own attributes • including relationships with other objects • A ‘person’ object for customer #1 may not be defined like a ‘person’ object for customer #2
  10. 10. CUSTOMER #1 CUSTOMER #2 • Person • Person • name • name • address <= Core Attributes => • address • date of birth • date of birth • employment history • gender • list: <= Customer Defined => • hobbies • company Attributes • list: • begin date • name • end date • job title
  11. 11. Customer Definable Objects • Give customers ability to define their own objects • collection of attribute names, types • relationships with other objects
  12. 12. Scalability • Will eventually have large amount of data • social net works, blogs, articles, etc. • email • Scaling should be (relatively) easy
  13. 13. What We Liked About MongoDB
  14. 14. What We Liked About MongoDB • Document datastore • easy to understand, visualize objects • cmd shell, log files, debuggers, viewers
  15. 15. What We Liked About MongoDB • Document datastore • easy to understand, visualize objects • cmd shell, log files, debuggers, viewers • Dynamic schemas (“schema-free”) • fit well with our extensibility requirements
  16. 16. Schema Flexibility Comparison • Relational vs. document-based • Support extensible ‘person’ object
  17. 17. Relational Database Example Date of Person Id Name Address Birth 1 Mike Brocious Malvern, PA 6/10/1984 2 Doug Smith Philadelphia, PA 2/24/1980 Person Id User Defined 1 User Defined 2 User Defined 3 User Defined 4 ... 1 Good Burger 3/20/1998 3/31/2010 Flipper 1 Visibiz 4/1/2010 <null> Tech Lead 2 10gen 7/9/2006 3/18/2009 Engineer Column Name Type User Defined 1 Company String User Defined 2 Begin Date Date User Defined 3 End Date Date User Defined 4 Job Title String
  18. 18. Document Datastore Example { "_id" : ObjectId("4d87a4d32739a23b3c834b67"), "name" : "Mike Brocious", "address" : "Malvern, PA", "dateOfBirth" : "Sun Jun 10 1984 00:00:00 GMT-0400 (EDT)", "employmentHistory" : [ { "company" : "Good Burger", "beginDate" : "Sun Mar 20 1998 00:00:00 GMT-0400 (EDT)", "endDate" : "Thu Mar 31 2010 00:00:00 GMT-0400 (EDT)", "jobTitle" : "Flipper" }, { "company" : "Visibiz", "beginDate" : "Fri Apr 01 2010 00:00:00 GMT-0400 (EDT)", "jobTitle" : "Engineer" } ] }
  19. 19. Document Datastore Example { "_id" : ObjectId("4d87a4d32739a23b3c834b67"), "name" : "Mike Brocious", "address" : "Malvern, PA", "dateOfBirth" : "Sun Jun 10 1984 00:00:00 GMT-0400 (EDT)", "employmentHistory" : [ { "company" : "Good Burger", "beginDate" : "Sun Mar 20 1998 00:00:00 GMT-0400 (EDT)", "endDate" : "Thu Mar 31 2010 00:00:00 GMT-0400 (EDT)", "jobTitle" : "Flipper" }, { "company" : "Visibiz", "beginDate" : "Fri Apr 01 2010 00:00:00 GMT-0400 (EDT)", "jobTitle" : "Engineer" } ] } { "_id": ObjectId("4d87a9732739a23b3c834b6d"), "name": "Julie Harper", "address": "Ocean City, NJ", "dateOfBirth": "Thu Sep 22 1994 00:00:00 GMT-0400 (EDT)", "gender": "F", "hobbies": [ "painting", "skateboarding", "reading", "cooking" ] }
  20. 20. More MongoDB Goodness
  21. 21. More MongoDB Goodness • Scalability • replication - easy to setup • sharding
  22. 22. More MongoDB Goodness • Scalability • replication - easy to setup • sharding • Uses JSON • “the new XML” • easy to build, parse and read • great support from tools, languages and ser vices
  23. 23. JSON • JSON * (MongoDB + Groovy + Grails + JavaScript) == LOVE • MongoDB == BSON/JSON • Groovy == Map • Grails == JSON converter/builder • JavaScript = Well...it’s the JS in JSON
  24. 24. JSON-eze • Every layer of the application understands common format Presentation • Not forced into transforming data View • DB result sets <--> DO/DTO J S Controller • DO/DTO <--> view O Services • view <--> presentation N Data Access • Enables rapid development DB
  25. 25. What Else We Liked About MongoDB • Active product development • Community support • plugins, drivers, viewers • 10gen support • developers very active on forum, JIRA • MongoDB conferences, webcasts • Deep list of sites already using MongoDB
  26. 26. Be Aware Of...
  27. 27. Be Aware Of... • No transactions • mitigation: schema design + atomic document updates • No really complex queries (JOINs, nested SELECTs) • mitigation: schema design • Felt we could minimize the impact
  28. 28. • OK, so that’s WHY • Let’s get into the HOW
  29. 29. Application Layers Browser Controller Application Svcs (bus. logic) Home-grown Light weight object- convenience wrapper MongoService Morphia mapping library around Java driver MongoDB Java driver MongoDB
  30. 30. Our Primary Collections • Things • Rels
  31. 31. Things • Our main collection: “things” • Domain objects (person, company, note, event, etc.) • Customer-defined objects • Takes advantage of ‘s chema-free’ nature of MongoDB • able to easily query across all types
  32. 32. ‘Person’ Thing { "_id" : ObjectId("4d10f60f39fe153be3316478"), "thingType" : "person", "name" : { "firstName" : "Gail", "lastName" : "Staudt" }, "owner" : ObjectId("4d10f47e39fe153b552e6478"), "createdDate" : "Tue Dec 21 2010 13:46:39 GMT-0500 (EST)", "tags" : [ { "tag" : "java", "score" : 2 }, { "tag" : "clojure", "score" : 2 } ], "addresses" : [ { "addr1" : "40 Lloyd Ave", "city" : "Malvern", "state" : "PA" } ], "emailAddresses" : [ { "type" : "work", "value" : "staudt@foo.com" } ] }
  33. 33. ‘Company’ Thing { "_id" : ObjectId("4d10ffe939fe153b193b6478"), "thingType" : "company", "name" : "We Be Coders, Inc." "owner" : ObjectId("4d10f60e39fe153b20316478"), "createdDate" : "Tue Dec 21 2010 14:28:41 GMT-0500 (EST)", "primaryBusiness" : "Software development consulting" "tags" : [ { "tag" : "software", "score" : 2 }, { "tag" : "development", "score" : 7 }, { "tag" : "clojure", "score" : 3 }, { "tag" : "java", "score" : 5 } ] }
  34. 34. ‘Company’ Thing { "_id" : ObjectId("4d10ffe939fe153b193b6478"), "thingType" : "company", "name" : "We Be Coders, Inc." "owner" : ObjectId("4d10f60e39fe153b20316478"), "createdDate" : "Tue Dec 21 2010 14:28:41 GMT-0500 (EST)", "primaryBusiness" : "Software development consulting" "tags" : [ { "tag" : "software", "score" : 2 }, { "tag" : "development", "score" : 7 }, { "tag" : "clojure", "score" : 3 }, { "tag" : "java", "score" : 5 } ] } • Find all things I own tagged with ‘java’ > db.things.find({owner: ObjectId(“4d10f60e39fe153b20316478”), tags.tag: “java”})
  35. 35. Relationships - Act One • Connections bet ween things • employment history • co-workers, friends
  36. 36. Relationships - Act One • Connections bet ween things • employment history • co-workers, friends • First cut: separate documents in the things collection • essentially a mapping/join table
  37. 37. Relationships - Act One A rel B
  38. 38. Relationships - Act One A rel B • Single atomic insert (good! no transaction concerns) • Made retrieval inefficient (no JOINs, nested SELECTs)
  39. 39. Relationships - Act One Find all things I’m related to me = ObjectId(“4d10f60e39fe153b20316478”) // Get all the relationships I’m involved in related = db.things.find({$or: [left.id: me, right.id:me]}) // Build up a list of ids I’m related to i = 0 relatedId = new Array() for(relationship in related) if (relationship.left.id == me) { relatedIds[i++] = relationship.right.id } else { relatedIds[i++] = relationship.left.id } ) // Now get the things relatedThings = db.things.find({_id: { $in : relatedIds } })
  40. 40. Relationships - Act Two • Moved relationships to nested document inside thing • natural approach for document-based datastores A B
  41. 41. Relationships - Act Two • Queries easy and fast - awesome! // Get all the things I’m related to relatedThings = db.things.find({$or: [rels.left.id: me, rels.right.id:me]})
  42. 42. Relationships - Act Two • Queries easy and fast - awesome! // Get all the things I’m related to relatedThings = db.things.find({$or: [rels.left.id: me, rels.right.id:me]}) • Two updates required to insert new relationship • relationship stored in both things • bad! - transaction concerns
  43. 43. Relationships - Act Three (final?) • Best of both worlds
  44. 44. Relationships - Act Three (final?) • Best of both worlds • Separate “rels” collection A rel B • master source of relationship details • atomic insert (good!) • unique index (good!)
  45. 45. Relationships - Act Three (final?) • Best of both worlds • Separate “rels” collection A rel B • master source of relationship details • atomic insert (good!) • unique index (good!) • Nested “rels” documents in things A B • easy, fast queries (good!)
  46. 46. Relationships - Act Three (final?) • Best of both worlds insert / delete • Separate “rels” collection A rel B • master source of relationship details • atomic insert (good!) • unique index (good!) • Nested “rels” documents in things A B • easy, fast queries (good!)
  47. 47. Relationships - Act Three (final?) • Best of both worlds insert / delete • Separate “rels” collection A rel B • master source of relationship details • atomic insert (good!) • unique index (good!) • Nested “rels” documents in things A B • easy, fast queries (good!)
  48. 48. Other MongoDB Collections • Workflow status log • Query log • Staging area for imported data • Users • Duplicates scan log
  49. 49. REST • RESTful interface on top of ser vices • Expose services for internal and external development (API)
  50. 50. REST • RESTful interface on top of ser vices • Expose services for internal and external development (API) • MongoDB doesn’t enforce datatypes or object shape • Need a way to validate data to prevent “garbage in” • JSON schema • http://json-schema.org/
  51. 51. person = [ Person Schema name: "person", type: "object", extends: "contact", properties: [ name: [type: "object", title: "Name", properties: [ firstName: [type: "string", title: "First Name", optional:true], middleName: [type: "string", title: "Middle Name", optional: true], lastName: [type: "string", title: "Last Name", optional: true], suffix: [type: "string", title: "Suffix", optional: true]]]]]
  52. 52. Thing Schemas • Separate “schemas” MongoDB collection • Every “thing” passes through validation before being stored • Uses: • validate incoming data • track customer-specific schema extensions • generate UI to display things
  53. 53. Why MongoDB Works For Us • Schema-free • Document-oriented • JSON • Scalable • Active product “MongoDB...The Database Chuck Norris Uses” • Free
  54. 54. Questions?
  • bunkertor

    Mar. 15, 2015
  • KevinQiangK

    Sep. 4, 2012
  • delinfo1

    Jan. 25, 2012
  • czhang

    Jul. 6, 2011
  • vedmalex

    Jun. 21, 2011
  • dermisek

    Apr. 23, 2011
  • aalbagarcia

    Mar. 30, 2011
  • ardic

    Mar. 29, 2011

Presentation describing why and how mongoDB is used at Visibiz, Inc.

Views

Total views

3,539

On Slideshare

0

From embeds

0

Number of embeds

1,128

Actions

Downloads

76

Shares

0

Comments

0

Likes

8

×