Scalable Mobile Media Site
using Drupal & MongoDB

Presented By:
Yash Badiani, Big Data Practice Lead, CIGNEX Datamatics
G...
About CIGNEX Datamatics

CIGNEX Datamatics Confidential

www.cignex.com

2
What Does CIGNEX Datamatics Do?
Since 2000, delivering Open Source
solutions for the enterprise through

adoption and inte...
Where We Can Help You
SOLUTIONS
Portals

Liferay, Drupal, JBoss,

User eXperience
Platform

Content
Enterprise Content
Man...
About the presenters
•

Yash Badiani is the Big Data Practice Lead at CIGNEX Datamatics and
focuses on Big Data Technologi...
Agenda
•
•
•
•
•
•
•

The Mobile Media Use Case
Requirements and Challenges
Solution : Mobile Media site using Drupal & Mo...
The Mobile Explosion!
By 2017, 82% of all phones purchased will be
smartphones.
By 2015, at least 60% of information worke...
Mobile Media Use Case
• Mobile Media site includes the following features:
–
–
–
–
–
–
–
–

Store a variety of Images & as...
Requirements of Mobile Media sites
User
experience
•
•
•
•

Rich UI features
Social features
Mobile access
Fast search

Ve...
Standard Three Layered Data Architecture
Application
layer
File System

Metadata in RDBMS

Search

Standard Three Layered
...
Introduction to NoSQL

Not
Only SQL
• They are schema less
• Designed to support huge data volumes
– Facebook 135 billion ...
NoSQL – Data Models
Column Families
Usage: Read/Write Intensive
Popular databases: Hbase,
Cassandra

Document Store
Usage:...
Requirements of Mobile Media sites - Recap
User
experience
•
•
•
•

Rich UI features
Social features
Mobile access
Fast se...
Drupal with MongoDB Solution

Themes
Core
Modules

Nodes

Tagging

Taxonomy

Workflow

Forums

User
Roles

Forms &
Menu

C...
Why Drupal?
HTML5 & CSS
Support

Pluggable
Architecture

User Management
& Permissions

Data Abstraction
Layer

3rd Party ...
Websites using Drupal

Website: Whitehouse.gov

Website: mtv.co.uk

CIGNEX Datamatics Confidential

Website: Data.gov.uk

...
Why MongoDB?

Agile and
Scalable

High Availability

Full Index
Support

Data Processing
and aggregation

Open Source
NoSQ...
Customers using MongoDB
• Centralized data management platform

• 2 billion+ documents
• 20 TB of photo metadata
• TV epis...
Demo
•
•
•
•
•

Media site on mobile simulator
Like & comment on an image on mobile simulator
Mobile site on web browser
V...
Solution Features

Architecture and Design

CIGNEX Datamatics Confidential

www.cignex.com

20
Architecture
Drupal API
Form API

Browser
Custom
/Mobile
Module
Theme
Image
GridFS
Metadata

User

Menu API

MongoDB PHP
D...
Flow
MongoDB Collections
User Metadata

Albums

GridFS

Image Metadata

Add Album
DBRef

Add Image
DBRef

User Actions

Vi...
Schema Design
User Metadata
•
•
•
•
•
•
•
•

User ID
DBRef (Album)
Tags
Thumbnail
Likes
View Counter
Comments
Permission

...
Schema Design
Image Metadata

Albums

User Metadata

CIGNEX Datamatics Confidential

www.cignex.com

24
MongoDB Monitoring Service (MMS)
• DB Storage
• Cursors
• Replica Sets
• Network
Connections
• Non Mapped
Virtual Memory
•...
Benefits
Drupal

MongoDB

Most advanced content management
solutions

Scalability – billions of content items,
millions of...
Summary and Key Takeaways
• MongoDB provides the RIGHT fit for CMS applications with
flexibility, scale & speed
• Drupal’s...
Thank You. Any Questions ?

Making Open Source WorkTM
To know how Open Source can be adopted and integrated into your ente...
Upcoming SlideShare
Loading in …5
×

Drupal MongoDB Integration benefits Mobile Media Site with Speed and Scalability

2,112 views

Published on

How to integrate Drupal with MongoDB’s NoSQL Database to successfully store and search millions of images and their associated metadata in a centralized and massively scalable repository.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,112
On SlideShare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Drupal MongoDB Integration benefits Mobile Media Site with Speed and Scalability

  1. 1. Scalable Mobile Media Site using Drupal & MongoDB Presented By: Yash Badiani, Big Data Practice Lead, CIGNEX Datamatics Gaurav Khambhala, Technical Lead, CIGNEX Datamatics 30th July, 2013 CIGNEX Datamatics Confidential www.cignex.com
  2. 2. About CIGNEX Datamatics CIGNEX Datamatics Confidential www.cignex.com 2
  3. 3. What Does CIGNEX Datamatics Do? Since 2000, delivering Open Source solutions for the enterprise through adoption and integration to: Portal Solutions Content Solutions • Address business goals • Increase business velocity • Lower the cost of doing business • Reduce TCO Big Data Analytics Solutions • Gain competitive advantage 400+ 450+ 200+ 12+ 5000+ Open Source Solutions Open Source Experts Open Source Integrations Books on Open Source Community Contributions Offices : America | India | UK | Europe | Singapore | Australia CIGNEX Datamatics Confidential www.cignex.com 3
  4. 4. Where We Can Help You SOLUTIONS Portals Liferay, Drupal, JBoss, User eXperience Platform Content Enterprise Content Management ZK, HTML5, Making Data Work Intranet Extranet EAI SOA • Social Collaboration • Big Data Portal • Mobile Portal • • • • • WCM DM RM CMS DAM • • • • MuleSoft Alfresco, Adobe CQ, Drupal, Magento, JBoss, Moodle, Ephesoft, Liferay Big Data Analytics • • • • Hadoop Ecosystem, MongoDB, Ne04j, Pentaho, Talend, Solr, Jaspersoft • Data Integration • Information Delivery • Data Analysis e-Commerce e-Learning ERP Imaging Solutions • Enterprise Search SERVICES UI • Development • Integration • Customization • Migration • Testing • Training • Support (24*7) Enterprise Mobility  Strategy • Mobile UX • App Development • MEAP/MDM Managed Cloud Services  Develop • Deploy • Manage VAR/Annual Product Subscription  Liferay • Alfresco • Cloudera Hadoop • MongoDB Extended Development Center  Center of Excellence CIGNEX Datamatics Confidential www.cignex.com 4
  5. 5. About the presenters • Yash Badiani is the Big Data Practice Lead at CIGNEX Datamatics and focuses on Big Data Technologies including MongoDB & Hadoop. He has worked extensively on large Data warehousing & Business Intelligence projects with tools such as Business Objects, Microsoft SQL Server, Microstrategy, IBM Cognos. • Gaurav Khambhala works at CIGNEX Datamatics as Technical Lead. He is the senior member of the PHP Practice at CIGNEX Datamatics and is involved on various technology initiatives like Big Data where he focuses on integration of PHP with NoSQL sources like MongoDB. He has a wide industry experience in software development & management in Open Source technologies such as Drupal, Moodle & Wordpress. CIGNEX Datamatics Confidential www.cignex.com 5
  6. 6. Agenda • • • • • • • The Mobile Media Use Case Requirements and Challenges Solution : Mobile Media site using Drupal & MongoDB Why Drupal and MongoDB? Demo and Solution Features Benefits Summary CIGNEX Datamatics Confidential www.cignex.com 6
  7. 7. The Mobile Explosion! By 2017, 82% of all phones purchased will be smartphones. By 2015, at least 60% of information workers will interact with their content applications via a mobile device People use digital assets (videos, images) longer on Tablets and Mobiles compared to desktops Based on a report by a leading IT advisory firm CIGNEX Datamatics Confidential www.cignex.com 7
  8. 8. Mobile Media Use Case • Mobile Media site includes the following features: – – – – – – – – Store a variety of Images & associated metadata Massively Scalable to store billions of images Access through Mobile Create /Edit Albums Add Images to the Albums Add / Edit Metadata of Images Search Images / Albums by date, metadata, albums, etc Social Media features – Likes, comments CIGNEX Datamatics Confidential www.cignex.com 8
  9. 9. Requirements of Mobile Media sites User experience • • • • Rich UI features Social features Mobile access Fast search Velocity • • • • Flexibility & Agility Fast performance Large user base Concurrent CRUD Access through various channels CIGNEX Datamatics Confidential • • • • • • • • • Easy integration Shorter dev cycle Faster deployment Ease of schema design Volume • Security & Availability HIGH availability Automatic failover User management Scalability Millions of digital assets Variety of content Complexity of data www.cignex.com • • • • Elastic scaling Cost effectiveness Centralized storage Ease of Maintenance 9
  10. 10. Standard Three Layered Data Architecture Application layer File System Metadata in RDBMS Search Standard Three Layered Storage CIGNEX Datamatics Confidential www.cignex.com 10
  11. 11. Introduction to NoSQL Not Only SQL • They are schema less • Designed to support huge data volumes – Facebook 135 billion messages/month; Twitter 7TB data/day • Scalable replication and distribution mechanism – Thousands of machines distributed around the world • • • • Massive write performance with asynchronous inserts and updates Designed to give high query performance Runs on commodity hardware Most NoSQL databases are Open Source CIGNEX Datamatics Confidential www.cignex.com 11
  12. 12. NoSQL – Data Models Column Families Usage: Read/Write Intensive Popular databases: Hbase, Cassandra Document Store Usage: Working with Occasionally changing/consistent data Popular databases: CouchDB, MongoDB NoSQL Databases Graph Database Usage: Spatial Data storage, Social Application Popular databases: Neo4j Key Value / Tulip Store Usage: Briskly changing data and high availability Popular databases: Riak, Redis, Azure Table storage • 4 broad data models • 120+ variants available in the market CIGNEX Datamatics Confidential www.cignex.com 12
  13. 13. Requirements of Mobile Media sites - Recap User experience • • • • Rich UI features Social features Mobile access Fast search Velocity • • • • Fast performance Large user base Concurrent CRUD Access through various channels CIGNEX Datamatics Confidential Flexibility & Agility • • • • Security & Availability Easy integration Shorter dev cycle Faster deployment Ease of schema design Volume • • • • • • HIGH availability Automatic failover User management Scalability Millions of digital assets Variety of content Complexity of data • • • • www.cignex.com Elastic scaling Cost effectiveness Centralized storage Ease of maintenance Mobile Media Site 13
  14. 14. Drupal with MongoDB Solution Themes Core Modules Nodes Tagging Taxonomy Workflow Forums User Roles Forms & Menu Comments & Ratings Web Services Custom Modules 3rd party & Internal Applications MongoDB Driver PHP Mongos Routing Process MongoDB MongoDB MongoDB Replica Set Replica Set CIGNEX Datamatics Confidential MongoDB www.cignex.com 14
  15. 15. Why Drupal? HTML5 & CSS Support Pluggable Architecture User Management & Permissions Data Abstraction Layer 3rd Party Tools Integration Easy to Upgrade Drupal Open Source CMS Scalable Secure Widely Adopted CIGNEX Datamatics Confidential Active Community www.cignex.com 15
  16. 16. Websites using Drupal Website: Whitehouse.gov Website: mtv.co.uk CIGNEX Datamatics Confidential Website: Data.gov.uk Website: pdx.edu www.cignex.com Website: research.yahoo.com Website: EndPoverty2015.org 16
  17. 17. Why MongoDB? Agile and Scalable High Availability Full Index Support Data Processing and aggregation Open Source NoSQL Solution Document Oriented Storage Atomic Updates Replication Querying CIGNEX Datamatics Confidential www.cignex.com 17
  18. 18. Customers using MongoDB • Centralized data management platform • 2 billion+ documents • 20 TB of photo metadata • TV episodes and series • Risk solutions auditing data Source: http://www.10gen.com/customers CIGNEX Datamatics Confidential www.cignex.com 18
  19. 19. Demo • • • • • Media site on mobile simulator Like & comment on an image on mobile simulator Mobile site on web browser Verify ‘Like’ & comment of the same image on web browser Search images & access control CIGNEX Datamatics Confidential www.cignex.com 19
  20. 20. Solution Features Architecture and Design CIGNEX Datamatics Confidential www.cignex.com 20
  21. 21. Architecture Drupal API Form API Browser Custom /Mobile Module Theme Image GridFS Metadata User Menu API MongoDB PHP Driver Mobile Device User Metadata Indexes CIGNEX Datamatics Confidential www.cignex.com Albums Image Metadata GridFS 21
  22. 22. Flow MongoDB Collections User Metadata Albums GridFS Image Metadata Add Album DBRef Add Image DBRef User Actions View Album DBRef DBRef View Individual Images Like Image Comment Image Add Tags to Images View Counter Search Images By Tags CIGNEX Datamatics Confidential www.cignex.com 22
  23. 23. Schema Design User Metadata • • • • • • • • User ID DBRef (Album) Tags Thumbnail Likes View Counter Comments Permission • • • GridFS User ID Tags Title • • • • • • Albums CIGNEX Datamatics Confidential FS.Files FS.Chunks • • Make Model Date Time Aperture Exposure DBRef (GridFS) Image Metadata www.cignex.com 23
  24. 24. Schema Design Image Metadata Albums User Metadata CIGNEX Datamatics Confidential www.cignex.com 24
  25. 25. MongoDB Monitoring Service (MMS) • DB Storage • Cursors • Replica Sets • Network Connections • Non Mapped Virtual Memory • Opcounters CIGNEX Datamatics Confidential www.cignex.com 25
  26. 26. Benefits Drupal MongoDB Most advanced content management solutions Scalability – billions of content items, millions of users Highly customized websites Performance – FAST writes through sharding, reads through indexes Most search friendly CMS Safety - Replication Less coding, high on automation Centralized Data Storage - Single System Powered by 7000+ plugins and extensions Monitoring & Administration - MMS Active community, real time assistance Enterprise Support - 10gen CIGNEX Datamatics Confidential www.cignex.com 26
  27. 27. Summary and Key Takeaways • MongoDB provides the RIGHT fit for CMS applications with flexibility, scale & speed • Drupal’s advanced & automated CMS features and tight integration with MongoDB makes it the right choice for building agile websites • Both Drupal & MongoDB are feature rich and being Open Source, provide significant cost benefits CIGNEX Datamatics Confidential www.cignex.com 27
  28. 28. Thank You. Any Questions ? Making Open Source WorkTM To know how Open Source can be adopted and integrated into your enterprise Sign Up for Free Assessment Workshop http://www.cignex.com/freeassessment Yash Badiani Big Data Practice Lead yash.badiani@cignex.com Gaurav Khambhala Technical Lead gaurav.khambhala@cignex.com info@cignex.com CIGNEX Datamatics Confidential www.cignex.com

×