Create a Scalable Mobile Media Site with Drupal and MongoDB
Upcoming SlideShare
Loading in...5
×
 

Create a Scalable Mobile Media Site with Drupal and MongoDB

on

  • 1,244 views

 

Statistics

Views

Total Views
1,244
Views on SlideShare
1,237
Embed Views
7

Actions

Likes
0
Downloads
25
Comments
0

1 Embed 7

http://dev.it.gosbook.ru 7

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Founded in 2000; Offices globallyPure Play Open Source SISolutions co-exist with proprietaryThought leaders: 8 Books on Open Source with 400+ expertsReduces Cost / TCO throughLicense (eliminating vendor lock-in and associated licensing cost)Infrastructure (Open Source has minimal infrastructure requirements compared to Proprietary technologies) AMC (reduction in Annual maintenance cost)Manpower (team size reduction with Open Source adoption)UBMi, Renesas, Sesame, Activision in excess of 10 millions/yearAverage saving $2.5 million x 400Saving of Million average/year – One example - 5 Millions in AMC (Documentum CMS to OS)For example, 200+ implementations and saving of at least $500,000 yields $100000000 (100 million), we have an example of Activision saving over 5 million and multiply with 3 years = 15 million but I am counting minimum of $500,000) 
  • GridFSIndexingShardingDBRefsGeo Spatial Index
  • Pluggable ArchitectureDatabase abstraction layer (support multiple databases)Easy to upgradeSecureHuge Active communityAdoption (2% of all websites are in Drupal CMS)Scalable3rd party tools integrationAdvance User management & permissionsTheme layer with HTML5 and CSS3 support

Create a Scalable Mobile Media Site with Drupal and MongoDB Create a Scalable Mobile Media Site with Drupal and MongoDB Presentation Transcript

  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Mobile Media Site using Drupal & MongoDB Presented By: Yash Badiani, Big Data Practice Lead, CIGNEX Datamatics Gaurav Khambhala, Technical Lead, CIGNEX Datamatics 30th July, 2013
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential About Acquia
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Acquia – Enterprise Guide to Drupal 3 Co-founded by the Drupal project’ creator in 2007, Acquia has helped more than 2,000 clients manage growth and scale of their digital experiences. Acquia provides wide range of Drupal products and services, including: Acquia Cloud Enterprise GardensAcquia Network Drupal Training Professional Services Drupal Commons
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential About CIGNEX Datamatics 4
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential What Does CIGNEX Datamatics Do? 5 Since 2000, CIGNEX Datamatics has implemented over 400 Open Source enterprise solutions addressing business requirements related to Portals, Content & Big Data
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Where We Can Help You 6 SOLUTIONS Managed Cloud Services - Develop, Deploy, Manage Annual Product Subscription: Liferay, Alfresco, Magento, Hadoop, Selenium Extended Development Center – Center of Excellence UI Development Integration Customization Migration Testing Training Support (24*7) User eXperience Platform Portals Liferay, Magento, Drupal, Adobe CQ • Intranet • Extranet • Social Collaboration • Mobile Portals • E-Commerce Enterprise Content Management Content Alfresco, Drupal, Magento, Adobe CQ, Moodle, Ephesoft • WCM • DM • RM • DAM • E-Commerce • E-learning • ERP • Imaging Solutions SERVICES Making Data Work Big Data Hadoop, MongoDB, Hbase, Neo4j, Solr • Analytics • Mobile • Social • Web • Real-time • DW - BI • Log Processing and Analysis • Enterprise Search Velocity Complexity Volume Variety
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Part of Datamatics (DGSL) • Mission – Experts in improving Enterprise productivity through Process Engineering & Information Management Solutions • Key Highlights – Founded in 1975 – Publicly listed in India – Annual consolidated revenue of US$100 Million – Fortune 500 clients – 4,400+ employees across 22 offices in 9 countries Strategic Alliances 7
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential About the presenters • Yash Badiani is the Big Data Practice Lead at CIGNEX Datamatics and focuses on Big Data Technologies including MongoDB & Hadoop. He has worked extensively on large Data warehousing & Business Intelligence projects with tools such as Business Objects, Microsoft SQL Server, Microstrategy, IBM Cognos. • Gaurav Khambhala works at CIGNEX Datamatics as Technical Lead. He is the senior member of the PHP Practice at CIGNEX Datamatics and is involved on various technology initiatives like Big Data where he focuses on integration of PHP with NoSQL sources like MongoDB. He has a wide industry experience in software development & management in Open Source technologies such as Drupal, Moodle & Wordpress. 8
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Agenda • The Mobile Media Use Case • Requirements and Challenges • Solution : Mobile Media site using Drupal & MongoDB • Why Drupal and MongoDB? • Demo and Solution Features • Benefits • Summary 9
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential The Mobile Explosion! 10 By 2015, at least 60% of information workers will interact with their content applications via a mobile device Employees work on proposals and presentation on mobile devices while travelling People use digital assets (videos, images) longer on Tablets and Mobiles compared to desktops Based on a report by a leading IT advisory firm
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Mobile Media Use Case • Mobile Media site includes the following features: – Store a variety of Images & associated metadata – Massively Scalable to store billions of images – Access through Mobile – Create /Edit Albums – Add Images to the Albums – Add / Edit Metadata of Images – Search Images / Albums by date, metadata, albums, etc – Social Media features – Likes, comments 11
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Requirements of Mobile Media sites 12 • Fast performance • Large user base • Concurrent CRUD • Access through various channels • Millions of digital assets • Variety of content • Complexity of data • Rich UI features • Social features • Mobile access • Fast search • Elastic scaling • Cost effectiveness • Centralized storage • Ease of Maintenance • HIGH availability • Automatic failover • User management Velocity Volume User experience Scalability Security & Availability • Easy integration • Shorter dev cycle • Faster deployment • Ease of schema design Flexibility & Agility
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Standard Three Layered Data Architecture 13 File System Metadata in RDBMS Search Standard Three Layered Storage Application layer
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Limitations of RDBMS • Support limited to terabytes – No support for petabytes to exabytes • Manage only structured data – No support for semi-structured and unstructured data • RDBMS don't scale inherently – Scale up/Scale out (Load Balancing & Replication) • Hard to shard / partition – Large data files • Both read / write throughput not possible – Transactional / Analytical databases • Specialized hardware - expensive 14 RDBMS can’t manage all dimensions of data with speed & at lower cost.
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential NoSQL is the right solution • They are schema less • Designed to support huge data volumes – Facebook 135 billion messages/month; Twitter 7TB data/day • Scalable replication and distribution mechanism – Thousands of machines distributed around the world • Massive write performance with asynchronous inserts and updates • Designed to give high query performance • Runs on commodity hardware • Most NoSQL databases are Open Source 15 Not SQLOnly
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential NoSQL – Data Models 16 Column Families Usage: Read/Write Intensive Popular databases: Hbase, Cassandra Document Store Usage: Working with Occasionally changing/consistent data Popular databases: CouchDB, MongoDB Graph Database Usage: Spatial Data storage Popular databases: Neo4j, Bigdata Key Value / Tulip Store Usage: Briskly changing data and high availability Popular databases: Riak, Redis, Azure Table storage NoSQL Databases • 4 broad data models • 120+ variants available in the market
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Requirements of Mobile Media sites - Recap 17 • Fast performance • Large user base • Concurrent CRUD • Access through various channels • Millions of digital assets • Variety of content • Complexity of data • Rich UI features • Social features • Mobile access • Fast search • Elastic scaling • Cost effectiveness • Centralized storage • Ease of maintenance • HIGH availability • Automatic failover • User management Velocity Volume User experience Scalability Security & Availability • Easy integration • Shorter dev cycle • Faster deployment • Ease of schema design Flexibility & Agility Mobile Media Site
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Drupal with MongoDB Solution 18 Themes Core Modules Nodes Taxonomy User Roles Forms & Menu Custom Modules Workflow Forums Comments & Ratings Tagging PHP Web Services 3rd party & Internal Applications MongoDB Driver Mongos Routing Process Replica Set MongoDB MongoDB Replica Set MongoDB MongoDB
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Why Drupal? 19 Pluggable Architecture Data Abstraction Layer Easy to Upgrade Secure Active CommunityWidely Adopted Scalable 3rd Party Tools Integration User Management & Permissions HTML5 & CSS Support Drupal Open Source CMS
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Websites using Drupal 20 Website: Whitehouse.gov Website: Data.gov.uk Website: mtv.co.uk Website: research.yahoo.com Website: pdx.edu Website: EndPoverty2015.org
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Why MongoDB? 21 Agile and Scalable Full Index Support Document Oriented Storage Replication Querying Atomic Updates Data Processing and aggregation High Availability Open Source NoSQL Solution
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Customers using MongoDB • Centralized data management platform • 2 billion+ documents • 20 TB of photo metadata • TV episodes and series • Risk solutions auditing data 22 Source: http://www.10gen.com/customers
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Demo 23 • Media site on mobile simulator • Like & comment on an image on mobile simulator • Mobile site on web browser • Verify ‘Like’ & comment of the same image on web browser • Search images & access control
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Solution Features Architecture and Design 24
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Architecture 25 User Metadata Indexes Albums Image Metadata GridFS Form API Drupal API Custom Module Browser /Mobile Theme MongoDB PHP Driver Menu API User Mobile Device Image Metadata GridFS
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Add Album Flow 26 Add Image View Album View Individual Images Like Image Comment Image Add Tags to Images View Counter Search Images By Tags User Metadata Albums GridFS Image Metadata DBRef DBRef DBRef DBRef MongoDB Collections UserActions
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential 27 Schema Design User Metadata GridFS Albums Image Metadata • User ID • DBRef (Album) • Tags • Thumbnail • Likes • View Counter • Comments • Permission • FS.Files • FS.Chunks • User ID • Tags • Title • Make • Model • Date Time • Aperture • Exposure • DBRef (GridFS)
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Schema Design 28 Image Metadata Albums User Metadata
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential MongoDB Monitoring Service (MMS) 29 • DB Storage • Cursors • Replica Sets • Network Connections • Non Mapped Virtual Memory • Opcounters
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Benefits Drupal MongoDB Most advanced content management solutions Scalability – billions of content items, millions of users Highly customized websites Performance – FAST writes through sharding, reads through indexes Most search friendly CMS Data safety through replication Less coding, high on automation Centralized single system for data storage Powered by 7000+ plugins and extensions Monitoring through MMS Active community, real time assistance Enterprise support through 10gen 30
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Summary & Key Takeaways • MongoDB provides the RIGHT fit for CMS applications with flexibility, scale & speed • Drupal’s advanced & automated CMS features and tight integration with MongoDB makes it the right choice for building agile websites • Both Drupal & MongoDB are feature rich and being Open Source, provide significant cost benefits 31
  • CIGNEX Datamatics Confidential www.cignex.com CIGNEX Datamatics Confidential Thank you. Questions? CIGNEX Datamatics makes Open Source work for you! 32 Yash Badiani Big Data Practice Lead yash.badiani@cignex.com Gaurav Khambhala Technical Lead gaurav.khambhala@cignex.com