Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Webinar: An Enterprise Architect’s View of MongoDB


Published on

In the world of big data, legacy modernization, siloed organizations, empowered customers, and mobile devices, making informed choices about your enterprise infrastructure has become more important than ever. The alternatives are abundant, and the successful Enterprise Architect must constantly discern which new technology is just a shiny object and which will add true business value.

MongoDB is more than just a great application database for developers; it gives Enterprise Architects new capabilities to solve previously difficult architectural requirements much more easily. Take for example the challenge of many siloed systems at MetLife – with MongoDB, the Metlife team was able to successfully provide a single view into those 70 systems, in only 3 months.

In this webinar, we will:

Explore real life challenges enterprises face with case studies of their solutions
Consider how best to introduce MongoDB in the enterprise
Give an overview of how to optimize the use of MongoDB

Published in: Technology
  • Be the first to comment

Webinar: An Enterprise Architect’s View of MongoDB

  1. 1. An Enterprise Architect’s View of MongoDB Matt Kalan Business Architect @matthewkalan
  2. 2. Agenda • Modern drivers of change on enterprises • Requirements these create • How traditional databases are handling changes • New capabilities needed • How MongoDB provides these capabilities • Case studies • Enterprise adoption 2
  3. 3. Modern Requirements
  4. 4. More Technologies and Requirements Than Ever Opportunity cost NoSQL Analytics Globalization JSON Big Data Datawarehouse Customer 360 Document Data Stores Key-value Hadoop ODS MongoDB Graph Wide-column Cloud Computing Cross-channel New Revenue Streams Faster Competition Emerging markets Agile Development Regulation Internet of Things Gamification More with less Mobile Social networking Empowered customers Consumerization Lowering TCO 4
  5. 5. Questions for Enterprise Architects • What current and future requirements does all this raise? • How to prepare my enterprise to handle these? • Which technologies and products will help me? • How to bring them into my enterprise successfully? • How does old and new technology work together? • What does the future state architecture look like? 5
  6. 6. Modern Application Requirements Data Types & OOP Volume of Data New Architectures • Object-oriented • Petabytes of data • Horizontal scaling • Variably structured • Trillions of records • Unstructured (not tabular) • Millions of queries per second • Commodity servers • Cloud computing Agile Development Single Views • Iterative • Disparate data • Short development cycles • Intraday • Fast time-to-market 6 RDBMS • Cross-channel/silo • Global
  7. 7. Impact of New Requirements Handled with 40-year old Technology • Customfield1…100 or separate tables • Caching & ORMs • Expensive hardware and storage • Schema migration project • One canonical schema • Application-specific partitioning • Use files instead of databases • Schema change takes 6 months 7 Slow time-to-market Agility lost High cost Failed projects Business frustrated
  8. 8. How Do I Prepare My Enterprise?
  9. 9. What Could a Modern Database Do to Make This Easier • Dynamic and variable schemas • Richly-structured data • Much faster performance • Easy horizontal scaling • Low TCO • Plus still maintaining capabilities – Rich querying – Strongly consistently data 10
  10. 10. Documents Support Modern Data Relational Document Data Structure { first_name: „Paul‟, surname: „Miller‟ city: „London‟, location: [45.123,47.232], cars: [ { model: „Bentley‟, year: 1973, value: 100000, picture: <binary>, … }, { model: „Rolls Royce‟, year: 1965, value: 330000, … } } } 11
  11. 11. MongoDB Supports Modern Requirements 1. Dynamic Document Schema Application 2. Native language drivers db.customer.insert({…}) db.customer.find({ name: ”John Smith”}) { name: “John Smith”, date: “2013-08-01”), address: “10 3rd St.”, phone: [ { home: 1234567890}, { mobile: 1234568138} ] } Driver Mongos 3. High availability Shard 2 Shard N Primary Primary Primary Secondary Secondary Secondary - Replica sets Shard 1 Secondary … 5. Horizontal scalability 12 - Sharding Secondary Secondary 4. High performance - Data locality - Rich Indexes - RAM
  12. 12. Global Deployment with Local Read/Writes Primary:LON Secondary:NYC Primary:NYC Secondary:SYD Secondary:LON Secondary:SYD Primary:SYD Secondary:LON Secondary:NYC 13
  13. 13. MongoDB Business Value Faster Time to Market Enabling New Apps 14 Lower TCO Response Time & Scalability
  14. 14. When Does MongoDB Help?
  15. 15. Database Landscape 2010 1990 2000 RDBMS Operational Database NoSQL RDBMS Key-Value/ Wide-column Document DB RDBMS Datawarehousing OLAP/DW Hadoop OLAP/DW 16
  16. 16. MongoDB-Hadoop Connector Operational Database Processing & Storage MongoDB-Hadoop Connector • • • • • 17 Low latency Rich fast querying Request/response Aggregations in database Known data relationships • • • • Longer jobs Batch analytics Highly parallel processing Unknown data relationships
  17. 17. Operational Database Use Cases MongoDB RDBMSs Key/Value or Wide Column Stores 18
  18. 18. MongoDB 5th Most Popular Database 19
  19. 19. Leading Organizations Rely on MongoDB 20
  20. 20. Criteria for benefitting most from MongoDB instead of RDBMS  You want to aggregate data from multiple sources  You want agile development and/or fastest time-to-market  You expect the schema to change often  You have variably or unstructured data (records might have different fields)  Your data is hierarchical (i.e. hard to model in RDBMS), e.g. JSON  You expect the data to grow quickly and want ease of scaling out  You want the best performance possible for real-time read/write  You want the lowest TCO and resources including with replication and caching  Performance of database directly impacts user experience  You want real-time analytics and aggregations  You want location-based querying (distance from locations, within regions, etc.)  You have challenges today with building canonical models, scale, TCO, or agility 21
  21. 21. Case Studies of Architectural Capabilities
  22. 22. Difficult Issues Today 1. Performance and agility issues with RDBMS 2. Building a single view across disparate systems 3. Legacy systems often not real-time enabled 4. Master data can be hard to change and distribute 5. Operational applications are siloed 23
  23. 23. Challenge: Performance and agility issues with RDBMS Code DB Schema Application 24 XML Config Object Relational Mapping Relational Database
  24. 24. Solution: Match Data to Application and Optimize Disk IOPS Code XML Config DB Schema Application Object Relational Mapping Relational Database Code Text Search Rich Queries Application 25 Geospatial Aggregatio n Map Reduce
  25. 25. Case Study Uses MongoDB to power enterprise social networking platform Problem • Complex SQL queries, highly normalized schema not aligned with new data types • Poor performance • Lack of horizontal scalability 26 Why MongoDB Results • Dynamic schemas using JSON • Flexibility to roll out new social features quickly • Ability to handle complex data while maintaining high performance • Sped up reads from 30 seconds to tens of milliseconds • Social network analytics with lightweight MapReduce • Dramatically increased write performance
  26. 26. Challenge: Building a single view across disparate systems Batch Datamar t Batch Datamar t Customer Accounts Loans Loans Silo 2 Loans Web … Deposits Deposits Silo 3 Cards Mobile 27 Batch Datamar t Batch Data Warehouse Reporting Cards Cards Silo 1 Banking Store Issues • Yesterday’s data • Details lost • Inflexible schema • Slow performance Impact • What happened today? • Worse customer satisfaction • Missed opportunities • Lost revenue
  27. 27. Solution: Using dynamic schema and easy scaling Operational Data Hub Customer Accounts CSR Application Real-time or Batch Customer Portal Loans Loans Silo 2 … … Deposits Deposits Silo 3 28 Operational Reporting Data Warehouse Strategic Reporting Cards Cards Silo 1 Benefits • Real-time • Complete details • Agile • Higher customer retention • Increase wallet share • Proactive exception handling
  28. 28. Case Study Insurance leader generates coveted 360-degree view of customers in 90 days – “The Wall” Problem • No single view of customer • 145 yrs of policy data, 70+ systems, 15+ apps Why MongoDB • Agility – prototype in 5 days; production in 90 days • 2 years, $25M in failing to aggregate in RDBMS • Dynamic schema & rich querying – combine disparate data into one data store • Poor customer experience • Hot tech to attract top talent 29 Results • Unified customer view available to all channels • Increased call center productivity • Better customer experience, reduced churn, more upsell opps • Dozens more projects on same data platform
  29. 29. Challenge: Legacy systems often not real-time enabled or too slow Data source 1 Batch copy Application 1 Often not ready to expose as enterprise services • Mainframe • Core systems • Data Warehouses • Not scalable system Application 2 Data source 2 … Slow request/response 30 Application 3 … Data source N Batch copying of data many times or requests are too slow Application X Changing source data affects X systems Impact • Slow time to market • Resource intensive • Hard to change interfaces and modernize system
  30. 30. Solution: Virtualize legacy systems with a persistent caching service Mainfram e Batch Batch copy API Batch copy Application 1 Application 2 EDW … … Pub/sub … Core system Application 3 Application X 31 Benefits • Faster time to market • More agile in changing sources • Can modernize data sources behind virtualization • Infinite scale with low TCO
  31. 31. Case Study: Global Custodial Bank Virtualize Enterprise Data Sources Create a central data hub for accessing data across the enterprise Problem • Found numerous pointto-point copies of data • Change in one system impacts multiple groups • Response time on EDW was too slow • Wanted one central data hub for most often accessed data 32 Why MongoDB Results • Dynamic schema: can • Data accessible by batch normalize data as needed or REST layer in one place and prioritized • Customer portal response • Performance: can handle times shrunk by 90% all data in one logical DB • Shorter development times • Sharding: can add data with more accessible hub easily by scaling out • Could modernize data sources without changing apps
  32. 32. Challenge: Master data can be hard to change and distribute Batch Batch Batch Golden Copy Common issues • Hard to change schema of master data • Data copied everywhere and gets out of sync Batch Batch Batch Batch Batch Impact • Process breaks from out of sync data • Business doesn’t have data it needs • Many copies creates 33 more management
  33. 33. Solution: Persistent dynamic cache replicated globally Real-time Real-time Real-time Real-time Real-time Solution: • Load into primary with any schema • Replicate to and read from secondaries Real-time Real-time Real-time Benefits • Easy & fast change at speed of business • Easy scale out for one stop shop for data • Low TCO 34
  34. 34. Case Study: Global bank Reference Data Distribution Distribute reference data globally in real-time for fast local accessing and querying Problem • Delays up to 36 hours in distributing data by batch • Charged multiple times globally for same data • Incurring regulatory penalties from missing SLAs • Had to manage 20 distributed systems with same data 35 Why MongoDB Results • Dynamic schema: easy to • Will save about load initially & over time $40,000,000 in costs and penalties over 5 years • Auto-replication: data distributed in real-time, • Only charged once for data read locally • Data in sync globally and • Both cache and database: read locally cache always up-to-date • Capacity to move to one • Simple data modeling & global shared data service analysis: easy changes and understanding
  35. 35. Reporting Reporting Silo 2 Transactions Silo 2 Systems Silo 3 Transactions 36 … Silo 1 Systems … Silo 1 Transactions Reporting Challenge: Operational applications are siloed Silo 3 Systems Impact • Views are siloed • Duplicate management and data access layer • Need another layer to aggregate
  36. 36. Solution: Unified data services Silo 2 Systems … … … … Common persistence framework Reporting Silo 1 Systems Silo 3 Systems 37 Benefit • Each application can still save its own data • Data is already aggregated for crosssilo reporting • One cluster and data access layer to manage
  37. 37. Case Study: Global Broker Dealer Trade Mart for all OTC Trades Distribute reference data globally in real-time for fast local accessing and querying Problem • Each application had its own persistence and audit trail • Wanted one unified framework and persistence for all trades and products • Needed to handle many variable structures across all securities 38 Why MongoDB Results • Dynamic schema: can • Fast time-to-market using save trade for all products the persistence framework in one data service • Store any structure of • Easy scaling: can easily products/trades without keep trades as long as changing a schema required with high • One consolidated trade performance store for auditing and reporting
  38. 38. Enterprise Adoption
  39. 39. Example Adoption Path Use of MongoDB Widespread Adoption Operationally Supported Certified MongoDB Practice Defined A Few Projects One Project Time 40
  40. 40. Traditional Data Integrity Enforcement Application 1 Application 2 Application 3 41 RDBMS • • • Apps access DB directly Data Integrity must be in the RDBMS Schema implemented by a DBA
  41. 41. Modern Apps (SOA) - Data Access Layer Should Enforce Data Integrity • • Data Integrity and validations done in Data Access Layer Implemented in code MongoDB Cluster Application 1 Application 2 … Application N 42 Data Access Layer API on TCP/IP … REST/API/WS
  42. 42. Data Governance Benefits • Greater adoption from natural developer framework on common data models • Easier for master data or upstream changes to flow into MongoDB-backed apps • MongoDB useful for distributing master data • ETL providers support MongoDB most in NoSQL 43
  43. 43. MongoDB Partners (360+) & Integration Software & Services Cloud & Channel 44 Hardware
  44. 44. Factors to Consider in Adoption • SDLC and data governance for an application • Enterprise-wide data governance (inter-app) • Roles and responsibilities • Training requirements • Operations/production support • Center of Excellence (COE) • Process for choosing which DB to use • How to work with other technologies in-house 45
  45. 45. Recommended Center of Excellence Database Engineering & CoE Database Advisory Services Operational Database CoE RDBMS Engineering Database SMEs MongoDB Engineering MongoDB Incubator (& cluster) RDBMS PaaS Engineering MongoDBaaS Engineering Datawarehousing CoE DW Product Engineering 46 Database SMEs Hadoop Incubator Clusters DW PaaS Engineering Hadoop PaaS Engineering
  46. 46. Summary • Enormous technology and business change today • Old technologies not suited for many of them • MongoDB is purpose built for today and future applications • And can help solve common architectural challenges • Bring MongoDB in to learn how to adopt it more widely when appropriate • Firms using MongoDB benefit from 50% time-to-market, 70% lower TCO, and making the infeasible possible 47
  47. 47. MongoDB Products and Services Subscriptions MongoDB Enterprise, Monitoring, Support, Commercial License Consulting Expert Resources for All Phases of MongoDB Implementations Training Online and In-Person for Developers and Administrators MongoDB Monitoring Service Free, Cloud-Based Service for Monitoring and Alerts MongoDB Backup Service Cloud-based service for backing up and restoring MongoDB 48
  48. 48. For More Information Resource MongoDB Downloads Free Online Training Webinars and Events White Papers Case Studies Presentations Documentation Additional Info 49 Location