An Enterprise Architect’s View of
MongoDB
Matt Kalan
Business Architect
matt.kalan@mongodb.com
@matthewkalan
Agenda
• Modern drivers of change on enterprises
• Requirements these create
• How traditional databases are handling chan...
Modern Requirements
More Technologies and Requirements
Than Ever
Opportunity cost
NoSQL
Analytics
Globalization
JSON
Big Data Datawarehouse
Cu...
Questions for Enterprise Architects
• What current and future requirements does all
this raise?
• How to prepare my enterp...
Modern Application Requirements
Data Types & OOP

Volume of Data

New Architectures

• Object-oriented

• Petabytes of dat...
Impact of New Requirements Handled
with 40-year old Technology
• Customfield1…100 or separate tables
• Caching & ORMs
• Ex...
How Do I Prepare My Enterprise?
What Could a Modern Database Do
to Make This Easier
• Dynamic and variable schemas
• Richly-structured data
• Much faster ...
Documents Support Modern Data
Relational

Document Data Structure
{
first_name: „Paul‟,
surname: „Miller‟
city: „London‟,
...
MongoDB Supports Modern
Requirements
1. Dynamic Document
Schema

Application

2. Native language drivers
db.customer.inser...
Global Deployment with Local
Read/Writes
Primary:LON

Secondary:NYC

Primary:NYC

Secondary:SYD

Secondary:LON
Secondary:S...
MongoDB Business Value

Faster Time to Market

Enabling New Apps

14

Lower TCO

Response Time & Scalability
When Does MongoDB Help?
Database Landscape
2010
1990

2000

RDBMS

Operational
Database

NoSQL

RDBMS

Key-Value/
Wide-column
Document DB

RDBMS
D...
MongoDB-Hadoop Connector

Operational
Database

Processing
& Storage
MongoDB-Hadoop
Connector

•
•
•
•
•

17

Low latency
...
Operational Database Use Cases

MongoDB
RDBMSs

Key/Value or
Wide Column Stores

18
MongoDB 5th Most Popular Database

19
Leading Organizations Rely on MongoDB

20
Criteria for benefitting most from
MongoDB instead of RDBMS
 You want to aggregate data from multiple sources
 You want ...
Case Studies of Architectural
Capabilities
Difficult Issues Today
1. Performance and agility issues with RDBMS
2. Building a single view across disparate systems
3. ...
Challenge: Performance and agility
issues with RDBMS

Code

DB Schema

Application

24

XML Config

Object Relational
Mapp...
Solution: Match Data to Application
and Optimize Disk IOPS
Code

XML Config

DB Schema

Application

Object Relational
Map...
Case Study
Uses MongoDB to power enterprise social
networking platform
Problem
• Complex SQL queries,
highly normalized
sc...
Challenge: Building a single view
across disparate systems
Batch

Datamar
t

Batch

Datamar
t

Customer
Accounts
Loans
Loa...
Solution: Using dynamic schema and
easy scaling
Operational Data Hub

Customer
Accounts

CSR Application

Real-time or
Bat...
Case Study
Insurance leader generates coveted 360-degree view of
customers in 90 days – “The Wall”
Problem
•

No single vi...
Challenge: Legacy systems often not
real-time enabled or too slow
Data
source 1

Batch copy

Application 1

Often not read...
Solution: Virtualize legacy systems
with a persistent caching service
Mainfram
e

Batch

Batch copy

API
Batch copy

Appli...
Case Study: Global Custodial Bank
Virtualize Enterprise Data Sources
Create a central data hub for accessing data across
t...
Challenge: Master data can be hard
to change and distribute
Batch

Batch

Batch
Golden
Copy

Common issues
• Hard to chang...
Solution: Persistent dynamic cache
replicated globally

Real-time

Real-time
Real-time

Real-time
Real-time

Solution:
• L...
Case Study: Global bank
Reference Data Distribution
Distribute reference data globally in real-time for
fast local accessi...
Reporting
Reporting

Silo 2
Transactions

Silo 2 Systems

Silo 3
Transactions

36

…

Silo 1 Systems

…

Silo 1
Transactio...
Solution: Unified data services

Silo 2 Systems

…

…

…
…

Common persistence framework

Reporting

Silo 1 Systems

Silo ...
Case Study: Global Broker Dealer
Trade Mart for all OTC Trades
Distribute reference data globally in real-time for
fast lo...
Enterprise Adoption
Example Adoption Path

Use of MongoDB

Widespread
Adoption

Operationally
Supported
Certified
MongoDB Practice
Defined
A F...
Traditional Data Integrity Enforcement

Application 1

Application 2

Application 3

41

RDBMS

•
•
•

Apps access DB dire...
Modern Apps (SOA) - Data Access
Layer Should Enforce Data Integrity
•
•

Data Integrity and validations done in
Data Acces...
Data Governance Benefits
• Greater adoption from natural developer
framework on common data models
• Easier for master dat...
MongoDB Partners (360+) &
Integration
Software & Services

Cloud & Channel

44

Hardware
Factors to Consider in Adoption
• SDLC and data governance for an application
• Enterprise-wide data governance (inter-app...
Recommended Center of Excellence
Database Engineering & CoE

Database
Advisory
Services

Operational Database CoE
RDBMS
En...
Summary
• Enormous technology and business change today
• Old technologies not suited for many of them
• MongoDB is purpos...
MongoDB Products and Services
Subscriptions
MongoDB Enterprise, Monitoring, Support, Commercial License

Consulting
Expert...
For More Information
Resource

MongoDB Downloads

mongodb.com/download

Free Online Training

education.mongodb.com

Webin...
Webinar: An Enterprise Architect’s View of MongoDB
Webinar: An Enterprise Architect’s View of MongoDB
Upcoming SlideShare
Loading in …5
×

Webinar: An Enterprise Architect’s View of MongoDB

5,049
-1

Published on

In the world of big data, legacy modernization, siloed organizations, empowered customers, and mobile devices, making informed choices about your enterprise infrastructure has become more important than ever. The alternatives are abundant, and the successful Enterprise Architect must constantly discern which new technology is just a shiny object and which will add true business value.

MongoDB is more than just a great application database for developers; it gives Enterprise Architects new capabilities to solve previously difficult architectural requirements much more easily. Take for example the challenge of many siloed systems at MetLife – with MongoDB, the Metlife team was able to successfully provide a single view into those 70 systems, in only 3 months.

In this webinar, we will:

Explore real life challenges enterprises face with case studies of their solutions
Consider how best to introduce MongoDB in the enterprise
Give an overview of how to optimize the use of MongoDB

Published in: Technology
0 Comments
17 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,049
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
212
Comments
0
Likes
17
Embeds 0
No embeds

No notes for slide
  • Here’s a relational model for an application. It has hundreds of tables.If you are the new developer who just joined the team, congratulations!!Here’s a map of the database, now go figure out how to add your new feature (or fix a bug).Good luck!
  • Point out what other NoSQL databases have (not rich querying and strong consistency)
  • One of the main reasons is the data model.Documents are just easier.If my app tracks car collections, I don’t need to know dozens of tables – all the data for an individual and their collection is in one document. (Walk through this example)Dynamic schema
  • Single view of a customer
  • Can store all accounts in one tableHave performance capacity and easy scaling to to do real-time, not just batch
  • Dynamic schema again importantAuto-sharding allow infinite capacity on commodity hardware
  • Compared to distributed cache - $ and fixed schema
  • Single view of a customer
  • Growing ~20% monthlyCertification: Cloud, BI/ETL, Analytics, Auditing/SecurityOther partners in BI (e.g., Pentaho, Jaspersoft) with many more comingIBM: Standardizing on BSON, MongoDB query language, and MongoDB wire protocol; integration with Guardium security product; integration with WebSphereRed Hat: Collaborating on a secure architecture for MongoDBInformatica: Integration with ETLAmazon: Easily deploy MongoDB on Amazon EC2; we have worked together to develop reference architectures and to use MongoDB with Amazon’s latest technologies, such as SSD instances and Provisioned IOPS (PIOPS)Rackspace: Rackspace offers a purpose-build database-as-a-service offering for MongoDB (through acquisition of ObjectRocket)Microsoft Azure: We have collaborated on tools to make it easy to deploy MongoDB on Microsoft AzureIntel, EMC, NetApp: We’re certified to work with their hardware. More to come.
  • Webinar: An Enterprise Architect’s View of MongoDB

    1. 1. An Enterprise Architect’s View of MongoDB Matt Kalan Business Architect matt.kalan@mongodb.com @matthewkalan
    2. 2. Agenda • Modern drivers of change on enterprises • Requirements these create • How traditional databases are handling changes • New capabilities needed • How MongoDB provides these capabilities • Case studies • Enterprise adoption 2
    3. 3. Modern Requirements
    4. 4. More Technologies and Requirements Than Ever Opportunity cost NoSQL Analytics Globalization JSON Big Data Datawarehouse Customer 360 Document Data Stores Key-value Hadoop ODS MongoDB Graph Wide-column Cloud Computing Cross-channel New Revenue Streams Faster Competition Emerging markets Agile Development Regulation Internet of Things Gamification More with less Mobile Social networking Empowered customers Consumerization Lowering TCO 4
    5. 5. Questions for Enterprise Architects • What current and future requirements does all this raise? • How to prepare my enterprise to handle these? • Which technologies and products will help me? • How to bring them into my enterprise successfully? • How does old and new technology work together? • What does the future state architecture look like? 5
    6. 6. Modern Application Requirements Data Types & OOP Volume of Data New Architectures • Object-oriented • Petabytes of data • Horizontal scaling • Variably structured • Trillions of records • Unstructured (not tabular) • Millions of queries per second • Commodity servers • Cloud computing Agile Development Single Views • Iterative • Disparate data • Short development cycles • Intraday • Fast time-to-market 6 RDBMS • Cross-channel/silo • Global
    7. 7. Impact of New Requirements Handled with 40-year old Technology • Customfield1…100 or separate tables • Caching & ORMs • Expensive hardware and storage • Schema migration project • One canonical schema • Application-specific partitioning • Use files instead of databases • Schema change takes 6 months 7 Slow time-to-market Agility lost High cost Failed projects Business frustrated
    8. 8. How Do I Prepare My Enterprise?
    9. 9. What Could a Modern Database Do to Make This Easier • Dynamic and variable schemas • Richly-structured data • Much faster performance • Easy horizontal scaling • Low TCO • Plus still maintaining capabilities – Rich querying – Strongly consistently data 10
    10. 10. Documents Support Modern Data Relational Document Data Structure { first_name: „Paul‟, surname: „Miller‟ city: „London‟, location: [45.123,47.232], cars: [ { model: „Bentley‟, year: 1973, value: 100000, picture: <binary>, … }, { model: „Rolls Royce‟, year: 1965, value: 330000, … } } } 11
    11. 11. MongoDB Supports Modern Requirements 1. Dynamic Document Schema Application 2. Native language drivers db.customer.insert({…}) db.customer.find({ name: ”John Smith”}) { name: “John Smith”, date: “2013-08-01”), address: “10 3rd St.”, phone: [ { home: 1234567890}, { mobile: 1234568138} ] } Driver Mongos 3. High availability Shard 2 Shard N Primary Primary Primary Secondary Secondary Secondary - Replica sets Shard 1 Secondary … 5. Horizontal scalability 12 - Sharding Secondary Secondary 4. High performance - Data locality - Rich Indexes - RAM
    12. 12. Global Deployment with Local Read/Writes Primary:LON Secondary:NYC Primary:NYC Secondary:SYD Secondary:LON Secondary:SYD Primary:SYD Secondary:LON Secondary:NYC 13
    13. 13. MongoDB Business Value Faster Time to Market Enabling New Apps 14 Lower TCO Response Time & Scalability
    14. 14. When Does MongoDB Help?
    15. 15. Database Landscape 2010 1990 2000 RDBMS Operational Database NoSQL RDBMS Key-Value/ Wide-column Document DB RDBMS Datawarehousing OLAP/DW Hadoop OLAP/DW 16
    16. 16. MongoDB-Hadoop Connector Operational Database Processing & Storage MongoDB-Hadoop Connector • • • • • 17 Low latency Rich fast querying Request/response Aggregations in database Known data relationships • • • • Longer jobs Batch analytics Highly parallel processing Unknown data relationships
    17. 17. Operational Database Use Cases MongoDB RDBMSs Key/Value or Wide Column Stores 18
    18. 18. MongoDB 5th Most Popular Database 19
    19. 19. Leading Organizations Rely on MongoDB 20
    20. 20. Criteria for benefitting most from MongoDB instead of RDBMS  You want to aggregate data from multiple sources  You want agile development and/or fastest time-to-market  You expect the schema to change often  You have variably or unstructured data (records might have different fields)  Your data is hierarchical (i.e. hard to model in RDBMS), e.g. JSON  You expect the data to grow quickly and want ease of scaling out  You want the best performance possible for real-time read/write  You want the lowest TCO and resources including with replication and caching  Performance of database directly impacts user experience  You want real-time analytics and aggregations  You want location-based querying (distance from locations, within regions, etc.)  You have challenges today with building canonical models, scale, TCO, or agility 21
    21. 21. Case Studies of Architectural Capabilities
    22. 22. Difficult Issues Today 1. Performance and agility issues with RDBMS 2. Building a single view across disparate systems 3. Legacy systems often not real-time enabled 4. Master data can be hard to change and distribute 5. Operational applications are siloed 23
    23. 23. Challenge: Performance and agility issues with RDBMS Code DB Schema Application 24 XML Config Object Relational Mapping Relational Database
    24. 24. Solution: Match Data to Application and Optimize Disk IOPS Code XML Config DB Schema Application Object Relational Mapping Relational Database Code Text Search Rich Queries Application 25 Geospatial Aggregatio n Map Reduce
    25. 25. Case Study Uses MongoDB to power enterprise social networking platform Problem • Complex SQL queries, highly normalized schema not aligned with new data types • Poor performance • Lack of horizontal scalability 26 Why MongoDB Results • Dynamic schemas using JSON • Flexibility to roll out new social features quickly • Ability to handle complex data while maintaining high performance • Sped up reads from 30 seconds to tens of milliseconds • Social network analytics with lightweight MapReduce • Dramatically increased write performance
    26. 26. Challenge: Building a single view across disparate systems Batch Datamar t Batch Datamar t Customer Accounts Loans Loans Silo 2 Loans Web … Deposits Deposits Silo 3 Cards Mobile 27 Batch Datamar t Batch Data Warehouse Reporting Cards Cards Silo 1 Banking Store Issues • Yesterday’s data • Details lost • Inflexible schema • Slow performance Impact • What happened today? • Worse customer satisfaction • Missed opportunities • Lost revenue
    27. 27. Solution: Using dynamic schema and easy scaling Operational Data Hub Customer Accounts CSR Application Real-time or Batch Customer Portal Loans Loans Silo 2 … … Deposits Deposits Silo 3 28 Operational Reporting Data Warehouse Strategic Reporting Cards Cards Silo 1 Benefits • Real-time • Complete details • Agile • Higher customer retention • Increase wallet share • Proactive exception handling
    28. 28. Case Study Insurance leader generates coveted 360-degree view of customers in 90 days – “The Wall” Problem • No single view of customer • 145 yrs of policy data, 70+ systems, 15+ apps Why MongoDB • Agility – prototype in 5 days; production in 90 days • 2 years, $25M in failing to aggregate in RDBMS • Dynamic schema & rich querying – combine disparate data into one data store • Poor customer experience • Hot tech to attract top talent 29 Results • Unified customer view available to all channels • Increased call center productivity • Better customer experience, reduced churn, more upsell opps • Dozens more projects on same data platform
    29. 29. Challenge: Legacy systems often not real-time enabled or too slow Data source 1 Batch copy Application 1 Often not ready to expose as enterprise services • Mainframe • Core systems • Data Warehouses • Not scalable system Application 2 Data source 2 … Slow request/response 30 Application 3 … Data source N Batch copying of data many times or requests are too slow Application X Changing source data affects X systems Impact • Slow time to market • Resource intensive • Hard to change interfaces and modernize system
    30. 30. Solution: Virtualize legacy systems with a persistent caching service Mainfram e Batch Batch copy API Batch copy Application 1 Application 2 EDW … … Pub/sub … Core system Application 3 Application X 31 Benefits • Faster time to market • More agile in changing sources • Can modernize data sources behind virtualization • Infinite scale with low TCO
    31. 31. Case Study: Global Custodial Bank Virtualize Enterprise Data Sources Create a central data hub for accessing data across the enterprise Problem • Found numerous pointto-point copies of data • Change in one system impacts multiple groups • Response time on EDW was too slow • Wanted one central data hub for most often accessed data 32 Why MongoDB Results • Dynamic schema: can • Data accessible by batch normalize data as needed or REST layer in one place and prioritized • Customer portal response • Performance: can handle times shrunk by 90% all data in one logical DB • Shorter development times • Sharding: can add data with more accessible hub easily by scaling out • Could modernize data sources without changing apps
    32. 32. Challenge: Master data can be hard to change and distribute Batch Batch Batch Golden Copy Common issues • Hard to change schema of master data • Data copied everywhere and gets out of sync Batch Batch Batch Batch Batch Impact • Process breaks from out of sync data • Business doesn’t have data it needs • Many copies creates 33 more management
    33. 33. Solution: Persistent dynamic cache replicated globally Real-time Real-time Real-time Real-time Real-time Solution: • Load into primary with any schema • Replicate to and read from secondaries Real-time Real-time Real-time Benefits • Easy & fast change at speed of business • Easy scale out for one stop shop for data • Low TCO 34
    34. 34. Case Study: Global bank Reference Data Distribution Distribute reference data globally in real-time for fast local accessing and querying Problem • Delays up to 36 hours in distributing data by batch • Charged multiple times globally for same data • Incurring regulatory penalties from missing SLAs • Had to manage 20 distributed systems with same data 35 Why MongoDB Results • Dynamic schema: easy to • Will save about load initially & over time $40,000,000 in costs and penalties over 5 years • Auto-replication: data distributed in real-time, • Only charged once for data read locally • Data in sync globally and • Both cache and database: read locally cache always up-to-date • Capacity to move to one • Simple data modeling & global shared data service analysis: easy changes and understanding
    35. 35. Reporting Reporting Silo 2 Transactions Silo 2 Systems Silo 3 Transactions 36 … Silo 1 Systems … Silo 1 Transactions Reporting Challenge: Operational applications are siloed Silo 3 Systems Impact • Views are siloed • Duplicate management and data access layer • Need another layer to aggregate
    36. 36. Solution: Unified data services Silo 2 Systems … … … … Common persistence framework Reporting Silo 1 Systems Silo 3 Systems 37 Benefit • Each application can still save its own data • Data is already aggregated for crosssilo reporting • One cluster and data access layer to manage
    37. 37. Case Study: Global Broker Dealer Trade Mart for all OTC Trades Distribute reference data globally in real-time for fast local accessing and querying Problem • Each application had its own persistence and audit trail • Wanted one unified framework and persistence for all trades and products • Needed to handle many variable structures across all securities 38 Why MongoDB Results • Dynamic schema: can • Fast time-to-market using save trade for all products the persistence framework in one data service • Store any structure of • Easy scaling: can easily products/trades without keep trades as long as changing a schema required with high • One consolidated trade performance store for auditing and reporting
    38. 38. Enterprise Adoption
    39. 39. Example Adoption Path Use of MongoDB Widespread Adoption Operationally Supported Certified MongoDB Practice Defined A Few Projects One Project Time 40
    40. 40. Traditional Data Integrity Enforcement Application 1 Application 2 Application 3 41 RDBMS • • • Apps access DB directly Data Integrity must be in the RDBMS Schema implemented by a DBA
    41. 41. Modern Apps (SOA) - Data Access Layer Should Enforce Data Integrity • • Data Integrity and validations done in Data Access Layer Implemented in code MongoDB Cluster Application 1 Application 2 … Application N 42 Data Access Layer API on TCP/IP … REST/API/WS
    42. 42. Data Governance Benefits • Greater adoption from natural developer framework on common data models • Easier for master data or upstream changes to flow into MongoDB-backed apps • MongoDB useful for distributing master data • ETL providers support MongoDB most in NoSQL 43
    43. 43. MongoDB Partners (360+) & Integration Software & Services Cloud & Channel 44 Hardware
    44. 44. Factors to Consider in Adoption • SDLC and data governance for an application • Enterprise-wide data governance (inter-app) • Roles and responsibilities • Training requirements • Operations/production support • Center of Excellence (COE) • Process for choosing which DB to use • How to work with other technologies in-house 45
    45. 45. Recommended Center of Excellence Database Engineering & CoE Database Advisory Services Operational Database CoE RDBMS Engineering Database SMEs MongoDB Engineering MongoDB Incubator (& cluster) RDBMS PaaS Engineering MongoDBaaS Engineering Datawarehousing CoE DW Product Engineering 46 Database SMEs Hadoop Incubator Clusters DW PaaS Engineering Hadoop PaaS Engineering
    46. 46. Summary • Enormous technology and business change today • Old technologies not suited for many of them • MongoDB is purpose built for today and future applications • And can help solve common architectural challenges • Bring MongoDB in to learn how to adopt it more widely when appropriate • Firms using MongoDB benefit from 50% time-to-market, 70% lower TCO, and making the infeasible possible 47
    47. 47. MongoDB Products and Services Subscriptions MongoDB Enterprise, Monitoring, Support, Commercial License Consulting Expert Resources for All Phases of MongoDB Implementations Training Online and In-Person for Developers and Administrators MongoDB Monitoring Service Free, Cloud-Based Service for Monitoring and Alerts MongoDB Backup Service Cloud-based service for backing up and restoring MongoDB 48
    48. 48. For More Information Resource MongoDB Downloads mongodb.com/download Free Online Training education.mongodb.com Webinars and Events mongodb.com/events White Papers mongodb.com/white-papers Case Studies mongodb.com/customers Presentations mongodb.com/presentations Documentation docs.mongodb.org Additional Info 49 Location info@mongodb.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×