Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
An Enterprise Architect’s View of
MongoDB
Matt Kalan
Sr. Solution Architect
matt.kalan@mongodb.com
@matthewkalan
2
• Modern drivers of change on enterprises
• Requirements these create
• How traditional databases are handling changes
•...
Today’s Requirements
4
More Technologies and Requirements
Than Ever
Big Data
NoSQL
Key-value
Wide-column
Document Data Stores
MongoDB
Mobile
Cl...
5
More Technologies and Requirements
Than Ever
Big Data
NoSQL
Key-value
Wide-column
Document Data Stores
MongoDB
Mobile
Cl...
6
• What current and future requirements does all
this raise?
• How to prepare my enterprise to handle these?
• Which tech...
Today’s Webinar Focus:
Application Development &
Reporting
8
The World Has Changed
Data
• Volume
• Velocity
• Variety
Time
• Iterative
• Agile
• Short Cycles
Risk
• Always On
• Scal...
9
Expressive
Query
Language
Strong
Consistency
Secondary
Indexes
Flexibility
Scalability
Performance
Relational
10
• Customfield1…100 or separate tables
• Caching & ORMs
• Expensive hardware and storage
• Schema migration project
• On...
12
NoSQL
Expressive
Query
Language
Strong
Consistency
Secondary
Indexes
Flexibility
Scalability
Performance
13
Expressive
Query
Language
Strong
Consistency
Secondary
Indexes
Flexibility
Scalability
Performance
Relational NoSQLNexu...
MongoDB Capabilities for
Today’s World
15
Documents Support Modern
Requirements
Relational Document Data Structure
{ customer_id : 1,
first_name : "Mark",
last_n...
16
Basic Insert/Query Examples
Objective Java Example Command-line
Javascript Shell
Insert a map Map m;
…
collection.inser...
17
Application
Driver
Query Router
Primary
Secondary
Secondary
Shard 1
Primary
Secondary
Secondary
Shard 2
…
Primary
Secon...
18
Better Data
Locality
Performance
In-Memory
Caching
In-Place
Updates
19
*Included with MongoDB Enterprise Advanced
BUSINESS NEEDS SECURITY FEATURES
Authentication SCRAM, LDAP*, Kerberos*, x.5...
20
Global Deployment with Local
Read/Writes
Primary:NYC
Secondary:NYC
Primary:LON
Primary:SYD
Secondary:LON
Secondary:NYC
...
21
MongoDB Business Value
Competitive Advantage Mitigating Risk
Lower TCOFaster Time to Value
When to Use MongoDB?
23
Data Management Revolution
2014
RDBMS
Key-Value/
Column Store
OLAP/DW
Hadoop
2000
RDBMS
OLAP/DW
1990
RDBMS
Operational
...
24
MongoDB-Hadoop Connector
• Low latency
• Rich fast querying
• Flexible indexing
• Aggregations in database
• Known data...
25
MongoDB 4th Most Popular Database
26
Leading Organizations Rely on MongoDB
Usage Patterns & Case Studies
28
1. Operational Data Store (ODS)
2. Enterprise Data Service
3. Datamart/Cache
4. Master Data Distribution
5. Single Oper...
Architectural Pattern –
Operational Data Store (ODS)
30
Challenge: Applications not agile nor
scalable enough
Requirement changes
Change
31
Solution: Match dynamic data model
to the application
32
Criteria for benefitting most from
MongoDB instead of RDBMS
Data
 Variably or
unstructured
 Hierarchical
 Geo-coordi...
33
One of the world's largest providers of payments solutions
constructs a completely reliable and robust mobile
experienc...
Architectural Pattern –
Enterprise Data Service
35
Challenge: Siloed operational
applications
Silo 1 Data
Silo 2 Data
Silo N Data
… Impact
• Views are siloed
• Duplicate ...
36
Solution: Unified data service
… Benefit
• Each application can still
save its own data
• Data is already aggregated
fo...
37
Distribute reference data globally in real-time for
fast local accessing and querying
Case Study: Global Broker Dealer
...
Architectural Pattern –
Datamart/Cache
39
Challenge: Response From Data
Warehouse or Other System is Slow
Cards
Loans
Deposits
…
Data
Warehouse
Issues
• Data sto...
40
Solution: Optimize Data Structure as a
Datamart In-memory or On-disk
Cards
Loans
Deposits
…
Data
Warehouse
Solution
• D...
41
Needed fast reporting for finance on global
banking transaction data (about 2 petabytes)
Case Study: Tier 1 Global Bank...
Architectural Pattern –
Master Data Distribution
43
Challenge: Master data can be hard
to change and distribute
Golden
Copy
Batch
Batch
Batch
Batch
Batch
Batch
Batch
Batch...
44
Solution: Persistent dynamic cache
replicated globally
Real-time
Real-time Real-time
Real-time
Real-time
Real-time
Real...
45
Distribute reference data globally in real-time for
fast local accessing and querying
Case Study: Global bank
Reference...
Architectural Pattern –
Single Operational View
47
Challenge: Aggregation of disparate
data is difficult
Cards
Loans
Deposits
…
Data
Warehouse
Batch
Cross-Silo
applicatio...
48
Solution: Using dynamic schema and
easy scaling
Data
Warehouse
Real-time or
Batch
…
Customer-facing
Applications
Regula...
49
Insurance leader generates coveted 360-degree view of
customers in 90 days – “The Wall”
Case Study
Problem Why MongoDB ...
50
Expanded Single View of ….
…
Single CSR
Application
Unified
Customer Portal
Operational
Reporting
Cards …CardsSilo 1
…
...
51
Processing + Data Access Paradigm
Processing
model
Data access
model
Request/response
Map-reduce
Batch, ETL, etc.
Analy...
52
Processing + Data Access Paradigm
Processing
model
Data access
model
Request/response
Map-reduce
Batch, ETL, etc.
Analy...
53
Processing + Data Access Paradigm
Processing
model
Data access
model
Request/response
Map-reduce
Batch, ETL, etc.
Analy...
Enterprise Adoption
55
Example Adoption PathUseofMongoDB
One Project
A Few Projects
Certified
Operationally
Supported
Widespread
Adoption
Time...
56
Traditional Data Integrity Enforcement
RDBMS
• Apps access DB directly
• Data Integrity must be in the RDBMS
• Schema i...
57
Modern Apps (SOA) - Data Access
Layer Should Enforce Data Integrity
Application 1
MongoDB Cluster
Application 2
Data
Ac...
58
• Greater adoption from offering an easy-to-use
developer framework on common data models
• Easier for master data or u...
59
Partner Ecosystem (500+)
60
• SDLC and data governance for an application
• Enterprise-wide data governance (inter-app)
• Enterprise-wide security
...
61
Recommended Center of Excellence
MongoDB
Engineering
RDBMS
Engineering
Operational Database CoE
MongoDB
Incubator
(& cl...
62
• The world has changed dramatically in 40 years
• Old technologies not suited for many uses today
• MongoDB is purpose...
63
We’re your partner
64
For More Information
Resource Location
MongoDB Downloads mongodb.com/download
Free Online Training university.mongodb.c...
Enterprise architectsview 2015-apr
Enterprise architectsview 2015-apr
Upcoming SlideShare
Loading in …5
×

Enterprise architectsview 2015-apr

In the world of big data, legacy modernization, siloed organizations, empowered customers, and mobile devices, making informed choices about your enterprise infrastructure has become more important than ever. The alternatives are abundant, and the successful Enterprise Architect must constantly discern which new technology is just a shiny object and which will add true business value.

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Enterprise architectsview 2015-apr

  1. 1. An Enterprise Architect’s View of MongoDB Matt Kalan Sr. Solution Architect matt.kalan@mongodb.com @matthewkalan
  2. 2. 2 • Modern drivers of change on enterprises • Requirements these create • How traditional databases are handling changes • New capabilities needed • How MongoDB provides these capabilities • Case studies • Enterprise adoption Agenda
  3. 3. Today’s Requirements
  4. 4. 4 More Technologies and Requirements Than Ever Big Data NoSQL Key-value Wide-column Document Data Stores MongoDB Mobile Cloud Computing Social networking JSON Internet of Things Hadoop Graph Agile Development ODS Datawarehouse Analytics Consumerization Gamification
  5. 5. 5 More Technologies and Requirements Than Ever Big Data NoSQL Key-value Wide-column Document Data Stores MongoDB Mobile Cloud Computing Social networking JSON Internet of Things Hadoop Graph Agile Development ODS Datawarehouse Analytics Consumerization Gamification Globalization Emerging markets Faster Competition Regulation Cross-channel Empowered customers Lowering TCO More with less New Revenue Streams Customer 360 Opportunity cost Data Monetization Common Services
  6. 6. 6 • What current and future requirements does all this raise? • How to prepare my enterprise to handle these? • Which technologies and products will help me? • How to bring them into my enterprise successfully? • How does old and new technology work together? • What does the future state architecture look like? Questions for Enterprise Architects
  7. 7. Today’s Webinar Focus: Application Development & Reporting
  8. 8. 8 The World Has Changed Data • Volume • Velocity • Variety Time • Iterative • Agile • Short Cycles Risk • Always On • Scale • Global Cost • Open-Source • Cloud • Commodity
  9. 9. 9 Expressive Query Language Strong Consistency Secondary Indexes Flexibility Scalability Performance Relational
  10. 10. 10 • Customfield1…100 or separate tables • Caching & ORMs • Expensive hardware and storage • Schema migration project • One canonical schema • Application-specific partitioning • Use files instead of databases • Schema change takes 6 months Impact of Relational DB Usage
  11. 11. 12 NoSQL Expressive Query Language Strong Consistency Secondary Indexes Flexibility Scalability Performance
  12. 12. 13 Expressive Query Language Strong Consistency Secondary Indexes Flexibility Scalability Performance Relational NoSQLNexus Architecture Relational + NoSQL
  13. 13. MongoDB Capabilities for Today’s World
  14. 14. 15 Documents Support Modern Requirements Relational Document Data Structure { customer_id : 1, first_name : "Mark", last_name : "Smith", city : "San Francisco", location : [40.74, -73.97], image : <binary>, phones: [ { number : “1-212-777-1212”, dnc : true, type : “home” }, { number : “1-212-777-1213”, type : “cell” }] }
  15. 15. 16 Basic Insert/Query Examples Objective Java Example Command-line Javascript Shell Insert a map Map m; … collection.insert( new BasicDBObject(m)); db.collection.insert(m); Find all contacts with at least one mobile phone DBObject expr = new BasicDBObject(); expr.put(“phones.type”, “cell”); List<DBObject> L = collection.find(expr).toArray(); db.contact.find( {"phones.type”:”cell”});
  16. 16. 17 Application Driver Query Router Primary Secondary Secondary Shard 1 Primary Secondary Secondary Shard 2 … Primary Secondary Secondary Shard N db.customer.insert({…}) db.customer.find({ name: ”John Smith”}) 1.Dynamic Document Schema { name: “John Smith”, date: “2013-08-01”), address: “10 3rd St.”, phone: [ { home: 1234567890}, { mobile: 1234568138} ] } 2. Native language drivers 4. High performance - Data locality - Rich Indexes - RAM 3. High availability - Replica sets 5. Horizontal scalability - Sharding MongoDB Technical Capabilities
  17. 17. 18 Better Data Locality Performance In-Memory Caching In-Place Updates
  18. 18. 19 *Included with MongoDB Enterprise Advanced BUSINESS NEEDS SECURITY FEATURES Authentication SCRAM, LDAP*, Kerberos*, x.509 Certificates Authorization Built-in Roles, User-Defined Roles, Field-Level Redaction Auditing* Admin, DML, DDL, Role-based Encryption Network: SSL (with FIPS 140-2), Disk: partner solutions Enterprise-Grade Security
  19. 19. 20 Global Deployment with Local Read/Writes Primary:NYC Secondary:NYC Primary:LON Primary:SYD Secondary:LON Secondary:NYC Secondary:SYD Secondary:LON Secondary:SYD
  20. 20. 21 MongoDB Business Value Competitive Advantage Mitigating Risk Lower TCOFaster Time to Value
  21. 21. When to Use MongoDB?
  22. 22. 23 Data Management Revolution 2014 RDBMS Key-Value/ Column Store OLAP/DW Hadoop 2000 RDBMS OLAP/DW 1990 RDBMS Operational Database Datawarehousing Document DB NoSQL
  23. 23. 24 MongoDB-Hadoop Connector • Low latency • Rich fast querying • Flexible indexing • Aggregations in database • Known data relationships • Great for any subset of data • Longer jobs • Batch analytics • Highly parallel processing • Unknown data relationships • Great for looking at all data or large subsets Applications Distributed Analytics MongoDB Connector for Hadoop
  24. 24. 25 MongoDB 4th Most Popular Database
  25. 25. 26 Leading Organizations Rely on MongoDB
  26. 26. Usage Patterns & Case Studies
  27. 27. 28 1. Operational Data Store (ODS) 2. Enterprise Data Service 3. Datamart/Cache 4. Master Data Distribution 5. Single Operational View Architecture Patterns System of Record System of Engagement
  28. 28. Architectural Pattern – Operational Data Store (ODS)
  29. 29. 30 Challenge: Applications not agile nor scalable enough Requirement changes Change
  30. 30. 31 Solution: Match dynamic data model to the application
  31. 31. 32 Criteria for benefitting most from MongoDB instead of RDBMS Data  Variably or unstructured  Hierarchical  Geo-coordinates  Disparate sources  Schema changes often Querying  Real-time analytics & aggregations  Location-based  Lowest latency  Performance affects user experience Requirements  Agile development & fastest time-to-market  Data will grow quickly  Best performance for request/response  Lowest TCO  Multiple sources aggregated  Challenges today with RDBMS
  32. 32. 33 One of the world's largest providers of payments solutions constructs a completely reliable and robust mobile experience ADP’s Global Mobile Platform Problem Why MongoDB Results • Needed a signature mobile app for customers • Must support millions of users • Needed to quickly change features & functionality • High availability was critically important • Built-in high availability architecture optimized for global, multi-data center distribution • Dynamic schema & rich querying – deep functionality from launch & new features easily added • Much lower TCO, especially with commodity hardware • iTunes App Store “Top 15” business app since 2012 launch • Over 1 million active users, 17 countries, 23 languages • Extremely high performance through predictive caching • Maintenance much easier => simple codebase, less hardware • New functionality easy and quick to add
  33. 33. Architectural Pattern – Enterprise Data Service
  34. 34. 35 Challenge: Siloed operational applications Silo 1 Data Silo 2 Data Silo N Data … Impact • Views are siloed • Duplicate management and data access layer • Need another layer to aggregate Silo 1 systems Silo 2 Systems Silo N Systems … ReportingReportingReporting
  35. 35. 36 Solution: Unified data service … Benefit • Each application can still save its own data • Data is already aggregated for cross-silo reporting • One cluster and data access layer to manage Silo 1 Systems Silo 2 Systems Silo N Systems … Reporting ……
  36. 36. 37 Distribute reference data globally in real-time for fast local accessing and querying Case Study: Global Broker Dealer Trade Mart for all OTC Trades Problem Why MongoDB Results • Each application had its own persistence and audit trail • Wanted one unified framework and persistence for all trades and products • Needed to handle many variable structures across all securities • Dynamic schema: can save trade for all products in one data service • Easy scaling: can easily keep trades as long as required with high performance • Rich querying: can query on any fields each business requires • Fast time-to-market using the persistence framework • Store any structure of products/trades without changing a schema • One consolidated trade store for auditing and reporting
  37. 37. Architectural Pattern – Datamart/Cache
  38. 38. 39 Challenge: Response From Data Warehouse or Other System is Slow Cards Loans Deposits … Data Warehouse Issues • Data stored normalized • Reports slow to generate • Data updated daily but user response must be fast Impact • Lost productivity • Dissatisfied users and business Reporting Cards Silo 1 Loans Silo 2 Deposits Silo 3
  39. 39. 40 Solution: Optimize Data Structure as a Datamart In-memory or On-disk Cards Loans Deposits … Data Warehouse Solution • Data stored in optimal structure for reports • Optionally in memory Impact • Response times is as fast as possible • Users and business satisfied FastReporting Cards Silo 1 Loans Silo 2 Deposits Silo 3 … Datamart/Cache
  40. 40. 41 Needed fast reporting for finance on global banking transaction data (about 2 petabytes) Case Study: Tier 1 Global Bank - Personalized In-memory Datamart Problem Why MongoDB Results • Data warehouse was too slow for reporting • No visibility into how long reports took • Could not generate multiple ad hoc reports • Users included regulators so even more demanding • Dynamic schema: store data in optimal structure • Performance: storing report results optimally • In-memory caching of results • Rich querying: can query on any field • Easy scaling: results spread across shards to generate report in parallel • Create a personalized in- memory data mart • Reports configured and notified when results ready • Data all in memory so fast to manipulate • Data spread across shards for ultra-fast reporting
  41. 41. Architectural Pattern – Master Data Distribution
  42. 42. 43 Challenge: Master data can be hard to change and distribute Golden Copy Batch Batch Batch Batch Batch Batch Batch Batch Common issues • Hard to change schema of master data • Data copied everywhere and gets out of sync Impact • Process breaks from out of sync data • Business doesn’t have data it needs • Many copies creates more management
  43. 43. 44 Solution: Persistent dynamic cache replicated globally Real-time Real-time Real-time Real-time Real-time Real-time Real-time Real-time Solution: • Load into primary with any schema • Replicate to and read from secondaries Benefits • Easy & fast change at speed of business • Easy scale out for one stop shop for data • Low TCO
  44. 44. 45 Distribute reference data globally in real-time for fast local accessing and querying Case Study: Global bank Reference Data Distribution Problem Why MongoDB Results • Delays up to 36 hours in distributing data by batch • Charged multiple times globally for same data • Incurring regulatory penalties from missing SLAs • Had to manage 20 distributed systems with same data • Dynamic schema: easy to load initially & over time • Auto-replication: data distributed in real-time, read locally • Both cache and database: cache always up-to-date • Simple data modeling & analysis: easy changes and understanding • Will save about $40,000,000 in costs and penalties over 5 years • Only charged once for data • Data in sync globally and read locally • Capacity to move to one global shared data service
  45. 45. Architectural Pattern – Single Operational View
  46. 46. 47 Challenge: Aggregation of disparate data is difficult Cards Loans Deposits … Data Warehouse Batch Cross-Silo applications Issues • Yesterday’s data • Details lost • Inflexible schema • Slow performance Datamar t Datamar t Datamar t Batch Impact • What happened today? • Worse customer satisfaction • Missed opportunities • Lost revenue Batch Batch Reporting Cards Silo 1 Loans Silo 2 Deposits Silo 3
  47. 47. 48 Solution: Using dynamic schema and easy scaling Data Warehouse Real-time or Batch … Customer-facing Applications Regulatory applications Operational Single View Benefits • Real-time • Complete details • Agile • Higher customer retention • Increase wallet share • Proactive exception handling Strategic Reporting Operational Reporting Cards Loans Deposits … Customer Accounts Cards Silo 1 Loans Silo 2 Deposits Silo N
  48. 48. 49 Insurance leader generates coveted 360-degree view of customers in 90 days – “The Wall” Case Study Problem Why MongoDB Results • No single view of customer • 145 yrs of policy data, 70+ systems, 15+ apps • 2 years, $25M in failing to aggregate in RDBMS • Poor customer experience • Agility – prototype in 5 days; production in 90 days • Dynamic schema: Imperative to combine disparate data • Rich querying: necessary for match data across silos • Hot tech to attract top talent • Unified customer view available to all channels • Increased call center productivity • Better customer experience, reduced churn, more upsell opps • Dozens more projects on same data platform
  49. 49. 50 Expanded Single View of …. … Single CSR Application Unified Customer Portal Operational Reporting Cards …CardsSilo 1 … Operational Data Layer • Request/response • Millisecond latency • Easily scalable • Flexible schema • Low TCO • Rich querying with indexes DW/Data Lake • Analytical/batch processing • Seconds to hours latency • Also scalable, low TCO, & flexible schema • Pre-defined slices of data (limited indexes) MongoDB Hadoop Connector … CardsCardsSilo 2 CardsCardsSilo N ETL Pub-sub/ETL Customer Clustering Churn Analysis Predictive analytics …
  50. 50. 51 Processing + Data Access Paradigm Processing model Data access model Request/response Map-reduce Batch, ETL, etc. Analytical Jobs Latency important (e.g. user waiting) Milliseconds to seconds Small to large subsets of data Indexes valuable Multiple seconds to hours Processing all or large sets of data Indexes not used TypicalMongoDB UseCase TypicalHadoop UseCase
  51. 51. 52 Processing + Data Access Paradigm Processing model Data access model Request/response Map-reduce Batch, ETL, etc. Analytical Jobs Latency important (e.g. user waiting) Milliseconds to seconds Small to large subsets of data Indexes valuable Multiple seconds to hours Processing all or large sets of data Indexes not used TypicalMongoDB UseCase TypicalHadoop UseCase
  52. 52. 53 Processing + Data Access Paradigm Processing model Data access model Request/response Map-reduce Batch, ETL, etc. Analytical Jobs Latency important (e.g. user waiting) Milliseconds to seconds Small to large subsets of data Indexes valuable Multiple seconds to hours Processing all or large sets of data Indexes not used TypicalMongoDB UseCase TypicalHadoop UseCase Data Discovery
  53. 53. Enterprise Adoption
  54. 54. 55 Example Adoption PathUseofMongoDB One Project A Few Projects Certified Operationally Supported Widespread Adoption Time MongoDB CoE Defined
  55. 55. 56 Traditional Data Integrity Enforcement RDBMS • Apps access DB directly • Data Integrity must be in the RDBMS • Schema implemented by a DBA Application 1 Application 2 Application 3
  56. 56. 57 Modern Apps (SOA) - Data Access Layer Should Enforce Data Integrity Application 1 MongoDB Cluster Application 2 Data Access Layer Application N … … REST/API/WS API on TCP/IP • Data Integrity and validations done in Data Access Layer • Implemented in code
  57. 57. 58 • Greater adoption from offering an easy-to-use developer framework on common data models • Easier for master data or upstream changes to flow into MongoDB-backed apps • MongoDB useful for distributing master data • ETL providers support MongoDB most in NoSQL Data Governance Benefits
  58. 58. 59 Partner Ecosystem (500+)
  59. 59. 60 • SDLC and data governance for an application • Enterprise-wide data governance (inter-app) • Enterprise-wide security • Roles and responsibilities • Training requirements • Operations/production support • Center of Excellence (COE) • Process for choosing which DB to use • How to work with other technologies in-house Factors to Consider in Adoption
  60. 60. 61 Recommended Center of Excellence MongoDB Engineering RDBMS Engineering Operational Database CoE MongoDB Incubator (& cluster) Database SMEs Database Engineering & CoE RDBMS PaaS Engineering MongoDBaaS Engineering Product Engineering DW & Analytics CoE Hadoop/Spark Incubator Clusters Product SMEs DW PaaS Engineering Hadoop/Spar k PaaS Engineering Database Advisory Services
  61. 61. 62 • The world has changed dramatically in 40 years • Old technologies not suited for many uses today • MongoDB is purpose built for today’s and future applications • And can help solve common architectural challenges • Firms using MongoDB benefit from 50% time-to-market, 70% lower TCO, less risk, and substantial competitive advantage • MongoDB, Inc. can help optimize the value and adoption in your enterprise Summary
  62. 62. 63 We’re your partner
  63. 63. 64 For More Information Resource Location MongoDB Downloads mongodb.com/download Free Online Training university.mongodb.com Webinars and Events mongodb.com/events White Papers mongodb.com/white-papers Case Studies mongodb.com/customers Presentations mongodb.com/presentations Documentation docs.mongodb.org Additional Info info@mongodb.com Resource Location

×