Enterprise Trends:
Database as a Service
Brandon.Newell@MongoDB.com
Senior Solutions Architect
MongoDB
Agenda
• Why Database as a Service?
• DBaaS in my Enterprise architecture
• Enterprise MongoDB-as-a-Service
• Get there with Ops Manager
3
Why Database as a Service?
• Empower developers
– Fast, easy, cheap
• Go BIG
– Seamlessly scale up
for prod
• Go SAFE
– Consistent security,
governance and
operations
iStock licensed (pixelfit)
Enterprise architecture
5
MongoDB and Enterprise IT Stack
EDW
Hadoop
Spark
Management&Monitoring
Security&Auditing
RDBMS
CRM, ERP, Collaboration, Mobile, BI
OS & Virtualization, Compute, Storage, Network
RDBMS
Applications
Infrastructure
Data Management
Online Data Offline Data
6
MongoDB and Enterprise IT Strategy
Legacy Strategic
Apps On-Premise SaaS, Mobile, Social
Database Oracle
Offline Data Teradata Hadoop, Spark
Compute Scale-Up Server Commodity HW / Cloud
Storage SAN Local Storage / Cloud
Network Routers and Switches Software-Defined Networks
7
Revolution in IT provisioning
• Hosting
– Public, Private, and Hybrid
• Stack
– Software | Platform | Infrastructure
…as a Service
• DB platform advantages
– Adoption
– Agility
– Governance
– Efficiency
Public PrivateHybrid
Cloud Clients
Web app, mobile app, thin client…
Software: SaaS
CRM, Email, virtual desktop, games…
Platform: PaaS
Execution runtime, DATABASE, web server, dev tools…
Infrastructure: IaaS
Virtual machines, containers, bare metal, storage, load balancers…
8
Public Cloud
• Commercial cloud
IaaS endless aisle
– Amazon Web Services
– Microsoft Azure
– Google Compute Engine
– Rackspace
– Many more…
• OpenStack
– Apache, Rackspace, NASA
– OpenStack Foundation
iStock licensed (4X-image)
9
In the Enterprise Cloud:
MongoDB as a Service
• Rewards
– Adoption
– Agility, move faster
– Governance
– Efficiency, cost gains
• Risks
– Systematize the wrong solution
– Standardize the wrong hardware
(especially storage)
– Unaffordable or inflexible:
unlimited apathy
– Too cheap: tragedy of the
commons
Bringing it all together
with an Enterprise Service
11
Customer First
• First Customer is your
cheerleader
• First app stakeholders
– Business owner
– Developers
– Ops
• Next few apps
– Same stakeholders iStock licensed (YanC)
12
Delivery Levels
• Application
• Data Service / Data Layer
– US Department of Veterans Affairs: http://goo.gl/8usttw
• Multi-tenancy
– Multiple app databases per MongoDB cluster
• Cluster per app
– Replica set only
– Sharded / replica sets
13
Implementation Considerations
• Server Hardware
• Virtualization or
Containers (Docker)
• Security & Entitlements
• Storage
• Operating System
• Infrastructure Management
• Backup and Recovery
• Accounting and chargeback
• Distributed computing
(Hadoop/Spark)
14
MACHINE 1
Multiple Apps
Multiple Discrete Replicas
Each app gets its own set of 3 machines.
Physical or Virtual Machines
PRIMARY
APP1APP2APP3
MACHINE 2
SECONDARY
MACHINE 3
SECONDARY
MACHINE 4
PRIMARY
MACHINE 5
SECONDARY
MACHINE 6
SECONDARY
MACHINE 7
PRIMARY
MACHINE 8
SECONDARY
MACHINE 9
SECONDARY
Traditional Design
15
Replica
Set1
MACHINE 1
Multiple Apps
Multiple Shared Replicas
Each database app still benefits from the
3 replicas
Physical or Virtual Machines
MongoDB Ops/Cloud Manager is key to
success!
PRIMARY
Replica
Set2
Replica
Set3
MACHINE 2
SECONDARY
MACHINE 3
SECONDARY
MACHINE 4
PRIMARY
MACHINE 5
SECONDARY
MACHINE 6
SECONDARY
MACHINE 7
PRIMARY
MACHINE 8
SECONDARY
MACHINE 9
SECONDARY
Multi-Tenancy Design
DB1
DB3
DB5 DB1
DB3
DB5 DB1
DB3
DB5
DB2
DB4
DB6 DB2
DB4
DB6 DB2
DB4
DB6
DB7
DB9
DB8 DB7
DB9
DB8 DB7
DB9
DB8
16
SECONDARY
cgroup/docker
cgroup/docker cgroup/docker cgroup/docker
cgroup/docker cgroup/docker
cgroup/dockercgroup/docker cgroup/docker
cgroup/dockercgroup/docker cgroup/docker
cgroup/dockercgroup/docker cgroup/docker
cgroup/dockercgroup/docker cgroup/docker
PRIMARY SECONDARY SECONDARY
PRIMARY SECONDARY
PRIMARYSECONDARY SECONDARY
PRIMARYSECONDARY SECONDARY
PRIMARYSECONDARY SECONDARY
PRIMARYSECONDARY SECONDARY
Same concept as Traditional Design,
except now we have more variation in the
sizing and provisioning. Maintaining
Primary and two Secondaries for HA per
application.
Linux cgroups or Docker Containers used
to isolate RAM / CPU for each mongod
instance.
MongoDB Ops/Cloud Manager is key to
success!
MACHINE 1
APP1
MACHINE 2 MACHINE 3
Container “Striping”APP2APP3APP4APP5APP6
SECONDARY
17
Best Practices
• Business case
– Cost matching
– First customers first
• Balance scalability, standardization, and
flexibility
– Don’t undershoot your customers
– Don’t boil the ocean
– Customize where required
• Find your performance limit
– Storage first (mongoperf)
– Network
– CPU
– RAM
• MongoDB engineering
– Schema
– Shard first
– Shard key
• 2+ data centers
– Consider hybrid for 3rd
– If only 2, see goo.gl/qy6P7X
• MongoDB, Inc.
– Let us help!
• Orchestration and Monitoring
– Ops Manager
18
Who is using MongoDB-as-a-Service
US Department of Veterans Affairs provides a centralized data
store to manage veterans’ electronic records.
Goldman Sachs developed an enterprise-scale private cloud to
host applications based on MongoDB due to its agile, scalable and
resilient architecture.
Square Enix gaming platform consolidated their infrastructure on
MongoDB, improved performance and created new profit center.
Huawei, global telco equipment manufacturer, provides a scalable
MongoDB database as a service for internal applications.
Quantitative investment manager with over $11B in assets under
management invests heavily in MongoDB as a Service to gain
competitive advantages in the systematic trading space.
Ops Manager
20
The Best Way to Run MongoDB
Ops Manager allows you leverage and automate the best
practices we’ve learned from thousands of deployments in
a comprehensive application that helps you run MongoDB
safely and reliably.
Benefits include:
10x-20x more efficient operations
Complete performance visibility
Protection from data loss
Assisted performance optimization
21
How It Works
Ops
Manager
mongod mongodmongod
Agent Agent Agent
22
How MongoDB Ops Manager helps you
Scale EasilyMeet SLAs
Best Practices,
Automated
Cut
Management
Overhead
23
Integrates with Existing Infrastructure
24
Need a first app? Here you go.
• Enterprise social network
– Short messages
– Followers
– Feeds
– Geolocation
– https://github.com/mongodb-
labs/socialite
• Active users: 60% of employees
• Indefinite retention
• Java application
iStock licensed (Erikona)
25
Takeaways
• Database revolution
• Enterprise-level innovation with DBaaS
• Start small with positive results
• Build on your wins
• Let MongoDB Global Consulting help!
MongoDB-as-a-Service Whitepaper:
http://goo.gl/PYqDCx

Webinar: Enterprise Trends for Database-as-a-Service

  • 1.
    Enterprise Trends: Database asa Service Brandon.Newell@MongoDB.com Senior Solutions Architect MongoDB
  • 2.
    Agenda • Why Databaseas a Service? • DBaaS in my Enterprise architecture • Enterprise MongoDB-as-a-Service • Get there with Ops Manager
  • 3.
    3 Why Database asa Service? • Empower developers – Fast, easy, cheap • Go BIG – Seamlessly scale up for prod • Go SAFE – Consistent security, governance and operations iStock licensed (pixelfit)
  • 4.
  • 5.
    5 MongoDB and EnterpriseIT Stack EDW Hadoop Spark Management&Monitoring Security&Auditing RDBMS CRM, ERP, Collaboration, Mobile, BI OS & Virtualization, Compute, Storage, Network RDBMS Applications Infrastructure Data Management Online Data Offline Data
  • 6.
    6 MongoDB and EnterpriseIT Strategy Legacy Strategic Apps On-Premise SaaS, Mobile, Social Database Oracle Offline Data Teradata Hadoop, Spark Compute Scale-Up Server Commodity HW / Cloud Storage SAN Local Storage / Cloud Network Routers and Switches Software-Defined Networks
  • 7.
    7 Revolution in ITprovisioning • Hosting – Public, Private, and Hybrid • Stack – Software | Platform | Infrastructure …as a Service • DB platform advantages – Adoption – Agility – Governance – Efficiency Public PrivateHybrid Cloud Clients Web app, mobile app, thin client… Software: SaaS CRM, Email, virtual desktop, games… Platform: PaaS Execution runtime, DATABASE, web server, dev tools… Infrastructure: IaaS Virtual machines, containers, bare metal, storage, load balancers…
  • 8.
    8 Public Cloud • Commercialcloud IaaS endless aisle – Amazon Web Services – Microsoft Azure – Google Compute Engine – Rackspace – Many more… • OpenStack – Apache, Rackspace, NASA – OpenStack Foundation iStock licensed (4X-image)
  • 9.
    9 In the EnterpriseCloud: MongoDB as a Service • Rewards – Adoption – Agility, move faster – Governance – Efficiency, cost gains • Risks – Systematize the wrong solution – Standardize the wrong hardware (especially storage) – Unaffordable or inflexible: unlimited apathy – Too cheap: tragedy of the commons
  • 10.
    Bringing it alltogether with an Enterprise Service
  • 11.
    11 Customer First • FirstCustomer is your cheerleader • First app stakeholders – Business owner – Developers – Ops • Next few apps – Same stakeholders iStock licensed (YanC)
  • 12.
    12 Delivery Levels • Application •Data Service / Data Layer – US Department of Veterans Affairs: http://goo.gl/8usttw • Multi-tenancy – Multiple app databases per MongoDB cluster • Cluster per app – Replica set only – Sharded / replica sets
  • 13.
    13 Implementation Considerations • ServerHardware • Virtualization or Containers (Docker) • Security & Entitlements • Storage • Operating System • Infrastructure Management • Backup and Recovery • Accounting and chargeback • Distributed computing (Hadoop/Spark)
  • 14.
    14 MACHINE 1 Multiple Apps MultipleDiscrete Replicas Each app gets its own set of 3 machines. Physical or Virtual Machines PRIMARY APP1APP2APP3 MACHINE 2 SECONDARY MACHINE 3 SECONDARY MACHINE 4 PRIMARY MACHINE 5 SECONDARY MACHINE 6 SECONDARY MACHINE 7 PRIMARY MACHINE 8 SECONDARY MACHINE 9 SECONDARY Traditional Design
  • 15.
    15 Replica Set1 MACHINE 1 Multiple Apps MultipleShared Replicas Each database app still benefits from the 3 replicas Physical or Virtual Machines MongoDB Ops/Cloud Manager is key to success! PRIMARY Replica Set2 Replica Set3 MACHINE 2 SECONDARY MACHINE 3 SECONDARY MACHINE 4 PRIMARY MACHINE 5 SECONDARY MACHINE 6 SECONDARY MACHINE 7 PRIMARY MACHINE 8 SECONDARY MACHINE 9 SECONDARY Multi-Tenancy Design DB1 DB3 DB5 DB1 DB3 DB5 DB1 DB3 DB5 DB2 DB4 DB6 DB2 DB4 DB6 DB2 DB4 DB6 DB7 DB9 DB8 DB7 DB9 DB8 DB7 DB9 DB8
  • 16.
    16 SECONDARY cgroup/docker cgroup/docker cgroup/docker cgroup/docker cgroup/dockercgroup/docker cgroup/dockercgroup/docker cgroup/docker cgroup/dockercgroup/docker cgroup/docker cgroup/dockercgroup/docker cgroup/docker cgroup/dockercgroup/docker cgroup/docker PRIMARY SECONDARY SECONDARY PRIMARY SECONDARY PRIMARYSECONDARY SECONDARY PRIMARYSECONDARY SECONDARY PRIMARYSECONDARY SECONDARY PRIMARYSECONDARY SECONDARY Same concept as Traditional Design, except now we have more variation in the sizing and provisioning. Maintaining Primary and two Secondaries for HA per application. Linux cgroups or Docker Containers used to isolate RAM / CPU for each mongod instance. MongoDB Ops/Cloud Manager is key to success! MACHINE 1 APP1 MACHINE 2 MACHINE 3 Container “Striping”APP2APP3APP4APP5APP6 SECONDARY
  • 17.
    17 Best Practices • Businesscase – Cost matching – First customers first • Balance scalability, standardization, and flexibility – Don’t undershoot your customers – Don’t boil the ocean – Customize where required • Find your performance limit – Storage first (mongoperf) – Network – CPU – RAM • MongoDB engineering – Schema – Shard first – Shard key • 2+ data centers – Consider hybrid for 3rd – If only 2, see goo.gl/qy6P7X • MongoDB, Inc. – Let us help! • Orchestration and Monitoring – Ops Manager
  • 18.
    18 Who is usingMongoDB-as-a-Service US Department of Veterans Affairs provides a centralized data store to manage veterans’ electronic records. Goldman Sachs developed an enterprise-scale private cloud to host applications based on MongoDB due to its agile, scalable and resilient architecture. Square Enix gaming platform consolidated their infrastructure on MongoDB, improved performance and created new profit center. Huawei, global telco equipment manufacturer, provides a scalable MongoDB database as a service for internal applications. Quantitative investment manager with over $11B in assets under management invests heavily in MongoDB as a Service to gain competitive advantages in the systematic trading space.
  • 19.
  • 20.
    20 The Best Wayto Run MongoDB Ops Manager allows you leverage and automate the best practices we’ve learned from thousands of deployments in a comprehensive application that helps you run MongoDB safely and reliably. Benefits include: 10x-20x more efficient operations Complete performance visibility Protection from data loss Assisted performance optimization
  • 21.
    21 How It Works Ops Manager mongodmongodmongod Agent Agent Agent
  • 22.
    22 How MongoDB OpsManager helps you Scale EasilyMeet SLAs Best Practices, Automated Cut Management Overhead
  • 23.
  • 24.
    24 Need a firstapp? Here you go. • Enterprise social network – Short messages – Followers – Feeds – Geolocation – https://github.com/mongodb- labs/socialite • Active users: 60% of employees • Indefinite retention • Java application iStock licensed (Erikona)
  • 25.
    25 Takeaways • Database revolution •Enterprise-level innovation with DBaaS • Start small with positive results • Build on your wins • Let MongoDB Global Consulting help! MongoDB-as-a-Service Whitepaper: http://goo.gl/PYqDCx

Editor's Notes

  • #2  Dana will be my co-presenter today and will help me to flag your questions. Please use the chat window on the left of the screen. For completeness I will repeat the question along with the answer. I will attempt to answer your questions in line with the presentation as well as leave plenty of time at the end for Q&A. There will be an email with recorded webinar sent to you as well as a survey, please be sure to put the absolute highest marks. I understand that I will be awarded with a pretty pony if I’m rated the top presenter. There have been a number of exciting changes at MongoDB since our last DBaaS webinar in June 2015.
  • #3  Why should we use it? Where does it fit in my architecture? The various deployment methods for MongoDB-as-a-Service And the tools that MongoDB Inc have developed to make operating MongoDB as a Service easy Who else is using DBaaS and how.
  • #4 Empower Developers: Giving them the win of overcoming technical challenges with traditional relational databases, we allowed them to produce results in hours or days vs months or longer. There was a time when only software companies had developers, now every enterprise has development teams, sometimes each department has a dev team and the demands on their time and talent is at a premium. Empowering them to achieve the organizations goals is at the forefront of the drive to have resources available at the click of a button. Delay the development, you delay the project, which delays the organizations business goals which could ultimately cost the organization MONEY. Nobody gets promoted by delaying corporate initiatives and losing money. You may ask “DIDN’T we just go through this when we virtualized everything? I mean we can spin up a VM in minutes vs racking and stacking physical boxes?” Yes, sort of. You moved the bottleneck from infrastructure over to the systems and DBA teams now. Once those VM’s are provisioned, someone still needs to install the Database, tune and secure it and turn it over to the dev team. You can’t read a news feed these days without Amazon Web Services or Google Compute Engine or some other “as-a-Service” product being mentioned as a disruptive force. Why is that? They make it easy for the developers to get stuff done. Try new ideas that are not tethered to infrastructure that was built to maintain the status quo. Mr Developer, if you think you need 128 cores, 2 Terabytes of RAM and a petabyte of SSD storage…swipe your credit card and we’ll have that up for you in a few minutes. It’s another reason why large enterprises are seeing competition come out of the woodworks because the tools are there for developers to build that ONE feature that everyone wants that you can’t seem to get out the door. BIG Once these new ideas have been qualified by the business, we need to operationalize them. Get them out of the science experiment phase and into production. Again with the risk of delays, do you really want to delay moving something from dev to production that the business has qualified to save or generate money? SAFE In the process of operationalizing our new idea, no Enterprise is willing to overlook security and governance requirements. Not to mention, business continuance and disaster recovery. We will cover these in more detail.
  • #6 This is where MongoDB fits into the existing enterprise IT stack. MongoDB is an operational data store used for online data, in the same way that Oracle is an operational data store. It supports applications that ingest, store, manage and even analyze data in real-time. (Compared to Hadoop and data warehouses, which are used for offline, batch analytical workloads.) The technical details around how MongoDB does that, and in most cases faster and more scalable is outside the scope of this webinar. The thought is that if you’ve joined this webinar, you’re probably already considering MongoDB or already use it in production and being able to deploy MongoDB-as-a-Service is the next logical step for your organizations initiatives.
  • #7 MongoDB is aligned with strategic IT initiatives – like accelerating development of mobile apps, Taking advantage of commodity hardware, Taking advantage of cloud computing resources, and even interoperating with Hadoop. Enterprises are increasingly looking at moving away from Oracle and other proprietary systems to modern data stores like MongoDB to support new app development, and to migrate legacy applications. Performance, High Availability and ease of scale are at the cornerstone of why MongoDB is the preferred database for many enterprises data strategies. It only makes sense to make this incredible resource easier to provision, secure and consume in the form of a Database-as-a-Service.
  • #8 Hosting Public hosting becomes tricky the second you go outside the US. Data Sovereignty Laws are pretty restrictive and essentially keep companies in Europe from adopting a public cloud strategy. Based on some industry news feeds, some European companies will stretch this by adopting a Hybrid approach where there are no data bearing nodes in the Public cloud. I’ll leave that to the legal scholars and EU based companies to flesh out themselves. However, here in the US this has been a HUGE enabler for companies to get the resources they need without waiting. Even so, in the US there are financial data, [PHI] Protected Health Information and [PII] Personally Identifiable Information that companies just aren’t willing to risk keeping in public cloud. Let’s not forget enterprises who have invested heavily in their Datacenter strategy to have all the geographically separated, highly available and high hardware so they don’t have to rely on outside resources. Stack Consisting of 3 layers, Software as a services, Platform as a service and finally Infrastructure as a service.
  • #10 Risks:     SAN - Noisy Neighbor problem     Need the efficiency/guaranteed randomized IOPS that come with localized storage     Too cheap:  land rush and "Of course I need 1TB for my project" x # of projects
  • #12 Build a raving fan out of the first customer. Each additional customer will be easier to please. Line of Business Owner, Developers, Operations – this is your gateway to DevOps approach if you haven’t already adopted it.
  • #13 Software as a service Platform as a service Infrastructure as a service. US Department of VA - provides a centralized data store to manage veterans’ electronic records. Problem: 3 consumers of their data: The Veterans and their family, Clinicians and Benefits adjudicators Not all treatment happens in a VA hospital. Dentists, Pediatrics, ER, pharmacy Consumed in different ways and often interacting with different systems, the applications still had a common goal. Maintain consistent view of the veterans services across all platforms. Results: Succeeded in rolling out system in 9 months, meeting Congressionally mandated deadline Common access mechanism to exchange and store veteran electronic records One place to store and manage veteran electronic records for the lifetime of the agency
  • #14 Distributed Computing – Hadoop / Spark
  • #16 Multi-tenancy in this scenario, while graphically busy, consists of each of these replica sets (having Primary and Secondaries) hosting multiple databases with Access Controls to maintain separation among different applications. Managing a Multi-tenant scenario is easily achieved with our Ops/Cloud Manager product due to the ability to monitor each databases performance as well as the instance of MongoD that runs it. Software as a services, Platform as a service and finally Infrastructure as a service.
  • #17 We scatter the primaries using affinity rules to average out new primary elections in the face of machine failure.   Each machine MUST be strong enough to carry, if necessary, ALL primaries.   In fact, secondaries are doing JUST AS MUCH WORK as the primaries.     Also, cgroup and docker can control blockIO too, which is very important for a DB! Note: Orchestration via Ops Manager is key to success in dense or multi-tenant environments Affinity rules in your orchestration tool should keep primaries and secondaries for each replica set on separate physical machines.
  • #19 The top 3 reasons these customers tell us that they’re pursuing internal DBaaS are: Compress development cycles. When companies can build products and turn around new features in weeks instead of months or years, good things happen. Empower developers. The ability to stand up database instances quickly helps developers experiment, go big (or fail fast), and build prototypes uber-quickly. This is how you build things no one has built before. Help ops. By owning the process of spinning up new database instances, ops can maintain control of the environments; standardize and enforce governance; rationalize the stack; and templatize based on best practices.
  • #21 MongoDB Ops Manager can do a lot for [ops teams]. Ops Manager:  3 primary functions   Automated provisioning   Monitoring, alerting   Backup and Restore Best Practices, Automated. Ops Manager takes best practices for running MongoDB and automates them. So you run ops the way MongoDB engineers would do it. This not only makes it more fool-proof, but it also helps you… Cut Management Overhead. No custom scripting or special setup needed. You can spend less time running and managing manual tasks because Ops Manager takes care of a lot of the work for you, letting you focus on other tasks. Meet SLAs. Automating critical management tasks makes it easier to meet uptime SLAs. This includes managing failover as well as doing rolling upgrades with no downtime. Scale Easily. Provision new nodes and systems with a single click.
  • #22 Ops Manager agents are installed on servers (where MongoDB will be deployed), either through configuration tools such as Chef or Puppet, or by an administrator. The administrator creates a new design goal for the system, either as a modification to an existing deployment (e.g., upgrade, oplog resize, new shard), or as a new system. The agents periodically check in with the Ops Manager central server and receive the new design instructions. Agents create and follow a plan for implementing the design. Using a sophisticated rules engine, agents continuously adjust their individual plans as conditions change. In the face of many failure scenarios – such as server failures and network partitions – agents will revise their plans to reach a safe state. Minutes later, the system is deployed – safely and reliably.
  • #23 OpsManager can do a lot for [ops teams]. Best Practices, Automated. Ops Manager takes best practices for running MongoDB and automates them. So you run ops the way MongoDB engineers would do it. This not only makes it more fool-proof, but it also helps you… Cut Management Overhead. No custom scripting or special setup needed. You can spend less time running and managing manual tasks because MMS takes care of a lot of the work for you, letting you focus on other tasks. Meet SLAs. Automating critical management tasks makes it easier to meet uptime SLAs. This includes managing failover as well as doing rolling upgrades with no downtime. Scale Easily. Provision new nodes and systems with a single click.
  • #24 Administrators can use the Ops Manager interface directly, or invoke the Ops Manager RESTful API from existing enterprise tools, including popular monitoring and orchestration frameworks.
  • #26 Survey