MongoDB
Pain Relief
Michael Lynn
Sr. Solutions Architect, MongoDB
@mlynn, @phillyMUG
michael.lynn@mongodb.com
OpsManager
OpsManager
Relieving the pain associated with manually
managing your large-scale MongoDB estate.
Goals
• Ops Manager, Cloud Manager, Atlas
– What are they?
– How do they differ?
– Why do I need them?
• Ansible
– What is it and how can I use these tools to relieve some
pain in my life?
Show of
Hands
• Who’s Here?
– DevOps/Ops?
– Site Reliability?
– Engineer?
– DBA?
– Developer?
Admin
Evolution
3. Puppet, Salt,
Ansible
Leveraging automated solution for
config mgt, some cloud deployment.
1. Manual
Download by hand, build server from
CD, Manually Start and Config Services.
2. Scripting
Write scripts to maintain
configurations, copy those to
servers and run them.
4. Cloud/Hybrid
Automation
Full Cloud-based pro-
visioning and
management
Managing databases manually can be...
• Painful
• Risky
• Complex
• Susceptible to
human error*
• Unnecessary...
* 80% of all downtime due to people/process error (Gartner)
What’s involved in managing MongoDB?
● Provision / Deploy
● Install/Upgrade Binaries
● Configure Database
● Administer Database
Deploying 42
42
Upgrading 35
70
Maintaining 23
100
Play Top 3 Things DBA’s Hate to Do Manually?
So… what does it take (at scale)?
It can take a lot of manual effort to care for a MongoDB
system in production
● Install + Configure
○ 150+ steps
● Scale out, move servers, resize oplog, etc
○ 10 - 180+ steps
● Upgrades, downgrades
○ 100+ stepsUpgrade
Deploy
Maintain
Up to 95%
Reduction in
Operational
Overhead
• Single-click provisioning, scaling &
upgrades, admin tasks!
• Monitoring, with charts, dashboards
and alerts on 100+ metrics
• Backup and restore, with
point-in-time recovery, support for
sharded clusters
MongoDB Ops Manager
The Best Way to Manage MongoDB In Your Data Center
Up to 95% Reduction in Operational Overhead
• Single-click provisioning, scaling &
upgrades, admin tasks!
• Monitoring, with charts, dashboards
and alerts on 100+ metrics
• Backup and restore, with
point-in-time recovery, support for
sharded clusters
MongoDB Cloud Manager
The Best Way to Manage MongoDB In The Cloud
Up to 95% Reduction in Operational Overhead
• Scalable back-end for your
application on-demand
• Secure by default
• Highly available, even while scaling
• Patch maintenance performed for
you
• Your own MongoDB cluster in the
cloud
MongoDB Atlas
MongoDB’s Database as a Service
OpsManager Demo
http://ec2-54-162-176-239.compute-1.amazonaws.com:8080
This finds and
leverages servers
where you’ve already
installed the
automation agent
This provisions hosts
AND deploys the
automation agents.
OpsManager Demo
1
2
Group/RBAC
Logical/Physical
3 Replica Set Members
3 MongoDB Version
1Add Replica Set
1Repset Name
2Version of MongoDB
3Server Pool Hostnames
4Port to Run MongoDB
1Member Roles
2DB Path on Server
3Advanced Options Settings
4CLICK APPLY
Activity
Deploy Server Instance
Upgrade (Done For You)
Configure
Administer Database
Maintain OS - Mongod
Maintain OS - OpsMgr
Activity
Deploy Server Instance
Upgrade MongoDB
Configure MongoDB
Administer Database
Maintain OS - Mongod
Maintain OS - OpsMgr
Activity
Deploy Server Instance
Upgrade MongoDB
Configure MongoDB
Administer Database
Maintain OS - Mongod
Maintain OS - OpsMgr
Ops
Manager
Cloud
Manager Atlas
CONTROL CONVENIENCE
MongoDB Enterprise Server
MongoDB Enterprise Advanced24x7Support
CommercialLicense
Platform
Certifications
Emergency
Patches
On-Demand
Online Training
Customer
Success
Program
MongoDB CompassMongoDB Ops Manager
Monitoring &
Alerting
Query
Optimization
Backup &
Recovery
Automation &
Configuration
Schema
Visualization
Data Exploration
Ad-Hoc Queries
MongoDB Connector for
BI
Visualization
Analysis
Reporting
LDAP & Kerberos Auditing FIPS 140-2Encryption at Rest
REST API
MongoDB Deployment - Ops Manager
Configure Host
Provision Host
Configure
Agent
1. AWS 2. New Host 3. Ops Manager
Install OM
Agent
Restart Agent
Install
MongoDB
Configure
Monitoring
Configure
Backups
Configure Host
Provision Host
Configure
Agent
Install OM
Agent
Restart Agent
Install
MongoDB
Configure
Monitoring
Configure
Backups
2. New Host 3. Ops Manager1. AWS
More?
More?
http://ec2-54-162-176-239.compute-1.amazonaws.com:8080
MongoDB Deployment - Cloud Manager & Atlas
Configure Host
Provision Host
Configure
Agent
Install OM
Agent
Restart Agent
Provision New
Hosts
Configure
Monitoring
Configure
Backups
AWS New Host Ops Manager
Provision New
Hosts
Configure
Monitoring
Configure
Backups
1. Cloud / Atlas
This provisions hosts
AND deploys the
automation agents.
MongoDB Deployment - Ops Manager
Configure Host
Provision Host
Configure
Agent
1. AWS 2. New Host 3. Ops Manager
Install OW
Agent
Restart Agent
Install
MongoDB
Configure
Monitoring
Configure
Backups
Configure Host
Provision Host
Configure
Agent
Install OW
Agent
Restart Agent
Install
MongoDB
Configure
Monitoring
Configure
Backups
2. New Host 3. Ops Manager1. AWS
Candidate
for
Automation?
26
WHAT IS ANSIBLE?
Confidential
It’s a simple automation language that
can perfectly describe an IT application
infrastructure in Ansible Playbooks.
It’s an automation engine that runs
Ansible Playbooks.
Ansible Tower is an enterprise
framework for controlling, securing
and managing your Ansible
automation with a UI and RESTful API.
27
SIMPLE POWERFUL AGENTLESS
App deployment
Configuration management
Workflow orchestration
Orchestrate the app lifecycle
Human readable automation
No special coding skills needed
Tasks executed in order
Get productive quickly
Agentless architecture
Uses OpenSSH & WinRM
No agents to exploit or update
More efficient & more secure
Confidential
28
ANSIBLE TOWER
CONTROL
SIMPLE POWERFUL AGENTLESS
KNOWLEDGE DELEGATION
TOWER EMPOWERS TEAMS TO AUTOMATE
AT ANSIBLE’S CORE IS AN OPEN-SOURCE AUTOMATION ENGINE
Scheduled and
centralized jobs
Visibility and
compliance
Role-based access
and self-service
Everyone speaks the
same language
Designed for
multi-tier deployments
Predictable, reliable,
and secure
Confidential
Getting Started with Ansible - Lexicon
• Commands
– ansible, ansible-playbook
• Configuration
– ansible.cfg
• Modules
– Hundreds of stand-alone scripts to solve common requirements
• Plays, Playbooks
– The execution tools to carry out your management tasks
• Tasks
– The components that connect ansible to the servers
Why Ansible?
• Simple
• Human Readable
• Agentless (uses SSH)
• Popular - Lots of Modules
• Used by Twitter, NASA, Spotify
Demo of Ansible
Follow Along
• Git (or download)
– git clone http://github.com/mrlynn/ansible-demo
– Thank you Torsten Spindler
• Ansible
– brew install ansible
• Pipe Viewer (optional)
– Simulates typing
• AWS->EC2
– Keys Set Up
Post-Demo Check-in
• Atlas
– DBaaS
– Deploys new MongoDB Clusters from zero to cluster in minutes.
• Ops Manager
– Automation for all the things you do with MongoDB - except deploying the agent.
• Ansible
– Automation for configs, deployment and more - especially deploying O/M Agents
I hope you learned some ways you can relieve the pain associated with manually
configuring, deploying and maintaining your database environment.
Questions?
Get These Scripts and Slides:
http://github.com/mrlynn/ansible-demo
1 - Ansible Configuration Files
2 - Ansible Hosts File
3 - Ansible Modules - Ping
4 - Ansible Environment Variables
5 - Ansible Modules - Script
6 - Ansible Playbooks
7 - Deploying O/M in Playbooks
BONUS
SLIDES!
• Slides probably
not used in the
presentation.
Replica Set Architecture (w/ Ops Manager)
43
APP SERVER
APPLICATION
DRIVER
OPS MANAGER
OM APP
APP BACKUP
REPLICA SET
PRIMARY
SECONDARY SECONDARY
OM
AGENT
OM
AGENT
OM
AGENT
Demonstration Setup
Command Description
ansible -m ping -i ansible-hosts all Using the ping module and the hosts file
ansible-hosts in the current directory,
ping all hosts
export ANSIBLE_HOSTS=./ansible_hosts Save some time - ansible uses
environment variables extensivel
ansible -m ping all Same as before - but now ansible
leverages the env var to find the ansible
hosts file.
Demonstration
Command Description
ansible -m script test.sh all Here, we take a local script and execute
it across all of our hosts. Ansible takes
that script and delivers it via scp to the
hosts, executes it and captures the
output.
export ANSIBLE_HOSTS=./ansible_hosts Save some time - ansible uses
environment variables extensivel
ansible -m ping all Same as before - but now ansible
leverages the env var to find the ansible
hosts file.
Appendix A - /etc/ansible/hosts sample
[opsManager]
ec2-54-93-114-205.eu-central-1.compute.amazonaws.com ansible_user=ec2-user
[amlReplicaSet]
ec2-54-93-79-122.eu-central-1.compute.amazonaws.com ansible_user=ec2-user
ec2-54-93-176-246.eu-central-1.compute.amazonaws.com ansible_user=ec2-user
ec2-54-93-207-148.eu-central-1.compute.amazonaws.com ansible_user=ec2-user
[amlReplicaSet:vars]
opsmanagerurl=http://ec2-54-93-114-205.eu-central-1.compute.amazonaws.com:8080
opsmanager=ec2-54-93-114-205.eu-central-1.compute.amazonaws.com
Appendix B - ansible.cfg
[defaults]
host_key_checking = False
private_key_file = PATH/TO/AWS/KEY.FILE
[ssh_connection]
control_path = %(directory)s/%%C

Ansible, MongoDB Ops Manager and AWS v1.1

  • 1.
    MongoDB Pain Relief Michael Lynn Sr.Solutions Architect, MongoDB @mlynn, @phillyMUG michael.lynn@mongodb.com OpsManager OpsManager Relieving the pain associated with manually managing your large-scale MongoDB estate.
  • 2.
    Goals • Ops Manager,Cloud Manager, Atlas – What are they? – How do they differ? – Why do I need them? • Ansible – What is it and how can I use these tools to relieve some pain in my life?
  • 3.
    Show of Hands • Who’sHere? – DevOps/Ops? – Site Reliability? – Engineer? – DBA? – Developer?
  • 4.
    Admin Evolution 3. Puppet, Salt, Ansible Leveragingautomated solution for config mgt, some cloud deployment. 1. Manual Download by hand, build server from CD, Manually Start and Config Services. 2. Scripting Write scripts to maintain configurations, copy those to servers and run them. 4. Cloud/Hybrid Automation Full Cloud-based pro- visioning and management
  • 5.
    Managing databases manuallycan be... • Painful • Risky • Complex • Susceptible to human error* • Unnecessary... * 80% of all downtime due to people/process error (Gartner)
  • 6.
    What’s involved inmanaging MongoDB? ● Provision / Deploy ● Install/Upgrade Binaries ● Configure Database ● Administer Database Deploying 42 42 Upgrading 35 70 Maintaining 23 100 Play Top 3 Things DBA’s Hate to Do Manually?
  • 7.
    So… what doesit take (at scale)? It can take a lot of manual effort to care for a MongoDB system in production ● Install + Configure ○ 150+ steps ● Scale out, move servers, resize oplog, etc ○ 10 - 180+ steps ● Upgrades, downgrades ○ 100+ stepsUpgrade Deploy Maintain Up to 95% Reduction in Operational Overhead
  • 8.
    • Single-click provisioning,scaling & upgrades, admin tasks! • Monitoring, with charts, dashboards and alerts on 100+ metrics • Backup and restore, with point-in-time recovery, support for sharded clusters MongoDB Ops Manager The Best Way to Manage MongoDB In Your Data Center Up to 95% Reduction in Operational Overhead
  • 9.
    • Single-click provisioning,scaling & upgrades, admin tasks! • Monitoring, with charts, dashboards and alerts on 100+ metrics • Backup and restore, with point-in-time recovery, support for sharded clusters MongoDB Cloud Manager The Best Way to Manage MongoDB In The Cloud Up to 95% Reduction in Operational Overhead
  • 10.
    • Scalable back-endfor your application on-demand • Secure by default • Highly available, even while scaling • Patch maintenance performed for you • Your own MongoDB cluster in the cloud MongoDB Atlas MongoDB’s Database as a Service
  • 11.
    OpsManager Demo http://ec2-54-162-176-239.compute-1.amazonaws.com:8080 This findsand leverages servers where you’ve already installed the automation agent
  • 12.
    This provisions hosts ANDdeploys the automation agents.
  • 13.
  • 14.
  • 15.
  • 16.
    1Repset Name 2Version ofMongoDB 3Server Pool Hostnames 4Port to Run MongoDB
  • 17.
    1Member Roles 2DB Pathon Server 3Advanced Options Settings 4CLICK APPLY
  • 18.
    Activity Deploy Server Instance Upgrade(Done For You) Configure Administer Database Maintain OS - Mongod Maintain OS - OpsMgr Activity Deploy Server Instance Upgrade MongoDB Configure MongoDB Administer Database Maintain OS - Mongod Maintain OS - OpsMgr Activity Deploy Server Instance Upgrade MongoDB Configure MongoDB Administer Database Maintain OS - Mongod Maintain OS - OpsMgr Ops Manager Cloud Manager Atlas CONTROL CONVENIENCE
  • 19.
    MongoDB Enterprise Server MongoDBEnterprise Advanced24x7Support CommercialLicense Platform Certifications Emergency Patches On-Demand Online Training Customer Success Program MongoDB CompassMongoDB Ops Manager Monitoring & Alerting Query Optimization Backup & Recovery Automation & Configuration Schema Visualization Data Exploration Ad-Hoc Queries MongoDB Connector for BI Visualization Analysis Reporting LDAP & Kerberos Auditing FIPS 140-2Encryption at Rest REST API
  • 20.
    MongoDB Deployment -Ops Manager Configure Host Provision Host Configure Agent 1. AWS 2. New Host 3. Ops Manager Install OM Agent Restart Agent Install MongoDB Configure Monitoring Configure Backups Configure Host Provision Host Configure Agent Install OM Agent Restart Agent Install MongoDB Configure Monitoring Configure Backups 2. New Host 3. Ops Manager1. AWS More? More?
  • 21.
  • 22.
    MongoDB Deployment -Cloud Manager & Atlas Configure Host Provision Host Configure Agent Install OM Agent Restart Agent Provision New Hosts Configure Monitoring Configure Backups AWS New Host Ops Manager Provision New Hosts Configure Monitoring Configure Backups 1. Cloud / Atlas
  • 23.
    This provisions hosts ANDdeploys the automation agents.
  • 24.
    MongoDB Deployment -Ops Manager Configure Host Provision Host Configure Agent 1. AWS 2. New Host 3. Ops Manager Install OW Agent Restart Agent Install MongoDB Configure Monitoring Configure Backups Configure Host Provision Host Configure Agent Install OW Agent Restart Agent Install MongoDB Configure Monitoring Configure Backups 2. New Host 3. Ops Manager1. AWS Candidate for Automation?
  • 26.
    26 WHAT IS ANSIBLE? Confidential It’sa simple automation language that can perfectly describe an IT application infrastructure in Ansible Playbooks. It’s an automation engine that runs Ansible Playbooks. Ansible Tower is an enterprise framework for controlling, securing and managing your Ansible automation with a UI and RESTful API.
  • 27.
    27 SIMPLE POWERFUL AGENTLESS Appdeployment Configuration management Workflow orchestration Orchestrate the app lifecycle Human readable automation No special coding skills needed Tasks executed in order Get productive quickly Agentless architecture Uses OpenSSH & WinRM No agents to exploit or update More efficient & more secure Confidential
  • 28.
    28 ANSIBLE TOWER CONTROL SIMPLE POWERFULAGENTLESS KNOWLEDGE DELEGATION TOWER EMPOWERS TEAMS TO AUTOMATE AT ANSIBLE’S CORE IS AN OPEN-SOURCE AUTOMATION ENGINE Scheduled and centralized jobs Visibility and compliance Role-based access and self-service Everyone speaks the same language Designed for multi-tier deployments Predictable, reliable, and secure Confidential
  • 29.
    Getting Started withAnsible - Lexicon • Commands – ansible, ansible-playbook • Configuration – ansible.cfg • Modules – Hundreds of stand-alone scripts to solve common requirements • Plays, Playbooks – The execution tools to carry out your management tasks • Tasks – The components that connect ansible to the servers
  • 30.
    Why Ansible? • Simple •Human Readable • Agentless (uses SSH) • Popular - Lots of Modules • Used by Twitter, NASA, Spotify
  • 31.
  • 32.
    Follow Along • Git(or download) – git clone http://github.com/mrlynn/ansible-demo – Thank you Torsten Spindler • Ansible – brew install ansible • Pipe Viewer (optional) – Simulates typing • AWS->EC2 – Keys Set Up
  • 33.
    Post-Demo Check-in • Atlas –DBaaS – Deploys new MongoDB Clusters from zero to cluster in minutes. • Ops Manager – Automation for all the things you do with MongoDB - except deploying the agent. • Ansible – Automation for configs, deployment and more - especially deploying O/M Agents I hope you learned some ways you can relieve the pain associated with manually configuring, deploying and maintaining your database environment.
  • 34.
    Questions? Get These Scriptsand Slides: http://github.com/mrlynn/ansible-demo
  • 35.
    1 - AnsibleConfiguration Files
  • 36.
    2 - AnsibleHosts File
  • 37.
    3 - AnsibleModules - Ping
  • 38.
    4 - AnsibleEnvironment Variables
  • 39.
    5 - AnsibleModules - Script
  • 40.
    6 - AnsiblePlaybooks
  • 41.
    7 - DeployingO/M in Playbooks
  • 42.
    BONUS SLIDES! • Slides probably notused in the presentation.
  • 43.
    Replica Set Architecture(w/ Ops Manager) 43 APP SERVER APPLICATION DRIVER OPS MANAGER OM APP APP BACKUP REPLICA SET PRIMARY SECONDARY SECONDARY OM AGENT OM AGENT OM AGENT
  • 44.
    Demonstration Setup Command Description ansible-m ping -i ansible-hosts all Using the ping module and the hosts file ansible-hosts in the current directory, ping all hosts export ANSIBLE_HOSTS=./ansible_hosts Save some time - ansible uses environment variables extensivel ansible -m ping all Same as before - but now ansible leverages the env var to find the ansible hosts file.
  • 45.
    Demonstration Command Description ansible -mscript test.sh all Here, we take a local script and execute it across all of our hosts. Ansible takes that script and delivers it via scp to the hosts, executes it and captures the output. export ANSIBLE_HOSTS=./ansible_hosts Save some time - ansible uses environment variables extensivel ansible -m ping all Same as before - but now ansible leverages the env var to find the ansible hosts file.
  • 46.
    Appendix A -/etc/ansible/hosts sample [opsManager] ec2-54-93-114-205.eu-central-1.compute.amazonaws.com ansible_user=ec2-user [amlReplicaSet] ec2-54-93-79-122.eu-central-1.compute.amazonaws.com ansible_user=ec2-user ec2-54-93-176-246.eu-central-1.compute.amazonaws.com ansible_user=ec2-user ec2-54-93-207-148.eu-central-1.compute.amazonaws.com ansible_user=ec2-user [amlReplicaSet:vars] opsmanagerurl=http://ec2-54-93-114-205.eu-central-1.compute.amazonaws.com:8080 opsmanager=ec2-54-93-114-205.eu-central-1.compute.amazonaws.com
  • 47.
    Appendix B -ansible.cfg [defaults] host_key_checking = False private_key_file = PATH/TO/AWS/KEY.FILE [ssh_connection] control_path = %(directory)s/%%C