Anthem is one of the largest healthcare companies in the USA. Transforming such a large enterprise and enabling true agile development depends on being able to get resources fast when you need them. How hard can it be to get a single server set up? The answer is - it can take weeks or even months! In this presentation, we will talk about how we shortened the delivery of MongoDB deployments to our developers to minutes using MongoDB OpsManager and, of course, some magic. We will cover the details of the approach and the challenges we had to face so that you can do it too.

1. MongoDB server provisioning:
From 2 months… to 2 minutes
Chetan Gavankar, DB Executive Advisor@ Anthem
September 2018

2. 22
Topics to Cover
• MongoDB in Anthem
• Demand for quick deployment
• Road Map to achieve our goal
• How we did it
• Lesson learned
• Future Ideas
• Q/A

3. MongoDB in Anthem
• Anthem as an organization
– 2nd largest HealthCare company
– Lots of development using SQL and NoSQL
• Why MongoDB
a) Flexible schema design
b) High availability
• From few serves to 200 servers in short time
3

4. 4
Demand for quick deployment
Current Process (1-2 months)
Mongodb
Server
App team
request
server Work in
Take
Process to
acquire
HW
Request to
create all
IDs
Verify HW
configurati
on
OS setup
and
verificatio
n
Firewall
/Port
setup
Centrify
setup
Install
MongoDB
Setup Ops
manager
DBA checklist

5. 5
Demand for quick deployment
Proposed Process (2 minutes)
Mongodb
Server
App team
request server
MongoDB
server
delivered from
Ops manager
server pool
Work in Take
Process
acquired new
servers
Servers are
added back to
server pool

6. 6
What’s the advantage of Automation
• Faster delivery of servers to customers i.e. happy customers
• Consistent setup when delivering these servers
• DBA efficiency
Time and effort saved to build these servers can be utilized to work closely with application
team on design review , performance testing and more interesting things for DBA to work on
.

7. 7
How We did it
• Automation of Server Build process
• Ops Manager’s Server Pool feature
• Automation Software using Ops Manager API and Python

8. 8
Automation of Server Build process
• Standard offering
– We developed SPO (server image) which allows to build servers with specific T-shirt sizes i.e.
predefined CPU/MEMORY/STORAGE (mount points)/OS
• Scripted OS configuration
– The script performs OS setup as recommended in MongoDB Production notes
https://docs.mongodb.com/manual/administration/production-notes
• No more individual firewall rules!
– We created a Firewall group for the given VLAN so required ports are open to allow
communications between DB servers and Ops Managers and within DB server itself
• Minimizing human intervention
– By the time servers are delivered to DBA team only manual process is to install ops manager
automation agent and edit the server pool configuration file (this also we will automate
soon)

9. 9
Ops Manager Server Pool Feature

10. 10
Ops Manager Server Pool Feature
Custom user defined properties
(stored in the pool properties file on each host)

11. 11
Automation Software
• Ops Manager REST API
– Gives incredible flexibility in building scripts/software to manage MongoDB deployments
• Python
– Easy to learn programming language
– OOO development
– Package to handle REST API calls

12. 12
Single Command to Deploy
A single command will create new Group in Ops manager , bind servers to new group depending on the
deployment topology and characteristics selected , install MongoDB version selected , setup monitoring ,
setup alerts , setup backup and put DB servers under Ops manager Automation …and bring you coffee and
fresh milk
python maintenance/creategroupanddeploy.py
--opsmgrUri http://***.**.com:80 --opsmgrapiuser <email user id> --opsmgrapikey ***
--organizationId 5af1482f99c8472f18541146
--groupName "FAMRTEST01"
--environments DEV
--deploymentTopology 'single-node'
--tShirtSize small --chipset intel --location dc1
--loglevel debug
--mongoDbVersion "3.6.5“
--deploymentName "farmDV01"
--disableAlerts --disableBackup

13. 13
Originally just a small single script
Automation SW Architecture Diagram

14. 14
Ops manager REST API - Example

15. 15
Calling Ops Manager API from Python - Example
import requests
Connector = OpsMgrConnector(args.opsMgrUri, args.opsMgrApiUser, args.opsMgrApiKey)
return creategroup(args)
_________________________________________________________________________________
def creategroup(args):
return addGroup(args.opsMgrUri, args.groupName, environmentType, organizationId)
def addGroup(opsMgrUri, groupName, groupType, organizationId):
addGroupResultJson = opsMgrConnector.addGroup(newGroupName, organizationId)
def addGroup(self, newGroupName, organizationId):
payload = { "name" : newGroupName, "orgId" : organizationId }
return self.post("{}/groups".format(self.apiURL), payload)
def post(self, url, payload):
result = requests.post(url, json=payload, auth=self.auth).json()
return result

16. 16
MongoDB Server Deployed in Ops manager

17. 17
Lesson Learned
• DNS Caching
– Nontrivial issues when DNS queries are taking long time
• Lots of testing
– Different test cases
– Different HW such as Intel , Power , Private Cloud, etc.
• Non-technical
– Project planning and roadmap is a must
– Everyone involved should understand project goals and individual responsibilities

18. 18
Future Ideas
• More automation
– Alert templates
• Explore integration with other automation/orchestration tools
– Chef, Puppet, Ansible
– Kubernetes

19. 19
Yes it is possible , we did it and All Clicked
Faster delivery of servers to customers i.e. happy customers
 Consistent setup when delivering these servers
 Time saved to build these servers can be utilized to work closely with application team
on design review , performance testing and more interesting things for DBA to work on .
Automation is flexible enough to be deployed on other platforms

20. 20
Q/A
Chetan Gavankar, 2018