Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MongoDB World 2018: Supercharge Your MongoDB Deployment with Ops Manager Automation

382 views

Published on

Speaker: Arkadiusz Borucki, Senior Service Reliability Engineer (MongoDB/Automation), Amadeus Data Processing GmbH

Published in: Technology
  • I like this service ⇒ www.HelpWriting.net ⇐ from Academic Writers. I don't have enough time write it by myself.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • To get professional research papers you must go for experts like ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • A professional Paper writing services can alleviate your stress in writing a successful paper and take the pressure off you to hand it in on time. Check out, please ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

MongoDB World 2018: Supercharge Your MongoDB Deployment with Ops Manager Automation

  1. 1. RESTRICTED Confidential ©AmadeusITGroupanditsaffiliatesandsubsidiaries RESTRICTED Confidential Amadeus Supercharge your MongoDB Deployment with Ops Manager Automation Arkadiusz Borucki, Senior Service Reliability Engineer
  2. 2. RESTRICTED Confidential ©AmadeusITGroupanditsaffiliatesandsubsidiaries _ We are the leading technology company dedicated to the global travel industry _ We are present in 190+ countries and employ 15,000+ people worldwide _ Our solutions enrich travel for billions of people every year _ We work together with our customers, partners and other players in the industry to improve business performance and shape the future of travel 630+ million total bookings processed in 2017 using the Amadeus distribution platform 1.6 billion passengers boarded in 2017 with Amadeus and Navitaire solutions 1 of the world’s top 15 software companies Forbes 2017 global rankings Amadeus in a few words 2 6th consecutive year included in the DJSI Recognised as world leader in the Software & Services industry sector in the Dow Jones Sustainability Index in 2017
  3. 3. RESTRICTED Confidential Providing local expertise, all over the world ©AmadeusITGroupanditsaffiliatesandsubsidiaries We work in 190+ markets 3
  4. 4. { NYC : MongoDB at Amadeus } ©AmadeusITGroupanditsaffiliatesandsubsidiaries 4 Supercharge your MongoDB Deployment with Ops Manager Automation
  5. 5. RESTRICTED Confidential RESTRICTED Confidential { Amadeus : MongoDB farm overview } 4 Ops Managers versions : - 3.6.*. MongoDB Clusters split between 4 Ops Managers. „Master” Ops Manager. 100 MongoDB clusters: size (100GB – 107TB): - 574 nodes, 3000 mongod. Micro sharding (even 120 shards per cluster). Automation: Jenkins CI/CD, Ansible, REST API, Python scripts. Security: SSL/TLS, Kerberos, LDAP, CA, Auditing, Encryption at Rest, QRadar. 5 ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  6. 6. RESTRICTED Confidential 6 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : MongoDB Architecture 2018 } 574 MongoDB nodes splitted between OM
  7. 7. { Amadeus : MongoDB use cases } Instant Search - flight recommendations data. - hundred billion of recommendations MongoDB for real-time analytics – 124 shards (micro-sharding). Apache Spark on MongoDB – fraud detection projects. MongoDB Amadeus Payments. MongoDB for Airlines - configuration, status, history. Next TI – travel intelligence, BI for airline decision-making. Dashboards: scheduling tools, cheange viewers, applications viewers, and more. 7 ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  8. 8. { Amadeus : SSP Instant Search } Search billions of recommendations in milliseconds! Speed! - search time is in average 2 milliseconds! In-house implementation of NoSQL. SSP - 3 MongoDB clusters – 200TB of data. always online! 8 ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  9. 9. { Amadeus : the biggest cluster in Europe ? } 9 ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  10. 10. { Amadeus : How to operate at huge scale ? } 10 ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  11. 11. 11 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : Automation } Infrastructure as Code - GIT. CI/CD with Jenkins - orchestrate the deployment processes. Ops Manager REST API. Ansible module for Ops Manager / MongoDB. Ansible playbooks / roles for MongoDB. Framework for online database upgrade. Framework for online rolling kernel upgrade. MongoDB as a Service.
  12. 12. 12 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { MongoDB : No Downtime }
  13. 13. { Amadeus : Infrastructure as a code } ©AmadeusITGroupanditsaffiliatesandsubsidiaries 13 Bitbucket - branches Reduce overhead, reduce human mistakes. speed-up processes, provide consistency.
  14. 14. { Ops Manager : REST API } ©AmadeusITGroupanditsaffiliatesandsubsidiaries 14 Ops Manager follows REST architectural style and provides internal resources which enable programmatic access to Ops Manager features. JSON entities, Digest authentication, Browsable interface WEB UI
  15. 15. RESTRICTED Confidential 15 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : OpsManager class } class OpsManager(object): def __init__(self, mmsurl, user, key, verify): self.mmsurl = mmsurl self.user = user self.key = key self ._session = requests.Session() self ._session.verify = verify self ._session.auth = requests.auth.HTTPDigestAuth(self.user, self.key)
  16. 16. RESTRICTED Confidential 16 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : OpsManager class } def post_maintenance_window(self, group, data): return self._post_json(data, *self._maintenance_url(group)) def upgrade_mongodb(self, group, ver): j = self.get_automation_config(group) j = self.enable_version(j, ver) j = self.compatibility_version(group, j, ver) for p in j.get('processes'): p['version'] = _enterprise_edition(ver) self._put_json(j, *self._automation_config_url(group)) self.deploy_change(group)
  17. 17. RESTRICTED Confidential 17 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : OpsManager class } def check_cluster_health(self, group): if not self.cluster_goal_status(group): raise OpsManInvalidState('Operation on cluster, try later') self.mongodb_alerts(group) self.check_sync(group) def shutdown_db(self, group, host, option): j = self.get_automation_config(group) for p in j.get('processes'): if p.get('hostname') == host and p.get('process Type') != 'mongos': p['disabled'] = option self.put_automation_config(group, j)
  18. 18. RESTRICTED Confidential 18 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : OpsManager methods }
  19. 19. RESTRICTED Confidential 19 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : OpsManager class } def delete_maintenance_window(self, group, idnum): return self._delete_json(*self._maintenance_url(group, idnum)) def cluster_goal_status(self, cluster): j = self._get('/api/public/v1.0/groups', cluster, 'automation Status') goal = j['goal Version'] return all([ goal == i.get('lastGoalVersionAchieved') for i in j.get('processes')]) def get_group_by_name(self, group): '''get group from Ops Manager via REST API call''' return self._get('/api/public/v1.0/groups/by Name', group).get('id')
  20. 20. RESTRICTED Confidential 20 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : OpsManager class }
  21. 21. RESTRICTED Confidential 21 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : Ansible OpsManager modules } mongodb start / stop / sync modules from ansible.module_utils.opsmanager import ansible_setup if __name__ == '__main__': module, opsmanager = ansible_setup() group = opsmanager.get_group_by_name(module.params['cluster']) response = opsmanager.up_db(group, module.params['host']) module.exit_json(changed=False, meta=response)
  22. 22. RESTRICTED Confidential 22 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : open-source } Python class for MongoDB Ops Manager and Ansible module https://github.com/AmadeusITGroup/opsmancombo The official global GitHub account of Amadeus IT Group. Amadeus provides the technology that keeps the travel sector moving.
  23. 23. RESTRICTED Confidential 23 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : Ansible-Galaxy } Installation : $ ansible-galaxy install AmadeusITGroup.opsmancombo Tags : automation, database, DevOps, MongoDB, NoSQL, Ops Manager https://galaxy.ansible.com/AmadeusITGroup/opsmancombo/
  24. 24. RESTRICTED Confidential 24 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : Jenkins } Jenkins is an open-source continuous integration (CI) tool that helps orchestrate the development processes (build, test, and deployment) with automation. Jenkins is one of the leading tools that would help a DevOps team orchestrate its processes.
  25. 25. RESTRICTED Confidential 25 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : Jenkins Pipeline } A continuous delivery pipeline is an automated expression of your process for getting software from version control right through to your deployment. Setting up a Pipeline project means writing a script that will sequentially apply some steps of the process we want to accomplish.
  26. 26. RESTRICTED Confidential 26 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : Jenkins Pipeline }
  27. 27. RESTRICTED Confidential 27 ©AmadeusITGroupanditsaffiliatesandsubsidiaries { Amadeus : projects we working on } • 200TB MongoDB cluster! – data repository at huge scale! • MongoDB on OpenShift (Microservices, Ops Manager operator). • Ops Manager and S3 Object Store. • Migraton from RDBMS to MongoDB! • Amadeus and MongoDB technical blog! • More automation and more OpenSource! Watch our webinar! stay tuned!!
  28. 28. ©AmadeusITGroupanditsaffiliatesandsubsidiaries You can follow us on: AmadeusITgroup amadeus.com amadeus.com/blog Thank you!

×