Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Automate Big Data with Ansible


Published on

AnsibleFest 2015 was a day-long series of learning seminars by Ansible developers. Our IT Operations Manager, Marius was one of the speakers and he presented “How to Automate Big Data with Ansible”.

Published in: Technology

How to Automate Big Data with Ansible

  1. 1. ANSiBLE How to automate Big Data with Ansible [EH8]? bigstep. com
  2. 2. Presentation Summary About Why Ansible / History of working with Ansible Why Ansible / Bigstep Architecture Apps / Integrating Ansible Going into specifics What we learned Under Development Go Baremetal @519]? bigstep. com
  3. 3. About Me Marius Boeru ' ‘I IT Operations lvlanagem-»Bit_1step . ~’ L? r’ 6 I . Tech Support 2007 — 2008 @ ‘ - Systems Administrator 2008 — late 2014 (from Junior to Seniorl - IT Operations Manager late 2014 — Present - Passionate about improvement and systems in general R -I r - Totally dislike repetitive tasks (§ github. com/ mboeru ,3 . I J m! .)o: :ru it ‘ flqstep“ bigstep. com
  4. 4. About Bigstep High performance, bare metal cloud (| aaS) purpose built for big data applications Cloning, snapshotting, instant provisioning, scaling (even faster than AWS) Pay—per—second Bare metal instances Flexible dedicated layer 2 broadcast domains Bare metal cut—through switching (very low latency) 40 Gbps at the instance level Distributed Al| —SSD storage In the works: managed single tenant scalable clusters of app stacks: Cloudera Hadoop, Elasticsearch, Couchbase, Riak etc controllable via the API and integrated together E elosticseorch. riokc33 QC°uChbaSe flqstep I bigstep. com
  5. 5. Why Ansible? imnmal Nmuork MySQL o1 gigstep l Load Balancar 01 MVSIA (I Ionts outbound tnffic llll Load Balancer 02 H Router 01 H Monltorlng 02 K? ) A - I I - A Backondsarvnr 01 l» — APP59|'V°' W l Backandsorvor OX ‘l ‘PP5°'V‘I' °x l l? A NTP Internal Resolver External Resolver Monllo O In 1 :1 IO . . REPO Logging collector T I bigstep. com
  6. 6. Why Ansible? Reusing the same playbook for provisioning new datacenters. I bigstep. com Egggm a w 4 , ,%g, .;%__: % flqstep
  7. 7. Why Ansible > tree —L 2 A~ README . md ~— production —— group_vars ~— ho5t_varS '—~ hosts -— roles —— automysqlbackup ~— base—apache —— bsiweb ~~ check_mk—agent —— common ~— graphite —— haproxy »— keepalived —— mysqlmastermaster ~— network —— onetime ~— pdns—recursor —— phpmyadmin ~— powerdns '—— redis ~— site. yml —— staging ~~ group_vars —— host_vars ‘~4 hosts fl9$teP' > tree —L 2 roies/ common/ tasks roies/ common/ tasks l~- av. ymI I—— bash_profiIe. ymI I—— epeI. ymI -— facts. ymI ~A hostname. yml I—— iptabIes. yml —— kerneIparams. ymI -- key5.ymI AA Ildp. ymI l~~ Iogging. ymI —— main. ymI —— ntp. ymT ~- packages. yml ~~ password. yml —— rkhunter. yml —— selinux. ymI —- snmpd. yml v~ ssh. yml —— stopservices. ymT —— timezone. yml ‘—— vmware. ymT I bigstep. com
  8. 8. lll Let's build apps flqstep" bigstep. com
  9. 9. Going into specifics - Bigstep Architecture Apps Drivers I APls Driver Storage Drivers I I Switch Drivers I I O5 Drivers I I Ansible Drivers I cloudera "5._, Datameer elosticseorch splunk » Q Couchbase DFlTFlSTFl: ;(. ..9 EX50L (soon) IESIPP" I bigstep. com
  10. 10. Bigstep Elasticsearch App elosticseorch. esmasternodes—611 I ‘ esdav. a.nodes-611 Ie‘ lI‘S‘: a1CES ‘ 7 Instances 2 , m rivi Rvnr w I imntnikallim eunaster-nodes-511.. . 11¢ Drives ' D 6 Master nodes Master node details Node-4164 HQ URL hrtp: l/ '9200l_ pluqiilH0 Data nodes Paramedic URL htlyrll : '.~I)| l0.v‘ pliigiliiparallbedic Nn(l(- 4 I62 """"‘ "M CPU Usage: 45% Node ID: IaNNpi. ':I'-. ’FlLalJN2—C. OoPYA Total memory: 94 44GB Memory used: 52% Total storage: 35 4368 Used storage: I 4866 Elasticsearch ver. : I 3 7 elcisilcseo F(l”i §t<, >|1 C luster Del: -ls: I luster (, au. :el figstep I bigstep. com
  11. 11. Bigstep Elasticsearch App ROLES grep " name es_common: — e5_common: — es, common: — es_Common: — es_common: — e5_common: — es_common: — es_common: - es_common: — es_common: — es_common: — es_common: — es_common’- e5_data_node: — nam name name I name I name I name 1 name I name I name I name I name I name I name I name ' : " */ tasks/ *.yml Install libselinux-python Copy Java rpm Install Java Configure Elasticsearch repository Install Elasticsearch Ensure Elasticsearch at startup Install ElasticHQ plugin Install Paramedic plugin Setting minimum master nodes. if the value is valid. Basic configuration for Elasticsearch service Copy file descriptors configuration file Set Elasticsearch system configuration Set Elasticsearch security limit for number of files e: Set ES node as data node es, master_node: — name: Set ES node as master node E45191?‘ elosticseorch. PLAYBOOK - hosts: elasticsearch_nodes roles: — es_common tags: - es_common — hosts: master_nodes roles: — es_master_node tags: « es_master_nodes — hosts: data_nodes roles: — es_data_node tags: — es_data_nodes bigstep. com
  12. 12. Bigstep Cloudera App cloudera heodnodesvtlda ll‘S‘: al(. ’r-3‘Z~ : rm lnI. il''A I- an INN Aw I s v 5 Master nodes Head node details Node-2712 HOST URL Heami Summary node—l7‘Il coon Dam “°d95 User: admin Password: """ Show password ~ooe,271:; ""09"" Hostname: rioo'e—27I2 Node ID: is: I63 I 23 "We 17“ Total memory: I93 657 aims "Me 0'6 CDH Version: 5 I 3 cloudera uoaenoz N(ideJ1ll. l N0de4104 NudeA105 2ig$t€p' bigstep. com
  13. 13. Bigstep Cloudera App API part (initially part of the Ansible playbook) A Adding hosts to the cluster via the CDH API A Headnodes get Namenode, Resourcewlanager. Zookeepers roles - Datanodes get Datanode. NodeManagers roles A (optional A in devel) Setting up Kerberos. principals. setting up encryption — Start services on all nodes - Calculate and configure YARN according to the size of the cluster, amount of RAM and cores available. Wnen shrinking or scaling out, evict data from a node to be replaced using node decoimri ssioning feature of IDFS. _§, "3/ Datameer fiqstep PLAYB(DK corimon: — conmon : — COHITIOHI * coiimon : A corirnon : — corimon : — COTIIIIOTI I * corirnon: A corirnon : A corrmon : — corirnon : — COTIITIOHZ ' corimon : A cormion : — manager : manager: manager: manage r: name: name: name: name: name: name: name: name: name: name: name: l'laIll€I name: name: name I name: name: name I cloudera python bindings selinux add node info to / etc/ sysconfig/ network remove dhcp hostname script set current hostname Download CM 5.1.1 repository update yum repositories install Oracle JDK 1.7 install cloudera manager agent install cloudera manager daemons configure swappines configure host configure port start agent service start ntp install CM server db start cloudera—scm—db start clouderaAscmAserver wait cloudera manager to actually open 7810 port E)(AsoL I bigstep. com
  14. 14. Why Ansible Again? Simple to use and agentless Idem potent and Reusable - simple yaml format - rerun many times and get the same result - uses ssh - provion nodes and add to cluster with ease - run same playbook on multiple app clusters Easily Extensible Easy to integrate with @519]? bigstep. com
  15. 15. What we learned - Always use name to describe the task even though it's optional - Test your playbooks using —syntax—check, -—check and --d‘i ‘Ff - Use anS‘i bl e—vau'| t to securely store variables - Use tags in playbooks so you could run just specific parts of the whole playbook - Also very useful ——star‘t—at and —step - Use -—"l ‘im‘i t to run on a subset of hosts (can be a group, a host, or multiple hosts web*l) - Time your runs for future reference (time ansible—playbook . ..) - Read the docs regularly as ansible add cool new features all the time (http: //docs. ansible. com) gigstep‘ bigstep. com
  16. 16. Under Development Integrate with Botstep (out hu bot based chatops bot) Use ansible with Rundeck Create our own Ansible Cloud Module for the Bigstep API (httpsi/ fullmetal. bigstep. com/ api) Deeper integration of the Apps module with Ansible Major app upgrades with ansible - Check the version of app actually installed. We use the version of control plugin that matches the version of the app already installed for any operation. - Upgrading is done using the apps system (eg: parcels for cdh) or yum/ apt. . Before update we do a snapshot on the storage. - lfanything happens during upgrade we can easily rollback. fiqstep bigstep. com
  17. 17. la 73.: , V . tji1?, ,I. i._I. .l. : 3*»? I. '.~: ..l ‘fr = f_i"r’»i: «.1:i-. .I‘r. rag . t‘; ‘r= .i"x= i.. i.-. :‘= n;iL ; :5 “jajliijl-I-L: ~ziii, ii ‘ ' : @l. :.l[, -l-ti‘/ —l. 'llL‘l