SlideShare a Scribd company logo
1 of 44
Download to read offline
Using Ansible for Deploying
to Cloud Environments
Andrew Hamilton
Who am I?
● Engineering Operations Lead for Prevoty
● Mentor META Lab at CSU, Northridge
● Formerly SRE @Twitter Search and Sys
Admin for Eucalyptus (now HP)
What will we discuss?
● What was the problem to solve?
● Why we chose Ansible
● Tips for using Ansible for deploys
● Working in the cloud and beyond
What was the problem to solve?
We needed a simple and repeatable way to
build services and push them out to properly
configured environments.
Previously used tools wouldn’t cut it
● Puppet
● Capistrano
● Fabric
Multiple languages
● We use:
○ Go
○ Java
○ PHP
● Needed a way to easily build and package
any of these
Moved from EC2 Classic to VPC
Image from: http://docs.aws.amazon.
com/opsworks/latest/userguide/workingstacks-
vpc.html
Moved from EC2 Classic to VPC
● Moved majority of services into private
subnets
● Direct access to instances now limited
○ ELBs are the default access method for pub traffic
○ Bastion host setup in each VPC for SSH access
○ SSH config used for “routing” access to bastions
SSH Config
~/.ssh/config
Host bastion
Hostname <public_dns_hostname_west>
IdentityFile <pem>
Host bastion-east
Hostname <public_dns_hostname_east>
IdentityFile <pem>
Host *.us-west-2.compute.internal
ProxyCommand ssh bastion nc -q 10 -w 10 %h %p 2>/dev/null
Host *.ec2.internal
ProxyCommand ssh bastion-east nc -q 10 -w 10 %h %p 2>/dev/null
Moved towards ephemeral instances
● Nodes are usually rolled between releases
● Use a blue-green deployment process
Why we chose Ansible
We had a developer working on a tool but this
isn’t our core competency so it was better to
move away from the responsibility of building
our own.
Focus on our core competencies
● We’re not in the deployment automation
business
● No need to build a tool if a sufficient one
already exists
Ansible has a simple execution model
● It is easier to understand than the
declarative model used by Puppet
● Execution happens in order
Open source core written in Python
● Easy to extend and update when needed
● Easy to run from HEAD or a branch
YaML is a simple language
● Easy for devs to also add and fix playbooks
SSH based communication
● Don’t need to install anything on new
instances
● Great for the cloud where instances are
created and destroyed often
● No changes needed to security groups
● Respects SSH configs
Simple secret storage
● ansible-vault command
● Integrates automatically with playbooks
● AES encrypted text can sit in version
control
Modules for almost everything
● Makes it super easy to get things done
● From file creation to working with load
balancers and beyond
● Majority are idempotent
Modules can be in any language
● Take in JSON and produce JSON
Ansible is well suited for the cloud
● Dynamic inventories
● Both configuration management and
remote command execution
● Run it when you need it
Use a dynamic inventory
● The cloud is ephemeral
● Standardize on a way to find instances
● ec2.py
○ Uses format tag_<tag_name>_<tag_value>
○ For new hosts: tag_<tag_name>_<tag_value>_new
Configure a dynamic inventory
● Configure it to work for you
● ec2.py and ec2.ini
○ Configured to provide the private DNS even if
public DNS does exist
Break up your playbooks
● Keep playbooks small
● We break ours on verbs:
○ Provision
○ Setup
○ Deploy
○ Promote
○ Terminate
Learn variable hierarchy
● From the Ansible docs
○ extra vars (-e in the command line) always win
○ then comes connection variables defined in inventory
(ansible_ssh_user, etc)
○ then comes "most everything else" (command line switches, vars in
play, included vars, role vars, etc)
○ then comes the rest of the variables defined in inventory
○ then comes facts discovered about a system
○ then "role defaults", which are the most "defaulty" and lose in
priority to everything.
Use common variables when possible
● group_vars/all
● Standardize as much as you can
Examples of what we put in there
group_vars/all
ansible_ssh_private_key_file: ~/.ssh/{{ key_name | default(service_name) }}.pem
ansible_ssh_user: "{{ remote_user }}"
remote_user:"{{aws_config[my_ec2_region]['remote_user'] | default('ec2-user')}}"
my_ec2_region: "{{ lookup('env', 'EC2_REGION') }}"
default_service_dir: /usr/local/prevoty
java_version: 1.8.0_25
java_home: /opt/jre{{ java_version }}
go_version: go1.4.1
Separate specific vars by service
● group_vars/<service_name>
● These will be vars specific to this service
○ ELB
○ VPC Subnet(s)
○ Configuration
Combine secrets by environment
● group_vars/all_<service_env>
● We’ve found that placing all secrets
together to be easier to deal with
● Single simple import
● Decryption happens automatically
Build generic playbooks
● Playbooks can be built on top of variables
● Use the “extra vars” (-e) to specify a service
% ansible-playbook --ask-vault-pass -i <inventory> -e
“service_name=<service> service_env=<env>” deploy.yml
Import var files based on extra vars
● Use the vars passed by the cli to specify
imports
- hosts: my_group
vars_files:
- group_vars/all
- group_vars/all_{{ service_env }}
- group_vars/{{ service_name }}
roles:
- my_role
Specify host groups with vars
● You can reference a host group based on
variables
● Host group can sit inside of a vars file
tag_{{ host_group_key }}_{{ host_group_value }}
Putting it together
deploy.yml
- hosts: tag_{{ host_group_tag }}_{{ host_group_value }}
vars_files:
- group_vars/all
- group_vars/all_{{ service_env }}
- group_vars/{{ service_name }}
role:
- deploy
% ansible-playbook --ask-vault-pass -i <inventory> -e “service_env=<env>
service_name=<service>” deploy.yml
Wrap it up with an old friend
● We use bash to wrap playbooks together
● Easily run a full deploy
● Restart at intermediate steps if needed
Allow hash merging
● Makes it so much easier for cloud
deployments
● Allows you to have one data structure
across files in group_vars that become
more easily accessible at runtime
● Enabled in ansible.cfg
Hash merging example
group_vars/all
aws_config : {
“us-west-1” : {
“ami_id” : “ami-00112233”,
“rds_url” : <west_url>,
},
“us-east-1” : {
“ami_id” : “ami-44556677”,
“rds_url” : <west_url>,
}
}
group_vars/<service_name>
aws_config : {
“us-west-1” : {
“elb_name” : <west_elb>,
“vpc_subnet” : [<west_subnet>],
},
“us-east-1” : {
“elb_name” : <east_elb>,
“vpc_subnet” : [<east_subnet>],
}
}
Hash merging example cont’d
result
aws_config : {
“us-west-1” : {
“ami_id” : “ami-00112233”,
“rds_url” : <west_url>,
“elb_name” : <west_elb>,
“vpc_subnet” : [<west_subnet>],
},
“us-east-1” : {
“ami_id” : “ami-44556677”,
result cont’d
“rds_url” : <west_url>,
“elb_name” : <east_elb>,
“vpc_subnet” : [<east_subnet>],
}
}
Hash merging example cont’d
● Easy access in playbooks based on region
ec2_region: {{ lookup(‘ENV’, ‘EC2_REGION’) }}
- or -
-e “ec2_region=<region>”
● Accessed by:
{{ aws_config[ec2_region][‘elb_name’] }}
Make sure it fails…
● It shouldn’t just fail when there’s an error!
● Don’t run other plays in a playbook if a
prerequisite isn’t met
● Ex: No hosts found in a host group
Test changes from start to finish
● Don’t consider a fix complete until you’ve
run the entire deploy from start to finish
● Commands issued while debugging an issue
can fix that issue without persistence
Working in the cloud and beyond
Our focus is in the cloud but it doesn’t always
work for customers when it comes to their
view of security
A VM is a VM
● We can automatically build a VM with tools
such as Cobbler or packer on VMWare, KVM
or XenServer
● Automated builds of the base OS that is the
same as we run on AWS
Only the endpoints changed
● IP of the VM added to a static inventory
● Same playbooks and roles used for setup of
the OS and build/deploy of the service
An example inventory
/tmp/inventory
[tag_<host_group_name>_<service_a>_new]
<service_a> ansible_ssh_host=10.0.xxx.yyy
[tag_<host_group_name>_<service_b>_new]
<service_b> ansible_ssh_host=10.0.xxx.yyy
[tag_<host_group_name>_<service_c>_new]
<service_c> ansible_ssh_host=10.0.xxx.yyy
Questions?

More Related Content

What's hot

What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
Simplilearn
 

What's hot (20)

Ansible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David KarbanAnsible Introduction - Ansible Brno #1 - David Karban
Ansible Introduction - Ansible Brno #1 - David Karban
 
Ansible fest Presentation slides
Ansible fest Presentation slidesAnsible fest Presentation slides
Ansible fest Presentation slides
 
Ansible is the simplest way to automate. MoldCamp, 2015
Ansible is the simplest way to automate. MoldCamp, 2015Ansible is the simplest way to automate. MoldCamp, 2015
Ansible is the simplest way to automate. MoldCamp, 2015
 
ansible why ?
ansible why ?ansible why ?
ansible why ?
 
Cyansible
CyansibleCyansible
Cyansible
 
Network Automation: Ansible 102
Network Automation: Ansible 102Network Automation: Ansible 102
Network Automation: Ansible 102
 
Advance discussion on Ansible - Rahul Inti
Advance discussion on Ansible - Rahul IntiAdvance discussion on Ansible - Rahul Inti
Advance discussion on Ansible - Rahul Inti
 
Ansible basics workshop
Ansible basics workshopAnsible basics workshop
Ansible basics workshop
 
AnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and TricksAnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and Tricks
 
Basics of Ansible - Sahil Davawala
Basics of Ansible - Sahil DavawalaBasics of Ansible - Sahil Davawala
Basics of Ansible - Sahil Davawala
 
Managing Your Cisco Datacenter Network with Ansible
Managing Your Cisco Datacenter Network with AnsibleManaging Your Cisco Datacenter Network with Ansible
Managing Your Cisco Datacenter Network with Ansible
 
Introduction to ansible galaxy
Introduction to ansible galaxyIntroduction to ansible galaxy
Introduction to ansible galaxy
 
Ansible 101, Gennadiy Mykhailiuta
Ansible 101, Gennadiy MykhailiutaAnsible 101, Gennadiy Mykhailiuta
Ansible 101, Gennadiy Mykhailiuta
 
Ansible with AWS
Ansible with AWSAnsible with AWS
Ansible with AWS
 
Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2Ansible 2 and Ansible Galaxy 2
Ansible 2 and Ansible Galaxy 2
 
Ansible Intro - June 2015 / Ansible Barcelona User Group
Ansible Intro - June 2015 / Ansible Barcelona User GroupAnsible Intro - June 2015 / Ansible Barcelona User Group
Ansible Intro - June 2015 / Ansible Barcelona User Group
 
Network automation (NetDevOps) with Ansible
Network automation (NetDevOps) with AnsibleNetwork automation (NetDevOps) with Ansible
Network automation (NetDevOps) with Ansible
 
DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!DevOps for Humans - Ansible for Drupal Deployment Victory!
DevOps for Humans - Ansible for Drupal Deployment Victory!
 
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
What Is Ansible? | How Ansible Works? | Ansible Tutorial For Beginners | DevO...
 
Go Faster with Ansible (AWS meetup)
Go Faster with Ansible (AWS meetup)Go Faster with Ansible (AWS meetup)
Go Faster with Ansible (AWS meetup)
 

Viewers also liked

[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
Insight Technology, Inc.
 
Diplome Maltherapeut
Diplome MaltherapeutDiplome Maltherapeut
Diplome Maltherapeut
Dino Toniolo
 

Viewers also liked (20)

The Open-Source Monitoring Landscape
The Open-Source Monitoring LandscapeThe Open-Source Monitoring Landscape
The Open-Source Monitoring Landscape
 
(SDD408) Amazon Route 53 Deep Dive: Delivering Resiliency, Minimizing Latency...
(SDD408) Amazon Route 53 Deep Dive: Delivering Resiliency, Minimizing Latency...(SDD408) Amazon Route 53 Deep Dive: Delivering Resiliency, Minimizing Latency...
(SDD408) Amazon Route 53 Deep Dive: Delivering Resiliency, Minimizing Latency...
 
Performance Tuning EC2 Instances
Performance Tuning EC2 InstancesPerformance Tuning EC2 Instances
Performance Tuning EC2 Instances
 
Let's Build an Angular App!
Let's Build an Angular App!Let's Build an Angular App!
Let's Build an Angular App!
 
Monitoramento de Ativos: Você sabe o que acontece na sua rede?
Monitoramento de Ativos: Você sabe o que acontece na sua rede?Monitoramento de Ativos: Você sabe o que acontece na sua rede?
Monitoramento de Ativos: Você sabe o que acontece na sua rede?
 
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016The ultimate container monitoring bake-off - Rancher Online Meetup October 2016
The ultimate container monitoring bake-off - Rancher Online Meetup October 2016
 
[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
[B34] MySQL最新ロードマップ – MySQL 5.7とその先へ by Ryusuke Kajiyama
 
AWSとAnsibleで実践!プロビジョニング入門‐Lamp+Laravel-
AWSとAnsibleで実践!プロビジョニング入門‐Lamp+Laravel-AWSとAnsibleで実践!プロビジョニング入門‐Lamp+Laravel-
AWSとAnsibleで実践!プロビジョニング入門‐Lamp+Laravel-
 
Darkfield LED Array
Darkfield LED ArrayDarkfield LED Array
Darkfield LED Array
 
Austin Benn Sales and Marketing Newsletter (clients) - Issue 3
Austin Benn Sales and Marketing Newsletter (clients) - Issue 3Austin Benn Sales and Marketing Newsletter (clients) - Issue 3
Austin Benn Sales and Marketing Newsletter (clients) - Issue 3
 
UK Export Finance : Presentation for China event
UK Export Finance : Presentation for China eventUK Export Finance : Presentation for China event
UK Export Finance : Presentation for China event
 
Ee15 presentation edinburgh
Ee15 presentation edinburghEe15 presentation edinburgh
Ee15 presentation edinburgh
 
Austin Benn Recruitment - Sales & Marketing Newsletter (Candidates) Issue 4
Austin Benn Recruitment - Sales & Marketing Newsletter (Candidates) Issue 4Austin Benn Recruitment - Sales & Marketing Newsletter (Candidates) Issue 4
Austin Benn Recruitment - Sales & Marketing Newsletter (Candidates) Issue 4
 
Nouns
NounsNouns
Nouns
 
Using Python
Using PythonUsing Python
Using Python
 
احمد السيد
احمد السيداحمد السيد
احمد السيد
 
Diplome Maltherapeut
Diplome MaltherapeutDiplome Maltherapeut
Diplome Maltherapeut
 
Linea de tiempo
Linea de tiempoLinea de tiempo
Linea de tiempo
 
Austin Benn Recruitment - Sales & Marketing Newsletter (Clients) Issue 4
Austin Benn Recruitment - Sales & Marketing Newsletter (Clients) Issue 4Austin Benn Recruitment - Sales & Marketing Newsletter (Clients) Issue 4
Austin Benn Recruitment - Sales & Marketing Newsletter (Clients) Issue 4
 
Ami Polymer Pvt Ltd
Ami Polymer Pvt LtdAmi Polymer Pvt Ltd
Ami Polymer Pvt Ltd
 

Similar to Using Ansible for Deploying to Cloud Environments

A3Sec Advanced Deployment System
A3Sec Advanced Deployment SystemA3Sec Advanced Deployment System
A3Sec Advanced Deployment System
a3sec
 

Similar to Using Ansible for Deploying to Cloud Environments (20)

Ansible is Our Wishbone(Automate DBA Tasks With Ansible)
Ansible is Our Wishbone(Automate DBA Tasks With Ansible)Ansible is Our Wishbone(Automate DBA Tasks With Ansible)
Ansible is Our Wishbone(Automate DBA Tasks With Ansible)
 
Ansible is Our Wishbone
Ansible is Our WishboneAnsible is Our Wishbone
Ansible is Our Wishbone
 
#OktoCampus - Workshop : An introduction to Ansible
#OktoCampus - Workshop : An introduction to Ansible#OktoCampus - Workshop : An introduction to Ansible
#OktoCampus - Workshop : An introduction to Ansible
 
Ansible intro
Ansible introAnsible intro
Ansible intro
 
Ansible - Hands on Training
Ansible - Hands on TrainingAnsible - Hands on Training
Ansible - Hands on Training
 
Ansible Tutorial.pdf
Ansible Tutorial.pdfAnsible Tutorial.pdf
Ansible Tutorial.pdf
 
An introduction to Rex - FLOSS UK DevOps York 2015
An introduction to Rex - FLOSS UK DevOps York 2015An introduction to Rex - FLOSS UK DevOps York 2015
An introduction to Rex - FLOSS UK DevOps York 2015
 
Declarative Infrastructure Tools
Declarative Infrastructure Tools Declarative Infrastructure Tools
Declarative Infrastructure Tools
 
DevOps for database
DevOps for databaseDevOps for database
DevOps for database
 
Cloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the CloudCloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the Cloud
 
Ansible Automation to Rule Them All
Ansible Automation to Rule Them AllAnsible Automation to Rule Them All
Ansible Automation to Rule Them All
 
Ansible a tool for dev ops
Ansible a tool for dev opsAnsible a tool for dev ops
Ansible a tool for dev ops
 
AWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp VaultAWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp Vault
 
A tour of Ansible
A tour of AnsibleA tour of Ansible
A tour of Ansible
 
Workflow story: Theory versus Practice in large enterprises by Marcin Piebiak
Workflow story: Theory versus Practice in large enterprises by Marcin PiebiakWorkflow story: Theory versus Practice in large enterprises by Marcin Piebiak
Workflow story: Theory versus Practice in large enterprises by Marcin Piebiak
 
Workflow story: Theory versus practice in Large Enterprises
Workflow story: Theory versus practice in Large EnterprisesWorkflow story: Theory versus practice in Large Enterprises
Workflow story: Theory versus practice in Large Enterprises
 
Introduction to ansible
Introduction to ansibleIntroduction to ansible
Introduction to ansible
 
Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.
 
A3Sec Advanced Deployment System
A3Sec Advanced Deployment SystemA3Sec Advanced Deployment System
A3Sec Advanced Deployment System
 
[HKOSCON][20180616][Containerized High Availability Virtual Hosting Deploymen...
[HKOSCON][20180616][Containerized High Availability Virtual Hosting Deploymen...[HKOSCON][20180616][Containerized High Availability Virtual Hosting Deploymen...
[HKOSCON][20180616][Containerized High Availability Virtual Hosting Deploymen...
 

Recently uploaded

JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
Max Lee
 

Recently uploaded (20)

AI Hackathon.pptx
AI                        Hackathon.pptxAI                        Hackathon.pptx
AI Hackathon.pptx
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
A Python-based approach to data loading in TM1 - Using Airflow as an ETL for TM1
 
How to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabberHow to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabber
 
IT Software Development Resume, Vaibhav jha 2024
IT Software Development Resume, Vaibhav jha 2024IT Software Development Resume, Vaibhav jha 2024
IT Software Development Resume, Vaibhav jha 2024
 
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
 
A Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data MigrationA Guideline to Gorgias to to Re:amaze Data Migration
A Guideline to Gorgias to to Re:amaze Data Migration
 
Studiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting softwareStudiovity film pre-production and screenwriting software
Studiovity film pre-production and screenwriting software
 
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
 
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
 
how-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdfhow-to-download-files-safely-from-the-internet.pdf
how-to-download-files-safely-from-the-internet.pdf
 
The Impact of PLM Software on Fashion Production
The Impact of PLM Software on Fashion ProductionThe Impact of PLM Software on Fashion Production
The Impact of PLM Software on Fashion Production
 
Crafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationCrafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM Integration
 
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAGAI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
 
APVP,apvp apvp High quality supplier safe spot transport, 98% purity
APVP,apvp apvp High quality supplier safe spot transport, 98% purityAPVP,apvp apvp High quality supplier safe spot transport, 98% purity
APVP,apvp apvp High quality supplier safe spot transport, 98% purity
 
What need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java DevelopersWhat need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java Developers
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)JustNaik Solution Deck (stage bus sector)
JustNaik Solution Deck (stage bus sector)
 

Using Ansible for Deploying to Cloud Environments

  • 1. Using Ansible for Deploying to Cloud Environments Andrew Hamilton
  • 2. Who am I? ● Engineering Operations Lead for Prevoty ● Mentor META Lab at CSU, Northridge ● Formerly SRE @Twitter Search and Sys Admin for Eucalyptus (now HP)
  • 3. What will we discuss? ● What was the problem to solve? ● Why we chose Ansible ● Tips for using Ansible for deploys ● Working in the cloud and beyond
  • 4. What was the problem to solve? We needed a simple and repeatable way to build services and push them out to properly configured environments.
  • 5. Previously used tools wouldn’t cut it ● Puppet ● Capistrano ● Fabric
  • 6. Multiple languages ● We use: ○ Go ○ Java ○ PHP ● Needed a way to easily build and package any of these
  • 7. Moved from EC2 Classic to VPC Image from: http://docs.aws.amazon. com/opsworks/latest/userguide/workingstacks- vpc.html
  • 8. Moved from EC2 Classic to VPC ● Moved majority of services into private subnets ● Direct access to instances now limited ○ ELBs are the default access method for pub traffic ○ Bastion host setup in each VPC for SSH access ○ SSH config used for “routing” access to bastions
  • 9. SSH Config ~/.ssh/config Host bastion Hostname <public_dns_hostname_west> IdentityFile <pem> Host bastion-east Hostname <public_dns_hostname_east> IdentityFile <pem> Host *.us-west-2.compute.internal ProxyCommand ssh bastion nc -q 10 -w 10 %h %p 2>/dev/null Host *.ec2.internal ProxyCommand ssh bastion-east nc -q 10 -w 10 %h %p 2>/dev/null
  • 10. Moved towards ephemeral instances ● Nodes are usually rolled between releases ● Use a blue-green deployment process
  • 11. Why we chose Ansible We had a developer working on a tool but this isn’t our core competency so it was better to move away from the responsibility of building our own.
  • 12. Focus on our core competencies ● We’re not in the deployment automation business ● No need to build a tool if a sufficient one already exists
  • 13. Ansible has a simple execution model ● It is easier to understand than the declarative model used by Puppet ● Execution happens in order
  • 14. Open source core written in Python ● Easy to extend and update when needed ● Easy to run from HEAD or a branch
  • 15. YaML is a simple language ● Easy for devs to also add and fix playbooks
  • 16. SSH based communication ● Don’t need to install anything on new instances ● Great for the cloud where instances are created and destroyed often ● No changes needed to security groups ● Respects SSH configs
  • 17. Simple secret storage ● ansible-vault command ● Integrates automatically with playbooks ● AES encrypted text can sit in version control
  • 18. Modules for almost everything ● Makes it super easy to get things done ● From file creation to working with load balancers and beyond ● Majority are idempotent
  • 19. Modules can be in any language ● Take in JSON and produce JSON
  • 20. Ansible is well suited for the cloud ● Dynamic inventories ● Both configuration management and remote command execution ● Run it when you need it
  • 21. Use a dynamic inventory ● The cloud is ephemeral ● Standardize on a way to find instances ● ec2.py ○ Uses format tag_<tag_name>_<tag_value> ○ For new hosts: tag_<tag_name>_<tag_value>_new
  • 22. Configure a dynamic inventory ● Configure it to work for you ● ec2.py and ec2.ini ○ Configured to provide the private DNS even if public DNS does exist
  • 23. Break up your playbooks ● Keep playbooks small ● We break ours on verbs: ○ Provision ○ Setup ○ Deploy ○ Promote ○ Terminate
  • 24. Learn variable hierarchy ● From the Ansible docs ○ extra vars (-e in the command line) always win ○ then comes connection variables defined in inventory (ansible_ssh_user, etc) ○ then comes "most everything else" (command line switches, vars in play, included vars, role vars, etc) ○ then comes the rest of the variables defined in inventory ○ then comes facts discovered about a system ○ then "role defaults", which are the most "defaulty" and lose in priority to everything.
  • 25. Use common variables when possible ● group_vars/all ● Standardize as much as you can
  • 26. Examples of what we put in there group_vars/all ansible_ssh_private_key_file: ~/.ssh/{{ key_name | default(service_name) }}.pem ansible_ssh_user: "{{ remote_user }}" remote_user:"{{aws_config[my_ec2_region]['remote_user'] | default('ec2-user')}}" my_ec2_region: "{{ lookup('env', 'EC2_REGION') }}" default_service_dir: /usr/local/prevoty java_version: 1.8.0_25 java_home: /opt/jre{{ java_version }} go_version: go1.4.1
  • 27. Separate specific vars by service ● group_vars/<service_name> ● These will be vars specific to this service ○ ELB ○ VPC Subnet(s) ○ Configuration
  • 28. Combine secrets by environment ● group_vars/all_<service_env> ● We’ve found that placing all secrets together to be easier to deal with ● Single simple import ● Decryption happens automatically
  • 29. Build generic playbooks ● Playbooks can be built on top of variables ● Use the “extra vars” (-e) to specify a service % ansible-playbook --ask-vault-pass -i <inventory> -e “service_name=<service> service_env=<env>” deploy.yml
  • 30. Import var files based on extra vars ● Use the vars passed by the cli to specify imports - hosts: my_group vars_files: - group_vars/all - group_vars/all_{{ service_env }} - group_vars/{{ service_name }} roles: - my_role
  • 31. Specify host groups with vars ● You can reference a host group based on variables ● Host group can sit inside of a vars file tag_{{ host_group_key }}_{{ host_group_value }}
  • 32. Putting it together deploy.yml - hosts: tag_{{ host_group_tag }}_{{ host_group_value }} vars_files: - group_vars/all - group_vars/all_{{ service_env }} - group_vars/{{ service_name }} role: - deploy % ansible-playbook --ask-vault-pass -i <inventory> -e “service_env=<env> service_name=<service>” deploy.yml
  • 33. Wrap it up with an old friend ● We use bash to wrap playbooks together ● Easily run a full deploy ● Restart at intermediate steps if needed
  • 34. Allow hash merging ● Makes it so much easier for cloud deployments ● Allows you to have one data structure across files in group_vars that become more easily accessible at runtime ● Enabled in ansible.cfg
  • 35. Hash merging example group_vars/all aws_config : { “us-west-1” : { “ami_id” : “ami-00112233”, “rds_url” : <west_url>, }, “us-east-1” : { “ami_id” : “ami-44556677”, “rds_url” : <west_url>, } } group_vars/<service_name> aws_config : { “us-west-1” : { “elb_name” : <west_elb>, “vpc_subnet” : [<west_subnet>], }, “us-east-1” : { “elb_name” : <east_elb>, “vpc_subnet” : [<east_subnet>], } }
  • 36. Hash merging example cont’d result aws_config : { “us-west-1” : { “ami_id” : “ami-00112233”, “rds_url” : <west_url>, “elb_name” : <west_elb>, “vpc_subnet” : [<west_subnet>], }, “us-east-1” : { “ami_id” : “ami-44556677”, result cont’d “rds_url” : <west_url>, “elb_name” : <east_elb>, “vpc_subnet” : [<east_subnet>], } }
  • 37. Hash merging example cont’d ● Easy access in playbooks based on region ec2_region: {{ lookup(‘ENV’, ‘EC2_REGION’) }} - or - -e “ec2_region=<region>” ● Accessed by: {{ aws_config[ec2_region][‘elb_name’] }}
  • 38. Make sure it fails… ● It shouldn’t just fail when there’s an error! ● Don’t run other plays in a playbook if a prerequisite isn’t met ● Ex: No hosts found in a host group
  • 39. Test changes from start to finish ● Don’t consider a fix complete until you’ve run the entire deploy from start to finish ● Commands issued while debugging an issue can fix that issue without persistence
  • 40. Working in the cloud and beyond Our focus is in the cloud but it doesn’t always work for customers when it comes to their view of security
  • 41. A VM is a VM ● We can automatically build a VM with tools such as Cobbler or packer on VMWare, KVM or XenServer ● Automated builds of the base OS that is the same as we run on AWS
  • 42. Only the endpoints changed ● IP of the VM added to a static inventory ● Same playbooks and roles used for setup of the OS and build/deploy of the service
  • 43. An example inventory /tmp/inventory [tag_<host_group_name>_<service_a>_new] <service_a> ansible_ssh_host=10.0.xxx.yyy [tag_<host_group_name>_<service_b>_new] <service_b> ansible_ssh_host=10.0.xxx.yyy [tag_<host_group_name>_<service_c>_new] <service_c> ansible_ssh_host=10.0.xxx.yyy