DevOps Krakow #Meet 1

DNS CLUSTER
Automated Internal DNS Service with Amazon VPC integration

Sławomir Skowron 	

System ...
DNS INTRODUCTION
WHAT IS DNS ?
•

Domain Name System is hierarchical and distributed naming
system	


•

Essentially name service for TCP/I...
NAMESPACE
Tree hierarchical structure starting from . (root)
ZONES
Delegated part of domain name space for administrative
responsibility.
DOMAIN NAME SERVERS
Software on servers that store, manage and serve information about own part
of domain namespace called...
DNS QUERIES
Two type of external queries: Recursive and Iterative

•

Recursive - querying other servers until positive re...
DNS AS A SERVICE
INSIDE AMAZON CLOUD
AMAZON EC2 DNS (VPC) PROBLEMS

•

Route-53 (right now) is not supporting internal DNS domains	


•

Amazon VPC Internal DN...
Our own DNS Service
USE CASE
Our own DNS Service
•

Available only in LAN and through VPN	


•

Only A and SRV - infrastructure DNS

•

Resolv...
SOLUTION
Our own DNS Service
•

Clustering for High Availability and Performance	


•

Integration with our VPC’s DHPC	


...
HIERARCHY
Hierarchy of private DNS at BaseCRM
DNSCLUSTER
RELEASE 1.0
:)
T
S
O
M
L
A
SOLUTION
•

Puppet 3 as Configuration Management solution	


•

Puppet Hiera, PuppetDB integration	


•

TheForeman - http:...
DNSCLUSTER
Integration with Puppet and TheForeman
WHAT’S WRONG WITH PUPPET ?
•

Puppet is slow	


•

Hard and slow flow developing with Puppet	


•

Hard to integrate on run...
PUPPET
ANSIBLE
Radicaly simple IT orchestration
ANSIBLE
•

Minimal setup - Python + Libs - pip install ansible	


•

Use existing auth (root, sudo) on SSH as default tran...
source: http://www.ansibleworks.com/
ANSIBLE @ BASE
•

Two months of work all in GIT

•

15 playbooks (Universal Flow)

•

25 roles

•

180 yaml files

•

52 te...
DNSCLUSTER
RELEASE 2.0
SOLUTION
•

Ansible

•

Unbound as DNSCluster core - local zones, forwarder, cache	


•

Git for store zones and versionin...
KISS

Keep it simple, stupid	


Core Thinking
IMPROVEMENT
KISS as core thinking
•

Simple workflow	


•

Faster development	


•

Fast Deploy with low memory/cpu consump...
DNSCLUSTER 2.0
Flow for DNSCluster Client
DNSCLUSTER 2.0 - MULTI-REGIONS
DNS CLUSTER
PERFORMANCE
DNSCLUSTER PERFORMANCE
Queries per second / Concurrency
2500

2000

AWS DNS
DNSCLUSTER 1 node (1 cpu core –
ec2.x1.small)
...
DNSCLUSTER PERFORMANCE
0.12

Latency / Concurrency

0.1

AWS DNS
DNSCLUSTER 1 node (1 cpu core –
ec2.x1.small)

Latensy [s...
SOON / NEXT TIME ?
Ansible Universal Template Flow
Created @ Base for simple consistent create/destroy instances

Monitori...
THE END
Dnscluster @ DevOps Krakow 2013
Upcoming SlideShare
Loading in …5
×

Dnscluster @ DevOps Krakow 2013

942 views

Published on

Simple infrastructure DNS service inside Amazon Cloud automated by Ansible, created @ BaseCRM

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
942
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
16
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Dnscluster @ DevOps Krakow 2013

  1. 1. DevOps Krakow #Meet 1 DNS CLUSTER Automated Internal DNS Service with Amazon VPC integration Sławomir Skowron System Engineer (DevOps Team) slawomir.skowron@getbase.com 2013
  2. 2. DNS INTRODUCTION
  3. 3. WHAT IS DNS ? • Domain Name System is hierarchical and distributed naming system • Essentially name service for TCP/IP networks • Allow IP address resolution mechanism • Adds tree based domain name space, • Name space is sub-divides into zones and start with root zone • One of the first NoSQL key-value database
  4. 4. NAMESPACE Tree hierarchical structure starting from . (root)
  5. 5. ZONES Delegated part of domain name space for administrative responsibility.
  6. 6. DOMAIN NAME SERVERS Software on servers that store, manage and serve information about own part of domain namespace called zone Two types of servers: master and slave
  7. 7. DNS QUERIES Two type of external queries: Recursive and Iterative • Recursive - querying other servers until positive response • Iterative - add local response (cache, local zone) or give info where to look for it. Cached Queries - DNS Cache - improve latency and throughput
  8. 8. DNS AS A SERVICE INSIDE AMAZON CLOUD
  9. 9. AMAZON EC2 DNS (VPC) PROBLEMS • Route-53 (right now) is not supporting internal DNS domains • Amazon VPC Internal DNS support only ec2.internal domains • Amazon VPC DHCP in default support only AWS DNS
  10. 10. Our own DNS Service
  11. 11. USE CASE Our own DNS Service • Available only in LAN and through VPN • Only A and SRV - infrastructure DNS • Resolv local and forward if not exist • No zone transfer, No slaves, No masters • Updates are simple, secure and fast
  12. 12. SOLUTION Our own DNS Service • Clustering for High Availability and Performance • Integration with our VPC’s DHPC • Availability in every Amazon Region • Caching • Fully Automated and Integrated with Instance Provisioning • Support for our name space
  13. 13. HIERARCHY Hierarchy of private DNS at BaseCRM
  14. 14. DNSCLUSTER RELEASE 1.0
  15. 15. :) T S O M L A
  16. 16. SOLUTION • Puppet 3 as Configuration Management solution • Puppet Hiera, PuppetDB integration • TheForeman - http://theforeman.org/ • Foreman integrates with BIND • Unbound as DNSCluster core - local zones, forwarder, cache • Git for store zones and versioning
  17. 17. DNSCLUSTER Integration with Puppet and TheForeman
  18. 18. WHAT’S WRONG WITH PUPPET ? • Puppet is slow • Hard and slow flow developing with Puppet • Hard to integrate on running machines before puppet. • PuppetDB is ok but it’s not scalable enough • Everything go through Foreman and BIND in our case
  19. 19. PUPPET
  20. 20. ANSIBLE Radicaly simple IT orchestration
  21. 21. ANSIBLE • Minimal setup - Python + Libs - pip install ansible • Use existing auth (root, sudo) on SSH as default transport or accelerated mode • Ad-hoc operations built in • async, sync and parallel operations • Predictable, easy to expand (plugins, connectors, filters, modules) • Use powerful templates in jinja2 • outputs in json, • configure in yaml
  22. 22. source: http://www.ansibleworks.com/
  23. 23. ANSIBLE @ BASE • Two months of work all in GIT • 15 playbooks (Universal Flow) • 25 roles • 180 yaml files • 52 template
  24. 24. DNSCLUSTER RELEASE 2.0
  25. 25. SOLUTION • Ansible • Unbound as DNSCluster core - local zones, forwarder, cache • Git for store zones and versioning • Amazon VPC DHCP integration - under development • ETCD integration - under development
  26. 26. KISS Keep it simple, stupid Core Thinking
  27. 27. IMPROVEMENT KISS as core thinking • Simple workflow • Faster development • Fast Deploy with low memory/cpu consumption • No central DB • All data are stored in 3 places and can be restored from running machines • Work as push or pull workflow • Integrated with VPC DHCP if new DNSCluster is created
  28. 28. DNSCLUSTER 2.0 Flow for DNSCluster Client
  29. 29. DNSCLUSTER 2.0 - MULTI-REGIONS
  30. 30. DNS CLUSTER PERFORMANCE
  31. 31. DNSCLUSTER PERFORMANCE Queries per second / Concurrency 2500 2000 AWS DNS DNSCLUSTER 1 node (1 cpu core – ec2.x1.small) 1500 QPS UNBOUND local cache (forwarders: 3 dnscluster nodes – 3 x ec2.x1.small) 1 pass – 1 unbound thread UNBOUND local cache (forwarders: 3 dnscluster nodes – 3 x ec2.x1.small) 2 pass – from cache – 1 unbound threads 1000 UNBOUND local cache (forwarders: 3 dnscluster nodes – 3 x ec2.x1.small) 2 pass – from cache – 2 unbound threads 500 0 1 500 Concurrency 1000
  32. 32. DNSCLUSTER PERFORMANCE 0.12 Latency / Concurrency 0.1 AWS DNS DNSCLUSTER 1 node (1 cpu core – ec2.x1.small) Latensy [seconds] 0.08 UNBOUND local cache (forwarders: 3 dnscluster nodes – 3 x ec2.x1.small) 1 pass – 1 unbound thread 0.06 UNBOUND local cache (forwarders: 3 dnscluster nodes – 3 x ec2.x1.small) 2 pass – from cache – 1 unbound threads 0.04 UNBOUND local cache (forwarders: 3 dnscluster nodes – 3 x ec2.x1.small) 2 pass – from cache – 2 unbound threads 0.02 0 1 500 Concurrency 1000
  33. 33. SOON / NEXT TIME ? Ansible Universal Template Flow Created @ Base for simple consistent create/destroy instances Monitoring and Alerting second element for our auto scaling
  34. 34. THE END

×