Gilt uses virtualization including Xen VMs, LXC containers, and Apache CloudStack to manage their infrastructure. They migrated to using CentOS and Scala services and now use technologies like PostgreSQL, Solr, Puppet, and RabbitMQ. Their in-house Galactica system integrates with CloudStack to provision and manage LXC containers at Gilt for their application stack, providing benefits like performance and automation. They are currently testing provisioning applications on CentOS and Ubuntu LXC containers to avoid issues with libvirt.

1. Virtualization at Gilt
Rangarajan Radhakrishnan, ranga@gilt.com
Infrastructure Engineering
Jun 6, 2013

2. What is Gilt?
> Founded in 2007, Gilt provides a curated,
personalized experience featuring products by today’
s top designer labels, at up to 60% off retail
> Products across Men, Women, Home, and Baby &
Kids stores

3. Gilt Tech:
> The engine behind the Gilt experience
Infrastructure Eng:
> Tasked with improving efficiency and
reliability

4. Tech at Gilt
Offices: NYC, Dublin and Portland
Infra Eng: NYC and Dublin
Architecture: NYC, Portland and Dublin
Engineering: NYC, Portland and Dublin

5. Gilt Tech History
> 2007: Small Debian footprint with Ruby on Rails
> 2008: Migrated to OpenSolaris (from RoR to Java
services)
> 2010: Migrated to CentOS (from Java services to
Scala services)
=> =>

6. Current Technology at Gilt

7. Technology
> Databases: PostgreSQL, MongoDB
> Search: Solr, Splunk
> Frameworks: Play
> Languages: Scala, Java, Ruby, Node.js
> Config mgmt: Puppet
> Continuous integration: Jenkins
> Messaging: RabbitMQ

8. Virtualization Goals
> Build on what is available
> Avoid a one-size-fits-all approach
> Offer seamless integration with existing
application stack
> Use lightweight containers that offer great
performance

9. Additional Virtualization Goals
> Automate provisioning as much as possible
> Feature immutable containers -- avoid uncertainties
as a result of changes over time
> Avoid repeated enforcement of config management
policies

10. Virtualization History at Gilt:
The Past
> Bare metal: Debian (2007 - 2008)
> OpenSolaris zones (2008 - 2010)
> Migration to CentOS (2010)

11. Virtualization History at Gilt:
The Present
> Xen VMs (since 2010)
> LXC (since 2012): strong performance, low
overhead
> EC2 (off and on since 2008), with varying degree of
prod/non-prod use (mostly spillover traffic)
> Apache CloudStack (since 2013): to increase
utilization and maximize efficiency

12. Why CloudStack for
Virtualization?
> open source
> support for multiple
hypervisors (KVM, Xenserver and, recently, LXC)
> great web UI and Rest API
> increasingly active community
> committer in-house

13. Virtualization at Gilt with
CloudStack and Galactica
> CloudStack primarily for use with LXC containers
> CloudStack integration with Galactica (in-house)
> Galactica implements a meta-data model
> Galactica integrates with Puppet/LDAP etc.

14. Galactica ...
> is a family of services
> allocates containers (via CloudStack)
> allows for provisioning of other services
> implements a meta-data model for:
>> HW/SW provisioning/management
>> User access provisioning/management
>> IP address/DNS provisioning/management

15. > maintains desired state of services
> provides DNS updates for services, especially
when provisioning or decommissioning
> updates load balancer config
> integrates with monitoring system/audit
> integrates with ioncannon (automated and
continuous deployment)
Galactica also:

16. Why LXC?
> lightweight: minimal performance overhead
> tested in staging with good results
> proof of concept with live traffic showed good
performance
> significant work to support LXC in CloudStack
"LXC will be implemented as a hypervisor in CloudStack and will be a
first class citizen similar to other hypervisors ..."
- Phong Nguyen, Gilt (CloudStack committer)

17. Other Key Pieces
> config management system: Puppet 3.1
(for first-time run on container, then turned off)
> Cobbler 2.2.3 for kickstarting bare metal servers
(with both Ubuntu and CentOS profiles)
> Jenkins: used for continuous integration and
testing

18. Recent and Ongoing Work
> Testing provisioning LXC containers with different
libvirt versions
>> Up to 1.0.5 on CentOS 6.3 and 0.9.8 on
Ubuntu
> Testing entire application stack on CentOS 6.4
LXC container on Ubuntu host
>> Trying to avoid libvirt issues that came up with
CentOS host

19. Recent and Ongoing Work
(cont.)
> Testing entire application stack on Ubuntu 12.04
LTS LXC container on Ubuntu host
>> Challenge: requires different packaging
mechanism
> Provisioning LXC with and without libvirt to identify
issues with libvirt

20. For More Info:
> Gilt Tech blog: http://tech.gilt.com
> Gilt Twitter: @gilttech
> Gilt Facebook: facebook.com/gilttech
> Work for us: gilt.com/careers/tech
Thanks to:
> CloudStack community
> Gilt Tech team
> NYCDevOps