This case study discusses Globo.com's implementation of an IaaS using Xen Cloud Platform (XCP) and Xen API (XAPI). Key points include automating infrastructure processes and integrating them into a single core application with REST APIs. Benefits of using XCP include pooling of resources, easy API, and large community support. The new system provides a unified front-end and integrates workflow and monitoring into the virtual machine lifecycle.

1. case study
IaaS using XCP and XAPI
marco sinhoreli
virtualization project leader
Monday, May 17, 2010

2. about globo.com
• Internet branch of Globo organizations
• Biggest media conglomerate in Latin America and 4th around the world
• Largest vertical web portals in Brazil
• More than 150 million page views per day
• Average of 27 million unique visitors in February 2010
• More than 1500 physical servers in the production datacenter
• More than 60 Gbps internet link
• Considered benchmark for high-volume web distribution
• Responsible for the largest simultaneous video streaming audiences in the country
Monday, May 17, 2010

3. making decisions
• Automate existing manual workflow between teams’ processes
• Keep existing automated systems used for infrastructure setup
• Identify new infrastructure processes to be automated
• The IaaS needs to fit the company and not the reverse
Monday, May 17, 2010

4. new infrastructure processes
Load
Switches
Balancers
NetworkAPI All infrastructure processes
core
are integrated in a single
R EST
core application responsible
REST
for the flux organization.
Each process has an API
I Monitoring
XAP Core application controlling its physical/
logical/virtual resources.
Physical Servers Virtual Machines Deploy
Monday, May 17, 2010

5. networkAPI
• Relationship between server and switch ports
• needed for adding tagged vlan into switch port trunk when a VM is installed or started
• Switch communication by SNMP or SSH
• check/add/remove/list vlans in switch ports
• Load-balancer communication by API or SNMP or SSH
• management of virtual IPs
• Users profiles
• implement policies for human or system users
Monday, May 17, 2010

6. monitoring
• The monitoring is a part of the VM life-cycle
• when the VM is started all information about the VM is included
in the monitoring system and then enabled
• when the VM is stopped or is tagged as “maintenance”, the
monitoring is disabled
• if the VM is removed, the VM monitoring is removed as well
Monday, May 17, 2010

7. deployment
• Used for internal application deployment
• Install all dependencies needed in OS
• Setup the virtual machine for applications
• Policies for users and groups
• Dev/QA/Staging/Production environment
Monday, May 17, 2010

8. motivation to use XCP
• Pooling
• One place to get status and setup all pool virtual resources
• Easy API
• Fast to program and maintain
• Many client bindings for many flavors
• Paravirtualization
• Large and devoted community
Monday, May 17, 2010

9. IaaS front-end component
• Django application running on Apache
• Users access front-end and authentication
• Unifies all others IaaS components
• Defines all workflow and system interactions
Monday, May 17, 2010

10. IaaS xapi integration component
• Django application running on Apache
• VNC console proxy encapsulated with token to identify client
• Translates REST calls from front-end application to native xapi calls
• All calls to XCP pools are only natively accessible from the integration
component
Monday, May 17, 2010

11. host and VM installation
• Hosts
• PXE boot
• answerfile for unattended installation
• rewrite /etc/firstboot.d files to not change the static IPs
• VM
• kickstart for unattended installation integrated into system
• static IP configured by system inserting information on Linux kernel cmdline
Monday, May 17, 2010

12. system view (traditional)
Monday, May 17, 2010

13. system view (logical)
Monday, May 17, 2010

14. benefits of this new approach
Monday, May 17, 2010

15. workflow
• Defines the workflow between teams
• Integrated with ticket system to request an human action
• Keeps the same flux previously used when the actions weren’t
automated
• Checklist of each item to release the resources
Monday, May 17, 2010

16. user profile
• Each profile has a group of actions in the system
• Some profiles inherit group actions of others
Monday, May 17, 2010

17. user profile
Monday, May 17, 2010

18. user profile
Project
administrator
Monday, May 17, 2010

19. user profile
Monday, May 17, 2010

20. user profile
Monday, May 17, 2010

21. roadmap
• Deployment (Integrate with the IaaS core to tag the deployment in projects)
• DNS (Manage the DNS using an API)
• Monitoring (rich details)
• XCP
• Permit many VMs to use the same shared storage block in web farms
• Master recovery
• Virtual Machine HA
Monday, May 17, 2010

22. thanks
Questions?
Monday, May 17, 2010