Xen in the Cloud

•

0 likes•625 views

Globo.com is the internet branch of Globo Organizations, the largest media conglomerate in Latin America. Globo.com has built a cloud platform called Project Orquestra using the Xen hypervisor to virtualize servers and reduce costs associated with power consumption and data center space. Project Orquestra uses queuing, authorization, and network services to manage virtual machine and resource requests across datacenters. Globo.com plans to contribute Project Orquestra to OpenStack and make it open source to help the OpenStack project.

Technology

globo Xen in the Cloud
.com Marco Sinhoreli
marco.sinhoreli@corp.globo.com

In a globalized world, one needs to brave
new routes, break through frontiers, fly in
search of new horizons.

More about us
‣ Internet branch of the Globo Organizations, the
largest Latin America media conglomerate and 4th in
the world
‣ World record owner in internet votes on reality-show
‣ Largest audience for video streaming in the country
‣ Considered benchmark for high-volume web
distribution

The globo.com leads the Brazilian
internet audience in the main segments

Are 11 years
building systems to support millions of users

‣ High performance hypervisor
‣ Security and reliability build-in and strong isolation
‣ Ready to cloud, standard-based API
‣ Proven technology and credibility
‣ Fully-engaged and global developer community
‣ Many leader technology vendors commited

Energy save in the cloud
Power consumption average

300

225

150

75

watts/hour
0

Stand-alone Server (250 w/h) Virtualized Server (20 w/h)

Data Center occupation
Occupied space in percent

100

75

50

25

Percent
0

Stand-alone Server Virtualized Server

‣ Cooling reduce
‣ Fast provisioning
‣ Resource consolidation
‣ Reduced Data Center TCO
‣ Increased ROI of projects

‣ Deﬁnes the polices to system access
‣ Each proﬁle has a speciﬁc dashboard to:
‣ Physical resources management
‣ Business resources management
‣ Network resources management
‣ VM administration segmented per project

‣ Sync calls are used to get informations about:
‣ VMs and Hosts
‣ RRDs to create the performance graphs
‣ Async calls are used to request actions like:
‣ VM creation, shutdown, start, stop
‣ Add host and pool creation
‣ Interact with NetAPI to set LBs and Switches

‣ Used to async processes
‣ The subscribers work in paralleling process to
instance resources in the cloud
‣ The users receive real-time information about the
queue consumption
‣ The users can manage the queue stopping,
resuming and removing tasks

‣ Used to manage network physical resources
‣ Include VLANs in the switch port trunk
‣ Apply ACLs
‣ VIP and reals creation
‣ Maintain the relationship between server and
switch ports
‣ Pluggable and vendor agnostic

‣ Layer to contact XCP servers
‣ Expose a VNC proxy with SSL and dynamic token
authentication to hosts and VMs consoles
‣ Translates all rest-full calls to XML-RPC using
xenapi.py

‣ Multi-datacenter support
‣ Integrate with OpenStack project
‣ Expose an API to integrate with internal PaaS and
Data Center management system (Energy, Cooling,
etc)
‣ Multi storage repository support
‣ Dynamic live-migration and host shutdown/start
based in consumption

‣ Better velocity in XAPI requests
‣ Today, doing many requests in a pool, the performance
is harmed
‣ Pool ejection without restart the Host
‣ Inter-pool migration and storage repository shared
between pools
‣ Link the object uuid to the task in an async call
(need a new ﬁeld to connect the objects)
‣ Ex: Async.VM.clean_shutdown(vm)
‣ In this case, the task for the async call, has a ﬁeld with
the Vm uuid
‣ Master recover decision enabled in XCP

‣ All components used in project orquestra are
open source
‣ After executing the roadmap, migrate the project to
OpenStack and turn it open source
‣ Help the OpenStack project with our development
force

Thanks
Marco Sinhoreli
marco.sinhoreli@corp.globo.com

Over the past few years, eBay has been re-architecting its infrastructure, automation tools, applications and processes to move eBay.com from a traditional architecture to a cloud model. During this journey, as we evaluated multiple infrastructure options, public cloud providers, tools and cloud automation solutions, we learned a lot in every domain. This presentation talks about our motivations, the technical options we evaluated, and our choices.

Cloud computingShailaja Patil

JetStor Storage Novus cloud

Gene Leyzarovich

Vps server 2

RicoVierra08

Cloud Computing | Virtualization | Docker | Go Language

Dweep Gogia

Cloud computing

iqa hanafi

How erp financials cloud applications could save you

SAPBPCANALYTICS G.P

Cloud Computing - Why it is so popular

Soham Mondal

Openstack and Reddwarf Overview

Craig Vyvial

Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen

confluent

What's hot

Containers for the Enterprise: It's Not That Simple

Mirantis

Microservices on the Edge

LaunchAny

Fred wuensch total virtualization and cloud computing

m_sushil29

Fred wuensch total virtualization and cloud computingf

Subhash Kumar

Basic Introduction to Cloud Computing

Satyam Jaiswal

Cloud computing and business impact

Jabbar Kiani

cloud storage

obrita youkhane

cloud computing.....

Gopibabu Srungavarapu

Cloud computingInes Bahram

20180307 climb cloud_computing

Lisa Marchioretto

Hybird Cloud - An adoption roadmap

John Georgiadis

The Future of the Cloud

New York Technology Council

Cloud@ebay

Jean-Christophe "JC" Martin

Cloud computingShailaja Patil

JetStor Storage Novus cloud

Gene Leyzarovich

Vps server 2

RicoVierra08

Cloud Computing | Virtualization | Docker | Go Language

Dweep Gogia

Cloud computing

iqa hanafi

How erp financials cloud applications could save you

SAPBPCANALYTICS G.P

Cloud Computing - Why it is so popular

Soham Mondal

What's hot (20)

Containers for the Enterprise: It's Not That Simple

Microservices on the Edge

Fred wuensch total virtualization and cloud computing

Fred wuensch total virtualization and cloud computingf

Basic Introduction to Cloud Computing

Cloud computing and business impact

cloud storage

cloud computing.....

Cloud computing

20180307 climb cloud_computing

Hybird Cloud - An adoption roadmap

The Future of the Cloud

Cloud@ebay

Cloud computing

JetStor Storage Novus cloud

Vps server 2

Cloud Computing | Virtualization | Docker | Go Language

Cloud computing

How erp financials cloud applications could save you

Cloud Computing - Why it is so popular

Similar to Xen in the Cloud

Openstack and Reddwarf Overview

Craig Vyvial

Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen

confluent

Day in the life event-driven workshop

Christina Lin

Oracle VM - the Heart of Oracle CloudSai Janakiram Penumuru

FreeSWITCH as a Microservice

Evan McGee

01282016 Aerospike-Docker webinar

Aerospike, Inc.

In this talk we review what Docker is and why it’s important to Developers, Admins and DevOps when they are using a NoSQL Database such as Aerospike, the high performance NoSQL Database. Persistence is a critical element for a successful multi-Container strategy. We also cover the following topics: Using Docker to Orchestrate a multi container application (Flask + Aerospike) Injecting HAProxy and other production requirements as we deploy to production Scaling the Web and Aerospike clusters to grow to meet demand This presentation led by Alvin Richards, VP of Product at Aerospike includes an interactive demo showcasing the core Docker components (Machine, Engine, Swarm and Compose) along with Aerospike’s integration. We hope you will see how much simpler Docker can make building and deploying multi-node Aerospike based applications.

How to run a bank on Apache CloudStack

gjdevos

VMworld 2013: Keep it Simple and Integrated - Out-of the Box Cross-System Aut...

VMworld

Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...

confluent

Microservices, events, containers, and orchestrators are dominating our vernacular today. As operations teams adapt to support these technologies in production, cloud-native platforms like Pivotal Cloud Foundry and Kubernetes have quickly risen to serve as force multipliers of automation, productivity and value. Apache Kafka® is providing developers a critically important component as they build and modernize applications to cloud-native architecture. This talk will explore: • Why cloud-native platforms and why run Apache Kafka on Kubernetes? • What kind of workloads are best suited for this combination? • Tips to determine the path forward for legacy monoliths in your application portfolio • Demo: Running Apache Kafka as a Streaming Platform on Kubernetes

DIMT 2023 SG - Hands-on Workshop_ Getting started with Confluent Cloud.pdf

confluent

Bridge to Cloud: Using Apache Kafka to Migrate to AWS

confluent

Watch this talk here: https://www.confluent.io/online-talks/bridge-to-cloud-apache-kafka-migrate-aws Speakers: Priya Shivakumar, Director of Product, Confluent + Konstantine Karantasis, Software Engineer, Confluent + Rohit Pujari, Partner Solutions Architect, AWS Most companies start their cloud journey with a new use case, or a new application. Sometimes these applications can run independently in the cloud, but often times they need data from the on premises datacenter. Existing applications will slowly migrate, but will need a strategy and the technology to enable a multi-year migration. In this session, we will share how companies around the world are using Confluent Cloud, a fully managed Apache Kafka service, to migrate to AWS. By implementing a central-pipeline architecture using Apache Kafka to sync on-prem and cloud deployments, companies can accelerate migration times and reduce costs. In this online talk we will cover: •How to take the first step in migrating to AWS •How to reliably sync your on premises applications using a persistent bridge to cloud •Learn how Confluent Cloud can make this daunting task simple, reliable and performant •See a demo of the hybrid-cloud and multi-region deployment of Apache Kafka

#VMUGMTL - Xsigo Breakout

1CloudRoad.com

Using Databases and Containers From Development to Deployment

Aerospike, Inc.

Accelerating Public Cloud Migration with Multi-Cloud Load Balancing

Avi Networks

Watch webinar on-demand https://info.avinetworks.com/webinars/accelerating-public-cloud-migration Avi Networks, now part of VMware, has helped many enterprises speed up their migration to the public clouds including Azure, AWS and Google Cloud. Learn why customers, especially those with a multi-cloud strategy, are choosing VMware NSX Advanced Load Balancer (by Avi Networks) to drive end-to-end automation, simpler operations and deeper visibility for load balancing and web application firewall. In this webinar, we will cover: - An insight into modern software load balancer architecture - A step-by-step guide to migrating applications to public clouds - Best practices for automating enterprise-grade load balancing - On-demand, elastic autoscaling of applications based on real-time analytics

OpenStack in an Ever Expanding World of Possibilities - Vancouver 2015 Summit

Lew Tucker

Over the past several years we have seen the continued adoption of OpenStack and it’s expansion into new areas: from cloud service providers, enterprise private clouds to large media companies, telecommunication giants, and big science. At the same time, open source based platforms for network functions virtualization (NFV) are fueling a movement toward cloud computing in almost all major telco’s. In the developer world, open source projects, such as Docker, Mesos, Kubernetes, and Spark are gaining a lot of attention and being integrated into OpenStack through projects Kolla and Magnum. This session will cover how these projects and activities relate to each other and further expand the utility and adoption OpenStack.

Agile Integration Workshop

Judy Breedlove

SD Times - Docker v2Alvin Richards

Understanding and deploying Network Virtualization

SDN Hub

Analogous to server virtualization, Network Virtualization decouples and isolates virtual networks (i.e. tenant) from the underlying network hardware. One of the key value propositions of Software-Defined Networking (SDN) is to enable the provisioning and operation of virtual networks. This tutorial motivates the need for network virtualization, describes the high-level requirements, provides an overview of all architectural approaches, and gives you a clear picture of the vendor landscape. Previously presented at ONUG Fall 2013 and Spring 2014.

China user group keynote

OpenCity Community

Orchestrating Shared Networks, Physical LB and DNS on Cloudstack

Marcus Vinicius Cesário

The adoption of Cloud Computing Solutions in datacenters worldwide has proven to be the right path, as it reduces costs, automates operation tasks and improve time-to-market for newly developed applications. However, cloud solutions are still in an early stage and do not meet all requirements for company environments, especially those with high capacity With that in mind, Globo.com embraced an open cloud solutions, Cloudstack, and began developing together with the community to fulfill all its high availability, high capacity and high throughput requirements. Integration between Cloudstack and Globo.com’s infrastructure happens in all levels, with an special attention to network equipment management, DNS automation and load balancing.

Similar to Xen in the Cloud (20)

Openstack and Reddwarf Overview

Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen

Day in the life event-driven workshop

Oracle VM - the Heart of Oracle Cloud

FreeSWITCH as a Microservice

01282016 Aerospike-Docker webinar

How to run a bank on Apache CloudStack

VMworld 2013: Keep it Simple and Integrated - Out-of the Box Cross-System Aut...

Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...

DIMT 2023 SG - Hands-on Workshop_ Getting started with Confluent Cloud.pdf

Bridge to Cloud: Using Apache Kafka to Migrate to AWS

#VMUGMTL - Xsigo Breakout

Using Databases and Containers From Development to Deployment

Accelerating Public Cloud Migration with Multi-Cloud Load Balancing

OpenStack in an Ever Expanding World of Possibilities - Vancouver 2015 Summit

Agile Integration Workshop

SD Times - Docker v2

Understanding and deploying Network Virtualization

China user group keynote

Orchestrating Shared Networks, Physical LB and DNS on Cloudstack

More from The Linux Foundation

ELC2019: Static Partitioning Made Simple

The Linux Foundation

Static partitioning is used to split an embedded system into multiple domains, each of them having access only to a portion of the hardware on the SoC. It is key to enable mixed-criticality scenarios, where a critical application, often based on a small RTOS, runs alongside a larger non-critical app, typically based on Linux. The two domains cannot interfere with each other. This talk will explain how to use Xen for static partitioning. It will introduce dom0-less, a new Xen feature written for the purpose. Dom0-less allows multiple VMs to start at boot time directly from the Xen hypervisor, decreasing boot times drastically. It makes it very easy to partition the system without virtualization overhead. Dom0 becomes unnecessary. This presentation will go into details on how to setup a Xen dom0-less system. It will show configuration examples and explain device assignment. The talk will discuss its implications for latency-sensitive and safety-critical environments.

XPDDS19: How TrenchBoot is Enabling Measured Launch for Open-Source Platform ...

The Linux Foundation

TrenchBoot is a cross-community OSS integration project for hardware-rooted, late launch integrity of open and proprietary systems. It provides a general purpose, open-source DRTM kernel for measured system launch and attestation of device integrity to trust-centric access infrastructure. TrenchBoot closes the UEFI Measurement Gap and reduces the need to trust system firmware. This talk will introduce TrenchBoot architecture and a recent collaboration with Oracle to launch the Linux kernel directly with Intel TXT or AMD SVM Secure Launch. It will propose mechanisms for integrating the Xen hypervisor into a TrenchBoot system launch. DRTM-enabled capabilities for client, server and embedded platforms will be presented for consideration by the Xen community.

XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...

The Linux Foundation

Artem will briefly cover what has been done since the first talk on Xen in Automotive domain back in 2013, what is going on now and what is still missing for broad adaptation of Xen in vehicles. The following topics will be covered: Embedded/automotive features of Xen Collaboration with AGL and GENIVI organizations for standardization Efforts on Functional Safety compliance Artem will also go over typical automotive use scenarios for Xen which may not be the same as generic computing use of hypervisor.

XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...

The Linux Foundation

XPDDS19 Keynote: Unikraft Weather Report

The Linux Foundation

In recent years unikernels have shown immense performance potential (e.g., boot times of only a few ms, image sizes of only hundreds of KBs).The fundamental drawback of unikernels is that they require that applications be manually ported to the underlying minimalistic OS, needing both expert work and often considerable amount of time. The Unikraft project provides a unikernel code base and build system that significantly simplifies the building of unikernels. In addition to support for a number CPU architectures, languages and frameworks, Unikraft provides debugging and tracing features that are generally sorely missing from unikernel projects. In this talk we will talk about these features, show a set of preliminary performance numbers, and provide a roadmap for the project's future.

XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...

The Linux Foundation

XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx

The Linux Foundation

This talk will introduce Dom0-less: a new way of using Xen to build mixed-criticality solutions. Dom0-less is a Xen feature that adds a novel approach to static partitioning based on virtualization. It allows multiple domains to start at boot time directly from the Xen hypervisor, decreasing boot times dramatically. Xen userspace tools, such as xl and libvirt, become optional. Dom0-less extends the existing device tree based Xen boot protocol to cover information required by additional domains. Binaries, such as kernels and ramdisks, are loaded by the bootloader (u-boot) and advertised to Xen via new device tree bindings. The audience will learn how to use Dom0-less to partition the system. Uboot and device tree configuration details will be explained to enable the audience to get the most out of this feature. The talk will include a status update and details on future plans.

XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...

The Linux Foundation

As the number of contributions grow, reviewer bandwidth becomes a bottleneck; and maintainers are always asking for more help. However, ultimately maintainers must at least Ack every patch that goes in; so if you're not a maintainer, how can you contribute? Why should anyone care about your opinion? This talk will try to lay out some advice and guidelines for non-maintainers, for how they can do code review in a way which will effectively reduce the load on maintainers when they do come to review a patch.

XPDDS19: Memories of a VM Funk - Mihai Donțu, Bitdefender

The Linux Foundation

This talk is a follow-up to our Summit 2017 presentation in which we covered our plans for Intel VMFUNC and #VE, as well as related use-cases. This year, we will provide a report on what we have accomplished in Xen 4.12, and what remains to be addressed. We will also give a brief status update of VMI on AMD hardware. The session will end with some real-world numbers of the Hypervisor Introspection solution running on Citrix Hypervisor 8.0 with #VE enabled.

OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...

The Linux Foundation

Safety certification is one of the essential requirements for software to be used in highly regulated industries. Besides technical and compliance issues (such as ISO 26262 vs IEC 611508) transitioning an existing project to become more easily safety certifiable requires significant changes to development practices within an open source project. In this session, we will lay out some challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the Xen Project has followed thus far and highlight lessons learned along the way. The talk will primarily focus on necessary process, tooling changes and community challenges that can prevent progress. We will be offering an in-depth review of how Xen Project is approaching this challenging goal and try to derive lessons for other projects and contributors.

OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...

The Linux Foundation

Safety certification is one of the essential requirements for software to be used in highly regulated industries. The Xen Project, a secure and stable hypervisor that is used in many different markets, has been exploring the feasibility of building safety certified products on top of Xen for a year, looking at key aspects of its code base and development practices. In this session, we will lay out the motivation and challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the project has followed thus far and highlight lessons learned along the way. The talk will cover technical enablers, necessary process and tooling changes and community challenges offering an in-depth review of how Xen Project is approaching this exciting and and challenging goal.

XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, Citrix

The Linux Foundation

2018 saw fundamental shifts in security boundaries which were previously taken for granted. A lot of work has been done in the past 2 years, and largely in secret under embargo, but there is plenty more work to be done to strengthen the existing mitigations and to try to recover some performance without reopening security holes. This talk will look at speculative execution sidechannels, the work which has already been done to mitigate the security holes, and future work which hopes to bring some improvements.

XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltd

The Linux Foundation

The Arm architecture provides a set of guidelines that any software should abide by when accessing the memory with MMU off and update page-tables. Failing to do so may result in getting TLB conflicts or breaking coherency. In a previous talk ("Keeping coherency on Arm"), we focused on updating safely the stage-2 (aka P2M) page-tables. This talk will focus on the boot code and Xen memory management. During this session, we will introduce some of the guidelines and when they should be used. We will also discuss how Xen boot sequence needs to be reworked to avoid breaking the guidelines.

XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...

The Linux Foundation

For many years the QEMU codebase has contained PV backends for Xen guests, giving them paravirtual access to storage, network, keyboard, mouse, etc. however these backends have not been configurable as QEMU devices as their implementation did not fully adhere to the QEMU Object Model (QOM). Particularly the PV storage backend not using proper QOM devices, or qdevs, meant that the QEMU block layer needed to maintain legacy code that was cluttering up the source. This was causing push-back from the maintainers who did not want to accept any patches relating to that Xen backend until it was 'qdevified'. In this talk, I'll explain the modifications I made to QEMU to achieve 'qdevification' of the PV storage backend, how compatibility with the libxl toolstack was maintained, and what the next steps in both QEMU and libxl development should be.

XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&D

The Linux Foundation

PCI is a local computer bus for attaching hardware devices in a computer, and is the main peripheral bus on modern x86 systems. As such, having a proper way to emulate it is crucial for Xen to be able to expose both fully emulated devices or passthrough devices to guests. This talk will focus on the current status of PCI emulation in Xen, how and where it is used, what are its main limitations and future plans to improve it in order to be more robust and modular.

XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems

The Linux Foundation

XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...

The Linux Foundation

Xen is a very powerful hypervisor with a talented and diverse developers community. Despite the fact it's almost everywhere (from the Cloud to the embedded world), it can be difficult to set up and manage as a system administrator. General purpose distros have Xen packages, but that's just a start in your Xen journey: you need some tooling and knowledge to have a working and scalable platform. XCP-ng was built to overcome those issues: by bringing Xen to the masses with a fully turnkey distro with Xen as its core. It's the logical sequel to the XCP project, with a community focus from the start. We'll see how it happened, what we did, and what's next. Finally, we'll see the impact of XCP-ng on the Xen Project.

XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...

The Linux Foundation

XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...

The Linux Foundation

High level toolstacks for server and cloud virtualization are very mature with large communities using and supporting them. Client virtualization is a much more niche community with unique requirements when compared to those found in the server space. In this talk, we’ll introduce a client virtualization toolstack for Xen (redctl) that we are using in Redfield, a new open-source client virtualization distribution that builds upon the work done by the greater virtualization and Linux communities. We will present a case for maturing libxl’s Go bindings and discuss what advantages Go has to offer for high level toolstacks, including in the server space.

XPDDS19: Core Scheduling in Xen - Jürgen Groß, SUSE

The Linux Foundation

Today Xen is scheduling guest virtual cpus on all available physical cpus independently from each other. Recent security issues on modern processors (e.g. L1TF) require to turn off hyperthreading for best security in order to avoid leaking information from one hyperthread to the other. One way to avoid having to turn off hyperthreading is to only ever schedule virtual cpus of the same guest on one physical core at the same time. This is called core scheduling. This presentation shows results from the effort to implement core scheduling in the Xen hypervisor. The basic modifications in Xen are presented and performance numbers with core scheduling active are shown.

More from The Linux Foundation (20)