2013/10/04
Leveraging OpenStack
for high performance clouds
Simon Pasquier
Slide 2/18
Outline
Introduction
XLcloud mission & architecture
XLcloud integration with OpenStack
Operating an OpenStack c...
Slide 3/18
Who am I?
Software engineer working at Bull SAS on the XLcloud project
OpenStack tester & contributor
Focusing ...
Slide 4/18
XLcloud project
http://xlcloud.org/
A three-year long collaborative project
Funded by the French FSN (Fonds nat...
Slide 5/18
Mission statement of XLcloud
The XLcloud project strives to establish the demonstration
of a High Performance C...
Slide 6/18
Common HPCC issues
Issues
● Virtualization overhead (5% to 30%)
especially for I/O
● Large data volume transfer...
Slide 7/18
Designed for DevOps minded users
to create and manage the life-cycle
of multi-tier middleware stacks
Vertically...
Slide 8/18
Integration of XLcloud with OpenStack
Slide 9/18
Walkthrough of stack creation
Slide 10/18
Operating an OpenStack cloud
You'll have to make some choices
Start with baby steps
Don't forget to take into ...
Slide 11/18
Which OS?
Most people use one of these:
Ubuntu (LTS!)
Debian
RedHat and derivatives
How to choose which distro...
Slide 12/18
Which installation method?
Several installers exist and might be helpful
But
Not ideal if you drift away from ...
Slide 13/18
Which network stack?
For new deployments, go with Neutron
OVS plugin ok for PoC, small and non-critical deploy...
Slide 14/18
Other stuff to consider
SQL backend
Keystone backend (SQL or memcache)
Message queue
Storage backend
Hyperviso...
Slide 15/18
Operating your cloud
Monitor your infrastructure
Identify the bottlenecks
Tune your configuration (both OpenSt...
Slide 16/18
Contributing to OpenStack
Very dynamic community with many skilled and creative people
Mailing lists and IRC c...
Slide 17/18
XLcloud efforts wrt OpenStack
Climate project
Ceilometer (KWAPI)
Heat
Nova (CPU affinity, GPU)
Neutron (Infini...
Thank you for your attention
Upcoming SlideShare
Loading in …5
×

Leveraging OpenStack for high performance clouds

471 views

Published on

Presentation on XLcloud that I've done at the Open World Forum 2013 in Paris. The talk happened during the "Open Source on Clouds, Making Cloud Interoperability a Reality" track organized by OW2 (http://www.openworldforum.org/en/tracks/7).

The XLcloud project strives to establish the demonstration of a High Performance Cloud Computing (HPCC) platform based on OpenStack, that is designed to run a representative set of compute intensive workloads, including more specifically interactive games, interactive simulations and 3D graphics. We will show how we build our solution on top of OpenStack, which services we use and how we contribute to the OpenStack project to address the requirements related to the HPC context.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
471
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The platform exposes an eXtended Infrastructure-as-a-Service REST API endpoint and GUI (we call XIaaS) which ops-minded developers can use to manage software stacks in the cloud of nearly any scale and complexity without sacrificing control. The main design paradigm is to automate and orchestrate the complete life-cycle management based on the declarative description of software a stack's end-state. This includes resource provisioning, configuration management, application deployment, software updates, monitoring, access control, high availability and auto-scaling The software stack artifacts produced by XLcloud are primarily targeted toward delivering two kinds of virtual cluster infrastructures but could easily be applied to other types of virtual cluster infrastructures: An HPC Virtual Cluster based on SLURM A Remote Rendering Virtual Cluster based on a high optimized graphic pipeline
  • Leveraging OpenStack for high performance clouds

    1. 1. 2013/10/04 Leveraging OpenStack for high performance clouds Simon Pasquier
    2. 2. Slide 2/18 Outline Introduction XLcloud mission & architecture XLcloud integration with OpenStack Operating an OpenStack cloud Contributing to OpenStack
    3. 3. Slide 3/18 Who am I? Software engineer working at Bull SAS on the XLcloud project OpenStack tester & contributor Focusing on Networking (Neutron) Orchestration (Heat)
    4. 4. Slide 4/18 XLcloud project http://xlcloud.org/ A three-year long collaborative project Funded by the French FSN (Fonds national pour la Société Numérique) programme, started early 2012 Apache 2.0 license Members ● Bull SAS ● Institut Telecom (Telecom Sud Paris) ● Silkan ● CEA List ● EISTI ● ATEME ● OW2 ● Inria RESO
    5. 5. Slide 5/18 Mission statement of XLcloud The XLcloud project strives to establish the demonstration of a High Performance Cloud Computing (HPCC) platform based on OpenStack that is designed to run a representative set of compute intensive workloads including more specifically interactive games, interactive simulations and 3D graphic applications.
    6. 6. Slide 6/18 Common HPCC issues Issues ● Virtualization overhead (5% to 30%) especially for I/O ● Large data volume transfers ● Low latency / high bandwidth networks ● Lack of mature management, orchestration and monitoring features ● Security concerns (data privacy, authorization) Solutions ● HVM, LXC, bare-metal instances ● GPU support for compute and graphics ● High performance full-bisection network with traffic control ● GUI & REST API (XMS) to manage and monitor full software stacks ● Strong and standard AuthN / AuthZ protocols ● Instances can be allocated on reserved, dedicated and homogeneous HW with CPU affinity
    7. 7. Slide 7/18 Designed for DevOps minded users to create and manage the life-cycle of multi-tier middleware stacks Vertically integrated with the OpenStack CMS and physical infra Advanced Identity & Access Management integration XLcloud platform architecture in a nutshell...
    8. 8. Slide 8/18 Integration of XLcloud with OpenStack
    9. 9. Slide 9/18 Walkthrough of stack creation
    10. 10. Slide 10/18 Operating an OpenStack cloud You'll have to make some choices Start with baby steps Don't forget to take into account Capacity planning Monitoring Redundancy Data recovery process
    11. 11. Slide 11/18 Which OS? Most people use one of these: Ubuntu (LTS!) Debian RedHat and derivatives How to choose which distro is right for me? Check Open vSwitch, KVM, libvirt, etc. versions Most important is your existing expertise
    12. 12. Slide 12/18 Which installation method? Several installers exist and might be helpful But Not ideal if you drift away from mainstream usage For serious workloads or when using bleeding-edge features, be ready to install from sources as well as patch & fix Invest in a configuration management system (Puppet, Chef, whatever) Be warned, upgrades are painful (but mandatory)!
    13. 13. Slide 13/18 Which network stack? For new deployments, go with Neutron OVS plugin ok for PoC, small and non-critical deployments Pending issues: Redundancy Stability & scalability Support of IPv6 both for the infra and the virtual But migrating from nova-network to Neutron will be hard
    14. 14. Slide 14/18 Other stuff to consider SQL backend Keystone backend (SQL or memcache) Message queue Storage backend Hypervisors KVM is the most used In our case, GPU support is a must
    15. 15. Slide 15/18 Operating your cloud Monitor your infrastructure Identify the bottlenecks Tune your configuration (both OpenStack and infra) Patch/fix if required Consider having a pre-prod/staging platform
    16. 16. Slide 16/18 Contributing to OpenStack Very dynamic community with many skilled and creative people Mailing lists and IRC channels are really important to learn, build relationships and credibility Don't wait for features to land, be proactive and show that you care Test new features Report bugs Write documentation Fix bugs
    17. 17. Slide 17/18 XLcloud efforts wrt OpenStack Climate project Ceilometer (KWAPI) Heat Nova (CPU affinity, GPU) Neutron (Infiniband)
    18. 18. Thank you for your attention

    ×