OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack


Published on

This talk wants to introduce you to Synnefo, an open source, scalable and production ready cloud stack. It consists of a vm management layer which is google ganeti and is essentially a cluster manager and delivers an integrated management of compute, network and storage. It also runs on of the shelf hardware and delivers live migration without or with shared storage.
Synnefo is the orchestration and presentation layer which talks to ganeti via an API. To the users it presents a simple web UI and also exposes an openstack compatible API for automation called cyclades.
Also it is possible to do flexible, L2 isolated networking.
Also it provides a “dropbox” like storage services called pithos. It features a sync client for different platforms and a web UI.

Synnefo offers all the benefits of an amazon like cloud but is geared towards persistent virtual machines. Also it is perfect as a replacement of the traditional virtualization stack and is easy to setup and use.

Published in: Software, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack

  1. 1. Introduction to Synnefo open source cloud stack Thomas Schend, Systems Architect, CompuGroup Medical AG Open Source Data Center Conference 2014 08. – 10. April, Berlin 4/24/2014 OSDC 20141
  2. 2. Agenda 4/24/2014 OSDC 20142 Introduction What do we need from IaaS? Introducing to Synnefo Components Ganeti Demo Q & A
  3. 3. successfully worldwide in 35 countries for approx. 385,000 customers 24.04.2014 OSDC 20143  = 19 countries with own offices  = additional 16 countries with customers 2012 Guidance 2013 Customers 385.000 Group sales € 451 Mio. € 458 – 463 Mio. EBITDA € 105 Mio. € 97 – 100 Mio. employees 3.700 Indicators Best provider of IT solutions for higher quality and efficiency in health care Worldwide leader for MIT (Medical Information Technology) High profitability and an average of 30% growth p.a. Objectives
  4. 4. Bio 4/24/2014 OSDC 20144 Systems Architect, responsible for Infrastructure Design OSS Enthusiast LPIC-2,MCITP,VCP,CCA,DCIE,… Contact: thomas.schend@gmail.com
  5. 5. Agenda 4/24/2014 OSDC 20145 Introduction What do we need from IaaS? Introducing to Synnefo Components Ganeti Demo Q & A
  6. 6. What do we need from IaaS? 4/24/2014 OSDC 20146 Reliable Scalable (Backend and VMs) Easy to manage Long lived workloads, not all VMs are EC2 instances ;-)
  7. 7. “VMs are not cattle, they are pets” Vangelis Koukis, Technical Lead, Synnefo Quote 4/24/2014 OSDC 20147
  8. 8. Agenda 4/24/2014 OSDC 20148 Introduction What do we need from IaaS? Introducing to Synnefo Components Ganeti Demo Q & A
  9. 9. Synnefo 4/24/2014 OSDC 20149 Open Source Production-quality IaaS made for persistent VMs commodity hardware – No SAN Large-scale Installations simple UI for the end-user uses Ganeti as a Cluster Manager
  10. 10. Synnefo History 4/24/2014 OSDC 201410 Design started late 2010 Production since July 2011 ~okeanos* public cloud running currently ~8700 VMs *http://okeanos.grnet.gr
  11. 11. Synnefo Layers 4/24/2014 OSDC 201411 Image provided by GRNET
  12. 12. Synnefo birdseye view 4/24/2014 OSDC 201412 Image provided by GRNET
  13. 13. Synnefo 4/24/2014 OSDC 201413 Build with Ganeti as a backend Astakos: Identity/Account services Cyclades: Compute/Network/Image/Volume services Pithos: File/Object Storage service snf-manage commandline tool Basic helpdesk tool snf-burnin / snf-burnin-run kamaki commandline client Code on github
  14. 14. Agenda 4/24/2014 OSDC 201414 Introduction What do we need from IaaS? Introducing to Synnefo Components Ganeti Demo Q & A
  15. 15. Synnefo Architecture I 4/24/2014 OSDC 201415 Image provided by GRNET
  16. 16. Synnefo features - Astakos 4/24/2014 OSDC 201416 Identity service Provides Dashboard Track Resource usage Project Management Multiple Auth methods Twitter / Google / LinkedIn / LDAP / Active-directory
  17. 17. Synnefo features - Cyclades 4/24/2014 OSDC 201417 Network management L3 Routed/L2 VLANS/L2 isolation with mac filter nfdhcpd Archipelago storage abstraction ceph/NFS/Gluster backend very fast, thin cloning Snapshots
  18. 18. Synnefo features - Pithos 4/24/2014 OSDC 201418 Object storage service Efficient syncing Hashing before upload Deduplication in backend Web UI written in GWT Clients for Windows, Macos and IOS Sharing public and private
  19. 19. Synnefo / Ganeti Communication 4/24/2014 OSDC 201419 Image provided by GRNET
  20. 20. written in python / django apache/gunicorn postgres database snf-gnt-eventd / snf-progress-mon rabbitmq snf-dispatcher snf-vncauthproxy build for wheezy, can run fully virtualized speaks openstack API but does not share any code Synnefo Architecture II 4/24/2014 OSDC 201420
  21. 21. Secure image deployment of untrusted images Isolated from the ganeti node Supports Linux,Windows and BSD customization tasks Assign hostname Delete SSH keys Change password Resize Filesystem Run unattended install (windows) Inject files Synnefo - snf-image 4/24/2014 OSDC 201422
  22. 22. Synnefo snf-image deployment process 4/24/2014 Central IT Services23 gnt-node evacuate -I hail node3 ImageprovidedbyGRNET
  23. 23. Agenda 4/24/2014 OSDC 201424 Introduction What we need from IaaS? Introducing to Synnefo Components Ganeti Demo Q & A
  24. 24. Ganeti History 4/24/2014 OSDC 201425 Very mature, production ready opensourced since August 30, 2007 by google uses Mature open source tools LVM,DRBD,python,haskell Different hypervisors (Xen,KVM,LXC) Low overhead for cluster management
  25. 25. Ganeti Usage at google 4/24/2014 OSDC 201426 Usage at google small Clusters in Offices (OIB) large Clusters in Datacenters for virtual desktops mostly using XEN Not used for user facing services
  26. 26. Ganeti features I 4/24/2014 Central IT Services27 Scale from 1-200 nodes and beyond Live migration without shared storage Noderoles: master,master candidate,regular nodes Nodegroups OVF import tool Backup with included tool Job queue Commands start with gnt-
  27. 27. Ganeti features II 4/24/2014 OSDC 201428 OS Interface to provision new instances snf-image secure image deployment ganeti-instance-debootstrap, image, ganeti-os-defs Htools to balance clusters, plan capacity and do maintenance supported file,plain,sharedfile,ceph,NFS,Gluster Ext storage provider with plugins for Archipelago, HP EVA,,IBM Storwize
  28. 28. Ganeti advanced features 4/24/2014 OSDC 201429 Hooks for every action (pre/post) Tags for every object Exclusion tags to keep machines apart from each other Deploy flash in every node for high performance (flashcache,bcache,dm-cache) OOB management interface Ganeti watcher
  29. 29. Ganeti advantages 4/24/2014 OSDC 201430 No need for special hardware (SAN) But SAN/Shared Storage is supported Manage cluster as an entity not single nodes (libvirt) Few dependencies No database Easy recovery in case of hardware outage No SPOF which can affect the reliability of VMs Very good documentation
  30. 30. Deployment automation puppet,chef,cfengine,.. Monitoring icinga/nagios/check_mk/… Openvswitch LACP + 10GBe Autorepair Packages for all major OS debian,ubuntu,rhel/centos Ganeti best practices 4/24/2014 OSDC 201431
  31. 31. Ganeti management interfaces 4/24/2014 OSDC 201432 cli with bash completion LUXI RAPI Different web UIs using the RAPI Ganeti web manager Ganetimgr Synnefo
  32. 32. Ganeti quick command reference 4/24/2014 OSDC 201433 gnt-cluster init / info / verify / command / copyfile gnt-node add / list / failover / evacuate gnt-instance add / info / replace-disks gnt-job list … gnt-network … gnt-backup … …
  33. 33. Ganeti htools - allocation and placement 4/24/2014 OSDC 201434 written in haskell hail hbal hspace harep hinfo hroller …
  34. 34. Typical Ganeti Clsuter 4/24/2014 OSDC 201435 Node01 Node02 Node03 gnt-cluster init [-s ip] --enabled-hypervisors=kvm cluster gnt-node add [-s ip] node2 gnt-node add [-s ip] node3 gnt-cluster verify gnt-instance add -t drbd -I hail -o debootstrap+default vm Veeamvirtualizationstencils
  35. 35. Ganeti Failover and Repair 4/24/2014 OSDC 201436 gnt-node modify -O yes node01 gnt-node failover --ignore-consistency node01 gnt-node evacuate -I hail node01 Node01 Node02 Node03 Veeamvirtualizationstencils
  36. 36. “Ganeti didn't have any generally known meaning before this project. And now it means kick-ass cluster management” Martin Beauchamp, on the Ganeti Users List Quote 4/24/2014 OSDC 201437
  37. 37. Agenda 4/24/2014 OSDC 201438 Introduction What do we need from IaaS? Introducing to Synnefo Components Ganeti Demo Q & A
  38. 38. Login to Astakos syncing and Pithos UI and App Create VMs with Cyclades UI Snapshot Feature OOB Console Hotplug of NICs and Networks Floating IPs Clean-up Demo 4/24/2014 OSDC 201439
  39. 39. Agenda 4/24/2014 OSDC 201440 Introduction What do we need from IaaS? Introducing to Synnefo Components Ganeti Demo Q & A
  40. 40. http://www.synnefo.org https://github.com/grnet/synnefo https://code.google.com/p/ganeti/ Try it! 4/24/2014 OSDC 201441
  41. 41. 4/24/2014 OSDC 201442 Q & A