This document discusses Apache Deltacloud, which allows clients to interact with different cloud platforms through common APIs. Deltacloud supports the EC2 and CIMI standards. It can act as an EC2 frontend to interface with clouds like OpenStack. Deltacloud also implements the CIMI standard developed by DMTF to provide a common REST API for cloud resources across platforms.
Cortex: Horizontally Scalable, Highly Available PrometheusGrafana Labs
In this talk we present Cortex - a horizontally scalable, highly available Prometheus implementation. Like Prometheus, Cortex is a CNCF (sandbox) project.
Cortex turns a lot of the Prometheus architectural assumptions on its head, by marrying a scale-out PromQL query engine with a storage layer based on NOSQL databases such as Bigtable, DynamoDB and Cassandra. We have disaggregated the Prometheus binary into a microservices-style architecture, with separate services for query, ingest, alerting and recording rules. By designing all these services as fungible replicas, this solution can be scaled out with ease and failure of any individual replica can be dealt with gracefully.
OpenNebulaConf2017EU: Elastic Clusters for Data Analysis by Carlos de Alfonso...OpenNebula Project
EUBra-BIGSEA is an european project aiming at developing cloud services for Big Data Analysis in the context of Traffic Recommendation. EUBra-BIGSEA leverages EC3 (Elastic Compute Cloud Clusters - ref) to deploy virtual elastic clusters managed by Apache Mesos, on top of an OpenNebula site.
YouTube: https://youtu.be/xShKKrEMTuQ
We are using Elasticsearch to power the search feature of our public frontend, serving 10k queries per hour across 8 markets in SEA.
Here we are sharing our experiences of running Elasticsearch on Kubernetes, presenting our general setup, configuration tweaks and possible pitfalls.
OpenNebulaConf2017EU: Hyper converged infrastructure with OpenNebula and Ceph...OpenNebula Project
Hyperconvergence is one of the big topics in datacenters at the moment. But is it more than an old wine in new bottles? Why we at Runtastic built an hyperconverged datacenter based on Opennebula with Ceph and what we learned.
YouTube: https://youtu.be/50Z4bmevTpg
Cortex: Horizontally Scalable, Highly Available PrometheusGrafana Labs
In this talk we present Cortex - a horizontally scalable, highly available Prometheus implementation. Like Prometheus, Cortex is a CNCF (sandbox) project.
Cortex turns a lot of the Prometheus architectural assumptions on its head, by marrying a scale-out PromQL query engine with a storage layer based on NOSQL databases such as Bigtable, DynamoDB and Cassandra. We have disaggregated the Prometheus binary into a microservices-style architecture, with separate services for query, ingest, alerting and recording rules. By designing all these services as fungible replicas, this solution can be scaled out with ease and failure of any individual replica can be dealt with gracefully.
OpenNebulaConf2017EU: Elastic Clusters for Data Analysis by Carlos de Alfonso...OpenNebula Project
EUBra-BIGSEA is an european project aiming at developing cloud services for Big Data Analysis in the context of Traffic Recommendation. EUBra-BIGSEA leverages EC3 (Elastic Compute Cloud Clusters - ref) to deploy virtual elastic clusters managed by Apache Mesos, on top of an OpenNebula site.
YouTube: https://youtu.be/xShKKrEMTuQ
We are using Elasticsearch to power the search feature of our public frontend, serving 10k queries per hour across 8 markets in SEA.
Here we are sharing our experiences of running Elasticsearch on Kubernetes, presenting our general setup, configuration tweaks and possible pitfalls.
OpenNebulaConf2017EU: Hyper converged infrastructure with OpenNebula and Ceph...OpenNebula Project
Hyperconvergence is one of the big topics in datacenters at the moment. But is it more than an old wine in new bottles? Why we at Runtastic built an hyperconverged datacenter based on Opennebula with Ceph and what we learned.
YouTube: https://youtu.be/50Z4bmevTpg
CloudConnect 云计算大会 China 2016
OpenStack Swift的性能调优
Performance Tuning of OpenStack Swift
李明宇 奥思数据 创始⼈& CTO
Email: li.mingyu@ostorage.com.cn
Swift is good at storing small objects like photos and documents.
Scheduling a Kubernetes Federation with AdmiraltyIgor Sfiligoi
Presented at OSG All-Hands Meeting 2020 - USCMS-USATLAS Session.
This talk presented the PRP experience with using Admiralty as a Kubernetes federation solution, with both discussion of why we need it, why Admiralty is the best (if not actually the only) solution for our needs, and how it works.
Persistent Storage with Containers with Kubernetes & OpenShiftRed Hat Events
Manually configuring mounts for containers to various network storage platforms and services is tedious and time consuming. OpenShift and Kubernetes provides a rich library of volume plugins that allow authors of containerized applications (Pods) to declaratively specify what the storage requirements for the containers are so that OpenShift can dynamically provision and allocate the storage assets for the specified containers. As the author of the Kubernetes Persistent Volume specification, I will provide an overview of how Persistent Volume plugins work in OpenShift, demo block storage and file storage volume plugins and close with the Red Hat storage roadmap.
Presented at LinuxCon/ContainerCon by Mark Turansky, Principal Software Engineer, Red Hat
Mark Turansky is a Principal Software Engineer at Red Hat and a full-time contributor to the Kubernetes Project. Mark is the author of the Kubernetes Persistent Volume specification and a member of the Red Hat OpenShift Engineering team.
OpenNebula Conf 2014 | Understanding the OpenNebula Model for Cloud Provision...NETWAYS
OpenNebula’s quest for simplicity touches every aspect of the software,and one of the greater effort has been put into the provisioning model. A smooth experience for users entails a proper design of the concepts, ironing out the flow of day to day operations, as well as proper tools for the administrators to manage its cloud.
For this reason, OpenNebula features Virtual Datacenters (vDCs), which are containers for the execution of virtual machines, as well as a way of hiding physical resources from group members. Three actors are identified in this model: the cloud administrator, the vDC administrator and the end user. In this talk we will see how this vDCs are created, how physical resources are associated to them and administrators of the vDCs are managed and given permissions. All this topped with an excellent interface, the Cloud View, adapted for each of the actors.
For Elasticsearch users, backups are done using the Elasticsearch snapshot facility. In this presentation I'll go through the design of an Elasticsearch backup system that you can use to create snapshots of your cluster's indices and documents.
Ceph & OpenStack talk given @ OpenStack Meetup @ Bangalore, June 2015Deepak Shetty
Talk that showcases the advantages of using Ceph as the storage of choice in openstack. It shows how Ceph integrates with all openstack storage services and the adv of using Ceph as __the__ Unified Storage solution for Openstack
CloudConnect 云计算大会 China 2016
OpenStack Swift的性能调优
Performance Tuning of OpenStack Swift
李明宇 奥思数据 创始⼈& CTO
Email: li.mingyu@ostorage.com.cn
Swift is good at storing small objects like photos and documents.
Scheduling a Kubernetes Federation with AdmiraltyIgor Sfiligoi
Presented at OSG All-Hands Meeting 2020 - USCMS-USATLAS Session.
This talk presented the PRP experience with using Admiralty as a Kubernetes federation solution, with both discussion of why we need it, why Admiralty is the best (if not actually the only) solution for our needs, and how it works.
Persistent Storage with Containers with Kubernetes & OpenShiftRed Hat Events
Manually configuring mounts for containers to various network storage platforms and services is tedious and time consuming. OpenShift and Kubernetes provides a rich library of volume plugins that allow authors of containerized applications (Pods) to declaratively specify what the storage requirements for the containers are so that OpenShift can dynamically provision and allocate the storage assets for the specified containers. As the author of the Kubernetes Persistent Volume specification, I will provide an overview of how Persistent Volume plugins work in OpenShift, demo block storage and file storage volume plugins and close with the Red Hat storage roadmap.
Presented at LinuxCon/ContainerCon by Mark Turansky, Principal Software Engineer, Red Hat
Mark Turansky is a Principal Software Engineer at Red Hat and a full-time contributor to the Kubernetes Project. Mark is the author of the Kubernetes Persistent Volume specification and a member of the Red Hat OpenShift Engineering team.
OpenNebula Conf 2014 | Understanding the OpenNebula Model for Cloud Provision...NETWAYS
OpenNebula’s quest for simplicity touches every aspect of the software,and one of the greater effort has been put into the provisioning model. A smooth experience for users entails a proper design of the concepts, ironing out the flow of day to day operations, as well as proper tools for the administrators to manage its cloud.
For this reason, OpenNebula features Virtual Datacenters (vDCs), which are containers for the execution of virtual machines, as well as a way of hiding physical resources from group members. Three actors are identified in this model: the cloud administrator, the vDC administrator and the end user. In this talk we will see how this vDCs are created, how physical resources are associated to them and administrators of the vDCs are managed and given permissions. All this topped with an excellent interface, the Cloud View, adapted for each of the actors.
For Elasticsearch users, backups are done using the Elasticsearch snapshot facility. In this presentation I'll go through the design of an Elasticsearch backup system that you can use to create snapshots of your cluster's indices and documents.
Ceph & OpenStack talk given @ OpenStack Meetup @ Bangalore, June 2015Deepak Shetty
Talk that showcases the advantages of using Ceph as the storage of choice in openstack. It shows how Ceph integrates with all openstack storage services and the adv of using Ceph as __the__ Unified Storage solution for Openstack
Beyond Golden Containers: Complementing Docker with Puppetlutter
Often, Docker or more generally containers and immutable infrastructure are viewed as a replacement for configuration management. This talk explains why that is not the case, and that they are in fact complementary.
Containers move the challenges that configuration management solves to different places in the application lifecycle. The talk explains where Puppet fits into this changed lifecycle, and what tools Puppet provides there.
Slides for a talk I gave at the Linux Foundation Colaboration Summit 2015
Introducing containers into your infrastructure brings new capabilities, but also new challenges, in particular around configuration. This talk will take a look under the hood at some of those operational challenges including:
* The difference between runtime and build-time configuration, and the importance of relating the two together.
* Configuration drift, immutable mental models and mutable container file systems.
* Who configures the orchestrators?
* Emergent vs. model driven configuration.
In the process we will identify some common problems and talk about potential solutions.
Talk from PuppetConf 2016
As clouds become commonplace, there is a need to manage virtual instances across a great variety of clouds and cloud-like environments. Aeolus, an open source project recently launched by Red Hat, lives in the space just below where traditional config management tools like Puppet live.
It provides tools for managing virtual instances and images across public, private and hybrid clouds, from image creation to the launch of complex deployments. It enhances existing clouds by providing user management, monitoring and quality-of-service metrics to make intelligent decisions about resource placement, or automate them based on policy, across multiple clouds. All fronted by a friendly self-service web application, with an API in the works.
Kyle Bassett's from @ Arctiq (www.arctiq.ca) Presentation from the Halifax DevOps Meet-up on July.19th - 2017.
Linux Container Platform on Azure
(Kubernetes, OpenShift, Ansible Automation)
Pipeline Automation
(From Code to Containers, Automated CI / CD on Azure
OCCIware Cloud Expo London 2016 - Docker Studio, Studio Factory, erocci bus &...Marc Dutoo
OCCIware @ Cloud Expo London 2016 - introduction to OCCI and OCCIware, year 1 main outputs (Docker Studio & Studio Factory, erocci OCCI Cloud management bus) & Linked Data as a Service (LDaaS)-themed end-to-end demo
OCCIware Year 1 Milestone: Docker Studio, Studio Factory, pluggable XaaS runt...OCCIware
The OCCIware project aims at managing in a unified manner all layers and domains of the Cloud (XaaS), by building on the Open Cloud Computing (OCCI) standard and formal and model-driven approaches.
This session highlights first year outcomes:
- the OCCIware Studio Factory, allowing to produce visually customizable diagram editors for any Cloud configuration business domain modeled in OCCI using the OCCI Extension Studio, such as the flagship Docker Studio ;
- erocci, a scalable OCCI middleware allowing to federate multiple XaaS Cloud runtimes, such as the Roboconf PaaS server and the ActiveEon Cloud Automation multi-IaaS connector are planned to be.
The session includes an end-to-end demo of PaaS and IaaS Cloud deployment, and 2016 plans.
The project, funded by French Ministry of Industry, involves 10 partners and 11 top expert advisors coming from both research and industry.
OpenShift is Red Hat's Platform-as-a-Service (PaaS) that lets developers quickly develop, host, and scale Docker container-based applications. OpenShift enables a uniform and standardised approach to container management across all hosting options including AWS/EC2 and other private/public cloud and on/off-premise variants. At this session, you will learn how Red Hat's enterprise clients are using OpenShift to enable their digital transformation initiatives. Examples will cover how realising a hybrid cloud strategy can simplify and reduce the risk of migrating and transitioning application workloads to containers in the cloud.
Alex Smith, Solutions Architect, Amazon Web Services, ASEAN
Stephen Bylo, Senior Solution Architect, Red Hat Asia Pacific Pte Ltd
OCCIware @ Cloud Computing World 2016 - year 1 milestone & Linked Data demoMarc Dutoo
OCCIware @ Cloud Computing World 2016 - introduction to OCCI and OCCIware, year 1 main outputs (Docker Studio & Studio Factory, erocci OCCI bus) & Linked Data as a Service (LDaaS)-themed end-to-end demo
OSDC 2018 | Three years running containers with Kubernetes in Production by T...NETWAYS
The talk gives a state of the art update of experiences with deploying applications in Kubernetes on scale. If in clouds or on premises, Kubernetes took over the leading role as a container operating system. The central paradigm of stateless containers connected to storage and services is the core of Kubernetes. However, it can be extended to distributed databases, Machine Learning, Windows VMs in Kubernetes. All these applications have been considered as edge cases a few years ago, however, are going more and more mainstream today.
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]Animesh Singh
Kubeflow Pipelines and TensorFlow Extended (TFX) together is end-to-end platform for deploying production ML pipelines. It provides a configuration framework and shared libraries to integrate common components needed to define, launch, and monitor your machine learning system. In this talk we describe how how to run TFX in hybrid cloud environments.
Cluster-as-code. The Many Ways towards KubernetesQAware GmbH
iSAQB Software Architecture Gathering – Digital 2022, November 2022, Mario-Leander Reimer (@LeanderReimer, Principal Software Architect bei QAware).
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Kubernetes is the de-facto standard when it comes to container orchestration. But why is there is no established, standard and uniform way to spin-up and manage a single or even a whole farm of Kubernetes clusters yet? Instead, a whole bunch of different and mostly incompatible ways towards Kubernetes exist today. Each with its own pros and cons in regards to ease of use, flexibility and many other requirements. In this session we will have a closer look at the different available options to create, manage and operate Kubernetes clusters at scale.
2. 2
Overview
● Cloud API's
● What is Apache Deltacloud ?
● What is CIMI ?
● Deltacloud's EC2 frontend
● Using Deltacloud with Openstack
● Questions
3. 3
Cloud API's
● Wild West – every cloud has one
● Material differences
● Annoying innovation
● de facto standards
● EC2: widely used and copied
● OpenStack: excellent governance
● de jure standards
● backed by Standards Developing Organization
● CIMI produced by DMTF
4. 4
Standard challenges
● Build broad community
● Allow standards development the open source way
● Bring proprietary players into the tent
● Governance/licensing
● Adaptability
● Enable wide range of implementations
● Let clients discover variations
8. 8
DMTF CIMI
● Distributed Management Task Force (DMTF)
● Broad industry participation
● Produced OVF and CIM
● Covers wide range of IaaS objects
● Published August 2012
● RESTful API
● Despite the name, not connected to CIM
● CIMI forces Deltacloud to keep state
9. 9
CIMI Model
Systems Machines Volumes Networks
CIMI Entry Point
Systems Machines Volumes Networks
Servers Storage volumesGrouping of
resources
SystemsMetadata
●
Flexibility through discovery
●
Many more supporting resources
●
including monitoring, events, jobs
●
Common patterns (Templates, Configs, Images)
Networks,
routing, ports
11. 11
Example: Creating a Server
POST /machines HTTP/1.1
{ "name": "myMachine1",
"description": "My very first machine",
"machineTemplate": {
"machineConfig": {"href": "/configs/small"},
"machineImage": {"href": "/images/WinXP-SP2"},
"credentials": {"href": "/creds/12345" }
}
}
HTTP/1.1 201 Created
Location: http://cimi.example.org/machines/42
12. 12
What Deltacloud is doing around CIMI
● CIMI frontend
● use against various clouds
● CIMI client web app
● CIMI test suite
● CIMI Ruby client (soon)
13. 13
Deltacloud's EC2 frontend
● Started as POC last spring
● Inspired by AWSOME
● Used by RHEV-M/oVirt
● Basic functionality
● Describe images/AZs
● Instance lifecycle management
● SSH keys
15. 15
Using Deltacloud with Openstack
● Start Deltacloud with Openstack driver
● Point at Keystone
● Users can dynamically switch endpoints
● Deployment options
● Close to user
● Close to Keystone
- with Red Hat for a long long time - Maintainer of Apache Deltacloud - Member of CIMI working group - on DMTF board
API adapter, not cloud RESTful service
m : n cloud API adapter passthrough authentication Three API frontends 'classic' Deltacloud API (RESTful) DMTF CIMI API (RESTful) EC2 Query API