WebCamp 2016: DevOps. Николай Дойков: Опыт создания клауда для потокового видео на docker swarm.

•

0 likes•351 views

История развития и построения решения для потокового видео. Подходы к построению географически распределенного Docker Swarm кластера и остальных компонент данного проекта.

Technology

Elastic streaming Cloud
How we build it .

Speaker
Nick Doikov
Sr.Devops Engineer @
nick.doikov@gmail.com

Introductory
- Over 1000 companies
- Over 4000 separate users
- Over 6000 streaming instances (cloud based and cloud attached)
One

- update mess (application and base OS )
- partial access to client's infrastructure
- To many resources need to support all this infrastructure
- Instances are in active usage approx 10% max (3% in general) time
- Users deploy own local servers (Distro? Kernel?)

- operational expenses
- constant resource freeze (used or not)
- core resources allocation (stats etc ...)
- not suitable for rapid start-stop (can`t be used for instant start on connection)
What we can do with all of this?

Docker containers :
- Can be runned in a seconds
- Real transparent start/stop for end customer
- No os CPU,RAM overhead
- Easily to update via images versioning

Streaming devices with bonding
Bonding and streaming server

Previous flow with binding streaming server (Sputnik) in Amazon EC2

Goals
- launch on device connect
- Stop after streaming finished and all files uploaded
- Centralized logging
- Every time up to date container (no problems with software updates)
- Oversell %)
- Multi regions
- Easy maintain and provisioning new points of presence

What we need
- Static entry point that do not change due device unregister in Core
- Static ports for stream capturing by 3rd party devices

Few notes :
- Consul cluster behind elb
- Swarm nodes announce themselves to Consul
- Regions separated by DOCKER_OPTS Label
- Container starts with region constraint
- Inside container there are monitors that check application status and send
metrics to core.
- After connections from devices and upload connections was closed container
“dies” after 30 minutes idle
- Jobber delete stopped containers each 5 second
- Centralized logging /alerting using async redis pubsub as a buffer and
elasticsearch as a storage

What we get :
- End user value (customer get enough resources exactly when he need it )
- Easy few steps node deploy (can be deployed on cloud providers as well as
on colo)
- Easy reached hybrid cloud model (cloud core and hardware calculation
nodes)
- Oversell %)
- Now we have 9 regions (cloud providers + hardware POP)

Since 2008, Harvard Research Computing has undertaken a significant scaling challenge increasing their available HPC and storage from 200 cores and 20TB to over 70,000 cores and 35PB of storage. James will discuss the journey and the highlights of extending the computing to support world class research and education. During the evolution of the computing platforms at Harvard they also helped to support and build the Massachusetts Green High Performance Computing Center which is a dedicated high performance research computing facility in Holyoke, MA. This facility continues to support large scale research computing with sustainable energy and advanced networking. Recently the NESE project (New England Storage Exchange) was funded by the National Science Foundation. This is a multi-petabyte object store that is supported by the existing MGHPCC facility supporting the region. The Data Science Initiative at Harvard has also been recently announced and will require even further advanced computation to support their research faculty. Now as the world takes a grip on "cloud" but more importantly remotely provisioned infrastructure, hybrid models for compute and storage are required along with flexibility to be able to further accelerate science. James will discuss their strategy moving forwards and the current and existing infrastructures in place to allow for seamless provisioning of research computing. Justin Riley Team Lead at Harvard, will follow this talk with a deep technical discussion of the specific implementation of the systems that Harvard are designing in concert with the development teams and leadership at OpenNebula to support research computing to make their platforms more resilient and able to continue to scale.

openstack源码分析(1)

cannium

This document introduces Nova, an open source cloud computing fabric controller. It describes Nova's core files, binaries, and services. Nova uses a flags module to manage configurable parameters. It has a scheduler module that uses drivers and algorithms to schedule virtual machines across compute hosts. Services include APIs for user interaction and internal RPC for communication between modules.

(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014

Amazon Web Services

"How can you reliably schedule tasks in an unreliable, autoscaling cloud environment? This presentation talks about the design of our Fenzo scheduler, built on Apache Mesos, that serves as the core of our stream-processing platform, Mantis, designed for real-time insights. We focus on the following aspects of the scheduler: - Resource granularity - Fault tolerance - Bin packing, task affinity, stream locality - Autoscaling of the cluster and of individual service jobs - Constraints (hard and soft) for individual tasks such as zone balancing, unique, and exclusive instances This talk also includes detailed information on a holistic approach to scheduling in a distributed, autoscaling environment to achieve both speed and advanced scheduling optimizations."

GoDocker presentation

Olivier Sallou

Network Automation: Ansible 101

APNIC

This document provides an overview and agenda for an Ansible 101 tutorial on network automation. It introduces Ansible concepts like playbooks, templates, variables, roles, and tasks. The tutorial will cover setting up an inventory and roles to generate network device configurations from templates using variables. It also discusses debugging Ansible and provides resources for learning more about network automation with Ansible.

Nagios Conference 2014 - Jeff Mendoza - Monitoring Microsoft Azure with Nagios

Nagios

Command line tools like PowerShell, Bash, and Bash allow you to manage Azure resources and perform operations through simple commands and syntax. These tools provide a unified standard to manage resources across platforms using REST APIs under the hood. SDKs act as wrappers that make REST API usage easier from various programming languages, while command line tools allow managing subscriptions, storage accounts, virtual machines, databases, and other resources through intuitive verbs and parameters. PowerShell cmdlets support all typical PowerShell features like variables, indexing, filtering and properties to provide full control and automation capabilities.

OpenNebula Conf 2014 | Lightning talk: Proactive Autonomic Management Feature...

NETWAYS

The document discusses an autonomic cloud architecture based on IBM's autonomic computing model. It uses OpenNebula as the cloud manager to provide self-awareness and self-configuration capabilities to virtual machines through OneGate, which exposes a REST interface. OneFlow is used for auto-scaling services. OneGate allows VMs to push monitoring metrics and receive information to enable self-awareness and handle configuration changes.

SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...

SaltStack

Google is making the power of its datacenter, network, and technology innovations available to the world through its Cloud services. This presentation will provide an overview of the Google Cloud Platform and a deeper dive on Google Compute Engine. Google recently made an open source contribution to SaltStack and now you can now use Salt Cloud to manage your Compute Engine resources (IaaS virtual machine services). Come find out more about Google's Cloud Platform and how you can leverage Google scale with SaltStack.

Cyansible

Alan Norton

Blue/Green deployments have been an important, if rarely implemented, technique in the Continuous Delivery playbook for years. Their aim is simple: provision, deploy, test — and optionally roll-back — your application before it's served to the public. Betterment's deployment architecture takes a similar, but more straightforward approach, accomplishing the important goals sought out by Blue/Green practitioners. Dubbed 'Cyan' (a mixture of Blue/Green), Betterment uses Ansible to provision new instances, push the latest artifacts to them, and ensure that they're healthy before marking them ready for production. All this ensures fast, stable, zero-downtime rollout with minimal human interaction. We'll discuss Betterment's philosophical approach to shipping new code and then dive into the nitty-gritty Ansible that powers the whole thing.

OpenNebula Conf 2014 | Puppet and OpenNebula - David Lutterkort

NETWAYS

Many facets of using an IaaS cloud like OpenNebula can be greatly simplified by using a configuration management tool such as Puppet. This includes the management of hosts as well as the management of cloud resources such as virtual machines and networks. Of course, Puppet can also play an important role in the management of the actual workload of virtual machine instances. Besides using it in the traditional, purely agent-based way, it is also possible to use Puppet during the building of machine images. This serves two purposes: firstly, it speeds up the initial Puppet run when an instance is launched off that image, sometimes quite dramatically. Secondly, it supports operating immutable infrastructure without losing Puppet’s benefits to organize and simplify the description of the entire infrastructure. This talk will show how Puppet can be used by adminsitrators to manage OpenNebula hosts, and by users to manage their infrastructure as well as how to use Puppet during image builds.

Monitoring of OpenNebula installations

NETWAYS

The complexity of a typical OpenNebula installation brings a special set of challenges on the monitoring side. In this talk, I will show monitoring of a full stack of from the physical servers to storage layer and ONE daemon. Providing an aggregated view of this information allows you see the real impact of a certain failure. I would like to also present a use case for a “closed-loop” setup where new VMs are automatically added to the monitoring without human intervention, allowing for an efficient approach to monitoring the services a OpenNebula setup provides.

Kubernetes 1.3 - Highlights

Matthew Barker

Using Embulk at Treasure Data

Muga Nishizawa

Embulk is an open-source tool for bulk loading data between various data sources. It provides parallel execution, data validation, error recovery, and other features. Customers use Embulk to upload various file formats and data sources to Treasure Data. The architecture includes a data connector that submits Embulk jobs to connector workers, which generate Embulk configurations and load data directly into Treasure Data using a private output plugin. To scale to large data loads, an optional MapReduce executor can run Embulk tasks on Hadoop clusters in parallel.

Recent Updates at Embulk Meetup #3

Muga Nishizawa

This document summarizes a presentation given by Muga Nishizawa on recent updates to Embulk. Some key points include: - Embulk is an open-source tool for parallel bulk data loading between various sources and destinations using plugins. - Recent updates include new plugins, performance improvements, and over 150 pull requests merged since the last meetup. - Future plans include improving tolerance to input source changes and a new "embulk-filter-calcite" plugin to transform data via SQL queries.

DSLing your System For Scalability Testing Using Gatling - Dublin Scala User ...

Aman Kohli

OpenNebula Conf 2014: CentOS, QA an OpenNebula - Christoph Galuschka

NETWAYS

CentOS, the Community Enterprise OS, uses Opennebula as virtualization plattform for its automated QA-process. The opennebula setup consists of 3 nodes, all running CentOS-6, who handle the following tasks: – sunstone as cloud controller – local mirror/DNS-Server/http-Server for the VMs to pull in packages – one VM to run a jenkins instance to launch the various tests (ci.de.centos.org) – nginx on the cloud controller to forward http traffic to the jenkins VM A public git repository (http://www.gitorious.org/testautomation) is used to allow whoever wants to contribute to pull the current test suite – t_functional, a series of bash scripts used to do funtional tests of various applications, binaries, configuration files and Trademark issues. As new tests are added to the repo via personal clones and merge requests, those tests first need to complete a test run via jenkins. Each test run currently consists of 4 VMs (one for each arch for C5 and C6 – C7 to come), which run the complete test suite. All VMs used for theses tests are instantiated and torn down on demand, whenever the call to testrun a personal clone is issued (via IRC). Once completed successfully, the request is merged into the main repo. The jenkins node monitors this repository and which automatically triggers another complete test run. Besides these triggered test runs, the test suite is automatically triggered daily to run. This is used to verify functionality of published updates – a handfull of failty updates have allready been discovered this way. Besides t_functional, the Linux Test Project Suite of tests is also run on a daily basis, also to verify functionality of the OS and all updates. The third setup is used to test the available and functional integrity of published docker images for CentOS. All these tests are later – during the QA-phase of a point release – used to verify functionality of new packages inside the CentOS QA-Setup.

Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013

Puppet

This document summarizes Sam Kottler's presentation on using Puppet at large scales. It discusses master-based vs masterless provisioning, certificate authority management, clustering patterns, node classification, external node classifiers, packaging for masterless deployments, distributed runs, deployment practices, and controlled releases. The presentation covers topics relevant to managing massive, multi-datacenter infrastructure with Puppet.

A user's perspective on SaltStack and other configuration management tools

SaltStack

[2018.10.19] Andrew Kong - Tunnel without tunnel (Seminar at OpenStack Korea ...

OpenStack Korea Community

The document discusses network architectures in OpenStack. It provides diagrams to illustrate the networking components including compute nodes, virtual machines, linux bridges, agents, and routers. MPLS is introduced as a solution to address issues with tenant network separation and performance challenges with other approaches like VxLAN. MPLS uses label switching to encapsulate and forward packets instead of relying on IP routing and overlays, improving east-west traffic performance between tenants.

Cloudinit

sriram_rajan

Cloud Init is a package that handles early initialization of cloud instances. It retrieves instance metadata like the server name and IP from the cloud provider. It then runs scripts and modules to configure the server, such as adding users, installing packages, and configuring services. It works across many Linux distributions and retrieves data through metadata services, config drives, or ISO images depending on the cloud platform. Administrators can write their own modules to customize the initialization and troubleshoot using logs and files stored in /var/lib/cloud.

[2018.10.19] 김용기 부장 - IAC on OpenStack (feat. ansible)

OpenStack Korea Community

Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...

SaltStack

OpenNebulaConf2015 1.09.02 Installgems Add-on - Alvaro Simon Garcia

OpenNebula Project

Ansible with AWS

Allan Denot

PuppetCamp Sydney 2012 - Building a Multimaster Environment

Greg Cockburn

This document discusses a solution for providing Puppet services globally across multiple regions with poor WAN connectivity. The solution involves building a "Puppeteer" master that acts as a central point of entry for code updates and certificate management. It ensures Puppet masters in each region are in sync. LDAP is used as an external node classifier to provide node definitions across regions. The Puppet file server replicates configuration between masters. F5 load balancers route clients to the nearest master and provide high availability if any master fails. Workflows for adding new servers and masters are also summarized.

WebCamp 2016: PHP.Алексей Петров.PHP at Scale: System Architect Toolbox

WebCamp 2016: DevOps. Николай Дойков: Опыт создания клауда для потокового видео на docker swarm.

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to WebCamp 2016: DevOps. Николай Дойков: Опыт создания клауда для потокового видео на docker swarm.

Similar to WebCamp 2016: DevOps. Николай Дойков: Опыт создания клауда для потокового видео на docker swarm. (20)

More from WebCamp

More from WebCamp (9)

Recently uploaded

Recently uploaded (20)

WebCamp 2016: DevOps. Николай Дойков: Опыт создания клауда для потокового видео на docker swarm.