Aptira presents OpenStack swift architecture and monitoring
•Download as PPT, PDF•
1 like•1,104 views
This document discusses OpenStack Swift architecture and monitoring options. It describes the key components of Swift including the proxy server, ring, object/storage nodes, account server, and replication. It recommends monitoring hardware/OS failures, cluster health using tools like swift-dispersion-report, and cluster telemetry using swift-recon. Existing monitoring options that can be used today include Nagios plugins for Swift and the Zenoss OpenStack Swift ZenPack, both of which retrieve metrics via the swift-recon API.
Report
Share
Report
Share
Recommended
Monitoring Swift - OpenStack Summit May 2015, Vancouver
Slides from a talk given at the OpenStack Summit, May 2015 in Vancouver.
This talk covered the following topics:
- short introduction to Swift itself
- what you need to monitor
- how to use statsd and graphite to visualize collected metrics
Recorded session available on Youtube: https://www.youtube.com/watch?v=z7gaew4OnDU
Changing the world with ZeroVM and Swift
ZeroVM is a secure execution environment that allows users to run applications in isolation. It is based on Google's Native Client and includes a runtime environment called ZRT that provides a subset of POSIX APIs. ZeroVM has very low overhead and fast startup times compared to traditional VMs and containers. The document discusses how ZeroVM could be embedded in OpenStack Swift to enable distributed computing on Swift storage nodes, such as for video transcoding, log searching, and distributed SQL.
How to integrate_custom_openstack_services_with_devstack
DevStack is a tool used to quickly deploy OpenStack from source code for development and testing purposes. Plugins allow custom OpenStack services to be integrated with DevStack. A plugin contains scripts that are executed at different points during deployment to install and configure the custom service. Functions are provided to help with common tasks like installing packages or configuring services.
PTG recap
This document summarizes discussions from the PTG-Denver meeting. It outlines the agenda which included an introduction to PTG, highlighting discussion topics, recapping accomplishments in the Pike release, and plans for ongoing work in the Queens release. Key topics discussed were skip-release upgrades, API-SIG, and accomplishments across various OpenStack projects including Neutron, Cinder, Nova, Ironic, Glance, and Kolla from Pike and planned work for Queens.
How OpenStack is Built - Anton Weiss - OpenStack Day Israel 2016
Openstack is one of the largest OSS projects today with hundreds of commits flowing in daily. This high rate of change requires an advanced CI infrastructure. The purpose of the talk is to provide an overview of this infrastructure, explaining the role of each tool and the pipelines along which changes have to travel before they find their way into the approved Openstack codebase.
DevOps and OpenStack December 2012
This document discusses OpenStack, an open source cloud computing platform. It provides OpenStack's mission statement to create a simple and scalable cloud platform. OpenStack controls compute, storage, and networking through projects like Nova (compute), Cinder (block storage), Quantum (networking), Glance (image registry), Keystone (identity), Swift (object storage), and Horizon (dashboard). The document outlines OpenStack's architecture, community involvement, code repositories, distributions, and the upcoming Grizzly release.
Introduction to tempest
Tempest is the OpenStack integration test suite. It uses unittest and nosetest frameworks to run API calls against OpenStack services like Nova, Glance, Keystone, etc. and validate the responses. Tempest tests include smoke, positive, negative, stress and white box tests. It has a modular structure with common, services, and tests directories. Tempest plays an important role in OpenStack continuous integration by running on proposed code changes to check for regressions.
Guts & OpenStack migration
GUTS is a workload migration engine that automatically migrates existing workloads and virtual machines from previous generation virtualization platforms to OpenStack. It supports migrating VMs, volumes, networks, users, and other resources between OpenStack environments or from platforms like VMware to OpenStack. GUTS has API, scheduler, and migration services to orchestrate the migrations. It can convert disk formats and manage hypervisor-specific tools during the migration process. Future plans include supporting more hypervisors and resource types.
Recommended
Monitoring Swift - OpenStack Summit May 2015, Vancouver
Slides from a talk given at the OpenStack Summit, May 2015 in Vancouver.
This talk covered the following topics:
- short introduction to Swift itself
- what you need to monitor
- how to use statsd and graphite to visualize collected metrics
Recorded session available on Youtube: https://www.youtube.com/watch?v=z7gaew4OnDU
Changing the world with ZeroVM and Swift
ZeroVM is a secure execution environment that allows users to run applications in isolation. It is based on Google's Native Client and includes a runtime environment called ZRT that provides a subset of POSIX APIs. ZeroVM has very low overhead and fast startup times compared to traditional VMs and containers. The document discusses how ZeroVM could be embedded in OpenStack Swift to enable distributed computing on Swift storage nodes, such as for video transcoding, log searching, and distributed SQL.
How to integrate_custom_openstack_services_with_devstack
DevStack is a tool used to quickly deploy OpenStack from source code for development and testing purposes. Plugins allow custom OpenStack services to be integrated with DevStack. A plugin contains scripts that are executed at different points during deployment to install and configure the custom service. Functions are provided to help with common tasks like installing packages or configuring services.
PTG recap
This document summarizes discussions from the PTG-Denver meeting. It outlines the agenda which included an introduction to PTG, highlighting discussion topics, recapping accomplishments in the Pike release, and plans for ongoing work in the Queens release. Key topics discussed were skip-release upgrades, API-SIG, and accomplishments across various OpenStack projects including Neutron, Cinder, Nova, Ironic, Glance, and Kolla from Pike and planned work for Queens.
How OpenStack is Built - Anton Weiss - OpenStack Day Israel 2016
Openstack is one of the largest OSS projects today with hundreds of commits flowing in daily. This high rate of change requires an advanced CI infrastructure. The purpose of the talk is to provide an overview of this infrastructure, explaining the role of each tool and the pipelines along which changes have to travel before they find their way into the approved Openstack codebase.
DevOps and OpenStack December 2012
This document discusses OpenStack, an open source cloud computing platform. It provides OpenStack's mission statement to create a simple and scalable cloud platform. OpenStack controls compute, storage, and networking through projects like Nova (compute), Cinder (block storage), Quantum (networking), Glance (image registry), Keystone (identity), Swift (object storage), and Horizon (dashboard). The document outlines OpenStack's architecture, community involvement, code repositories, distributions, and the upcoming Grizzly release.
Introduction to tempest
Tempest is the OpenStack integration test suite. It uses unittest and nosetest frameworks to run API calls against OpenStack services like Nova, Glance, Keystone, etc. and validate the responses. Tempest tests include smoke, positive, negative, stress and white box tests. It has a modular structure with common, services, and tests directories. Tempest plays an important role in OpenStack continuous integration by running on proposed code changes to check for regressions.
Guts & OpenStack migration
GUTS is a workload migration engine that automatically migrates existing workloads and virtual machines from previous generation virtualization platforms to OpenStack. It supports migrating VMs, volumes, networks, users, and other resources between OpenStack environments or from platforms like VMware to OpenStack. GUTS has API, scheduler, and migration services to orchestrate the migrations. It can convert disk formats and manage hypervisor-specific tools during the migration process. Future plans include supporting more hypervisors and resource types.
Deploying Kubernetes without scaring off your security team - KubeCon 2017
Kubernetes provides plenty of enhancements for deploying software, but it can cause anxiety on the corporate security team. This talk explains how to approach your security team and how to push them to provide guardrails, not deployments.
CDK Meetup: Rule the World through IaC
How CDK for Terraform make developers use favorite program language to implement IaC, and how to face the IaC Day 2 challenge!
Open stack nova reverse engineer
This document provides an overview of OpenStack Nova's architecture and code structure. It begins with an introduction to Nova's mission to provide scalable on-demand access to compute resources. It then covers Nova's core components, data flows, and code organization. Key aspects summarized include Nova's use of Cells to partition compute nodes, its facade design pattern for APIs, and its reliance on common OpenStack libraries like Oslo and Stevedore for configuration, logging, and extensibility.
Role of sdn controllers in open stack
This document discusses the role of SDN controllers in OpenStack. It provides background on SDN controllers and OpenStack. SDN controllers can be integrated with OpenStack via the Neutron module to manage network flows and enable programmability. Several SDN controllers that integrate with Neutron are discussed, including OpenDaylight, OpenContrail, and ONOS. The document outlines how these controllers plug into Neutron and their current status in OpenStack. It provides guidance on how new SDN controllers can join OpenStack.
OPNFV & OpenStack
This document summarizes a presentation about Open Platform for Network Functions Virtualization (OPNFV). It discusses NFV challenges for telecom operators and introduces OPNFV as an open source platform that aims to develop and test an integrated virtual network functions infrastructure. Key aspects of OPNFV covered include its reference architecture, goals of contributing to relevant open source projects and establishing an NFV ecosystem, and examples of feature development and community labs/testing activities.
Using OpenStack Swift for Extreme Data Durability
Openstack Swift is a very powerful object storage that is used in several of the largest object storage deployments around the globe. It ensures a very high level of data durability and can withstand epic disasters if setup in the right way.
OpenDaylight OpenStack Integration
Networking-odl and ODL Neutron Northbound are the key components for integrating OpenStack Neutron and OpenDaylight. They are actively developed open source projects. The document encourages giving the integration a try, providing feedback, and contributing to help further the integration of OpenStack and OpenDaylight networking.
Batch Applications for the Java Platform
Presentation on the Batch JSR (JSR-352) in JavaOne India, Hyderabad 2013.
Thanks to http://www.slideshare.net/reza_rahman and http://www.slideshare.net/arungupta1 for the source slides.
Neutron upgrades
This presentation covers the governance requirements to use upgrade tags and how neutron is in compliance with those tags
Monitoring the Hashistack with Prometheus
Prometheus has become the defacto monitoring system for cloud native applications, but for a while was eschewed by the Hashistack in favour of more traditional technologies. Thats all changing: Hashicorp's project are beginning to export metrics in the native Prometheus format, and many exporters exist to bridge the gap.
In this talk Tom will give a brief introduction to Prometheus, show you how to piece it all together, and give some recommendation on what to monitor and alert on.
OpenStack Astara
This document discusses network orchestration and Astara, an open source project for provisioning network functions in virtual environments. It provides background on software defined networking (SDN) and network function virtualization (NFV). It then describes Astara, including its architecture and goal of tying together OpenStack neutron. It follows the "FOUR O" principles of being open source, community, development and design.
OpenStack @ Workday - CI/CD
During the OpenStack Tokyo Summit we provided an overview on how Workday started the production deployment with a very robust and efficient CI/CD process that it explained here.
OpenStack Tempest and REST API testing
Here are the basic steps to download the latest version of DevStack:
1. Install Git:
- On Ubuntu/Debian: `sudo apt-get install git`
- On CentOS/RHEL: `sudo yum install git`
2. Clone the DevStack repository:
`git clone https://git.openstack.org/openstack-dev/devstack`
3. Change to the DevStack directory:
`cd devstack`
4. Checkout the latest stable branch (e.g. stable/ussuri):
`git checkout stable/ussuri`
5. Download the latest code:
`git pull`
This will download the latest version
Moving to Nova Cells without Destroying the World
This document discusses using cells in OpenStack Nova to scale deployments. Cells create a hierarchy with a top-level API cell and multiple compute cells. Each cell has its own database, message queue, and services. The document outlines planning the conversion, preparing the environment by expanding RabbitMQ and splitting services, configuring the compute and API cells, importing data to the API cell, and restarting services. It notes some caveats like limitations on certain notifications and objects between cells.
Fuel's current use cases, architecture and next steps
Fuel is open source initiative which provides great automation and flexibility for:
* CentOS and Ubuntu provisioning on bare metal servers
* Disk partitioning, including software RAIDs. Support of hardware RAIDs is in the roadmap
* Complex network configurations, including VLANs, GRE and bonding
* Network verification (VLANs, DHCP) before deployment
* Automated deployment of OpenStack components and Ceph
* Health Check - a set of short-running tests to ensure functionality of deployed environment
Demo how to efficiently evaluate nf-vi performance by leveraging opnfv testi...
Liang Gao, Huawei, Trevor Cooper, Intel
NFV environments are highly flexible and this introduces unique challenges for testing performance of NFVI and Network Services. This presentation introduces OPNFV performance test projects and explains their role as part of the testing ecosystem. Examples from three performance testing categories will be demonstrated showing test results and their interpretation. Test cases discussed will include data-path performance, live migration performance and storage performance.
OpenStack: running manually installed components on VirtualBox
This document discusses running OpenStack on VirtualBox and provides details on a demonstration environment using two VirtualBox VMs - one configured as the controller node and one as the compute node. Key services are outlined for each VM. Networking is configured using four network adapters per VM and requirements include VirtualBox 4.3, two preconfigured ova images, and a shell script to automate deployment.
Containerizing Network Services - Alon Harel - OpenStack Day Israel 2016
With the increasing maturity of OpenStack Neutron, and increasing support for a large number of services , we'll examine how SDNs can leverage the container technology for supporting "as a Service" components.
Tips Tricks and Tactics with Cells and Scaling OpenStack - May, 2015
The document discusses using cells in OpenStack to scale cloud infrastructure across multiple geographic locations. Key points include using cells to distribute OpenStack compute services around Australia, with over 6000 users, 700 hypervisors, and 30,000 cores spread across 8 sites and 14 cells. It also discusses strategies for operating, upgrading, and scheduling across multiple cells.
Understanding and Extending Prometheus AlertManager
The document discusses Prometheus AlertManager, including its purpose of ingesting, grouping, deduplicating, silencing, throttling, and notifying alerts from Prometheus. It describes AlertManager's routes, receivers, silencers, inhibitors, and grouping functionality. It also covers high availability, the AlertManager UI, and enhancing AlertManager to support viewing alert history.
Eran Rom, IBM - Bringing Compute to Openstack Swift, OpenStack Israel 2015
The document discusses IBM's Storlet engine, which extends the OpenStack Swift object storage system. The Storlet engine allows computations to be pushed down to storage nodes rather than bringing data to computation nodes. It uses Docker containers to run user-defined storlet programs on Swift storage nodes. Storlets can be invoked on GET and PUT requests to objects in Swift to transform or analyze data during storage operations. The Storlet engine is open source and available on GitHub for others to experiment with and integrate into applications.
Couch to open_stack_keystone
This document provides an overview and introduction to the Keystone identity service in OpenStack. It defines key concepts in Keystone including users, tenants, roles, tokens, services, and endpoints. It also provides examples of using the keystone CLI to list and get information about users, tenants, services, and endpoints. The document concludes by assigning homework of setting up additional roles, endpoints, and services in Keystone to enable the Glance image service, and installing Glance in preparation for the next session.
More Related Content
What's hot
Deploying Kubernetes without scaring off your security team - KubeCon 2017
Kubernetes provides plenty of enhancements for deploying software, but it can cause anxiety on the corporate security team. This talk explains how to approach your security team and how to push them to provide guardrails, not deployments.
CDK Meetup: Rule the World through IaC
How CDK for Terraform make developers use favorite program language to implement IaC, and how to face the IaC Day 2 challenge!
Open stack nova reverse engineer
This document provides an overview of OpenStack Nova's architecture and code structure. It begins with an introduction to Nova's mission to provide scalable on-demand access to compute resources. It then covers Nova's core components, data flows, and code organization. Key aspects summarized include Nova's use of Cells to partition compute nodes, its facade design pattern for APIs, and its reliance on common OpenStack libraries like Oslo and Stevedore for configuration, logging, and extensibility.
Role of sdn controllers in open stack
This document discusses the role of SDN controllers in OpenStack. It provides background on SDN controllers and OpenStack. SDN controllers can be integrated with OpenStack via the Neutron module to manage network flows and enable programmability. Several SDN controllers that integrate with Neutron are discussed, including OpenDaylight, OpenContrail, and ONOS. The document outlines how these controllers plug into Neutron and their current status in OpenStack. It provides guidance on how new SDN controllers can join OpenStack.
OPNFV & OpenStack
This document summarizes a presentation about Open Platform for Network Functions Virtualization (OPNFV). It discusses NFV challenges for telecom operators and introduces OPNFV as an open source platform that aims to develop and test an integrated virtual network functions infrastructure. Key aspects of OPNFV covered include its reference architecture, goals of contributing to relevant open source projects and establishing an NFV ecosystem, and examples of feature development and community labs/testing activities.
Using OpenStack Swift for Extreme Data Durability
Openstack Swift is a very powerful object storage that is used in several of the largest object storage deployments around the globe. It ensures a very high level of data durability and can withstand epic disasters if setup in the right way.
OpenDaylight OpenStack Integration
Networking-odl and ODL Neutron Northbound are the key components for integrating OpenStack Neutron and OpenDaylight. They are actively developed open source projects. The document encourages giving the integration a try, providing feedback, and contributing to help further the integration of OpenStack and OpenDaylight networking.
Batch Applications for the Java Platform
Presentation on the Batch JSR (JSR-352) in JavaOne India, Hyderabad 2013.
Thanks to http://www.slideshare.net/reza_rahman and http://www.slideshare.net/arungupta1 for the source slides.
Neutron upgrades
This presentation covers the governance requirements to use upgrade tags and how neutron is in compliance with those tags
Monitoring the Hashistack with Prometheus
Prometheus has become the defacto monitoring system for cloud native applications, but for a while was eschewed by the Hashistack in favour of more traditional technologies. Thats all changing: Hashicorp's project are beginning to export metrics in the native Prometheus format, and many exporters exist to bridge the gap.
In this talk Tom will give a brief introduction to Prometheus, show you how to piece it all together, and give some recommendation on what to monitor and alert on.
OpenStack Astara
This document discusses network orchestration and Astara, an open source project for provisioning network functions in virtual environments. It provides background on software defined networking (SDN) and network function virtualization (NFV). It then describes Astara, including its architecture and goal of tying together OpenStack neutron. It follows the "FOUR O" principles of being open source, community, development and design.
OpenStack @ Workday - CI/CD
During the OpenStack Tokyo Summit we provided an overview on how Workday started the production deployment with a very robust and efficient CI/CD process that it explained here.
OpenStack Tempest and REST API testing
Here are the basic steps to download the latest version of DevStack:
1. Install Git:
- On Ubuntu/Debian: `sudo apt-get install git`
- On CentOS/RHEL: `sudo yum install git`
2. Clone the DevStack repository:
`git clone https://git.openstack.org/openstack-dev/devstack`
3. Change to the DevStack directory:
`cd devstack`
4. Checkout the latest stable branch (e.g. stable/ussuri):
`git checkout stable/ussuri`
5. Download the latest code:
`git pull`
This will download the latest version
Moving to Nova Cells without Destroying the World
This document discusses using cells in OpenStack Nova to scale deployments. Cells create a hierarchy with a top-level API cell and multiple compute cells. Each cell has its own database, message queue, and services. The document outlines planning the conversion, preparing the environment by expanding RabbitMQ and splitting services, configuring the compute and API cells, importing data to the API cell, and restarting services. It notes some caveats like limitations on certain notifications and objects between cells.
Fuel's current use cases, architecture and next steps
Fuel is open source initiative which provides great automation and flexibility for:
* CentOS and Ubuntu provisioning on bare metal servers
* Disk partitioning, including software RAIDs. Support of hardware RAIDs is in the roadmap
* Complex network configurations, including VLANs, GRE and bonding
* Network verification (VLANs, DHCP) before deployment
* Automated deployment of OpenStack components and Ceph
* Health Check - a set of short-running tests to ensure functionality of deployed environment
Demo how to efficiently evaluate nf-vi performance by leveraging opnfv testi...
Liang Gao, Huawei, Trevor Cooper, Intel
NFV environments are highly flexible and this introduces unique challenges for testing performance of NFVI and Network Services. This presentation introduces OPNFV performance test projects and explains their role as part of the testing ecosystem. Examples from three performance testing categories will be demonstrated showing test results and their interpretation. Test cases discussed will include data-path performance, live migration performance and storage performance.
OpenStack: running manually installed components on VirtualBox
This document discusses running OpenStack on VirtualBox and provides details on a demonstration environment using two VirtualBox VMs - one configured as the controller node and one as the compute node. Key services are outlined for each VM. Networking is configured using four network adapters per VM and requirements include VirtualBox 4.3, two preconfigured ova images, and a shell script to automate deployment.
Containerizing Network Services - Alon Harel - OpenStack Day Israel 2016
With the increasing maturity of OpenStack Neutron, and increasing support for a large number of services , we'll examine how SDNs can leverage the container technology for supporting "as a Service" components.
Tips Tricks and Tactics with Cells and Scaling OpenStack - May, 2015
The document discusses using cells in OpenStack to scale cloud infrastructure across multiple geographic locations. Key points include using cells to distribute OpenStack compute services around Australia, with over 6000 users, 700 hypervisors, and 30,000 cores spread across 8 sites and 14 cells. It also discusses strategies for operating, upgrading, and scheduling across multiple cells.
Understanding and Extending Prometheus AlertManager
The document discusses Prometheus AlertManager, including its purpose of ingesting, grouping, deduplicating, silencing, throttling, and notifying alerts from Prometheus. It describes AlertManager's routes, receivers, silencers, inhibitors, and grouping functionality. It also covers high availability, the AlertManager UI, and enhancing AlertManager to support viewing alert history.
What's hot (20)
Deploying Kubernetes without scaring off your security team - KubeCon 2017
Deploying Kubernetes without scaring off your security team - KubeCon 2017
Fuel's current use cases, architecture and next steps
Fuel's current use cases, architecture and next steps
Demo how to efficiently evaluate nf-vi performance by leveraging opnfv testi...
Demo how to efficiently evaluate nf-vi performance by leveraging opnfv testi...
OpenStack: running manually installed components on VirtualBox
OpenStack: running manually installed components on VirtualBox
Containerizing Network Services - Alon Harel - OpenStack Day Israel 2016
Containerizing Network Services - Alon Harel - OpenStack Day Israel 2016
Tips Tricks and Tactics with Cells and Scaling OpenStack - May, 2015
Tips Tricks and Tactics with Cells and Scaling OpenStack - May, 2015
Understanding and Extending Prometheus AlertManager
Understanding and Extending Prometheus AlertManager
Viewers also liked
Eran Rom, IBM - Bringing Compute to Openstack Swift, OpenStack Israel 2015
The document discusses IBM's Storlet engine, which extends the OpenStack Swift object storage system. The Storlet engine allows computations to be pushed down to storage nodes rather than bringing data to computation nodes. It uses Docker containers to run user-defined storlet programs on Swift storage nodes. Storlets can be invoked on GET and PUT requests to objects in Swift to transform or analyze data during storage operations. The Storlet engine is open source and available on GitHub for others to experiment with and integrate into applications.
Couch to open_stack_keystone
This document provides an overview and introduction to the Keystone identity service in OpenStack. It defines key concepts in Keystone including users, tenants, roles, tokens, services, and endpoints. It also provides examples of using the keystone CLI to list and get information about users, tenants, services, and endpoints. The document concludes by assigning homework of setting up additional roles, endpoints, and services in Keystone to enable the Glance image service, and installing Glance in preparation for the next session.
Making Glance tasks work for you - OpenStack Summit May 2015 Vancouver
It's not widely known that the OpenStack Images API v2 contains an implementation of a "tasks" API that can be customized by operators to enable asynchronous processing of long-running operations. For example, a deployer might want to enable end users to upload their own custom images ... but only after such images have been approved by some thorough, computation-intensive validation process. The Glance tasks API provides a common interface across OpenStack installations, but allows the implementation of tasks to be customizable to a particular cloud environment. Join Brian Rosmaita, Compute Control Plane Product Manager at Rackspace to see how Glance tasks are being used at Rackspace and to learn how you can use Glance tasks in your OpenStack cloud.
Vagrant
Vagrant allows users to easily create and configure virtual development environments. The document outlines a 5 step process to get started with Vagrant: 1) select a virtualization provider like VirtualBox, 2) install Vagrant, 3) download a virtual machine image or "box", 4) initialize and start the VM with Vagrant commands, and 5) log into the VM via SSH. It also discusses additional features like version controlling Vagrant files, customizing the VM, using multiple VMs, and provisioning VMs with tools like Puppet, Chef, or Ansible.
Openstack swift introduction
Swift is OpenStack's scalable object storage system that stores unstructured data and provides a RESTful API similar to S3. It is designed to be highly available, distributed, and scalable up to several petabytes. Swift uses a consistent hashing algorithm to partition and replicate data across multiple servers. Components include proxy, account, container, and object servers that each have specific roles in storing and managing objects, metadata, and namespaces.
OpenStack Swift on virtualbox
This document provides instructions for installing OpenStack Swift on Virtualbox by fetching an installation script from GitHub, selecting a version (Folsom, Grizzly, or Havana) to install, running the script as superuser, sourcing credentials, and testing the deployment, with assumptions that it is not intended for production and requires Ubuntu 12.04, Git, and Virtualbox.
Hybrid Container for OpenStack Swift
OpenStack Swift currently allows one storage policy for a container. In a typical setup, a storage policy represents backend devices with similar quality of service (QoS) characteristics like IOPS, time-to-first-byte, etc. This conflation of logical namespace with physical resources puts the burden of placing objects based on QoS through namespace partitioning on the application i.e., store objects with QoS1 in Container1, and objects with QoS2 in Container2, etc. This reduces the flexibility of applications in utilizing containers for organizing objects into logically related buckets.
This slide deck describes Hybrid Container for OpenStack Swift. A Hybrid Container allows objects within a container to be stored on backends with different QoS characteristics e.g., different storage media like SSD/HDD/tape, or different storage policies such as replication and erasure coding.
Viewers also liked (7)
Eran Rom, IBM - Bringing Compute to Openstack Swift, OpenStack Israel 2015
Eran Rom, IBM - Bringing Compute to Openstack Swift, OpenStack Israel 2015
Making Glance tasks work for you - OpenStack Summit May 2015 Vancouver
Making Glance tasks work for you - OpenStack Summit May 2015 Vancouver
Similar to Aptira presents OpenStack swift architecture and monitoring
Testing kubernetes and_open_shift_at_scale_20170209
This document discusses testing Kubernetes and OpenShift at scale. It describes installing large clusters of 1000+ nodes, using scalability test tools like the Kubernetes performance test repo and OpenShift SVT repo to load clusters and generate traffic. Sample results show loading clusters with thousands of pods and projects, and peaks in master node resource usage when loading and deleting hundreds of pods simultaneously.
Openstack devops challenges
Openstack DevOps Challenges outlines the journey of CloudRX, a fictitious company, to setup a production-grade Openstack cloud using DevOps practices. It discusses challenges faced in implementing continuous integration/delivery pipelines for Openstack and its heterogeneous components, managing configurations, automated testing of environments, packaging applications, and baremetal server management.
Flexible compute
Introducing our new OpenStack environment and image workflows as a complimentary platform to Sangers existing HPC infrastructure
Sanger, upcoming Openstack for Bio-informaticians
Delivery of a new Bio-informatics infrastructure at the Wellcome Trust Sanger Center. We include how to programatically create, manage and provide providence for images used both at Sanger and elsewhere using open source tools and continuous integration.
Modern application development with oracle cloud sangam17
How Oracle cloud helps in building modern application development. This explains Oracle Application container cloud with developer cloud service and etc. Spring boot application deployed in Oracle ACCS and CI/CD part done in Oracle Developer cloud service.
Introduction to kubernetes
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery called Pods. ReplicaSets ensure that a specified number of pod replicas are running at any given time. Key components include Pods, Services for enabling network access to applications, and Deployments to update Pods and manage releases.
Fluo CICD OpenStack Summit
This document describes eBay's use of Fluo for continuous integration and deployment using OpenStack. Fluo provides a single interface for configuring, building, testing, and deploying code changes. It provisions instances on OpenStack to run tasks defined in a configuration file like running tests, building packages, and deploying code. Fluo replicates code, packages, and configuration management code across regions and datacenters. It supports common workflows from code review through integration testing, releases, and periodic jobs. Fluo aims to provide a fully automated and scalable continuous delivery system to deploy code changes to eBay's global infrastructure on OpenStack.
Getting to Walk with DevOps
Moving from "Crawl" to "walk" stage on Devops. Part 2 of 4 for Complete DevOps implementation in an organization.
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...Rahul Krishna Upadhyaya
Slide was presented at Dr. Dobb's Conference in Bangalore.
Talks about Openstack Introduction in general
Projects under Openstack.
Contributing to Openstack.
This was presented jointly by CB Ananth and Rahul at Dr. Dobb's Conference Bangalore on 12th Apr 2014.Open shift and docker - october,2014
This document provides an overview and summary of OpenShift v3 and containers. It discusses how OpenShift v3 uses Docker containers and Kubernetes for orchestration instead of the previous "Gears" system. It also summarizes the key architectural changes in OpenShift v3, including using immutable Docker images, separating development and operations, and abstracting operational complexity.
OpenNaaS Overview Complete
This document provides an overview of the OpenNaaS project, which aims to create an open source Network as a Service (NaaS) software stack. The key goals of OpenNaaS are to enable long-term network research, leverage past research assets, and create an open community of stakeholders. OpenNaaS is based on OSGi and uses a lightweight abstraction model centered around resources, capabilities, actions, and profiles. It provides reusable platform components and aims to be embeddable, interoperable, and provide a foundation for NaaS plugins. The roadmap discusses ongoing and future extensions related to different network domains.
Oct meetup open stack 101 clean
The document provides information about an OpenStack Meetup happening on October 7th, 2014 in San Antonio, Texas. It includes details about the event location and organizers, as well as an agenda for introductory OpenStack training modules covering an overview of OpenStack, its architecture, and installing OpenStack.
DEVNET-1169 CI/CT/CD on a Micro Services Applications using Docker, Salt & Ni...
Nowadays, we heard a lot regarding micro services and DevOps but then, what are the impacts for an application development and how to really achieve this? The demo will demonstrate the benefits of using Docker (and related tools / technologies) for a micro services application and then having a continuous integration / tests / deployment workflow on CCS/Nimbus.
OpenStack London Meetup, 18 Nov 2015
This document summarizes feedback from operator sessions at the Mitaka/Tokyo Design Summit and OpenStack London Meetup on November 18, 2015. It discusses upgrades being difficult due to unknown compatibility and integration issues. It also provides an overview of the OpenStack-Ansible project including its history, deliverables for the Liberty release, and plans for the Mitaka release.
Solution Manager in Denodo Platform 7.0: Admin Made Simple
Watch this Fast Data Strategy Session here: https://goo.gl/c1Rkoy
Denodo Platform 7.0 Solution Manger provides a unified interface for management and simplifies large-scale deployments.
Watch this session to learn more about:
• The new features of Denodo Platform 7.0 Solution Manager
• What’s new in Monitoring and Diagnostics
• Additional tools to enable a cohesive DevOps approach
OpenStack@Mini-Deb Conf'16 Mumbai
OpenStack is a cloud operating system that controls large pools of computing, storage, and networking resources throughout a datacenter. It is made up of several components like Horizon (dashboard), Keystone (identity), Nova (compute), Neutron (networking), Swift (object storage), and Cinder (block storage) that are all managed through a central dashboard. DevStack is a shell script used to deploy a complete OpenStack development environment locally and is useful for OpenStack development by beginners. The presentation demonstrated installing and using DevStack to deploy an OpenStack development environment.
Kubernetes presentation
This document provides an overview of Kubernetes, including its architecture, components, concepts, and configuration. It describes that Kubernetes is an open-source container orchestration system designed by Google to manage containerized applications across multiple hosts. The key components include the master nodes which run control plane components like the API server, scheduler, and controller manager, and worker nodes which run the kubelet and containers. It also explains concepts like pods, services, deployments, networking, storage, and role-based access control (RBAC).
Scaling Up Logging and Metrics
This document discusses scaling up logging and metrics in OpenShift Container Platform (OCP). It provides an overview of the logging stack including Elasticsearch, Fluentd, and Kibana. It also summarizes the metrics stack including Cassandra, Heapster, and Hawkular. The document outlines testing done to evaluate limits and scaling of these components on large OCP clusters with thousands of nodes and pods. It provides recommendations for configuring and deploying the infrastructure to support high throughput logging and metrics collection.
Neutrondev ppt
Neutron is OpenStack's networking service that exposes a REST API to manage network resources using plugins. It provides a framework for plugins to implement specific networking strategies using agents that manage state on compute and gateway nodes. Typical requests update the database, asynchronously dispatch state changes to agents, and return a status to the client.
Nex clipper 1905_summary_eng
NexClipper is open source for Kubernetes monitoring in an enterprise environments by complementing and integration with Prometheus
Similar to Aptira presents OpenStack swift architecture and monitoring (20)
Testing kubernetes and_open_shift_at_scale_20170209
Testing kubernetes and_open_shift_at_scale_20170209
Modern application development with oracle cloud sangam17
Modern application development with oracle cloud sangam17
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
Openstack - An introduction/Installation - Presented at Dr Dobb's conference...
DEVNET-1169 CI/CT/CD on a Micro Services Applications using Docker, Salt & Ni...
DEVNET-1169 CI/CT/CD on a Micro Services Applications using Docker, Salt & Ni...
Solution Manager in Denodo Platform 7.0: Admin Made Simple
Solution Manager in Denodo Platform 7.0: Admin Made Simple
More from OpenStack
Swinburne University of Technology - Shunde Zhang & Kieran Spear, Aptira
We recently teamed up with our good friends at SUSE to build a very high-performing and scalable storage landscape at a fraction of the cost than with traditional storage systems for the Swinburne University of Technology.
The challenge that the IT team at Swinburne were facing is how to accommodate the ever-growing need for performance and capacity while sticking to a tight budget. With SUSE Enterprise Storage we have been able to deploy a compelling and affordable solution to support Swinburne’s storage needs. Their new storage platform is fast and flexible, meaning the IT team at Swinburne can support the needs of researchers more effectively.
https://aptira.com/swinburne-university-technology/
Related OSS Projects - Peter Rowe, Flexera Software
Audience Level
Intermediate
Synopsis
Today’s fast-paced development environment has changed the compliance landscape. Many software projects consist of more than 50% Open Source Software (OSS) components, but as much as 99% are undocumented, increasing the complexities of managing your company’s software compliance process.
Of particular concern is “Zombie software”, or software that is outdated and contains vulnerable versions of certain components. Zombies can live in your code forever if you’re not aware of them. The acceleration of modern development lifecycles and the breakdown of an undocumented software supply chain have opened up new pathways for zombies to enter your software – leaving you exposed to security threats.
This presentation discusses best practices for implementing an Open Source Software management strategy that covers common pitfalls and commercial licence issues as well as the optimal way to track and eliminate the risks associated with Zombies!
Speaker Bio:
Involved in and around IT development for over 20 years, starting as a web developer using NotePad in 1995 when the most exciting thing online was Sun’s animated Java coffee cup, through Numega Pre-Sales selling BoundsChecker and now into the brave, new World of Open Source and software composition analysis.
Supercomputing by API: Connecting Modern Web Apps to HPC
Audience Level
Intermediate
Synopsis
The traditional user experience for High Performance Computing (HPC) centers around the command line, and the intricacies of the underlying hardware. At the same time, scientific software is moving towards the cloud, leveraging modern web-based frameworks, allowing rapid iteration, and a renewed focus on portability and reproducibility. This software still has need for the huge scale and specialist capabilities of HPC, but leveraging these resources is hampered by variation in implementation between facilities. Differences in software stack, scheduling systems and authentication all get in the way of developers who would rather focus on the research problem at hand. This presentation reviews efforts to overcome these barriers. We will cover container technologies, frameworks for programmatic HPC access, and RESTful APIs that can deliver this as a hosted solution.
Speaker Bio
Dr. David Perry is Compute Integration Specialist at The University of Melbourne, working to increase research productivity using cloud and HPC. David chairs Australia’s first community-owned wind farm, Hepburn Wind, and is co-founder/CTO of BoomPower, delivering simpler solar and battery purchasing decisions for consumers and NGOs.
Federation and Interoperability in the Nectar Research Cloud
Audience Level
Beginner
Synopsis
The Nectar Research Cloud provides an OpenStack cloud for Australia’s academic researchers. Since its inception in 2012 it has grown steadily to over 30,000 CPUs, with over 10,000 registered users from more than 50 research institutions. It is different to many clouds in being a federation across eight organisations, each of which runs cloud infrastructure in one or more data centres and contributes to a distributed help desk and user support. A Nectar core services team runs centralised cloud services. This presentation will give an overview of the experiences, challenges and benefits of running a federated OpenStack cloud and a short demonstration on using the Nectar cloud. We will also describe some current approaches that are looking to extend this federation to encompass other institutions including some in New Zealand, to extend the infrastructure using commercial cloud providers, and to move towards interoperability with the growing number of international science and research clouds through the new Open Research Cloud initiative.
Speaker Bio
Dr Paul Coddington is a Deputy Director of Nectar, responsible for the Nectar national Research Cloud, and also Deputy Director of eResearch SA. He has over 30 years experience in eResearch including computational science, high performance and distributed computing, cloud computing, software development, and research data management.
I am text block. Click edit button to change this text. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
Simplifying the Move to OpenStack
Audience Level
Intermediate
Synopsis
In this presentation, Shunde will show you how to simplify the migration process with a workload migration engine, making the move to OpenStack easy. This talk will address the various difficulties operators and administrators face when migrating workloads and resources between various cloud platforms, including removing time consuming, repetitive and complicated steps.
This tool can be applied to many cloud migrations, including between Virtual Machines and OpenStack, between Public and Private clouds, as well as between OpenStack and OpenStack. This tool integrates completely with other OpenStack projects minimising deployment and maintenance efforts. So whether you’re looking to upgrade from your existing traditional virtualisation platform, setup a new OpenStack instance, or upgrade to a newer version of OpenStack, we will show you how to simplify this process using GUTS.
Speaker Bio
Shunde is a senior software developer in Aptira with over 15 years experience in software development, automation and system administration. He has worked with OpenStack since the Diablo cycle and has been involved in projects from OpenStack infrastructure to distributed systems running on top of OpenStack.
Hyperconverged Cloud, Not just a toy anymore - Andrew Hatfield, Red Hat
Audience Level
Intermediate
Synopsis
Hypercoverged Compute, Network and Storage is ready for production workloads – where it makes sense.
Whether you’re a telecommunications carrier, service provider or enterprise; implementing Network Function Virtualisation (NFV), focusing on specific known workloads or simply a dev / test cloud – deploying a hypercoverged OpenStack cloud makes a lot of sense.
Come along and discover which workloads fit a hyperconverged architecture, see examples and look into the very near future and learn how OpenStack is truly ready to serve your every need.
Speaker Bio
Andrew has over 20 years experience in the IT industry across APAC, specialising in Databases, Directory Systems, Groupware, Virtualisation and Storage for Enterprise and Government organisations. When not helping customers slash costs and increase agility by moving to the software-defined future, he’s enjoying the subtle tones of Islay Whisky and shredding pow pow on the world’s best snowboard resorts.
Migrating your infrastructure to OpenStack - Avi Miller, Oracle
The document discusses challenges in migrating from a traditional infrastructure to OpenStack. It outlines some initial steps such as mapping current infrastructure components to OpenStack services and identifying initial workloads. It also covers some technical challenges around deploying and managing OpenStack components like Compute, Networking, Storage, Identity Management and upgrades. The document shares a case study of Oracle Product Development IT's migration of 14,000 VMs to OpenStack.
A glimpse into an industry Cloud using Open Source Technologies - Adrian Koh,...
Audience Level
All levels
Synopsis
Often times, prospects and existing OpenStack users wonder if there is indeed strong business and technical value proposition for cloud platforms serving a specific industry vertical.
In this session, EasyStack would like to share with the participants our experience engaging with an industry leader to build a credible solution platform catering to their current and future business and technology roadmap.
Speaker Bio:
Adrian is Director Global Business Development in EasyStack and has 20 years working experience in leading tech companies in the IT industry.
Prior to joining EasyStack, Adrian was with IBM Singapore and IBM China and served in roles such as Offering Manager, Engagement Manager, Solution Architect, Services Consultant, IT Specialist.
Adrian holds a Bachelor of Science degree with honors in Computer Sciences from University of Texas at Austin.
Enabling OpenStack for Enterprise - Tarso Dos Santos, Veritas
Audience Level
All levels
Synopsis
OpenStack offers many advantages for organisations building out their cloud environments, including flexibility and community-driven innovation. However, enterprises looking to deploy OpenStack in production typically find its storage management capabilities wanting from the perspective of management complexity and business resiliency. Enterprises are also challenged when it comes to ensuring protection of their data and providing the necessary performance – especially for their tier one applications. Meeting these fundamental needs is critical for enterprises to proceed confidently with their OpenStack deployments.
Veritas HyperScale for OpenStack is a software-defined storage management solution uniquely developed for OpenStack based clouds. It leverages direct attached storage (DAS) and provides enterprise-strength capabilities that enable robust, production-scale deployment while meeting performance and data protection needs. Learn how this innovative solution, coupled with other relevant Veritas offerings, solve the remaining issues around implementing OpenStack within the enterprise.
Speaker Bio:
Tarso dos Santos works as a Technical Account Manager at Veritas, directly engaging with customers to develop strategies, architectures and solutions with focus on Cloud – Openstack, Containers, Data Protection, High Availability and Compliance.
He has over 21 years in the IT industry architecting, delivering and positioning solutions such as private clouds, distributed systems, hpc, storage, and high available platforms.
Tarso has a great interest in distributed systems performance, and scientific organizations that push the boundaries of existing technologies, but also need to link these into the Enterprise.
Tarso in his life has enjoyed working in some of the most amazing projects ranging from mission critical systems protecting Australian lives, to IT infrastructure projects that are looking at the sky and discovering new planets out in the space.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Understanding blue store, Ceph's new storage backend - Tim Serong, SUSE
Audience Level
Intermediate
Synopsis
Ceph – the most popular storage solution for OpenStack – stores all data as a collection of objects. This object store was originally implemented on top of a POSIX filesystem, an approach that turned out to have a number of problems, notably with performance and complexity.
BlueStore, a new storage backend for Ceph, was created to solve these issues; the Ceph Jewel release included an early prototype. The code and on-disk format were declared stable (but experimental) for Ceph Kraken, and now in the upcoming Ceph Luminous release, BlueStore will be the recommended default storage backend.
With a 2-3x performance boost, you’ll want to look at migrating your Ceph clusters to BlueStore. This talk goes into detail about what BlueStore does, the problems it solves, and what you need to do to use it.
Speaker Bio:
Tim works for SUSE, hacking on Ceph and related technologies. He has spoken often about distributed storage and high availability at conferences such as linux.conf.au. In his spare time he wrangles pigs, chickens, sheep and ducks, and was declared by one colleague “teammate most likely to survive the zombie apocalypse”.
OpenStack Networks the Web-Scale Way - Scott Laffer, Cumulus Networks
Audience Level
Beginner
Synopsis
Layer 2 versus Layer 3, MLAG, Spanning-Tree, switch mechanism drivers, overlays and routing-on-the-host — What scales and what does not? The underlying plumbing of an OpenStack network is something you’d rather not have to think about. This presentation examines the network architectures of web-scale and large enterprise OpenStack users and how those same efficiencies can be used in deployments of all sizes.
Speaker Bio:
Scott is a Member of Technical Staff at Cumulus Networks where he designs, supports and deploys web-scale technologies and architectures in enterprise networks globally. Prior to becoming a founding member of the Cumulus office in Australia, Scott started his career as a network administrator before joining Cisco Systems to support their data centre products.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Diving in the desert: A quick overview into OpenStack Sahara capabilities - A...
This document provides an overview of using Hadoop on OpenStack. It discusses Hadoop and MapReduce, the benefits of running Hadoop on OpenStack including scalability by decoupling storage and compute. It also notes that major enterprise Hadoop distributions are available as images for OpenStack and concludes with a promise of a demo of these capabilities.
Building a GPU-enabled OpenStack Cloud for HPC - Blair Bethwaite, Monash Univ...
Audience Level
Intermediate
Synopsis
M3 is the latest generation system of the MASSIVE project, an HPC facility specializing in characterization science (imaging and visualization). Using OpenStack as the compute provisioning layer, M3 is a hybrid HPC/cloud system, custom-integrated by Monash’s R@CMon Research Cloud team. Built to support Monash University’s next-gen high-throughput instrument processing requirements, M3 is half-half GPU-accelerated and CPU-only.
We’ll discuss the design and tech used to build this innovative platform as well as detailing approaches and challenges to building GPU-enabled and HPC clouds. We’ll also discuss some of the software and processing pipelines that this system supports and highlight the importance of tuning for these workloads.
Speaker Bio
Blair Bethwaite: Blair has worked in distributed computing at Monash University for 10 years, with OpenStack for half of that. Having served as team lead, architect, administrator, user, researcher, and occasional hacker, Blair’s unique perspective as a science power-user, developer, and system architect has helped guide the evolution of the research computing engine central to Monash’s 21st Century Microscope.
Lance Wilson: Lance is a mechanical engineer, who has been making tools to break things for the last 20 years. His career has moved through a number of engineering subdisciplines from manufacturing to bioengineering. Now he supports the national characterisation research community in Melbourne, Australia using OpenStack to create HPC systems solving problems too large for your laptop.
OpenStack and Red Hat: How we learned to adapt with our customers in a maturi...
Audience Level
All levels
Synopsis
Peter has been involved in OpenStack community since its B-release, and he has been enabling and helping customers across various industries adopt OpenStack in strategic ways. In this session, you will learn from his experience what Red Hat’s perspective is on the current state of affairs in the OpenStack community and the path we see ahead that Red Hat is putting its efforts in. OpenStack is not a product that tries to solve any one business problem in particular, but a technology that aims to be usable for many – what are the required steps to make sure that your organisation is ready for the OpenStack-based cloudification and transformation.
Speaker Bio:
Peter Jung is a Senior Business Development Manager at Red Hat where he leads the practice in the areas of Cloud, SDN/NFV and IoT across Australia and New Zealand. He is passionate about open innovation and open source software development model as the foundation for next generation society and ICT systems. Prior to Red Hat, he had various roles at Cisco and Dell for 15 years. He holds a BSEE and an MBA.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Meshing OpenStack and Bare Metal Networks with EVPN - David Iles, Mellanox Te...
Audience Level
Intermediate
Synopsis
The latest SDN revolution is centered on creating efficient virtualized data center networks using VXLAN & EVPN. We will talk about the scale, performance, and cost advantages of using a modern controller-free virtualized network solution built on 100 Gigabit Ethernet switches with hardware based VXLAN Routing. We will explore the ease of automating such a network in an OpenStack environment and take you through a real world use case of using OpenStack Network Node bridging between a bare metal cloud (EVPN) and a fully virtualized cloud environments (orchestrated by Neutron).
Speaker Bio:
David has held leadership roles at 3COM, Cisco Systems, Nortel Networks, and IBM where he promoted advanced network technologies including High Speed Ethernet, Layer 4-7 switching, Virtual Machine-aware networking, and Software Defined Networking.
David’s current focus is on the evolving landscape of data center networking, scale out storage, Open Networking, and cloud computing.
The Why and How of HPC-Cloud Hybrids with OpenStack - Lev Lafayette, Universi...
Audience Level
Intermediate
Synopsis
High performance computing and cloud computing have traditionally been seen as separate solutions to separate problems, dealing with issues of performance and flexibility respectively. In a diverse research environment however, both sets of compute requirements can occur. In addition to the administrative benefits in combining both requirements into a single unified system, opportunities are provided for incremental expansion.
The deployment of the Spartan cloud-HPC hybrid system at the University of Melbourne last year is an example of such a design. Despite its small size, it has attracted international attention due to its design features. This presentation, in addition to providing a grounding on why one would wish to build an HPC-cloud hybrid system and the results of the deployment, provides a complete technical overview of the design from the ground up, as well as problems encountered and planned future developments.
Speaker Bio
Lev Lafayette is the HPC and Training Officer at the University of Melbourne. Prior to that he worked at the Victorian Partnership for Advanced Computing for several years in a similar role.
Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Ba...
Audience Level
All levels
Synopsis
Our journey towards solving our Application and Infrastructure Problems using Immutability, Codification, Mesos, Docker and Ironic.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Traditional Enterprise to OpenStack Cloud - An Unexpected Journey
Audience Level
Intermediate
Synopsis
Hostworks is a part of the Inabox group and an Australian based Managed Hosting and Services Provider with a strong background in management of infrastructure, servers, and applications following Traditional Enterprise management practices. 2 years ago, we began our transition to adopt and deploy cloud technologies and mindsets with an on On-Premises OpenStack platform as part of a larger hybrid cloud offering. This presentation details how we did it, what went wrong, and what went right, important lessons and recommendations for other businesses who wish to follow the same path.
Speaker Bio:
Daniel is Platform Engineer at Hostworks and specialises in their On-Premises OpenStack cloud offering.
Building a GPU-enabled OpenStack Cloud for HPC - Lance Wilson, Monash University
Audience Level
Intermediate
Synopsis
M3 is the latest generation system of the MASSIVE project, an HPC facility specializing in characterization science (imaging and visualization). Using OpenStack as the compute provisioning layer, M3 is a hybrid HPC/cloud system, custom-integrated by Monash’s R@CMon Research Cloud team. Built to support Monash University’s next-gen high-throughput instrument processing requirements, M3 is half-half GPU-accelerated and CPU-only.
We’ll discuss the design and tech used to build this innovative platform as well as detailing approaches and challenges to building GPU-enabled and HPC clouds. We’ll also discuss some of the software and processing pipelines that this system supports and highlight the importance of tuning for these workloads.
Speaker Bio
Blair Bethwaite: Blair has worked in distributed computing at Monash University for 10 years, with OpenStack for half of that. Having served as team lead, architect, administrator, user, researcher, and occasional hacker, Blair’s unique perspective as a science power-user, developer, and system architect has helped guide the evolution of the research computing engine central to Monash’s 21st Century Microscope.
Lance Wilson: Lance is a mechanical engineer, who has been making tools to break things for the last 20 years. His career has moved through a number of engineering subdisciplines from manufacturing to bioengineering. Now he supports the national characterisation research community in Melbourne, Australia using OpenStack to create HPC systems solving problems too large for your laptop.
Monitoring Uptime on the NeCTAR Research Cloud - Andy Botting, University of ...
Audience Level
Intermediate
Synopsis
We will discuss how we do monitoring on the Nectar research cloud, utilising tools like OpenStack tempest, Nagios and translating this into a user facing dashboard.
Speaker Bio:
Andy is a DevOps engineer working at the University of Melbourne in the Core Services team for the Nectar Research Cloud.
More from OpenStack (20)
Swinburne University of Technology - Shunde Zhang & Kieran Spear, Aptira
Swinburne University of Technology - Shunde Zhang & Kieran Spear, Aptira
Related OSS Projects - Peter Rowe, Flexera Software
Related OSS Projects - Peter Rowe, Flexera Software
Supercomputing by API: Connecting Modern Web Apps to HPC
Supercomputing by API: Connecting Modern Web Apps to HPC
Federation and Interoperability in the Nectar Research Cloud
Federation and Interoperability in the Nectar Research Cloud
Hyperconverged Cloud, Not just a toy anymore - Andrew Hatfield, Red Hat
Hyperconverged Cloud, Not just a toy anymore - Andrew Hatfield, Red Hat
Migrating your infrastructure to OpenStack - Avi Miller, Oracle
Migrating your infrastructure to OpenStack - Avi Miller, Oracle
A glimpse into an industry Cloud using Open Source Technologies - Adrian Koh,...
A glimpse into an industry Cloud using Open Source Technologies - Adrian Koh,...
Enabling OpenStack for Enterprise - Tarso Dos Santos, Veritas
Enabling OpenStack for Enterprise - Tarso Dos Santos, Veritas
Understanding blue store, Ceph's new storage backend - Tim Serong, SUSE
Understanding blue store, Ceph's new storage backend - Tim Serong, SUSE
OpenStack Networks the Web-Scale Way - Scott Laffer, Cumulus Networks
OpenStack Networks the Web-Scale Way - Scott Laffer, Cumulus Networks
Diving in the desert: A quick overview into OpenStack Sahara capabilities - A...
Diving in the desert: A quick overview into OpenStack Sahara capabilities - A...
Building a GPU-enabled OpenStack Cloud for HPC - Blair Bethwaite, Monash Univ...
Building a GPU-enabled OpenStack Cloud for HPC - Blair Bethwaite, Monash Univ...
OpenStack and Red Hat: How we learned to adapt with our customers in a maturi...
OpenStack and Red Hat: How we learned to adapt with our customers in a maturi...
Meshing OpenStack and Bare Metal Networks with EVPN - David Iles, Mellanox Te...
Meshing OpenStack and Bare Metal Networks with EVPN - David Iles, Mellanox Te...
The Why and How of HPC-Cloud Hybrids with OpenStack - Lev Lafayette, Universi...
The Why and How of HPC-Cloud Hybrids with OpenStack - Lev Lafayette, Universi...
Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Ba...
Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Ba...
Traditional Enterprise to OpenStack Cloud - An Unexpected Journey
Traditional Enterprise to OpenStack Cloud - An Unexpected Journey
Building a GPU-enabled OpenStack Cloud for HPC - Lance Wilson, Monash University
Building a GPU-enabled OpenStack Cloud for HPC - Lance Wilson, Monash University
Monitoring Uptime on the NeCTAR Research Cloud - Andy Botting, University of ...
Monitoring Uptime on the NeCTAR Research Cloud - Andy Botting, University of ...
Recently uploaded
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Project Management Semester Long Project - Acuity
Acuity is an innovative learning app designed to transform the way you engage with knowledge. Powered by AI technology, Acuity takes complex topics and distills them into concise, interactive summaries that are easy to read & understand. Whether you're exploring the depths of quantum mechanics or seeking insight into historical events, Acuity provides the key information you need without the burden of lengthy texts.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Choosing The Best AWS Service For Your Website + API.pptx
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Skybuffer SAM4U tool for SAP license adoption
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Recently uploaded (20)
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
Aptira presents OpenStack swift architecture and monitoring
- 1. Aptira OpenStack Architecture and Monitoring Indian Openstack users group 5th May 2012
- 2. Agenda ● Explain OpenStack Swift Architecture ● Look at key components to be monitored ● Look at various options available that work today
- 4. Swift Components • Proxy Server • The Ring • The Object Server/Storage nodes • Account Server • Replication • Updaters and Auditors
- 5. What to monitor? ● Hardware failure ● Operating System failure ● Swift Cluster health ● Swift Cluster telemetry
- 6. Hardware Failure and Operating Systems ● Hard drive failure detection in Swift Use swift-drive-audit Other Common metrics like • • CPU usage • RAM usage • Network usage etc.
- 7. OpenStack Swift Cluster health monitoring ● Use swift-dispersion-report tool Based on config file /etc/swift/dispersion.conf
- 8. Cluster Telemetry and Monitoring ● swift-recon middleware The Swift Recon middleware can provide general machine stats (load average, socket stats, /proc/meminfo contents, etc.) as well as Swift-specific metrics: The MD5 sum of each ring file. The most recent object replication time. Count of each type of quarantined file: account, container, or object. Count of “async_pendings” (deferred container updates) on disk.
- 9. Swift-Recon Config section to be added in object-server.conf Following information available
- 10. What works today? ● A lot of development going on around OpenStack in the community. Existing enterprise grade systems have written modules for OpenStack ● Nagios ● Zenoss
- 11. What works today? ● Nagios Exchange Addon check_swift – http://exchange.nagios.org/directory/Plugins/Clustering-and-High- 2DAvailability/check_swift/details ● The components – check_swift : tries to upload, download and delete a file in a Swift container to check that it works correctly – check_swift_dispersion: uses swift-dispersion tools to report dispersion analysis and checks that all copies of objects are OK – check_swift_object_servers : uses swift-recon to query all clusters servers and ensure they all have the same copy of the object ring. ● Compatible with Nagios 3.x
- 12. What works today? ● Zenoss Zen Pack Open Stack Swift – https://github.com/zenoss/ZenPacks.zenoss.OpenStackSwift ● All of the monitoring currently performance is done through the optional swift-recon API endpoint that can be enabled on all of your Swift object servers. Before using this ZenPack you must install and configure swift-recon on your Swift object servers. ● Compatible with versions 3.2 to 4.0