OSS Presentation OpenStack Swift by Joe Arnold
•
3 likes•1,178 views
Swift is an object storage system that can store millions to billions of objects with a URL for each. It uses a proxy tier, ring mapping, storage zones for fault isolation, and continuous replication to provide scalability and reliability. Example uses of Swift include Cloud Files storage, long tail data storage, running VMs, database volumes, and asset storage.
Report
Share
Report
Share
![Joe Arnold [email_address] http://swiftstack.com OpenStack Swift](https://image.slidesharecdn.com/openstackswift-joearnold-openstoragesummit-2011-10-26-111206154221-phpapp01/85/OSS-Presentation-OpenStack-Swift-by-Joe-Arnold-1-320.jpg)
Recommended
Understanding AWS Storage Options
AWS provides multiple storage options to meet your varying needs. This presentation provides an overview of how AWS Cloud storage services can be used to support application development and delivery, backup, archive, disaster recovery, and virtualized compute.
Cloud db
This document discusses using relational database management systems (RDBMS) in the cloud. It outlines the steps to provision a MySQL database instance on Amazon RDS, including choosing the database and instance type, version, storage, password, input/output options, ports, security groups, and backups. It also mentions using R and RODBC to connect to and analyze data in Cloud MySQL databases, and compares other cloud platforms like Google, Oracle, and Windows for hosting RDBMS and analytics workloads.
AWS Customer Presentation - AideRss
Ilya Grigorik of AideRSS presents at AWS Start-Up Event - Toronto on their extensive use of AWS for their RSS aggregator and organizer.
MidLink - re:Invent 2018 meetup presentation - new amazon s3 features
This document summarizes Amazon S3 storage services including S3 storage classes like Standard, Infrequent Access, and Intelligent Tiering. It provides examples of common use cases like backup/restore, disaster recovery, and data lakes. Details are given on S3 Intelligent Tiering such as minimum object size, pricing, and storage duration. The document also describes AWS Batch Operations for managing large numbers of S3 objects and AWS Transfer for using S3 as an SFTP server without managing your own SFTP infrastructure. Pricing details are provided for AWS Batch Operations and AWS Transfer.
Amazon EMR Facebook Presto Meetup
Steve McPherson presented on Amazon EMR (Elastic MapReduce) at the Facebook Presto Meetup on March 19, 2015. Amazon EMR makes cluster management easy by handling setup and configuration, node monitoring and replacement, log aggregation, and integration with CloudWatch and Spot instances. Amazon EMR supports various big data frameworks like MapReduce, Spark, Hive, Pig, Presto and data stores like Amazon S3. It allows running different clusters optimized for different workloads like Hive/Pig, Presto or Spark.
If you doing file uploads with rails you're gonna have a bad time
Slide from SFRails February 21, 2013 - A quick overview of the perrils of uploading files with Ruby on Rails and how using Filepicker.io can help. https://github.com/Filepicker/filepicker-rails
Bring Your Cloud Bill Back Down to Earth
Is your Cloud Bill soaring too high? In this presentation we will be exploring how to:
-Drastically lower cloud spend and to more efficiently deploy, use, grow, and manage your cloud infrastructure.
- Optimize storage to compute and everything in between.
-Identify the best techniques, and how and when to use them.
While the discussion will center around AWS, the same techniques can generally be applied to Azure, GCP, and others.
You’ll leave this session ready to employ impactful cost optimization solutions to your environment that will start saving you money immediately.
AWS EMR (Elastic Map Reduce) explained
This presentation explains the fastest way to build a Hadoop cluster on cloud and be able to use it within minutes.
Recommended
Understanding AWS Storage Options
AWS provides multiple storage options to meet your varying needs. This presentation provides an overview of how AWS Cloud storage services can be used to support application development and delivery, backup, archive, disaster recovery, and virtualized compute.
Cloud db
This document discusses using relational database management systems (RDBMS) in the cloud. It outlines the steps to provision a MySQL database instance on Amazon RDS, including choosing the database and instance type, version, storage, password, input/output options, ports, security groups, and backups. It also mentions using R and RODBC to connect to and analyze data in Cloud MySQL databases, and compares other cloud platforms like Google, Oracle, and Windows for hosting RDBMS and analytics workloads.
AWS Customer Presentation - AideRss
Ilya Grigorik of AideRSS presents at AWS Start-Up Event - Toronto on their extensive use of AWS for their RSS aggregator and organizer.
MidLink - re:Invent 2018 meetup presentation - new amazon s3 features
This document summarizes Amazon S3 storage services including S3 storage classes like Standard, Infrequent Access, and Intelligent Tiering. It provides examples of common use cases like backup/restore, disaster recovery, and data lakes. Details are given on S3 Intelligent Tiering such as minimum object size, pricing, and storage duration. The document also describes AWS Batch Operations for managing large numbers of S3 objects and AWS Transfer for using S3 as an SFTP server without managing your own SFTP infrastructure. Pricing details are provided for AWS Batch Operations and AWS Transfer.
Amazon EMR Facebook Presto Meetup
Steve McPherson presented on Amazon EMR (Elastic MapReduce) at the Facebook Presto Meetup on March 19, 2015. Amazon EMR makes cluster management easy by handling setup and configuration, node monitoring and replacement, log aggregation, and integration with CloudWatch and Spot instances. Amazon EMR supports various big data frameworks like MapReduce, Spark, Hive, Pig, Presto and data stores like Amazon S3. It allows running different clusters optimized for different workloads like Hive/Pig, Presto or Spark.
If you doing file uploads with rails you're gonna have a bad time
Slide from SFRails February 21, 2013 - A quick overview of the perrils of uploading files with Ruby on Rails and how using Filepicker.io can help. https://github.com/Filepicker/filepicker-rails
Bring Your Cloud Bill Back Down to Earth
Is your Cloud Bill soaring too high? In this presentation we will be exploring how to:
-Drastically lower cloud spend and to more efficiently deploy, use, grow, and manage your cloud infrastructure.
- Optimize storage to compute and everything in between.
-Identify the best techniques, and how and when to use them.
While the discussion will center around AWS, the same techniques can generally be applied to Azure, GCP, and others.
You’ll leave this session ready to employ impactful cost optimization solutions to your environment that will start saving you money immediately.
AWS EMR (Elastic Map Reduce) explained
This presentation explains the fastest way to build a Hadoop cluster on cloud and be able to use it within minutes.
Summer'14 release presentation by Manish
The Salesforce Summer '14 Release includes the following key updates:
1. Increased limits for workflow, assignment, auto-response, and escalation rules, as well as character limits for text area fields.
2. New capabilities for manipulating multiple Salesforce fields and records at once using sObject and collection variables in Visual Flows.
3. Enhancements to Visual Flow elements like Fast Lookup, Fast Create, and loops to simplify common workflows.
Demonstrating 100 Gbps in and out of the public Clouds
Poster presented at PEARC20.
There is increased awareness and recognition that public Cloud providers do provide capabilities not found elsewhere, with elasticity being a major driver. The value of elastic scaling is however tightly coupled to the capabilities of the networks that connect all involved resources, both in the public Clouds and at the various research institutions. This poster presents results of measurements involving file transfers inside public Cloud providers, fetching data from on-prem resources into public Cloud instances and fetching data from public Cloud storage into on-prem nodes. The networking of the three major Cloud providers, namely Amazon Web Services, Microsoft Azure and the Google Cloud Platform, has been benchmarked. The on-prem nodes were managed by either the Pacific Research Platform or located at the University of Wisconsin – Madison. The observed sustained throughput was of the order of 100 Gbps in all the tests moving data in and out of the public Clouds and throughput reaching into the Tbps range for data movements inside the public Cloud providers themselves. All the tests used HTTP as the transfer protocol.
Atlantisで実現するTerraformのGitOps
The document discusses using Terraform and Atlantis for infrastructure as code on AWS. It provides an overview of Terraform and how it can be used to define AWS resources like S3 buckets. It then introduces Atlantis as a tool that integrates with version control systems like GitHub to run Terraform plans on pull requests and apply changes once approved. Key features of Atlantis mentioned are that it allows pull request based planning and applying of Terraform code.
How to reduce hosting costs for Redis based applications on Java
In this presentation we are going to look how Redisson PRO can reduce hosting costs and rise Redis overall performance.
Lighthouse - an open-source library to build data lakes - Kris Peeters
Data Minded built an open-source library to build data lakes
6th Data Science Leuven Meetup: https://github.com/datamindedbe/lighthouse
Amazon DynamoDB by Aswin
DynamoDB is a fully managed NoSQL database service from Amazon that provides unlimited storage and throughput scaling without downtime. The document discusses getting started with DynamoDB, including defining primary keys like hash and hash-range, provisioning throughput, and using secondary indexes. Best practices include uploading big data to S3 and having DynamoDB only store pointers, as well as designing uniform workloads that avoid limited hash keys and use distinct values.
Bleeding Edge Databases
This document provides information on and demonstrations of several bleeding edge database technologies: Aerospike, Algebraix Data, and Google BigQuery. It includes benchmark results, architecture diagrams, pricing and deployment details for each one. Example use cases and instructions for getting started with the technologies are also provided.
Azure: Lessons From The Field
This is a presentation I delivered at CodeMash 2.0.1.0 dealing with lessons learned while building an application for handling the post-processing of scientific data using the Windows Azure platform.
Aws best practices
The document discusses best practices for architecting applications in AWS. It recommends choosing AWS regions based on factors like proximity, availability of services, and cost. It also recommends building security into every layer, leveraging different storage options like S3 and DynamoDB, implementing elasticity through auto-scaling, using caching to improve performance, and designing systems to be fault-tolerant by eliminating single points of failure and using features like multiple availability zones.
Windows Azure: Lessons From The Field
Presentation at CodeStock 2010 on the lessons learned while building an application that handles post-processing of scientific data in the Cloud.
AWS Cloud Kata 2014 | Jakarta - 2-1 AWS Intro and Scale 2014
This document provides an overview of strategies for building scalable applications on AWS. It recommends starting simply with EC2, RDS, and Route 53, then adding services like S3, DynamoDB, ElastiCache, and CloudFront to optimize performance. Auto Scaling is introduced to automatically scale resources based on demand. The document discusses best practices like separating databases by function, implementing sharding, and leveraging serverless options. The goal is to demonstrate how these techniques can help applications scale to millions of users on AWS.
(SOV204) Scaling Up to Your First 10 Million Users | AWS re:Invent 2014
Cloud computing gives you a number of advantages, such as the ability to scale your application on demand. If you have a new business and want to use cloud computing, you might be asking yourself, andquot;Where do I start?andquot; Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
Azure, Cloud Computing & Services
Azure provides cloud computing services including Windows Azure, which offers an operating system for the cloud with features like service management, compute, storage, and developer experience. Windows Azure storage provides scalable and durable storage of blobs, tables, and queues to store and retrieve data. It ensures high availability, durability, and scalability through data replication and automatic load balancing.
Getting Started with Docker on AWS - DevDay Austin 2017
This document provides an overview of Docker containers and Amazon ECS for running containers on AWS. It discusses how containers can help deploy applications across different environments consistently. It then introduces Docker and how images are built. Amazon ECS is presented as a fully managed service for running Docker containers across a cluster of EC2 instances, providing scheduling and management of containers. Key features of ECS like load balancing, scaling, and integration with other AWS services are highlighted.
AWS Storage Options
This is a presentation that covers AWS's storage options and how they compare to traditional on-premises storage options.
Scaling on AWS for the First 10 Million Users (ARC206) | AWS re:Invent 2013
Cloud computing gives you a number of advantages in being able to scale on demand, easily replace whole parts of your infrastructure, and much more. As a new business looking to use the cloud, you inevitably ask yourself, Where do I start? Join us at this session to understand some of the common patterns and recommended areas of focus you can expect to work through while scaling an infrastructure to handle going from zero to millions of users. From leveraging highly scalable AWS services to making smart decisions on building out your application, you'll learn a number of best practices for scaling your infrastructure in the cloud. The patterns and practices reviewed in this session will get you there.
Getting Started with Docker on AWS - DevDay Los Angeles 2017
The document discusses getting started with Docker on AWS. It provides an overview of containers and Docker, and introduces Amazon ECS for managing Docker containers on AWS. Key points include: Docker provides portable application environments; Amazon ECS manages container scheduling across EC2 instances; and services in ECS allow deploying containers behind a load balancer for long-running applications.
Leveraging Amazon's Elastic Block Store
RightScale Webinar: August 28, 2008 – This webinar focuses on Amazon's Elastic Block Store (EBS) with specific instructions and a live demo. Watch the video at http://vimeo.com/rightscale/leveraging-amazons-elastic-block-store.
Amazon Aurora: Under the Hood
The document discusses Amazon Aurora, a database service from AWS that is compatible with PostgreSQL and MySQL. It provides summaries of Aurora's architecture, performance advantages, and customer benefits compared to traditional databases. Specifically, the document notes that Aurora achieves higher performance and availability than PostgreSQL by using a distributed, scalable storage system and replicating data across Availability Zones. It shares performance test results showing that Aurora can be up to 3x faster than PostgreSQL for various workloads. Customers have also cited lower costs and easier management with Aurora compared to commercial databases.
Deep Dive: Scaling Up to Your First 10 Million Users
This document discusses best practices for scaling infrastructure on AWS to support over 10 million users. It begins by recommending using multiple AWS regions and availability zones for redundancy. It then walks through scaling a simple single-instance application to be horizontally and vertically scaled across multiple instances, database read replicas, caching, and content delivery. Key services discussed include EC2, RDS, DynamoDB, ElastiCache, S3, CloudFront, Route 53, and Auto Scaling. Automating management using tools and separating concerns like static assets are also recommended.
Tech-Spark: Exploring the Cosmos DB
The event, held on 27th April 2019, was part of the Global Azure Bootcamp and covered Microsoft's Cosmos DB, more specifically:
- Introduction to Cosmos DB, its features, internals, resource models, and request units.
- DEMO: Create an SQL API. Download sample .NET app. Simple queries.
- Covered Change Feed and showcased various use case scenarios.
- Detailed Global Distribution and Consistency Models implications.
- DEMO: Mongo - Lift and shift. Run simple .NET code against a MongoDB (in docker container) and cosmos.
- Introduction to Tinkerpop graphs
- DEMO: Graphs API. Download sample .NET app. Simple queries.
https://techspark.mt/global-azure-bootcamp-27th-april-2019/
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Organizations face significant challenges moving their applications to the cloud when they require a standard file system interface for accessing their cloud data. In this technical session, we will explore the world’s first cloud-scale file system and its targeted use cases. Attendees will learn about the Amazon Elastic File System (EFS) features and benefits, how to identify applications that are appropriate for use with Amazon EFS, and details about its performance and security models. We will highlight and demonstrate how to deploy Amazon EFS in one of our most common use cases and will share tips for success throughout.
Learning Objectives:
• Recognize why and when to use Amazon EFS
• Understand key technical/security concepts
• Learn how to leverage EFS’s performance
• See a demo of EFS in action
• Review EFS’s economics
More Related Content
What's hot
Summer'14 release presentation by Manish
The Salesforce Summer '14 Release includes the following key updates:
1. Increased limits for workflow, assignment, auto-response, and escalation rules, as well as character limits for text area fields.
2. New capabilities for manipulating multiple Salesforce fields and records at once using sObject and collection variables in Visual Flows.
3. Enhancements to Visual Flow elements like Fast Lookup, Fast Create, and loops to simplify common workflows.
Demonstrating 100 Gbps in and out of the public Clouds
Poster presented at PEARC20.
There is increased awareness and recognition that public Cloud providers do provide capabilities not found elsewhere, with elasticity being a major driver. The value of elastic scaling is however tightly coupled to the capabilities of the networks that connect all involved resources, both in the public Clouds and at the various research institutions. This poster presents results of measurements involving file transfers inside public Cloud providers, fetching data from on-prem resources into public Cloud instances and fetching data from public Cloud storage into on-prem nodes. The networking of the three major Cloud providers, namely Amazon Web Services, Microsoft Azure and the Google Cloud Platform, has been benchmarked. The on-prem nodes were managed by either the Pacific Research Platform or located at the University of Wisconsin – Madison. The observed sustained throughput was of the order of 100 Gbps in all the tests moving data in and out of the public Clouds and throughput reaching into the Tbps range for data movements inside the public Cloud providers themselves. All the tests used HTTP as the transfer protocol.
Atlantisで実現するTerraformのGitOps
The document discusses using Terraform and Atlantis for infrastructure as code on AWS. It provides an overview of Terraform and how it can be used to define AWS resources like S3 buckets. It then introduces Atlantis as a tool that integrates with version control systems like GitHub to run Terraform plans on pull requests and apply changes once approved. Key features of Atlantis mentioned are that it allows pull request based planning and applying of Terraform code.
How to reduce hosting costs for Redis based applications on Java
In this presentation we are going to look how Redisson PRO can reduce hosting costs and rise Redis overall performance.
Lighthouse - an open-source library to build data lakes - Kris Peeters
Data Minded built an open-source library to build data lakes
6th Data Science Leuven Meetup: https://github.com/datamindedbe/lighthouse
Amazon DynamoDB by Aswin
DynamoDB is a fully managed NoSQL database service from Amazon that provides unlimited storage and throughput scaling without downtime. The document discusses getting started with DynamoDB, including defining primary keys like hash and hash-range, provisioning throughput, and using secondary indexes. Best practices include uploading big data to S3 and having DynamoDB only store pointers, as well as designing uniform workloads that avoid limited hash keys and use distinct values.
Bleeding Edge Databases
This document provides information on and demonstrations of several bleeding edge database technologies: Aerospike, Algebraix Data, and Google BigQuery. It includes benchmark results, architecture diagrams, pricing and deployment details for each one. Example use cases and instructions for getting started with the technologies are also provided.
What's hot (7)
Demonstrating 100 Gbps in and out of the public Clouds
Demonstrating 100 Gbps in and out of the public Clouds
How to reduce hosting costs for Redis based applications on Java
How to reduce hosting costs for Redis based applications on Java
Lighthouse - an open-source library to build data lakes - Kris Peeters
Lighthouse - an open-source library to build data lakes - Kris Peeters
Similar to OSS Presentation OpenStack Swift by Joe Arnold
Azure: Lessons From The Field
This is a presentation I delivered at CodeMash 2.0.1.0 dealing with lessons learned while building an application for handling the post-processing of scientific data using the Windows Azure platform.
Aws best practices
The document discusses best practices for architecting applications in AWS. It recommends choosing AWS regions based on factors like proximity, availability of services, and cost. It also recommends building security into every layer, leveraging different storage options like S3 and DynamoDB, implementing elasticity through auto-scaling, using caching to improve performance, and designing systems to be fault-tolerant by eliminating single points of failure and using features like multiple availability zones.
Windows Azure: Lessons From The Field
Presentation at CodeStock 2010 on the lessons learned while building an application that handles post-processing of scientific data in the Cloud.
AWS Cloud Kata 2014 | Jakarta - 2-1 AWS Intro and Scale 2014
This document provides an overview of strategies for building scalable applications on AWS. It recommends starting simply with EC2, RDS, and Route 53, then adding services like S3, DynamoDB, ElastiCache, and CloudFront to optimize performance. Auto Scaling is introduced to automatically scale resources based on demand. The document discusses best practices like separating databases by function, implementing sharding, and leveraging serverless options. The goal is to demonstrate how these techniques can help applications scale to millions of users on AWS.
(SOV204) Scaling Up to Your First 10 Million Users | AWS re:Invent 2014
Cloud computing gives you a number of advantages, such as the ability to scale your application on demand. If you have a new business and want to use cloud computing, you might be asking yourself, andquot;Where do I start?andquot; Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
Azure, Cloud Computing & Services
Azure provides cloud computing services including Windows Azure, which offers an operating system for the cloud with features like service management, compute, storage, and developer experience. Windows Azure storage provides scalable and durable storage of blobs, tables, and queues to store and retrieve data. It ensures high availability, durability, and scalability through data replication and automatic load balancing.
Getting Started with Docker on AWS - DevDay Austin 2017
This document provides an overview of Docker containers and Amazon ECS for running containers on AWS. It discusses how containers can help deploy applications across different environments consistently. It then introduces Docker and how images are built. Amazon ECS is presented as a fully managed service for running Docker containers across a cluster of EC2 instances, providing scheduling and management of containers. Key features of ECS like load balancing, scaling, and integration with other AWS services are highlighted.
AWS Storage Options
This is a presentation that covers AWS's storage options and how they compare to traditional on-premises storage options.
Scaling on AWS for the First 10 Million Users (ARC206) | AWS re:Invent 2013
Cloud computing gives you a number of advantages in being able to scale on demand, easily replace whole parts of your infrastructure, and much more. As a new business looking to use the cloud, you inevitably ask yourself, Where do I start? Join us at this session to understand some of the common patterns and recommended areas of focus you can expect to work through while scaling an infrastructure to handle going from zero to millions of users. From leveraging highly scalable AWS services to making smart decisions on building out your application, you'll learn a number of best practices for scaling your infrastructure in the cloud. The patterns and practices reviewed in this session will get you there.
Getting Started with Docker on AWS - DevDay Los Angeles 2017
The document discusses getting started with Docker on AWS. It provides an overview of containers and Docker, and introduces Amazon ECS for managing Docker containers on AWS. Key points include: Docker provides portable application environments; Amazon ECS manages container scheduling across EC2 instances; and services in ECS allow deploying containers behind a load balancer for long-running applications.
Leveraging Amazon's Elastic Block Store
RightScale Webinar: August 28, 2008 – This webinar focuses on Amazon's Elastic Block Store (EBS) with specific instructions and a live demo. Watch the video at http://vimeo.com/rightscale/leveraging-amazons-elastic-block-store.
Amazon Aurora: Under the Hood
The document discusses Amazon Aurora, a database service from AWS that is compatible with PostgreSQL and MySQL. It provides summaries of Aurora's architecture, performance advantages, and customer benefits compared to traditional databases. Specifically, the document notes that Aurora achieves higher performance and availability than PostgreSQL by using a distributed, scalable storage system and replicating data across Availability Zones. It shares performance test results showing that Aurora can be up to 3x faster than PostgreSQL for various workloads. Customers have also cited lower costs and easier management with Aurora compared to commercial databases.
Deep Dive: Scaling Up to Your First 10 Million Users
This document discusses best practices for scaling infrastructure on AWS to support over 10 million users. It begins by recommending using multiple AWS regions and availability zones for redundancy. It then walks through scaling a simple single-instance application to be horizontally and vertically scaled across multiple instances, database read replicas, caching, and content delivery. Key services discussed include EC2, RDS, DynamoDB, ElastiCache, S3, CloudFront, Route 53, and Auto Scaling. Automating management using tools and separating concerns like static assets are also recommended.
Tech-Spark: Exploring the Cosmos DB
The event, held on 27th April 2019, was part of the Global Azure Bootcamp and covered Microsoft's Cosmos DB, more specifically:
- Introduction to Cosmos DB, its features, internals, resource models, and request units.
- DEMO: Create an SQL API. Download sample .NET app. Simple queries.
- Covered Change Feed and showcased various use case scenarios.
- Detailed Global Distribution and Consistency Models implications.
- DEMO: Mongo - Lift and shift. Run simple .NET code against a MongoDB (in docker container) and cosmos.
- Introduction to Tinkerpop graphs
- DEMO: Graphs API. Download sample .NET app. Simple queries.
https://techspark.mt/global-azure-bootcamp-27th-april-2019/
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Organizations face significant challenges moving their applications to the cloud when they require a standard file system interface for accessing their cloud data. In this technical session, we will explore the world’s first cloud-scale file system and its targeted use cases. Attendees will learn about the Amazon Elastic File System (EFS) features and benefits, how to identify applications that are appropriate for use with Amazon EFS, and details about its performance and security models. We will highlight and demonstrate how to deploy Amazon EFS in one of our most common use cases and will share tips for success throughout.
Learning Objectives:
• Recognize why and when to use Amazon EFS
• Understand key technical/security concepts
• Learn how to leverage EFS’s performance
• See a demo of EFS in action
• Review EFS’s economics
Deep Dive on Amazon Aurora
Amazon Aurora is a MySQL-compatible relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Amazon Aurora is disruptive technology in the database space, bringing a new architectural model and distributed systems techniques to provide far higher performance, availability and durability than previously available using conventional monolithic database techniques. In this session, we will do a deep-dive into some of the key innovations behind Amazon Aurora, discuss best practices and configurations, and share early customer experience from the field.
Deep Dive into EC2
This document provides a summary of Amazon Web Services (AWS) regions, availability zones, and the architecture behind EC2 instances and storage options. It explains that resources are bound to either an AWS account, region, or availability zone. The document describes how EC2 instances can use either Amazon Elastic Block Store (EBS) volumes or instance store for root disks, and how EBS volumes and machine images are stored, replicated, and made available across availability zones and regions. It also summarizes the two main types of AWS storage options - regular disks and key-value/NoSQL databases.
Aws for Startups Building Cloud Enabled Apps
The document discusses how to build cloud-enabled apps that can scale on AWS. It covers scaling vertically by increasing instance sizes, scaling horizontally by adding more instances, using auto-scaling to dynamically scale based on demand, distributing load with an ELB, scaling databases using read replicas and sharding, and taking advantage of managed database services like RDS and DynamoDB for easier administration. It also discusses decomposing applications into small, stateless components and using infrastructure as code for continuous deployment and agility.
Best Practices for Backup and Recovery: Windows Workload on AWS
Backing up Windows workloads can be a challenge, and cumbersome for many companies. Backup and recovery for Windows workloads on AWS, however, can be easy. This session will cover best practices for backup and recovery, how to configure Windows workloads to back up to AWS; pitfalls to look out for; and recommended reference architectures.
Deep Dive on Amazon Aurora
Amazon Aurora is a MySQL-compatible relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Amazon Aurora is disruptive technology in the database space, bringing a new architectural model and distributed systems techniques to provide far higher performance, availability and durability than previously available using conventional monolithic database techniques. In this session, we will do a deep-dive into some of the key innovations behind Amazon Aurora, discuss best practices and configurations, and share early customer experience from the field.
Similar to OSS Presentation OpenStack Swift by Joe Arnold (20)
AWS Cloud Kata 2014 | Jakarta - 2-1 AWS Intro and Scale 2014
AWS Cloud Kata 2014 | Jakarta - 2-1 AWS Intro and Scale 2014
(SOV204) Scaling Up to Your First 10 Million Users | AWS re:Invent 2014
(SOV204) Scaling Up to Your First 10 Million Users | AWS re:Invent 2014
Getting Started with Docker on AWS - DevDay Austin 2017
Getting Started with Docker on AWS - DevDay Austin 2017
Scaling on AWS for the First 10 Million Users (ARC206) | AWS re:Invent 2013
Scaling on AWS for the First 10 Million Users (ARC206) | AWS re:Invent 2013
Getting Started with Docker on AWS - DevDay Los Angeles 2017
Getting Started with Docker on AWS - DevDay Los Angeles 2017
Deep Dive: Scaling Up to Your First 10 Million Users
Deep Dive: Scaling Up to Your First 10 Million Users
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Best Practices for Backup and Recovery: Windows Workload on AWS
Best Practices for Backup and Recovery: Windows Workload on AWS
More from OpenStorageSummit
OSS Presentation DRMC by Keith Brennan
This document summarizes Keith Brennan's implementation of a VMware View virtual desktop infrastructure (VDI) solution at Delano Regional Medical Center using Nexenta storage. Key points include:
1) The initial VDI deployment using an IBM N3600 storage array struggled with performance issues due to simultaneous updates and reboots overwhelming the storage.
2) Brennan migrated the VDI workload to a Nexenta storage solution using SSDs, achieving a 20x improvement in IOPS and reducing latency from 18ms to 2ms.
3) The Nexenta implementation provided reliable performance for 175 active VMs using caching, compression, and avoiding deduplication due to use of a golden image.
4)
OSS Presentation Accelerating VDI by Daniel Beveridge
The document discusses storage design options for virtual desktop infrastructure (VDI) deployments using a virtual storage appliance (VSA) based on ZFS. It describes how VSA designs can provide high IOPS per user, burst isolation, low latency, and scalability. It compares stateless and persistent VDI models and outlines storage configuration hints for implementing stateless and persistent VDI with local disks or in a hybrid configuration using a VSA with local SSDs and SAN/NAS storage. While intricate ZFS configurations are involved, VSA designs can deliver cost-effective high performance for VDI workloads.
OSS Presentation Vesk
This document discusses launching a successful VDI hosted service using OpenStorage and Citrix XenDesktop. It provides an overview of the VDI market, different types of VDI deployments, and a case study of a VESK implementation for Brixton Council with 550 users across 185 applications. The implementation involved upgrading storage to Nexenta with RAIDz2 JBODs, SSDs, and 72 2TB drives totaling over 140TB of raw storage. Some problems were encountered during the Nexenta 3.1 upgrade including issues with the zpool and disk groups that required troubleshooting.
OSS Presentation NexentaStor™
VAAI, VASA, and SVA are technologies that improve storage efficiency in VMware environments. VAAI offloads storage operations to storage arrays to reduce host CPU usage. VASA allows storage characteristics to be displayed in vSphere. SVA allows NexentaStor to run as a virtual appliance, turning direct-attached storage into shared storage. These technologies provide benefits such as more efficient provisioning and management of storage resources.
OSS Presentation VMWorld 2011 by Andy Bennett & Craig Morgan
This document summarizes Nexenta Systems' involvement in providing storage for the Hands-on-Labs (HOL) at VMworld 2011 in Las Vegas and Copenhagen. It describes the architecture developed using Nexenta storage and VMware technologies on a commodity "SuperRack". Key statistics are provided on the number of VMs created, IOPS performance achieved, and how Nexenta storage handled the workload with high availability. Custom analytics dashboards were also developed to visualize storage metrics for attendees.
OSS Presentation Keynote by Per Sedihn
The document discusses the need for IT organizations to adapt to business demands for faster delivery of services and information. It summarizes the background and services of Proact IT Group, a large European IT services company. The rest of the document outlines challenges facing IT, including the rise of public cloud usage outside of IT policies, and explores how technologies like virtualization, storage tiering, and data classification can help IT deliver services in a more automated, flexible and cost effective manner aligned with business needs.
OSS Presentation Keynote by Jason Hoffman
The document discusses emerging trends in technology including everything becoming connected through web applications and real-time data synchronization between devices replacing the traditional client-server model. It also summarizes that the goal of technology should be to just work, scale, and be transparent. The rest of the document discusses how data, compute resources, and networking can be viewed as economic goods and services that are delivered over time, with datacenters serving as hubs for this transportation of digital information and computing power.
OSS Presentation Keynote by Hal Stern
The document discusses how networks and applications can become more aware of each other to improve the experience for end users. Currently, networks and applications operate independently without much visibility into each other. The document proposes that applications share information about end users and traffic with networks, and networks share information about topology, bandwidth, and resources with applications. This would allow applications to optimize content placement and resource usage, and networks to gain insights to better optimize traffic and provide new services. The document argues this type of programmable network can improve areas like security, performance, analytics and more.
OSS Presentation Keynote by Evan Powell
The document discusses abstraction, commoditization, and two bridges. It notes that abstraction and virtualization are driving changes in data storage technology. Commoditization of storage is occurring through the use of open standards and x86 hardware. Two bridges are used as a metaphor - one expensive bridge built in Silicon Valley, and one much less expensive community-built bridge in Guatemala.
OSS Presentation by Kevin Halgren
The document summarizes Washburn University's efforts to consolidate its enterprise storage using an open source storage solution called Nexenta. It describes the university's existing fragmented storage environment consisting of various vendors and protocols. The implementation plan involves acquiring an initial Nexenta cluster and storage expansions, building an iSCSI fabric, migrating CIFS/NFS storage, integrating VMware, and connecting a fiber channel server. The end goal is to manage all storage and backups through a single Nexenta interface using the data integrity and flexibility benefits of the ZFS file system.
OSS Presentation by Bryan Badger
This document discusses Integral Networks' transition from legacy SAN storage solutions like NetApp and Dell EqualLogic to NexentaStor software-defined storage. It provides an overview of Integral Networks, their infrastructure workload, and the pain points they experienced with their previous storage platforms. It then outlines how the Nexenta architecture and NexentaStor software address their scalability, availability, and cost needs without compromising on features.
OSS Presentation by Stefano Maffulli
OpenStack is an open source cloud operating system that is used by enterprises and service providers to build both private and public clouds. It began development in 2010 with a mission to produce an ubiquitous open source cloud platform. OpenStack provides common services for compute, storage, networking and identity management and is made up of several modular components like Nova (compute), Swift (object storage), Glance (images), and Keystone (identity). The OpenStack community includes over 1,200 active participants from 17 countries working together to rapidly evolve the software through six major releases. Companies can participate by using OpenStack to power their clouds or by contributing code to the project.
OSS Presentation Arista
AWS has experienced exponential growth since its inception, growing from $0 to $500M in revenue in just 4 years. It releases new features doubling every year and has over 600,000 virtual machines running on just 80,000 physical servers across only 2 hardware vendors and 10 configurations. KT, a telecom provider, successfully converted to being a cloud service provider by building private, mobile, and public clouds based on best practices from real cloud competitors rather than traditional enterprise IT approaches. This allowed them to exceed expectations by outperforming AWS on pricing and provisioning times while reducing their capex spend by 79%.
OSS Presentation Metro Cluster by Andy Bennett & Roel De Frene
The document describes a stretched metro cluster configuration using NexentaStor software and Coraid SRX storage arrays. Key features include high availability across geographically separate sites, non-disruptive failover for block and file storage, and synchronous replication providing a disaster recovery solution. The configuration was implemented for a Belgian managed services provider, delivering over 10,000 IOPS and 200MB/s of throughput between sites 15km apart with failover times under a minute. A video demo and performance benchmarks are presented.
OSS Presentation DDR Drive ZIL Accelerator by Christopher George
The document discusses the ZFS Intent Log (ZIL) and ZIL Accelerator. It explains that the ZIL logs all file system transactions and the ZIL Accelerator is an optional device that can accelerate performance by handling the synchronous writes. It emphasizes that any device used as a ZIL Accelerator must have volatile cache power protection to ensure data integrity in the event of a power loss. It also lists several Intel SSD models and whether they do or do not have this necessary volatile cache protection.
More from OpenStorageSummit (15)
OSS Presentation Accelerating VDI by Daniel Beveridge
OSS Presentation Accelerating VDI by Daniel Beveridge
OSS Presentation VMWorld 2011 by Andy Bennett & Craig Morgan
OSS Presentation VMWorld 2011 by Andy Bennett & Craig Morgan
OSS Presentation Metro Cluster by Andy Bennett & Roel De Frene
OSS Presentation Metro Cluster by Andy Bennett & Roel De Frene
OSS Presentation DDR Drive ZIL Accelerator by Christopher George
OSS Presentation DDR Drive ZIL Accelerator by Christopher George
Recently uploaded
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentationsGraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Building RAG with self-deployed Milvus vector database and Snowpark Container...
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Join us to introduce Milvus Lite, a vector database that can run on notebooks and laptops, share the same API with Milvus, and integrate with every popular GenAI framework. This webinar is perfect for developers seeking easy-to-use, well-integrated vector databases for their GenAI apps.
Recently uploaded (20)
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
OSS Presentation OpenStack Swift by Joe Arnold
- 1. Joe Arnold [email_address] http://swiftstack.com OpenStack Swift
- 2. Uses of Swift Overview of Architecture Example Deployments
- 3. Uses of Swift Overview of Architecture Example Deployments
- 5. Stores Millions to Billions of Objects All objects have a URL Data Storage REST API Get / Put / Delete Similar to Amazon S3
- 6. Long Tail Data $/GB frequency $/IO
- 7. Swift Running VMs Database Volumes Asset Storage “Ephemeral” DAS EBS S3
- 10. Uses of Swift Overview of Architecture Example Deployments
- 12. Proxy Tier handles incoming requests Scales Horizontally
- 13. The Ring maps data to disk
- 14. Storage Zones isolate physical failures
- 15. Quorum Writes
- 17. Replication process runs continuously
- 18. Uses of Swift Overview of Architecture Example Deployments
- 19. Zone Zone Zone Zone Proxy Proxy
- 20. Zone Zone Zone Zone Zone Agg ToR ToR ToR ToR ToR Proxy
- 21. Joe Arnold [email_address] http://swiftstack.com Thank You!
Editor's Notes
- isolate physical failures