Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss the performance implications of our new larger and faster SSD volumes (up to 16 TB with increased max throughput levels), as well as Amazon EBS encryption. Throughout, we share tips for success.
Learn tips and techniques that will improve the performance of your applications and databases running on Amazon EC2 instance storage and/or Amazon Elastic Block Store (EBS). This advanced session discusses when to use HI1, HS1, and Amazon EBS. We will share an "under the hood" view to tune the performance of your Elastic Block Store and best practices for running workloads on Amazon EBS, such as relational databases (MySQL, Oracle, SQL Server, postgres) and NoSQL data stores, such as MongoDB and Riak.
Deep Dive on Amazon EBS Elastic Volumes - March 2017 AWS Online Tech TalksAmazon Web Services
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage for use with Amazon EC2 instances. In this technical session, we will present and demonstrate how you can increase capacity, tune performance, and modify volume types on the fly with the latest Amazon EBS innovation, Elastic Volumes. You will learn how Elastic Volumes can significantly reduce both operational complexity and downtime enabling you to right-size your deployment and dynamically adapt as your business needs change. We will describe best practices and share tips for success throughout.
Learning Objectives:
- Learn how to increase capacity, tune performance, and modify volume types
- Learn how you can automate modifications to align with changing business needs.
- Review the different Amazon EBS volume types and receive best practices for each.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
Amazon EBS provides persistent block-level storage volumes for use with Amazon EC2 instances. In this technical session, you will discover how Amazon EBS can take your application deployments on EC2 to the next level. Session attendees will learn about the Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, and details about its performance and volume types. We discuss how to maximize Amazon EBS performance, with a special emphasis on low-latency, high-throughput applications like transactional and NoSQL databases, and big data analysis frameworks like Hadoop and Kafka. We will also dive deep and discuss Elastic Volumes, our latest EBS feature that allows you to dynamically increase capacity, tune performance, and change the type of EBS volumes on the fly. Throughout, we share tips for success.
Learn tips and techniques that will improve the performance of your applications and databases running on Amazon EC2 instance storage and/or Amazon Elastic Block Store (EBS). This advanced session discusses when to use HI1, HS1, and Amazon EBS. We will share an "under the hood" view to tune the performance of your Elastic Block Store and best practices for running workloads on Amazon EBS, such as relational databases (MySQL, Oracle, SQL Server, Postgres) and NoSQL data stores, such as MongoDB and Riak.
Review of how AWS EC2 storage options have evolved, and making the right selection for your workload. Covering Amazon Elastic Block Storage, EBS and Amazon Elastic File System, EFS.
Elastic Block Store is one of the fundamental components of a best-practices cloud architecture. Join Senior Solutions Architect Miles Ward for a detailed review of the service, a clear breakdown on cost and approaches for price estimates, easy methods for extracting maximum performance from both Standard and Provisioned-IOPS EBS volumes, configuration nuances for both Windows and Linux users, RAID guidance, RDBMS and NoSQL storage configuration tuning, as well as several examples of EBS delivering amazing value at scale.
Learn tips and techniques that will improve the performance of your applications and databases running on Amazon EC2 instance storage and/or Amazon Elastic Block Store (EBS). This advanced session discusses when to use HI1, HS1, and Amazon EBS. We will share an "under the hood" view to tune the performance of your Elastic Block Store and best practices for running workloads on Amazon EBS, such as relational databases (MySQL, Oracle, SQL Server, postgres) and NoSQL data stores, such as MongoDB and Riak.
Deep Dive on Amazon EBS Elastic Volumes - March 2017 AWS Online Tech TalksAmazon Web Services
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage for use with Amazon EC2 instances. In this technical session, we will present and demonstrate how you can increase capacity, tune performance, and modify volume types on the fly with the latest Amazon EBS innovation, Elastic Volumes. You will learn how Elastic Volumes can significantly reduce both operational complexity and downtime enabling you to right-size your deployment and dynamically adapt as your business needs change. We will describe best practices and share tips for success throughout.
Learning Objectives:
- Learn how to increase capacity, tune performance, and modify volume types
- Learn how you can automate modifications to align with changing business needs.
- Review the different Amazon EBS volume types and receive best practices for each.
Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas
Après la petite intro sur le stockage distribué et la description de Ceph, Jian Zhang réalise dans cette présentation quelques benchmarks intéressants : tests séquentiels, tests random et surtout comparaison des résultats avant et après optimisations. Les paramètres de configuration touchés et optimisations (Large page numbers, Omap data sur un disque séparé, ...) apportent au minimum 2x de perf en plus.
Amazon EBS provides persistent block-level storage volumes for use with Amazon EC2 instances. In this technical session, you will discover how Amazon EBS can take your application deployments on EC2 to the next level. Session attendees will learn about the Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, and details about its performance and volume types. We discuss how to maximize Amazon EBS performance, with a special emphasis on low-latency, high-throughput applications like transactional and NoSQL databases, and big data analysis frameworks like Hadoop and Kafka. We will also dive deep and discuss Elastic Volumes, our latest EBS feature that allows you to dynamically increase capacity, tune performance, and change the type of EBS volumes on the fly. Throughout, we share tips for success.
Learn tips and techniques that will improve the performance of your applications and databases running on Amazon EC2 instance storage and/or Amazon Elastic Block Store (EBS). This advanced session discusses when to use HI1, HS1, and Amazon EBS. We will share an "under the hood" view to tune the performance of your Elastic Block Store and best practices for running workloads on Amazon EBS, such as relational databases (MySQL, Oracle, SQL Server, Postgres) and NoSQL data stores, such as MongoDB and Riak.
Review of how AWS EC2 storage options have evolved, and making the right selection for your workload. Covering Amazon Elastic Block Storage, EBS and Amazon Elastic File System, EFS.
Elastic Block Store is one of the fundamental components of a best-practices cloud architecture. Join Senior Solutions Architect Miles Ward for a detailed review of the service, a clear breakdown on cost and approaches for price estimates, easy methods for extracting maximum performance from both Standard and Provisioned-IOPS EBS volumes, configuration nuances for both Windows and Linux users, RAID guidance, RDBMS and NoSQL storage configuration tuning, as well as several examples of EBS delivering amazing value at scale.
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
An overview of the Amazon ElastiCache managed service, with examples of how it can be used to increase performance, lower costs and augment other database services and databases to make things faster, easier and less expensive.
In this popular session, discover how Amazon EBS can take your application deployments on Amazon EC2 to the next level. Learn about Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, and details about its performance and volume types. The target audience is storage administrators, application developers, applications owners, and anyone who wants to understand how to optimize performance for Amazon EC2 using the power of Amazon EBS.
발표 영상 다시보기: https://kr-resources.awscloud.com/data-databases-and-analytics/%EC%A7%80%EA%B8%88-%EB%8B%B9%EC%9E%A5-dynamo-db-%ED%99%9C%EC%9A%A9%ED%95%98%EA%B8%B0-%EA%B0%95%EB%AF%BC%EC%84%9D-aws-database-modernization-day-%EC%98%A8%EB%9D%BC%EC%9D%B8-2
DynamoDB는 대량의 트래픽에 대해 빠른 응답시간을 보장하는 AWS의 NOSQL Database 서비스 입니다. 본 세션에서는 DynamoDB를 생성하고 테이블 디자인 후 데이터 입력, 삭제, 업데이트 및 성능에 관련된 설정에 대해서 진행합니다. 이 세션후 참석자들은 DyanmoDB에 대한 이해하며 직접 구성 및 사용할 수 있습니다.
최근 국내와 글로벌 서비스에서 MongoDB를 사용하는 사례가 급증하고 있습니다. 다만 전통적인 RDBMS에 비해, 아직 지식과 경험의 축적이 적게 되어 있어 손쉬운 접근과 트러블 슈팅등에 문제가 있는 것도 사실입니다. 이 세션에서는 MongoDB 와 AWS의 DocumentDB의 Architecure를 간단히 살펴보고 MongoDB 및 DocumentDB의 비교를 진행하며 특히 MongoDB와 DocumentDB를 사용할때 주의해야할 중요 포인트에 대해서 알아봅니다.
An overview of the Amazon ElastiCache managed service, with examples of how it can be used to increase performance, lower costs and augment other database services and databases to make things faster, easier and less expensive.
In this session we will explore the world’s first cloud-scale file system and its targeted use cases. Session attendees will learn about EFS’s benefits, how to identify applications that are appropriate for use with EFS, and details about its performance and security models. The target audience is file system administrators, application developers, and application owners that operate or build file-based applications.
In this session we will review Amazon EFS and how it delivers fully managed, petabyte-scale file storage for Amazon EC2 instances. Large scale and consistent performance make Amazon EFS ideal for web and content serving, enterprise applications, media processing, container storage, and Big Data analytics use cases. Session attendees will learn how to identify appropriate applications for use with Amazon EFS, understand performance details and security models, and hear how established customers are using it in production. The target audience is file system administrators, application developers, and application owners that operate or build file-based applications that require consistent latencies at cloud scale.
In this presentation, you will get a look under the covers of Amazon Redshift, a fast, fully-managed, petabyte-scale data warehouse service for less than $1,000 per TB per year. Learn how Amazon Redshift uses columnar technology, optimized hardware, and massively parallel processing to deliver fast query performance on data sets ranging in size from hundreds of gigabytes to a petabyte or more. We'll also walk through techniques for optimizing performance and, you’ll hear from a specific customer and their use case to take advantage of fast performance on enormous datasets leveraging economies of scale on the AWS platform.
In this popular session, discover how Amazon EBS can take your application deployments on Amazon EC2 to the next level. Learn about Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, snapshots, and details about its performance and volume types. The target audience is storage administrators, application developers, applications owners, and anyone who wants to understand how to optimize performance for Amazon EC2 using the power of Amazon EBS.
by Martin Schade, Enterprise Solutions Architect, & Gareth Eagar, Solutions Architect, AWS
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from one to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
Optimizing Delta/Parquet Data Lakes for Apache SparkDatabricks
This talk outlines data lake design patterns that can yield massive performance gains for all downstream consumers. We will talk about how to optimize Parquet data lakes and the awesome additional features provided by Databricks Delta. * Optimal file sizes in a data lake * File compaction to fix the small file problem * Why Spark hates globbing S3 files * Partitioning data lakes with partitionBy * Parquet predicate pushdown filtering * Limitations of Parquet data lakes (files aren't mutable!) * Mutating Delta lakes * Data skipping with Delta ZORDER indexes
Speaker: Matthew Powers
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
Speakers:
Tom Maddox, AWS Solutions Architect
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
An overview of the Amazon ElastiCache managed service, with examples of how it can be used to increase performance, lower costs and augment other database services and databases to make things faster, easier and less expensive.
In this popular session, discover how Amazon EBS can take your application deployments on Amazon EC2 to the next level. Learn about Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, and details about its performance and volume types. The target audience is storage administrators, application developers, applications owners, and anyone who wants to understand how to optimize performance for Amazon EC2 using the power of Amazon EBS.
발표 영상 다시보기: https://kr-resources.awscloud.com/data-databases-and-analytics/%EC%A7%80%EA%B8%88-%EB%8B%B9%EC%9E%A5-dynamo-db-%ED%99%9C%EC%9A%A9%ED%95%98%EA%B8%B0-%EA%B0%95%EB%AF%BC%EC%84%9D-aws-database-modernization-day-%EC%98%A8%EB%9D%BC%EC%9D%B8-2
DynamoDB는 대량의 트래픽에 대해 빠른 응답시간을 보장하는 AWS의 NOSQL Database 서비스 입니다. 본 세션에서는 DynamoDB를 생성하고 테이블 디자인 후 데이터 입력, 삭제, 업데이트 및 성능에 관련된 설정에 대해서 진행합니다. 이 세션후 참석자들은 DyanmoDB에 대한 이해하며 직접 구성 및 사용할 수 있습니다.
최근 국내와 글로벌 서비스에서 MongoDB를 사용하는 사례가 급증하고 있습니다. 다만 전통적인 RDBMS에 비해, 아직 지식과 경험의 축적이 적게 되어 있어 손쉬운 접근과 트러블 슈팅등에 문제가 있는 것도 사실입니다. 이 세션에서는 MongoDB 와 AWS의 DocumentDB의 Architecure를 간단히 살펴보고 MongoDB 및 DocumentDB의 비교를 진행하며 특히 MongoDB와 DocumentDB를 사용할때 주의해야할 중요 포인트에 대해서 알아봅니다.
An overview of the Amazon ElastiCache managed service, with examples of how it can be used to increase performance, lower costs and augment other database services and databases to make things faster, easier and less expensive.
In this session we will explore the world’s first cloud-scale file system and its targeted use cases. Session attendees will learn about EFS’s benefits, how to identify applications that are appropriate for use with EFS, and details about its performance and security models. The target audience is file system administrators, application developers, and application owners that operate or build file-based applications.
In this session we will review Amazon EFS and how it delivers fully managed, petabyte-scale file storage for Amazon EC2 instances. Large scale and consistent performance make Amazon EFS ideal for web and content serving, enterprise applications, media processing, container storage, and Big Data analytics use cases. Session attendees will learn how to identify appropriate applications for use with Amazon EFS, understand performance details and security models, and hear how established customers are using it in production. The target audience is file system administrators, application developers, and application owners that operate or build file-based applications that require consistent latencies at cloud scale.
In this presentation, you will get a look under the covers of Amazon Redshift, a fast, fully-managed, petabyte-scale data warehouse service for less than $1,000 per TB per year. Learn how Amazon Redshift uses columnar technology, optimized hardware, and massively parallel processing to deliver fast query performance on data sets ranging in size from hundreds of gigabytes to a petabyte or more. We'll also walk through techniques for optimizing performance and, you’ll hear from a specific customer and their use case to take advantage of fast performance on enormous datasets leveraging economies of scale on the AWS platform.
In this popular session, discover how Amazon EBS can take your application deployments on Amazon EC2 to the next level. Learn about Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, snapshots, and details about its performance and volume types. The target audience is storage administrators, application developers, applications owners, and anyone who wants to understand how to optimize performance for Amazon EC2 using the power of Amazon EBS.
by Martin Schade, Enterprise Solutions Architect, & Gareth Eagar, Solutions Architect, AWS
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from one to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
Optimizing Delta/Parquet Data Lakes for Apache SparkDatabricks
This talk outlines data lake design patterns that can yield massive performance gains for all downstream consumers. We will talk about how to optimize Parquet data lakes and the awesome additional features provided by Databricks Delta. * Optimal file sizes in a data lake * File compaction to fix the small file problem * Why Spark hates globbing S3 files * Partitioning data lakes with partitionBy * Parquet predicate pushdown filtering * Limitations of Parquet data lakes (files aren't mutable!) * Mutating Delta lakes * Data skipping with Delta ZORDER indexes
Speaker: Matthew Powers
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
Speakers:
Tom Maddox, AWS Solutions Architect
Learn tips and techniques that will improve the performance of your applications and databases running on Amazon EC2 instance storage and/or Amazon Elastic Block Store (EBS). This advanced session discusses when to use HI1, HS1, and Amazon EBS. We will share an "under the hood" view to tune the performance of your Elastic Block Store and best practices for running workloads on Amazon EBS, such as relational databases (MySQL, Oracle, SQL Server, postgres) and NoSQL data stores, such as MongoDB and Riak.
Overview and Best Practices for Amazon Elastic Block Store - September 2016 W...Amazon Web Services
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we present the differences between the types of Amazon EBS block storage so that you can best understand which storage type to use for your different application deployments. We discuss how to maximize Amazon EBS performance with a special eye towards low-latency and high-throughput applications. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
Learning Objectives:
• Learn about the latest updates to EBS
• Learn about best practices for using EBS.
Who Should Attend:
• Application admins, DBAs, database and big data architects
AWS - an introduction to bursting (GP2 - T2)Rasmus Ekman
An introduction to bursting on AWS. The presentation includes an introductory explanation of the new EC2 family, T2, and the new EBS volume type GP2.
Learn about the new services on AWS to get the most bang for your buck.
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
Amazon EBS provides persistent block-level storage volumes for use with Amazon EC2 instances. In this technical session, you will discover how Amazon EBS can take your application deployments on EC2 to the next level. Session attendees will learn about the Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, and details about its performance and volume types. We discuss how to maximize Amazon EBS performance, with a special emphasis on low-latency, high-throughput applications like transactional and NoSQL databases, and big data analysis frameworks like Hadoop and Kafka. We will also dive deep and discuss Elastic Volumes, our latest EBS feature that allows you to dynamically increase capacity, tune performance, and change the type of EBS volumes on the fly. Throughout, we share tips for success.
Maximizing EC2 and Elastic Block Store Disk Performance (STG302) | AWS re:Inv...Amazon Web Services
Learn tips and techniques that will improve the performance of your applications and databases running on EC2 instance storage and/or Elastic Block Store. This advanced session discusses when to use HI1, HS1, and EBS. It shares an under the hood view on how to tune the performance of Elastic Block Store. The presenter(s) will share best practices on running workloads on EBS such as relational databases (MySQL, Oracle, SQL Server, postgres) and NoSQL data stores such as MongoDB and Riak.
AWS re:Invent 2016: Deep Dive on Amazon Elastic Block Store (STG301)Amazon Web Services
In this popular session, you will learn about the latest features and use cases for Amazon EBS, including best practices, an overview of newly introduced features, and brand-new re:Invent announcements. In particular we will cover the expanded portoflio of volume types, including provisioned IOPS, cold storage, and throughput-optimized. This session will help database admins and application architects understand how to blend performance and cost with applicaitns for big data analytics, data warehousing, and transactional and NoSQL databases.
Amazon EBS provides persistent block-level storage volumes for use with Amazon EC2 instances. In this technical session, you will discover how Amazon EBS can take your application deployments on EC2 to the next level. Session attendees will learn about the Amazon EBS features and benefits, how to identify applications that are appropriate for use with Amazon EBS, best practices, and details about its performance and volume types. We discuss how to maximize Amazon EBS performance, with a special emphasis on low-latency, high-throughput applications like transactional and NoSQL databases, and big data analysis frameworks like Hadoop and Kafka. We will also dive deep and discuss Elastic Volumes, our latest EBS feature that allows you to dynamically increase capacity, tune performance, and change the type of EBS volumes on the fly. Throughout, we share tips for success.
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we discuss how to maximize Amazon EBS performance, with a special eye toward low-latency, high-throughput applications like databases. We explain how to monitor your application and share real-world examples.
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...Amazon Web Services
As the cloud continues to grow, organizations need IT talent with cloud skills. AWS Certifications validate cloud knowledge with an industry-recognized credential that can help advance your career.
Join this webinar to learn more about why AWS Certifications matter and to hear tips from an AWS expert about how to prepare for certification exams. During this webinar, you’ll hear about the AWS training, self-paced labs, and online resources that can help you on your path toward preparing for any one of our Associate exams including: Solutions Architect, Developer, and SysOps Administrator. We’ll also walk you through sample questions and study tips so you can learn how to think through typical associate-level exam questions. Finally, you’ll have the chance to have your questions answered live by an AWS expert.
Learning Objectives:
• Hear about a recommended preparation path for the career-enhancing AWS associate certification exams
• Learn more about how AWS Training can help you prepare to take the exam
• Hear study tips, work through a practice question, and have your questions answered live
AWS Webcast - Cost and Performance Optimization in Amazon RDSAmazon Web Services
Amazon RDS makes it easy to set up, operate, and scale relational databases in the cloud. The service offers a variety of options for optimizing the performance level delivered, as well as optimizing your spending. In this webinar, we will show a variety of techniques for implementing the right performance level for your application.
Learning Objectives:
• Understand the Amazon RDS options that change database performance and cost
• Select the appropriate performance and cost level for your specific application Who Should Attend:
• Technical Amazon RDS customers and prospective customers
Understanding Elastic Block Store Availability and PerformanceAmazon Web Services
Depending on your application needs, Elastic Block Store’s volumes can be configured for optimal performance and higher availability. In this session, we will present the different design characteristics of EBS Standard and Provisioned IOPS volumes, provide technical insights on how to think about EBS performance and availability, and share best practices to achieve higher availability and performance.
AWS Summit London 2014 | Maximising EC2 and EBC Performance (400)Amazon Web Services
This advanced technical session is ideal for customers that are looking to maximise the performance of AWS Elastic Block Store (EBS) storage to support workloads with demanding IO performance requirements. If you need to run high IO workloads on EBS such as NoSQL or RBDMS systems then attend this session to find out how to optimise your EBS configuration to enable this.
Consistent High IO Performance with Amazon Elastic Block StoreAmazon Web Services
Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high IO for their service
Amazon Elastic Block Store (Amazon EBS) provides flexible, persistent storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of all types of Amazon EBS block storage including General Purpose SSD (gp2) and Provisioned IOPS SSD (io1). Along the way, we will share Amazon EBS best practices for optimizing performance, managing snapshots and securing data.
Similar to Deep Dive: Maximizing EC2 and EBS Performance (20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS.
In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup.
Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti.
Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.
Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi
Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
L’utilizzo dei container è in continua crescita.
Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili.
I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!
In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th.
Event Agenda :
Open banking so far (short recap)
• PSD2, OB UK, OB Australia, OB LATAM, OB Israel
Intro to Open Finance marketplace
• Scope
• Features
• Tech overview and Demo
The role of the Cloud
The Future of APIs
• Complying with regulation
• Monetizing data / APIs
• Business models
• Time to market
One platform for all: a Strategic approach
Q&A
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc.
AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta.
Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity.
AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet.
Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.
Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.
Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti.
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti.
Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire.
Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito.
In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.
Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline.
Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.
Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
7. What is EBS?
• Network block storage
• Designed for five nines of availability
• Attaches to Amazon EC2 within the same
Availability Zone
• Provides point-in-time snapshots to
Amazon S3
8. More about EBS
• It’s a service!
• It’s independent of EC2
• It has regional and AZ availability goals
– All EBS volumes are designed for 99.999% availability
• Over 1.5 million volumes are created per day
9. A few definitions…
• IOPS: Input/output operations per second (#)
• Throughput: Read/write rate to storage (MB/s)
• Latency: Delay between request and completion (ms)
• Capacity: Volume of data that can be stored (GB)
• Block size: Size of each I/O (KB)
10. EBS volume types
• General Purpose (SSD)
• Provisioned IOPS (SSD)
• Magnetic
When performance matters, use SSD-backed volumes
11. EBS SSD volumes
• Applies to both General Purpose and
Provisioned IOPS
• IOPS measured up to 256 KB
• Single-digit ms latency
• Designed for 99.999% availability
12. EBS General Purpose volumes (SSD)
New default volume type for EBS
Every volume can burst up to 3,000 IOPS
• Larger volumes can burst for longer periods
3 IOPS per GB baseline performance,
maximum of 10,000 IOPS
99% performance consistency
Up to 160 MB/s throughput
14. (2) Max I/O credit per bucket is 5.4M
(1) Always accumulating 3
IOPS per GB per second
(3) You can spend up to
3000 IOPS per second
Understanding General Purpose (SSD) bursting
Baseline performance = 3 IOPS per GB
15. Minutes to empty a full I/O credit bucket for various volume sizes
The larger the volume, the longer it takes to empty the I/O credit bucket
1 TB or larger volume will never exhaust its I/O credit bucket
16. Minutes to empty a full IO credit bucket for various sizes
The larger the volume, the longer it takes to empty the I/O credit bucket
1 TB or larger volume will never exhaust its I/O credit bucket
17. General Purpose (SSD) volumes example
Microsoft Windows 30 GB boot volume:
• Gets initial I/O credit of 5.4M
• Could burst for up to 30 mins @ 3000 IOPS
• Always accumulating 90 I/O credits per
second
18. Improved instance boot time
m3.medium
Volume type Boot time Access time OS
GP2 3:31 4:33 Windows Server
2012
Magnetic 4:30 7:16 Windows Server
2012
GP2 0:36 0:45 CentOS6
Magnetic 0:57 1:16 CentOS6
40% Reduction in boot times by using General Purpose SSD
19. 1 TB PIOPS volume with 4K IOPS = $526.40 per month per volume
GP2 1 TB volume with 3000 IOPS = $102.40
GP2 2 x 500 GB volume at 3K, Burst to 6K = $102.40
80% cost savings, 50% more peak I/O with
General Purpose SSD
Database volume
20. Guidelines for sizing General Purpose (SSD)
volumes
Generic boot, developer, test/dev, and web apps:
Provision GB required for your applications
Database apps:
1. Calculate the IOPS required in steady state
2. Perform this calculation: (steady state IOPS) ÷ 3 = GB to
provision
Note: I/O bursts will support:
• Database load or table scan operations
• Spike in I/O workload
20
21. EBS PIOPS (SSD) volumes
• Best for I/O intensive databases that require highest
consistency
• Throughput up to 320 MB/sec
• Provision up to 20,000 IOPS per volume
(supports IOPS:GB ratio of 30)
• Designed for 99.9% performance consistency
22. EBS Magnetic volumes
• Best for cold workloads (rarely accessed data that needs
always-on access)
• IOPS: ~100 IOPS steady-state, with best-effort bursts
• Throughput: variable by workload, best effort to 10s of MBs
• Latency: Varies, reads typically ~20-40 ms, writes typically
~2-10 ms
23. EBS volume types - summary
General Purpose (SSD) Provisioned IOPS (SSD) Magnetic
Recommend use cases
Boot volumes
Small to med DBs
Dev and test
I/O-intensive workloads
Large DBs
Cold storage
Storage media SSD-backed SSD-backed Magnetic-backed
Volume size 1 GB - 16 TB 4 GB - 16 TB 1 GB - 1 TB
Max IOPS per volume 10,000 IOPS 20,000 IOPS ~100 IOPS
Burst < 1 TB to 3000 IOPS baseline baseline
Read and write peak throughput 160 MB/s 320 MB/s ~50-90 MBps
Max IOPS per node (16k) 48,000 48,000 48,000
Peak throughput node 800 MB/s 800 MB/s 800 MB/s
Latency (random read) 1-2 ms 1-2 ms 20-40 ms
API Name gp2 io1 standard
Price* $.10/GB-month
$.125/GB-month
$.065/provisioned IOPS
$.05/GB-month
$.05/ 1M I/O
24. Why is General Purpose SSD the default?
High baseline level of performance
Burst to higher level of IOPS
Single, capacity-based pricing dimension
• Makes forecasting very easy
• Eliminates sizing complexity
Attractive price/gigabyte/price/IOPS density
26. Migrating to General Purpose (SSD) volumes
Change volume type during launch
Use EBS snapshots
You may be able to resize the file system
Use General Purpose (SSD) sizing guide
27. Benefits of using EBS snapshots
More durable than an EBS volume
• Stored in Amazon S3
Differential (space-efficient)
• First snapshot is a clone
• Pay only for what you use
Availability Zone-independent
• Clone into any AZ
Can be copied efficiently across regions
28. Tagging snapshots
Use tags to add
metadata to snapshots:
• Type (daily, weekly)
• Version
• Instance Id
• Volume Id
• Application stack
30. Queuing theory – Little’s law
Little’s law is the foundation for performance
tuning theory
• Mathematically proven by John Little in 1961
L = A * W
L = Queue length = average number of requests waiting
A = Arrival rate = the rate of requests arriving
W = Wait time = average wait time
EBS performance is related to this law
31. Performance optimization is measured by:
IOPS: Read/write I/O rate (IOPS)
Latency: Time between I/O submission
and completion (ms)
Throughput: Read/write transfer rate
(MB/s); throughput = IOPS X I/O size
32. Four key components of performance optimization
1. EC2 instance
2. I/O
4. EBS
3. Network
link
33. Tools available for performance tuning:
1. EC2 instance: Network bandwidth (Mbps)
2. EBS-optimized instance: EC2 instance option (On/Off)
3. Workload: Block size, read/write ratio, serialization
4. Queue depth: The number of outstanding I/Os
5. RAID: Stripe volumes to maximize performance
6. Pre-warming: Eliminate first-touch penalty
34. 1. EC2 instance
Compute-optimized – C3,C4
Memory-optimized – R3
General-Purpose – M3
EBS
EC2
Select the EC2 instance that has the right network,
RAM, and CPU resources for your applications
35. 2. EBS-optimized instance
Most instance families support the EBS-optimized flag
EBS-optimized instances now support up to 4 GB/s
• Drive 32,000 16K IOPS or 500 MB/s
EC2 *.8xlarge instances support 10 Gb/s network
Max IOPS per node supported is ~48,000 IOPS @ 16K I/O
37. 3. Workload
I/O size:
• 4 KB to 64 MB
I/O pattern:
• Sequential and random
I/O type:
• Read and write
I/O concurrency:
• Number of concurrent I/O
EBS SSD-backed volumes measure I/O size up to 256 KB
EBS SSD-backed volumes deliver same performance for read and write
38. EBS IOPS and throughput limits
20,000 IOPS
PIOPS volume
20,000 IOPS
320 MB/s
throughput
You can achieve 20,000 IOPS when
driving smaller I/O operations
You can achieve up to 320 MB/s
when driving larger I/O operations
39. EBS IOPS and throughput limits
8,000 IOPS
PIOPS volume
8,000 IOPS
320 MB/s
throughput
8,000 x 64 KB=512 MB/s
1,250 x 256 KB = 320 MB/s
8,000 X 8 KB = 64 MB/s
8,000 X 16 KB = 128 MB/s
16,000 x 8 KB = 128 MB/s
8,000 x 32 KB = 256 MB/s
40. Block (I/O) size determines whether your
application is IOPS bound or throughput bound
41. 4. Queue depth
An I/O operation
EBS
After it’s gone, it’s gone
EC2
Queue depth is the pending I/O for a volume
43. I/O latency
• Elapsed time between I/O submission and its completion
time
• Performance requirements may be driven by IOPS or
latency or both
• There is an interdependency among IOPS, queue depth,
and latency
55. Optimal queue depth to achieve lower latency and highest IOPS
is typically between 4-8; ~1 queue depth per 500 IOPS
EBS-optimized instances provide consistent latency experience
Use SSD volumes with latest-generation EC2 instances
56. 5. RAID
Increases performance, or capacity, or both
Over 320 MB/sec or 20K IOPS, striping needed
Don’t mix volume types
Typically RAID 0 or LVM stripe
Avoid RAID for redundancyEBS
EC2
57. Maximum performance per instance
How should you think about taking snapshots on a striped volume?
• Quiesce file systems and take snapshot
• Unmount file system and take snapshot
• Use OS-specific tools
12×400 GB PIOPS, pre-warmed, RAID 0 LVM, stripe size 128 KB, attached to CR1 instance
59. 6. Pre-warming
• Eliminates first-access penalty
• Typically 5%, extreme worst case of 50% performance reduction in IOPS and latency
when volumes are used without pre-warming:
– Performance is as provisioned when all the chunks are accessed
• Recommendations before benchmarking:
– For new volumes:
• Linux: DD write
• Windows: NTFS full format
– Takes roughly an hour to pre-warm 1 TB PIOPS/General Purpose (SSD) volumes
• Always check latest documentation
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-prewarm.html
60. Use large block size to speed up your pre-warming
Example: sudo dd if=/dev/xvdf
of=/dev/xvdf conv=notrunc bs=1M
61. Final tips
• Try to use Ext4 or XFS
• Alignment can matter; check tools use 4k
62. Workload/
software
Typical block
size
Random/
Seq?
Max EBS @ 500
MB/s instances
Max EBS @
1 GB/s instances
Max EBS @ 10 GB/s
instances
Oracle DB Configurable:2 KB
–16 KB
Default 8 KB
random ~7,800 IOPS ~15,600 IOPS ~96,000 IOPS
Microsoft SQL
Server
8 KB w/ 64 KB
extents
random ~7,800 IOPS ~15,600 IOPS ~80,000 IOPS
MySQL 16 KB random ~4,000 IOPS ~7,800 IOPS ~48,000 IOPS
PostgreSQL 8 KB random ~7,800 IOPS ~15,600 IOPS ~96,000 IOPS
MongoDB 4 KB serialized ~15,600 IOPS ~31,000 IOPS ~96,000 IOPS
Apache
Cassandra
4 KB random ~15,600 IOPS ~31,000 IOPS ~96,000 IOPS
GlusterFS 128 KB sequential ~500 IOPS ~1,000 IOPS ~6,000 IOPS
Cheat sheet sample: Storage workloads on AWS
66. Why encrypt data volumes?
Security:
Protects against someone who might gain unauthorized physical access to the volume
Can help with internal or external compliance efforts:
• Chief Information Security Officer wants encryption to protect sensitive corporate
data
• 3rd-party auditors want to see evidence that sensitive customer data is encrypted
Ease of use and operating cost reduction:
Unlike open-source or 3rd-party solutions, such as Trend Micro SecureCloud, SafeNet
ProtectV, etc., EBS encryption offers:
• “Checkbox” encryption at no extra cost
• Automated, secure key management
67. AWS KMS
A service that simplifies encrypting data and managing keys
Allows customers to create, use, and manage encryption keys from within
their own applications and supported AWS services (Amazon S3, EBS,
Amazon Redshift)
Key management functions include:
• Create, enable, disable, rotate, and define usage policy on master keys
• Generate a data key that can be exported from the service after it’s
encrypted by a master key
• Audit use of master keys in AWS CloudTrail
Available in 9 commercial regions
69. How AWS services integrate with AWS KMS
2-tiered key hierarchy using envelope
encryption
Unique data key encrypts customer data
AWS KMS master keys encrypt data keys
Benefits of envelope encryption:
• Limits risk of a compromised data key
• Better performance for encrypting large data
• Easier to manage a small number of master
keys than millions of data keys
Master key(s)
Data key 1
S3 object EBS
volume
Amazon
Redshift
cluster
Data key 2 Data key 3 Data key 4
Custom
application
KMS
70. Summary
Use encryption if
you need it
Take snapshotsSelect the right
instance for your
workload
Select the right
volume for your
workload