by Joyjeet Banerjee, Enterprise Solutions Architect, AWS
Amazon Aurora is a MySQL- and PostgreSQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. In this deep dive session, we’ll discuss best practices and explore new features in areas like high availability, security, performance management and database cloning. Level 300
What’s New in Amazon RDS for Open-Source and Commercial Databases: Amazon Web Services
by Kwesi Edwards, Business Development Manager, AWS
In the past year, Amazon RDS has continued to expand functionality, scalability, availability and ease of use for all supported database engines: PostgreSQL, MySQL, MariaDB, Oracle and Microsoft SQL Server. We’ll take a close look at RDS use cases and new capabilities, splitting the time between open-source and commercial database engines. Level 300
Cloud storage provides education with a reliable, scalable, and secure alternative to on-premises storage systems. AWS offers eight different object, file, and block storage options supporting applications, archiving and compliance options. This webinar will provide an overview to the services and key education use cases ranging from data center replacement to video storage and file sharing. Services covered include Amazon S3, Amazon EFS, Amazon EBS, Amazon Glacier, Amazon Storage Gateway, and the Snowball family.
Learn how Amazon RDS makes it easy to deploy and operate a highly available and scalable SQL Server database in the cloud with cost-efficient and resizable capacity.
by Darin Briskman, Technical Evangelist, AWS
Elasticsearch is the most popular open-source search and analytics engine - it's easy to use, but not always easy to configure an manage. Learn about Amazon's fully managed service that provides easier deployment, operation, and scale for Elasticsearch. Level: 200
Use AWS DMS to Securely Migrate Your Oracle Database to Amazon Aurora with Mi...Amazon Web Services
Changing database engines is often daunting to customers. However, the value of a highly scalable, cost-effective, and fully managed service, such as Amazon Aurora, can make the challenge worth it. In this hand-on lab, we demonstrate how to take advantage of the AWS Schema Conversion Tool (SCT) and AWS Database Migration Service (DMS) to facilitate and simplify migrating an Oracle database to the Amazon Aurora PostgreSQL-compatible Edition. We connect to an Oracle (source) and a PostgreSQL (target) instance, and convert the Oracle database schema and code objects to PostgreSQL using AWS SCT. Then, we migrate and replicate the data using AWS DMS. AWS credits are provided. Bring your laptop, and have an active AWS account.
Amazon Aurora is a MySQL and PostgreSQL compatible relational database built for the cloud, that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. AWS Database Migration Service helps you migrate databases to AWS quickly and securely. The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database. In this session, we explore features of Amazon Aurora and demonstrate database migration using the AWS Database Migration Service.
by Joyjeet Banerjee, Enterprise Solution Architect, AWS
Amazon RDS allows you to launch an optimally configured, secure and highly available database with just a few clicks. It provides cost-efficient and resizable capacity while managing time-consuming database administration tasks, freeing you to focus on your applications and business. We’ll discuss Amazon RDS fundamentals, learn about the seven available database engines, and examine customer success stories. Level 100
What’s New in Amazon RDS for Open-Source and Commercial Databases: Amazon Web Services
by Kwesi Edwards, Business Development Manager, AWS
In the past year, Amazon RDS has continued to expand functionality, scalability, availability and ease of use for all supported database engines: PostgreSQL, MySQL, MariaDB, Oracle and Microsoft SQL Server. We’ll take a close look at RDS use cases and new capabilities, splitting the time between open-source and commercial database engines. Level 300
Cloud storage provides education with a reliable, scalable, and secure alternative to on-premises storage systems. AWS offers eight different object, file, and block storage options supporting applications, archiving and compliance options. This webinar will provide an overview to the services and key education use cases ranging from data center replacement to video storage and file sharing. Services covered include Amazon S3, Amazon EFS, Amazon EBS, Amazon Glacier, Amazon Storage Gateway, and the Snowball family.
Learn how Amazon RDS makes it easy to deploy and operate a highly available and scalable SQL Server database in the cloud with cost-efficient and resizable capacity.
by Darin Briskman, Technical Evangelist, AWS
Elasticsearch is the most popular open-source search and analytics engine - it's easy to use, but not always easy to configure an manage. Learn about Amazon's fully managed service that provides easier deployment, operation, and scale for Elasticsearch. Level: 200
Use AWS DMS to Securely Migrate Your Oracle Database to Amazon Aurora with Mi...Amazon Web Services
Changing database engines is often daunting to customers. However, the value of a highly scalable, cost-effective, and fully managed service, such as Amazon Aurora, can make the challenge worth it. In this hand-on lab, we demonstrate how to take advantage of the AWS Schema Conversion Tool (SCT) and AWS Database Migration Service (DMS) to facilitate and simplify migrating an Oracle database to the Amazon Aurora PostgreSQL-compatible Edition. We connect to an Oracle (source) and a PostgreSQL (target) instance, and convert the Oracle database schema and code objects to PostgreSQL using AWS SCT. Then, we migrate and replicate the data using AWS DMS. AWS credits are provided. Bring your laptop, and have an active AWS account.
Amazon Aurora is a MySQL and PostgreSQL compatible relational database built for the cloud, that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. AWS Database Migration Service helps you migrate databases to AWS quickly and securely. The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database. In this session, we explore features of Amazon Aurora and demonstrate database migration using the AWS Database Migration Service.
by Joyjeet Banerjee, Enterprise Solution Architect, AWS
Amazon RDS allows you to launch an optimally configured, secure and highly available database with just a few clicks. It provides cost-efficient and resizable capacity while managing time-consuming database administration tasks, freeing you to focus on your applications and business. We’ll discuss Amazon RDS fundamentals, learn about the seven available database engines, and examine customer success stories. Level 100
What’s New in Amazon RDS for Open-Source and Commercial DatabasesAmazon Web Services
In the past year, Amazon RDS has continued to expand functionality, scalability, availability and ease of use for all supported database engines: PostgreSQL, MySQL, MariaDB, Oracle and Microsoft SQL Server. We’ll take a close look at RDS use cases and new capabilities, splitting the time between open-source and commercial database engines.
AWS ofrece una gran variedad de servicios de base de datos que se adaptan a los requisitos de su aplicación. Los servicios de bases de datos están totalmente administrados y se pueden implementar en cuestión de minutos con tan solo unos clics. Los servicios de AWS incluyen Amazon Relational Database Service (Amazon RDS), compatible con 6 motores de bases de datos comunes, Amazon Aurora, base de datos relacional compatible con MySQL con un desempeño 5 veces superior, Amazon DynamoDB, servicio de bases de datos NoSQL rápido y flexible, Amazon Redshift, almacén de datos a escala de petabytes, y Amazon Elasticache, servicio de caché en memoria compatible con Memcached y Redis. AWS también proporciona AWS Database Migration Service, un servicio que permite migrar las bases de datos a la nube de AWS de forma sencilla y rentable.
Traditional data warehouses become expensive and slow down as the volume of your data grows. Amazon Redshift is a fast, petabyte-scale data warehouse that makes it easy to analyze all of your data using existing business intelligence tools for 1/10th the traditional cost. This session will provide an introduction to Amazon Redshift and cover the essentials you need to deploy your data warehouse in the cloud so that you can achieve faster analytics and save costs.
Amazon Aurora is a cloud-optimized relational database that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. The recently announced PostgreSQL-compatibility, together with the original MySQL compatibility, are perfect for new application development and for migrations from overpriced, restrictive commercial databases. In this session, we’ll do a deep dive into the new architectural model and distributed systems techniques behind Amazon Aurora, discuss best practices and configurations, look at migration options and share customer experience from the field.
Amazon Aurora is a MySQL- and PostgreSQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. In this deep dive session, we’ll discuss best practices and explore new features in areas like high availability, security, performance management and database cloning.
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017Amazon Web Services
Attend this session for a technical deep dive about RDS Postgres and Aurora Postgres. Come hear from Mark Porter, the General Manager of Aurora PostgreSQL and RDS at AWS, as he covers service specific use cases and applications within the AWS worldwide public sector community. Learn More: https://aws.amazon.com/government-education/
Introducing Amazon EMR Release 5.0 - August 2016 Monthly Webinar SeriesAmazon Web Services
Amazon EMR is a managed Hadoop service that makes it easy for customers to use big data frameworks and applications like Hadoop, Spark, and Presto to analyze data stored in HDFS or on Amazon S3 , Amazon’s highly scalable object storage service. In this webinar, we will introduce the latest release of Amazon EMR. With Amazon EMR release 5.0, customers can now launch the latest versions of popular open source frameworks including Apache Spark 2.0, Hive 2.1, Presto 0.151, Tez 0.8.4, and Apache Hadoop 2.7.2. We will walk through a demo to show you how to deploy a Hadoop environment within minutes. We will cover common use cases and best practices to lower costs using Amazon S3 as your data store and Amazon EC2 Spot Instances, which allow you to bid on space Amazon computing capacity.
Learning Objectives:
• Describe the new features and updated frameworks in Amazon EMR 5.0
• Learn best practices and real-world applications for Amazon EMR
• Understand how to use EC2 Spot pricing to save costs
• Explain the advantages of decoupling storage and compute with Amazon S3 as storage layer for EMR workloads
It’s been an exciting year for Amazon Aurora, the database with MySQL-compatible and PostgreSQL-compatible database engines. Amazon Aurora combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. In this deep dive session, we’ll discuss best practices and explore new features, including high availability options, new integrations with AWS services, and the performance management with Amazon RDS Performance Insights.
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017Amazon Web Services
Amazon Relational Database Service (Amazon RDS) simplifies setup, operation, and management of databases in the cloud. In this session, we will explore Amazon RDS features and best practices that offer graceful migration, high performance, elastic scaling, and high availability for Oracle databases. You will also learn from the Chief Architect for Intuit’s Small Business Division how the QuickBooks Online team is using Amazon RDS for Oracle to scale the world's largest online accounting platform.
This is an introduction to Amazon Redshift and cover the essentials you need to deploy your data warehouse in the cloud so that you can achieve faster analytics and save costs.
Amazon RDS enables you to launch an optimally configured, secure, and highly available relational database with just a few clicks. It provides cost-efficient and resizable capacity while managing time consuming administration tasks, freeing you to focus on your applications and business. In this session, we take a closer look at how Amazon RDS works, and we review best practices to achieve performance, flexibility, and cost savings for your MySQL, PostgreSQL, MariaDB, Oracle, and SQL Server databases on Amazon RDS. We also discuss AWS Database Migration Service, a quick and secure means for migrating your existing relational database management system investments to Amazon RDS.
Best Practices for Running Oracle Databases on Amazon RDS (DAT317) - AWS re:I...Amazon Web Services
Amazon Relational Database Service (Amazon RDS) continues to be a popular choice for Oracle DBAs moving new and legacy workloads to the cloud. In this session, we discuss how Amazon RDS for Oracle helps DBAs focus their time where it matters most. We cover recent RDS Oracle features, and we go deep on key functionality that enables license optimization, performance, and high availability for Oracle databases. We also hear directly from an AWS customer about their journey to Amazon RDS and the best practices that helped make their move successful.
Amazon Relational Database Service – How is it different to what you do today ?Amazon Web Services
Whether you are new to Amazon Relational Database Service (RDS) or just need a refresher on the latest features - this session is for you. We will take you through RDS from the perspective of how you manage your databases on-premise today. That is, we will describe the full lifecycle of a database from provisioning to decommissioning while taking into consideration your performance, scalability, availability, security and maintenance requirements.
Learn the fundamentals of Amazon DynamoDB and see the DynamoDB console first-hand as we walk through a demo of building a serverless web application using this high-performance key-value and JSON document store.
Strategic Uses for Cost Efficient Long-Term Cloud StorageAmazon Web Services
Compared to storing long-term datasets on-premises, archiving in the cloud is a smart alternative whether you’re looking for an active archive solution, tape replacement, or to fulfill a compliance requirement. Learn how AWS customers are simplifying their archiving strategy and meeting compliance needs using Amazon Glacier. Hear how customers have evolved their backup and disaster recovery architectures and replaced tape solutions by turning to AWS for a more cost efficient, durable and agile solution. We will showcase Sony DADC's active archive deployment on Glacier and demo how some of our financial service customers have set up compliant archives to meet their regulatory objectives.
Organizations need to perform increasingly complex analysis on data — streaming analytics, ad-hoc querying, and predictive analytics — in order to get better customer insights and actionable business intelligence. Apache Spark has recently emerged as the framework of choice to address many of these challenges. In this session, we show you how to use Apache Spark on AWS to implement and scale common big data use cases such as real-time data processing, interactive data science, predictive analytics, and more. We will talk about common architectures, best practices to quickly create Spark clusters using Amazon EMR, and ways to integrate Spark with other big data services in AWS.
Learning Objectives:
• Learn why Spark is great for ad-hoc interactive analysis and real-time stream processing.
• How to deploy and tune scalable clusters running Spark on Amazon EMR.
• How to use EMR File System (EMRFS) with Spark to query data directly in Amazon S3.
• Common architectures to leverage Spark with Amazon DynamoDB, Amazon Redshift, Amazon Kinesis, and more.
Making (Almost) Any Database Faster and Cheaper with CachingAmazon Web Services
Learn how to make your AWS databases up to 10x faster and up to 90% less expensive with Amazon ElastiCache for Redis. We’ll look at how to determine whether caching will benefit your database environment and show how to easily test and implement a high speed solution.
Streaming Data Analytics with Amazon Redshift and Kinesis FirehoseAmazon Web Services
by Joyjeet Banerjee, Enterprise Solutions Architect, AWS
Evolving your analytics from batch processing to real-time processing can have a major business impact, but ingesting streaming data into your data warehouse requires building complex streaming data pipelines. Amazon Kinesis Firehose solves this problem by making it easy to transform and load streaming data into Amazon Redshift so that you can use existing analytics and business intelligence tools to extract information in near real-time and respond promptly. In this session, we will dive deep using Amazon Kinesis Firehose to load streaming data into Amazon Redshift reliably, scalably, and cost-effectively. Level: 200
What’s New in Amazon RDS for Open-Source and Commercial DatabasesAmazon Web Services
In the past year, Amazon RDS has continued to expand functionality, scalability, availability and ease of use for all supported database engines: PostgreSQL, MySQL, MariaDB, Oracle and Microsoft SQL Server. We’ll take a close look at RDS use cases and new capabilities, splitting the time between open-source and commercial database engines.
AWS ofrece una gran variedad de servicios de base de datos que se adaptan a los requisitos de su aplicación. Los servicios de bases de datos están totalmente administrados y se pueden implementar en cuestión de minutos con tan solo unos clics. Los servicios de AWS incluyen Amazon Relational Database Service (Amazon RDS), compatible con 6 motores de bases de datos comunes, Amazon Aurora, base de datos relacional compatible con MySQL con un desempeño 5 veces superior, Amazon DynamoDB, servicio de bases de datos NoSQL rápido y flexible, Amazon Redshift, almacén de datos a escala de petabytes, y Amazon Elasticache, servicio de caché en memoria compatible con Memcached y Redis. AWS también proporciona AWS Database Migration Service, un servicio que permite migrar las bases de datos a la nube de AWS de forma sencilla y rentable.
Traditional data warehouses become expensive and slow down as the volume of your data grows. Amazon Redshift is a fast, petabyte-scale data warehouse that makes it easy to analyze all of your data using existing business intelligence tools for 1/10th the traditional cost. This session will provide an introduction to Amazon Redshift and cover the essentials you need to deploy your data warehouse in the cloud so that you can achieve faster analytics and save costs.
Amazon Aurora is a cloud-optimized relational database that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. The recently announced PostgreSQL-compatibility, together with the original MySQL compatibility, are perfect for new application development and for migrations from overpriced, restrictive commercial databases. In this session, we’ll do a deep dive into the new architectural model and distributed systems techniques behind Amazon Aurora, discuss best practices and configurations, look at migration options and share customer experience from the field.
Amazon Aurora is a MySQL- and PostgreSQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. In this deep dive session, we’ll discuss best practices and explore new features in areas like high availability, security, performance management and database cloning.
RDS Postgres and Aurora Postgres | AWS Public Sector Summit 2017Amazon Web Services
Attend this session for a technical deep dive about RDS Postgres and Aurora Postgres. Come hear from Mark Porter, the General Manager of Aurora PostgreSQL and RDS at AWS, as he covers service specific use cases and applications within the AWS worldwide public sector community. Learn More: https://aws.amazon.com/government-education/
Introducing Amazon EMR Release 5.0 - August 2016 Monthly Webinar SeriesAmazon Web Services
Amazon EMR is a managed Hadoop service that makes it easy for customers to use big data frameworks and applications like Hadoop, Spark, and Presto to analyze data stored in HDFS or on Amazon S3 , Amazon’s highly scalable object storage service. In this webinar, we will introduce the latest release of Amazon EMR. With Amazon EMR release 5.0, customers can now launch the latest versions of popular open source frameworks including Apache Spark 2.0, Hive 2.1, Presto 0.151, Tez 0.8.4, and Apache Hadoop 2.7.2. We will walk through a demo to show you how to deploy a Hadoop environment within minutes. We will cover common use cases and best practices to lower costs using Amazon S3 as your data store and Amazon EC2 Spot Instances, which allow you to bid on space Amazon computing capacity.
Learning Objectives:
• Describe the new features and updated frameworks in Amazon EMR 5.0
• Learn best practices and real-world applications for Amazon EMR
• Understand how to use EC2 Spot pricing to save costs
• Explain the advantages of decoupling storage and compute with Amazon S3 as storage layer for EMR workloads
It’s been an exciting year for Amazon Aurora, the database with MySQL-compatible and PostgreSQL-compatible database engines. Amazon Aurora combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. In this deep dive session, we’ll discuss best practices and explore new features, including high availability options, new integrations with AWS services, and the performance management with Amazon RDS Performance Insights.
Running Oracle Databases on Amazon RDS - DAT313 - re:Invent 2017Amazon Web Services
Amazon Relational Database Service (Amazon RDS) simplifies setup, operation, and management of databases in the cloud. In this session, we will explore Amazon RDS features and best practices that offer graceful migration, high performance, elastic scaling, and high availability for Oracle databases. You will also learn from the Chief Architect for Intuit’s Small Business Division how the QuickBooks Online team is using Amazon RDS for Oracle to scale the world's largest online accounting platform.
This is an introduction to Amazon Redshift and cover the essentials you need to deploy your data warehouse in the cloud so that you can achieve faster analytics and save costs.
Amazon RDS enables you to launch an optimally configured, secure, and highly available relational database with just a few clicks. It provides cost-efficient and resizable capacity while managing time consuming administration tasks, freeing you to focus on your applications and business. In this session, we take a closer look at how Amazon RDS works, and we review best practices to achieve performance, flexibility, and cost savings for your MySQL, PostgreSQL, MariaDB, Oracle, and SQL Server databases on Amazon RDS. We also discuss AWS Database Migration Service, a quick and secure means for migrating your existing relational database management system investments to Amazon RDS.
Best Practices for Running Oracle Databases on Amazon RDS (DAT317) - AWS re:I...Amazon Web Services
Amazon Relational Database Service (Amazon RDS) continues to be a popular choice for Oracle DBAs moving new and legacy workloads to the cloud. In this session, we discuss how Amazon RDS for Oracle helps DBAs focus their time where it matters most. We cover recent RDS Oracle features, and we go deep on key functionality that enables license optimization, performance, and high availability for Oracle databases. We also hear directly from an AWS customer about their journey to Amazon RDS and the best practices that helped make their move successful.
Amazon Relational Database Service – How is it different to what you do today ?Amazon Web Services
Whether you are new to Amazon Relational Database Service (RDS) or just need a refresher on the latest features - this session is for you. We will take you through RDS from the perspective of how you manage your databases on-premise today. That is, we will describe the full lifecycle of a database from provisioning to decommissioning while taking into consideration your performance, scalability, availability, security and maintenance requirements.
Learn the fundamentals of Amazon DynamoDB and see the DynamoDB console first-hand as we walk through a demo of building a serverless web application using this high-performance key-value and JSON document store.
Strategic Uses for Cost Efficient Long-Term Cloud StorageAmazon Web Services
Compared to storing long-term datasets on-premises, archiving in the cloud is a smart alternative whether you’re looking for an active archive solution, tape replacement, or to fulfill a compliance requirement. Learn how AWS customers are simplifying their archiving strategy and meeting compliance needs using Amazon Glacier. Hear how customers have evolved their backup and disaster recovery architectures and replaced tape solutions by turning to AWS for a more cost efficient, durable and agile solution. We will showcase Sony DADC's active archive deployment on Glacier and demo how some of our financial service customers have set up compliant archives to meet their regulatory objectives.
Organizations need to perform increasingly complex analysis on data — streaming analytics, ad-hoc querying, and predictive analytics — in order to get better customer insights and actionable business intelligence. Apache Spark has recently emerged as the framework of choice to address many of these challenges. In this session, we show you how to use Apache Spark on AWS to implement and scale common big data use cases such as real-time data processing, interactive data science, predictive analytics, and more. We will talk about common architectures, best practices to quickly create Spark clusters using Amazon EMR, and ways to integrate Spark with other big data services in AWS.
Learning Objectives:
• Learn why Spark is great for ad-hoc interactive analysis and real-time stream processing.
• How to deploy and tune scalable clusters running Spark on Amazon EMR.
• How to use EMR File System (EMRFS) with Spark to query data directly in Amazon S3.
• Common architectures to leverage Spark with Amazon DynamoDB, Amazon Redshift, Amazon Kinesis, and more.
Making (Almost) Any Database Faster and Cheaper with CachingAmazon Web Services
Learn how to make your AWS databases up to 10x faster and up to 90% less expensive with Amazon ElastiCache for Redis. We’ll look at how to determine whether caching will benefit your database environment and show how to easily test and implement a high speed solution.
Streaming Data Analytics with Amazon Redshift and Kinesis FirehoseAmazon Web Services
by Joyjeet Banerjee, Enterprise Solutions Architect, AWS
Evolving your analytics from batch processing to real-time processing can have a major business impact, but ingesting streaming data into your data warehouse requires building complex streaming data pipelines. Amazon Kinesis Firehose solves this problem by making it easy to transform and load streaming data into Amazon Redshift so that you can use existing analytics and business intelligence tools to extract information in near real-time and respond promptly. In this session, we will dive deep using Amazon Kinesis Firehose to load streaming data into Amazon Redshift reliably, scalably, and cost-effectively. Level: 200
Operations: Production Readiness Review – How to stop bad things from HappeningAmazon Web Services
There is more to deploying code than pushing the deploy button. A good practice that many companies follow is a Production Readiness Review (PRR) which is essentially a pre-flight check list before a service launches. This helps ensure new services are properly architected, monitored, secured, and more. We’ll walk through an example PRR and discuss the value of ensuring each of these is properly taken care of before your service launches.
Apache Spark Streaming + Kafka 0.10 with Joan ViladrosarieraSpark Summit
Spark Streaming has supported Kafka since it’s inception, but a lot has changed since those times, both in Spark and Kafka sides, to make this integration more fault-tolerant and reliable.Apache Kafka 0.10 (actually since 0.9) introduced the new Consumer API, built on top of a new group coordination protocol provided by Kafka itself. So a new Spark Streaming integration comes to the playground, with a similar design to the 0.8 Direct DStream approach. However, there are notable differences in usage, and many exciting new features. In this talk, we will cover what are the main differences between this new integration and the previous one (for Kafka 0.8), and why Direct DStreams have replaced Receivers for good. We will also see how to achieve different semantics (at least one, at most one, exactly once) with code examples. Finally, we will briefly introduce the usage of this integration in Billy Mobile to ingest and process the continuous stream of events from our AdNetwork.
The presentation at DevFest Tokyo 2017 / @__timakin__
An introduction of blockchain and why go is nice to implement blockchain.
Additionally described about the blockchain projects that are based on Go.
Andrew Betts Web Developer, The Financial Times at Fastly Altitude 2016
Running custom code at the Edge using a standard language is one of the biggest advantages of working with Fastly’s CDN. Andrew gives you a tour of all the problems the Financial Times and Nikkei solve in VCL and how their solutions work.
Go's simplicity and concurrency model make it an appealing choice for backend systems, but how does it fare for latency-sensitive applications? In this talk, we explore the other side of the coin by providing some tips on writing high-performance Go and lessons learned in the process. We do a deep dive on low-level performance optimizations in order to make Go a more compelling option in the world of systems programming, but we also consider the trade-offs involved.
Want to build a custom app for Google Home or Google Assistant? Learn the basic concepts and how you can create a custom app to reach your users on new platforms (Google Home, Android, iPhone, and more) and help them get things done.
We'll use serverless tools like Google Cloud Functions as well as API.AI to do intelligent routing of commands to entities and intents.
Video of this talk available at: https://www.youtube.com/watch?v=C492KgDMO0c&list=PLlCd2ljeqltbJQQ79eyxbresnaKkP0TgS&index=1
Announcing Amazon Aurora with PostgreSQL Compatibility - January 2017 AWS Onl...Amazon Web Services
Amazon Aurora is now PostgreSQL compatible. With Amazon Aurora’s new PostgreSQL support, customers can get several times better performance than the typical PostgreSQL database and take advantage of the scalability, durability, and security capabilities of Amazon Aurora – all for one-tenth the cost of commercial grade databases. Amazon Aurora is a fully managed relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Amazon Aurora is built on a cloud native architecture that is designed to offer greater than 99.99 percent availability and automatic failover with no loss of data.
Learning Objectives:
• Learn about the capabilities and features of Amazon Aurora with PostgreSQL Compatibility
• Learn about the benefits and different use cases
• Learn how to get started using Amazon Aurora with PostgreSQL Compatibility
NEW LAUNCH! Introducing PostgreSQL compatibility for Amazon AuroraAmazon Web Services
After we launched Amazon Aurora, a cloud-native relational database with region-wide durability, high availability, fast failover, up to 15 read replicas, and up to five times the performance of MySQL, many of you asked us whether we could deliver the same features - but with PostgreSQL compatibility. We are now delivering a preview of Amazon Aurora with this functionality: we have built a PostgreSQL-compatible edition of Amazon Aurora, sharing the core Amazon Aurora innovations with the object-oriented capabilities, language interfaces, JSON compatibility, ANSI:SQL:2008 compliance, and broad functional richness of PostgreSQL. Amazon Aurora will provide full PostgreSQL compatibility while delivering more than twice the performance of the community PostgreSQL database on many workloads. At this session, we will be discussing the newest addition to Amazon Aurora in detail.
This presentation was used by Blair during his talk on Aurora and PostgreSQl compatibility for Aurora at pgDay Asia 2017. The talk was part of dedicated PostgreSQL track at FOSSASIA 2017
DAT340_Hands-On Journey for Migrating Oracle Databases to the Amazon Aurora P...Amazon Web Services
"In this workshop, we focus on the hands-on journey for migrating Oracle databases to the Aurora PostgreSQL-compatible Edition. Participants deploy an instance of Amazon Aurora, migrate or generate a test workload, and manually monitor the database to understand the workload. Participants also review multiple ways to track queries and their execution plans, and they determine how to optimize the queries. Finally, participants also learn how to use Amazon RDS Performance Insights for query-analysis and tuning.
Below are the prerequisites for the workshop.
Active AWS account with Admin privileges. (IAM user should have administrator access). Please refer the link on how to create IAM administrator user here
Existing EC2 key pair created in the AWS region you are launching the CloudFormation template in. Please refer below on how to first create a new Key pair as shown here
Pre-installed AWS Schema Conversion Tool software on your machine. Details on how to download and install AWS Schema Conversion Tool shown below
Install and launch SCT on your local machine from http://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_SchemaConversionTool.Installing.html
Download required drivers from links in the “Installing the Required Database Drivers” section from the above link. You will need to download Oracle and PostgreSQL drivers for this workshop. Alternatively, you can download the required drivers for this lab from
http://bit.ly/2phVpPk -> Oracle JDBC driver
http://bit.ly/2pt04ZT -> PostgreSQL JDBC driver
Download the Workshop Hands on lab guide http://bit.ly/2zYpnvS"
AWS January 2016 Webinar Series - Amazon Aurora for Enterprise Database Appli...Amazon Web Services
Relational databases are a cornerstone of the enterprise IT landscape, powering business-critical applications of many kinds. Though they have been around for a while, current commercial relational databases have lagged behind in innovation. Amazon Aurora, a managed database service built for the cloud, is intended to change that. It targets the high-performance needs of business-critical applications with an emphasis on cost-effectiveness.
In this session, we will look into how Aurora fits the needs of applications built and bought by enterprises to power their business.
Learning Objectives:
Learn about the overall architecture, capabilities, and cost-effectiveness of Aurora, comparing it to current commercial database offerings
Explore best practices for enterprises adopting Aurora for existing and new applications, as well as strategies, tools, and techniques for migrating existing databases to Aurora
Who Should Attend:
IT Managers, DBAs, Enterprise and Solution Architects , DevOps Engineers and Developers
Amazon Aurora for the Enterprise - August 2016 Monthly Webinar SeriesAmazon Web Services
Relational databases are a cornerstone of the enterprise IT landscape, powering business-critical applications of many kinds. Though they have been around for a while, current commercial relational databases have lagged behind in innovation. Amazon Aurora, a managed database service built for the cloud, is intended to change that. It fulfils the high-performance, high-availability needs of business-critical applications with an emphasis on cost-effectiveness. In this session, we will look into how Aurora fits the needs of applications built and bought by enterprises to power their business.
Learning Objectives:
• Explore the overall architecture, capabilities, and cost-effectiveness of Aurora and see how it compares to commercial database offerings
• Learn best practices for enterprises adopting Aurora for existing and new workloads, as well as strategies, tools, and techniques for migrating existing databases to Aurora
Amazon Aurora is a MySQL-compatible relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Amazon Aurora is disruptive technology in the database space, bringing a new architectural model and distributed systems techniques to provide far higher performance, availability, and durability than was previously available using conventional monolithic database techniques. In this session, we dive deep into some of the key innovations behind Amazon Aurora, discuss best practices and migration from other databases to Amazon Aurora, and share early customer experiences from the field.
Relational databases are a cornerstone of the enterprise IT landscape, powering business-critical applications of many kinds. Though they have been around for a while, current commercial relational databases have lagged behind in innovation. Amazon Aurora, a managed database service built for the cloud, is intended to change that. It targets the high-performance needs of business-critical applications with an emphasis on cost-effectiveness. In this session, we will look into how Aurora fits the needs of applications built and bought by enterprises to power their business. You will learn about the overall architecture, capabilities, and cost-effectiveness of Aurora, comparing it to current commercial database offerings. We will explore best practices for enterprises adopting Aurora for existing and new workloads, as well as strategies, tools, and techniques for migrating existing databases to Aurora. You will also hear from Expedia, one of world’s leading travel companies on how they are using Amazon Aurora to power application with high performance database needs.
Amazon Aurora is a MySQL-compatible relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Amazon Aurora is disruptive technology in the database space, bringing a new architectural model and distributed systems techniques to provide far higher performance, availability and durability than previously available using conventional monolithic database techniques. In this session, we will do a deep-dive into some of the key innovations behind Amazon Aurora, discuss best practices and configurations, and share early customer experience from the field.
Amazon Aurora is a MySQL-compatible relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. Amazon Aurora is disruptive technology in the database space, bringing a new architectural model and distributed systems techniques to provide far higher performance, availability and durability than previously available using conventional monolithic database techniques. In this session, we will do a deep-dive into some of the key innovations behind Amazon Aurora, discuss best practices and configurations, and share early customer experience from the field.
Amazon Aurora is a MySQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. This session introduces you to Amazon Aurora, explains common use cases for the service, and helps you get started with building your first Amazon Aurora–powered application.
AWS re:Invent 2016: Workshop: Stretching Scalability: Doing more with Amazon ...Amazon Web Services
Easy scalability is a powerful feature of Amazon Aurora. Scalability in its actual definition refers to being able to get larger or smaller depending on the need. Amazon Aurora allows you to easily achieve this by scaling the database instance up or down and adding or removing read replicas. Scaling across regions brings additional resilience to your architectures and could boost your application performance due to geographic proximity. You can perform all of these scaling operations through the Aurora console. You can also automate instance and read scaling using lambda function or scripts based on the usage pattern you define. You can extend the automation by feeding your database usage data from Aurora enhanced monitoring into Machine Learning to provide more sophisticated predictive patterns to drive your automation. In this session we will do a deep dive into how scalability works in Aurora and how to make the best use of it to reduce your cost, increase application performance and architect resilient applications.
You should have good database knowledge and at least some experience with Amazon RDS or Amazon Aurora and should bring your own laptop.
Build on Amazon Aurora with MySQL Compatibility (DAT348-R4) - AWS re:Invent 2018Amazon Web Services
Amazon Aurora is a MySQL- and PostgreSQL-compatible relational database with the speed, reliability, and availability of commercial databases at one-tenth the cost. Join this session, and get started with the MySQL-compatible edition, discuss your existing application running on Aurora, or learn about recently announced features, such as Serverless or Parallel Query.
Amazon Aurora is a MySQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. This session introduces you to Amazon Aurora, explains common use cases for the service, and helps you get started with building your first Amazon Aurora–powered application.
It’s been an exciting year for Amazon Aurora, the MySQL-compatible relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. In this deep dive session, we’ll discuss best practices and explore new features, include high availability options and new integrations with AWS services. We’ll also discuss the recently-announced Aurora with PostgreSQL compatibility.
Amazon Aurora is a MySQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. This session introduces you to Amazon Aurora, explains common use cases for the service, and helps you get started with building your first Amazon Aurora–powered application.
Migrating Your Oracle & SQL Server Databases to Amazon Aurora (DAT318) - AWS ...Amazon Web Services
Organizations today are looking to free themselves from the constraints of on-premises commercial databases and leverage the power of cloud-native and open-source systems. Amazon Aurora is a MySQL- and PostgreSQL-compatible relational database that is built for the cloud, with the speed, reliability, and availability of commercial databases at one-tenth the cost. In this session, we provide an overview of Aurora and its features. We talk about the latest advances in migration tooling and automation, and we explain how many of the common legacy features of Oracle and SQL Server map to modern cloud variants. We also hear from Dow Jones about its migration journey to the cloud.
AWS re:Invent 2016: Getting Started with Amazon Aurora (DAT203)Amazon Web Services
Amazon Aurora is a MySQL-compatible relational database engine with the speed, reliability, and availability of high-end commercial databases at one-tenth the cost. This session introduces you to Amazon Aurora, explores the capabilities and features of Aurora, explains common use cases, and helps you get started with Aurora. Debanjan Saha, general manager for Aurora, explains how Aurora differs from other commonly available databases while staying compatible with MySQL and providing a high-end, cost-effective alternative to commercial and open-source database engines. In addition, Linda Xu, data architect at Ticketmaster, walks you through Ticketmaster's journey to Amazon Aurora, starting with evaluation through production migration of a critical Ticketmaster database to Amazon Aurora. Ticketmaster is one of the world's top 10 e-commerce companies and the global market leader in ticketing. In this session, Linda discusses how Aurora lets Ticketmaster provide better services to their fans, customers, and clients, and helps reduce the cost and operational burden while giving greater flexibility to support heavy traffic spikes.
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS.
In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup.
Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti.
Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.
Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi
Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
L’utilizzo dei container è in continua crescita.
Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili.
I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!
In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th.
Event Agenda :
Open banking so far (short recap)
• PSD2, OB UK, OB Australia, OB LATAM, OB Israel
Intro to Open Finance marketplace
• Scope
• Features
• Tech overview and Demo
The role of the Cloud
The Future of APIs
• Complying with regulation
• Monetizing data / APIs
• Business models
• Time to market
One platform for all: a Strategic approach
Q&A
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc.
AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta.
Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity.
AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet.
Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.
Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.
Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti.
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti.
Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire.
Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito.
In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.
Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline.
Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.
Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.
2. Agenda
§ Why did we build Amazon Aurora?
§ Why add PostgreSQL compatibility?
§ Durability and Availability Architecture
§ Performance Results vs. PostgreSQL
§ Performance Architecture
§ Announcing Performance Insights
§ Getting Data In
§ Feature Roadmap
§ Preview Information & Questions
+
3. Traditional relational databases are hard to scale
Multiple layers of
functionality all in a
monolithic stack
SQL
Transactions
Caching
Logging
Storage
4. Traditional approaches to scale databases
Each architecture is limited by the monolithic mindset
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application Application
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Storage
Application
Storage Storage
SQL
Transactions
Caching
Logging
Storage
SQL
Transactions
Caching
Logging
Storage
5. Reimagining the relational database
What if you were inventing the database today?
You would break apart the stack
You would build something that:
ü Can scale out…
ü Is self-healing…
ü Leverages distributed services…
6. A service-oriented architecture applied to the database
Move the logging and storage layer into a
multitenant, scale-out, database-optimized
storage service
Integrate with other AWS services like
Amazon EC2, Amazon VPC, Amazon
DynamoDB, Amazon SWF, and Amazon
Route 53 for control & monitoring
Make it a managed service – using Amazon
RDS. Takes care of management and
administrative functions.
Amazon
DynamoDB
Amazon SWF
Amazon Route 53
Logging + Storage
SQL
Transactions
Caching
Amazon S3
1
2
3
Amazon RDS
7. Cloud-optimized relational database
Performance and availability of
commercial databases
Simplicity and cost-effectiveness of
open source databases,
with MySQL compatibility
What is Amazon Aurora?
9. In 2014, we launched Amazon Aurora with MySQL compatibility.
Now, we are adding PostgreSQL compatibility.
Customers can now choose how to use Amazon’s
cloud-optimized relational database, with the performance and
availability of commercial databases and the simplicity and cost-
effectiveness of open source databases.
Making Amazon Aurora Better
12. Customer Migration Scenarios to Amazon Aurora
Migrate from Amazon EC2 or on-premises
Migrate from Amazon RDS for PostgreSQL
Migrate Oracle and SQL Server applications
Build new applications
13. Open source database
In active development for 20 years
Owned by a foundation, not a single company
Permissive innovation-friendly open source license
PostgreSQL Fast Facts
Open Source Initiative
14. High performance out of the box
Object-oriented and ANSI-SQL:2008 compatible
Most geospatial features of any open-source database
Supports stored procedures in 12 languages (Java, Perl,
Python, Ruby, Tcl, C/C++, its own Oracle-like PL/pgSQL,
etc.)
PostgreSQL Fast Facts
15. Most Oracle-compatible open-source database
Highest AWS Schema Conversion Tool automatic
conversion rates are from Oracle to PostgreSQL
PostgreSQL Fast Facts
16. What does PostgreSQL compatibility mean?
PostgreSQL 9.6 + Amazon Aurora cloud-optimized storage
Performance: Up to 2x+ better performance than PostgreSQL alone
Availability: failover time of < 30 seconds
Durability: 6 copies across 3 Availability Zones
Read Replicas: single-digit millisecond lag times on up to 15 replicas
Amazon Aurora Storage
17. What does PostgreSQL compatibility mean?
Cloud-native security and encryption
AWS Key Management Service (KMS) and AWS
Identity and Access Management (IAM)
Easy to manage with Amazon RDS
Easy to load and unload
AWS Database Migration Service and AWS Schema
Conversion Tool
Fully compatible with PostgreSQL, now and for the
foreseeable future
Not a compatibility layer – native PostgreSQL
implementation
AWS DMS
Amazon RDS
PostgreSQL
20. Amazon Aurora Storage Engine Overview
Data is replicated 6 times across 3 Availability
Zones
Continuous backup to Amazon S3
(built for 11 9s durability)
Continuous monitoring of nodes and disks for
repair
10GB segments as unit of repair or hotspot
rebalance
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Storage volume automatically grows up to 64 TB
AZ 1 AZ 2 AZ 3
Amazon S3
Database
Node
Storage
Node
Storage
Node
Storage
Node
Storage
Node
Storage
Node
Storage
Node
Storage
Monitoring
21. What can fail?
Segment failures (disks)
Node failures (machines)
AZ failures (network or datacenter)
Optimizations
4 out of 6 write quorum
3 out of 6 read quorum
Peer-to-peer replication for repairs
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
Amazon Aurora Storage Engine Fault-tolerance
22. Amazon Aurora Replicas
Availability
Failing database nodes are automatically
detected and replaced
Failing database processes are
automatically detected and recycled
Replicas are automatically promoted to
primary if needed (failover)
Customer specifiable fail-over order
AZ 1 AZ 3AZ 2
Primary
Node
Primary
Node
Primary
Database
Node
Primary
Node
Primary
Node
Read
Replica
Primary
Node
Primary
Node
Read
Replica
Database
and
Instance
Monitoring
Performance
Customer applications can scale out read traffic
across read replicas
Read balancing across read replicas
23. Amazon Aurora Continuous Backup
Segment snapshot Log records
Recovery point
Segment 1
Segment 2
Segment 3
Time
• Take periodic snapshot of each segment in parallel; stream the logs to Amazon S3
• Backup happens continuously without performance or availability impact
• At restore, retrieve the appropriate segment snapshots and log streams to storage nodes
• Apply log streams to segment snapshots in parallel and asynchronously
24. Traditional databases
Have to replay logs since the last
checkpoint
Typically 5 minutes between checkpoints
Single-threaded in MySQL and
PostgreSQL; requires a large number of
disk accesses
Amazon Aurora
No replay at startup because storage system
is transaction-aware
Underlying storage replays log records
continuously, whether in recovery or not
Coalescing is parallel, distributed, and
asynchronous
Checkpointed Data Log
Crash at T0 requires
a re-application of the
SQL in the log since
last checkpoint
T0 T0
Crash at T0 will result in logs being applied to
each segment on demand, in parallel,
asynchronously
Amazon Aurora Instant Crash Recovery
25. Faster, more predictable failover with Amazon Aurora
App
RunningFailure Detection DNS Propagation
Recovery
Database
Failure
Amazon RDS for PostgreSQL is good: failover times of ~60 seconds
Replica-Aware App Running
Failure Detection DNS Propagation
Recovery
Database
Failure
Amazon Aurora is better: failover times < 30 seconds
1 5 - 2 0 s e c 3 - 1 0 s e c
App
Running
27. PostgreSQL
Benchmark System Configurations
Amazon Aurora
AZ 1
EBS EBS EBS
45,000 total IOPS
AZ 1 AZ 2 AZ 3
Amazon S3
m4.16xlarge
database
instance
Storage
Node
Storage
Node
Storage
Node
Storage
Node
Storage
Node
Storage
Node
c4.8xlarge
client driver
m4.16xlarge
database
instance
c4.8xlarge
client driver
ext4 filesystem
m4.16xlarge (64 VCPU, 256GiB), c4.8xlarge (36 VCPU, 60GiB)
28. Amazon Aurora is >=2x Faster on PgBench
pgbench “tpcb-like” workload, scale 2000 (30GiB). All configurations run for 60 minutes
29. Amazon Aurora is 2x-3x Faster on SysBench
Amazon Aurora delivers 2x the absolute peak of PostgreSQL and 3x
PostgreSQL performance at high client counts
SysBench oltp(write-only) workload with 30 GB database with 250 tables and 400,000 initial rows per table
30. Amazon Aurora: Over 120,000 Writes/Sec
OLTP test statistics:
queries performed:
read: 0
write: 432772903
other:(begin + commit) 216366749
total: 649139652
transactions: 108163671 (30044.73 per sec.)
read/write requests: 432772903 (120211.75 per sec.)
other operations: 216366749 (60100.40 per sec.)
ignored errors: 39407 (10.95 per sec.)
reconnects: 0 (0.00 per sec.)
sysbench write-only 10GB workload with 250 tables and 25,000 initial rows per table. 10-minute warmup, 3,076 clients
Ignored errors are key constraint errors, designed into sysbench
Sustained sysbench throughput over 120K writes/sec
31. Amazon Aurora Loads Data 3x Faster
Database initialization is three times faster than PostgreSQL using the
standard PgBench benchmark
Command: pgbench -i -s 2000 –F 90
32. Amazon Aurora Gives >2x Faster Response Times
Response time under heavy write load >2x faster than PostgreSQL
(and >10x more consistent)
SysBench oltp(write-only) 23GiB workload with 250 tables and 300,000 initial rows per table. 10-minute warmup.
33. Amazon Aurora Has More Consistent Throughput
While running at load, performance is more than three times
more consistent than PostgreSQL
PgBench “tpcb-like” workload at scale 2000. Amazon Aurora was run with 1280 clients. PostgreSQL was run with
512 clients (the concurrency at which it delivered the best overall throughput)
34. Amazon Aurora is 3x Faster at Large Scale
Scales from 1.5x to 3x faster as database grows from 10 GiB to 100 GiB
SysBench oltp(write-only) – 10GiB with 250 tables & 150,000 rows and 100GiB with 250 tables & 1,500,000 rows
75,666
27,491
112,390
82,714
0
20,000
40,000
60,000
80,000
100,000
120,000
10GB 100GB
writes/sec
SysBench Test Size
SysBench write-only
PostgreSQL Amazon Aurora
35. Amazon Aurora Delivers up to 85x Faster Recovery
SysBench oltp(write-only) 10GiB workload with 250 tables & 150,000 rows
Writes per Second 69,620
Writes per Second 32,765
Writes per Second 16,075
Writes per Second 92,415
Recovery Time (seconds) 102.0
Recovery Time (seconds) 52.0
Recovery Time (seconds) 13.0
Recovery Time (seconds) 1.2
0 20 40 60 80 100 120 140
0 20,000 40,000 60,000 80,000
PostgreSQL
12.5GB
Checkpoint
PostgreSQL
8.3GB
Checkpoint
PostgreSQL
2.1GB
Checkpoint
Amazon Aurora
No Checkpoints
Recovery Time in Seconds
Writes Per Second
Crash Recovery Time - SysBench 10GB Write Workload
Transaction-aware storage system recovers almost instantly
36. Amazon Aurora with PostgreSQL Compatibility
Performance By The Numbers
Measurement Result
PgBench >= 2x faster
SysBench 2x-3x faster
Data Loading 3x faster
Response Time >2x faster
Throughput Jitter >3x more consistent
Throughput at Scale 3x faster
Recovery Speed Up to 85x faster
38. Do fewer IOs
Minimize network packets
Offload the database engine
DO LESS WORK
Process asynchronously
Reduce latency path
Use lock-free data structures
Batch operations together
BE MORE EFFICIENT
How Does Amazon Aurora Achieve High Performance?
DATABASES ARE ALL ABOUT I/O
NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND
HIGH-THROUGHPUT PROCESSING NEEDS CPU AND MEMORY OPTIMIZATIONS
39. Write IO Traffic in Amazon RDS for PostgreSQL
WAL DATA COMMIT LOG & FILES
RDS FOR POSTGRESQL WITH MULTI-AZ
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
EBS
Amazon Elastic
Block Store (EBS)
Primary
Database
Node
Standby
Database
Node
1
2
3
4
5
Issue write to Amazon EBS, EBS issues to mirror,
acknowledge when both done
Stage write to standby instance
Issue write to EBS on standby instance
IO FLOW
Steps 1, 3, 5 are sequential and synchronous
This amplifies both latency and jitter
Many types of writes for each user operation
OBSERVATIONS
T Y P E O F W R I T E
Write IO Traffic in Amazon RDS for PostgreSQL
40. Write IO Traffic in an Amazon Aurora Database Node
AZ 1 AZ 3
Primary
Database
Node
Amazon S3
AZ 2
Read
Replica /
Secondary
Node
AMAZON AURORA
ASYNC
4/6 QUORUM
DISTRIBUTED
WRITES
DATAAMAZON AURORA + WAL LOG COMMIT LOG & FILES
IO FLOW
Only write WAL records; all steps asynchronous
No data block writes (checkpoint, cache replacement)
6X more log writes, but 9X less network traffic
Tolerant of network and storage outlier latency
OBSERVATIONS
2x or better PostgreSQL Community Edition performance on
write-only or mixed read-write workloads
PERFORMANCE
Boxcar log records – fully ordered by LSN
Shuffle to appropriate segments – partially ordered
Boxcar to storage nodes and issue writes
WAL
T Y P E O F W R I T E
Read
Replica /
Secondary
Node
41. Write IO Traffic in an Amazon Aurora Storage Node
LOG RECORDS
Primary
Database
Node
INCOMING QUEUE
STORAGE NODE
AMAZON S3 BACKUP
1
2
3
4
5
6
7
8
UPDATE
QUEUE
ACK
HOT
LOG
DATA
BLOCKS
POINT IN TIME
SNAPSHOT
GC
SCRUB
COALESCE
SORT
GROUP
PEER TO PEER GOSSIPPeer
Storage
Nodes
All steps are asynchronous
Only steps 1 and 2 are in foreground latency path
Input queue is far smaller than PostgreSQL
Favors latency-sensitive operations
Uses disk space to buffer against spikes in activity
OBSERVATIONS
IO FLOW
① Receive record and add to in-memory queue
② Persist record and acknowledge
③ Organize records and identify gaps in log
④ Gossip with peers to fill in holes
⑤ Coalesce log records into new data block versions
⑥ Periodically stage log and new block versions to Amazon
S3
⑦ Periodically garbage collect old versions
⑧ Periodically validate CRC codes on blocks
42. Applications Restart Faster With Survivable Caches
Cache normally lives inside the
operating system database process–
and goes away when/if that database
dies
Aurora moves the cache out of the
database process
Cache remains warm in the event of a
database restart
Lets the database resume fully loaded
operations much faster
Cache lives outside the database
process and remains warm across
database restarts
SQL
Transactions
Caching
SQL
Transactions
Caching
SQL
Transactions
Caching
RUNNING CRASH AND RESTART RUNNING
43. Amazon Aurora with PostgreSQL Compatibility
Performance monitoring and management
44. First Step: Enhanced Monitoring
Released 2016
O/S Metrics
Process & thread List
Up to 1 second granularity
46. Why Database Tuning?
RDS is all about managed databases
Customers want performance managed too:
q Want easy tool for optimizing cloud database workloads
q May not have deep tuning expertise
à Want a single pane of glass to achieve this
47. What makes Database Load
such a useful metric?
• Based on sampling active database requests
• Frequent sampling builds a time model of usage
• Visualizations illuminate the time model in one chart
48.
49.
50. Performance Insights at a glance
Automates sampling of data
Exposes data via API
Provides UI to show Database Load
Database Load: