In this session, we explore features and functions of AWS Storage Services. We give context on the portfolio, cover the most common use cases for AWS offerings for object, file, block and migration technologies, including thepartner ecosystem, and then go into each service with customer case studiy examples. Leave this session with an understanding of how to select storage and start moving workloads or building new ones.
Building Hybrid Cloud Storage Architectures with AWS @scaleAmazon Web Services
The document discusses building hybrid cloud storage architectures with AWS. It provides an overview of AWS storage services including Amazon S3, Glacier, EBS, and EFS. It also describes the AWS Storage Gateway family of on-premises appliances that enable hybrid storage between on-premises and AWS cloud storage. Specifically, it covers the File Gateway for accessing S3 storage as files, Volume Gateway for iSCSI volumes, and Tape Gateway for migrating tape backups to S3.
The introductory morning session will discuss big data challenges and provide an overview of the AWS Big Data Platform. We will also cover:
• How AWS customers leverage the platform to manage massive volumes of data from a variety of sources while containing costs.
• Reference architectures for popular use cases, including: connected devices (IoT), log streaming, real-time intelligence, and analytics.
• The AWS big data portfolio of services, including Amazon S3, Kinesis, DynamoDB, Elastic MapReduce (EMR) and Redshift.
• The latest relational database engine, Amazon Aurora - a MySQL-compatible, highly-available relational database engine which provides up to five times better performance than MySQL at a price one-tenth the cost of a commercial database.
• Amazon Machine Learning – the latest big data service from AWS provides visualization tools and wizards that guide you through the process of creating machine learning (ML) models without having to learn complex ML algorithms and technology.
AWS Summit Singapore - Architecting a Serverless Data Lake on AWSAmazon Web Services
Unni Pillai, Specialist Solution Architect, ASEAN, AWS.
Daniel Muller, Head of Cloud Infrastructure, Spuul.
As the volume and types of data continues to grow, customers often have valuable data that is not easily discoverable and available for analytics. A common challenge for data engineering teams is architecting a data lake that can cater to the needs of diverse users - from developers to business analysts to data scientists.
In this session, we will dive deep into building a data lake using Amazon S3, Amazon Kinesis, Amazon Athena and AWS Glue. We will also see how AWS Glue crawlers can automatically discover your data, extracting and cataloguing relevant metadata to reduce operations in preparing your data for downstream consumers.
Furthermore, learn from our customer Spuul, on how they moved from a Data Warehouse based analytics to a serverless data lake. Why and how did Spuul undertake this journey? Hear about the benefits and challenges they encountered.
by Drew Meyer, Sr. Product Marketing Manager, AWS
This session will provide an overview of the AWS storage portfolio, including block, file, object, and cloud data migration services. We will touch on new offerings, outline some of the most common use cases, and prepare you for the individual deep dive sessions, customer sessions, and new announcements. The session will also address our partner network and what it means for a storage provider to have the APN Storage Competency.
This document provides an overview of AWS databases and analytics services. It discusses AWS's broad portfolio of purpose-built databases including relational databases like RDS and Aurora, non-relational databases like DynamoDB and Neptune, data lakes with S3 and Glue, data movement services, and analytics services like Redshift, EMR, and Athena. It also covers key concepts around relational and non-relational data models and provides examples of common use cases for different database types.
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...Amazon Web Services
Level 200: Visualize Your Data in Data Lake with AWS Athena and AWS Quicksight
Nowadays, enterprises are building Data Lake which store lots of structured and unstructured data for data analysis. But it takes lots of time for building the data modeling and infrastructure that is required. How to make quick data queries without servers and databases is the next big question for every enterprises.
In this workshop, eCloudvalley, the first and only Premier Consulting Partner in GCR, will demonstrate how to use serverless architecture to visualize your data using Amazon Athena and Amazon Quicksight.
You can easily query and visualize the data in your S3, and get business insights with the combination of these two services. Also, you can also build business reports with other tools such as AWS IoT, Amazon Kinesis Firehose.
Reason to Attend:
Learn how to quickly search for thousands of data on S3 via serverless Amazon's Athena
Learn how to use AWS QuickSight to retrieve information from your database quickly and create detailed reports
Hybrid as a Stepping Stone: It’s Not All or Nothing for Your Cloud Transforma...Amazon Web Services
The implementation of highly scalable, easy-to-deploy technology is transforming the public sector, but it's not a one-size-fits-all approach. Organizations begin their cloud adoption journeys in many ways. Some start with pilot projects and others jump into mission critical programs, but they are all starting with an existing infrastructure. Adopting cloud doesn't mean scrapping it all and starting over. This session explores how organizations can extend their existing IT platforms into the cloud to enable hybrid capabilities capable of supporting every phase of their transformation. Learn More: https://aws.amazon.com/government-education/
Building Hybrid Cloud Storage Architectures with AWS @scaleAmazon Web Services
The document discusses building hybrid cloud storage architectures with AWS. It provides an overview of AWS storage services including Amazon S3, Glacier, EBS, and EFS. It also describes the AWS Storage Gateway family of on-premises appliances that enable hybrid storage between on-premises and AWS cloud storage. Specifically, it covers the File Gateway for accessing S3 storage as files, Volume Gateway for iSCSI volumes, and Tape Gateway for migrating tape backups to S3.
The introductory morning session will discuss big data challenges and provide an overview of the AWS Big Data Platform. We will also cover:
• How AWS customers leverage the platform to manage massive volumes of data from a variety of sources while containing costs.
• Reference architectures for popular use cases, including: connected devices (IoT), log streaming, real-time intelligence, and analytics.
• The AWS big data portfolio of services, including Amazon S3, Kinesis, DynamoDB, Elastic MapReduce (EMR) and Redshift.
• The latest relational database engine, Amazon Aurora - a MySQL-compatible, highly-available relational database engine which provides up to five times better performance than MySQL at a price one-tenth the cost of a commercial database.
• Amazon Machine Learning – the latest big data service from AWS provides visualization tools and wizards that guide you through the process of creating machine learning (ML) models without having to learn complex ML algorithms and technology.
AWS Summit Singapore - Architecting a Serverless Data Lake on AWSAmazon Web Services
Unni Pillai, Specialist Solution Architect, ASEAN, AWS.
Daniel Muller, Head of Cloud Infrastructure, Spuul.
As the volume and types of data continues to grow, customers often have valuable data that is not easily discoverable and available for analytics. A common challenge for data engineering teams is architecting a data lake that can cater to the needs of diverse users - from developers to business analysts to data scientists.
In this session, we will dive deep into building a data lake using Amazon S3, Amazon Kinesis, Amazon Athena and AWS Glue. We will also see how AWS Glue crawlers can automatically discover your data, extracting and cataloguing relevant metadata to reduce operations in preparing your data for downstream consumers.
Furthermore, learn from our customer Spuul, on how they moved from a Data Warehouse based analytics to a serverless data lake. Why and how did Spuul undertake this journey? Hear about the benefits and challenges they encountered.
by Drew Meyer, Sr. Product Marketing Manager, AWS
This session will provide an overview of the AWS storage portfolio, including block, file, object, and cloud data migration services. We will touch on new offerings, outline some of the most common use cases, and prepare you for the individual deep dive sessions, customer sessions, and new announcements. The session will also address our partner network and what it means for a storage provider to have the APN Storage Competency.
This document provides an overview of AWS databases and analytics services. It discusses AWS's broad portfolio of purpose-built databases including relational databases like RDS and Aurora, non-relational databases like DynamoDB and Neptune, data lakes with S3 and Glue, data movement services, and analytics services like Redshift, EMR, and Athena. It also covers key concepts around relational and non-relational data models and provides examples of common use cases for different database types.
Visualize your data in Data Lake with AWS Athena and AWS Quicksight Hands-on ...Amazon Web Services
Level 200: Visualize Your Data in Data Lake with AWS Athena and AWS Quicksight
Nowadays, enterprises are building Data Lake which store lots of structured and unstructured data for data analysis. But it takes lots of time for building the data modeling and infrastructure that is required. How to make quick data queries without servers and databases is the next big question for every enterprises.
In this workshop, eCloudvalley, the first and only Premier Consulting Partner in GCR, will demonstrate how to use serverless architecture to visualize your data using Amazon Athena and Amazon Quicksight.
You can easily query and visualize the data in your S3, and get business insights with the combination of these two services. Also, you can also build business reports with other tools such as AWS IoT, Amazon Kinesis Firehose.
Reason to Attend:
Learn how to quickly search for thousands of data on S3 via serverless Amazon's Athena
Learn how to use AWS QuickSight to retrieve information from your database quickly and create detailed reports
Hybrid as a Stepping Stone: It’s Not All or Nothing for Your Cloud Transforma...Amazon Web Services
The implementation of highly scalable, easy-to-deploy technology is transforming the public sector, but it's not a one-size-fits-all approach. Organizations begin their cloud adoption journeys in many ways. Some start with pilot projects and others jump into mission critical programs, but they are all starting with an existing infrastructure. Adopting cloud doesn't mean scrapping it all and starting over. This session explores how organizations can extend their existing IT platforms into the cloud to enable hybrid capabilities capable of supporting every phase of their transformation. Learn More: https://aws.amazon.com/government-education/
"Wipro is one of India's largest publicly traded companies and the seventh largest IT services firm in the world. In this session, we showcase the structured methods that Wipro has used in enabling enterprises to take advantage of the cloud. These cover identifying workloads and application profiles that could benefit, re-structuring enterprise application and infrastructure components for migration, rapid and thorough verification and validation, and modifying component monitoring and management.
Several of these methods can be tailored to the individual client or functional context, so specific client examples are presented. We also discuss the enterprise experience of enabling many non-IT functions to benefit from the cloud, such as sales and training. More functions included in the cloud increase the benefit drawn from a cloud-enabled IT landscape.
Session sponsored by Wipro."
講師: Ivan Cheng, Solution Architect, AWS
Join us for a series of introductory and technical sessions on AWS Big Data solutions. Gain a thorough understanding of what Amazon Web Services offers across the big data lifecycle and learn architectural best practices for applying those solutions to your projects.
We will kick off this technical seminar in the morning with an introduction to the AWS Big Data platform, including a discussion of popular use cases and reference architectures. In the afternoon, we will deep dive into Machine Learning and Streaming Analytics. We will then walk everyone through building your first Big Data application with AWS.
This session provides a foundational overview of the AWS storage portfolio, including block, file, object, and cloud data migration services. This session will touch on the significant new offerings, outline some of the most common use cases and prepare you for the individual deep dive sessions, customer sessions and new announcements.
We will cover the core AWS storage services, which include Amazon Simple Storage Service (Amazon S3), Amazon Glacier, Amazon Elastic File System (Amazon EFS), and Amazon Elastic Block Store (Amazon EBS). We also discuss data transfer services such as AWS Snowball, Snowball Edge, and AWS Snowmobile, and hybrid storage solutions such as AWS Storage Gateway.
Building Serverless Web Applications - DevDay Los Angeles 2017Amazon Web Services
The document provides information about building serverless web applications using AWS Lambda and other AWS services. It begins with an overview of serverless computing using AWS Lambda and how it avoids the need to provision and manage servers. It then discusses various AWS compute offerings and when to use EC2, ECS, or Lambda. The rest of the document discusses serverless design patterns, demonstrates building a serverless web application using services like API Gateway and DynamoDB, and how to define and manage serverless applications using the AWS Serverless Application Model (SAM).
AWS Big Data and Analytics Services Speed Innovation | AWS Public Sector Summ...Amazon Web Services
Data-driven agencies face extreme data integration and analytics challenges. Decades of point solutions have solved specific mission problems while creating valuable data stores. However, these data stores are not integrated and are stored in information silos. AWS's powerful data ingestion and integration services now allow agencies to rapidly store more in data lakes for deeper analytics. Join this discussion on how FAA and other agencies have leveraged AWS data integration and analytic services to optimize and innovate with their previously untapped information silos. Learn More: https://aws.amazon.com/government-education/
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...Amazon Web Services
Amazon RDS allows you to launch an optimally configured, secure and highly available database with just a few clicks. It provides cost-efficient and resizable capacity, automates time-consuming database administration tasks, and provides you with six familiar database engines to choose from: Amazon Aurora, Oracle, Microsoft SQL Server, PostgreSQL, MySQL and MariaDB. In this session, we will take a close look at the capabilities of Amazon RDS and explain how it works. We’ll also discuss the AWS Database Migration Service and AWS Schema Conversion Tool, which help you migrate databases and data warehouses with minimal downtime from on-premises and cloud environments to Amazon RDS and other Amazon services. Gain your freedom from expensive, proprietary databases while providing your applications with the fast performance, scalability, high availability, and compatibility they need.
AWS offers numerous services to migrate data at a petabyte scale. You can easily move large volumes of data from onsite to the cloud and utilize the cloud as a backup target using data transfer services, such as AWS Snowball, AWS Snowball Edge, or AWS Storage Gateway. Learn about available data migration options and which one is the right fit for your requirements.
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMRAmazon Web Services
Customers are migrating their analytics, data processing (ETL), and data science workloads running on Apache Hadoop, Spark, and data warehouse appliances from on-premise deployments to Amazon EMR in order to save costs, increase availability, and improve performance. Amazon EMR is a managed service that lets you process and analyze extremely large data sets using the latest versions of over 15 open-source frameworks in the Apache Hadoop and Spark ecosystems. This session will focus on identifying the components and workflows in your current environment and providing the best practices to migrate these workloads to Amazon EMR. We will explain how to move from HDFS to Amazon S3 as a durable storage layer, and how to lower costs with Amazon EC2 Spot instances and Auto Scaling. Additionally, we will go over common security recommendations and tuning tips to accelerate the time to production.
Join us for an in-depth look at the current state of big data at AWS. Learn about the latest big data trends and industry use cases. Hear how other organizations are using the AWS big data platform to innovate and remain competitive. Take a look at some of the most recent AWS big data developments.
Backup and Recovery with Cloud-Native Deduplication and Use Cases from the Fi...Amazon Web Services
by Hugh Emberson, CTO, StorReduce
Designing and deploying cloud-enabled backup & recovery solutions often leads to opportunities for reducing storage requirements and increasing efficiencies. Having effective cloud-native deduplication capabilities as part of your backup & recovery strategy can optimize migration, decrease the need for purpose built backup appliances like Data Domains, large tape archives, and enable cost reductions of up to 95%. In this session, StorReduce will provide best practices around data deduplication in relation to designing and deploying solutions around backup, archive, and general unstructured file data. They will also demonstrate how using a cloud native interface with scale-out deduplication enables generic cloud services like search inside all backups moved to cloud. They will guide the audience through two customer use cases from the financial services and healthcare industries.
by PD Dutta, Sr. Product Manager, Object Storage, AWS
We will explain how to design and build an IoT cloud platform on top of Amazon S3. You will get to review the best practices for architecting a cost-effective, durable, and secure storage solution to store and analyze your IoT data on Amazon S3. In addition, we’ll cover how to collect, ingest and analyze the data in-place using different AWS Services such as AWS IoT, Amazon Kinesis, Amazon Athena, and Amazon Redshift Spectrum.
We will introduce key concepts for a data lake and present aspects related to its implementation. Also discussing critical success factors, pitfalls to avoid operational aspects, and insights on how AWS enables a server-less data lake architecture.
Speaker: Sebastien Menant, Solutions Architect, Amazon Web Services
A Look Under the Hood – How Amazon.com Uses AWS Services for Analytics at Mas...Amazon Web Services
Amazon’s consumer business continues to grow, and so does the volume of data and the number and complexity of the analytics done in support of the business. In this session, we talk about how Amazon.com uses AWS technologies to build a scalable environment for data and analytics. We look at how Amazon is evolving the world of data warehousing with a combination of a data lake and parallel, scalable compute engines such as Amazon EMR and Amazon Redshift.
AWS platform has developed rapidly over the past few years through continuous iteration and innovations. In this session we provide a high level overview of the AWS platform and how customers leverage this to create highly available and scalable infrastructure. This session provides the required knowledge on how to get started with AWS.
Citrix Moves Data to Amazon Redshift Fast with Matillion ETLAmazon Web Services
Citrix moved large amounts of customer usage data to Amazon Redshift for analytics using Matillion ETL. Initially, Citrix built custom workflows to transform and load the data, but this required more maintenance. Using Matillion, Citrix can now load millions of rows into Redshift in minutes, allowing faster and more granular analysis of user data to optimize their applications. The speed and simplicity of Matillion has increased the efficiency of Citrix's analytics initiatives.
Eugene Kim takes us on a detailed overview of the AWS Cloud, and how SAP ERP workloads can be implemented. He discusses instance sizing in terms of SAPS, High Availability and Disaster Recovery scenarios. SAP Hana and certified solutions are presented as well.
Compared to storing long-term datasets on-premise, archiving in the cloud is a smart alternative whether you’re looking for an active archive solution, tape replacement, or to fulfill a compliance requirement. Learn how AWS customers are simplifying their archiving strategies and meeting compliance needs using Amazon Glacier.
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...Amazon Web Services
DigitalGlobe, Inc., the world’s leading provider of high-resolution Earth imagery, data, and analysis, is migrating its IT infrastructure, supporting imagery production and storage as well as satellite flight operations, to AWS with plans to close its commercial data centers within four years. DigitalGlobe has utilized AWS Snowmobile to move its 100PB image archive to the cloud. DigitalGlobe built its Geospatial Big Data platform, GBDX, natively on AWS. GBDX utilizes the image archive and combines geospatial big data and analytic tools, partner and customer data and tools, and dynamic cloud compute all in one place. This session will explore cost optimization for data management on AWS, highlighting various storage tiers and data import opportunities. We will focus on cost optimal usage of S3, S3-IA, Glacier, Snowball Edge and Snowmobile – balancing imagery access time with storage costs. Hear how DigitalGlobe utilized some of the newest features of the AWS platform to minimize their costs from storage. Learn More: https://aws.amazon.com/government-education/
This document summarizes a presentation on data lifecycle and storage management techniques for Amazon S3. It discusses lifecycle management rules for transitioning or expiring objects based on age, S3 inventory for listing objects, object tagging for classification and policy filtering, storage class analysis for monitoring usage and optimizing storage, and monitoring tools like CloudWatch and CloudTrail. The presentation provides an overview and best practices for these S3 management features.
Overview of AWS Services for Data Storage and Migration - SRV205 - Anaheim AW...Amazon Web Services
In this session, we explore the features and functions of AWS storage services. We provide context on the portfolio, and we cover the most common use cases for AWS offerings for object, file, block, and migration technologies, including the partner ecosystem. We then describe each service through customer case studies. Expect to leave this session understanding how to select a storage service and start moving workloads or building new ones.
An Overview of AWS Services for Data Storage and Migration - SRV205 - Atlanta...Amazon Web Services
In this session, we explore the features and functions of AWS storage services. We provide context on the AWS storage portfolio, and we cover the most common use cases for AWS offerings for object, file, block, and migration technologies, including the AWS Partner Network (APN) ecosystem. Then we examine each service, using customer case studies as examples. You gain an understanding of how to select storage and start moving workloads or building new ones.
"Wipro is one of India's largest publicly traded companies and the seventh largest IT services firm in the world. In this session, we showcase the structured methods that Wipro has used in enabling enterprises to take advantage of the cloud. These cover identifying workloads and application profiles that could benefit, re-structuring enterprise application and infrastructure components for migration, rapid and thorough verification and validation, and modifying component monitoring and management.
Several of these methods can be tailored to the individual client or functional context, so specific client examples are presented. We also discuss the enterprise experience of enabling many non-IT functions to benefit from the cloud, such as sales and training. More functions included in the cloud increase the benefit drawn from a cloud-enabled IT landscape.
Session sponsored by Wipro."
講師: Ivan Cheng, Solution Architect, AWS
Join us for a series of introductory and technical sessions on AWS Big Data solutions. Gain a thorough understanding of what Amazon Web Services offers across the big data lifecycle and learn architectural best practices for applying those solutions to your projects.
We will kick off this technical seminar in the morning with an introduction to the AWS Big Data platform, including a discussion of popular use cases and reference architectures. In the afternoon, we will deep dive into Machine Learning and Streaming Analytics. We will then walk everyone through building your first Big Data application with AWS.
This session provides a foundational overview of the AWS storage portfolio, including block, file, object, and cloud data migration services. This session will touch on the significant new offerings, outline some of the most common use cases and prepare you for the individual deep dive sessions, customer sessions and new announcements.
We will cover the core AWS storage services, which include Amazon Simple Storage Service (Amazon S3), Amazon Glacier, Amazon Elastic File System (Amazon EFS), and Amazon Elastic Block Store (Amazon EBS). We also discuss data transfer services such as AWS Snowball, Snowball Edge, and AWS Snowmobile, and hybrid storage solutions such as AWS Storage Gateway.
Building Serverless Web Applications - DevDay Los Angeles 2017Amazon Web Services
The document provides information about building serverless web applications using AWS Lambda and other AWS services. It begins with an overview of serverless computing using AWS Lambda and how it avoids the need to provision and manage servers. It then discusses various AWS compute offerings and when to use EC2, ECS, or Lambda. The rest of the document discusses serverless design patterns, demonstrates building a serverless web application using services like API Gateway and DynamoDB, and how to define and manage serverless applications using the AWS Serverless Application Model (SAM).
AWS Big Data and Analytics Services Speed Innovation | AWS Public Sector Summ...Amazon Web Services
Data-driven agencies face extreme data integration and analytics challenges. Decades of point solutions have solved specific mission problems while creating valuable data stores. However, these data stores are not integrated and are stored in information silos. AWS's powerful data ingestion and integration services now allow agencies to rapidly store more in data lakes for deeper analytics. Join this discussion on how FAA and other agencies have leveraged AWS data integration and analytic services to optimize and innovate with their previously untapped information silos. Learn More: https://aws.amazon.com/government-education/
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...Amazon Web Services
Amazon RDS allows you to launch an optimally configured, secure and highly available database with just a few clicks. It provides cost-efficient and resizable capacity, automates time-consuming database administration tasks, and provides you with six familiar database engines to choose from: Amazon Aurora, Oracle, Microsoft SQL Server, PostgreSQL, MySQL and MariaDB. In this session, we will take a close look at the capabilities of Amazon RDS and explain how it works. We’ll also discuss the AWS Database Migration Service and AWS Schema Conversion Tool, which help you migrate databases and data warehouses with minimal downtime from on-premises and cloud environments to Amazon RDS and other Amazon services. Gain your freedom from expensive, proprietary databases while providing your applications with the fast performance, scalability, high availability, and compatibility they need.
AWS offers numerous services to migrate data at a petabyte scale. You can easily move large volumes of data from onsite to the cloud and utilize the cloud as a backup target using data transfer services, such as AWS Snowball, AWS Snowball Edge, or AWS Storage Gateway. Learn about available data migration options and which one is the right fit for your requirements.
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMRAmazon Web Services
Customers are migrating their analytics, data processing (ETL), and data science workloads running on Apache Hadoop, Spark, and data warehouse appliances from on-premise deployments to Amazon EMR in order to save costs, increase availability, and improve performance. Amazon EMR is a managed service that lets you process and analyze extremely large data sets using the latest versions of over 15 open-source frameworks in the Apache Hadoop and Spark ecosystems. This session will focus on identifying the components and workflows in your current environment and providing the best practices to migrate these workloads to Amazon EMR. We will explain how to move from HDFS to Amazon S3 as a durable storage layer, and how to lower costs with Amazon EC2 Spot instances and Auto Scaling. Additionally, we will go over common security recommendations and tuning tips to accelerate the time to production.
Join us for an in-depth look at the current state of big data at AWS. Learn about the latest big data trends and industry use cases. Hear how other organizations are using the AWS big data platform to innovate and remain competitive. Take a look at some of the most recent AWS big data developments.
Backup and Recovery with Cloud-Native Deduplication and Use Cases from the Fi...Amazon Web Services
by Hugh Emberson, CTO, StorReduce
Designing and deploying cloud-enabled backup & recovery solutions often leads to opportunities for reducing storage requirements and increasing efficiencies. Having effective cloud-native deduplication capabilities as part of your backup & recovery strategy can optimize migration, decrease the need for purpose built backup appliances like Data Domains, large tape archives, and enable cost reductions of up to 95%. In this session, StorReduce will provide best practices around data deduplication in relation to designing and deploying solutions around backup, archive, and general unstructured file data. They will also demonstrate how using a cloud native interface with scale-out deduplication enables generic cloud services like search inside all backups moved to cloud. They will guide the audience through two customer use cases from the financial services and healthcare industries.
by PD Dutta, Sr. Product Manager, Object Storage, AWS
We will explain how to design and build an IoT cloud platform on top of Amazon S3. You will get to review the best practices for architecting a cost-effective, durable, and secure storage solution to store and analyze your IoT data on Amazon S3. In addition, we’ll cover how to collect, ingest and analyze the data in-place using different AWS Services such as AWS IoT, Amazon Kinesis, Amazon Athena, and Amazon Redshift Spectrum.
We will introduce key concepts for a data lake and present aspects related to its implementation. Also discussing critical success factors, pitfalls to avoid operational aspects, and insights on how AWS enables a server-less data lake architecture.
Speaker: Sebastien Menant, Solutions Architect, Amazon Web Services
A Look Under the Hood – How Amazon.com Uses AWS Services for Analytics at Mas...Amazon Web Services
Amazon’s consumer business continues to grow, and so does the volume of data and the number and complexity of the analytics done in support of the business. In this session, we talk about how Amazon.com uses AWS technologies to build a scalable environment for data and analytics. We look at how Amazon is evolving the world of data warehousing with a combination of a data lake and parallel, scalable compute engines such as Amazon EMR and Amazon Redshift.
AWS platform has developed rapidly over the past few years through continuous iteration and innovations. In this session we provide a high level overview of the AWS platform and how customers leverage this to create highly available and scalable infrastructure. This session provides the required knowledge on how to get started with AWS.
Citrix Moves Data to Amazon Redshift Fast with Matillion ETLAmazon Web Services
Citrix moved large amounts of customer usage data to Amazon Redshift for analytics using Matillion ETL. Initially, Citrix built custom workflows to transform and load the data, but this required more maintenance. Using Matillion, Citrix can now load millions of rows into Redshift in minutes, allowing faster and more granular analysis of user data to optimize their applications. The speed and simplicity of Matillion has increased the efficiency of Citrix's analytics initiatives.
Eugene Kim takes us on a detailed overview of the AWS Cloud, and how SAP ERP workloads can be implemented. He discusses instance sizing in terms of SAPS, High Availability and Disaster Recovery scenarios. SAP Hana and certified solutions are presented as well.
Compared to storing long-term datasets on-premise, archiving in the cloud is a smart alternative whether you’re looking for an active archive solution, tape replacement, or to fulfill a compliance requirement. Learn how AWS customers are simplifying their archiving strategies and meeting compliance needs using Amazon Glacier.
Optimizing Data Management Using AWS Storage and Data Migration Products | AW...Amazon Web Services
DigitalGlobe, Inc., the world’s leading provider of high-resolution Earth imagery, data, and analysis, is migrating its IT infrastructure, supporting imagery production and storage as well as satellite flight operations, to AWS with plans to close its commercial data centers within four years. DigitalGlobe has utilized AWS Snowmobile to move its 100PB image archive to the cloud. DigitalGlobe built its Geospatial Big Data platform, GBDX, natively on AWS. GBDX utilizes the image archive and combines geospatial big data and analytic tools, partner and customer data and tools, and dynamic cloud compute all in one place. This session will explore cost optimization for data management on AWS, highlighting various storage tiers and data import opportunities. We will focus on cost optimal usage of S3, S3-IA, Glacier, Snowball Edge and Snowmobile – balancing imagery access time with storage costs. Hear how DigitalGlobe utilized some of the newest features of the AWS platform to minimize their costs from storage. Learn More: https://aws.amazon.com/government-education/
This document summarizes a presentation on data lifecycle and storage management techniques for Amazon S3. It discusses lifecycle management rules for transitioning or expiring objects based on age, S3 inventory for listing objects, object tagging for classification and policy filtering, storage class analysis for monitoring usage and optimizing storage, and monitoring tools like CloudWatch and CloudTrail. The presentation provides an overview and best practices for these S3 management features.
Overview of AWS Services for Data Storage and Migration - SRV205 - Anaheim AW...Amazon Web Services
In this session, we explore the features and functions of AWS storage services. We provide context on the portfolio, and we cover the most common use cases for AWS offerings for object, file, block, and migration technologies, including the partner ecosystem. We then describe each service through customer case studies. Expect to leave this session understanding how to select a storage service and start moving workloads or building new ones.
An Overview of AWS Services for Data Storage and Migration - SRV205 - Atlanta...Amazon Web Services
In this session, we explore the features and functions of AWS storage services. We provide context on the AWS storage portfolio, and we cover the most common use cases for AWS offerings for object, file, block, and migration technologies, including the AWS Partner Network (APN) ecosystem. Then we examine each service, using customer case studies as examples. You gain an understanding of how to select storage and start moving workloads or building new ones.
This document provides an overview and summary of AWS storage services that can be used for migrating data to AWS. It discusses AWS Snowball and Snowmobile appliances that can physically move large amounts of data to AWS storage services like S3. It also describes the AWS Storage Gateway, which allows on-premises applications to access AWS storage using standard storage protocols. Additional services covered include Amazon Kinesis Firehose for loading streaming data, AWS Direct Connect for private connectivity, and AWS Migration Hub and Application Discovery Service for discovery and tracking of servers and databases during migration.
Data is gravity. Your workloads and processing is dependent on where your data is and how it is stored. With AWS, you have a host of storage options and the key to successfully leverage them is to know when to use which option. This session will explain in details about each of the AWS Storage offerings along with data ingestion optins into the Cloud using Snowball and Snowmobile
Marc Trimuschat,
Head - Business Developement, AWS Storage, AWS APAC
Eric Durand once again takes us to a journey of Storage solutions for digital media, using the AWS Cloud.
This presentation was delivered at AWS Toronto, during the Media and Entertainment Symposium.
Darry Osborne takes us on a journey across the AWS Cloud-based storage solutions. He explains S3, Glacier, Snowball and ends with Snowmobile, petabyte-scale data migration. He also talks about use cases, and customer stories. Presented in Montreal at the AWS Innovate show.
Storage is the most clear requirement for digital media. The AWS Cloud has customized solutions that cater to digital media storage, and present an array of options to ingest, store and move digital media, using the Cloud as a transport and storage mechanism.
Erik Durand, the Principal Business Development Manager for AWS Storage, takes us on this analysis of the options, benefits and characteristics of each one.
Presented during the AWS Media and Entertainment Symposium in Toronto
SRV403 Deep Dive on Object Storage: Amazon S3 and Amazon GlacierAmazon Web Services
In this session, storage experts will walk you through Amazon S3 and Amazon Glacier, bulk data repositories that can deliver 99.999999999% durability and scale past trillions of objects worldwide – with cost points competitive against tape archives. Learn about the different ways you can accelerate data transfer into S3 and get a close look at new tools to secure and manage your data more efficiently. See how Amazon Athena runs serverless analytics on your data and hear about expedited and bulk retrievals from Amazon Glacier. Learn how AWS customers have built solutions that turn their data from a cost into a strategic asset, and bring your toughest questions straight to our experts.
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...Amazon Web Services
Most organizations have data that they need to retain, but is accessed infrequently, if ever. In cases where this data needs to be accessible at a moment’s notice, it’s hard to save money by moving to an archival storage because access times on these platforms are slower. Now, customers are using Amazon S3 & Glacier for “Active Archiving” to reduce storage costs while maintaining the flexibility of instant access. In this tech talk, we’ll show you how implement Active Archiving with AWS Object Storage services, and we’ll provide some real world examples of how AWS customers are saving money with these capabilities today.
Learning Outcomes:
• Define Active Archiving, and understand how it is different from traditional cold archiving
• Review the cost modeling tools available to determine if Active Archiving is a good fit for your organization
• Learn about best practices for using AWS Object Storage features & functionality to enable Active Archiving
Introduction to Storage on AWS - AWS Summit Cape Town 2017Amazon Web Services
With AWS, you can choose the right storage service for the right use case. This session shows the range of AWS choices that are available to you: Amazon S3, Amazon EBS, Amazon EFS, Amazon Glacier and Cloud Data Migration solutions.
When evaluating and planning migrating your data from on premises to the Cloud, you might encounter physical limitations. Amazon offers a suite of tools to help you surmount these limitations by moving data using networks, roads, and technology partners. In this session, we discuss how to move large amounts of data into and out of the Cloud in batches, increments, and streams.
This document discusses various options for migrating large data sets to AWS, including AWS Snowball, Snowball Edge, Snowmobile, Storage Gateway, and S3 Transfer Acceleration. It provides an overview of each solution's key capabilities and features. Example use cases are given to illustrate how customers can use these tools to migrate data to AWS cost effectively and simplify ongoing data transfer processes.
This document summarizes various AWS storage services that can be used for migrating large data sets to the cloud, including Snowball, Snowball Edge, Snowmobile, Storage Gateway, Amazon S3 Transfer Acceleration, Amazon EFS over AWS Direct Connect, and Kinesis. It provides descriptions of each service and examples of how customers have used them for applications like active archive transport, hybrid storage solutions, data backup and disaster recovery, and streaming data ingestion and analysis. The document also discusses partner solutions that can help with storage tiering and migration by extending on-premises storage capabilities to AWS.
Matt Nowina, AWS Toronto Enterprise Solutions Architect, takes us on an overview of Cloud Storage solutions, data migration solutions and strategies, and practical examples of how to move data to the Cloud.
AWS Data Transfer Services - AWS Storage Gateway, AWS Snowball, AWS Snowball ...Amazon Web Services
AWS offers a suite of tools to help you surmount limitations associated to data migration from on premise to the cloud. Attend this session to learn about moving data by using networks, roads, and AWS technology partners. We will also discuss how to move data into and out of the Cloud in batches, increments, and streams.
AWS re:Invent 2016: Strategic Planning for Long-Term Data Archiving with Amaz...Amazon Web Services
Without careful planning, data management can quickly turn complex with a runaway cost structure. Enterprise customers are turning to the cloud to solve long-term data archive needs such as reliability, compliance, and agility while optimizing the overall cost. Come to this session and hear how AWS customers are using Amazon Glacier to simplify their archiving strategy. Learn how customers architect their cloud archiving applications and share integration to streamline their organization's data management and establish successful IT best practices.
Learn how AWS customers save money, time and effort by using AWS's backup and archive services. Organizations of all sizes rely on AWS services to durably safeguard their data off-premises at a surprisingly low cost. This session will illustrate backup and archive architectures that AWS customers are benefitting from today.
• Overview of AWS Storage Services including block, file, and object
• AWS data migration tools and approaches
• Description of AWS data migration programs aimed at accelerating your journey to the cloud
As the volume and types of data continues to grow, customers often have valuable data that is not easily discoverable and available for analytics. A common challenge for data engineering teams is architecting a data lake that can cater to the needs of diverse users - from developers to business analysts to data scientists. In this session, dive deep into building a data lake using Amazon S3, Amazon Kinesis, Amazon Athena and AWS Glue. Learn how AWS Glue crawlers can automatically discover your data, extracting and cataloguing relevant metadata to reduce operations in preparing your data for downstream consumers.
Similar to An Overview of AWS services for Data Storage and Migration - SRV205 - Toronto AWS Summit (20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS.
In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup.
Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti.
Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.
Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi
Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
L’utilizzo dei container è in continua crescita.
Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili.
I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!
In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th.
Event Agenda :
Open banking so far (short recap)
• PSD2, OB UK, OB Australia, OB LATAM, OB Israel
Intro to Open Finance marketplace
• Scope
• Features
• Tech overview and Demo
The role of the Cloud
The Future of APIs
• Complying with regulation
• Monetizing data / APIs
• Business models
• Time to market
One platform for all: a Strategic approach
Q&A
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc.
AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta.
Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity.
AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet.
Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.
Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.
Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti.
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti.
Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire.
Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito.
In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.
Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline.
Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.
1) The document discusses building a minimum viable product (MVP) using Amazon Web Services (AWS).
2) It provides an example of an MVP for an omni-channel messenger platform that was built from 2017 to connect ecommerce stores to customers via web chat, Facebook Messenger, WhatsApp, and other channels.
3) The founder discusses how they started with an MVP in 2017 with 200 ecommerce stores in Hong Kong and Taiwan, and have since expanded to over 5000 clients across Southeast Asia using AWS for scaling.
This document discusses pitch decks and fundraising materials. It explains that venture capitalists will typically spend only 3 minutes and 44 seconds reviewing a pitch deck. Therefore, the deck needs to tell a compelling story to grab their attention. It also provides tips on tailoring different types of decks for different purposes, such as creating a concise 1-2 page teaser, a presentation deck for pitching in-person, and a more detailed read-only or fundraising deck. The document stresses the importance of including key information like the problem, solution, product, traction, market size, plans, team, and ask.
This document discusses building serverless web applications using AWS services like API Gateway, Lambda, DynamoDB, S3 and Amplify. It provides an overview of each service and how they can work together to create a scalable, secure and cost-effective serverless application stack without having to manage servers or infrastructure. Key services covered include API Gateway for hosting APIs, Lambda for backend logic, DynamoDB for database needs, S3 for static content, and Amplify for frontend hosting and continuous deployment.
This document provides tips for fundraising from startup founders Roland Yau and Sze Lok Chan. It discusses generating competition to create urgency for investors, fundraising in parallel rather than sequentially, having a clear fundraising narrative focused on what you do and why it's compelling, and prioritizing relationships with people over firms. It also notes how the pandemic has changed fundraising, with examples of deals done virtually during this time. The tips emphasize being fully prepared before fundraising and cultivating connections with investors in advance.
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
This document discusses Amazon's machine learning services for building conversational interfaces and extracting insights from unstructured text and audio. It describes Amazon Lex for creating chatbots, Amazon Comprehend for natural language processing tasks like entity extraction and sentiment analysis, and how they can be used together for applications like intelligent call centers and content analysis. Pre-trained APIs simplify adding machine learning to apps without requiring ML expertise.
Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.
2. 163 ZB by 2025
- IDC
This is 10x the amount of data generated in 2016.
1 Zettabyte = 1021 = 1 000 000 000 000 000 000 000 bytes
Data growth is not slowing
3. Data as a flywheel for innovation
Deliver new insights
(data lakes, analytics)
Accelerate innovation
(active archive, IoT,
Artificial Intelligence)
Realize benefits
(cost, management, scale)
Build or migrate
an application
DATA
5. Compliance
Industry
certifications
Lockable with audit
trails
Secure
Enterprise
Applications
Easier lift-and-shift
migrations
Integrated with
major vendors
Fully managed
infrastructure
Active
Archive
Media workflows
Tape replacement
Public Sector, FinServ,
Healthcare/Life
Sciences
Databases &
Analytics
Tailored database or
Hadoop workloads
Bespoke database
lift-and-shift projects
Backup &
Restore
Non-disruptive
Easy place to start
Integrated w/all
major vendors
Data Lakes
& IoT
400% faster queries
Built for
streaming data
Optional data
visualization
Common storage workloads on AWS
6. The best reliability and
largest scale
The most complete
portfolio
The most data
movement choices
The most comprehensive
support and consulting
More than twice
the partners
The most secure,
compliant, and auditable
Why AWS Storage?
7. “…AWS has made considerable efforts to make S3 smarter for analytics workloads
by building query capabilities into the storage layer itself.”
- Gartner’s Critical Capabilities for Public Cloud Storage Services, Worldwide
Raj Bala, Julie Palmer, August 27, 2018
Amazon S3 holds trillions of objects and regularly peaks at millions
of requests per second.
TIME
OBJECTS
Storage with analytics capabilities
8. Enterprise ApplicationsDisaster Recovery Analytics
Primary Storage Backup & Restore Archive
Complete partner list at https://aws.amazon.com/backup-recovery/partner-solutions/
Twice as many partnerships
9. Data movement Data security
and management
AWS storage services
AWS Snow Family
AWS Storage Gateway
AWS Direct Connect
Amazon EFS File Sync
Amazon S3 Transfer
Acceleration
Third-party
Applications
Amazon Kinesis Firehose
AWS KMS
AWS IAM
Amazon CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
Amazon QuickSight
Amazon
EFS
Amazon
EBS
Amazon
S3
Amazon
Glacier
10. Getting started moving data
A private
connection
between your
data center,
office, or
colocation
environment and
AWS
AWS
Snow Family
(Snowball, Snowball
Edge, and
Snowmobile) Secure,
physical transport
appliances that can
pre-process and
move up to exabytes
of data into and out
of AWS
AWS
Storage
Gateways
Hybrid storage
that seamlessly
connects on-
premises
applications to
AWS storage.
Ideal for backup,
DR, bursting,
tiering, or
migration
Amazon
Kinesis
Firehose
Capture, trans-
form, & load
streaming data
into Amazon S3
for use with
Amazon business
intelligence and
analytics tools
Up to 5x faster file
transfers than
open-source tools.
Ideal for migrating
data into Amazon
EFS or moving
between cloud file
systems
Amazon
S3 Transfer
Acceleration
Up to 300% faster
transfers into and
out of Amazon S3.
Ideal when
working with long
geographic
distances
APN
Competency
Partners
Integrations
between third-
party vendors and
AWS services.
Ideal for
leveraging existing
software licenses
and skills
Amazon
EFS File Sync
Amazon
Direct
Connect
11. Amazon S3
Analyze
Store
Collect
Built for
• More than a decade of experience and continuous innovation
• Multiple storage classes and integrated lifecycle management
• Reporting on object metadata, compliance and usage with S3 Inventory
• Multiple storage ingestion options and partner integrations
• Multiple encryption options, security integrations and compliance
• Increase data access performance by up to 400% with S3 Select
• Query data in place with Amazon Athena and Amazon Redshift Spectrum
• Optimize storage utilization with S3 Analytics
Backup &
restore
Data lakes &
analytics
Cloud-native
applications
12. Amazon Glacier
Cost-effective
Secure
Durable
• Certifications supporting compliance requirements for virtually every
regulatory agency
• Locking, encryption, audit and alerting tools to prevent tampering
• Built on AWS’s global infrastructure
• Withstands multiple facility failures
• Replication options across global regions
• Designed for archives and backup
• Expedited retrievals in minutes, bulk retrievals in hours
• Opens archives to analytics applications with Glacier Select
Built for Active
archive
Tape
replacement
Regulatory
compliance
13. Object lifecycle management
S3 Standard GlacierS3 Standard -
Infrequent Access
Active data
Milliseconds
$0.023/GB/mo
Archive data
Minutes to hours
$0.004/GB/mo
Infrequently accessed data
Milliseconds
$0.0125/GB/mo
Automated Lifecycle Policies
S3 One Zone -
Infrequent Access
Infrequently accessed data
in just one AWS Region
Milliseconds
$0.01/GB/mo
14. Designed for
99.999999999% durability
Glacier
S3
Standard
S3 - IA
OR
99.999% durability
99.99% durability
Traditional model with two copies in one site
Traditional model with copies in two sites
S3 & Glacier data durability
S3 One Zone
- IA
15. “Zones”
Or worse, this:
AWS Region
This: Not this:
“Region”
Availability Zone
Availability Zone
Availability Zone
Object Storage Availability & Durability
16. AWS Storage Gateway family
Cost-effective
Hybrid
Cloud
Integrated
• Connects on-premises applications to AWS storage services
• Low-latency access with local caching and cloud scalability
• Optimized, fully managed data transfer between appliance and AWS
• Standard storage protocols for file, block & tape
• Stores data in Amazon S3, Amazon Glacier and EBS Snapshots
• AWS-native: Simplifying management, monitoring and automation
• Reduces on-premises storage and backup systems and management
• Unlocks cloud economics for data storage, processing and recovery
Built for Backup &
Archiving
File Storage for
Apps & Content
Hybrid
workloads
17. Canada’s largest biotech firm
Data sovereignty required local hot files
and tape archives in each of 10 global offices
• AWS Volume Gateway eliminated 50-hour
backup windows and tape archive systems
• Cut on-premises storage CAPEX 40%; dropped
RTO from 48 hours to 10 minutes
• Meets cloud strategy while retaining local
ownership and data sovereignty
• Enabled data center exit in next 12 months
“It made no sense to keep buying
big disk siloes, especially as we opened up
new global offices, and now we can
recover in the cloud from a snapshot if we
ever had to.”
- Adam Leggett
IT Manager
Hybrid Cloud Storage & Restore
18. On-premises infrastructure took weeks to produce
customer content
Needed performant, secure,
economical media distribution solution
• Workflow pipelines are now highly parallel and
elastic
• New one-hour content delivery SLA
• Fully migrating away from on-premises
infrastructure economics
Active Archive
“We have 20 petabytes of content on AWS, the
equivalent of more than 800,000 hours of video,
available on our platform. We can only move all
that content around the world with the
scalability we’re getting on the AWS Cloud. “
- Andy Shenkler
Chief Solutions and Technology Officer
19. Threat analysis company ingesting
and analyzing 50 TB daily
Right-sizing clusters cost weeks and lost data
• Saved 95% through re-architecting to a “hot” index
on Amazon EBS with an analytics data lake on
Amazon S3
• Amazon EBS shortened indexing times from weeks
to hours while cutting OPEX
• Now getting consistent 1–3 sec. search response
times across 5 PB of growing data in Amazon S3
• Managing 1 billion Amazon S3 objects and 2,500
instances with just six engineers
“AWS storage completely changed our business
operations, time to market and manpower. EBS
volumes cut our cluster indexing times from weeks to
hours. Moving data into Amazon S3 saved us 95% and
our data lake now outperforms our clusters—the
harder we push it the faster it gets for extremely large
datasets. We simply could not do this anywhere else.”
- Gene Stevens
CTO and Cofounder
A m a z o n S 3 D A T A L A K E
Data Lakes
20. Security-as-a-Service for 4000 customers
using 25 PB and growing 110% per year
Colocation not agile enough or cost effective
• Built an Amazon S3 data lake and avoided $1.6M
CAPEX - in the first year alone
• Stress-tested 100x larger load with zero CAPEX
• 4x better “I/O per $” ratio
• Gained new insights into their customers through
Amazon S3 data management capabilities
• No 40-Gbps network infrastructure worries
“AWS storage is
Fully redundant, multi-region,
more secure, and faster
at less than half the cost.”
+
- Paul Fisher
Technical Fellow
A m a z o n S 3 D A T A L A K E
Data Lakes
21. ProofPoint controls and enforces over 1M daily
social media posts for corporate customers
Needed to integrate this social media content
into their regulated email archive solution
• Built fully compliant archive/purge workflows
using Amazon S3 and Amazon Glacier
• Created a compliant two-step legal hold with
vault-level tags and Glacier Vault Lock
“What would it have cost us to build
a WORM data store,
get it certified for SEC Rule 17(a)-4(f)
and CFTC Rule 1.31 (b)-(c),
and then scale it?”
- Rich Sutton
VP of Engineering
Regulatory Compliance
22. Amazon EBS
Performant
Persistent
Reliable
• Dedicated, detachable volumes for EC2 instances
• Helps customers manage compute and storage separately
• Highly secure Multi-AZ design
• Built-in backup options
• Performance options to fit most workloads
• Optimized for latency, throughput, or cost
• Elastic volumes expand capacity on the fly
Built for
Hadoop/Amazon EMR,
relational and NoSQL databases,
log processing, and data warehousing
23. Databases and analytics
Global broadband service operator processing 17 TB
of daily device data streams at 200 MB/s
Modifying Kafka clusters required an
8 hours resync every time
• Moved from instance stores to EBS volumes
• Cut storage costs by 25%
• Cut production cluster node count by 33%
• Dropped resync times to 20 minutes
“Our AWS service use is about making the
necessary easy. Storage should be as boring
as possible—it should just work. Amazon
EBS makes it trivial to do things that were
impractical before, driving experimentation,
creativity, and faster delivery.”
- Daniel Woodlins
Software Engineer
24. Amazon EFS
Scalable
Simple
Elastic
Web serving, content management, media and
entertainment workflows, home directories, container
storage, big data, and analytics
• Share files between EC2 instances in minutes
• True file system interface with file system semantics
• Fully managed – no capacity planning surprises
• Pay-as-you-go consumption and pricing
• Automatically grows and shrinks
• Much lower TCO than DIY or third-party workarounds
• Consistent performance even as data grows
Built for
25. Newly acquired streaming media product depended
on a local file server
Had to launch at global scale in 90 days – with
minimal changes
• DIY was too complex and took too long
• Lift-and-shift to Amazon EFS took 2 hours
• EFS with EC2 autoscaling met global scale agility
needs
• Seamless integration between partner
application and existing AWS systems
• Post-mortem TCO analysis showed that EFS was
still the best choice
Enterprise applications
“Good, fast, and cheap. We picked two and got
all three with Amazon EFS. It gave us the agility
to deliver a new product on schedule, eliminated
scale and performance concerns, and operates
below our
OPEX expectations.”
- Chris DeAcosta
Sr. Director Software Engineering
26. Prior to Amazon EFS, we experienced timeouts for
up to 10% of uploads over 100 MB. Now, all of the
JFrog build artifacts (from infrastructure-as-code
components to Docker images) are in one place,
and we’ve increased large file transfer speeds by
38%.”
- Suresh Prem, Murty Chitti,
and Rajesh Sivaraman
System Engineers
Enterprise applications
Builds 3d digital maps relying on 28 TB of
waypoints generated daily
Unreliable on-premises repository and
high maintenance DIY cloud version
• Amazon EFS dropped infrastructure provisioning
time from 90 days to 7
• Now handling 800,000 daily file transfers up to
38% faster with zero failures
• Seamless JFrog workflow integration
• Gained high availability at no extra cost
• Also tiering JFrog backups into Amazon S3 and
Amazon Glacier
27. AWS Snow family
Cost-effective
Secure
Portable
• Certifications supporting nearly any regulatory compliance program
• Encryption, audit and alerting tools to flag tampering
• Ruggedized for capturing data in remote locations
• Optional computing power (Snowball Edge models)
• No networking modifications required
• Automatic return shipping labels
• Integrates with existing protocols and workflows
Built for Populating
Data Lakes
Media
Migration
Edge computing
28. “The AWS solution with Storage Gateway helps us
reduce backend service costs; our estimated costs for
this new hybrid cloud service are one tenth of the
prior on-premises infrastructure costs. As we pass
those savings on to our customers, we become
significantly more competitive in the energy-
exploration and production market.”
- William Rivera
Manager Of Global Cloud Operations
Landmark provides technology solutions for oil & gas
exploration and production
Clients use Landmark to store & manage TBs of data
for energy discovery research
• Initial bulk data transfer to Amazon S3 done with
AWS Snowball
• Created hybrid storage infrastructure with AWS
Storage Gateway – File Gateway
• Automated operations on the stored data with
AWS Lambda functions
• Able to query and organize data to according to
specific file structure
Hybrid Cloud Storage & Active Archive
29. Halliburton Landmark Use Case
Amazon
Glacier
Amazon S3
Standard
S3-Infrequent
Access
File Gateway
PetroBank
Application
ServersLTO
NAS
AWS Direct
Connect
Halliburton data center
1
2
Use AWS Snowball to ship data from on-premises offline archives1
2 Online access to all data through AWS Storage Gateway - File Gateway
Minimal on-premises storage reduces cost
Time-to-date by reduced by days or weeks
AWS Snowball
30. Compliance
Industry
certifications
Lockable with audit
trails
Secure
Enterprise
Applications
Easier lift-and-shift
migrations
Integrated with
major vendors
Fully managed
infrastructure
Active
Archive
Media workflows
Tape replacement
Public Sector, FinServ,
Healthcare/Life
Sciences
Databases &
Analytics
Tailored database or
Hadoop workloads
Bespoke database
lift-and-shift projects
Backup &
Restore
Non-disruptive
Easy place to start
Integrated w/all
major vendors
Data Lakes
& IoT
400% faster queries
Built for
streaming data
Optional data
visualization
Let’s get started: which is top of mind?