SlideShare a Scribd company logo
AWS Users’ Group Updates
David “Mac” McDaniel
Sr. Solution & Cloud Architect - Independent Consultant
david@mobile-360.com
LinkedIn: https://www.linkedin.com/in/davidbmcdaniel
Twitter: @CloudKegGuy, @ServerlessJava
Twitter list: https://twitter.com/CloudKegGuy/lists/aws/members
Getting Connected
Slack Channel: https://DenverAWSUsersGroup.slack.com
You will need an invitation to join, please email me: david@mobile-360.com.
We are now listed on AWS UG site:
https://aws.amazon.com/usergroups/americas/
We are sponsored by CloudAcademy! They have a free portal for our members at:
https://cloudacademy.com/aws-usergroup/?code=newawsugs
We are also sponsored and a member of the official Global AWS Communities!
See them at https://awsug.support
What we’re going to do tonight
1. Describe Amazon Redshift
2. Talk about how it’s different from regular SQL Databases
3. Talk about storage options for Redshift
a. Standard Disk-based storage
b. Spectrum and S3 (CSV & Parquet) storage
4. Describe ways to load data
a. S3, EMR, DynamoDB or Remote Hosts
5. Compare to Athena
What is Redshift?
Amazon Redshift is a fast, fully managed data warehouse that makes it simple and cost-effective to analyze
all your data using standard SQL and your existing Business Intelligence (BI) tools. It allows you to run
complex analytic queries against petabytes of structured data, using sophisticated query optimization,
columnar storage on high-performance local disks, and massively parallel query execution. Most results come
back in seconds. With Amazon Redshift, you can start small for just $0.25 per hour with no commitments and
scale out to petabytes of data for $1,000 per terabyte per year, less than a tenth the cost of traditional
solutions.
Amazon Redshift also includes Redshift Spectrum, allowing you to directly run SQL queries against exabytes
of unstructured data in Amazon S3. No loading or transformation is required, and you can use open data
formats, including CSV, TSV, Parquet, Sequence, and RCFile. Redshift Spectrum automatically scales query
compute capacity based on the data being retrieved, so queries against Amazon S3 run fast, regardless of
dataset size.
Recently announced 4x compression improvement in Redshift.
How Redshift is Different
Redshift is a column-oriented database whereas regular SQL databases are row-oriented in nature. This
means that Redshift stores groups of columns together rather than groups of rows. This can be hugely
beneficial when processing many rows, but only a few columns, which is typical in BI and Analytical
processing. Many data warehouse databases will be denormalized to reduce joins and therefore tables
will be very wide (many columns) to provide the most value, even though individual queries will only use a
small number of columns.
Columnar vs. Row Oriented
Storage Options
1. Local Disk Storage
a. Traditional, SSD-based, ties storage to compute.
b. Ties compute to storage.
c. Must make FULL read-only copies to scale.
2. S3 - Used with Redshift Spectrum
a. Uses Amazon Athena Meta-data to understand files in S3.
b. Decouples storage from compute.
c. Still must make read-only copies, but of meta-data only, so smaller & faster to scale.
How do we load data?
Multiple ways:
1. Preferred way: Use COPY command to load data from files in one of many
formats from:
a. S3
b. EMR
c. Remote EC2 Hosts
d. DynamoDB Tables
2. Use DML:
How is it different from Athena?
Athena Redshift
Storage on S3 Storage on attached SSD disks
Automatically scales Must add more instances/change instance
size
Massive parallelism Only as parallel as you configure
Data can be stored in multiple formats per
table
Data can be loaded from files in multiple
formats
Demo!
1. Create Schemas for Redshift tables
2. Load data in multiple formats from S3
3. Create Redshift Spectrum Schemas
4. Load data (really, meta-data)
5. Execute queries
6. Tableau visualization
Next Month’s
TOPIC:
????
We need speakers!
Chipe: Cheesy, Sarcastic
Questions?

More Related Content

What's hot

Amazon Web Services: Lessons for Architecting Data in the Cloud
Amazon Web Services: Lessons for Architecting Data in the CloudAmazon Web Services: Lessons for Architecting Data in the Cloud
Amazon Web Services: Lessons for Architecting Data in the Cloud
Safe Software
 
AWS SSA Webinar 21 - Getting Started with Data lakes on AWS
AWS SSA Webinar 21 - Getting Started with Data lakes on AWSAWS SSA Webinar 21 - Getting Started with Data lakes on AWS
AWS SSA Webinar 21 - Getting Started with Data lakes on AWS
Cobus Bernard
 
Getting Started with Amazon EMR
Getting Started with Amazon EMRGetting Started with Amazon EMR
Getting Started with Amazon EMR
Arman Iman
 
How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013
How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013
How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013
Amazon Web Services
 
Unit1 dbms
Unit1 dbmsUnit1 dbms
Unit1 dbms
gowrivageesan87
 
AWS EMR (Elastic Map Reduce) explained
AWS EMR (Elastic Map Reduce) explainedAWS EMR (Elastic Map Reduce) explained
AWS EMR (Elastic Map Reduce) explained
Harsha KM
 
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWSAWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
Cobus Bernard
 
Hadoop in the cloud with AWS' EMR
Hadoop in the cloud with AWS' EMRHadoop in the cloud with AWS' EMR
Hadoop in the cloud with AWS' EMR
rICh morrow
 
Interactively Querying Large-scale Datasets on Amazon S3
Interactively Querying Large-scale Datasets on Amazon S3Interactively Querying Large-scale Datasets on Amazon S3
Interactively Querying Large-scale Datasets on Amazon S3
Amazon Web Services
 
AWS RDS
AWS RDSAWS RDS
AWS RDS
Mahesh Raj
 
Rethinking the database for the cloud (iJAWS)
Rethinking the database for the cloud (iJAWS)Rethinking the database for the cloud (iJAWS)
Rethinking the database for the cloud (iJAWS)Rasmus Ekman
 
Best Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon RedshiftBest Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon Redshift
Amazon Web Services
 
AWS Tutorial-Part2:Exam Intro-2018
AWS Tutorial-Part2:Exam Intro-2018AWS Tutorial-Part2:Exam Intro-2018
AWS Tutorial-Part2:Exam Intro-2018
SaM theCloudGuy
 
Introduction to Amazon Athena
Introduction to Amazon AthenaIntroduction to Amazon Athena
Introduction to Amazon Athena
Amazon Web Services
 
Announcing Amazon Athena - Instantly Analyze Your Data in S3 Using SQL
Announcing Amazon Athena - Instantly Analyze Your Data in S3 Using SQLAnnouncing Amazon Athena - Instantly Analyze Your Data in S3 Using SQL
Announcing Amazon Athena - Instantly Analyze Your Data in S3 Using SQL
Amazon Web Services
 
Aws Atlanta meetup Amazon Athena
Aws Atlanta meetup Amazon AthenaAws Atlanta meetup Amazon Athena
Aws Atlanta meetup Amazon Athena
Adam Book
 
Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...
Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...
Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...
Amazon Web Services
 
Deep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduceDeep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduce
Amazon Web Services
 
AWS RDS Migration Tool
AWS RDS Migration Tool AWS RDS Migration Tool
Amazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration ServiceAmazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration Service
Amazon Web Services
 

What's hot (20)

Amazon Web Services: Lessons for Architecting Data in the Cloud
Amazon Web Services: Lessons for Architecting Data in the CloudAmazon Web Services: Lessons for Architecting Data in the Cloud
Amazon Web Services: Lessons for Architecting Data in the Cloud
 
AWS SSA Webinar 21 - Getting Started with Data lakes on AWS
AWS SSA Webinar 21 - Getting Started with Data lakes on AWSAWS SSA Webinar 21 - Getting Started with Data lakes on AWS
AWS SSA Webinar 21 - Getting Started with Data lakes on AWS
 
Getting Started with Amazon EMR
Getting Started with Amazon EMRGetting Started with Amazon EMR
Getting Started with Amazon EMR
 
How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013
How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013
How Amazon.com is Leveraging Amazon Redshift (DAT306) | AWS re:Invent 2013
 
Unit1 dbms
Unit1 dbmsUnit1 dbms
Unit1 dbms
 
AWS EMR (Elastic Map Reduce) explained
AWS EMR (Elastic Map Reduce) explainedAWS EMR (Elastic Map Reduce) explained
AWS EMR (Elastic Map Reduce) explained
 
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWSAWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
 
Hadoop in the cloud with AWS' EMR
Hadoop in the cloud with AWS' EMRHadoop in the cloud with AWS' EMR
Hadoop in the cloud with AWS' EMR
 
Interactively Querying Large-scale Datasets on Amazon S3
Interactively Querying Large-scale Datasets on Amazon S3Interactively Querying Large-scale Datasets on Amazon S3
Interactively Querying Large-scale Datasets on Amazon S3
 
AWS RDS
AWS RDSAWS RDS
AWS RDS
 
Rethinking the database for the cloud (iJAWS)
Rethinking the database for the cloud (iJAWS)Rethinking the database for the cloud (iJAWS)
Rethinking the database for the cloud (iJAWS)
 
Best Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon RedshiftBest Practices for Migrating your Data Warehouse to Amazon Redshift
Best Practices for Migrating your Data Warehouse to Amazon Redshift
 
AWS Tutorial-Part2:Exam Intro-2018
AWS Tutorial-Part2:Exam Intro-2018AWS Tutorial-Part2:Exam Intro-2018
AWS Tutorial-Part2:Exam Intro-2018
 
Introduction to Amazon Athena
Introduction to Amazon AthenaIntroduction to Amazon Athena
Introduction to Amazon Athena
 
Announcing Amazon Athena - Instantly Analyze Your Data in S3 Using SQL
Announcing Amazon Athena - Instantly Analyze Your Data in S3 Using SQLAnnouncing Amazon Athena - Instantly Analyze Your Data in S3 Using SQL
Announcing Amazon Athena - Instantly Analyze Your Data in S3 Using SQL
 
Aws Atlanta meetup Amazon Athena
Aws Atlanta meetup Amazon AthenaAws Atlanta meetup Amazon Athena
Aws Atlanta meetup Amazon Athena
 
Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...
Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...
Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...
 
Deep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduceDeep Dive: Amazon Elastic MapReduce
Deep Dive: Amazon Elastic MapReduce
 
AWS RDS Migration Tool
AWS RDS Migration Tool AWS RDS Migration Tool
AWS RDS Migration Tool
 
Amazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration ServiceAmazon Aurora and AWS Database Migration Service
Amazon Aurora and AWS Database Migration Service
 

Similar to July 2017 Meeting of the Denver AWS Users' Group

AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18
Neal Davis
 
Building Data Lakes in the AWS Cloud
Building Data Lakes in the AWS CloudBuilding Data Lakes in the AWS Cloud
Building Data Lakes in the AWS Cloud
Amazon Web Services
 
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Precisely
 
Redshift
RedshiftRedshift
Redshift
OmarFaroque16
 
AWS Big Data Landscape
AWS Big Data LandscapeAWS Big Data Landscape
AWS Big Data Landscape
Crishantha Nanayakkara
 
Data warehousing in the era of Big Data: Deep Dive into Amazon Redshift
Data warehousing in the era of Big Data: Deep Dive into Amazon RedshiftData warehousing in the era of Big Data: Deep Dive into Amazon Redshift
Data warehousing in the era of Big Data: Deep Dive into Amazon Redshift
Amazon Web Services
 
Owning Your Own (Data) Lake House
Owning Your Own (Data) Lake HouseOwning Your Own (Data) Lake House
Owning Your Own (Data) Lake House
Data Con LA
 
2017 AWS DB Day | AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?
2017 AWS DB Day |  AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?2017 AWS DB Day |  AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?
2017 AWS DB Day | AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?
Amazon Web Services Korea
 
Module 2 - Datalake
Module 2 - DatalakeModule 2 - Datalake
Module 2 - Datalake
Lam Le
 
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
Amazon Web Services
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Amazon Web Services
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Amazon Web Services
 
Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Amazon Web Services
 
Aplicaciones a gran escala: Cómo servir a millones de usuarios
Aplicaciones a gran escala: Cómo servir a millones de usuariosAplicaciones a gran escala: Cómo servir a millones de usuarios
Aplicaciones a gran escala: Cómo servir a millones de usuarios
Amazon Web Services
 
AWS Webcast - Tableau Big Data Solution Showcase
AWS Webcast - Tableau Big Data Solution ShowcaseAWS Webcast - Tableau Big Data Solution Showcase
AWS Webcast - Tableau Big Data Solution Showcase
Amazon Web Services
 
Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2Amazon Web Services
 
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Amazon Web Services
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Amazon Web Services
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Ian Massingham
 
Amazon-Redshift-dBT-Best-Practices_paper.pdf
Amazon-Redshift-dBT-Best-Practices_paper.pdfAmazon-Redshift-dBT-Best-Practices_paper.pdf
Amazon-Redshift-dBT-Best-Practices_paper.pdf
Hoang CHi THang
 

Similar to July 2017 Meeting of the Denver AWS Users' Group (20)

AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18
 
Building Data Lakes in the AWS Cloud
Building Data Lakes in the AWS CloudBuilding Data Lakes in the AWS Cloud
Building Data Lakes in the AWS Cloud
 
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...
 
Redshift
RedshiftRedshift
Redshift
 
AWS Big Data Landscape
AWS Big Data LandscapeAWS Big Data Landscape
AWS Big Data Landscape
 
Data warehousing in the era of Big Data: Deep Dive into Amazon Redshift
Data warehousing in the era of Big Data: Deep Dive into Amazon RedshiftData warehousing in the era of Big Data: Deep Dive into Amazon Redshift
Data warehousing in the era of Big Data: Deep Dive into Amazon Redshift
 
Owning Your Own (Data) Lake House
Owning Your Own (Data) Lake HouseOwning Your Own (Data) Lake House
Owning Your Own (Data) Lake House
 
2017 AWS DB Day | AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?
2017 AWS DB Day |  AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?2017 AWS DB Day |  AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?
2017 AWS DB Day | AWS 데이터베이스 개요 - 나의 업무에 적합한 데이터베이스는?
 
Module 2 - Datalake
Module 2 - DatalakeModule 2 - Datalake
Module 2 - Datalake
 
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
(BDT322) How Redfin & Twitter Leverage Amazon S3 For Big Data
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
 
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech TalksMigrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
Migrating Your Oracle Database to PostgreSQL - AWS Online Tech Talks
 
Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
Migrating your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...
 
Aplicaciones a gran escala: Cómo servir a millones de usuarios
Aplicaciones a gran escala: Cómo servir a millones de usuariosAplicaciones a gran escala: Cómo servir a millones de usuarios
Aplicaciones a gran escala: Cómo servir a millones de usuarios
 
AWS Webcast - Tableau Big Data Solution Showcase
AWS Webcast - Tableau Big Data Solution ShowcaseAWS Webcast - Tableau Big Data Solution Showcase
AWS Webcast - Tableau Big Data Solution Showcase
 
Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud - DevDay Austin 2017 Day 2
 
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit Dublin
 
Scaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit DublinScaling on AWS for the First 10 Million Users at Websummit Dublin
Scaling on AWS for the First 10 Million Users at Websummit Dublin
 
Amazon-Redshift-dBT-Best-Practices_paper.pdf
Amazon-Redshift-dBT-Best-Practices_paper.pdfAmazon-Redshift-dBT-Best-Practices_paper.pdf
Amazon-Redshift-dBT-Best-Practices_paper.pdf
 

More from David McDaniel

Denver AWS Users' Group Meetup - May 2020
Denver AWS Users' Group Meetup - May 2020Denver AWS Users' Group Meetup - May 2020
Denver AWS Users' Group Meetup - May 2020
David McDaniel
 
January 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' Group
January 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' GroupJanuary 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' Group
January 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' Group
David McDaniel
 
Denver AWS Meetup - March 2019 slides
Denver AWS Meetup - March 2019 slidesDenver AWS Meetup - March 2019 slides
Denver AWS Meetup - March 2019 slides
David McDaniel
 
Denver AWS Meetup - February 2019
Denver AWS Meetup - February 2019Denver AWS Meetup - February 2019
Denver AWS Meetup - February 2019
David McDaniel
 
Denver AWS Users' Group Meetup - October 2018
Denver AWS Users' Group Meetup - October 2018Denver AWS Users' Group Meetup - October 2018
Denver AWS Users' Group Meetup - October 2018
David McDaniel
 
Denver AWS Meetup -- August 2018
Denver AWS Meetup -- August 2018Denver AWS Meetup -- August 2018
Denver AWS Meetup -- August 2018
David McDaniel
 
Denver AWS Users' Group Meeting - July 2018 Slides - Cloud Optimization
Denver AWS Users' Group Meeting - July 2018 Slides - Cloud OptimizationDenver AWS Users' Group Meeting - July 2018 Slides - Cloud Optimization
Denver AWS Users' Group Meeting - July 2018 Slides - Cloud Optimization
David McDaniel
 
Denver AWS Users' Group Meeting - July 2018 Slides
Denver AWS Users' Group Meeting - July 2018 SlidesDenver AWS Users' Group Meeting - July 2018 Slides
Denver AWS Users' Group Meeting - July 2018 Slides
David McDaniel
 
Denver AWS Users' Group Meeting - May 2018 Slides
Denver AWS Users' Group Meeting - May 2018 SlidesDenver AWS Users' Group Meeting - May 2018 Slides
Denver AWS Users' Group Meeting - May 2018 Slides
David McDaniel
 
Denver AWS Users' Group meeting - September 2017
Denver AWS Users' Group meeting - September 2017Denver AWS Users' Group meeting - September 2017
Denver AWS Users' Group meeting - September 2017
David McDaniel
 
June 2017 Denver AWS Users' Group intro slides
June 2017 Denver AWS Users' Group intro slidesJune 2017 Denver AWS Users' Group intro slides
June 2017 Denver AWS Users' Group intro slides
David McDaniel
 
DevOps on AWS
DevOps on AWSDevOps on AWS
DevOps on AWS
David McDaniel
 
May 2017
May 2017May 2017
May 2017
David McDaniel
 
January 2017 - Deep dive on AWS Lambda and DevOps
January 2017 - Deep dive on AWS Lambda and DevOpsJanuary 2017 - Deep dive on AWS Lambda and DevOps
January 2017 - Deep dive on AWS Lambda and DevOps
David McDaniel
 
October 2016
October 2016October 2016
October 2016
David McDaniel
 

More from David McDaniel (15)

Denver AWS Users' Group Meetup - May 2020
Denver AWS Users' Group Meetup - May 2020Denver AWS Users' Group Meetup - May 2020
Denver AWS Users' Group Meetup - May 2020
 
January 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' Group
January 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' GroupJanuary 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' Group
January 2020 - re:Invent reCap slides - Denver Amazon Web Services Users' Group
 
Denver AWS Meetup - March 2019 slides
Denver AWS Meetup - March 2019 slidesDenver AWS Meetup - March 2019 slides
Denver AWS Meetup - March 2019 slides
 
Denver AWS Meetup - February 2019
Denver AWS Meetup - February 2019Denver AWS Meetup - February 2019
Denver AWS Meetup - February 2019
 
Denver AWS Users' Group Meetup - October 2018
Denver AWS Users' Group Meetup - October 2018Denver AWS Users' Group Meetup - October 2018
Denver AWS Users' Group Meetup - October 2018
 
Denver AWS Meetup -- August 2018
Denver AWS Meetup -- August 2018Denver AWS Meetup -- August 2018
Denver AWS Meetup -- August 2018
 
Denver AWS Users' Group Meeting - July 2018 Slides - Cloud Optimization
Denver AWS Users' Group Meeting - July 2018 Slides - Cloud OptimizationDenver AWS Users' Group Meeting - July 2018 Slides - Cloud Optimization
Denver AWS Users' Group Meeting - July 2018 Slides - Cloud Optimization
 
Denver AWS Users' Group Meeting - July 2018 Slides
Denver AWS Users' Group Meeting - July 2018 SlidesDenver AWS Users' Group Meeting - July 2018 Slides
Denver AWS Users' Group Meeting - July 2018 Slides
 
Denver AWS Users' Group Meeting - May 2018 Slides
Denver AWS Users' Group Meeting - May 2018 SlidesDenver AWS Users' Group Meeting - May 2018 Slides
Denver AWS Users' Group Meeting - May 2018 Slides
 
Denver AWS Users' Group meeting - September 2017
Denver AWS Users' Group meeting - September 2017Denver AWS Users' Group meeting - September 2017
Denver AWS Users' Group meeting - September 2017
 
June 2017 Denver AWS Users' Group intro slides
June 2017 Denver AWS Users' Group intro slidesJune 2017 Denver AWS Users' Group intro slides
June 2017 Denver AWS Users' Group intro slides
 
DevOps on AWS
DevOps on AWSDevOps on AWS
DevOps on AWS
 
May 2017
May 2017May 2017
May 2017
 
January 2017 - Deep dive on AWS Lambda and DevOps
January 2017 - Deep dive on AWS Lambda and DevOpsJanuary 2017 - Deep dive on AWS Lambda and DevOps
January 2017 - Deep dive on AWS Lambda and DevOps
 
October 2016
October 2016October 2016
October 2016
 

Recently uploaded

Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using SmithyGenerating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using Smithy
g2nightmarescribd
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 

Recently uploaded (20)

Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Generating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using SmithyGenerating a custom Ruby SDK for your web service or Rails API using Smithy
Generating a custom Ruby SDK for your web service or Rails API using Smithy
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 

July 2017 Meeting of the Denver AWS Users' Group

  • 1. AWS Users’ Group Updates David “Mac” McDaniel Sr. Solution & Cloud Architect - Independent Consultant david@mobile-360.com LinkedIn: https://www.linkedin.com/in/davidbmcdaniel Twitter: @CloudKegGuy, @ServerlessJava Twitter list: https://twitter.com/CloudKegGuy/lists/aws/members
  • 2. Getting Connected Slack Channel: https://DenverAWSUsersGroup.slack.com You will need an invitation to join, please email me: david@mobile-360.com. We are now listed on AWS UG site: https://aws.amazon.com/usergroups/americas/ We are sponsored by CloudAcademy! They have a free portal for our members at: https://cloudacademy.com/aws-usergroup/?code=newawsugs We are also sponsored and a member of the official Global AWS Communities! See them at https://awsug.support
  • 3. What we’re going to do tonight 1. Describe Amazon Redshift 2. Talk about how it’s different from regular SQL Databases 3. Talk about storage options for Redshift a. Standard Disk-based storage b. Spectrum and S3 (CSV & Parquet) storage 4. Describe ways to load data a. S3, EMR, DynamoDB or Remote Hosts 5. Compare to Athena
  • 4. What is Redshift? Amazon Redshift is a fast, fully managed data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and your existing Business Intelligence (BI) tools. It allows you to run complex analytic queries against petabytes of structured data, using sophisticated query optimization, columnar storage on high-performance local disks, and massively parallel query execution. Most results come back in seconds. With Amazon Redshift, you can start small for just $0.25 per hour with no commitments and scale out to petabytes of data for $1,000 per terabyte per year, less than a tenth the cost of traditional solutions. Amazon Redshift also includes Redshift Spectrum, allowing you to directly run SQL queries against exabytes of unstructured data in Amazon S3. No loading or transformation is required, and you can use open data formats, including CSV, TSV, Parquet, Sequence, and RCFile. Redshift Spectrum automatically scales query compute capacity based on the data being retrieved, so queries against Amazon S3 run fast, regardless of dataset size. Recently announced 4x compression improvement in Redshift.
  • 5. How Redshift is Different Redshift is a column-oriented database whereas regular SQL databases are row-oriented in nature. This means that Redshift stores groups of columns together rather than groups of rows. This can be hugely beneficial when processing many rows, but only a few columns, which is typical in BI and Analytical processing. Many data warehouse databases will be denormalized to reduce joins and therefore tables will be very wide (many columns) to provide the most value, even though individual queries will only use a small number of columns.
  • 6. Columnar vs. Row Oriented
  • 7. Storage Options 1. Local Disk Storage a. Traditional, SSD-based, ties storage to compute. b. Ties compute to storage. c. Must make FULL read-only copies to scale. 2. S3 - Used with Redshift Spectrum a. Uses Amazon Athena Meta-data to understand files in S3. b. Decouples storage from compute. c. Still must make read-only copies, but of meta-data only, so smaller & faster to scale.
  • 8. How do we load data? Multiple ways: 1. Preferred way: Use COPY command to load data from files in one of many formats from: a. S3 b. EMR c. Remote EC2 Hosts d. DynamoDB Tables 2. Use DML:
  • 9. How is it different from Athena? Athena Redshift Storage on S3 Storage on attached SSD disks Automatically scales Must add more instances/change instance size Massive parallelism Only as parallel as you configure Data can be stored in multiple formats per table Data can be loaded from files in multiple formats
  • 10. Demo! 1. Create Schemas for Redshift tables 2. Load data in multiple formats from S3 3. Create Redshift Spectrum Schemas 4. Load data (really, meta-data) 5. Execute queries 6. Tableau visualization
  • 11. Next Month’s TOPIC: ???? We need speakers! Chipe: Cheesy, Sarcastic