Title: Amazon Redshift Vs Google BigQuery Speaker: Kostas Pardalis (https://gr.linkedin.com/in/kostaspardalis/) Date: Wednesday, May 3, 2017 Event: https://meetup.com/Athens-Big-Data/events/239088154/

1. BigQuery
Redshift
VS

4. Amazon Redshift
Released on 2012 (beta)
based on ParAccel (PostgreSQL clone)
Designed for OLAP and BIapplications
Relationaland Columnardatabase
Petabyteto Exabytescale (Spectrum)

5. Google BigQuery
evolution of Dremel(2006)
Initially launched in 2010
WebService on top of Dremel Technology
More of a hybridsystem (columnar + nested data)
Petabytescale

6. Amazon Redshift Google BigQuery
Build on top of a proven technology
Relational
SQL
Analysts
Build something from scratch
Nested data structures are a first class citizen
NoSQL
Developers
VS

7. Loading Data

8. Amazon Redshift Google BigQuery
S3
Kinesis
CSV, Avro, JSON
Google Cloud Storage
Streaming Inserts
Google Analytics Premium
CSV, Avro, JSON
VS

9. Data Modeling

10. Amazon Redshift Google BigQuery
Schemas
Tables
Datasets
Tables
VS

11. Data Types

12. Data Types
Redshift:Closer to the StandardSQLdatatypes (e.g. INT4,
INT8) but doesnotsupport the full range of PostgreSQL
data types
BigQuery: Smaller set of data types supported. But...

13. Data Types
Redshift:Very basic support for JSON
BigQuery: Support for Array and STRUCT types. Nesteddata
structures are first class citizens.

14. Working with
Data

15. Data Manipulation
BigQuery used to be appendonly, now it supportsUpdates
andDeletes (DML). But still limited.
Redshift always had this Supported via SQL but with a
catch (Vacuum)

16. Table Manipulation
BigQuery: Limited and expensive via standard SQL, or via
HTTP API (but you have to unload and reload the table).
Redshift: Supported via SQL
Both support views but not materialized

17. Data Consistency

18. Data Consistency
Redshift supports transactions VS BigQuery No
Deduplication harder to be achieved on BigQuery (costly
also).
Even more complex when we go streaming.

19. Data Consistency
Kinesis: At least once semantics
BigQuery: best-effort deduplication time window +
insertId.

20. Cluster
Management

21. Cluster Management
Here is where BigQueryreallyshines. It is fullymanaged with
supportforHA.
Redshift doesnotabstract completely the hardware from the
user and it is difficult to implement it as a HA service.
This changes with Spectrum.

22. Cluster
Management
Connectivity

23. Connectivity
Redshift:API+ FullJDBC/ODBCsupport.
& access to all the standardPostgreSQL tools.
BigQuery: Mainly through the RESTAPI, JDBC/ODBC drivers
only for queries.

24. Authentication
Redshift:AWSIAM
BigQuery: OAuth

25. Cluster
Management
Quotas

26. Amazon Redshift Google BigQuery
Resources capped by your cluster size
No quotas related to inserts/updates
etc
2,000 slots per account
Encourages the append only model with
strict DML quotas
Both have a limit of 50 concurrent Queries
Cluster resizing a pain with Redshift.

27. Cluster
Management
Optimization

28. Optimization
Redshift:DistributionKeys(Partitioning),SortKeys,Column
Compression
BigQuery: You don’t have to worry about all these, but it
allows you to define time based partitioning.

29. Optimization
Redshift:DistributionKeys(Partitioning),SortKeys,Column
Compression
With great power comes great responsibility: Vacuuming

30. Optimization
With BigQueryyou can only optimize your queries on the
statement level and partition over time.

31. Cluster
Management
Costs

32. Costs
BigQuery:more difficulttoestimatecosts but costs scale
based on usage. $5 / TB (on demand)
Redshift: you know exactlyhow much you will pay but
you payregardlessofusage.

33. Ecosystem

35. Performance

36. Performance
BenchmarkbyPanoply
DebunkingMisleadingBenchmarksOfRedshiftvsBigQuery
HNDiscussion
BenchmarkbyPeriscopeData

37. BigQuery
Redshift
OR
??

38. Amazon Redshift Google BigQueryVS
similarities are greater than the differences

39. Amazon Redshift Google BigQuery
More predictable costs
More intuitive data modeling (Analysts)
Options for optimizations
Easier & cheaper to start with
Good for nested data
Easier to work with time series
VS

40. Thanks!kostas@blendo.co
www.blendo.co
Get the free Amazon Redshift guide
http://bit.ly/redshift-guide