BDT303 Data Science with Elastic MapReduce - AWS re: Invent 2012
Report
Amazon Web ServicesFollow
Dec. 4, 2012•0 likes•3,054 views
BDT303 Data Science with Elastic MapReduce - AWS re: Invent 2012
Dec. 4, 2012•0 likes•3,054 views
Download to read offline
Report
In this talk, we dive into the Netflix Data Science & Engineering architecture. Not just the what, but also the why. Some key topics include the big data technologies we leverage (Cassandra, Hadoop, Pig + Python, and Hive), our use of Amazon S3 as our central data hub, our use of multiple persistent Amazon Elastic MapReduce (EMR) clusters, how we leverage the elasticity of AWS, our data science as a service approach, how we make our hybrid AWS / data center setup work well, and more.
Amazon Web ServicesFollow
Recommended
(BDT403) Netflix's Next Generation Big Data Platform | AWS re:Invent 2014Amazon Web Services
Data Science at Netflix with Amazon EMR (BDT306) | AWS re:Invent 2013Amazon Web Services
Next Generation Big Data Platform at Netflix 2014Eva Tse
(BDT303) Running Spark and Presto on the Netflix Big Data PlatformAmazon Web Services
Presto @ Netflix: Interactive Queries at Petabyte ScaleDataWorks Summit
SF Big Analytics: Machine Learning with Presto by Christopher BernerChester Chen
More Related Content
What's hot
AWS re:Invent 2016: How to Scale and Operate Elasticsearch on AWS (DEV307)Amazon Web Services
AWS re:Invent 2016: How Citus Enables Scalable PostgreSQL on AWS (DAT207)Amazon Web Services
(BDT210) Building Scalable Big Data Solutions: Intel & AOLAmazon Web Services
Scaling Traffic from 0 to 139 Million Unique VisitorsYelp Engineering
Analyzing IOT Data in Apache Spark Across Data Centers and Cloud with NetApp ...Databricks
Analytics at Scale with Apache Spark on AWS with Jonathan FritzDatabricks
Viewers also liked
RMG202 Rainmakers: How Netflix Operates Clouds for Maximum Freedom and Agilit...Amazon Web Services
ARC203 Highly Available Architecture at Netflix - AWS re: Invent 2012Amazon Web Services
AWS Summit Auckland 2014 | Desktops in the Cloud Amazon Web Services
CPN211 My Datacenter Has Walls That Move - AWS re: Invent 2012Amazon Web Services
CPN301 The Best Amazon EC2 Features You Never Knew About - AWS re: Invent 2012Amazon Web Services
AWS Partner Presentation-Sonian-AWS Cloud Storage for the Enterprise 2012Amazon Web Services
Similar to BDT303 Data Science with Elastic MapReduce - AWS re: Invent 2012
Deep Dive on Amazon S3Adrian Hornsby
2017 AWS DB Day | Amazon Athena 서비스 최신 기능 소개 Amazon Web Services Korea
Getting Buzzed on Buzzwords: Using Cloud & Big Data to Pentest at ScaleBishop Fox
AWS re:Invent 2016: Workshop: AWS S3 Deep-Dive Hands-On Workshop: Deploying a...Amazon Web Services
Running Presto and Spark on the Netflix Big Data PlatformEva Tse
AWS re:Invent 2016: Case Study: How Startups Like Smartsheet and Quantcast Ac...Amazon Web Services
More from Amazon Web Services
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
Costruire Applicazioni Moderne con AWSAmazon Web Services
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
Open banking as a serviceAmazon Web Services
BDT303 Data Science with Elastic MapReduce - AWS re: Invent 2012
- 2. What is Netflix’s data warehouse? a) Cassandra b) Teradata c) Hive d) S3
- 4. DSE Platform
- 7. DSE Platform Chukwa S3
- 9. Aegisthus
- 11. DSE Platform Chukwa Aegisthus S3
- 12. Sting
- 15. DSE Platform Sting Chukwa Aegisthus S3
- 16. What is Netflix’s data warehouse? a) Cassandra b) Teradata c) Hive d) S3
- 17. DSE Platform Sting Chukwa Aegisthus S3
- 18. S3
- 19. S3
- 20. 99.999999999%
- 22. S3
- 23. High SLA S3 Query
- 24. HDFS ?
- 26. “Data Science as a Service” • Execution Service / Genie • Event Service • Metadata Service
- 32. High SLA Cluster Job High SLA S3 Query Cluster Job Query
- 33. High SLA S3 Query Cluster Job Query
- 34. High SLA Cluster Job High SLA S3 Query Cluster Job Query
- 35. Super SLA Cluster Job Super SLA S3 High SLA Cluster Job High SLA Query Cluster Job Query
- 36. Super SLA Cluster Job High SLA Cluster Job High SLA S3 Query Cluster Job Query
- 38. Questions? http://jobs.netflix.com kurtbrown@netflix.com