• Like
  • Save
Introduction to Elastic MapReduce
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Introduction to Elastic MapReduce

  • 3,579 views
Published

An introduction to Elastic MapReduce, including a demonstration of how to create a pre-configured, scalable Hadoop cluster in minutes.

An introduction to Elastic MapReduce, including a demonstration of how to create a pre-configured, scalable Hadoop cluster in minutes.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,579
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
62
Comments
0
Likes
5

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Elastic Map Reduce Matt Wood T E C H N O L O G Y E VA N G E L I S T
  • 2. Hello.
  • 3. Thank you.
  • 4. 3
  • 5. 1Building blocks
  • 6. Infrastructure services
  • 7. 5 years young
  • 8. ?
  • 9. On demand
  • 10. Pay as you go
  • 11. Pay for what you use
  • 12. Elastic capacity
  • 13. Capacity Estimated demand Time
  • 14. Capacity Infrastructure Investment Estimated demand Time
  • 15. Capacity Infrastructure Real demand Time
  • 16. Capacity Elastic capacity Real demand Time
  • 17. Undifferentiated heavy lifting
  • 18. Focus on your stuff
  • 19. Idea Product
  • 20. Idea Product Heavy lifting
  • 21. Idea Product VERY Heavy lifting
  • 22. Idea Product
  • 23. Scalable storageScalable compute Scalable tools
  • 24. 2Enter the Cloud
  • 25. S3 Scalable storage Scalable compute Scalable tools EC2
  • 26. Elastic Map Reduce
  • 27. Hosted Hadoop
  • 28. Without the ‘muck’
  • 29. S3Input data
  • 30. S3 Input dataCode Elastic MapReduce
  • 31. S3 Input dataCode Elastic Name MapReduce node
  • 32. S3 Input dataCode Elastic Name MapReduce node Elastic cluster
  • 33. S3 Input dataCode Elastic Name MapReduce node HDFS Elastic cluster
  • 34. S3 Input dataCode Elastic Name MapReduce node Queries HDFS + BI Via JDBC, Pig, Hive Elastic cluster
  • 35. S3 Input dataCode Elastic Name Output MapReduce node S3 + SimpleDB Queries HDFS + BI Via JDBC, Pig, Hive Elastic cluster
  • 36. S3 Input data Elastic OutputMapReduce S3 + SimpleDB
  • 37. It’s all just Hadoop
  • 38. HDFS + S3
  • 39. Hive, Pig,Cascading,Streaming
  • 40. API driven
  • 41. Data movement
  • 42. Import/Export
  • 43. Multipart upload
  • 44. Multipart,parallel results delivery
  • 45. Scale control
  • 46. Resize running job flows
  • 47. 14 hoursTime remaining: 14 hours
  • 48. 14 hoursTime remaining: 7 hours
  • 49. Time remaining: 3 hours
  • 50. Balance cost and performance
  • 51. Resize based on usage patterns
  • 52. Steady state Steady state Batch processing
  • 53. Cluster types
  • 54. Small
  • 55. High memory High CPU or both
  • 56. HPC
  • 57. Nehalem Quad core HPC 10 gig E GPU
  • 58. Access control
  • 59. Private
  • 60. Location
  • 61. Identity and Access
  • 62. 3EMR ByExample
  • 63. Bioinformatics Web indexing Financial modellingFile processing Data mining and BI Data warehousing Fraud detectionTargeted advertising
  • 64. Click stream analysis for Best Buy 3.5 billion records 71 million unique cookies 1.7 million targeted ads 13 Tb of clickstream logs Each day
  • 65. Click stream analysis for Best Buy Workflow time from 2 days to 8 hoursProcurement time from 2 months to 5 minutes $13k per month500% increase return on advertising spend
  • 66. Web log analysis and recommendation engine $29.9 million in sales 842 million page views 434 Gb of page logs 97 million ‘favourites’
  • 67. Elastic Map Reduce
  • 68. Undifferentiated heavy lifting
  • 69. Managed Hadoop
  • 70. Hive, Pig, Cascading
  • 71. Data movement
  • 72. Scale control
  • 73. HPC instances
  • 74. aws.amazon.com
  • 75. Thank you!
  • 76. Q U E S T I O N S + C O M M E N T Smatthew@amazon.com @mza O N T W I T T E R