A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013

1,219 views

Published on

If you've ever developed code for processing data, you know what a mess it can be—especially on Hadoop. You lack debugging tools, instant feedback, automated tests, and a sane deploy. Mortar has developed a modern framework for data processing on Hadoop and Amazon Elastic MapReduce. It is a free, open framework providing instant, step-by-step execution visibility, automated testing, reusable components, and one-button deployment. See how Mortar demonstrates this framework on Amazon EMR on a sample data set to solve a big data problem.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,219
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

A Modern Framework for Amazon Elastic MapReduce (BDT309) | AWS re:Invent 2013

  1. 1. A Modern Framework for EMR Mortar Data: K Young (CEO), Jeremy Karn (Lead Engineer) November 15, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Friday, November 15, 13
  2. 2. K Young Friday, November 15, 13 Jeremy Karn
  3. 3. This talk is technical Friday, November 15, 13
  4. 4. Great: Hadoop Greater: Amazon EMR Friday, November 15, 13
  5. 5. A Modern Framework Friday, November 15, 13
  6. 6. A Modern Framework Friday, November 15, 13
  7. 7. Goal: 10x productivity • Collaboration • Efficient, Free Development • Testing / Debug • Reproducibility • And More... Friday, November 15, 13
  8. 8. For data science (not a database) Friday, November 15, 13
  9. 9. Iterate Locally Deploy Friday, November 15, 13 Cloud Execution EMR + Mortar
  10. 10. Demo Find the most popular projects on GitHub Friday, November 15, 13
  11. 11. Goal: Collaboration • Easily run code from others • Contribute changes back Friday, November 15, 13
  12. 12. Goal: Efficient, Free Development • Rapid iteration • No cost • Resilient to errors Friday, November 15, 13
  13. 13. Goal: Testing & Debug • Automated testing • See what is happening at runtime Friday, November 15, 13
  14. 14. Goal: Reproducibility • Know what was run • 1-button deploy • Rollback Friday, November 15, 13
  15. 15. Goal: Miscellaneous goals • Easy scheduling • Easily use other technologies: Python, Amazon DynamoDB, MongoDB • Results easy to locate • Managed cluster lifetime • More granular API Friday, November 15, 13
  16. 16. A Modern Framework for Amazon EMR What you saw • Collaboration • Efficient, Free Development • Testing / Debug • Reproducibility • And More... Friday, November 15, 13
  17. 17. Next steps • bit.ly/mortar-reinvent • Documentation: help.mortardata.com • Follow: @mortardata Friday, November 15, 13
  18. 18. Please give us your feedback on this presentation BDT309 As a thank you, we will select prize winners daily for completed surveys! Friday, November 15, 13 Thank You

×