Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MapReduce is History:
Deep Integration of Tez with YARN's
Timeline Server
P R E S E N T E D B Y J o n a t h a n E a g l e ...
Introduction
https://flic.kr/p/rmJkUB
https://flic.kr/p/7PeBhM
https://flic.kr/p/9F5bop
Formation of a Team
The team
7 Yahoo Confidential & Proprietary
 Hitesh Shah – Tez Core Expert
 Prakash Ramachandran – Tez Core / Tez UI
 R...
Design with a purpose
8 Yahoo Confidential & Proprietary
 Create a plan
 Get feedback
 Iterate
Ready Set Deploy
Rollout Phases
10 Yahoo Confidential & Proprietary
 Backend
 Tez UI
 Application Enablement
Rollout methodology
11 Yahoo Confidential & Proprietary
https://flic.kr/p/bJuv9k
Demo
Demo
Demo
Demo
Are we there yet?
16 Yahoo Confidential & Proprietary
 Yes and No.
› Hive and Pig
 What will it take to get a yes
› It’s...
Getting Technical
17 Yahoo Confidential & Proprietary
 What is the architecture.
› REST API
› Events
› Types
CORS
18 Yahoo Confidential & Proprietary
 Cross-Origin Resource Sharing.
› Ember
• Remote REST API
• Local Javascript
› E...
Even more technical
Data Design
Data Design
21 Yahoo Confidential & Proprietary
 Small keys are keys
 Consider compression
 Consider Sparse Counters
 ...
Data Design (cont.)
22 Yahoo Confidential & Proprietary
 Use Relations Sparingly
 Use Primary Keys Sparingly
 Promote A...
https://flic.kr/p/cqgshw
Future
Questions.
Upcoming SlideShare
Loading in …5
×

MapReduce is History: Deep Integration of Tez with YARN's Timeline Server

844 views

Published on

Published in: Technology
  • Login to see the comments

MapReduce is History: Deep Integration of Tez with YARN's Timeline Server

  1. 1. MapReduce is History: Deep Integration of Tez with YARN's Timeline Server P R E S E N T E D B Y J o n a t h a n E a g l e s ⎪ J u n e 9 , 2 0 1 5
  2. 2. Introduction
  3. 3. https://flic.kr/p/rmJkUB
  4. 4. https://flic.kr/p/7PeBhM
  5. 5. https://flic.kr/p/9F5bop
  6. 6. Formation of a Team
  7. 7. The team 7 Yahoo Confidential & Proprietary  Hitesh Shah – Tez Core Expert  Prakash Ramachandran – Tez Core / Tez UI  Ram Venkatesh – Team lead and UX  Sreenath Somarajapuram – Tez UI Expert  Myself - YARN developer / Tez developer  And more
  8. 8. Design with a purpose 8 Yahoo Confidential & Proprietary  Create a plan  Get feedback  Iterate
  9. 9. Ready Set Deploy
  10. 10. Rollout Phases 10 Yahoo Confidential & Proprietary  Backend  Tez UI  Application Enablement
  11. 11. Rollout methodology 11 Yahoo Confidential & Proprietary https://flic.kr/p/bJuv9k
  12. 12. Demo
  13. 13. Demo
  14. 14. Demo
  15. 15. Demo
  16. 16. Are we there yet? 16 Yahoo Confidential & Proprietary  Yes and No. › Hive and Pig  What will it take to get a yes › It’s technical
  17. 17. Getting Technical 17 Yahoo Confidential & Proprietary  What is the architecture. › REST API › Events › Types
  18. 18. CORS 18 Yahoo Confidential & Proprietary  Cross-Origin Resource Sharing. › Ember • Remote REST API • Local Javascript › Events › Types
  19. 19. Even more technical
  20. 20. Data Design
  21. 21. Data Design 21 Yahoo Confidential & Proprietary  Small keys are keys  Consider compression  Consider Sparse Counters  Consider a Dictionary of Counters
  22. 22. Data Design (cont.) 22 Yahoo Confidential & Proprietary  Use Relations Sparingly  Use Primary Keys Sparingly  Promote Aggregate Values › Lessons computation › Enables artificial-paging
  23. 23. https://flic.kr/p/cqgshw
  24. 24. Future
  25. 25. Questions.

×