Your SlideShare is downloading. ×
WebHDFS at King - May 2014 Hadoop MeetUp
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

WebHDFS at King - May 2014 Hadoop MeetUp

373
views

Published on

The latest developments at King on their work with WebHDFS .

The latest developments at King on their work with WebHDFS .

Published in: Technology, Sports

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
373
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 2 How to turbo charge your data transfers with WebHDFS Andy Done, Data Platform Lead andy.done@king.com
  • 2. Last time…
  • 3. Since then…
  • 4. 100 40 Hadoop
  • 5. 1 0.5 Storage
  • 6. 15 10 Events
  • 7. 10 4 ExaSol
  • 8. 2.5 6 Load times
  • 9. Problem WebHDFS 12
  • 10. Old way WebHDFS
  • 11. Old way hadoop fs –cat /some/path/* | bulk_load my_table WebHDFS
  • 12. WebHDFS way WebHDFS
  • 13. WebHDFS way IMPORT INTO TABLE my_table FROM FILE ‘http://namenode/webhdfs/v1/some/path/file_1’ FILE ‘http://namenode/webhdfs/v1/some/path/file_2’ … FILE ‘http://namenode/webhdfs/v1/some/path/file_n’ WebHDFS
  • 14. WebHDFS benefits •  Simple •  Efficient •  Ubiquitous •  Parallelisable •  Bidirectional •  Fast WebHDFS
  • 15. 18 Conclusion WebHDFS
  • 16. Thank you 19
  • 17. We're hiring! 20