Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Alluxio: The missing piece of on-demand clusters at Alluxio Meetup 2016

606 views

Published on

Presented by Calvin Jia, Alluxio
Alluxio Meetup at Samsung
http://www.meetup.com/Alluxio/

Published in: Technology
  • Be the first to comment

Alluxio: The missing piece of on-demand clusters at Alluxio Meetup 2016

  1. 1. The Missing Piece of On-Demand Clusters September 2016 @ Samsung Calvin Jia
  2. 2. About Me •  Calvin Jia •  Software Engineer @ Alluxio, Inc. •  Alluxio PMC •  #1 Alluxio Contributor •  Twitter: @JiaCalvin 2
  3. 3. Alluxio Inc. •  Founded by Alluxio creators and top committers •  Formerly Tachyon Nexus, Inc. •  $7.5 million Series A by Andreessen Horowitz •  Committed to the Alluxio Open Source Project •  Company Website: http://www.alluxio.com •  We are hiring! 3  
  4. 4. Outline •  Cloud Architectures •  Alluxio in Cloud Architectures •  Takeaways 4  
  5. 5. Cloud Architectures – Overview •  Mostly service based, from providers –  Amazon Web Services –  Google Cloud Platform •  Separate compute and storage clusters •  Compute clusters are ephemeral 5  
  6. 6. Cloud Architectures – Pros & Cons Pros •  Low maintenance •  Pay as you go •  Elastic and scalable •  Cost effective storage Cons •  Lower Performance 6  
  7. 7. Outline •  Cloud Architectures •  Alluxio in Cloud Architectures •  Takeaways 7  
  8. 8. Alluxio in Cloud Architectures – Overview •  Deployed in compute clusters –  Memory speed data access –  Transparent data access to any storage •  Simple to deploy –  Mount storage systems similar to local disks 8  
  9. 9. Alluxio in Cloud Architectures – Example 9  
  10. 10. Alluxio in Cloud Architectures – Benefits 10   •  Remedies the performance drawback •  Acceleration due to memory-speed I/O •  Designed to improve the affinity of compute and storage
  11. 11. Alluxio in Cloud Architectures – Data Path 11   FAST 104 - 105 MB/s MODERATE 103 - 104 MB/s SLOW 102 - 103 MB/s Only when necessary Limited Often SSD   HDD   Mem  
  12. 12. Takeaways – Experiment Results 12   0   100   200   300   400   500   600   Ini/al  Read   Subsequent  Read   Read  from   Separate  Job   Run+me  (seconds)   Spark  -­‐  No  Persist   Spark  -­‐  Persist   Alluxio  
  13. 13. Takeaways – Alluxio & Cloud Architectures 13   •  Cloud architectures have significant upsides •  Alluxio alleviates the major downsides
  14. 14. Try out Alluxio 1.2.0 http://www.alluxio.org/releases 14  
  15. 15. Read the blog at http://www.alluxio.com/2016/08/accelerating-on-demand-data-analytics-with-alluxio Read the whitepaper at www.alluxio.com. Check out www.alluxio.org. Thank you! 15

×