Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Amazon S3 Integration


Published on

A preview of the FileCatalyst Direct 3.5 focusing on the Amazon S3 cloud integration

Published in: Software
  • Be the first to comment

  • Be the first to like this

Amazon S3 Integration

  1. 1. Webinar: FileCatalyst 3.5 Preview - Amazon S3 Integration March 19th, 2015
  2. 2. © Copyright FileCatalyst, 2015 Presenter Chris Bailey, CEO and Co-Founder FileCatalyst / Unlimi-Tech
  3. 3. © Copyright FileCatalyst, 2015 Agenda 1. What is Amazon S3 2. Architecture 3. Demo 4. Ways to Integrate 5. Ways to Connect 6. What’s Next?
  4. 4. © Copyright FileCatalyst, 2015 What is Amazon S3 Amazon S3 (Simple Storage Service) is an online file storage web service offered by Amazon Web Services. ● High bandwidth (but hard to utilize) ● It scales on demand and is much more durable than local storage ● Capex vs Opex Lots of options to transfer files into S3 ● File system drivers (map/mount a drive) ● FTP-like clients ● Web interfaces All these options have one thing in common, they use HTTP to transfer across the WAN. Slow transfer over high bandwidth connections or where latency is present (i.e. always) Many of these solutions cache files locally and copy them to S3 in the background to create the illusion of speed. Files are not accessible right away.
  5. 5. © Copyright FileCatalyst, 2015 We have integrated Amazon’s SDK into a file system driver that plugs into FileCatalyst using Java NIO.2 FileCatalyst is able to treat the S3 storage as a file system, so all functions are available: - Support resume, MD5 verification, etc… - Files are streamed direct to S3, nothing lands locally Architecture FileCatalys t Server S3Java NIO Amazon SDK EC2 Instance WANFIleCatalyst Client UDP Acceleration Multipart HTTP Acceleration Amazon Web Services Higher Latency Lower latency
  6. 6. © Copyright FileCatalyst, 2015 Demo
  7. 7. © Copyright FileCatalyst, 2015 Ways to integrate S3 buckets/folders can be assigned in numerous ways: ● Map a User home directory to a bucket or a sub-folder within a bucket ● Create a Virtual Folder that points to a bucket or a sub-folder within a bucket ● Create a Virtual File that points to a single object. Virtual Files/Folders may be assigned to a User or a Group and appear off the root of the user’s home directory Using this approach, a user could gain access to one or more files/folders on S3 along with local files.
  8. 8. © Copyright FileCatalyst, 2015 Ways to Connect
  9. 9. © Copyright FileCatalyst, 2015 Whats next? Finalize and push performance ● Highly dependent on instance type and region ● Uses HTTPS, which is CPU intensive. Recommend compute optimized for high speeds ● Use of “Enhanced Networking” available on R3, C3, I2 instance types ● Higher packet per second (PPS) performance ● Requires use of Amazon VPC Integration with FileCatalyst HotFolder ● Automated transfer from S3 to a remote server location ● Migration of files between cloud storage (i.e. Azure to S3 or vise versa) Integrations of more File Systems and Object stores ● Encrypted File System ● Microsoft Azure, OpenStack Swift, Google Cloud Storage ● Any File System that implements NIO.2 (make your own, open source, etc...)
  10. 10. © Copyright FileCatalyst, 2015 Questions?