Your SlideShare is downloading. ×
  • Like
Managing terabytes on aws
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Managing terabytes on aws

  • 599 views
Published

JFrog is handling huge amount of binaries files for all our customers. Since each customer has its own space and domain, using a global Object Store can be tricky. …

JFrog is handling huge amount of binaries files for all our customers. Since each customer has its own space and domain, using a global Object Store can be tricky.
Also our application cannot work with an “eventually consistent” storage, and cannot deliver customer requirements with current S3 performance.
Learn in this session how we managed fast upload, critical replication and backups, and global download availability of the terabytes of JFrog customer binaries files.

Published in Technology , Sports
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
599
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. RequestStorage File Listing ChecksumStorage
  • 2. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB request
  • 3. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSrequestDB request
  • 4. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSrequestDB requestGET request Cache check, FSrequestDB + FS request
  • 5. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSrequestDB requestGET request Cache check, FSrequestDB + FS requestPUT request FS request DB + FS request
  • 6. EBS + LVM= WIN!
  • 7. /data250GB 250GB
  • 8. /data250GB 250GB 250GB 250GB
  • 9. /data250GB 250GB 250GB 250GB 250GB 250GB
  • 10. Backup files
  • 11. Backup filesBackup database
  • 12. Backup filesBackup databaseKeep them in sync!
  • 13. Backup filesBackup databaseKeep them in sync!
  • 14. Backup only deltas
  • 15. Backup only deltasManage threads toec2 efficiently
  • 16. Backup only deltasManage threads toec2 efficientlyKeep the dataisolated
  • 17. Backup only deltasManage threads toec2 efficientlyKeep the dataisolated
  • 18. CustomerEBSCustomerEBSCustomerEBS
  • 19. De-dupedEBSHA StorageCustomerEBSCustomerEBSCustomerEBS
  • 20. De-dupedGlacierDe-dupedEBSBackupHA StorageCustomerEBSCustomerEBSCustomerEBS
  • 21. Create empty Artifactory instance
  • 22. Create empty Artifactory instancePopulate from backstorage on demand
  • 23. Create empty Artifactory instancePopulate from backstorage on demandDrinkCosmopolitan
  • 24. Conclusions time1. Be lazy.
  • 25. Conclusions time1. Be lazy.2. Be smart.
  • 26. Conclusions time1. Be lazy.2. Be smart.3. Be agile.
  • 27. DrinkCosmopolitan