RequestStorage File Listing ChecksumStorage
RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB request
RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSre...
RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSre...
RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSre...
EBS + LVM= WIN!
/data250GB 250GB
/data250GB 250GB 250GB 250GB
/data250GB 250GB 250GB 250GB 250GB 250GB
Backup files
Backup filesBackup database
Backup filesBackup databaseKeep them in sync!
Backup filesBackup databaseKeep them in sync!
Backup only deltas
Backup only deltasManage threads toec2 efficiently
Backup only deltasManage threads toec2 efficientlyKeep the dataisolated
Backup only deltasManage threads toec2 efficientlyKeep the dataisolated
CustomerEBSCustomerEBSCustomerEBS
De-dupedEBSHA StorageCustomerEBSCustomerEBSCustomerEBS
De-dupedGlacierDe-dupedEBSBackupHA StorageCustomerEBSCustomerEBSCustomerEBS
Create empty Artifactory instance
Create empty Artifactory instancePopulate from backstorage on demand
Create empty Artifactory instancePopulate from backstorage on demandDrinkCosmopolitan
Conclusions time1. Be lazy.
Conclusions time1. Be lazy.2. Be smart.
Conclusions time1. Be lazy.2. Be smart.3. Be agile.
DrinkCosmopolitan
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Managing terabytes on aws
Upcoming SlideShare
Loading in...5
×

Managing terabytes on aws

662

Published on

JFrog is handling huge amount of binaries files for all our customers. Since each customer has its own space and domain, using a global Object Store can be tricky.
Also our application cannot work with an “eventually consistent” storage, and cannot deliver customer requirements with current S3 performance.
Learn in this session how we managed fast upload, critical replication and backups, and global download availability of the terabytes of JFrog customer binaries files.

Published in: Technology, Sports
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
662
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Managing terabytes on aws

  1. 1. RequestStorage File Listing ChecksumStorage
  2. 2. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB request
  3. 3. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSrequestDB request
  4. 4. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSrequestDB requestGET request Cache check, FSrequestDB + FS request
  5. 5. RequestStorage File Listing ChecksumStorageMetadatarequestsCache check, FSrequestDB requestHEAD requests Cache check, FSrequestDB requestGET request Cache check, FSrequestDB + FS requestPUT request FS request DB + FS request
  6. 6. EBS + LVM= WIN!
  7. 7. /data250GB 250GB
  8. 8. /data250GB 250GB 250GB 250GB
  9. 9. /data250GB 250GB 250GB 250GB 250GB 250GB
  10. 10. Backup files
  11. 11. Backup filesBackup database
  12. 12. Backup filesBackup databaseKeep them in sync!
  13. 13. Backup filesBackup databaseKeep them in sync!
  14. 14. Backup only deltas
  15. 15. Backup only deltasManage threads toec2 efficiently
  16. 16. Backup only deltasManage threads toec2 efficientlyKeep the dataisolated
  17. 17. Backup only deltasManage threads toec2 efficientlyKeep the dataisolated
  18. 18. CustomerEBSCustomerEBSCustomerEBS
  19. 19. De-dupedEBSHA StorageCustomerEBSCustomerEBSCustomerEBS
  20. 20. De-dupedGlacierDe-dupedEBSBackupHA StorageCustomerEBSCustomerEBSCustomerEBS
  21. 21. Create empty Artifactory instance
  22. 22. Create empty Artifactory instancePopulate from backstorage on demand
  23. 23. Create empty Artifactory instancePopulate from backstorage on demandDrinkCosmopolitan
  24. 24. Conclusions time1. Be lazy.
  25. 25. Conclusions time1. Be lazy.2. Be smart.
  26. 26. Conclusions time1. Be lazy.2. Be smart.3. Be agile.
  27. 27. DrinkCosmopolitan
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×