Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Technical Overview        Bill Branan   DuraCloud Technical Lead
Agenda•   Introduction•   Storage•   Services•   Reporting•   UI•   Security•   Tools•   Roadmap
Introduction• DuraCloud is:  – Hosted service     • Runs on cloud-based compute systems     • Connects to cloud-based stor...
DuraCloud in ContextUser Data Center                                            Data                            File Syste...
DuraCloud Storage                     Administrative User Interface (DurAdmin)  Storage Management          Service Manage...
DuraCloud Storage                           User Data Center                                 Amazon Storage      Amazon S3...
Storage REST Interface         Space Actions                  Content Actions•   Add Space                   •   Add Conte...
Storage AdaptersTransform calls from DuraCloud interface intocalls understood by a specific storage provider• Determine ho...
DuraCloud Services                     Administrative User Interface (DurAdmin)  Storage Management          Service Manag...
DuraCloud Services                            Media                           Streamer             Duplicate              ...
DuraCloud Service Deployment           Services           Service Management          Available?             (DuraService)...
DuraCloud Service Deployment               Deploy                     Service Management               Service            ...
Service REST Interface•   Get Services•   Deploy Service•   Get (Deployed) Service•   Get Deployed Service Properties•   U...
DuraCloud                                                   DuraCloud        Instance                                     ...
DuraCloud Bulk Service Deployment                              Service Management          1                      (DuraSer...
DuraCloud Reporting                     Administrative User Interface (DurAdmin)  Storage Management          Service Mana...
DuraCloud Report Generation                              Report Management                                 (DuraReport)   ...
Report REST Interface       Storage Report                 Service Report•   Start Storage Report        • Get Deployed Se...
DuraCloud UI                     Administrative User Interface (DurAdmin)  Storage Management          Service Management ...
DuraCloud UI Interaction                Administrative User Interface (DurAdmin)         Store Client        Service Clien...
DuraCloud Security                                                   DurAdmin   Storage   Provider                        ...
DuraCloud Tools  Transfer to DuraCloud           Transfer from DuraCloud• Upload Tool                   • Retrieval Tool  ...
DuraCloud ToolsUser Data Center             Store Client                               Store Client        Chunk Tool     ...
Near Term Road MapNew features• Access control at the space level• Improved format identification• Automated bit integrity...
Long Term Road MapNew Features• Video transcoding• Document format transformation• Improved image support services• Media ...
Questions?
Upcoming DuraCloud Webinars    DSpace and DuraCloud     November 30, 2011 at 1pm ET    Fedora and DuraCloud      January 1...
Thank You!  Further Questions?  info@duracloud.org
Upcoming SlideShare
Loading in …5
×

A Technical Overview of DuraCloud

3,381 views

Published on

Interested in learning more about the technical architecture of DuraCloud? Curious about how it runs in the cloud? This presentation details the technical underpinnings of DuraCloud.
Presented by:
Bill Branan, DuraCloud Lead Developer, DuraSpace

Published in: Technology, Business
  • Be the first to comment

A Technical Overview of DuraCloud

  1. 1. A Technical Overview Bill Branan DuraCloud Technical Lead
  2. 2. Agenda• Introduction• Storage• Services• Reporting• UI• Security• Tools• Roadmap
  3. 3. Introduction• DuraCloud is: – Hosted service • Runs on cloud-based compute systems • Connects to cloud-based storage systems • Provides a service platform – Open source software suite• Webinar Focus: – Components of software platform – Interacting with hosted DuraCloud
  4. 4. DuraCloud in ContextUser Data Center Data File System Repository DuraCloud Management Console • Account Management • Instance Management
  5. 5. DuraCloud Storage Administrative User Interface (DurAdmin) Storage Management Service Management Report Management (DuraStore) (DuraService) (DuraReport) REST API REST API REST API Service ContainerDuraCloud Instance
  6. 6. DuraCloud Storage User Data Center Amazon Storage Amazon S3 Adapter REST API Rackspace Rackspace Storage Storage Adapter Cloud Files Provider Storage Interface Mediation Azure Storage Microsoft Azure Adapter Storage SDSC StorageStorage Management Adapter SDSC Cloud (DuraStore) Storage
  7. 7. Storage REST Interface Space Actions Content Actions• Add Space • Add Content• Get/Set Space Properties • Get/Set Content Properties• Get Spaces List • Get Content• Get Space Content List • Copy Content• Get/Set Space Access • Delete Content• Delete Space Storage Provider Interface Other Actions • Get Stores • Get Tasks List • Perform Task
  8. 8. Storage AdaptersTransform calls from DuraCloud interface intocalls understood by a specific storage provider• Determine how DuraCloud “spaces” translate into top level provider containers (buckets)• Define how space and content properties are stored and retrieved• Manage eventual consistency concerns
  9. 9. DuraCloud Services Administrative User Interface (DurAdmin) Storage Management Service Management Report Management (DuraStore) (DuraService) (DuraReport) REST API REST API REST API Service ContainerDuraCloud Instance
  10. 10. DuraCloud Services Media Streamer Duplicate Duplicate on on Demand Change Bit Bit BitIntegrity Integrity IntegrityChecker Checker Checker Tools Bulk Image Image Transformer Transformer Bulk Image Server … so how are they deployed?
  11. 11. DuraCloud Service Deployment Services Service Management Available? (DuraService) 1 REST API 2 Service List Service Manager Services In Plan? 4 Service Registry Service List 3 Service BundlePart 1: Service Config Ask which services are available Service Plan
  12. 12. DuraCloud Service Deployment Deploy Service Management Service (DuraService) Service 1 Config REST API 2 Service ManagerPart 2: Retrieve Deploy Service Service Bundle Deploy Service 4 Service Service Bundle Config Service Service Registry Bundle Service Container Service Bundle 3 Service Bundle Service Configurator Deployed Deployed Service Config Service Service Config Service Plan
  13. 13. Service REST Interface• Get Services• Deploy Service• Get (Deployed) Service• Get Deployed Service Properties• Update Service Configuration• UnDeploy Service
  14. 14. DuraCloud DuraCloud Instance Distributed Services Services Media Streamer Duplicate Duplicate on on Demand Change Bit Bit Bit Integrity Integrity Integrity Checker Checker Checker Tools Bulk Image Image Transformer Transformer Bulk Image Server • Runs primarily outside of• Runs on DuraCloud Instance DuraCloud Instance• Connects to DuraStore • Connects to DuraStore• Can be direct Java service • Makes use of cloud network• Can be deployed web app or computation features
  15. 15. DuraCloud Bulk Service Deployment Service Management 1 (DuraService) 2 REST APITask Management Service Manager (DuraStore) REST API Run Service Registry Service 4 Run Task 3 6 Hadoop 5 Bundle Job Service Container Config Hadoop Deployed Service Plan Service Config
  16. 16. DuraCloud Reporting Administrative User Interface (DurAdmin) Storage Management Service Management Report Management (DuraStore) (DuraService) (DuraReport) REST API REST API REST API Service ContainerDuraCloud Instance
  17. 17. DuraCloud Report Generation Report Management (DuraReport) Storage Service Report Report Builder BuilderDepositReports Message Broker Storage Data REST API REST API Service Storage Management Event Service Management Messages (DuraStore) (DuraService)
  18. 18. Report REST Interface Storage Report Service Report• Start Storage Report • Get Deployed Services• Get Latest Storage Report Report• Get Storage Report List • Get Latest Completed Services Report• Get Storage Report • Get Completed Services• Get Storage Report Info Report List• Cancel Storage Report • Get Services Report• Schedule Storage Report• Cancel Storage Report Schedule
  19. 19. DuraCloud UI Administrative User Interface (DurAdmin) Storage Management Service Management Report Management (DuraStore) (DuraService) (DuraReport) REST API REST API REST API Service ContainerDuraCloud Instance
  20. 20. DuraCloud UI Interaction Administrative User Interface (DurAdmin) Store Client Service Client Report Client REST API REST API REST APIStorage Management Service Management Report Management (DuraStore) (DuraService) (DuraReport)
  21. 21. DuraCloud Security DurAdmin Storage Provider DuraStore DuraService DuraReport Application SecurityStorage Provider Access Control Instance Firewall Transport Security User Data Center
  22. 22. DuraCloud Tools Transfer to DuraCloud Transfer from DuraCloud• Upload Tool • Retrieval Tool – GUI to select files and – Retrieve files from folders one, selected, or all – Visual transfer progress spaces• Sync Tool • Stitch Tool – Monitors for changes – Retrieve large files – Run one time or – Acts on a single file continually• Chunk Tool – Transfer large files – Acts on a single file
  23. 23. DuraCloud ToolsUser Data Center Store Client Store Client Chunk Tool Stitch Tool Sync Tool Retrieval Tool Upload Tool REST API Storage Management (DuraStore)
  24. 24. Near Term Road MapNew features• Access control at the space level• Improved format identification• Automated bit integrity checking• Email notification on service completion• Multi-Account DuraCloud instances• Content indexing and Search – Improved discovery – Improved content browsing• Folder-based navigation• Improved handling of very large files• Multi-Item updates/deletesIntegrations• SDSC storage provider (production version)• Fedora-in-the-cloud prototype• DSpace-in-the-cloud prototype
  25. 25. Long Term Road MapNew Features• Video transcoding• Document format transformation• Improved image support services• Media streaming access controls• Metadata extraction servicesIntegrations• Azure storage provider (production version)• Eucalyptus Walrus storage provider• Rackspace compute instances• Managed Fedora-in-the-cloud• Managed DSpace-in-the-cloud
  26. 26. Questions?
  27. 27. Upcoming DuraCloud Webinars DSpace and DuraCloud November 30, 2011 at 1pm ET Fedora and DuraCloud January 11, 2012 at 1pm ET
  28. 28. Thank You! Further Questions? info@duracloud.org

×