Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scaling digital asset repositories with aem dam and s7 by Devang Shah


Published on

Published in: Technology

Scaling digital asset repositories with aem dam and s7 by Devang Shah

  1. 1. Building Large Scale Digital Asset Repositories with AEM DAM & Scene7 Devang Shah, I-Cubed
  2. 2. INTRODUCTION About Me: • Platform Architect, Emerging Technologies at I-Cubed • Prior Life: • Acquity Group, Part of Accenture Interactive • ContextMine • eFORCE Global • Lived in Reading & Swindon during early stages of my career “I-Cubed helps enterprises connect with their customers across multiple online channels through the implementation, support and training of Digital Experience Management solutions”
  3. 3. The Growing Demand for Rich Media Web Site Then & Now With the growing demand of digital content and rich media; Digital Asset Management (DAM) systems allow for organizing, enhancing, sharing and distributing of digital assets
  4. 4. Digital Asset Management - Content Life Cycle & Use Cases DAM: Asset Storage Asset Creation: • Support for wide variety of formats • Metadata Embed Support • Encoding / Decoding Asset Management: • Advanced Metadata & Tagging • Search & Browse • Lock, Publish, Version, Archive • Workflow support • Authentication, Authorization, DRM Publishing & Delivery: • Web &/or Print publishing • Secure v/s non-secure access • End users: B2B or B2C • File Serving, Streaming • CDN Content Producers Content Editors & Collaborators Content Consumers Overview of Digital Asset Management Ecosystem
  5. 5. AEM: Digital Asset Manager (DAM) Adobe Scene7 • Digital asset management platform for business users • Tightly integrated with AEM Web Content Management tool • Support for wide range of rich media asset types and formats • Metadata & Workflow driven asset handling • Integration with Adobe Creative Cloud to allow collaboration with asset creative team • Hosted solution for managing and publishing dynamic media with local caching & CDN • Dynamic generation of asset renditions (or variations) • Multi-screen engagement using Adaptive content delivery • Management and publishing of videos to web and mobile platforms • Integration with SiteCatalyst for enhanced tracking & reporting AEM DAM & Adobe Scene7: Scaling Asset Delivery Product Highlights
  6. 6. Basic Integration Improved Asset synchronization Dynamic Media Capabilities CQ 5.5 • Configure scene7 cloud service • Manual asset upload to S7 • Auto asset upload via workflow launcher • OOTB Scene7 Asset Viewer components AEM 5.6.1 • Configure polling importer, with support for two way synchronization between AEM DAM & Scene7 • S7 Video Component & Options to S7 Video Presents in AEM • Scene7 content browser for viewing all S7 assets in AEM AEM 6.0 • Dynamic media capabilities with option to access S7 functionality within AEM DAM Integration Highlights AEM DAM & Adobe Scene7: Scaling Asset Delivery Integration Overview
  7. 7. AEM DAM & Adobe Scene7: Scaling Asset Delivery OOTB Integration Options
  8. 8. AEM DAM & Adobe Scene7: Scaling Asset Delivery OOTB Integration Options
  9. 9. AEM DAM & Adobe Scene7: Scaling Asset Delivery Integration Limitations & Custom Enhancements Few Limitations of the Integration (v 5.5 / 5.6.1) & Customizations: - Custom Scene7 Image Component & corresponding Image Tag - Asset deletion in AEM DAM not reflected in Scene7 - Inconsistent Scene7 publish complete status update (daycare hotfix available) - DAM Admin Interface customized to display Scene7 Upload Status
  10. 10. But, what happens when you try to open up the flood gate .. err.. expand the DAM repository usage without optimizing & scaling it first? How to Scale Asset Ingestion?
  11. 11. Images slow to get processed ObservationDispatcher More than 20000 events in the queue Out of Memory Exceptions Blocked Replication Queue Unstable Author Environment Unable to size & scope system architecture Unable to complete System Processes (TarPM & GC) Author Challenges Developer Challenges Sys Admin Challenges Image not getting uploaded to S7 Scaling DAM Asset Ingestion - Challenges
  12. 12. Performance != Scalability • Performance: Capability of a system to provide certain response time. • Scalability: Characteristic of a system to increase performance by adding additional resources. • Scalability can help increase performance; but scaling system without fine tuning and optimizing it first, may not yield much benefit and will waste system resources. Step1: Performance Optimization & Fine Tuning • How can we fine tune and optimize the DAM asset ingestion process? Step 2: Scalability • How can we scale the system so that it can handle ‘x’ number of assets knowing that it takes ‘y’ number of resources per asset. Optimize & Scale (In that order)
  13. 13. Understand How Assets Are Processed & Verify Asset Processing Steps Question To Ask: • Is DMA used to store videos? OR • Is XMP metadata functionality needed? Approach: • If no, trim down DAM Update asset processing workflow by removing unnecessary steps DAM Performance Optimization & Fine Tuning Know Your Use Cases Asset Node Created Trigger Observation Event Event Placed in Event Queue Extract Asset Metadata Process Workflow Event Create Thumbnails FFMpeg Video Processing Create Web Rendition InDesign Extraction DAM Update Asset Workflow
  14. 14. Verify Asset Size Requirement Resources required to process an asset is directly proportional to asset resolution Question To Ask: • How are assets being used? Are assets used only for web display or for Web + Print? Approach: • If assets are used only for web display, restrict &/or reduce asset sizes and ensure that assets uploaded to DAM are optimized per web consumption DAM Performance Optimization & Fine Tuning Know Your Use Cases
  15. 15. If a rendition is used frequently and consistently across site pages: Recommended Approach • Generate renditions at asset creation time via asset update workflow Benefit: • Increase in page load performance • Consistent access and generation of renditions • Author can override a rendition, if needed If a rendition is used sparingly across site pages : Recommended Approach • Generate rendition at run time via image processing script Benefit: • Reduced asset processing time and author instance resource usage • Reduced disk space utilization DAM Performance Optimization & Fine Tuning Understand How Asset Renditions are Used
  16. 16. Workflow & Audit Purge Service • Periodic deletion of archived workflow events. For version 5.5, download it from Adobe KB Base page. Version 5.6+ built-in service. Asset Hierarchy • Organize asset hierarchy such a way that it limits number of direct sub nodes within a particular node (< 1000). Event Queue Size • Parameter: jackrabbit.maxQueuedEvents • Default limit is 20000, can be changed by setting value via start up script Vertical scaling of infrastructure • but, additional RAM and CPU may not solve all the issues, as I/O & network might all become bottleneck DAM Performance Optimization & Fine Tuning Various Configuration Parameters & Other Options
  17. 17. Use Case: Constant stream of new digital asset ingestion – unfiltered and without manual intervention. For example: Large news & media sites Approach: Create separate job or asset processing server which ingests & processes assets and replicates to the main author instance. CQ / Asset Ingestor CQ Author CQ Author CQ Publish CQ Publish Replication (auto asset ingestion Replication (editorial changes) Architecture Scaling for Asset Processing Separate Instance for Ingesting & Processing Assets Scene7 Batch / Auto Asset Upload Process S7 Auto Asset Upload
  18. 18. Use Case: High volume of author ingested data or asset ingestion requiring manual updates. Approach: (5.6.1) Offload execution of complex and resource intensive tasks (for example DAM Update Asset workflow) to a separate instance. CQ Author CQ Author CQ Publish CQ Publish Replication CQ / Asset Processor Asset Process - Task offloading Architecture Scaling for Asset Processing Separate Instance for Executing Process Intensive Tasks Scene7 Batch / Auto Asset Upload Process S7 Auto Asset Upload
  19. 19. Use Case: Large scale repositories with high storage requirements. Options If using Scene7: Publish asset to Scene7 & Only metadata to CQ Publisher Instances. CQ Author CQ Author CQ Publish CQ Publish Replication • Replicate only asset metadata. Actual asset will be served via S7 Shared Data Store & Repository Architecture Scaling for Asset Processing Scene7 S7 Asset Upload
  20. 20. Questions? Feel free to reach out.. Devang Shah Platform Architect, I-Cubed | +1.919.615.4365