The new repository in AEM 6
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

The new repository in AEM 6

  • 3,280 views
Uploaded on

AEM6 comes with a fresh new repository backend designed for improved performance and scalability. This session introduces the new repository architecture and describes the key differences and......

AEM6 comes with a fresh new repository backend designed for improved performance and scalability. This session introduces the new repository architecture and describes the key differences and improvements for developers and operations teams. Topics covered include content migration, backwards compatibility, key deployment scenarios and configuration options, and custom search indexes.

More in: Software , Technology , Sports
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Great slides, thnx

    Re. Slide #18 Migration between TarMK and MongoMk & backup/restore approach.
    Not sure if I get it right ? Is it already supported and documented somewhere ?
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
3,280
On Slideshare
3,082
From Embeds
198
Number of Embeds
3

Actions

Shares
Downloads
132
Comments
1
Likes
12

Embeds 198

https://twitter.com 187
http://www.slideee.com 10
https://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. The New Repository in AEM 6.0 Jukka Zitting Senior Developer Adobe
  • 2. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Agenda Introduction Deployment MigrationCompatibility Search Questions Questions? • ask any time for clarification • broader questions at the end
  • 3. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. CRX3 aka Oak The new repository in AEM 6.0
  • 4. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Project Oak • Large repositories • Distributed repositories Scalability • Improved performance • Improved concurrency Throughput • Flat hierarchies • Complex ACLs Features • Pluggable components • OSGi-friendly Flexibility Part of Apache Jackrabbit • fully open source, ALv2 • http://jackrabbit.apache.org/oak/ • one of the most active Apache projects by commit counts Fresh implementation of JCR 2.0 • mostly backwards compatible • some optional features excluded to avoid bad performance/scalability tradeoffs Project timeline: • 2008: initial design ideas • 2011: prototyping • 2012: project launched • 2014: Oak 1.0 released
  • 5. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Key differences Oak • MVCC • tree persistence • designed for scalability • plugin architecture • pluggable query indices CRX2 • synchronous updates • key-value persistence • clustering as add-on • static extension points • one index per workspace
  • 6. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Architecture MK Core JCR oak-jcr oak-core TarMK MongoMK JCR • JCR API binding • convenience and safety features like auto-refresh and thread-safety guards Core • high-level functionality • search, versioning, security, etc. • most features implemented as pluggable extensions MicroKernel • versioned tree storage • clustering, caching, etc. http://www.slideshare.net/jukka/ oak-the-architecture-of-apache- jackrabbit-3
  • 7. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Deployment scenarios
  • 8. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. AEM6 Author default TarMK reliability TarMK + failover scalability MongoMK compatibility CRX2 Which repository backend to use for AEM 6.0 author deployments? Instructions: • start from the top • follow lines to add required features • stop when happy Note: • each step adds complexity • most steps can be postponed to when actually needed
  • 9. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. AEM6 Publish Publishing • TarMK • farm of replicas User generated content • MongoMK • publish cluster Which repository backend to use for AEM 6.0 publish deployments? Instructions: • pick your main use case Note: • in most cases it will be possible to switch the backend later on, though the migration may be a bit costly
  • 10. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. TarMK Simple default deployment option Heavily optimized for single-node performance • memory mapping (64bit JVMs) • compactness • locality of reference Optionally with a data store for large binaries • migration from crx2 • external storage (NAS/SAN, S3) TarMK
  • 11. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. TarMK + failover For added reliability • continuous, incremental backup to a separate failover server • requires an external smart firewall, load balancer or monitoring tool (Nagios, etc.) for triggering the failover TarMK Failover TarMK
  • 12. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. MongoMK Reliability + scalability Any number of AEM6 cluster nodes can share an underlying MongoDB cluster • standard MongoDB replication and sharding features • all shared state in MongoDB • eventually consistent mongos mongos mongod primary mongod secondary mongod secondary replicas shards mongod config
  • 13. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Performance benchmarks (lower is better)
  • 14. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Scalability benchmarks
  • 15. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Content migration https://www.flickr.com/photos/belsymington/4102783610/
  • 16. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Content Migration Migrate from CRX2 to Oak see next slide Do a full backup see backup instructions Upgrade to AEM6 with CRX2 see upgrade instructionsHow to migrate an existing AEM 5.x deployment to AEM 6.0 with Oak? BTW, why the migration? • radically different storage format • need for an explicit decision because of changes in backwards compatibility http://docs.adobe.com/content/docs /en/aem/6-0/deploy/upgrade.html
  • 17. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Content migration Start AEM 6 with crx3 Replace repository with migrated content segmentstore from crx2oak datastore from crx2 Create a vanilla AEM 6 repository with FileDataStore new crx-quickstart folder FileDataStore.cfg SegmentNodeStoreService.cfg Run crx2oak Very roughly: 10 mins / GB Does not copy binaries Stop AEM 6 with crx2 How to migrate a CRX2 repository to CRX3 with TarMK? Prerequisites: • use Java 7 or higher • first upgrade to AEM 6 For MongoMK: • -r crx3,crx3mongo • -Doak.mongo.uri • DocumentNodeStoreService.cfg • see documentation for details For custom data stores: • see documentation for details http://docs.adobe.com/content/docs /en/aem/6-0/deploy/upgrade.html
  • 18. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Migration between TarMK and MongoMK • Backend choice mostly transparent to higher level code • Migrating the full repository – use backup/restore • Migrating selected subtrees – use content packages
  • 19. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Backwards compatibility https://www.flickr.com/photos/exfordy/344603717/
  • 20. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Significant changes • See next sectionQuery • Single-node observation mostly unchanged • Commit boundaries, user info, etc. not available across cluster Observation • Significant changes in performance and flexibility of access controls • Backwards compatibility a priority, so few direct problems expected, but review for performance Security Functionality with significant changes in implementation and whose use in client code should be reviewed and, if needed, adapted
  • 21. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Minor differences • Sessions not always up to date with latest changes • Auto-refresh feature avoids most compatibility issues Session refresh • Only referenceable nodes have UUIDs • Other nodes have path identifiers Identifiers • Different frozen identifiers • Updated access control Versioning Functional changes that in most cases require few or no changes in client code
  • 22. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Missing features • Only a single workspace per repository • Unneeded in normal AEM deployments Workspaces • Only very limited support (essentially read-only) • Not very frequently used Same-name siblings • Only “soft locking” implemented • Use as a guideline (“I’m working on this page”) instead of as a strict synchronization tool Locking Functionality that is either completely missing or significantly less useful than before and thus require significant rewrite in affected client code
  • 23. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Custom search indexes https://www.flickr.com/photos/dwmoran/3729048272/
  • 24. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Search Engine Query parser • Statement parsed into an abstract syntax tree Index selection • Cost estimate from each index • Select the index with least cost Index lookup • Look up matching paths from selected index Filtering • Load matching nodes • Filter results based on ACL restrictions and other constraints Features not covered here: • joins • ordering Features not yet available: • facets • Aggregates Trouble with a query? • try the EXPLAIN feature!
  • 25. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Index definitions Nodes under /oak:index • Index data as hidden content or in an external location Creating an index: • add a definition node Removing an index: • remove the node Re-indexing: • set reindex=true Update frequency: • async=“async” Fallback: • traversing index
  • 26. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Property index SELECT * FROM [mix:language] WHERE [jcr:language]=?
  • 27. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Constraints on multiple properties SELECT * FROM [nt:base] WHERE foo=? AND bar=? • one index on foo and another on bar • engine automatically selects best index • other constraints applied as extra filters • future extension: multi-property index
  • 28. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Unique index SELECT * FROM [rep:Authorizable] WHERE [rep:principalName]=?
  • 29. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Ordered index SELECT * FROM [nt:base] WHERE [cq:lastModified] > ? AND [cq:lastModified] < ? ORDER BY [cq:lastModified]
  • 30. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Lucene index SELECT * FROM [nt:base] WHERE CONTAINS(*, ?) Lucene index files stored as hidden content inside the repository!
  • 31. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Questions?
  • 32. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc.