CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
The New Repository
in AEM 6.0
Jukka Zitting
Senior Develope...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Agenda
Introduction Deployment
MigrationCompatibility
Searc...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
CRX3 aka Oak
The new repository in AEM 6.0
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Project Oak
• Large repositories
• Distributed repositories...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Key differences
Oak
• MVCC
• tree persistence
• designed fo...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Architecture
MK
Core
JCR oak-jcr
oak-core
TarMK MongoMK
JCR...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Deployment scenarios
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
AEM6 Author
default
TarMK
reliability
TarMK + failover
scal...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
AEM6 Publish
Publishing
• TarMK
• farm of
replicas
User gen...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
TarMK
Simple default deployment option
Heavily optimized fo...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
TarMK + failover
For added reliability
• continuous, increm...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
MongoMK
Reliability + scalability
Any number of AEM6 cluste...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Performance benchmarks (lower is better)
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Scalability benchmarks
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Content migration
https://www.flickr.com/photos/belsymingto...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Content Migration
Migrate from CRX2 to Oak
see next slide
D...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Content migration
Start AEM 6 with crx3
Replace repository ...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Migration between TarMK and MongoMK
• Backend choice mostly...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Backwards compatibility
https://www.flickr.com/photos/exfor...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Significant changes • See next sectionQuery
• Single-node o...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Minor differences
• Sessions not always up to
date with lat...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Missing features
• Only a single workspace per
repository
•...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Custom search indexes
https://www.flickr.com/photos/dwmoran...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Search Engine
Query
parser
• Statement parsed into an abstr...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Index definitions
Nodes under /oak:index
• Index data as hi...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Property index
SELECT * FROM [mix:language]
WHERE [jcr:lang...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Constraints on multiple properties
SELECT * FROM [nt:base]
...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Unique index
SELECT * FROM [rep:Authorizable]
WHERE [rep:pr...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Ordered index
SELECT * FROM [nt:base]
WHERE [cq:lastModifie...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Lucene index
SELECT * FROM [nt:base]
WHERE CONTAINS(*, ?)
L...
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Questions?
CIRCUIT – An Adobe Developer Event
Presented by CITYTECH, Inc.
Upcoming SlideShare
Loading in...5
×

The new repository in AEM 6

7,110

Published on

AEM6 comes with a fresh new repository backend designed for improved performance and scalability. This session introduces the new repository architecture and describes the key differences and improvements for developers and operations teams. Topics covered include content migration, backwards compatibility, key deployment scenarios and configuration options, and custom search indexes.

Published in: Software, Technology, Sports
1 Comment
23 Likes
Statistics
Notes
No Downloads
Views
Total Views
7,110
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
287
Comments
1
Likes
23
Embeds 0
No embeds

No notes for slide

The new repository in AEM 6

  1. 1. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. The New Repository in AEM 6.0 Jukka Zitting Senior Developer Adobe
  2. 2. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Agenda Introduction Deployment MigrationCompatibility Search Questions Questions? • ask any time for clarification • broader questions at the end
  3. 3. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. CRX3 aka Oak The new repository in AEM 6.0
  4. 4. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Project Oak • Large repositories • Distributed repositories Scalability • Improved performance • Improved concurrency Throughput • Flat hierarchies • Complex ACLs Features • Pluggable components • OSGi-friendly Flexibility Part of Apache Jackrabbit • fully open source, ALv2 • http://jackrabbit.apache.org/oak/ • one of the most active Apache projects by commit counts Fresh implementation of JCR 2.0 • mostly backwards compatible • some optional features excluded to avoid bad performance/scalability tradeoffs Project timeline: • 2008: initial design ideas • 2011: prototyping • 2012: project launched • 2014: Oak 1.0 released
  5. 5. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Key differences Oak • MVCC • tree persistence • designed for scalability • plugin architecture • pluggable query indices CRX2 • synchronous updates • key-value persistence • clustering as add-on • static extension points • one index per workspace
  6. 6. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Architecture MK Core JCR oak-jcr oak-core TarMK MongoMK JCR • JCR API binding • convenience and safety features like auto-refresh and thread-safety guards Core • high-level functionality • search, versioning, security, etc. • most features implemented as pluggable extensions MicroKernel • versioned tree storage • clustering, caching, etc. http://www.slideshare.net/jukka/ oak-the-architecture-of-apache- jackrabbit-3
  7. 7. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Deployment scenarios
  8. 8. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. AEM6 Author default TarMK reliability TarMK + failover scalability MongoMK compatibility CRX2 Which repository backend to use for AEM 6.0 author deployments? Instructions: • start from the top • follow lines to add required features • stop when happy Note: • each step adds complexity • most steps can be postponed to when actually needed
  9. 9. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. AEM6 Publish Publishing • TarMK • farm of replicas User generated content • MongoMK • publish cluster Which repository backend to use for AEM 6.0 publish deployments? Instructions: • pick your main use case Note: • in most cases it will be possible to switch the backend later on, though the migration may be a bit costly
  10. 10. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. TarMK Simple default deployment option Heavily optimized for single-node performance • memory mapping (64bit JVMs) • compactness • locality of reference Optionally with a data store for large binaries • migration from crx2 • external storage (NAS/SAN, S3) TarMK
  11. 11. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. TarMK + failover For added reliability • continuous, incremental backup to a separate failover server • requires an external smart firewall, load balancer or monitoring tool (Nagios, etc.) for triggering the failover TarMK Failover TarMK
  12. 12. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. MongoMK Reliability + scalability Any number of AEM6 cluster nodes can share an underlying MongoDB cluster • standard MongoDB replication and sharding features • all shared state in MongoDB • eventually consistent mongos mongos mongod primary mongod secondary mongod secondary replicas shards mongod config
  13. 13. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Performance benchmarks (lower is better)
  14. 14. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Scalability benchmarks
  15. 15. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Content migration https://www.flickr.com/photos/belsymington/4102783610/
  16. 16. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Content Migration Migrate from CRX2 to Oak see next slide Do a full backup see backup instructions Upgrade to AEM6 with CRX2 see upgrade instructionsHow to migrate an existing AEM 5.x deployment to AEM 6.0 with Oak? BTW, why the migration? • radically different storage format • need for an explicit decision because of changes in backwards compatibility http://docs.adobe.com/content/docs /en/aem/6-0/deploy/upgrade.html
  17. 17. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Content migration Start AEM 6 with crx3 Replace repository with migrated content segmentstore from crx2oak datastore from crx2 Create a vanilla AEM 6 repository with FileDataStore new crx-quickstart folder FileDataStore.cfg SegmentNodeStoreService.cfg Run crx2oak Very roughly: 10 mins / GB Does not copy binaries Stop AEM 6 with crx2 How to migrate a CRX2 repository to CRX3 with TarMK? Prerequisites: • use Java 7 or higher • first upgrade to AEM 6 For MongoMK: • -r crx3,crx3mongo • -Doak.mongo.uri • DocumentNodeStoreService.cfg • see documentation for details For custom data stores: • see documentation for details http://docs.adobe.com/content/docs /en/aem/6-0/deploy/upgrade.html
  18. 18. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Migration between TarMK and MongoMK • Backend choice mostly transparent to higher level code • Migrating the full repository – use backup/restore • Migrating selected subtrees – use content packages
  19. 19. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Backwards compatibility https://www.flickr.com/photos/exfordy/344603717/
  20. 20. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Significant changes • See next sectionQuery • Single-node observation mostly unchanged • Commit boundaries, user info, etc. not available across cluster Observation • Significant changes in performance and flexibility of access controls • Backwards compatibility a priority, so few direct problems expected, but review for performance Security Functionality with significant changes in implementation and whose use in client code should be reviewed and, if needed, adapted
  21. 21. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Minor differences • Sessions not always up to date with latest changes • Auto-refresh feature avoids most compatibility issues Session refresh • Only referenceable nodes have UUIDs • Other nodes have path identifiers Identifiers • Different frozen identifiers • Updated access control Versioning Functional changes that in most cases require few or no changes in client code
  22. 22. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Missing features • Only a single workspace per repository • Unneeded in normal AEM deployments Workspaces • Only very limited support (essentially read-only) • Not very frequently used Same-name siblings • Only “soft locking” implemented • Use as a guideline (“I’m working on this page”) instead of as a strict synchronization tool Locking Functionality that is either completely missing or significantly less useful than before and thus require significant rewrite in affected client code
  23. 23. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Custom search indexes https://www.flickr.com/photos/dwmoran/3729048272/
  24. 24. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Search Engine Query parser • Statement parsed into an abstract syntax tree Index selection • Cost estimate from each index • Select the index with least cost Index lookup • Look up matching paths from selected index Filtering • Load matching nodes • Filter results based on ACL restrictions and other constraints Features not covered here: • joins • ordering Features not yet available: • facets • Aggregates Trouble with a query? • try the EXPLAIN feature!
  25. 25. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Index definitions Nodes under /oak:index • Index data as hidden content or in an external location Creating an index: • add a definition node Removing an index: • remove the node Re-indexing: • set reindex=true Update frequency: • async=“async” Fallback: • traversing index
  26. 26. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Property index SELECT * FROM [mix:language] WHERE [jcr:language]=?
  27. 27. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Constraints on multiple properties SELECT * FROM [nt:base] WHERE foo=? AND bar=? • one index on foo and another on bar • engine automatically selects best index • other constraints applied as extra filters • future extension: multi-property index
  28. 28. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Unique index SELECT * FROM [rep:Authorizable] WHERE [rep:principalName]=?
  29. 29. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Ordered index SELECT * FROM [nt:base] WHERE [cq:lastModified] > ? AND [cq:lastModified] < ? ORDER BY [cq:lastModified]
  30. 30. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Lucene index SELECT * FROM [nt:base] WHERE CONTAINS(*, ?) Lucene index files stored as hidden content inside the repository!
  31. 31. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc. Questions?
  32. 32. CIRCUIT – An Adobe Developer Event Presented by CITYTECH, Inc.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×