SlideShare a Scribd company logo
#evolverocks
CRX2OAK – ALL THE SECRETS OF
REPOSITORY MIGRATION
TOMEK RĘKAWEK, ADOBE RESEARCH
Aug 30, 2016
#evolverocks 2
• Overview of CRX2Oak
• CRX2Oak command line
• Features
• Case study: large migration
• General migration tips
• Using CRX2Oak for AEM upgrade
• Q & (hopefully) A
AGENDA
#evolverocks 3
OVERVIEW OF THE CRX2OAK
UPGRADE FROM CRX2
CQ 5.x – CRX2 AEM 6.x – Jackrabbit Oak
#evolverocks 4
OVERVIEW OF THE CRX2OAK
UPGRADE OR SIDEGRADE
CQ 5.x – CRX2
AEM 6.x – Jackrabbit Oak
AEM 6.x – Oak
#evolverocks 5
OVERVIEW OF THE CRX2OAK
MIGRATING BINARIES
#evolverocks 6
• CRX2Oak is a command-line tool:
• java -jar crx2oak.jar [options] [datastore-options] SOURCE TARGET
• Source and target defines the repositories. Supported formats:
• path to the CRX2 “repository” directory, eg.
crx-quickstart/repository
• path to the Oak SegmentMK “repository” directory, as above
• Mongo URI, eg.
mongodb://localhost:27017/aem
• JDBC URI, eg.
jdbc:mysql://localhost:3306/sakila?profileSQL=true
CRX2OAK COMMAND LINE
REPOSITORY PARAMETER TYPES
#evolverocks 7
• java -jar crx2oak.jar [options] [datastore-options] SOURCE TARGET
• The source blob store is defined using: --src-datastore or --src-s3datastore.
• If there’s no blob store defined for source, CRX2Oak assumes embedded
• If the source blob store is defined, it will be used for target as well (only
references will be copied, not actual binaries)
• It can be overridden with --copy-binaries
• Destination blob store can be defined with: --datastore or --s3datastore
CRX2OAK COMMAND LINE
DEFINING DATASTORE TO BE USED
#evolverocks 8
FEATURES
SELECTING PATHS TO MIGRATE
#evolverocks 9
FEATURES
MIGRATING VERSION STORAGE
#evolverocks 10
• Client requirements
• CQ 5.6.1 instance with a large number of sites and assets, storing binaries in S3
• The content is being authored 24/7
• The migration of the whole content takes about 20h
• The migration is being done offline and the instance can’t be down so long
• The upgraded instance has to be tested before going live
• Strategy
• Snapshot the instance and migrate the copy
• Perform tests on it
• Top-up the changes introduced after snapshot
CASE STUDY
INTRODUCTION
#evolverocks 11
CASE STUDY
STRATEGY
#evolverocks 12
• The migration (4) will be much faster, as only the diff will be migrated
• In the (4) use --skip-init, so the existing repository won’t be reinitialized
• Also, use --include-paths=/content/mysite to migrate only the modified
subtree
CASE STUDY
REMARKS
#evolverocks 13
• When using Mongo (either as source or destination), run CRX2Oak on the same
machine as Mongo primary
• If you don’t need version history for deleted nodes, use --copy-orphaned-
versions=false to make the migration faster
• CRX2Oak may be used to copy content between existing repositories. Use
following parameters:
• --skip-init, so the destination is not initialized with the index definitions,
• --{include,merge}-paths to refer which subtrees should be copied
• --copy-orphaned-versions=false
GENERAL MIGRATION TIPS
#evolverocks 14
• When upgrading CQ 5.x + S3, crx2oak calls AWS asking for length of each binary
• the lengths are stored in Oak but not in CRX2, so we have to ask about it
• For a large repositories it may slow down the whole migration
• It’s possible to pre-fetch all lengths, store them in a text file and configure CRX
(and therefore CRX2Oak) to use it
• More information:
• https://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/oak/upgrade
/blob/LengthCachingDataStore.html
• Sample configuration files:
• http://bit.ly/cq5-s3-upgrade
GENERAL MIGRATION TIPS
UPGRADING CQ 5.X STORING BINARIES IN AWS S3
#evolverocks 15
• UUID conflict exception
• may occur if the destination repository already exists (iterative migration)
• remember to add --copy-orphaned-versions=false
• when using --include-paths, include all modified paths:
• otherwise, if the page has been moved and we include only the destination path,
CRX2Oak won’t remove the page from its original position
• BlobId not found exception
• either source or destination blob store is not configured correctly
• Unable to delete referenced node
• probably CRX2Oak tries to overwrite the whole version storage (removing existing
versions)
• add --copy-orphaned-versions=false
TROUBLESHOOTING
#evolverocks 16
Official docs describes using the extension:
• java -jar aem-quickstart-6.2.0.jar -unpack # unpack the AEM jar
• java -jar aem-quickstart-6.2.0.jar -v -x crx2oak # prepare extension
config
• java -jar aem-quickstart-6.2.0.jar -v -x crx2oak # prepare OSGi config
• java -Xmx4096m -XX:MaxPermSize=2048M -jar aem-quickstart-6.2.0.jar -v -
x crx2oak -xargs -- -o migrate
For running the CRX2Oak manually, the last command should be replaced with:
• java -Xmx4096m -XX:MaxPermSize=2048M -jar crx-
quickstart/opt/helpers/crx2oak/crx2oak.jar [source] [destination]
USING EXTENSION VS RUNNING CRX2OAK
MANUALLY
#evolverocks 17
• All CRX2Oak versions offer similar features
• They differ in:
• Oak version used underneath (as the CRX2Oak starts a normal Oak repository)
• Index definitions created during the repository initialisation
• These both things are assigned to the AEM version and shouldn’t be mismatched
• Table of truth:
• CRX2Oak 1.2.x can be used with AEM 6.1 too, but it won’t have all the
advanced features
VERSIONS
AEM Oak CRX2Oak
AEM 6.0 1.0.x 1.0.x
AEM 6.1 1.2.x 1.3.x (sic!)
AEM 6.2 1.4.x 1.4.x
#evolverocks 18
• CRX2Oak downloads:
• https://repo.adobe.com/nexus/content/groups/public/com/adobe/granite/crx2oak/
• CRX2Oak documentation
• https://docs.adobe.com/docs/en/aem/6-2/deploy/upgrade/using-crx2oak.html
• oak-upgrade documentation:
• https://jackrabbit.apache.org/oak/docs/migration.html
RESOURCES
#evolverocks
THANK YOU!
http://tomek.rekawek.eu
@Tomek1024
rekawek@adobe.com

More Related Content

What's hot

Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0
Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0
Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0
オラクルエンジニア通信
 
Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)
Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)
Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)
オラクルエンジニア通信
 
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best PracticesOracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Markus Michalewicz
 
Understanding oracle rac internals part 1 - slides
Understanding oracle rac internals   part 1 - slidesUnderstanding oracle rac internals   part 1 - slides
Understanding oracle rac internals part 1 - slides
Mohamed Farouk
 
Oracle RAC Virtualized - In VMs, in Containers, On-premises, and in the Cloud
Oracle RAC Virtualized - In VMs, in Containers, On-premises, and in the CloudOracle RAC Virtualized - In VMs, in Containers, On-premises, and in the Cloud
Oracle RAC Virtualized - In VMs, in Containers, On-premises, and in the Cloud
Markus Michalewicz
 
Oracle RAC 19c: Best Practices and Secret Internals
Oracle RAC 19c: Best Practices and Secret InternalsOracle RAC 19c: Best Practices and Secret Internals
Oracle RAC 19c: Best Practices and Secret Internals
Anil Nair
 
How to Use EXAchk Effectively to Manage Exadata Environments
How to Use EXAchk Effectively to Manage Exadata EnvironmentsHow to Use EXAchk Effectively to Manage Exadata Environments
How to Use EXAchk Effectively to Manage Exadata Environments
Sandesh Rao
 
Oracle GoldenGate 概要 2020年11月版
Oracle GoldenGate 概要 2020年11月版Oracle GoldenGate 概要 2020年11月版
Oracle GoldenGate 概要 2020年11月版
オラクルエンジニア通信
 
はじめてのOracle Cloud Platform
はじめてのOracle Cloud PlatformはじめてのOracle Cloud Platform
はじめてのOracle Cloud Platform
オラクルエンジニア通信
 
Hit Refresh with Oracle GoldenGate Microservices
Hit Refresh with Oracle GoldenGate MicroservicesHit Refresh with Oracle GoldenGate Microservices
Hit Refresh with Oracle GoldenGate Microservices
Bobby Curtis
 
CLUB DB2 第122回 DB2管理本の著者が教える 簡単運用管理入門
CLUB DB2 第122回  DB2管理本の著者が教える 簡単運用管理入門CLUB DB2 第122回  DB2管理本の著者が教える 簡単運用管理入門
CLUB DB2 第122回 DB2管理本の著者が教える 簡単運用管理入門
Akira Shimosako
 
Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...
Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...
Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...
HostedbyConfluent
 
SQL Monitoring in Oracle Database 12c
SQL Monitoring in Oracle Database 12cSQL Monitoring in Oracle Database 12c
SQL Monitoring in Oracle Database 12c
Tanel Poder
 
HA環境構築のベスト・プラクティス
HA環境構築のベスト・プラクティスHA環境構築のベスト・プラクティス
HA環境構築のベスト・プラクティス
EnterpriseDB
 
Oracle GoldenGate Performance Tuning
Oracle GoldenGate Performance TuningOracle GoldenGate Performance Tuning
Oracle GoldenGate Performance Tuning
Bobby Curtis
 
Apache tinkerpopとグラフデータベースの世界
Apache tinkerpopとグラフデータベースの世界Apache tinkerpopとグラフデータベースの世界
Apache tinkerpopとグラフデータベースの世界
Yuki Morishita
 
The Oracle RAC Family of Solutions - Presentation
The Oracle RAC Family of Solutions - PresentationThe Oracle RAC Family of Solutions - Presentation
The Oracle RAC Family of Solutions - Presentation
Markus Michalewicz
 
Oracle RAC - New Generation
Oracle RAC - New GenerationOracle RAC - New Generation
Oracle RAC - New Generation
Anil Nair
 
One PDB to go, please!
One PDB to go, please!One PDB to go, please!
One PDB to go, please!
Christian Gohmann
 
KafkaとPulsar
KafkaとPulsarKafkaとPulsar

What's hot (20)

Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0
Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0
Oracle GoldenGate 19c を使用した 簡単データベース移行ガイド_v1.0
 
Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)
Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)
Oracle GoldenGateでの資料採取(トラブル時に採取すべき資料)
 
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best PracticesOracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
Oracle Real Application Clusters (RAC) 12c Rel. 2 - Operational Best Practices
 
Understanding oracle rac internals part 1 - slides
Understanding oracle rac internals   part 1 - slidesUnderstanding oracle rac internals   part 1 - slides
Understanding oracle rac internals part 1 - slides
 
Oracle RAC Virtualized - In VMs, in Containers, On-premises, and in the Cloud
Oracle RAC Virtualized - In VMs, in Containers, On-premises, and in the CloudOracle RAC Virtualized - In VMs, in Containers, On-premises, and in the Cloud
Oracle RAC Virtualized - In VMs, in Containers, On-premises, and in the Cloud
 
Oracle RAC 19c: Best Practices and Secret Internals
Oracle RAC 19c: Best Practices and Secret InternalsOracle RAC 19c: Best Practices and Secret Internals
Oracle RAC 19c: Best Practices and Secret Internals
 
How to Use EXAchk Effectively to Manage Exadata Environments
How to Use EXAchk Effectively to Manage Exadata EnvironmentsHow to Use EXAchk Effectively to Manage Exadata Environments
How to Use EXAchk Effectively to Manage Exadata Environments
 
Oracle GoldenGate 概要 2020年11月版
Oracle GoldenGate 概要 2020年11月版Oracle GoldenGate 概要 2020年11月版
Oracle GoldenGate 概要 2020年11月版
 
はじめてのOracle Cloud Platform
はじめてのOracle Cloud PlatformはじめてのOracle Cloud Platform
はじめてのOracle Cloud Platform
 
Hit Refresh with Oracle GoldenGate Microservices
Hit Refresh with Oracle GoldenGate MicroservicesHit Refresh with Oracle GoldenGate Microservices
Hit Refresh with Oracle GoldenGate Microservices
 
CLUB DB2 第122回 DB2管理本の著者が教える 簡単運用管理入門
CLUB DB2 第122回  DB2管理本の著者が教える 簡単運用管理入門CLUB DB2 第122回  DB2管理本の著者が教える 簡単運用管理入門
CLUB DB2 第122回 DB2管理本の著者が教える 簡単運用管理入門
 
Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...
Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...
Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...
 
SQL Monitoring in Oracle Database 12c
SQL Monitoring in Oracle Database 12cSQL Monitoring in Oracle Database 12c
SQL Monitoring in Oracle Database 12c
 
HA環境構築のベスト・プラクティス
HA環境構築のベスト・プラクティスHA環境構築のベスト・プラクティス
HA環境構築のベスト・プラクティス
 
Oracle GoldenGate Performance Tuning
Oracle GoldenGate Performance TuningOracle GoldenGate Performance Tuning
Oracle GoldenGate Performance Tuning
 
Apache tinkerpopとグラフデータベースの世界
Apache tinkerpopとグラフデータベースの世界Apache tinkerpopとグラフデータベースの世界
Apache tinkerpopとグラフデータベースの世界
 
The Oracle RAC Family of Solutions - Presentation
The Oracle RAC Family of Solutions - PresentationThe Oracle RAC Family of Solutions - Presentation
The Oracle RAC Family of Solutions - Presentation
 
Oracle RAC - New Generation
Oracle RAC - New GenerationOracle RAC - New Generation
Oracle RAC - New Generation
 
One PDB to go, please!
One PDB to go, please!One PDB to go, please!
One PDB to go, please!
 
KafkaとPulsar
KafkaとPulsarKafkaとPulsar
KafkaとPulsar
 

Similar to CRX2Oak - all the secrets of repository migration

Postgre sql linuxcontainers by Jignesh Shah
Postgre sql linuxcontainers by Jignesh ShahPostgre sql linuxcontainers by Jignesh Shah
Postgre sql linuxcontainers by Jignesh Shah
PivotalOpenSourceHub
 
Productionizing Spark and the Spark Job Server
Productionizing Spark and the Spark Job ServerProductionizing Spark and the Spark Job Server
Productionizing Spark and the Spark Job Server
Evan Chan
 
Productionizing Spark and the REST Job Server- Evan Chan
Productionizing Spark and the REST Job Server- Evan ChanProductionizing Spark and the REST Job Server- Evan Chan
Productionizing Spark and the REST Job Server- Evan Chan
Spark Summit
 
The power of linux advanced tracer [POUG18]
The power of linux advanced tracer [POUG18]The power of linux advanced tracer [POUG18]
The power of linux advanced tracer [POUG18]
Mahmoud Hatem
 
les01.pdf
les01.pdfles01.pdf
les01.pdf
VAMSICHOWDARY61
 
Ippevent : openshift Introduction
Ippevent : openshift IntroductionIppevent : openshift Introduction
Ippevent : openshift Introduction
kanedafromparis
 
OGG Architecture Performance
OGG Architecture PerformanceOGG Architecture Performance
OGG Architecture Performance
Enkitec
 
Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
Bobby Curtis
 
Benchmarking Solr Performance at Scale
Benchmarking Solr Performance at ScaleBenchmarking Solr Performance at Scale
Benchmarking Solr Performance at Scale
thelabdude
 
Docking postgres
Docking postgresDocking postgres
Docking postgres
rycamor
 
Tanel Poder Oracle Scripts and Tools (2010)
Tanel Poder Oracle Scripts and Tools (2010)Tanel Poder Oracle Scripts and Tools (2010)
Tanel Poder Oracle Scripts and Tools (2010)
Tanel Poder
 
Oracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture PerformanceOracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture Performance
Enkitec
 
Simon Bennetts - Automating ZAP
Simon Bennetts - Automating ZAP Simon Bennetts - Automating ZAP
Simon Bennetts - Automating ZAP
DevSecCon
 
Automating OWASP ZAP - DevCSecCon talk
Automating OWASP ZAP - DevCSecCon talk Automating OWASP ZAP - DevCSecCon talk
Automating OWASP ZAP - DevCSecCon talk
Simon Bennetts
 
Running Spark on Cloud
Running Spark on CloudRunning Spark on Cloud
Running Spark on Cloud
Qubole
 
Java and Containers - Make it Awesome !
Java and Containers - Make it Awesome !Java and Containers - Make it Awesome !
Java and Containers - Make it Awesome !
Dinakar Guniguntala
 
Performance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons LearnedPerformance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons Learned
Tim Callaghan
 
Training Slides: 203 - Backup & Recovery
Training Slides: 203 - Backup & RecoveryTraining Slides: 203 - Backup & Recovery
Training Slides: 203 - Backup & Recovery
Continuent
 
Creating a Benchmarking Infrastructure That Just Works
Creating a Benchmarking Infrastructure That Just WorksCreating a Benchmarking Infrastructure That Just Works
Creating a Benchmarking Infrastructure That Just Works
Tim Callaghan
 
Summary of JDK10 and What will come into JDK11
Summary of JDK10 and What will come into JDK11Summary of JDK10 and What will come into JDK11
Summary of JDK10 and What will come into JDK11
なおき きしだ
 

Similar to CRX2Oak - all the secrets of repository migration (20)

Postgre sql linuxcontainers by Jignesh Shah
Postgre sql linuxcontainers by Jignesh ShahPostgre sql linuxcontainers by Jignesh Shah
Postgre sql linuxcontainers by Jignesh Shah
 
Productionizing Spark and the Spark Job Server
Productionizing Spark and the Spark Job ServerProductionizing Spark and the Spark Job Server
Productionizing Spark and the Spark Job Server
 
Productionizing Spark and the REST Job Server- Evan Chan
Productionizing Spark and the REST Job Server- Evan ChanProductionizing Spark and the REST Job Server- Evan Chan
Productionizing Spark and the REST Job Server- Evan Chan
 
The power of linux advanced tracer [POUG18]
The power of linux advanced tracer [POUG18]The power of linux advanced tracer [POUG18]
The power of linux advanced tracer [POUG18]
 
les01.pdf
les01.pdfles01.pdf
les01.pdf
 
Ippevent : openshift Introduction
Ippevent : openshift IntroductionIppevent : openshift Introduction
Ippevent : openshift Introduction
 
OGG Architecture Performance
OGG Architecture PerformanceOGG Architecture Performance
OGG Architecture Performance
 
Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
Oracle GoldenGate Presentation from OTN Virtual Technology Summit - 7/9/14 (PDF)
 
Benchmarking Solr Performance at Scale
Benchmarking Solr Performance at ScaleBenchmarking Solr Performance at Scale
Benchmarking Solr Performance at Scale
 
Docking postgres
Docking postgresDocking postgres
Docking postgres
 
Tanel Poder Oracle Scripts and Tools (2010)
Tanel Poder Oracle Scripts and Tools (2010)Tanel Poder Oracle Scripts and Tools (2010)
Tanel Poder Oracle Scripts and Tools (2010)
 
Oracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture PerformanceOracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture Performance
 
Simon Bennetts - Automating ZAP
Simon Bennetts - Automating ZAP Simon Bennetts - Automating ZAP
Simon Bennetts - Automating ZAP
 
Automating OWASP ZAP - DevCSecCon talk
Automating OWASP ZAP - DevCSecCon talk Automating OWASP ZAP - DevCSecCon talk
Automating OWASP ZAP - DevCSecCon talk
 
Running Spark on Cloud
Running Spark on CloudRunning Spark on Cloud
Running Spark on Cloud
 
Java and Containers - Make it Awesome !
Java and Containers - Make it Awesome !Java and Containers - Make it Awesome !
Java and Containers - Make it Awesome !
 
Performance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons LearnedPerformance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons Learned
 
Training Slides: 203 - Backup & Recovery
Training Slides: 203 - Backup & RecoveryTraining Slides: 203 - Backup & Recovery
Training Slides: 203 - Backup & Recovery
 
Creating a Benchmarking Infrastructure That Just Works
Creating a Benchmarking Infrastructure That Just WorksCreating a Benchmarking Infrastructure That Just Works
Creating a Benchmarking Infrastructure That Just Works
 
Summary of JDK10 and What will come into JDK11
Summary of JDK10 and What will come into JDK11Summary of JDK10 and What will come into JDK11
Summary of JDK10 and What will come into JDK11
 

More from Tomasz Rękawek

Radio ad blocker
Radio ad blockerRadio ad blocker
Radio ad blocker
Tomasz Rękawek
 
Deep-dive into cloud-native AEM deployments based on Kubernetes
Deep-dive into cloud-native AEM deployments based on KubernetesDeep-dive into cloud-native AEM deployments based on Kubernetes
Deep-dive into cloud-native AEM deployments based on Kubernetes
Tomasz Rękawek
 
Emulating Game Boy in Java
Emulating Game Boy in JavaEmulating Game Boy in Java
Emulating Game Boy in Java
Tomasz Rękawek
 
Zero downtime deployments for the Sling-based apps using Docker
Zero downtime deployments for the Sling-based apps using DockerZero downtime deployments for the Sling-based apps using Docker
Zero downtime deployments for the Sling-based apps using Docker
Tomasz Rękawek
 
SlingQuery
SlingQuerySlingQuery
SlingQuery
Tomasz Rękawek
 
Code metrics
Code metricsCode metrics
Code metrics
Tomasz Rękawek
 
Inter-Sling communication with message queue
Inter-Sling communication with message queueInter-Sling communication with message queue
Inter-Sling communication with message queue
Tomasz Rękawek
 
Sling Dynamic Include
Sling Dynamic IncludeSling Dynamic Include
Sling Dynamic Include
Tomasz Rękawek
 
Shooting rabbits with sling
Shooting rabbits with slingShooting rabbits with sling
Shooting rabbits with sling
Tomasz Rękawek
 

More from Tomasz Rękawek (9)

Radio ad blocker
Radio ad blockerRadio ad blocker
Radio ad blocker
 
Deep-dive into cloud-native AEM deployments based on Kubernetes
Deep-dive into cloud-native AEM deployments based on KubernetesDeep-dive into cloud-native AEM deployments based on Kubernetes
Deep-dive into cloud-native AEM deployments based on Kubernetes
 
Emulating Game Boy in Java
Emulating Game Boy in JavaEmulating Game Boy in Java
Emulating Game Boy in Java
 
Zero downtime deployments for the Sling-based apps using Docker
Zero downtime deployments for the Sling-based apps using DockerZero downtime deployments for the Sling-based apps using Docker
Zero downtime deployments for the Sling-based apps using Docker
 
SlingQuery
SlingQuerySlingQuery
SlingQuery
 
Code metrics
Code metricsCode metrics
Code metrics
 
Inter-Sling communication with message queue
Inter-Sling communication with message queueInter-Sling communication with message queue
Inter-Sling communication with message queue
 
Sling Dynamic Include
Sling Dynamic IncludeSling Dynamic Include
Sling Dynamic Include
 
Shooting rabbits with sling
Shooting rabbits with slingShooting rabbits with sling
Shooting rabbits with sling
 

Recently uploaded

美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
widenerjobeyrl638
 
How GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdfHow GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdf
Zycus
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
Jhone kinadey
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
Yara Milbes
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Vince Scalabrino
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
kgyxske
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
kalichargn70th171
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
Reetu63
 
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Luigi Fugaro
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
XfilesPro
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
Alina Yurenko
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
gapen1
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
Maitrey Patel
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
Tier1 app
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and MoreManyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
narinav14
 

Recently uploaded (20)

美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
 
How GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdfHow GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdf
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
一比一原版(sdsu毕业证书)圣地亚哥州立大学毕业证如何办理
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
 
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
Voxxed Days Trieste 2024 - Unleashing the Power of Vector Search and Semantic...
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
 
ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.ACE - Team 24 Wrapup event at ahmedabad.
ACE - Team 24 Wrapup event at ahmedabad.
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and MoreManyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
 

CRX2Oak - all the secrets of repository migration

  • 1. #evolverocks CRX2OAK – ALL THE SECRETS OF REPOSITORY MIGRATION TOMEK RĘKAWEK, ADOBE RESEARCH Aug 30, 2016
  • 2. #evolverocks 2 • Overview of CRX2Oak • CRX2Oak command line • Features • Case study: large migration • General migration tips • Using CRX2Oak for AEM upgrade • Q & (hopefully) A AGENDA
  • 3. #evolverocks 3 OVERVIEW OF THE CRX2OAK UPGRADE FROM CRX2 CQ 5.x – CRX2 AEM 6.x – Jackrabbit Oak
  • 4. #evolverocks 4 OVERVIEW OF THE CRX2OAK UPGRADE OR SIDEGRADE CQ 5.x – CRX2 AEM 6.x – Jackrabbit Oak AEM 6.x – Oak
  • 5. #evolverocks 5 OVERVIEW OF THE CRX2OAK MIGRATING BINARIES
  • 6. #evolverocks 6 • CRX2Oak is a command-line tool: • java -jar crx2oak.jar [options] [datastore-options] SOURCE TARGET • Source and target defines the repositories. Supported formats: • path to the CRX2 “repository” directory, eg. crx-quickstart/repository • path to the Oak SegmentMK “repository” directory, as above • Mongo URI, eg. mongodb://localhost:27017/aem • JDBC URI, eg. jdbc:mysql://localhost:3306/sakila?profileSQL=true CRX2OAK COMMAND LINE REPOSITORY PARAMETER TYPES
  • 7. #evolverocks 7 • java -jar crx2oak.jar [options] [datastore-options] SOURCE TARGET • The source blob store is defined using: --src-datastore or --src-s3datastore. • If there’s no blob store defined for source, CRX2Oak assumes embedded • If the source blob store is defined, it will be used for target as well (only references will be copied, not actual binaries) • It can be overridden with --copy-binaries • Destination blob store can be defined with: --datastore or --s3datastore CRX2OAK COMMAND LINE DEFINING DATASTORE TO BE USED
  • 10. #evolverocks 10 • Client requirements • CQ 5.6.1 instance with a large number of sites and assets, storing binaries in S3 • The content is being authored 24/7 • The migration of the whole content takes about 20h • The migration is being done offline and the instance can’t be down so long • The upgraded instance has to be tested before going live • Strategy • Snapshot the instance and migrate the copy • Perform tests on it • Top-up the changes introduced after snapshot CASE STUDY INTRODUCTION
  • 12. #evolverocks 12 • The migration (4) will be much faster, as only the diff will be migrated • In the (4) use --skip-init, so the existing repository won’t be reinitialized • Also, use --include-paths=/content/mysite to migrate only the modified subtree CASE STUDY REMARKS
  • 13. #evolverocks 13 • When using Mongo (either as source or destination), run CRX2Oak on the same machine as Mongo primary • If you don’t need version history for deleted nodes, use --copy-orphaned- versions=false to make the migration faster • CRX2Oak may be used to copy content between existing repositories. Use following parameters: • --skip-init, so the destination is not initialized with the index definitions, • --{include,merge}-paths to refer which subtrees should be copied • --copy-orphaned-versions=false GENERAL MIGRATION TIPS
  • 14. #evolverocks 14 • When upgrading CQ 5.x + S3, crx2oak calls AWS asking for length of each binary • the lengths are stored in Oak but not in CRX2, so we have to ask about it • For a large repositories it may slow down the whole migration • It’s possible to pre-fetch all lengths, store them in a text file and configure CRX (and therefore CRX2Oak) to use it • More information: • https://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/oak/upgrade /blob/LengthCachingDataStore.html • Sample configuration files: • http://bit.ly/cq5-s3-upgrade GENERAL MIGRATION TIPS UPGRADING CQ 5.X STORING BINARIES IN AWS S3
  • 15. #evolverocks 15 • UUID conflict exception • may occur if the destination repository already exists (iterative migration) • remember to add --copy-orphaned-versions=false • when using --include-paths, include all modified paths: • otherwise, if the page has been moved and we include only the destination path, CRX2Oak won’t remove the page from its original position • BlobId not found exception • either source or destination blob store is not configured correctly • Unable to delete referenced node • probably CRX2Oak tries to overwrite the whole version storage (removing existing versions) • add --copy-orphaned-versions=false TROUBLESHOOTING
  • 16. #evolverocks 16 Official docs describes using the extension: • java -jar aem-quickstart-6.2.0.jar -unpack # unpack the AEM jar • java -jar aem-quickstart-6.2.0.jar -v -x crx2oak # prepare extension config • java -jar aem-quickstart-6.2.0.jar -v -x crx2oak # prepare OSGi config • java -Xmx4096m -XX:MaxPermSize=2048M -jar aem-quickstart-6.2.0.jar -v - x crx2oak -xargs -- -o migrate For running the CRX2Oak manually, the last command should be replaced with: • java -Xmx4096m -XX:MaxPermSize=2048M -jar crx- quickstart/opt/helpers/crx2oak/crx2oak.jar [source] [destination] USING EXTENSION VS RUNNING CRX2OAK MANUALLY
  • 17. #evolverocks 17 • All CRX2Oak versions offer similar features • They differ in: • Oak version used underneath (as the CRX2Oak starts a normal Oak repository) • Index definitions created during the repository initialisation • These both things are assigned to the AEM version and shouldn’t be mismatched • Table of truth: • CRX2Oak 1.2.x can be used with AEM 6.1 too, but it won’t have all the advanced features VERSIONS AEM Oak CRX2Oak AEM 6.0 1.0.x 1.0.x AEM 6.1 1.2.x 1.3.x (sic!) AEM 6.2 1.4.x 1.4.x
  • 18. #evolverocks 18 • CRX2Oak downloads: • https://repo.adobe.com/nexus/content/groups/public/com/adobe/granite/crx2oak/ • CRX2Oak documentation • https://docs.adobe.com/docs/en/aem/6-2/deploy/upgrade/using-crx2oak.html • oak-upgrade documentation: • https://jackrabbit.apache.org/oak/docs/migration.html RESOURCES