The Art of the Upgrade
Kyle Adams
Solutions Engineer at Alfresco
@kylefadams

#SummitNow
Objectives
• Arm you with the strategy and tactics to
perform a successful upgrade of the core
Alfresco repository.

#Summ...
Strategy and Tactics
"Strategy without tactics is the slowest route
to victory.”
“Tactics without strategy is the noise be...
Strategy and Tactics
“...tactics are the actual means used to gain
an objective, while strategy is the overall
campaign pl...
Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:...
Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:...
Prepare
"To ... not prepare is the greatest of crimes;
to be prepared beforehand for any
contingency is the greatest of vi...
Prepare – Current State
•
•
•
•
•
•
•

Count of all database tables
Size and number of files in content store
Size of luce...
Prepare – Upgrade Path
•

http://docs.alfresco.com/4.2/topic/com.alfresco.enterprise.doc/concepts/up
grade-path.html

#Sum...
Prepare – Backup/Restore
• Backup Prod Database
• Restore into new instance
• Backup Prod lucene/solr indices
• Restore in...
Prepare – Backup/Restore
• Backup Prod ContentStore
• What if your ContentStore is too large to
restoring into a lower env...
Prepare – Backup/Restore
• No excuse!
• We can mock up the ContentStore
against the database using small sample
files.
• h...
Prepare – Alfresco Instances
• Install an out-of-the-box Alfresco instance for
each upgrade segment
• Current Version: 3.3...
Prepare – Alfresco Instances
• Do not perform an in-place upgrade!

#SummitNow
#SummitNow
Prepare – Patches
• Know exactly which patches will run
during each segment of the upgrade.
• alfresco/WEBINF/classes/alfr...
Prepare – Patches
• Still unsure?
• Perform a dry run of the upgrade with
OOTB installation.
• Set logging to debug for th...
Prepare – Customizations
• Only include bare minimum extensions
and overrides (Example: content model)
• Restore a Prod ba...
Prepare – Tools for Monitoring
•
•
•

Logs
• tail, text editor, Splunk
JVM
• YourKit, JProfiler
DB Queries
• Vendor-specif...
Prepare – Tools for Monitoring
• Hardware & OS (CPU, Memory, Disk, etc)
• top, nmon, ProcessExplorer

#SummitNow
#SummitNo...
Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:...
Execute
• Validate your newly restored instance is
working as expected.
• Example: 3.3.4
• Shutdown the instance
• Startup...
Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:...
Monitor - Logs
• Based on logging output from your dry
run, you should be able to determine the
overall progress of your u...
Monitor - Logs
• Stalled?

#SummitNow
#SummitNow
Monitor - JVM
• Leverage a Java profiling tool such as
YourKit
• YourKit will tell you exactly which
class and methods are...
Monitor – JVM
• Still Stalled???

#SummitNow
#SummitNow
Monitor – SQL Queries
• Use your DB vendor-specific tools to monitor
the queries being executed
• Don’t have MySQL Workben...
Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:...
Validate – Smoke Tests
• Test basic Alfresco functionality.
• Test one or more of your major use cases.
• For the latest 4...
Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:...
Gotchas – Cyclical ACL’s
• FixAclInheritancePatch could fail with
StackOverflowException during 3.4
upgrade.
• Added cycli...
Gotchas – Orphaned Nodes
• Orphaned Nodes and Associations to
deleted nodes
• Reindex could fail with
“InvalidNodeRefExcep...
Gotchas – Orphaned Nodes
• Find orphaned nodes:
SELECT n.*
FROM alf_node n, alf_qname q, alf_namespace ns
where q.id = n.t...
Gotchas – Orphaned Nodes
• Find associations to deleted nodes:
SELECT a.*
FROM alf_child_assoc a, alf_node n, alf_qname q,...
Gotchas – Orphaned Nodes
• CAUTION: Do not attempt to fix these by
hacking the database!
• Detection and fix via Java has ...
Gotchas – Nodes without Txn’s
• May cause schema updates to fail with
*ContraintViolationExceptions.

#SummitNow
#SummitNo...
Gotchas – Nodes without Txn’s
• Find nodes without transactions:
SELECT *
FROM alf_node node
WHERE node.transaction_id
NOT...
Gotchas – Nodes without Txn’s
• CAUTION: Do not alter the database
without the consent or the assistance of
Alfresco Suppo...
Gotchas – Long-running
Patches

• MigrateVersionStorePatch (2.x  3.x)
• Run asynchronously as a scheduled job
#
Never:
* ...
Gotchas – Long-running
Patches

• ContentUrlConverterPatch (...  3.2.x)
• Run asynchronously as a scheduled job
#
Never:
...
Gotchas – Long-running
Patches

• NodeDeleted.sql (…  4.1.x)
• Renames alf_node to t_alf_node
• Recreate the alf_node tab...
Gotchas – Database Statistics
• Don’t forget to recompute your database
statistics!
• Without histograms if you’re using
O...
Summary
• Be meticulous!
• Have attention to detail!
• Leave nothing up to mystery!
• There should be no surprises.
• “Kno...
Summary
• “The supreme art of war is to subdue the
enemy without fighting.”
[1]

•

[1] Sun Tzu – The Art of War

#SummitN...
Contact Information
•
•
•
•

Kyle Adams
Solutions Engineer at Alfresco
Email: kyle.adams@alfresco.com
Twitter: @kylefadams...
#SummitNow
Upcoming SlideShare
Loading in …5
×

Alfresco Summit 2013 - The Art of the Upgrade

882 views
764 views

Published on

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
882
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
39
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Alfresco Summit 2013 - The Art of the Upgrade

  1. 1. The Art of the Upgrade Kyle Adams Solutions Engineer at Alfresco @kylefadams #SummitNow
  2. 2. Objectives • Arm you with the strategy and tactics to perform a successful upgrade of the core Alfresco repository. #SummitNow #SummitNow
  3. 3. Strategy and Tactics "Strategy without tactics is the slowest route to victory.” “Tactics without strategy is the noise before defeat.” – Sun Tzu, The Art of War #SummitNow #SummitNow
  4. 4. Strategy and Tactics “...tactics are the actual means used to gain an objective, while strategy is the overall campaign plan…” [1] [1] http://en.wikipedia.org/wiki/Tactic_(method) #SummitNow #SummitNow
  5. 5. Upgrade Strategy Strategy: Accurately predict how my Alfresco instance will behave throughout the upgrade process. Tactic: Validate Tactic: Monitor Tactic: Execute Tactic: Prepare #SummitNow #SummitNow
  6. 6. Upgrade Strategy Strategy: Accurately predict how my Alfresco instance will behave throughout the upgrade process. Tactic: Validate Tactic: Monitor 80% of Work Tactic: Execute Tactic: Prepare #SummitNow #SummitNow
  7. 7. Prepare "To ... not prepare is the greatest of crimes; to be prepared beforehand for any contingency is the greatest of virtues.” [1] [1] Sun Tzu, The Art of War #SummitNow #SummitNow
  8. 8. Prepare – Current State • • • • • • • Count of all database tables Size and number of files in content store Size of lucene or Solr indexes Custom Content Models Custom Process definitions Installed modules Customizations #SummitNow #SummitNow
  9. 9. Prepare – Upgrade Path • http://docs.alfresco.com/4.2/topic/com.alfresco.enterprise.doc/concepts/up grade-path.html #SummitNow #SummitNow
  10. 10. Prepare – Backup/Restore • Backup Prod Database • Restore into new instance • Backup Prod lucene/solr indices • Restore into new instance • Backup Prod ContentStore • Restore into new instance #SummitNow #SummitNow
  11. 11. Prepare – Backup/Restore • Backup Prod ContentStore • What if your ContentStore is too large to restoring into a lower environment for testing? #SummitNow #SummitNow
  12. 12. Prepare – Backup/Restore • No excuse! • We can mock up the ContentStore against the database using small sample files. • https://github.com/kylefernandadams/alfre sco-mock-content-generator • CAUTION: Only use this for testing purposes. #SummitNow #SummitNow
  13. 13. Prepare – Alfresco Instances • Install an out-of-the-box Alfresco instance for each upgrade segment • Current Version: 3.3.4 • Target Version: 4.2.0 • Path: 3.3.4 -> 3.3.5 -> 4.2.0 • Don’t forget to request license files for each minor version from Alfresco support! • Point each instance to common content store, database, and indices. #SummitNow • Do not perform an in-place upgrade! #SummitNow
  14. 14. Prepare – Alfresco Instances • Do not perform an in-place upgrade! #SummitNow #SummitNow
  15. 15. Prepare – Patches • Know exactly which patches will run during each segment of the upgrade. • alfresco/WEBINF/classes/alfresco/patch/patchservices-context.xml • Patch Java docs:http://dev.alfresco.com/resource/docs/java/reposi tory/org/alfresco/repo/admin/patch/impl/packagesummary.html #SummitNow #SummitNow
  16. 16. Prepare – Patches • Still unsure? • Perform a dry run of the upgrade with OOTB installation. • Set logging to debug for the following packages • org.alfresco.repo.admin.patch • org.alfresco.repo.domain.patch • Review the logging output #SummitNow #SummitNow
  17. 17. Prepare – Customizations • Only include bare minimum extensions and overrides (Example: content model) • Restore a Prod backup into the extensionRoot #SummitNow #SummitNow
  18. 18. Prepare – Tools for Monitoring • • • Logs • tail, text editor, Splunk JVM • YourKit, JProfiler DB Queries • Vendor-specific tools (MySQL Workbench, SQL Developer, PgAdmin, SQL Server Mgmt Studio) • Quickly toggle log4j logging for java.sql.* #SummitNow #SummitNow
  19. 19. Prepare – Tools for Monitoring • Hardware & OS (CPU, Memory, Disk, etc) • top, nmon, ProcessExplorer #SummitNow #SummitNow
  20. 20. Upgrade Strategy Strategy: Accurately predict how my Alfresco instance will behave throughout the upgrade process. Tactic: Validate Tactic: Monitor Tactic: Execute Tactic: Prepare #SummitNow #SummitNow
  21. 21. Execute • Validate your newly restored instance is working as expected. • Example: 3.3.4 • Shutdown the instance • Startup the instance for the first segment of the upgrade. • Example: 3.3.4 -> 3.3.5 #SummitNow #SummitNow
  22. 22. Upgrade Strategy Strategy: Accurately predict how my Alfresco instance will behave throughout the upgrade process. Tactic: Validate Tactic: Monitor Tactic: Execute Tactic: Prepare #SummitNow #SummitNow
  23. 23. Monitor - Logs • Based on logging output from your dry run, you should be able to determine the overall progress of your upgrade. • What if the logging output appears to be stalled? #SummitNow #SummitNow
  24. 24. Monitor - Logs • Stalled? #SummitNow #SummitNow
  25. 25. Monitor - JVM • Leverage a Java profiling tool such as YourKit • YourKit will tell you exactly which class and methods are being called. • What if it appears that the upgrade has stalled on a given method call? #SummitNow #SummitNow
  26. 26. Monitor – JVM • Still Stalled??? #SummitNow #SummitNow
  27. 27. Monitor – SQL Queries • Use your DB vendor-specific tools to monitor the queries being executed • Don’t have MySQL Workbench, SQL Developer, PgAdmin, or SQL Server Mgmt Studio? • Quickly toggle log4j logger for java.sql.* • You must add java.sql to customlog4j.properties #SummitNow #SummitNow
  28. 28. Upgrade Strategy Strategy: Accurately predict how my Alfresco instance will behave throughout the upgrade process. Tactic: Validate Tactic: Monitor Tactic: Execute Tactic: Prepare #SummitNow #SummitNow
  29. 29. Validate – Smoke Tests • Test basic Alfresco functionality. • Test one or more of your major use cases. • For the latest 4.x versions, leverage WebDrone. #SummitNow #SummitNow
  30. 30. Upgrade Strategy Strategy: Accurately predict how my Alfresco instance will behave throughout the upgrade process. Tactic: Validate Repeat for each segment Tactic: Monitor Tactic: Execute Tactic: Prepare #SummitNow #SummitNow
  31. 31. Gotchas – Cyclical ACL’s • FixAclInheritancePatch could fail with StackOverflowException during 3.4 upgrade. • Added cyclical ACL detection in 3.4.13 • Jira: MNT-2580 #SummitNow #SummitNow
  32. 32. Gotchas – Orphaned Nodes • Orphaned Nodes and Associations to deleted nodes • Reindex could fail with “InvalidNodeRefException: Node does not exist” #SummitNow #SummitNow
  33. 33. Gotchas – Orphaned Nodes • Find orphaned nodes: SELECT n.* FROM alf_node n, alf_qname q, alf_namespace ns where q.id = n.type_qname_id and ns.id = q.ns_id and not (ns.uri = 'http://www.alfresco.org/model/system/1.0' AND q.local_name = 'deleted') AND NOT EXISTS (SELECT 1 FROM alf_store where root_node_id = n.id) AND NOT EXISTS (SELECT 1 FROM alf_child_assoc where child_node_id = n.id) #SummitNow #SummitNow
  34. 34. Gotchas – Orphaned Nodes • Find associations to deleted nodes: SELECT a.* FROM alf_child_assoc a, alf_node n, alf_qname q, alf_namespace ns where (n.id = a.child_node_id or n.id = a.parent_node_id) and q.id = n.type_qname_id and ns.id = q.ns_id and ns.uri = 'http://www.alfresco.org/model/system/1.0' AND q.local_name = 'deleted' #SummitNow #SummitNow
  35. 35. Gotchas – Orphaned Nodes • CAUTION: Do not attempt to fix these by hacking the database! • Detection and fix via Java has been added as of 4.1.4 • Jira: MNT-8531 #SummitNow #SummitNow
  36. 36. Gotchas – Nodes without Txn’s • May cause schema updates to fail with *ContraintViolationExceptions. #SummitNow #SummitNow
  37. 37. Gotchas – Nodes without Txn’s • Find nodes without transactions: SELECT * FROM alf_node node WHERE node.transaction_id NOT IN ( SELECT id FROM alf_transaction ); #SummitNow #SummitNow
  38. 38. Gotchas – Nodes without Txn’s • CAUTION: Do not alter the database without the consent or the assistance of Alfresco Support or Alfresco Consulting. • Fix: UPDATE alf_node set transaction_id = (SELECT MIN(id) from alf_transaction) WHERE transaction_id NOT IN (SELECT id FROM alf_transaction); #SummitNow #SummitNow
  39. 39. Gotchas – Long-running Patches • MigrateVersionStorePatch (2.x  3.x) • Run asynchronously as a scheduled job # Never: * * * * * ? 2099 # Once every thirty minutes: 0 0/30 * * * ? version.store.migrateVersionStore.cronExpression=* * * * * ? 2099 # Limit number of version histories to migrate per job cycle, where -1 = unlimited. Note: if limit > 0 then need to schedule job to run regularly in order to complete the migration. version.store.migrateVersionStore.limitPerJobCycle=-1 version.store.migrateVersionStore.runAsScheduledJob=false #SummitNow #SummitNow
  40. 40. Gotchas – Long-running Patches • ContentUrlConverterPatch (...  3.2.x) • Run asynchronously as a scheduled job # Never: * * * * * ? 2099 # Once every thirty minutes: 0 0/30 * * * ? system.content.contentUrlConverter.cronExpression=* * * * * ? 2099 system.content.contentUrlConverter.threadCount=2 system.content.contentUrlConverter.batchSize=500 system.content.contentUrlConverter.runAsScheduledJob=false #SummitNow #SummitNow
  41. 41. Gotchas – Long-running Patches • NodeDeleted.sql (…  4.1.x) • Renames alf_node to t_alf_node • Recreate the alf_node table without the node_deleted column • Insert all rows from t_alf_node into alf_node • Drop the temporary t_alf_node table #SummitNow #SummitNow
  42. 42. Gotchas – Database Statistics • Don’t forget to recompute your database statistics! • Without histograms if you’re using Oracle #SummitNow #SummitNow
  43. 43. Summary • Be meticulous! • Have attention to detail! • Leave nothing up to mystery! • There should be no surprises. • “Know thy self, know thy enemy. A thousand battles, a thousand victories” • [1] Sun Tzu, The Art of War [1] #SummitNow #SummitNow
  44. 44. Summary • “The supreme art of war is to subdue the enemy without fighting.” [1] • [1] Sun Tzu – The Art of War #SummitNow #SummitNow
  45. 45. Contact Information • • • • Kyle Adams Solutions Engineer at Alfresco Email: kyle.adams@alfresco.com Twitter: @kylefadams #SummitNow #SummitNow
  46. 46. #SummitNow

×