Foundation APIs and Repository Internals


Published on

In this session we will start by examining some of features that developers have available when using the Foundation service interfaces: how to initiate and use transactions; how and when to make use of transactional resources; using different types of search; controlling behaviors e.g. 'cm:auditable'; changing CopyService behavior. Following this, some repository internals will be examined, including: typical content lifecycles and parameters that control these; schema generation files, upgrade scripts and runtime SQL (3.4-specific); considerations for large-scale custom data structures.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • 1:00
  • 1:00Focus on development; Possible to pick up details from the Javadocs and proceed; I will give details of supporting infrastructure and considerations that will improve your code.
  • 1:00
  • 1:00
  • ~2:40Later demo: Write batching (demo 1)Note: 3.4
  • ~4:40
  • ~6:45Usually, user-driven changes are usually evenly distributed; more danger from server-initiated process (background processes or logic derived from customer events).
  • 10:30Yellow: Taking these limits other transactionsGreen: No effect on other transactions
  • 18:20Illustrate the difference between Implicit and Explicit transactions and why developers should care about themWill see how cost of continuing txn is negligible
  • 21:20
  • 23:45Demo 0Fewer CPU cycles
  • 29:45Demo 1Effects of index.tracking.disableInTransactionIndexing: that Mike Farman and I will be showing
  • 34:00Demo 2Can go into optimistic locking, but it is less prevalent on 3.4 as writes occur immediately, etc.
  • 43:00Leave non-propagating alone unless ...Use read-only for all read scenarios. If your code is writing in the reads, then change it. In an emergency, start a new transaction (non-propagating, but should use conditional logic.Will discuss an example later.
  • 47:00
  • Foundation APIs and Repository Internals

    1. 1. Alfresco Repository Internals<br />1<br />Derek Hulley<br />Senior Engineer, Alfresco<br />twitter: @derekhulley<br />
    2. 2. Alfresco Repository Internals<br />2<br />Resource Contention<br />Node Creation and Modification<br />Actions<br />Scheduled Jobs<br />Transactions<br />Resources<br />Implicit and Explicit<br />Using Alfresco’s Transaction Support<br />Agenda (1)<br />
    3. 3. Alfresco Repository Internals<br />3<br />Navigating Hierarchies<br />Lucene-based APIs<br />NodeService-based APIs<br />Walking Child Associations<br />Policies and Behaviour<br />Policy Behaviour Filters<br />CopyService<br />Agenda (2)<br />
    4. 4. Alfresco Repository Internals<br />4<br />Content Lifecycles<br />ContentData Properties<br />Binary Files and Transactions<br />Orphaned Content<br />System Properties<br />Application Bootstrap<br />Spring init<br />Lifecycle Classes<br />Modules<br />Questions<br />Agenda (3)<br />
    5. 5. Resource Contention<br />5<br />Node Creation<br /><ul><li>Row inserts
    6. 6. Read-committed isolation: Invisibleuntil commit
    7. 7. Database, Caches, Lucene Indexes: Low contention</li></li></ul><li>Resource Contention<br />6<br />Node Modification<br /><ul><li>Update type, aspects, properties, ACLs; Move; Delete; etc
    8. 8. Invisible until commit, but can hold resource locks
    9. 9. Transactions rejected e.g. ConcurrencyFailureException</li></li></ul><li>Resource Contention<br />7<br />Actions and Scheduled Jobs<br /><ul><li>Danger of background jobs moving ‘up’ a hierarchy</li></ul>L1N1<br />?<br />L2N1<br />L2N2<br />L3N1<br />L3N1<br />Only one winner (at a time)<br /><ul><li>Individual node modifications are serialized
    10. 10. Pick up small junks, commit and give way</li></li></ul><li>Transactions: Resources<br />8<br /><ul><li>Database row locking
    11. 11. ‘version’ column: optimistic locking
    12. 12. Each transaction requires a thread
    13. 13. Possibly multiple transactions on a thread</li></ul>Connection Pool<br />Thread Pools<br /><ul><li>One transaction – one connection
    14. 14. Connection housekeeping has a cost</li></ul>Caches<br />Transaction<br /><ul><li>Index deltas
    15. 15. Heavy on IO</li></ul>Lucene Indexes<br />Content Binaries<br /><ul><li>New content binaries only
    16. 16. Temporary files</li></ul>Database Rows<br /><ul><li>Caches are transaction-aware
    17. 17. Conflicts drop cache entries</li></ul>It pays to think about resource contention.<br />Replaying transactions means:<br /><ul><li>Reclaiming resources
    18. 18. CPU cycles
    19. 19. Lower response times</li></li></ul><li>Transactions<br />9<br />Implicit<br /><ul><li>Defined against public service (Foundation) APIs
    20. 20. Bean naming convention: NodeServicevsnodeService
    21. 21. Cost is in starting a transaction, not continuing one
    22. 22. public-services-context.xml and ServiceRegistry
    23. 23. Spring customization and interceptors</li></li></ul><li>Transactions<br />10<br />Explicit<br /><ul><li>Wrap all atomic operations including groups of reads
    24. 24. Use RetryingTransactionHelper rather than UserTransaction</li></li></ul><li>Transactions<br />11<br />Explicit: Demo: Read-only Batching<br /><ul><li> Time lost to transaction initiation
    25. 25. 3ms lost per low-level operation ... How much per user click?</li></ul>Get Stores<br />Get Children<br />Lucene Query<br />
    26. 26. Transactions<br />12<br />Explicit: Demo: Write Batching<br /><ul><li> Time lost to initiate transactions. Well, yes, but ...
    27. 27. Unnecessary additional indexing</li></ul>CIFS and FTP<br />index<br />Create Node<br />Get Writer<br />index<br />Add Content<br />
    28. 28. 1<br />Transactions<br />13<br />Direct Contention: Demo<br />begin<br />0<br />0<br />1<br />1<br />1<br />1<br />1<br />0<br />0<br />0<br /><br />Eg: Retrying OptimisticLockingDemo-8: count 0; wait: 0.1s; msg: "Failed to update node 14575"; exception: ...ConcurrencyFailureException<br />
    29. 29. Transactions<br />14<br />Alfresco Transaction Support<br /><ul><li>RetryingTransactionHelper
    30. 30. Reliable, resolves contention
    31. 31. “non propagating”: Transaction suspended and a new one started.
    32. 32. TransactionService.getRetryingTransactionHelper -> your instance
    33. 33. TransactionalListener and TransactionListenerAdapter
    34. 34. Bind to events associated with a transaction
    35. 35. AlfrescoTransactionSupport
    36. 36. Helper around Spring’s TransactionSynchronizationManager
    37. 37. getTransactionReadState: Allows logic conditional on the state of the transaction.
    38. 38. bindResource and getResource: Bind objects to current transaction. This is like ThreadLocal but is safer i.e. Resources are bound to the transaction and go away when the transaction is terminated.
    39. 39. TransactionalResourceHelper
    40. 40. <K,V> getMap(Object resourceKey), etc: Helper to get transactionally-bound collections.</li></li></ul><li>Navigating Hierarchies<br />15<br />Lucene-Driven APIs<br /><ul><li>SearchService.query()
    41. 41. Versatile
    42. 42. Not always transactionally consistent
    43. 43. Cluster: Transactions are replayed for indexes</li></ul>SQL-Driven APIs<br /><ul><li>SearchService.selectNodes()
    44. 44. Fast for simple path-based searches only: e.g.:“/app:company_home/app:data_dictionary”
    45. 45. Always consistent!
    46. 46. NodeService.*
    47. 47. Use for consistent views
    48. 48. FileFolderService.*
    49. 49. Limited to cm:folder-based lookups
    50. 50. Fast lookups on cm:name (via NodeService)</li></li></ul><li>Navigating Hierarchies<br />16<br />Walking Child Associations<br /><ul><li>Hierarchy traversal is fast if the child associations can be isolated.
    51. 51. Put the correct data into createNode</li></ul>P1<br /><ul><li>No uniqueness on path QName, but can be used to trim results to a meaningful few.
    52. 52. Use type QName for better search selectivity.
    53. 53. Use child associations that have <duplicate>false</duplicate> to enforce uniqueness (cm:name) and use FileFolderService.</li></ul>Child Association indexes:<br /><ul><li> Parent and child
    54. 54. typeQName
    55. 55. qname
    56. 56. Also unique cm:name</li></ul>N2<br />
    57. 57. Policies and Behaviour<br />17<br />Policy Behaviour Filters<br /><ul><li>Temporarily disable polices
    58. 58. Bound to the currenttransaction
    59. 59. Bean: “behaviourFilter”</li></ul>cm:versionable<br /><ul><li> Prevent a change from forcing a new version
    60. 60. Force versioning on metadata change: cm:autoVersionOnUpdateProps</li></ul>cm:auditable<br /><ul><li> Manually set cm:modified date
    61. 61. Prevent cm:modified from being recorded</li></li></ul><li>Alfresco Repository Internals<br />18<br />Application Bootstrap<br />Server Startup<br /><ul><li>DDL script execution: lock table
    62. 62. Spring init(): no resources available
    63. 63. Alfresco Bootstrap: order of AbstractLifecycleBean
    64. 64. Module Bootstrap: module startup on AbstractModuleComponent</li></li></ul><li>Q & A<br />19<br />