Foundation APIs and Repository Internals

  • 1,185 views
Uploaded on

In this session we will start by examining some of features that developers have available when using the Foundation service interfaces: how to initiate and use transactions; how and when to make use …

In this session we will start by examining some of features that developers have available when using the Foundation service interfaces: how to initiate and use transactions; how and when to make use of transactional resources; using different types of search; controlling behaviors e.g. 'cm:auditable'; changing CopyService behavior. Following this, some repository internals will be examined, including: typical content lifecycles and parameters that control these; schema generation files, upgrade scripts and runtime SQL (3.4-specific); considerations for large-scale custom data structures.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,185
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
64
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • 1:00
  • 1:00Focus on development; Possible to pick up details from the Javadocs and proceed; I will give details of supporting infrastructure and considerations that will improve your code.
  • 1:00
  • 1:00
  • ~2:40Later demo: Write batching (demo 1)Note: 3.4
  • ~4:40
  • ~6:45Usually, user-driven changes are usually evenly distributed; more danger from server-initiated process (background processes or logic derived from customer events).
  • 10:30Yellow: Taking these limits other transactionsGreen: No effect on other transactions
  • 18:20Illustrate the difference between Implicit and Explicit transactions and why developers should care about themWill see how cost of continuing txn is negligible
  • 21:20
  • 23:45Demo 0Fewer CPU cycles
  • 29:45Demo 1Effects of index.tracking.disableInTransactionIndexing: that Mike Farman and I will be showing
  • 34:00Demo 2Can go into optimistic locking, but it is less prevalent on 3.4 as writes occur immediately, etc.
  • 43:00Leave non-propagating alone unless ...Use read-only for all read scenarios. If your code is writing in the reads, then change it. In an emergency, start a new transaction (non-propagating, but should use conditional logic.Will discuss an example later.
  • 47:00

Transcript

  • 1. Alfresco Repository Internals
    1
    Derek Hulley
    Senior Engineer, Alfresco
    twitter: @derekhulley
  • 2. Alfresco Repository Internals
    2
    Resource Contention
    Node Creation and Modification
    Actions
    Scheduled Jobs
    Transactions
    Resources
    Implicit and Explicit
    Using Alfresco’s Transaction Support
    Agenda (1)
  • 3. Alfresco Repository Internals
    3
    Navigating Hierarchies
    Lucene-based APIs
    NodeService-based APIs
    Walking Child Associations
    Policies and Behaviour
    Policy Behaviour Filters
    CopyService
    Agenda (2)
  • 4. Alfresco Repository Internals
    4
    Content Lifecycles
    ContentData Properties
    Binary Files and Transactions
    Orphaned Content
    System Properties
    Application Bootstrap
    Spring init
    Lifecycle Classes
    Modules
    Questions
    Agenda (3)
  • 5. Resource Contention
    5
    Node Creation
    • Row inserts
    • 6. Read-committed isolation: Invisibleuntil commit
    • 7. Database, Caches, Lucene Indexes: Low contention
  • Resource Contention
    6
    Node Modification
    • Update type, aspects, properties, ACLs; Move; Delete; etc
    • 8. Invisible until commit, but can hold resource locks
    • 9. Transactions rejected e.g. ConcurrencyFailureException
  • Resource Contention
    7
    Actions and Scheduled Jobs
    • Danger of background jobs moving ‘up’ a hierarchy
    L1N1
    ?
    L2N1
    L2N2
    L3N1
    L3N1
    Only one winner (at a time)
    • Individual node modifications are serialized
    • 10. Pick up small junks, commit and give way
  • Transactions: Resources
    8
    • Database row locking
    • 11. ‘version’ column: optimistic locking
    • 12. Each transaction requires a thread
    • 13. Possibly multiple transactions on a thread
    Connection Pool
    Thread Pools
    • One transaction – one connection
    • 14. Connection housekeeping has a cost
    Caches
    Transaction
    • Index deltas
    • 15. Heavy on IO
    Lucene Indexes
    Content Binaries
    • New content binaries only
    • 16. Temporary files
    Database Rows
    • Caches are transaction-aware
    • 17. Conflicts drop cache entries
    It pays to think about resource contention.
    Replaying transactions means:
    • Reclaiming resources
    • 18. CPU cycles
    • 19. Lower response times
  • Transactions
    9
    Implicit
    • Defined against public service (Foundation) APIs
    • 20. Bean naming convention: NodeServicevsnodeService
    • 21. Cost is in starting a transaction, not continuing one
    • 22. public-services-context.xml and ServiceRegistry
    • 23. Spring customization and interceptors
  • Transactions
    10
    Explicit
    • Wrap all atomic operations including groups of reads
    • 24. Use RetryingTransactionHelper rather than UserTransaction
  • Transactions
    11
    Explicit: Demo: Read-only Batching
    • Time lost to transaction initiation
    • 25. 3ms lost per low-level operation ... How much per user click?
    Get Stores
    Get Children
    Lucene Query
  • 26. Transactions
    12
    Explicit: Demo: Write Batching
    • Time lost to initiate transactions. Well, yes, but ...
    • 27. Unnecessary additional indexing
    CIFS and FTP
    index
    Create Node
    Get Writer
    index
    Add Content
  • 28. 1
    Transactions
    13
    Direct Contention: Demo
    begin
    0
    0
    1
    1
    1
    1
    1
    0
    0
    0
    log4j.logger.org.alfresco.repo.transaction.RetryingTransactionHelper=warn
    Eg: Retrying OptimisticLockingDemo-8: count 0; wait: 0.1s; msg: "Failed to update node 14575"; exception: ...ConcurrencyFailureException
  • 29. Transactions
    14
    Alfresco Transaction Support
    • RetryingTransactionHelper
    • 30. Reliable, resolves contention
    • 31. “non propagating”: Transaction suspended and a new one started.
    • 32. TransactionService.getRetryingTransactionHelper -> your instance
    • 33. TransactionalListener and TransactionListenerAdapter
    • 34. Bind to events associated with a transaction
    • 35. AlfrescoTransactionSupport
    • 36. Helper around Spring’s TransactionSynchronizationManager
    • 37. getTransactionReadState: Allows logic conditional on the state of the transaction.
    • 38. bindResource and getResource: Bind objects to current transaction. This is like ThreadLocal but is safer i.e. Resources are bound to the transaction and go away when the transaction is terminated.
    • 39. TransactionalResourceHelper
    • 40. <K,V> getMap(Object resourceKey), etc: Helper to get transactionally-bound collections.
  • Navigating Hierarchies
    15
    Lucene-Driven APIs
    • SearchService.query()
    • 41. Versatile
    • 42. Not always transactionally consistent
    • 43. Cluster: Transactions are replayed for indexes
    SQL-Driven APIs
    • SearchService.selectNodes()
    • 44. Fast for simple path-based searches only: e.g.:“/app:company_home/app:data_dictionary”
    • 45. Always consistent!
    • 46. NodeService.*
    • 47. Use for consistent views
    • 48. FileFolderService.*
    • 49. Limited to cm:folder-based lookups
    • 50. Fast lookups on cm:name (via NodeService)
  • Navigating Hierarchies
    16
    Walking Child Associations
    • Hierarchy traversal is fast if the child associations can be isolated.
    • 51. Put the correct data into createNode
    P1
    • No uniqueness on path QName, but can be used to trim results to a meaningful few.
    • 52. Use type QName for better search selectivity.
    • 53. Use child associations that have <duplicate>false</duplicate> to enforce uniqueness (cm:name) and use FileFolderService.
    Child Association indexes:
    N2
  • 57. Policies and Behaviour
    17
    Policy Behaviour Filters
    • Temporarily disable polices
    • 58. Bound to the currenttransaction
    • 59. Bean: “behaviourFilter”
    cm:versionable
    • Prevent a change from forcing a new version
    • 60. Force versioning on metadata change: cm:autoVersionOnUpdateProps
    cm:auditable
    • Manually set cm:modified date
    • 61. Prevent cm:modified from being recorded
  • Alfresco Repository Internals
    18
    Application Bootstrap
    Server Startup
    • DDL script execution: lock table
    • 62. Spring init(): no resources available
    • 63. Alfresco Bootstrap: order of AbstractLifecycleBean
    • 64. Module Bootstrap: module startup on AbstractModuleComponent
  • Q & A
    19