Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data replication in Sling

2,173 views

Published on

Overview of Sling Replication module with usecases and design prinicples

Published in: Technology
  • Be the first to comment

Data replication in Sling

  1. 1. APACHE SLING & FRIENDS TECH MEETUP BERLIN, 22-24 S EPTEMBER 2014 Data replication in Sling Tommaso Teofili, Adobe Systems
  2. 2. Use cases adaptTo() 2014 2
  3. 3. Replication in AEM § Moving authored content to publish servers author publish adaptTo() 2014 3
  4. 4. Replication in AEM § Moving user generated content back to author for moderation author publish adaptTo() 2014 4
  5. 5. Replication in AEM § Moving user generated content to other publish instances author publish publish adaptTo() 2014 5
  6. 6. Replication module should ... § transfer resources between Sling instances § push from server A to server B § pull from server C to server D adaptTo() 2014 6
  7. 7. Example: replication of /foo/bar § HTTP request for pushing /foo/bar § Resources get § packaged § sent § received § persisted adaptTo() 2014 7
  8. 8. we’re not in 1999 ... anymore adaptTo() 2014 8
  9. 9. NRT sync on N servers publish1 publish2 author1 author2 publish3 publish4 publish5 adaptTo() 2014 9
  10. 10. Oak instances served by same MongoDB publish1 author publish2 oak oak mongo mongo mongo mongo p1 or p2 adaptTo() 2014 10
  11. 11. Cloud based infrastructures P1 to pN publish1 author1 author2 publish2 publish3 publish4 publish4 publish4 publish4 publish4 publishN adaptTo() 2014 11
  12. 12. Overview adaptTo() 2014 12
  13. 13. Sling Replication § Contributed to Sling in November 2013 § First release ? § Main goals § Simple § Resilient § Fast adaptTo() 2014 13
  14. 14. Replication agents § Execute replication requests by: § exporting replication packages from a (remote) Sling instance § importing replication packages into a (remote) Sling instance adaptTo() 2014 14
  15. 15. Replication agents § A “push” agent has § A “local” exporter § Creating a package locally for the resources to be replicated (e.g. from the underlying JCR repo) § A “remote” importer § Importing the exported package remotely by sending it to a designated endpoint to persist it adaptTo() 2014 15
  16. 16. Replication agents § A “pull” agent has § A “remote” exporter § Pulling a package from a remote Sling instance designated endpoint § A “local” importer § Importing the package locally by persisting it into the Sling instance adaptTo() 2014 16
  17. 17. Replication agents § A “coordinating” agent has § A “remote” exporter § Pulling a package from a remote endpoint § A “remote” importer § Importing the package remotely into a Sling instance adaptTo() 2014 17
  18. 18. Replication agents § A “queuing” agent has § A “local” exporter § Creating a package locally for the resources to be replicated (e.g. from the underlying JCR repo) § No importer adaptTo() 2014 18
  19. 19. Replication package serialization § Payload to be sent / received § Package builders for (de)serialization § Jackrabbit FileVault based package builder adaptTo() 2014 19
  20. 20. Replication queues § Multiple queue providers § Sling Jobs based (sling.event bundle) § In memory § Multiple queue distribution strategies § Single § Error aware § Priority adaptTo() 2014 20
  21. 21. Replication rules § Rules can be defined within agents § To trigger replications upon resource changes § To schedule periodic replications § To chain replicate § ... adaptTo() 2014 21
  22. 22. Accessing replication resources § Agents as OSGi services § Can be defined via ConfigurationAdmin § Resource providers control § Service access § CRUD operations on configs adaptTo() 2014 22
  23. 23. Agent configuration example (1/3) § Config for a ‘push agent’ { "jcr:primaryType" : "sling:OsgiConfig", "name" : "publish", "type" : "simple", "packageExporter": [ "type=local”, "packageBuilder/type=vlt", ... ], ... adaptTo() 2014 23
  24. 24. Agent configuration example (2/3) ... "packageImporter" : [ "type=remote", "endpoints[0]=http://.../replication/services/importers/default”, "authenticationFactory/type=service", "authenticationFactory/name=user", ... "packageBuilder/type=vlt", ... ], ... adaptTo() 2014 24
  25. 25. Agent configuration example (3/3) ... "queueProvider" : [ "type=service", "name=sjh" ], "queueDistributionStrategy" : [ "type=service", "name=error" ] } adaptTo() 2014 25
  26. 26. Anatomy of a forward replication request adaptTo() 2014 26
  27. 27. Anatomy of a fwd. replication (1/4) § HTTP POST on /libs/sling/replication/ service/agents/publish with form parameters § Action = ADD § Path = /content/replication § Agent with name ‘publish’ gets picked up by the agent resource provider adaptTo() 2014 27
  28. 28. Anatomy of a fwd. replication (2/4) § Agent ‘publish’ calls its Exporter (local) to create the payload § the ‘local’ exporter calls its Package builder § the ‘vlt’ package builder creates a FileVault package for resources under /foo/bar adaptTo() 2014 28
  29. 29. Anatomy of a fwd. replication (3/4) § Agent ‘publish’ dispatches the package to its queue provider and distribution algorithm § The queue provider is asked to provide queues depending on the distribution strategy § The request gets queued adaptTo() 2014 29
  30. 30. Anatomy of a fwd. replication (4/4) § When queue entry gets processed the (remote) importer is called § Package sent over the wire to an endpoint bound to a local importer on the receiving server § The local importer on the receiving side deserializes and persists the replication package via its package builder adaptTo() 2014 30
  31. 31. Reverse replication § Pull agent on author § Periodically polling § Queuing agent on publish adaptTo() 2014 31
  32. 32. Event based reverse replication § No scheduled polling § The publish instance notifies author when to pull § Notification through server sent events adaptTo() 2014 32
  33. 33. Looking forward § Get information on other instances § e.g. replicate to all instances with run mode ‘xyz’ § NRT publish sync § Coordinate agents++ § Performance adaptTo() 2014 33
  34. 34. Thanks! § Questions? adaptTo() 2014 34

×