1. APACHE SLING&FRIENDSTECHMEETUP
2- 4SEPTEMBER 2019
Deep-diveinto cloud-native AEM deployments based on Kubernetes
Tomek Rękawek, Adobe

2. 2
Cloud: whatand why?

3. Wewanttomovefrom
3

4. to
4

5. Cloud:why?
5
 RunningAEM at scale
 Hassle-freedeployments
 The cloud provider (AWS, Azure) should worry about
infrastructure

6. Agenda
6
 Jobs
 Content migration
 New indexes
 QA
 Docker&Kubernetesintroduction
 Architectureoverview
 Publishpersistence
 Cloud Segment Store
 Golden publish
 Compaction
 Sidecarservices

7. 7
Docker & Kubernetesintroduction

8. Thepowerofstandardization
8

9. DockerizedAEM
9
 AEM in Docker image
 Composite Node Store
 /apps& /libs storedin thecontainer
 Actualcontentlives outside,in the VOLUMEorMongoDB
 OSGi Feature Model
 DefinesAEMandcustomerapplication
 Featurelauncherstartsit inside container
 Covered in Karl& David’spresentation
 SeeadaptTo() 2017 talk for moredetails

10. Mini intro toKubernetes
10
 Launches Docker containers without worrying about the underlying VMs
 Dictionary
 Pod–a groupofcontainersstartingtogetheron asingle machine
 Service –internalloadbalancer,exposinganumberofpodreplicas underasingle address
 Ingress– exposesa service underanpublic httpaddress
 Everyentity is represented by a YAML object in K8s API server
 TheYAML is the desired state, K8s knows how to get there

11. DeploymentswithHelm
11
 Helm – K8s apps management
 Chart – a bunch of K8s YAML descriptors,
with a simple templating
 Whole AEMdeployment canbe
installed/upgraded with a single command
Image:https://helm.sh

12. 12
Architecture overview

13. AEMKubernetessetup
13

14. 14
Publishpersistence evolution

15. Problemdefinition
15
 Author persistence is easy-ish with MongoDB
 Publish is harder –local SegmentMK, no clustering
 Thepublish farm is kept up-to-date with replication
 However:
 weneed toprovidethenew publishinstanceswitha segment store,
 copyit fromanotherinstance.
 Problems:
 copyingfiles betweenpodsis hardandhacky,
 whatif there’snopublishto copyfrom?

16. Cloud SegmentStore
16
 A new plugin for the Segment Node Store
 Nodes are stored in a cloud storage service
 No tar files, raw segments grouped in dirs
 Can be used in RW or RO modes

17. Goldenpublish
17
 A designated publish instance
 Not connectedto LB
 It maintainsa “golden copy” of the
segmentstore
 New publish just clone it

18. Problem:duplicatedbinaries andstartuptime
18
Golden publish
Segment 1
Segment 2
Segment 4
Segment 3
Segment x
Publish 1
Segment 1
Segment 2
Segment 4
Segment 3
Segment y
Publish 2
Segment 1
Segment 2
Segment 4
Segment 3
Segment z
Publish 3 (starting)
Segment 1
Segment 2
Segment 3
Segment 4
 Multiple copies of the same segments 1-4 ($$$)
 Cloning a bucket takes time during the publish start ( 🕑🕑🕑)

19. Optimization:a singlesegmentstore
19

20. 20
Publishpersistence: compaction

21. Compaction
21

22. Out-of-bandpublish update
22
 This pattern willbe usefulin many cases
 We may clone thepublish repository,
modify it and re-deploy instanceson
top of it
 Persisted message queuewill apply
missingchanges

23. 23
Sidecar services

24. Sidecar approach
24
 A single pod can run many containers, sharing their volumes and
localhost interface
 We can use them for the auxiliaryservices (sidecars):
 Dispatcher in the publish pod
 Upload logs to Splunk
 Warmup service

25. AEMpod withsidecars
25Sidecaricon: Flaticons.com

26. 26
Jobs

27. Kubernetesjob
27
 Starts a pod
 Meantto perform a specific task and finish
 Unlikethe deployment, which run indefinitely
 Willbe restarted if fails
 Jobs provides a way to interactwiththe deployed AEM

28. 28
Contentmigration

29. Contentmigration
29
 How to migrate old content to K8s?
 Access problem
 old AEM envs shouldn’t have access to the K8s (encapsulation)
 K8s shouldn’t be able to access oldAEMs (they can be installed
anywhere)
 Solution:demilitarized zone

30. 2-phasemigration
30
 Phase 1
 Themigrator tool (crx2oak-like jar) is used to
export old AEM content into a cloud storage
service
 Phase 2
 A Kubernetes job is used to apply the migrated
content on the cloud instances

31. 31
Newindexes

32. Addingnew index
32
 Indexesaretricky
 /oak:index is a part of the mutable content
 But indexdefinitions belongs to the application
 Onlyaddingnew indexesis supported
 Whena new indexis addedin /oak:index,it’ll haveanextra
useIfExistspropertyreferencing immutable part/apps
 This bounds the index definition tothe application version andDocker
image
 This /appspathhaveto beaddedas well
• /oak:index/my-new-index
• useIfExists: /apps/indexes@v3
• /apps/indexes
• v3: true
Mutable part
Immutable part

33. Mutablecontent:addingnew index
33
 Indexingjob should be runbefore the actual app deployment
 Thejob will:
1. Look forthe new index defs in /oak:index
2. Perform out of band indexing ofthe content
3. Save the new indexing content tothe production repository
 TheuseIfExists will makesure that the newindexis ignored, untilthe
newimageis installed
 When the new image is deployed, the /apps part will be updated and the
new index will beused

34. 34
Other topics

35. Relatedtopics
35
 Feature model usage in Docker (covered in David’s and Karl’stalk)
 Replication (covered in Timothee’s talk)
 Monitoring with Prometheus and Grafana
 CI/CD pipeline
 Network policies
 …

36. 36
Thanks!