The document outlines strategies for improving developer experience in cloud-native environments by achieving consistency between local and production setups. It discusses the twelve-factor app methodology, various tools like Skaffold and ksync for synchronizing code and deployments, and techniques for efficient debugging and source reloading. Overall, it emphasizes the importance of maintaining parity across environments to streamline the development and deployment process.

1. Developer Experience Cloud Native
-
Become Efficient and Achieve Parity
Michael Hofmann
https://hofmann-itconsulting.de

2. Developer Experience Cloud Native
create code
Docker build
& push
deploy to
(local) K8S
test & debug
remote
Git push
build & test
(local)

3. The Twelve-Factor App
“methodology for building software-as-a-service apps that”:
● “Use declarative formats for setup automation, to minimize time and cost for new
developers joining the project”;
● “Have a clean contract with the underlying operating system, offering maximum
portability between execution environments”;
● “Are suitable for deployment on modern cloud platforms, obviating the need for
servers and systems administration”;
● “Minimize divergence between development and production, enabling
continuous deployment for maximum agility”;
● “And can scale up without significant changes to tooling, architecture, or
development practices”.
https://12factor.net

4. The Twelve-Factor App
I. Codebase
One codebase tracked in revision control, many
deploys
II. Dependencies
Explicitly declare and isolate dependencies
III. Config
Store config in the environment
IV. Backing services
Treat backing services as attached resources
V. Build, release, run
Strictly separate build and run stages
VI. Processes
Execute the app as one or more stateless processes
VII. Port binding
Export services via port binding
VIII. Concurrency
Scale out via the process model
IX. Disposability
Maximize robustness with fast startup and graceful
shutdown
X. Dev/prod parity
Keep development, staging, and production as similar as
possible
XI. Logs
Treat logs as event streams
XII. Admin processes
Run admin/management tasks as one-off processes

5. Deviation in Parity: local vs cloud
● operation system
● JDK version
● environment
– ENV vars
– volume mounts
– DNS (myservice.mynamespace.svc.cluster.local)
● backing service (single server / cluster)
(e.g. Redis: SingleServerConfig, ClusterServersConfig)
● Java SecureRandom: in cloud too slow (missing entropy with isolated
CPUs)

6. The Quest:
How to get
new source code
quick
into container or pod
to keep
parity?

7. Possible Solutions
● synchronous deployment in Kubernetes
● source reload in Kubernetes pod
● swap deployment of Kubernetes pod
● for completeness: source reload in local container
– no focus in this session
– to many manual tasks necessary (see summary)

8. Sync Deployment: Skaffold
● handles the workflow for
– building (application and image)
– pushing to registry
– deployment to Kubernetes
● keeps artifacts in sync
● workflow automated or manually
https://skaffold.dev

9. Sync Deployment
Demo Skaffold

10. Skaffold Summary
● always full build and deployment cycle
– deactivate JUnits for shorter roundtrip
● remote debug only with manual intervention (pod restart)
● automatic sync of deployment files and Docker images
● cleanup of all (skaffold-ed) resources in cluster
● main usage scenario: keep local and remote in sync
● remote debug is not a first class citizen

11. ● OpenLiberty can reload code on every change
● setting in server.xml
<applicationManager autoExpand=”true”>
● on file basis (app.war) or on individual class files
● concept: sync local class files to remote pod for
reload in running app server
● remote debug with JDWP (java debug wire protocol)
Source Reload: OpenLiberty/Ksync

12. Ksync
● installs DaemonSet in K8s cluster
● works bi-directional
● local watch process keeps local folder and pod
folder in sync
● new pods will also be synced (scale --replicas=2)
https://github.com/ksync/ksync

13. OpenLiberty/Ksync Summary
● full parity: code evolution inside running pod
● roundtrip takes only 1-2s
● JDWP is well-engineered
● stable debug connection (no pod restart)
● concept can also be applied to other app servers (e.g.
tomcat)
● remove JDWP agent and debug port in production

14. Source Reload: Quarkus Live Coding
● start app (in pod) in remote dev mode:
– package app in mutable jar format
– env setting: QUARKUS_LAUNCH_DEVMODE=true
– same as mvn compile quarkus:dev for local dev mode
● intercepts HTTP requests: redeploys app if new code exists
– redeploy takes less than 1 second (even for larger applications)
● connect local agent to remote host:
mvn quarkus:remote-dev -Dquarkus.live-reload.url=
http://my-remote-host:8080
● sync can be delegated to e.g. Ksync (without url in previous command)

15. Source Reload
Demo Quarkus Live Coding

16. Quarkus Live Coding Summary
● full parity: code evolution inside running pod
● no need for additional tools
● very quick roundtrip (< 1s)
● stable debug connection (JDWP)
● do not use dev-mode in production
– app must be repackaged for production
– env setting and debug port must be deleted

17. Service-C (Pod)
Service-B (Pod)
swapped
Service-A (Pod)
Service-B (Pod)
Swap Deployment
local
Kubernetes
Env
Volume
Env
Volume

18. Swap Deployment
● debugging a service mesh
● substitutes K8S pod with a two-way network proxy
● proxies data from K8S environment (e.g., TCP connections,
environment variables, volumes) to local process
● local process has its networking transparently overridden:
DNS calls and TCP connections are routed to K8S
● Bridge to Kubernetes or Telepresence
https://marketplace.visualstudio.com/items?itemName=mindaro.mindaro
https://www.telepresence.io

19. Swap Deployment
Demo Bridge to Kubernetes

20. Swap Deployment Summary
● parity broken only on OS or JDK (env, mount, DNS parity)
● local development with local debugging
● no manipulation of image or settings necessary
● Non root account: can be set but a little complicated (Docker or env-settings)
● individual traffic
– Telepresence additional service (commercial)
– Bridge to Kubernetes out-of-the-box
● main usage scenarios: develop and debug service mesh local with nearly
full parity

21. Tool OS/JDK ENV DNS Mount Debug Efficient
Source Reload
(Local Container)
mvn
liberty:devc
manually
--env
kubefwd
manually
--mount
Remote
Sync Deployment Skaffold Remote
Source Reload
(Kubernetes Pod)
OpenLiberty/
Ksync
Quarkus Live
Coding
Remote
Swap Deployment
Bridge to
Kubernetes
telepresence
Local
Summary