Instant developer onboarding with self contained repositories

Instant Self-contained Development
Environments for Everyone
Yshay Yaacobi
CTO @ Livecycle
yshay@livecycle.io
yshayy

@yshayy
About me
● CTO & co-founder of livecycle
● Full-stack developer for quite a time
● Passionate about cloud development,
backend architecture, UX/DX, functional
programming and Docker
● Creator & maintainer of Tweek - an open-source, “cloud-
native” feature management solution
● Things I care about in code: simplicity, consistency and elegance

@yshayy
About Livecycle
● Next generation collaboration tools for development teams
● Continuous playground environments
● Bridging the gap between coders & non-coders
● Soon in public beta

How does it feel like to start working on a new
complex codebase?

@yshayy
Try to build & run
● Wrong OS
● Missing or conflicting SDKs and/or PL runtimes
● Package managers throw “random” errors

@yshayy
Read README.md and try to make it work
● Run some magic scripts (and watch them fails)
● Change hosts file
● Install required tools & dependencies on the environment
● Install Root CA?

@yshayy
Try to develop
● Debugging doesn’t work
● IDE have problem with autocomplete or dependencies
● Code watch & build doesn’t work and we need watchman
● HMR doesn’t work because of websocket issues
● Problems with external dependencies
● CORS

Do it all over again after few months of working
on something else....

@yshayy
Why is it so difficult?
● Different OSes (or versions)
● Lots of versions and fragmentation of sdk & runtimes
● Work on my machine syndrome
● Vast amount of different toolchains, IDEs, extensions
● Complex development flows that are difficult to setup and easily break:
○ Debugging, watching+building, hot-reloading, docker mounts
● Developers’ machines are polluted and overloaded with tools
● Environments and tools tend to change rapidly in active repositories
● ----> Waste of time and frustration!

The dream
Development environments that are...

Consistent
Provide the same predictable experience...

Reproducible
It’s possible to destroy & rebuild them

Isolated
Don’t or get affected by other environments

Self-Contained
All dependencies and tools needed for development
are defined & packaged inside

UNBREAKABLE
We won’t struggle countless hours to get things
working again

@yshayy
Running our environments in containers

@yshayy
Before we begin
● All examples & slides are available on Github
● All tools used in this presentation are OSS or free-to-use projects
● Most examples here are far from bullet-proof and some of them use tools that
can be considered experimental

@yshayy
Example #1 - qeesung/image2ascii
CLI tool for creating ASCII art from images
● Project written in Go
● Before Go modules

@yshayy
The Development Container
● Integration with SCM
● Remote code editing
● Remote terminal

@yshayy
Configuring our environment
● Setting Runtime/SDKS/CLIs
● Setting environment variables and path
● Configure our shell
● Defining extensions

@yshayy
Example #2 - yshayy/email-sender
Simple Flask app to send email based on SendGrid example
New challenges
● Running & interacting with a server
● Managing secrets
● Debugging

@yshayy
Secrets Encryption
● Secrets sit inside the repository
● Using Mozilla Sops for encrypting secrets
● GPG keys are nice for start, but it can also integrate with cloud encryptions-as-
service solution such as KMS, KeyVault
● MetaData is saved unencrypted which make it easy to do diffing and check
history
● Practice usually used in GitOps context
● Other solutions - git-secret, git-crypt, Kamus, SealedSecrets, etc...

@yshayy
IDE settings
● Launch settings - launch.json
● Port forwarding - forwarding port on the localhost

@yshayy
#3 HabitRPG/Habitica
OSS Web-based RPG for organizing your life
New Challenges
● Huge project
● Front end
● Backend
● DB

@yshayy
Full-stack application
● Docker-compose
● DB Image
● Additional tools

@yshayy
Data seeding
● Basic scripts
● Alternatively, we can clone data from staging/production

@yshayy
Use reverse proxy
● Route services to several subdomains instead of ports
● Wildcard “localhost” dns-es (localtest.me, xip)
● Traefik - a very simple and developer friendly reverse proxy

@yshayy
#4 - Soluto/Tweek
Cloud-native open-source feature flags and configuration management
New challenges
● Several microservices
● Several DBs/Messaging systems
● Cross service communications
● Polyglot environment

@yshayy
“Nested” containers
● Docker-in-docker vs docker-from-docker
● Development in nested containers with Tilt + Docker-Compose
○ Watching & rebuilding on every code commit
○ Remote debugging
○ “Hot” code reloading if possible
● Things can get slower...

@yshayy
“Mock” cloud dependencies
● Docker images of databases (redis, mongo, postgres, etc…)
● Wire-compatible solutions (Minio, OIDC mock server)
● Manual mocks
● Full frameworks (localstack)
● Encrypted credentials with dedicated tenants. (or dynamic provisioning)

@yshayy
#5 - kubecost/cost-model
Tool for managing kubernetes costs
New challenges
● We need kubernetes
● Metrics server
● Prometheus

@yshayy
What can we do with Kubernetes?
● Kubernetes local development is already difficult
○ Fragmentation - Mini-kube, Docker for Desktop k8s, micro-k8s, kind, k3s, etc…
○ Versioning
○ Upgrading
● Using a single kubernetes distro+version can make life easy

@yshayy
Kubernetes in dev-container
● K3S - minimal kubernetes distribution
● K3D - Make it easy to run k3s and a dedicated registry inside Docker
● Stable and cluster can be re-created
● Helm controller for installing helm charts declaratively
● Tilt facilitate building/pushing/running

@yshayy
Docker Host
Dev Container
IDE
Tilt
Docker-in-Docker
Registry
K3S Node
ContainerD
App
To put it “simply”....

@yshayy
Containerized development environments
● Development environment configuration is also source-controlled and
correspond to the application code.
● Developer machine stays clean
● Can scale well to multiple environments without conflicts
● Can run locally or remotely

@yshayy
Our setup
● ~10 microservices in Golang & typescript/js
● Frontend with HMR
● Our own Kubernetes CR and controllers
● External dependencies (GH api, sendgrid, auth0, etc…)
● DB + Graphql engine
● A full blown CI system
● Container Registry
● Dynamic dns subdomains
● SSL Certs for local development
● Multiple CLI/SDKs for kubernetes and code-generation

@yshayy
Results
● Time to teardown and download/rebuild all cluster and dependencies <15m
● Time to build/run/test code changes < 10s
● Time to onboard new developer < 3h
(including remote provisioning of a dedicated host on AWS)
● Time to introduce new tool and update dev-environments if needed <5m)
● No “works on my machine” occurrences
● No strain on developer machines
● Developers need to deal less with secrets
● Our team work on both M1 and Intel Macs

@yshayy
Future optimizations
● Shared build cache
● Snapshots (for reducing time or sharing state)
● Using a cloud provider optimized for dev-machines (cost, location, hibernation,
cpu/ram, etc...)

@yshayy
Drawbacks
● Creating the initial setup can take some time
● Many tools, some are bleeding-edge
● Additional code to manage
● Dev-Environments are not standardized yet
● Coupling to VS-Code, Docker, Git and Linux
● Some performance issues
● Security challenges between development and production context

@yshayy
Alternatives to VS Code?
● It’s possible to use terminal based code-editors
● GitPod.io & Theia have similar features with gitpod.yaml
● Jetbrains Projector
● Run local IDE with Docker mounts

@yshayy
What about serverless?
● Should work although some pieces might be missing
● FaaS frameworks and sdks usually can run locally
● For the other PaaS features - cloud mocking frameworks, emulators or wire
compatible solutions. (localstack, minio, etc…)
● If necessary, throw IaC tools to the mix (pulumi/terraform) for dynamic
provisioning

@yshayy
What about native mobile?
● The problem still exists, developers can engage in epic battles with mobile IDEs,
workspace, build and debug.
● It might be possible to stream applications with VNC/WebRTC during
development, but experience is not optimal
● Mobile-emulators are heavy and can require nested virtualization to perform
● Container ecosystem is optimized for Linux
● IDEs for mobile are very tailored for mobile development
● Might be easier/possible with more cross-platform frameworks such as react-
native/flutter

@yshayy
Part of a larger trend to put more stuff in the repository
● Linting, style guidelines
● Declarative application dependencies
● Documentation
● OpenAPI Specification
● CI Pipelines definition
● Workflows (PR)
● Design systems
● Infrastructure-As-Code
● Secrets (encrypted)
● Dashboards/alerts/SLA configuration
● Notebooks
● ...

@yshayy
Self-Contained Repositories
● All code, tools, knowledge, definitions and processes related to project resides
in the repository.
● Git as the single source of truth
● Code is more accessible, lowering the barrier of entry
● Applications are portable
● Fine-grained developer experience
● Emerging tools ecosystem

@yshayy
Patterns & Cheatsheet
Challenge Solution Exampletools
Basic SDK/Runtime
dependencies
Development in
container
VSCode+Docker, GitPod
Cloud dependencies Compatible dockerized
implementations
Minio, redis
Initial application data Data seeding Scripts, replicating from cloud
More hardware Remote environment Docker-machine, codespaces
Multi-Container Apps Nested Containers Dind, compose, tilt
Kubernetes Apps Nested Kubernetes Kind, k3d, tilt, skaffold
Exposing network
dependencies
Reverse proxy, wild card
development dns
Traefik, nginx, *.xip, *.localtest.me
Serverless Mock cloud frameworks Localstack

Thank you
@yshayy
https://github.com/yshayy/self-contained-repositories

Instant developer onboarding with self contained repositories

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Instant developer onboarding with self contained repositories

Similar to Instant developer onboarding with self contained repositories (20)

Recently uploaded

Recently uploaded (20)

Instant developer onboarding with self contained repositories