Kubernetes is extremely powerful, and that configurability leads to complexity. Here are the practical considerations that one must be aware of when dealing with Kubernetes as the deployment target. These slides go through Setup & maintenance, Application developer experience, Monitoring, Cost, and Setup evolution from a startup perspective.

1. Kubernetes: Practical Considerations
For Startups
Things to
keep in mind
Kubernetes version updates may break applications due to deprecated APIs.
Don’t try to manage Kubernetes clusters by yourself using kubeadm unless
there is a constraint.
Command-line tools exist to specify the configurations. Terraform modules can
help in provisioning this.
Setup and Maintenance
01 Managed Kubernetes
providers
Your initial setup can be through
managed services by cloud providers
Amazon EKS, Google GKE, Azure AKS,
DOKS.
02 Kubernetes -
wrappers
There are some startups moving k8s
under the hood. Automatically
manage Kubernetes pods and
clusters.

2. Kubernetes:
Practical Considerations For Startups
An Improved Developer
Experience
01 Build
Containerization leads to long build times, especially if there is no
caching enabled for the build. Use podman, kaniko, or similar over
docker for builds.
02 Deploy
The complexity of defining the deployment and maintianing it is
pretty high and requires k8s-specific knowledge. Use tools for
automatic generation of k8s manifests like skaffold, devspace, dokku.

3. An Improved Developer
Experience
04 Autoscaling
Nodes: The number of EC2 instances required for the application.
The cluster-autoscaler or karpenter projects are great for this.
Horizontal Scaling: Creating more instances of the application to keep up with
the load. Keda is the tool for this.
Vertical Scaling: Increasing resources available to each instance of the
application based on load.
03 Debug
Replicating bugs from remote environments to local is tough. Tools like
devspace and telepresence help. Sometimes when its needed to login to
the remote container for debugging, you’d want to use a Kubernetes
dashboard - k8s lens, or k8s cli.
Kubernetes:
Practical Considerations For Startups

4. Kubernetes:
Practical Considerations For Startups
Monitoring Applications
01 Logs
You should avoid directly accessing prod
environments. Prometheus + Grafana + Loki is
the simplest self-hosted solution. Datadog etc.
has excellent experiences but at a (literal) cost.
02 APM + Tracing
Please avoid tool fragmentation by using
the same solution across. The tools
mentioned in the previous point work
just fine for this use case as well.
Things to
keep in mind
Infra monitoring with the right permissions and access control can get
complicated for startups. The container logs from each node are sent to the log
server with appropriate labels, which is a huge advantage in k8s.

5. Kubernetes:
Practical Considerations For Startups
Setup Evolution
01 Setup multiple clusters across regions and connect them to
operate as a single entity as far as apps are concerned. This
needs a service mesh like linkerd.
02 Setup policy around what resource requirements can be
requested by an app per environment. OPA and gatekeeper or
kyverno can help.
03 Setup access control for who can create or modify apps.
04 Setup a VPN solution.
05 Accrete ~15 tools to help with k8s management over time.
06 Have a dedicated infra team.
Here are some things you will eventually need to do over the next few years.

6. Kubernetes:
Practical Considerations For Startups
Closing Notes
01 Use Kubernetes as a startup if you have cloud credits and more than 3-5
services, and a couple of environments.
02 Use lambda container images if you have 1-2 services and low utilization.
03 Setup a VPN solution.
04 Kubernetes adoption increases the cloud bill as there is a $72/month cost
associated with just running each cluster on EKS, GKE, or AKS + compute cost.
05 Tools like kubecost could help with reducing the costs by providing visibility into
the utilization of the pods and adjusting resources appropriately.