This talk is a journey through the wonders and mysteries of Kubernetes namespaces. While being a known feature of Kubernetes, there are a number of not so well known things to know about them that can teach a lot about Kubernetes. During the talk we will not only take a look at the details of Kubernetes namespaces, but also show how they are used in different production scenarios.

1. R A F F A E L E D I F A Z I O
Author
2 0 2 1 - 1 1 - 1 7
Date
OMG Namespaces!
A journey through the wonders and mysteries of Kubernetes namespaces

2. Hi!
•
•
•
Software Engineer at GitHub.
Created my first Kubernetes cluster in
2015.
Creator and maintainer of ExternalDNS.

3. Let's talk about Namespaces!
< / >

4. Deployment resource
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80

5. Namespace
apiVersion: v1
kind: Namespace
metadata:
name: pineapple pizza

6. Deployment with namespace
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: pineapple pizza
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80

7. The end, thank you for coming!
< / >

8. No, no, I'm just joking
SORRY 😅
< / >

9. Goals
•
•
We'll use Namespaces as a mean to
explore Kubernetes and some of its
design decisions.
Tools used: kind, kubernetes/kubernetes, the
official docs.

10. index.html
1.
2.
3.
4.
5.
Read the docs
Not all namespaces are created equal
Internals, resources, tools
How people use namespaces
Namespaces evolution

11. Chapter 1: read the docs
< / >

12. Namespaces de nition (v1.21)
Kubernetes supports multiple virtual clusters backed by the same
physical cluster. These virtual clusters are called namespaces.

15. Namespaces de nition (latest)
In Kubernetes, namespaces provides a mechanism for isolating groups of resources
within a single cluster. Names of resources need to be unique within a namespace, but
not across namespaces. Namespace-based scoping is applicable only for namespaced
objects (e.g. Deployments, Services, etc) and not for cluster-wide objects (e.g.
StorageClass, Nodes, PersistentVolumes, etc).

16. When to Use Multiple Namespaces (docs)
Namespaces are intended for use in environments with many users spread across multiple teams, or
projects. For clusters with a few to tens of users, you should not need to create or think about
namespaces at all. Start using namespaces when you need the features they provide.
Namespaces provide a scope for names. Names of resources need to be unique within a namespace,
but not across namespaces. Namespaces cannot be nested inside one another and each Kubernetes
resource can only be in one namespace.
Namespaces are a way to divide cluster resources between multiple users (via resource quota).
It is not necessary to use multiple namespaces to separate slightly different resources, such as
different versions of the same software: use labels to distinguish resources within the same
namespace.

17. Recap from the docs
•
•
•
•
•
Some resources are namespaced.
Namespaced resources are unique
within a namespace.
Can be used for isolation.
Namespaces can be used to divide the
clusters between users.
The docs don't really prescribe how to
use namespaces.

18. Questions
•
•
Is isolation really there?
What does it even mean to divide
resources between users?

19. Chapter 2: not all namespaces are created equal
K O O B C E E T E E E L L
< / >

20. K U B E R N E T E S N A M E S P A C E I N F R E S H K I N D C L U S T E R

21. L O O K I N G A T S O M E R E S O U R C E S ( K U B E - P U B L I C )

22. L O O K I N G A T S O M E R E S O U R C E S ( K U B E - S Y S T E M )

23. L O O K I N G A T S O M E R E S O U R C E S ( D E F A U L T )

24. Let's delete the "kubernetes" service
🔥🧨🎇
< / >

25. D E L E T I N G T H E D E F A U L T K U B E R N E T E S S E R V I C E

26. K U B E R N E T E S D E F A U L T P O D S

27. K U B E R N E T E S D E F A U L T E N D P O I N T S

28. Service, endpoints, no pods?
🤔
< / >

29. Let's delete the "default" namespace
🔥🧨🎇
< / >

30. D E L E T I N G D E F A U L T N A M E S P A C E

32. Keeping the kubernetes namespace around
RunKubernetesNamespaces periodically makes sure that all internal namespaces exist
func (c *Controller) RunKubernetesNamespaces(ch chan struct{}) {
wait.Until(func() {
Loop the system namespace list, and create them if they do not exist
for _, ns range c.SystemNamespaces {
if err createNamespaceIfNeeded(c.NamespaceClient, ns); err nil {
runtime.HandleError(fmt.Errorf("unable to create required kubernetes system
namespace %s: %v", ns, err))
}
}
}, c.SystemNamespacesInterval, ch)
}

33. Keeping the kubernetes service around
RunKubernetesService periodically updates the kubernetes service
func (c *Controller) RunKubernetesService(ch chan struct{}) {
wait until process is ready
wait.PollImmediateUntil(100 time.Millisecond, func() (bool, error) {
var code int
c.readyzClient.Get().AbsPath("/readyz").Do(context.TODO()).StatusCode(&code)
return code http.StatusOK, nil
}, ch)
wait.NonSlidingUntil(func() {
Service def nition is not reconciled after f rst
run, ports and type will be corrected only during
start.
if err c.UpdateKubernetesService(false); err nil {
runtime.HandleError(fmt.Errorf("unable to sync kubernetes service: %v", err))
}
}, c.EndpointInterval, ch)
}

34. Keeping the kubernetes service around
func (c *Controller) CreateOrUpdateMasterServiceIfNeeded() error {
if s, err c.ServiceClient.Services(metav1.NamespaceDefault).Get(serviceName); err nil {
The service already exists.
if reconcile {
if svc, updated reconcilers.GetMasterServiceUpdateIfNeeded(serviceName); updated {
_, err c.ServiceClient.Services(metav1.NamespaceDefault).Update(svc)
return err
}
}
return nil
}
c.ServiceClient.Services(metav1.NamespaceDefault).Create(svc)
}

35. Why do we need the Kubernetes service?
< / >

36. Kubernetes default service
We need this service to talk with the API service, it's the ClusterIP of the Kubernetes API
server, used by users (humans) and controllers.

37. But why in the default namespace?
< / >

40. Chapter 3: internals, resources, tools
< / >

41. Resources
•
•
•
Not all resources are namespaced.
How are namespaces represented?
What about custom resources?

42. Namespaced resource
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: pineapple pizza
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80

43. Non namespaced resource
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster admin
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- '*'
- nonResourceURLs:
- '*'
verbs:
- '*'

44. Namespaced resource

45. Are namespaces... namespaced?
< / >

46. A namespaced namespace
apiVersion: v1
kind: Namespace
metadata:
name: foo
namespace: kube system

47. A namespaced namespace

48. A look at etcd

49. How does it even work?
< / >

50. A piece of code that creates namespaces...
kubernetes/pkg/master/client_util.go
func createNamespaceIfNeeded(c corev1client.NamespacesGetter, ns string) error {
if _, err c.Namespaces().Get(context.TODO(), ns, metav1.GetOptions{}); err nil {
the namespace already exists
return nil
}
newNs &corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Name: ns,
Namespace: "",
},
}
_, err c.Namespaces().Create(context.TODO(), newNs, metav1.CreateOptions{})
if err nil errors.IsAlreadyExists(err) {
err = nil
}
return err
}

51. A piece of code that creates namespaces...
kubernetes/pkg/master/client_util.go
func createNamespaceIfNeeded(c corev1client.NamespacesGetter, ns string) error {
if _, err c.Namespaces().Get(context.TODO(), ns, metav1.GetOptions{}); err nil {
the namespace already exists
return nil
}
newNs &corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Name: ns,
Namespace: "",
},
}
_, err c.Namespaces().Create(context.TODO(), newNs, metav1.CreateOptions{})
if err nil errors.IsAlreadyExists(err) {
err = nil
}
return err
}

52. Another gem: IsCertainlyClusterScoped 😅
IsCertainlyClusterScoped returns true for Node, Namespace, etc. and
false for Pod, Deployment, etc. and kinds that aren't recognized in the
openapi data. See:
https: kubernetes.io/docs/concepts/overview/working with objects/namespaces
func IsCertainlyClusterScoped(typeMeta yaml.TypeMeta) bool {
nsScoped, found IsNamespaceScoped(typeMeta)
return found !nsScoped
}

53. Namespaces are not namespaced :-)
from pkg/registry/core/namespace/strategy.go
NamespaceScoped is false for namespaces.
func (namespaceStrategy) NamespaceScoped() bool {
return false
}
staging/src/k8s.io/apiextensions apiserver/pkg/registry/customresource/strategy.go
objectMetaFieldsSet returns a f elds that represent the ObjectMeta.
func objectMetaFieldsSet(objectMeta metav1.Object, namespaceScoped bool) f elds.Set {
if namespaceScoped {
return f elds.Set{
"metadata.name": objectMeta.GetName(),
"metadata.namespace": objectMeta.GetNamespace(),
}
}
return f elds.Set{
"metadata.name": objectMeta.GetName(),
}
}

54. Namespacing CRDs

55. Kubernetes is tolerant, not all tools are

56. Chapter 4: how people use namespaces
< / >

57. Different usages
•
•
•
•
Naive: everything in the default
namespace.
One namespace per team.
One namespace per environment.
One namespace per application.

58. Kubernetes does not have an opinion on
how you should use Namespaces.
< / >

59. Everything in default a.k.a. kubetcl YOLO
< / >

60. CONs
PROs
•
•
Easy to understand.
What you do following most tutorials.
•
•
Hard to manage with a lot of resources.
Who owns what?

61. One namespace per team
< / >

62. CONs
PROs
•
•
Logical separation between services of
different teams.
Good for "team centric" companies.
• In some companies teams are
extremely fluid.

63. One namespace per environment
< / >

64. CONs
PROs
•
•
Logical separation between staging,
production, etc.
Can work well for small companies.
• Are things really separated? 😬

65. The great misunderstanding
< / >

66. There's no separation between things in
different namespaces.
< / >

67. Network policies

68. It's likely a bad idea to mix prod and non
prod workloads.
< / >

69. RBAC and Namespaces
< / >

70. RBAC for Namespaces
•
•
Creating/deleting arbitrary namespaces -> very high privileges.
Custom controllers often need to get access to all namespaces which
give them lots of permissions.

71. Permission model weaknesses

72. One namespace per application
< / >

73. CONs
PROs
•
•
•
•
Good logical separation.
You can fine tune what talks with what.
Keeps the number of resources per
namespace low.
Deleting a namespace is like deleting
an app... sort of.
•
•
If you have a monolith, what's
even an app?
You could end up with a lot of
namespaces.

74. Chapter 5: Namespaces evolution
< / >

75. Hierarchical Namespaces

76. Virtual Clusters using Namespaces
https://youtu.be/QddWNqchD9I

77. Conclusions
< / >

78. Thank You
< / >