In this talk, Connor and Niklas will talk about their thoughts on the next decade of cluster computing. They have worked on Apache Mesos, Kubernetes and Mesos Frameworks; from design of subsystems to tooling and operationalizing at scale. They will discuss past, present and future trends in public and private cloud computing and unique opportunities for the cluster computing communities. By the end of the talk, they hope you will leave with a fresh perspective on scheduling and orchestration, at a deeper level than "Mesos vs. Kubernetes vs. Omega vs. Borg …" KubeCon schedule link: http://sched.co/4Wgx

1. Hoverboards, Jetpacks,
Clusters and Flux Capacitors
Connor Doyle and Niklas Nielsen, Intel Corporation
Software Defined Infrastructure

2. Your speakers today
Connor Doyle
@nor0101
Niklas Nielsen
@quarfot

3. An evaluation of Kubernetes,
Mesos, Omega …?
Not exactly

4. Just thoughts from engineers
in the scheduling space

5. Let’s take a look at
Yesterday’s
Infrastructure

6. You buy a box
Application
Application

7. Change and upgrade it to meet demand
Application
Application

8. Is this bad?
It’s another trade off that was the
obvious one at the time

9. Legacy infrastructure is starting to cause
pain
Expensive at scale
Encouraged snowflakes
Less automation
Change and iteration is hard

10. The Amazons,
Azures and
Googles

11. Public Cloud Providers shared their secret sauce
“Legacy
infrastructure”
Deployment and image formats
Services (Blob and block Storage,
Load balancing, Databases)
Cost is correlated to fine grained
allocation
“Infinite” capacity
Explicit fault domains in form of
availability zones

12. Public Cloud Providers shared their secret sauce
“Legacy
infrastructure”
Interference
Limited exposure to hardware features
Subsidizing other users
Vendor lock-in?
“Infinite capacity”? This is a business.

13. The Facebooks,
Twitters, and
Apples

14. Latency is king
Legacy infrastructure

15. Private Cloud
● Requires high expertise
● Reinvention
● Resource silos
● Workload portability?
Can we get the best of both worlds?
● Exclusive access to best
hardware
● Predicable cost and
performance
● Deploy custom hardware
● Meet security and compliance
requirements

16. Hybrid: best of both worlds
Legacy infrastructure
“Hybrid”

17. Meanwhile…
The container
revolution!
Google Trends
Docker (Software)
Interest Over Time
January 2013 - November 2015

18. To many people, it felt like this

19. Now for jetpacks
and flux capacitors

20. Prediction 1
Cloud computing will be a
commodity.

21. Let 1000 clouds bloom
Amazon EC2
Google
Compute
Platform
Twitter
Private
Cloud
Yelp
Private
Cloud
eBay
Private
Cloud
Public
Cloud
FooPublic
Cloud
Bar
Public
Cloud
Baz
Private
Cloud
Baz

22. Prediction 2
Grid computing returns.*

23. Computing can float between providers
Amazon EC2
Google
Compute
Platform
Twitter
Private
Cloud
Yelp
Private
Cloud
eBay
Private
Cloud
Public
Cloud
FooPublic
Cloud
Bar
Public
Cloud
Baz
Private
Cloud
Baz

24. Form factor is different this time
Google
Compute
Platform
Amazon EC2
Public Cloud BarCluster
X
Cluster
Z
Cluster
Y
Cluster Y
Über cluster Y
Milky way clusterScalable global
insights
Local control

26. Boundaries of ‘form factors’ blur
1000 cores
1 million x
0.001
shares

27. Think about how you describe your intent
across these layers?
Manage complexity with abstractions.
How do you delegate decision making?
How do we avoid creating a mega-language for
the future form factor?
How do we provide portable, composable
interfaces?

28. Prediction 3
Declarative configuration won’t
be enough by itself

29. Visualizing the curse of dimensionality in configuration spaces
cpus: 1 cpus: 1
mem: 512
cpus: 1
mem: 512
disk: 8192
cpus: 1
mem: 512
disk: 8192
iops: 100kps
...

30. Exhibit A: Mesos TaskInfo
/**
* Describes a task. Passed from the scheduler all the way to an
* executor (see SchedulerDriver::launchTasks and
* Executor::launchTask). Either ExecutorInfo or CommandInfo should be set.
* A different executor can be used to launch this task, and subsequent tasks
* meant for the same executor can reuse the same ExecutorInfo struct.
*/
message TaskInfo {
...
}

31. Exhibit B: Kubernetes Pod Manifest
apiVersion: v1 <- API Server
kind: ReplicationController
metadata:
name: redis
spec:
replicas: 1 <- Replication controller
selector:
name: redis <- Scheduler
template:
metadata:
labels:
name: redis
spec:
containers:
- name: redis
image: kubernetes/redis:v1
ports:
- containerPort: 6379
resources:
limits:
cpu: "0.1"
volumeMounts:
- mountPath: /redis-master-data <- Kubelet / runtime
name: data
volumes:
- name: data
emptyDir: {}

32. Don’t ask users for too much
We need communication between layers.

33. Don’t ask users for too much
Think about interacting with a managed
runtime.

34. Don’t ask users for too much
Learn more about your users’ needs by
observation.

35. How will our interfaces evolve?
● Usability paradox
○ We understand the problem better
○ Systems get more advanced
○ Conceptual surface area increases
● Alternatives…
○ Turing-complete, imperative control
● A principled mix?
○ Rules to determine where knobs belong?
○ Patterns to “drop down” and specify lower-
level logic?

36. Upcalls and delegation across layers of abstraction
Hardware
OS
Cluster Runtime
Cluster manager
Cluster federation

37. Thanks for tuning in!
@nor0101 @quarfot
Intel is hiring!

38. Backup slides

39. An age-old problem
Inheritance Plugins,
Extensions,
Modules
Composition Shims, Collectors,
Adaptors

41. Let’s take a look at
Yesterday’s
Infrastructure

42. Snowflakes caused growing pains

43. The Amazons,
Azures and
Googles

45. Remember this?
Application
Application

46. Multiply commodity infrastructure
Application
Application
Application
Application
Application
Application
Application
Application
Application
Application

47. The Facebooks,
Twitters, and Yelps

48. Internet companies followed the path of
Google (minus 10 years)

49. The inbetweeners
Can hybrid cloud deliver?