Delivered at Interop ITX 2017: http://info.interop.com/itx/2017/scheduler/session/cloud-native-designing-change-tolerant-software Cloud-native applications are characterized by highly distributed topologies consisting of many relatively small components (yup, usually called microservices). But the thing that sets them apart from the previous generation of apps is that they are expected to function flawlessly even while the environment they are running in is constantly changing, or even failing. All of this requires applying a new set of design patterns and practices and this session will introduce the key ones. The Twelve Factor App (12factor.net) is a high-level articulation of some of these techniques that you may well have heard of, but its descriptions are relatively dense and the industry knowledge has evolved a fair bit since its publication. Cornelia Davis will go through the best practices for cloud-native applications and clear some of the mystery that shrouds 12-factor today. At the conclusion, attendees will understand what is needed for cloud-native applications, as well as why and how to deliver on those requirements.

1. @cdavisafc
CLOUD-NATIVE
DESIGNING CHANGE-TOLERANT SOFTWARE
Cornelia Davis, Sr. Director of Technology, Pivotal

2. @cdavisafc
TEXT
A Seattle book store
deploys code, on average,
every second

3. @cdavisafc
TEXT
A Seattle technology company
hosts applications for thousands
of major corporations

4. @cdavisafc
TEXT
Major Outage on 20 September 2015
Airbnb, Nest, IMDB and many more experienced downtime
… Including Netflix
Outage lasted 5+ hours
“brief availability blip”
experienced a

5. @cdavisafc
TEXT
ME?
Developer (wasn’t Ops)
Web architectures for >10 years
Cloud-native for 5 years
Cloud Foundry for 5 years
More recently
Discount code 39% off!: ctwintitx

6. MICROSERVICES

7. It is not that simple

8. Mental Model

9. @cdavisafc
▸ Cloud-Native App
(yup, the microservices)
APP APP
APP
APP
APP

10. @cdavisafc
▸ Cloud-Native App
(yup, the microservices)
APP APP
APP
APP
APP

11. @cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
APP APP
APP
APP
APP

12. @cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps)
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
APP

13. @cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps
… and other are other
types of services)
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
DATA DATA
DATA
APP

14. @cdavisafc
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps
… and other are other
types of services)
▸ Cloud-Native Data
(a distributed data fabric)
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
DATA DATA
DATA
?
APP

15. @cdavisafc
CLOUD-NATIVE SOFTWARE
▸ Cloud-Native App
(yup, the microservices)
▸ Cloud-Native Services
(yup, some are apps
… and other are other types
of services)
▸ Cloud-Native Data
(a distributed data fabric)
▸ The Cloud-Native Collective
(implicit and explicit connections)
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP
DATA DATA
DATA
?

16. @cdavisafc
AGENDA (THE REST OF IT)
▸ Cloud-Native App
▸ Cloud-Native Services
▸ Cloud-Native Data
The Cloud-Native Collective

17. CLOUD-NATIVE APP

18. @cdavisafc
CLOUD-NATIVE APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

19. @cdavisafc
CLOUD-NATIVE APP
APP APP
APP
APP
APP
APP APP
APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

20. @cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APP
APP
APP
APP APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
APP APP
APP
APP
APP

21. @cdavisafc
CLOUD-NATIVE APP
APPAPP
APPAPPAPPAPPAPP
APPAPPAPPAPPAPP
APP APP
APP
APP
APP APP
APP
APP
APP APP
APP
APP
APP
SCALE HORIZONTALLY
▸ Need an abstraction to treat set of
instances as one logical unit
▸ Load Balancing - DNS? Something
else?
▸ Dynamic!!!
▸ IP Addresses of instances always
changing
APP APP
APP
APP
APP
ROUTER
ROUTER

22. @cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken

23. @cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
ValidTokens: []

24. @cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
Unauthorized ValidTokens: []
token=UserToken

25. @cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you
think!) the instance you are sticky to
will disappear
▸ Do make your instances
completely stateless
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
/login
token=UserToken
token=UserToken
APP
(INSTANCE 2)
Unauthorized ValidTokens: []
token=UserToken

26. @cdavisafc
CLOUD-NATIVE APP
STATELESSNESS
▸ Do NOT use sticky sessions!
▸ Eventually (and sooner than you
think!) the instance you are sticky to
will disappear
▸ Do make your instances
completely stateless
▸ Externalize state
ROUTER
APP
(INSTANCE 1)
ValidTokens:
[UserToken]
token=UserToken
token=UserToken
APP
(INSTANCE 2)
token=UserToken
STATE
STORE

27. @cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Property Files are the abstraction used
to externalize configuration
▸ They do not hold actual values!
▸ Values coming from the environment
are supplied via environment variables
▸ Other configuration is controlled just
like source code and injected
RUNTIME CONTEXT
WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S)
DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp=
${CF_INSTANCE_IP}
specialization=
${SPECIALIZATION:
Science}
APP
CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController {
@Value("${hostIp}")
private String hostIp;
@Value("${specialization}")
private String specialization;
...
}
SYSTEM PROPERTIES
PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29

28. @cdavisafc
CLOUD-NATIVE APP
APP CONFIGURATION
▸ Other configuration is controlled just
like source code and injected
RUNTIME CONTEXT
WHERE CONFIGURATION PARAMETERS ARE DEFINED IN ENVIRONMENT VARIABLES
PROPERTY FILE(S)
DEFINE THE CONFIGURATION
PARAMETERS AND DEFAULT VALUES
hostIp=
${CF_INSTANCE_IP}
specialization=
${SPECIALIZATION:
Science}
APP
CODE THAT REFERENCES THE CONFIGURATION PARAMETERS
DEFINED IN THE PROPERTY FILE
public class HelloController {
@Value("${hostIp}")
private String hostIp;
@Value("${specialization}")
private String specialization;
...
}
SYSTEM PROPERTIES
PROVIDED BY THE RUNTIME ENVIRONMENT, VIA
ENVIRONMENT VARIABLES
CF_INSTANCE_IP=10.10.148.29
APP PROPERTIES
STORED AND VERSIONED IN A
REPOSITORY SUCH AS GIT
specialization
=Sports
CONFIG SERVER
ROLE IS TO INJECT MANAGED APP
PROPERTIES INTO THE APP

29. @cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have
rippling effects through the collective
INVOICE APP
ordersvcIP
=10.24.1.35
ORDER SERVICE APP
10.24.1.3510.24.63.116

30. @cdavisafc
CLOUD-NATIVE APP
APPLICATION LIFECYCLE
▸ Application lifecycle events have
rippling effects through the collective
▸ The application must broadcast
lifecycle change events
▸ (Note: I strongly recommend you
use a framework to help you with
this!)
▸ And an app must be able to absorb
that configuration at the right time (run
time?)
INVOICE APP
ordersvcIP
=10.24.1.35
ORDER SERVICE APP
Some “magic” happens
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

31. CLOUD-NATIVE
SERVICE

32. @cdavisafc
CLOUD-NATIVE SERVICE
LET’S TALK ABOUT AUTONOMY
A major benefit of cloud-native architectures is independent:
▸ Application Scaling
▸ Team Scaling
▸ Development Cycles
▸ Experimentation
▸ Resilience

33. @cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making
deployments a non-event
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
APP
… AND PARALLEL DEPLOYS

34. @cdavisafc
CLOUD-NATIVE SERVICE
VERSIONED SERVICES
▸ Use Semantic Versioning
▸ major.minor.patch
▸ Use Blue/Green deploys
▸ Deployments needn’t be replacements
▸ Powerful lever in making
deployments a non-event
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
APP
… AND PARALLEL DEPLOYS
SERVICE
APP
1.0.0 2.0.0

35. @cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Let’s dig into the “magic”
(it’s not really magic)
INVOICE APP
orderSvcCoords...
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116
Some “magic” happens

36. @cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=orderSvc.example.com
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
ROUTER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …

37. @cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=10.24.1.13
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

38. @cdavisafc
CLOUD-NATIVE SERVICE
SERVICE DISCOVERY
▸ Dynamic Router maintains routing
table
▸ Messaging pattern to decouple
service from router
▸ Must handle lost broadcast
messages
▸ Request pass through router INVOICE APP
orderSvcCoords
=10.24.63.116
ORDER SERVICE APP
10.24.63.116
Here I am!
IP Address: 10.24.63.116
Version: 3.4.239
…
SERVICE DISCOVERY SERVER
OrderService: 10.24.63.116, …
CustomerService: 10.24.3.1, …
ORDER SERVICE APP
10.24.1.13
I’ll adapt to these changes!
Orders Service new IP:
10.24.63.116

39. @cdavisafc
CLOUD-NATIVE SERVICE
DISTRIBUTED TRACING

40. @cdavisafc
CLOUD-NATIVE SERVICE

41. @cdavisafc
CLOUD-NATIVE SERVICE
DISTRIBUTED TRACING
▸ App must look for and leave
breadcrumbs
▸ Use a framework to help you do
this!!!
▸ Tools can then read and correlate logs
SERVICE
APP
SERVICE
APP
APP
SERVICE
APP
SERVICE
APP
SERVICE
APP

42. CLOUD-NATIVE DATA

43. @cdavisafc
CLOUD-NATIVE DATA
WE ARE LOOSELY COUPLED!

44. @cdavisafc
CLOUD-NATIVE DATA
OR ARE WE?

45. @cdavisafc
CLOUD-NATIVE DATA
THIS IS NOT CLOUD-NATIVE DATA

46. @cdavisafc
CLOUD-NATIVE DATA
DATA APIS (SERVICES)
▸ Microservices do not access data layer directly
▸ Except for those that implement the data API
▸ A surface area to:
▸ Implement access control
▸ Implement throttling
▸ Perform logging
▸ Other policies…

47. @cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned
microservices…
V1 V2

48. @cdavisafc
CLOUD-NATIVE DATA
VERSIONED DATA APIS (SERVICES)
▸ We are already familiar with versioned
microservices…
… and
V1 V2
PARALLEL DEPLOYS

49. @cdavisafc
CLOUD-NATIVE DATA
ANTI-PATTERN: DATA APIS THAT JUST PROXY

50. @cdavisafc
CLOUD-NATIVE DATA
EVERY MICROSERVICE NEEDS A CACHE

51. @cdavisafc
CLOUD-NATIVE DATA
“Caching at Netflix: The Hidden Microservice” https://www.youtube.com/watch?v=Rzdxgx3RC0Q

52. @cdavisafc
CLOUD-NATIVE DATA
DATABASE PER MICROSERVICES

53. @cdavisafc
CLOUD-NATIVE DATA
DATABASE PER MICROSERVICES
▸ Supports Polyglot persistence
▸ Independent availability, backup/restore, access
patterns, etc.

54. @cdavisafc
CLOUD-NATIVE DATA
EVENT LOG AS A SYSTEM OF RECORD
Unified Log
Source of
Record

55. @cdavisafc
CLOUD-NATIVE
▸ Cloud-native Apps
▸ Scale out
▸ Statelessness
▸ Externalize configuration
▸ Implications from changes in the application
lifecycle
▸ Services
▸ Versioned services (autonomy)
▸ Service Discovery
▸ Distributed tracing
▸ Data
▸ Breaking the Data monolith
▸ Data APIs
▸ Caching
▸ Polyglot Persistence
▸ Event sourcing

56. THANK YOU
Discount code 39% off!: ctwintitx