Oracle: Building Cloud Native Applications

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.
Building Cloud Native Applications
Kelly Goetsch
Director, Product Management
Microservices
January 31st 2016

Software Is Overtaking the World
Time-to-market is key to success
2

Business Cycles Are Faster Than Ever Before
3
Over the last 50 years, the average
lifespan of companies on the S&P 500
has shrunk from 60 to 18 years
Years
75
65
55
45
35
25
15
5
1930 1940 1950 1960 1970 1980 1990 2000 2010

What Do Today’s Winners Have in Common?
Speed of Business Innovation, Enabled by Software
4Copyright © 2016, Oracle and/or its affiliates. All rights reserved.

Winners Don’t Have These Problems
Must adopt a new approach to IT
It’ll take six
months to build
your development
environment
To get that done,
you’ll need to file a
ticket
That change will
have to wait for our
next build in three
months
Apache
Zookeeper went
down again...
We already spent all
of the money we
allocated for hardware
I’m frustrated that my
important cool new
app is prioritized the
same as payroll
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. 5

Software is How Companies Differentiate Themselves
6
"More and more major businesses and industries are
being run on software and delivered as online services—
from movies to agriculture to national defense. Many of
the winners are Silicon Valley-style entrepreneurial
technology companies that are invading and overturning
established industry structures"

Copyright © 2016, Oracle and/or its affiliates. All rights reserved.Copyright © 2016, Oracle and/or its affiliates. All rights reserved. 7
Better Segment Software
Different software has different needs
Find the Next Business
Run the Current Business
Run the Back Office
New IT
Old IT

Different Types of Software Requires Different Practices
A payroll system should be treated very different from a customer-facing .com
8
Innovation Software - Find the Next Business
Differentiation Software - Run Current Business
Core Software - Keep the Lights On
Release Hourly
Fail Early
Agile
Business-centric
Top Line Growth
Bespoke Software
Product-based
Release Quarterly
Fail Late
Waterfall
IT-centric
Bottom Line Savings
Packaged Software
Project-based

Most Build Software for Innovation and Differentiation
9
75%
By 2020, 75 percent of
application purchases
supporting digital business
will be "build," not "buy.”
Forecast Analysis: Enterprise Application
Software, Worldwide, 2Q15 Update.

Innovation and Differentiation Systems are Different!
Stop treating them like systems of record
IT-centric
Focus on Saving Money
Release Quarterly
Waterfall
Traditional Infrastructure
Dev/Ops Separated
Project-based
One Large, Monolithic Application
Software Manually Installed
Manual Scaling
Heavy Weight Governance
Tight Coupling
Stateful Middle Tiers

We Must Re-invent Our Profession
We can now do better

Introducing Cloud Native Architecture
Used to build Cloud Native applications
12

Microservices
• Minimal Function
• Service Discovery
• API-first
3 • Polyglot
• Choreography
• Loose Coupling
DevOps
• Automated Provisioning
• Automated Setup
• Continuous Integration
1 • Continuous Delivery
• Automated Testing
• Agile
• Culture Change
* as a Service
• Consume Infrastructure and
Software as a Service
• Fault Tolerant by Definition
2 • Auto-scaling
• Infinite Elasticity
What is Cloud Native?
A new style of architecture
Distributed Computing
• Multi-master
• Many Data Centers
• Many Fault Domains
4 • Many Regions
• Global Server Load Balancing
• Replication
Competency

Prerequisite #1 - DevOps
14
A prerequisite to consuming infrastructure and software as a service
“Done” Means Released
Avoid Blaming
Culture Technology
Discuss
Respect
Don’t Fix Anything
One Step Build/Deploy
Shared Version Control
Infrastructure as Code
Culture Constrained by Technology
Get development and
ops to work together

Prerequisite #2 - * as a Service
Requires fundamental shift in how applications are built
Your Code
Highly innovative,
differentiated, etc
Configuration
Caching
Load Balancing Eventing
Logging
Monitoring
Database
NoSQLState
Messaging
3rd Party Cloud – On or Off Premise
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Building
Block
Where you’re not
differentiating,
consume building
blocks from 3rd party
cloud vendor

Prerequisite #3 – Microservices
• Single monolithic application -> small,
independently deployable
microservices
• Each microservice:
– Has its own team that designs, builds,
deploys and maintains it
– Exposes an API, which can be consumed
elsewhere
– Has its own datastore/database
• Microservices are loosely coupled,
with most communication over REST
and async messaging
16
Break up your application into many small pieces to get features to market quickly
User Interface
Application
Datastore
Infrastructure
Status Quo
One Application
Microservices
Many Small Microservices
API
Application
Datastore
Infrastructure
Inventory
Microservice
API
Application
Datastore
Infrastructure
Payment
Microservice
API
Application
Datastore
Infrastructure
Profile
Microservice
API
Application
Datastore
Infrastructure
Product Catalog
Microservice

Prerequisite #4 – Distributed Computing
Run your applications across multiple data centers, fault domains, regions, etc
Cloud
Region
Fault Domain
Data Center
Host
Container
Microservice
Microservice
Microservice
Microservice
Hundreds of
milliseconds
of latency
Everything is now distributed
Even within the same data center

What’s the Value of Cloud Native?
SPEED
Quickly move changes
into production
RESILIENCY
Survive failures. Stay
available in production
AGILITY
Change your
practices as needed
#1 Value – Quickly Capitalize on Business Opportunities

Cloud Native Originated in Consumer-facing Tech Companies
Consumer-facing Tech
•Spend 20%+ of revenue on R&D
•Employ highly paid developers
•Internet-scale
•Handful of apps
•Technology is their business
Where Cloud Native Originated
Traditional Enterprises
•Spend 2-4% of revenue on R&D
•Employ “normal” people
•Enterprise-scale
•Thousands of apps
•Technology seen as a tax
Most Every Other Enterprise

DevOps + * as a Service + Microservices + Distributed
Make Cloud Native Available to Everyone Else

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted
Cloud Native is Difficult
But not doing it is worse
52% of the Fortune 500 have been merged,
acquired, and gone bankrupt since 2000
Which path are you going to take?

Building Cloud Native Applications
22

Changes Required to Adopt Cloud Native
23
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Cloud Native Takes Real Organizational Commitment
The legacy style has inertia
• LARGE HORIZONTAL LAYERS FOCUSED ON
PROJECTS-> SMALL VERTICAL TEAMS
FOCUSED ON PRODUCTS
• CHANGE PEOPLE’S JOBS
• CHANGE HOW PEOPLE ARE REWARDED
RETHINK

Horizontally Tiered Enterprises == Horizontally Tiered Apps
25
Conway’s Law: Software reflects the structure of the organization that produced it
User Interface
Application
Datastore
Infrastructure
Resulting SoftwareTypical Enterprise Organization Structure
Head of IT
Head of
Operations
Head of DBAs
Head of
Infrastructure
Head of App
Dev
Head of UI
Head of
Development
An Enormous Monolith

Even Simple Changes Are Hard to Implement With Monoliths
26
Organizational boundaries introduce the need to extensively coordinate
User Interface
Application
Datastore
Infrastructure
New requirement: Add a birthdate
property to the customer’s profile. How
does this get implemented?
Application developer tickets DBAs to have them
add that property as a column in the database
1.
Application developer tickets UI team to have
them add that property to the profile screens
3.
Application developer adds the new property to
the application-level code
2. Different Teams
Different Timelines
Different Priorities
Different Ticketing

Characteristics of Different Organization Types
27
 Produce microservices
 A team can make any change to
their microservice
 Architecture more clean because
it’s not done by central committee
 A lot of freedom and
responsibilities
 True ownership
 Produce monoliths
 Simple change requires extensive
coordination across all of the
different layers
 Business logic is spread
everywhere because it’s easier to
bury it in the layer you own
 No real ownership
Centralized Organizations
Focused on Technology Layers
Distributed Organizations
Focused on Products

Re-structure Your Organization – Put Conway’s Law to Work
28
Build small product-focused teams – strict one team to one microservice mapping
Resulting SoftwareMicroservices Organization Structure
Many Small Microservices
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
Product
Lead
Project
Manager
Sys Admin DBA
JavaScript
Developer
Developer
Developer
Sys Admin
Storage
Admin
Graphic ArtistNoSQL Admin
Product
Lead
Project
Manager
Sys Admin DBA
JavaScript
Developer
Developer
Developer
Sys Admin
Storage
Admin
Product
Lead
Project
Manager
Sys Admin DBA
JavaScript
Developer
Developer
Developer
Sys Admin
Storage
Admin
Product
Lead
Project
Manager
Sys Admin DBA
JavaScript
Developer
Developer
Developer
Sys Admin
Storage
Admin

Small Teams = Much Better Communication
Low Latency/High Bandwidth Communication
Legacy Microservices

Much Faster Turnaround With Microservices
The profile microservice team – three people total, all sitting together
Turn around changes in hours vs. months
Hey Jim, can you add
that column to the
database before lunch?
Low latency, high
bandwidth communication

Adopt DevOps
31
Requires changes throughout your entire organization
Culture Technology
Respect
Discuss
Avoid Blaming
“Done” Means
Released
Infra as Code
Shared Version
Control
One Step
Build/Deploy
Don’t Fix
Anything
• Dev respect for ops
• Ops respect for dev
• Ops should be in dev discussions
• Dev should be in ops discussions
• Shared runbooks
• No fingerpointing!
• Everyone should have some
culpability
• Dev’s responsibility shouldn’t ever
end – production support required
• “Throwing it over the wall” is dead
• Don’t build envs by hand
• Scripts checked in and
managed as src
• Single system
• Ship trunk
• Enable features through flags
• One button build/deploy
• If verification fails, stop and
alert or take action
• If something breaks, re-deploy.
Don’t fix
• Fix environment setup scripts

More About DevOps
32
http://www.slideshare.net/KellyGoetsch/mastering-devops-with-oracle

Review: Organization Structure Changes for Cloud Native
33
Organize People Into Small, Vertical Teams
Change How People Are Rewarded
Focus From Projects to Products
Get Dev and Ops Working Together

34
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Make a Project Manager a Permanent Member of Teams
35
Small, cross-functional teams
Product Lead
Project Manager Sys Admin DBA
JavaScript
Developer
Developer
Developer
Sys Admin
Storage Admin

Do One Thing and
Do It Well
Focus on Business
Capabilities
Avoid Inter-
dependencies
Start Managing Small, Vertical Teams
Can have hundreds of microservices for a larger application
Large
Medium
Small
11-15 People
Example: Order Microservice
4-10 People
Example: Inventory Microservice
1-3 People
Example: Order Status Microservice
36

Deploy
Test
Change to a Form of Agile Project Management
37
Develop
Waterfall
Design
Develop
Test
Deploy
Discover
Design
Discover
Design
Develop
Test
Deploy
Discover
Design
Develop
Test
Deploy
Discover
Design
Develop
Test
Deploy
Discover
Agile

Hold Each Person on Each Team Accountable
38
Much easier with microservices-style architecture

Owners
Support in
Production
Everything
is Owned
by a Small
Team
Owners
Implement
Owners
Architect
Owners
Care
Owners Can
Fix Things
Ownership is Key
to the Success of
Cloud Native
In traditional enterprises, any one individual
has very low ownership of anything. It’s classic
tragedy of the commons

Review: Project Management Changes for Cloud Native
40
Make a Project Manager a Permanent
Member of Each Team
Start Managing Small, Vertical Teams
Change to a Form of Agile Project Management
Hold Each Person on Each Team Accountable

41
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Change How You Procure Software
42
Focus is on bringing new features to market very quickly
Competitive
Differentiation
Buy as SaaS Build
Innovation Software
Differentiation Software
Developer-led
Procurement
Core Software

Understand What You’re Differentiating On
Outsource the rest. Focus just on your application
Focus on your
application

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted
Consume Application Building Blocks as Software
Cloud (*-as-a-Service) is key to innovation
Start consuming resources as a service
Infrastructure
Compute, Networking, Storage
Platforms
Java EE, Java SE, Node, etc
Building Blocks
Database, NoSQL, Messaging, etc

Empower Developers to Make Procurement Decisions
Core Software Differentiation Software Innovation Software
• #1 focus of cloud native: time to market. Long-term
maintenance should not be a big consideration
• Let developers who are innovating pick the absolute best
technology for their own use
• Each small team supports their own microservice in perpetuity.
No need to have large maintenance teams
• Standardization is best for system of record-style applications

Teams Do Not Have 100% Freedom
46
Standardize where it makes sense. Be pragmatic
Custom Code
Communication Protocol
Data Format
Infrastructure
Datastore
SourceControl
ConfigurationManagement
DevelopmentTooling
Alerting
Monitoring
Standardize on One
Offer a Menu of 2-3 Options
Team Has Complete Choice
Programming Language
Messaging

Each Microservice Can Now Use Its Own Database/Datastore
RDBMS is great but not necessary for all use cases
Relational Database
ACID-compliant, suitable for a
wide range of workloads.
Trusted, reliable, wide client
support, easy to use
Object GridsNoSQL
Store objects in and move
business logic into the server-
side grid.
Non-relational organization of
data, including key/value, graph,
document, tabular, etc. Always
BASE and sometimes ACID
compliant
47

Review: Procurement Changes for Cloud Native
48
Segment Your Software
Empower Developers to Make Procurement
Decisions
Standardize Where it Makes Sense
Look At Non-relational Databases

49
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Everything is Now Software
Provision a new server just like you would provision a new object in Java
Person p = new Person();
$ docker run -v /host:/container example/myapp
curl -X POST "name=MyFirstApp" "runtime=java” "archiveURL=/binaries/myapp.zip"
$ puppet module install puppetlabs-java
Today, Everything is
100% Code.
Automation at scale
is standard

Three Rules for Application Startup/Shutdown
51
Start with one script
– no manual
intervention. This
script will be called
when container is
provisioned
Start up quickly – try
for under 10 seconds.
If it takes minutes,
auto-scaling won’t
work properly
Shut down cleanly
without warning.
Containers will be
killed with no
warning whatsoever

Start Using Containers
52
Helpful to microservices but not a requirement
Hardware
Hypervisor
VM 1
OS
App
VM 2
OS
App
Hardware Virtualization
Hardware
Operating System
Hypervisor
VM 1
OS
App
VM 2
OS
App
Para-virtualization
Hardware
Operating System
Container 1
App
Container 2
App
Containers
• #1 value – app
packaging
• Microservices doesn't
rely on containers but
they do help:
– Higher density
– Easy to start/stop
– Portability
• Containers are
lightweight, just like
microservices
themselves
• Containers can run in
VMs too

Containers Are Now the Standard
53
Four main use cases
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Hardware
Operating System
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Container
App
Application Packaging
Continuous Integration DIY PaaS
Infrastructure Consolidation
Neatly package applications
and supporting environment
in immutable, portable
containers
All changes to an app are
contained in one immutable
container image. Container is tested
and deployed as one atomic unit
Get infrastructure utilization up to
100% (vs 5-10% with VMs) due to
over-subscription of resources and
near bare metal performance.
Build a simple PaaS by wiring up
containers to a load balancer.
New code, patches, etc pushed
as new immutable containers.

Strict Requirement: One Instance Per Container
• Run ONE instance (unique
host/port combination) per
container
• Running multiple instances of
the same application or
different applications will
make scheduling very difficult
• Expose one port per container
54
Physical Host
Operating System
Container
App
Container
App
Just One Per Container

Isn’t a Container Just Like a VM Image? NO
55
Key differences
Hard to over-subscribe physical
hardware resources like CPU and
memory
Utilization
Very easy to get to 100% consume
physical hardware resources. Can
easily move containers off of hosts
when the hosts become too utilized
Performance
Heavy weight – must often go
through abstraction layers to
access physical hardware resources
Light weight – a container is just an
operating system process. 100%
native access to all physical
hardware resources
Must use configuration
management tool to apply updates.
Or must update every single gold
image and then re-deploy
Updates
Can apply diffs to containers, or
most often, the entire container is
rebuilt and re-deployed
A human must build the VM and
then it must be snapshotted. Or
you can use a configuration
management tool to build it
Creation
Same options as VMs + can also
declaratively construct one using a
Dockerfile
VMs Containers

How Do You Deploy Containers to Physical Hosts?
56
The emerging space of container orchestration
What Do Container
Orchestration Solutions Do?
• Map containers back to physical
hosts, taking into account user-
defined placement rules, the
utilization of each host, and the
needs of each container. Can be
very complex
• Set up overlay networking,
firewalls, ensure network QoS, etc
• Auto-scaling
• Local and external load balancers
• Service registry / discovery
Host
Host
Host
Host
Host
Host
Host
Host
Host
Host
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
App
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
• Inventory
Microserv
ice
• AcmeCo
• v1.2
Container
App
Many Containers
Host
Host
Host
Host
Host
Host
Host
Host
Host
Host
Many Hosts
Docker Swarm
Emerging space. Solutions are very early and lack any real
notion of an application. Still very much infrastructure-focused

Start Dynamically Auto-scaling
57
Requires the use of cloud, stateless middle tiers, fast/automated startup, etc
Hardware Provisioned Without Auto-scaling
Time
Traffic
Resources Provisioned
Try to
track

Review: Provisioning Changes for Cloud Native
58
Start up/shut down
Use Containers
Use Container Orchestration
One Instance per Container
Auto-scale

59
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Monolithic Applications
Single, Monolithic App
Must Deploy Entire App
One Database for Entire App
Organized Around Technology Layers
State In Each Runtime Instance
One Technology Stack for Entire App
In-process Calls Locally, SOAP Externally
Microservices
Many, Smaller Minimal Function Microservices
Can Deploy Each Microservice Independently
Each Microservice Has Its Own Datastore
Organized Around Business Capabilities
State is Externalized
Choice of Technology for Each Microservice
REST Calls Over HTTP, Messaging, or Binary
What Are Microservices?
Minimal function services that are deployed separately but can interact together to
achieve a broader use-case

Three Key Rules to Microservices
61
Don’t share a datasource across microservices
Can release each microservice independently
Can build a microservice independently

Everything Is Now Distributed. Get Used To It
62
Applications Infrastructure Teams
Applications are now
broken up into small
microservices, with
separate frontends and
backends
Different data centers, fault
domains, regions, etc. Even
within the same data
center, latency may be
unacceptably high
Many small teams, each
responsible for their own
microservice. Each team is
often geographically
distributed

Rules of Distributed Computing
63
Computer science is about tradeoffs
Consistency
Each node shows the same data at all times
Availability
Each node is available for writes at all times
Partition Tolerance
Able to handle network outages
CAP Theorem – Pick Any Two
C
A P
Theory Practice
Pick One
Partition Tolerance is non-negotiable
because we have networks that can
always fail
Enterprise IT Systems: Often CP
Microservice Systems: Often AP
Each microservice can be CP, AP or
CA but the system as a whole is
always CP or APMore Information
Pick
Any
Two

Microservices Forces Move To Distributed Computing
64
Introduces enormous complexity – monoliths don’t suffer from this
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
Microservice A Microservice B Microservice C Microservice D
• Distributed computing is a natural consequence of microservices
because each microservice has its own datastore
• Sharing datastores across microservices introduces coupling – very bad!
• There will always be latency between microservices
• Latency = eventual consistency
• All data exchange between
microservices must be
through API layer or
messaging – no accessing
datastores cross-
microservices
• Must implement high-
speed messaging for
synchronous calls between
microservices. REST + HTTP
probably isn’t fast enough
• May end up duplicating
data across datastores –
e.g. a customer’s profile

Make Your Middle Tier Stateless
65
Push all state and configuration down to highly available cloud services
Application Server
File System
Application Server
File System
Application Server
File System
Application Server
File System
Application
Instance
File System
Load Balancer
Sticky to an
Individual
Instance
Application
InstanceApplication
InstanceApplication
InstanceApplication
Instance
Load Balancer
NOT Sticky to
an Individual
Instance
State
Service
Configuration
Service
Application
Instance
Key to Cloud Native
Session State
Shopping cart contents, page
view data, personalization, etc
Application Configuration
Port numbers, file system
paths, host names, etc
Legacy Cloud Native

Remove All Hard-coded IPs, Host Names, etc
66
Use service discovery, DNS, etc instead. Everything should be dynamic

Multiple data centers
Multiple Fault Domains
Multiple Regions
Multiple Vendors
Geographically Distribute Your Workloads

Orchestration
• Top-down coordination of
discrete actions
• Used in centralized,
monolithic applications
• Brittle – centralized by
nature
• Each “action” registers with
centralized system – single
point of failure that is not
very flexible
Choreography
• Bottom-up coordination
of discrete actions
• Used in distributed,
microservice applications
• Resilient – distributed by
nature
• Each microservice
asynchronously throws up a
message that other
microservices can consume
68
Choreography Over Orchestration Between Microservices

Scenario: eCommerce user returns a widget through web-facing .com
Example of Orchestration
Centralized Workflow
Self Service
Help
Initiate
Return
Workflow
Increment
Inventory
Done
Inventory
Record
Refund
Done
Customer
Profile
Done
Payment
Refund
Money
Brittle | Centralized | Tightly Coupled

Scenario: Inventory microservice
Example of Choreography
Inventory
Microservice
All asynchronous
Event Bus
New
Inventory
Events This Microservice Cares About Events This Microservice Emits
Product
Returned
Product Sold
Product
Recalled
Inventory
Incremented
Inventory
Created
Inventory
Decremented
Inventory
Deleted
For Anyone
Who Cares...

Assume Your Infrastructure is Unreliable
71
Even though it is actually pretty reliable these days
Some public cloud vendors (not Oracle)
use velcro to attach components to
motherboards, given how frequently
these components fail

Adopt an API Gateway
72
API gateways provide a "backend for each frontend"
Client
Public Internet
Microservice Microservice Microservice
Data Center
API Gateway
• Builds a XML or JSON response for each type
of client – web, mobile, etc
• Asynchronously calls each of the N
microservices required to build a response
• Handles security and hides back-end
• Load balances
• Applies limited business logic
• Meters APIs
• Logs centrally

Ship Your Applications Headless
Put your front and back ends in different geographies, fault domains, etc
Design, develop, deploy and manage your front and back ends differently
73
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
API
Application
Datastore
Infrastructure
BackEnd
Add to Cart Inventory Product Details Search
FrontEnd
API Gateway
Web Content
Management System
Custom Application
Very Different
Requirements
• Security
• Elasticity
• Performance
• Traffic patterns
or

Consider Deploying Front/Back Ends To Different Locations
74
Offers strong uptime. Hard to take down the system!
Cloud Data Center
Front End
Application
Cloud Data Center
Front End
Application
Cloud Data Center
Front End
Application
Cloud Data Center
Back End
Application(s)
Cloud Data Center
Back End
Application(s)
Cloud Data Center
Back End
Application(s)
Any front end should be able
to connect to any back end

Eliminate Singletons
75
Singletons are an evil necessity. They do not have to be “fixed”
Dynamic
Available at well known service
name. Instances are dynamically
elected at runtime. If instance
goes down, another will take over
Fixed
Available at IP/ports. Instances
are long-lived, though IP/ports
change. If instance goes down,
people will know

Review: Architecture Changes for Cloud Native
76
Adopt Microservices
Adapt to a Distributed World
Make Your Middle Tier
Stateless
Remove Hard Coded
IPs, host names, etc
Geographically
Distribute Workloads
Adopt Coreography
Over Orchestration
Plan for Unreliable
Infrastructure
Adopt an API Gateway
Ship Your Applications
Headless
Deploy Your Front and Back
Ends To Multiple Locations
Eliminate Singletons

77
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Developers Want Choice in Programming Language
• Different languages for
different specialized
microservices
• Remember – the goal of
cloud native is time to
market. It is NOT about
long-term maintainability
• If maintenance becomes an
issue, rewrite the
microservice over a
weekend
78
Use the language that works best
API
Application
Datastore
Infrastructure
Inventory
Microservice
API
Application
Datastore
Infrastructure
Chat
Microservice
API
Application
Datastore
Infrastructure
Product Recommendation
Microservice

Remember That Latency is Everywhere
Asynchronously make calls to all remote systems
High latency within a single cloud.
Can no longer assume 2ms between
application and data tiers
Application is now comprised of
many different microservices, each
with its own datastore/database
Application is now deployed to
multiple data centers, in multiple
availability zones, in multiple regions

Synchronous Calls With Microservices Are Very Bad
80
Product Catalog Product Pricing Inventory
Chaining == coupling == downtime
The availability of microservice A depends on B, B depends on C, etc

Avoid Synchronous Calls for Read-only Data By Copying
81
Product Pricing
Inventory
Promotions
Product Pricing
Inventory
Promotions
Product Pricing
Inventory
Promotions
Product Catalog
• Synchronous in-memory calls
• Data is 100% consistent
• No data is duplicated
Request for
Category Page
Product Catalog
Product
Pricing
Inventory
Promotions
• Synchronous calls to product catalog microservice
• Product, pricing, inventory and promotions
microservices are systems of record; they publish
asynchronously to Product Catalog microservice when
updated
• Product Catalog microservice is not always consistent
• Data is duplicated – two or more microservices may
contain the same data
Traditional
Monoliths
New
Microservices

Circuit Breakers Prevent Cascading Failures
• Rule #1 of microservices – avoid coupling!
– Synchronous = two systems are coupled
– Asynchronous = no coupling
• Cascading failures happen when request-handling threads
are waiting on a response from a remote system
• Circuit breakers make synchronous calls from another
thread pool to avoid binding up request-handling threads
• Hystrix (Java-based) is well-known and solves this problem
82
Cascading failures are more common with microservices

Start Using a Higher Level of REST
83
• Level 0
– Use HTTP as a tunneling mechanism only
– Interact with /OrderService
– Use HTTP GET/POST only
• Level 1
– Start requesting individual resources
– Interact with /OrderService/12345
• Level 2
– Start using HTTP verbs - HTTP GET/POST/PUT/DELETE/etc
– Responses come back using correct HTTP codes – e.g. HTTP 409 for a conflict
• Level 3
– Application itself tells client how to interact with it - similar to hyperlinks in a web page
– <link rel = "delete" uri = "/OrderService/12345/delete"/> under <order id="12345">
REST is the currency of cloud native

Emit Logs as Event Streams
84
Can’t do anything with log files sitting on a container’s local storage volume
Log Entry Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry Log Entry
Container
Instance of Application
Log Entry Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry Log Entry
Container
Log Entry Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry
Log Entry Log Entry
Container
Capture, Aggregate, Search, Troubleshoot, etc

Review: Code Writing Changes for Cloud Native
85
Allow Developers to Select Programming Languages
Code Assuming Latency Is Everywhere
Don't Synchronously Couple Anything
Use Circuit Breakers
Emit Logs as Event Streams
Start Using a Higher Level of REST

86
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Test Everything, All the Time. Preferably Automated
87
Acceptance Testing Usability Testing Component Testing
Did we build the right
thing? Do business
requirements make
sense?
Performed manually
directly by business
users
Entire system is
tested using end-
clients
How usable is the
system? Will end-
users like it? Is it fast
enough?
Performed manually
by business users
Entire system is
tested using end-
clients
Does each
microservice work in
isolation? Is it fast
enough?
Performed
automatically with
each microservice
release
Each microservice is
tested in isolation.
Dependencies
stubbed out
Does each fragment
of code work in
isolation?
Performed
automatically with
each build
Each method, or
similar fragment is
captured
Unit Testing
Frequency of Testing

Artifacts Are Now Immutable Containers – Not EARs, WARs
88
Containers can have anything in them and are highly portable
Hardware
Operating System
Hypervisor
VM 1 VM 2
Legacy
Hardware
Operating System
Container 1 Container 2
Cloud Native
• No more installing a JVM,
app server, and then
deploying the artifacts to
them
• Build the container once,
deploy it anywhere. Can
include complex
environment variables,
scripts, etc
• Containers should be free of
state and configuration
• Containers should not
assume they are able to
write to a persistent local
file system
OS
App Server
EAR/WAR
OS
App Server
EAR/WAR The Artifact You Deploy

Best to Declaratively Build Containers Using Dockerfiles
89
FROM centos:centos6
# Enable Extra Packages for Enterprise Linux (EPEL) for CentOS
RUN yum install -y epel-release
# Install Node.js and npm
RUN yum install -y nodejs npm
# Install app dependencies
COPY package.json /src/package.json
RUN cd /src; npm install
# Bundle app source
COPY . /src
EXPOSE 8080
CMD ["node", "/src/index.js"]

Everyone Should Be Able to Build Anything At Any Time
• Manual one button
build/deploy
• Scheduled builds - every
day, every week, etc
• Builds triggered by code
checkins
• If post-build validation fails,
report it
Set it and forget it
90

Review: Code Building Changes for Cloud Native
91
Use Automation to Test Everything, All the Time
Switch to Immutable Containers
Allow Anyone to Build Anything At Any Time

92
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Constantly Deploy Each Microservice
93
No need to batch together many changes to single app
One Application Many Microservices – Deploy Many Times/Day
API
Application
Datastore
Infrastructure
Inventory
Microservice
API
Application
Datastore
Infrastructure
Payment
Microservice
API
Application
Datastore
Infrastructure
Profile
Microservice
API
Application
Datastore
Infrastructure
Product Catalog
Microservice
By definition, each microservice can be
written, built, and deployed independently
User Interface
Application
Datastore
Infrastructure
Deploy Constantly – Even HourlyDeploy Quarterly
Long, serial process to
deploy anything

Run Many Versions of the Same Microservice Concurrently
94
Monolithic
Application
v1.1
Microservice A
Microservice A
Microservice A
Microservice A
Microservice AMicroservice B
v1.1
Microservice A
Microservice A
Microservice A
Microservice A
Microservice A
Microservice A
Microservice A
Microservice A
Microservice A
Microservice AMicroservice B
v1.2
Microservice A
Microservice A
Microservice A
Microservice A
Microservice A
Run only one version of the same
application in the same environment
Run many versions of each
microservice in the same environment
Microservice A
v1.2
Microservice A
v1.1

Use Blue/Green Deployments When One Version Can Be Live
95
Blue is current. Deploy new code to green. Switch load balancer from blue to green
Application
Live in production - v1.1
Application
In production but no
traffic from load
balancer
v1.2
“Blue” Environment “Green” Environment
Identical Environments
Each capable of handling 100% of production traffic
“Blue” can be shut down after code deployment is successful.
New CodeCurrent Code

Review: Code Deployment Changes for Cloud Native
96
Constantly Deploy Each Microservice
Run Many Versions of the Same Microservice
Concurrently
Use Blue/Green When You Have to

97
Maintaining
Deploying
Code
Building
Code
Writing
Code
Architecture
Provisioning
Procuring
Project
Management
Structuring
Organization

Ongoing Operations is Now a Shared Responsibility
98
In perpetuity. No more “throwing it over the wall”
New Goal
Add new features and keep the
system stable, fast and available
Developers
Paid to add new features
that may break things
Operations
Paid to keep system
available, stable, and fast

Containers Should Be Immutable
99
Patches to
System Software
New Version of
Application
Configuration
Changes
Build and deploy a new container
Never touch a container that’s already been built

Stop Manually Fixing Problems
100
In no case should an administrator fix issues by hand. Should be 100% automated
Auto-scaling will automatically launch a new container on new
hardware as load dictates
Hardware Failure
Example: motherboard failed
Auto-scaling will automatically launch new containers as load
dictates
Network Failure
Example: switch failed
Health checking should fail and the container will be culled.
Auto-scaling will automatically launch a new container as load
dictates
System Software Failure
Example: kernel panic
Application Software Failure
Example: bad file permissions
Fix the source (your application, your container, your
Dockerfile, etc) and re-deploy your entire application

Review: Maintenance Changes for Cloud Native
101
Share Development and Operations Responsibilities
Make Containers Immutable
Stop Manually Fixing Problems

Oracle: Building Cloud Native Applications

Oracle: Building Cloud Native Applications

More Related Content

What's hot

Viewers also liked

Similar to Oracle: Building Cloud Native Applications

Recently uploaded

Oracle: Building Cloud Native Applications

Editor's Notes