Cloud-Native-Data with Cornelia Davis

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Cloud-Native Data
Extending Cloud-Native Beyond the App
Cornelia Davis
Sr. Director of Technology
Pivotal
@cdavisafc

@cdavisafc
Me?
Developer (wasn’t Ops)
Web architectures for >10 years
Cloud-native for 5 years
Cloud Foundry for 5 years
M...

A Seattle book store
deploys code, on average,
every second

© 2015 Pivotal Software, Inc. All rights reserved. 4
Search µservice .
Image µservice .
Item Master µservice
Reviews µserv...

Why?
• Scale Applications
• Scale Teams
• Independent Development Cycles
• Experimentation
• Resilience

6
Continuous Delivery
of
Customer & Business
Value

Obstacles
• Silos: Dev, QA, Operations is
typical. No shared common goal
• Dissimilar Environments - “It
works on my machi...

Enabling Patterns
• Reinventing the Software
(Delivery) Value Chain
• Cloud-native Software
Architectures
• The Right Plat...

© 2015 Pivotal Software, Inc. All rights reserved. 9
Search µservice .
Image µservice .
Item Master µservice
Reviews µserv...

10
Presentation
Frontend-integration
Order
Status
µService
Search
µService
Item 
Master
µService
Item
MetaDatµS
ervice
Car...

New LIGHTWEIGHT ARCHITECTURES are emerging  Microservices addressing speed to market and cloud scale
Monolithic / Layered ...

12 http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

Spring Cloud Services 1.0.0
3
Spring Cloud Services
Config Server Service Registry Circuit Breaker
Dashboard

14
Operational Visibility: Distributed Tracing
• Latency visibility into a request’s end-to-end call graph
• Quickly ident...

• PCF Developers can redirect application traffic to a desired request
path in order to use logging, authentication or rat...

New LIGHTWEIGHT ARCHITECTURES are emerging  Microservices addressing speed to market and cloud scale
Monolithic / Layered ...

18
What is Cloud-native
Data?

It’s not this

At the Intersection of Cloud-native App Architecture & Data
What is Cloud-native Data?
Patterns
• Data-services Topology
•...

Data-services topology
Goals/Needs:
• Resilience
• Tenancy
• Security - networks?
• Day 2 Ops - Backup/Recovery

Data-services topology: Resilience
PCF
Data Center 1
PCF
Data Center 2

PCF PCF
Pipelines Distribute Deployments
Active/Active
Data Center 1 Data Center 2

PCF PCF
Pipelines Distribute Deployments
“Active/Active”
Data Center 1 Data Center 2

PCF PCF
Pipelines Distribute Deployments
“Active/Active”
Data Center 1 Data Center 2

PCF PCF
Pipelines Distribute Deployments
“Active/Active”
Data Center 1 Data Center 2
WAN Replication

Interfacing to Existing Data Sources
Goals/Needs:
• Cost reduction - offloading MIPS
• Agility

Pattern: Data API
• Microservices do not access data layer directly
• Except for the micro services that implement the dat...

Anti-pattern: Stateless Data APIs*
29
* We will maintain statelessness
at the app level
This is the architecture that domi...

Pattern: Microservice Needs a Cache
30
We’ll have a lot more to discuss with respect to caching
… stay tuned

Pattern: Data API
• Microservices do not access data layer directly
• Except for the micro services that implement the dat...

Pattern: Data API
• Microservices do not access data layer directly
• Except for the micro services that implement the dat...

Pattern: Versioned Data API
• We are already familiar with versioned
micro services…
V1 V2
Possibly coupled with
Pattern: ...

Caching Rethought
Goals/Needs:
• Performance
• Interface to existing data sources
• Resilience
https://content.pivotal.io/...

Caching Patterns
Look Aside
• Attempt retrieval from cache
• Client retrieves from source
• Write into cache
! ?
"
#
Advan...

Caching Patterns
Read-through
• Attempt retrieval from cache
• Cache retrieves from source and stores
in cache
• Return va...

Caching Patterns
Write-through
• Write to cache
• Cache writes to source
• ack sent to client
!
"
#
Advantages
• Simpler c...

Caching Patterns
Write-behind
• Write to cache
• ack sent to client
• Cache writes to source asynchronously
!
"
#
Advantag...

© Copyright 2014 Pivotal. All rights reserved.

40
Why Cloud-Native Apps Need an In-Memory Cache
As an integrated service on a world class platform
Fast, available
micros...

Pattern: Cache Warming
• Loading the
cache can be
expensive
• Spring Cloud
Data Flow for
modern ETL
Sources
Destination
Sp...

Microdatabases & Event Sourcing
Goals/Needs:
• Agility ∴ Autonomy
• Resilience

Pattern: Database per Microservice
• Supports Polyglot
persistence
• Independent availability,
backup/restore, access
patt...

My
Connections
API
Posts
API
New from
Network
API

Client Side Join
47
My
Connections
API
Posts
API
New from
Network
API

Event-driven Systems
48
My
Connections
API
Posts
API
New from
Network
API

Independent Databases - Shared Entities
• We’ve started to break up the
data monolith
• BUT our data integration
“strategy...

My
Connections
API
Posts
API
New from
Network
API

My
Connections
API
Posts
API
New from
Network
API

My
Connections
API
Posts
API
New from
Network
API
Unified Log

CQRS is what allows
the unified log to be
the source of truth
54
My
Connections
API
Posts
API
New from
Network
API
Unified...

But then, what about people?

Info Sec
Srv Build
Cap PlanNetwork
OpsMid. Eng.
SW Arch
SW Dev
Client SW Dev
Svc Govern
CUSTOMER FACING APP TEAM
Ops
Cap P...

57
Cloud-native Patterns
Summary

Legacy Data
Access
Service APIs
Data APIs
Shared DB
Database Per
Service
Data Integration
Client-side “Joins”
Event Sourci...

Transforming The Way the World Builds Software
59
And Software Needs Data

Thank you!
Cornelia Davis
Sr. Director of Technology
Pivotal
@cdavisafc
Discount code 40% off!: 40cloudnat

Cloud-Native-Data with Cornelia Davis

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

Share
Email

Infinite power at your fingertips w... by Activeeon 1455 views
Agile Development and DevOps in the... by jeckels 920 views
Microsoft Azure Big Data Analytics by Mark Kromer 4156 views
Cloud Native, Cloud First, and Hybr... by Amazon Web Services 1236 views
Building Cloud Native Software by Paul Fremantle 8291 views
Cloud Native Architectures for Devops by cornelia davis 2229 views

Top clipped slide

1 of 61 Ad

1 of 61 Ad