While a number of patterns and architectural guidelines exist for cloud-native applications, a discussion about data often leads to more questions than answers. For example, what are some of the typical data problems encountered, why are they different, and how can they be overcome? Join Prasad Radhakrishnan from Pivotal and Dave Nielsen from Redis Labs as they discuss: - Expectations and requirements of cloud-native data - Common faux pas and strategies on how you can avoid them Presenters: Prasad Radhakrishnan, Platform Architecture for Data at Pivotal Dave Nielsen, Head of Ecosystem Programs at Redis Labs

1. Cloud-Native Data: What Data Questions to Ask
When Building Cloud-Native Apps Webinar
Dave Nielsen
@DaveNielsen
Prasad Radhakrishnan
@Prasad_0101

2. Cloud Native Data

3. Data in the context of Cloud-Native Apps
AppsData
IaaS / Cloud
Platform

4. Quick glance at the world of
Cloud-Native Apps

5. App world: the move to Microservices
Exit SoA Embrace Microservice design pattern
The current trend

6. App world: the move to Microservices
The Twelve-Factor App (https://12factor.net/)
● Fourth Amendment says “Treat backing services as attached
resources”
● Fifth Amendment says “Execute the app as one or more
stateless processes”
“Backing Services” refer to
Databases and Datastores
Microservices demand dedicated
single ownership of schema and
the Datastore
Microservices are stateless
Databases and Datastores are
stateful

7. App world: the move to Microservices
Few other things to note
● Microservices approach is incomplete without full adoption of
CI/CD
● Microservices development and Agile methodologies: a perfect
marriage!
● With the help of Cloud Foundry and other Cloud-Native
platforms, Microservices are multi-cloud deployable
● More investment in the SRE model and approach as opposed to
traditional support model - early stages of adoption but
increased awareness.
Microservices + Agile + DevOps
mindset/model is helping
Enterprises to aspire to be like
Amazon, as in, deploy Code to
Production every few seconds or
minutes, instead of once a
quarter!

8. Flow of Information and time delays
App
[producer of
Data]
DB
Data Lake
Apps
dependent
on SoR
Other Apps
dependent
on SoR
BI Analytics
App from a
diff BU
ODS [stale
by few
hours]DataReplication
Queries
t0
t1
t2
Insights
from
Analytical
Models t3

9. Flow of Information and time delays
App
[producer of
Data]
DB
Data Lake
Apps
dependent
on SoR
Other Apps
dependent
on SoR
BI Analytics
App from a
diff BU
ODS [stale
by few
hours]DataReplication
Queries
t0
t1
t2
Insights
from
Analytical
Models t3

10. Flow of Information and time delays
App
[producer of
Data]
DB
Data Lake
Apps
dependent
on SoR
Other Apps
dependent
on SoR
BI Analytics
App from a
diff BU
ODS [stale
by few
hours]DataReplication
Queries
t0
t1
t2
Insights
from
Analytical
Models t3
Traditional approach to Data
Movement is time prohibitive
‘Data’ Impedes overall Agility
Increases time to Market

11. Cloud-Native Apps & noticeable
impacts on “Data”

12. Monolith vs. Non-Monolith
No. of Databases increased and
projected to further increase.
Additional HTTP calls got introduced.

13. A shift in balance of power
Back then
● Sub-millisecond response time reqt was for few
specialty Apps.
● Data Modeling
● Regimented approach to schema changes in
relational DBs
● Stored Procedures for complex business logic
that Producer app couldn’t handle.
● Database Triggers.
● Data Replication in batch mode
● New App gets direct access to an existing
Database
At present / going forward
● Microservices more often than not demand
sub-millisecond response time.
● Domain Modeling.
● Increased NoSQL adoption for Applications to
handle Schema changes & versioning.
● Externalize the complex business logic to
additional Microservices.
● Eventing.
● Event Store & Event Sourcing
● Microservice owns the schema + data, New App
gets API only access. No direct access

14. Key Concerns / Risks / Considerations
Will there be an explosion in number of Databases or Datastores that
an Enterprise will end up provisioning and supporting?
● Highly likely!
● The solution includes Cloud or Platform managed Data Services.
● One among the biggest reasons for public cloud adoption (AWS, GCP, Azure) is their
ever growing list of different types of DataServices. Like Aurora, Cosmos DB, Spanner,
Dynamo DB etc.
● There are Pros and Cons to “managed data services”. It’s an opinionated install with
less control over the underlying configurations. It’s important to understand the
trade-offs.
● Self-Service!

15. Key Concerns / Risks / Considerations
Eventing, Streams and Real-Time Data Processing
● Eventing is cool!
● Idempotency needs to be explicitly handled and can potentially be tricky!
● Understand the differences: At most once vs. At least once vs. Exactly once.

16. Key Concerns / Risks / Considerations
High Availability is a shared responsibility (between Apps & Data)
● High Availability: A Service or DataStore’s percentage of uptime in a given year.
● Understand your environment’s lower to higher order failure domains. E.g., PID, VM,
Rack, Datacenter etc. and how it can impact HA.
● There is no such a thing as 100% uptime.
● Embrace and Architect for Failures.
● Cloud-Native App Developers should choose a Datastore knowing its RPO, RTO, HA
characteristics and Failover strategy.

17. Key Concerns / Risks / Considerations
“Data” for Multi-Cloud
● Active-Active or Active-Passive?
● Would you use both Cosmos DB and Spanner?
● Data in most scenarios can only be Eventually Consistent across Clouds (or Networks).
● Understand Conflict Resolution in an active-active topology and try hard to avoid the
need for explicit Conflict Resolution.
● Portability is key. Keep an eye on K8S and its ability to support Stateful workloads.

18. Cloud Native Data
QUESTIONS TO ASK WHEN BUILDING CLOUD NATIVE APPS
Dave Nielsen
Head of Ecosystem, Redis Labs

19. ▪ Polyglot microservices
▪ Globally distributed data / Active-Active
▪ Publish and subscribe / Pub-sub
▪ Caching transient high traffic data via
endpoint integrations
▪ Message queue
▪ Eventing / Asynchronous messaging
▪ Transactions / Event sourcing
▪ Application session state / User identity /
Personalization
▪ Streaming / Time series
Some Cloud Native Data Use Cases
19
▪ Relational data
▪ Document / Graph / Key-value store
▪ Fast Lookups and Low latency / Stock
quotes / Real time bidding
▪ Metering / API rate limiting
▪ Event logs
▪ Data created in one place, duplicated
elsewhere
▪ Big Data / Analytics
▪ Object store / storing media files where
durability matters

20. Characteristics of Cloud Native Data
20

21. General Lifecycle Considerations
• How the code is written, tested, released,
and managed in production?
Cloud Native-specific Lifecycle
• Continuous Change (CI/CD)
• Spin-up and spin-down instances
• Agile / DevOps
• Automation
Considerations for the Cloud Native Application Lifecycle
21

22. Considerations for Cloud Native Data Lifecycle
22
Data Lifecycle is Different than App Lifecycle
• Production data cannot be replaced
with dev/test data. Instead it must be
protected during app updates.
• Some data needs to be duplicated for
fail-over or geographic distribution
• Some data, such as customer orders,
needs to be backed up
• Other data, such as session data, may

23. • Most data isn’t ephemeral, but your database instances can be!
– Database instances will be spun-up and down for dev, test and production
• Where your data is deployed should not drive how it is designed
– Database needs to run across multiple clouds, regions and platforms
– Data should be portable across multiple deployment pipelines (dev, test, prod)
– Built-in role-based access controls, managed by your platform
– Database configuration is managed by your platform
Characteristics of Cloud Native Data
23

24. • Multi-cloud, Multi-region, Multi-platform
• Data portability
– Database instances are ephemeral, data is not
– Database should support different data
strategies across multi-region deployments
• Database config for dev, test and prod
can be set via a platform config API
– Ex: Service broker can pass config details for
each deployment
Characteristics of a Cloud Native Database
24
Ex: USER PROFILE/SESSION MANAGEMENT

25. Example: Active-Active with Geographic Distribution
25
App
App
App
• CRDT’s proven technology
backed by deep academic
research
– Local latencies guaranteed
with consensus free protocol
– Built-in conflict resolution
– Strong eventual consistency

26. 26
Choose a Database that Supports All Cloud Native Data
Requirements
Graph
RelationalCachingHigh-Speed
Transaction
s
Fast Data
Ingest
Time Series Geospatial
Indexing
MessagingAnalytics
Message
Queue
Document

27. Cloud Native Data Deployments
Who’s Responsible for what?
27

28. – Application/Database configuration should
be managed by the Platform
– Database HA and scale should be automated
by the platform
– Aim for consistency in the app and database
across all deployments
What is the Platform Responsible For?
28

29. • The data is managed by the database, the
platform and the DBA (who leans on
platform)
• Let the platform automate as much as possible
• Data solutions
• HA, Rebalancing, etc.
• Let the database do the work for you!
• Minimal admin overhead
What is the Database Responsible For?
29

30. • Focus on the application requirements
• Some changes to data can be made
directly without requiring the DBA
• Iterate quickly
Make it Easy For The Developer
30

31. “Cloud Native is about how applications/data
are deployed, not where.”
31

32. Purpose-built to Support Cloud Native Data in Your Cloud Native Applications
32
Redis Enterprise for Pivotal Cloud Foundry

33. 33
The Redis Enterprise Advantage
Integrated
ModulesAutomation
& Support
Redis on Flash
+ More Savings
Performance
at Scale
Built-in
Search
Active-Active
Geo
Distribution
(CRDTs) Robust
Security
HA, Durability,
DR

34. Redis Enterprise: Uniquely Suited to Cloud Native Applications
Redise
Node Redise
Cluster
Enterprise Layer
34
Integrated Modules

35. Redis Tops Popular Database Rankings,
and is the “Most Loved Database”
35

36. Redis Tops Database Popularity Rankings
36

37. What Makes Redis So Popular?
Simplicity VersatilityPerformance
ListsSorted Sets
Hashes
Hyperloglo
g
Geospatial
Indexes
Bitmaps
SetsStrings
Bit field
NoSQL Benchmark
1
Redis Data Structures
2 3
Widely Applicable
37

38. 38
Redis Uses Span Many Solutions
Real-time
…AND MANY MORE
IoT
Metering
Fraud
Mitigation Social Apps
Personal-
ization
Ecommerce
Search

39. Redis for Microservices
39

40. Redis for Polyglot Microservices
40
• From module to service
• Loosely coupled
components
• Easy of deployment
(containerize, …)
• Scale per service
• DevOps approach
• Polyglot persistence
• Variety of protocols
• Implemented using several
programming languages &
frameworks
• Different kinds of
developers
• Monolithic vs distributed
• Self-contained services
• Domain-specific separation
• One service can use
another
• Aggregate services
Why MicroservicesMicroservices Definition Redis for Polyglot

41. Global Financial Use Case
41

42. Case Study Of A Global Financial Company Architecture
42
• Run across 8 multi-cloud environments, including public clouds and private data centers
• Performance of transactions < 5 milliseconds
• Cross datacenter availability and geo replication to maintain data consistency
• Scale to handle large amounts of constantly changing data
• Handle all types of data including session data, user data, credit card information, location, etc.
• Distributed highly reliable database across all 8 datacenters for fast response
• Security and performance to ensure authentication and fraud detection
CachingHigh-Speed
Transactions
MessagingAnalytics
Geo
replication
Time Series

43. • Fast time to market – can deploy consistently across
dev, test and production
• Scale from single-instance to Redis clusters with no
application changes or downtime
• Enjoy the added flexibility of persistence, geographically
distributed active-active or active-passive models,
tunable consistency
• Design and build for microservice-based applications
orchestrated in PCF environments
• Fast provisioning with BOSH
• Enhanced security
• Assured high performance for a multitude of functions –
Redis Enterprise on PCF

44. Get Started with Redis Enterprise on Pivotal
44

45. Get Started Right Away
45
https://network.pivotal.io/products/redis-e
nterprise-pack
Pivotal Cloud Foundry Pivotal Web Services
Free Redis Cloud service available on the
Pivotal Marketplace

46. promo code SVCLOUDNATIVE
50% off RedisConf conference pass
-or-
50% off pre-conference workshop (Intro or Advanced)
AND free conference pass

47. Embedded OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code & Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
>cf push >kubectl run
YOU build the containerWE build the container
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concourse
Concourse
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
OpenServiceBrokerAPI
Repair
— CVEs
Repave Rotate
— Credhub

48. Cloud-Native Data: What Data Questions to Ask
When Building Cloud-Native Apps Webinar
Dave Nielsen
@DaveNielsen
Prasad Radhakrishnan
@Prasad_0101

49. Transforming How The World Builds Software
© Copyright 2017 Pivotal Software, Inc. All rights Reserved.