Cloud native applications offer scalability, flexibility, and optimal use of compute resources. Serverless functions interacting through events, leveraging cloud capabilities for persistent storage and automated operations take organization to the next level in IT. This session demonstrates polyglot Functions interacting with native cloud services for events and persistence (Object Storage and NoSQL Database) and leveraging the Key and Secrets Vault, Monitoring and Notifications services for operational control. A lightweight API Gateway is used to expose APIs to external consumers. Infrastructure as Code is the guiding principle in deploying both cloud resources and application components, through OCI CLI and Terraform. This session leverages many cloud native (enabling) services in Oracle Cloud Infrastructure. The session will introduce concepts, then spend most of the time on live demonstrations. All sources are shared with the audience, to allow participants to create the same application in their own cloud tenancy.
What is so great about Cloud Native Applications? How do you create one? I will explain the first and demonstrate the second. On Oracle Cloud Infrastructure, using services that anyone can use for free, I will live create a cloud native application that streams, persists, notifies, scales, monitors
1. Cloud Native
Application
Development
- build fast, cheap,
scalable and agile
software
on Oracle Cloud
Infrastructure
DOAG 2020| Oracle Cloud Native Application Development
DOAG 2020, November 2020
Lucas Jellema, CTO & Architect AMIS | Conclusion
2. Lucas Jellema
CTO for AMIS | Conclusion
Cloud Solution Architect
DOAG 2020| Oracle Cloud Native Application Development
lucas.jellema@amis.nl | technology.amis.nl | @lucasjellema | lucas-jellema
3. A traditional application
DOAG 2020| Oracle Cloud Native Application Development
WebApp
Database
(RDBMS?)JVM
Java EE
Operating System
Hardware
3rd party libraries
& frameworks
Monolith,
Microservice,
Module
Platform & Language
runtime features
(transaction
management, HTTP
request handling, state
management & in
memory caching, IAM,
…
Custom Code
4. IaaS
PaaS
Cloud Native Application
DOAG 2020| Oracle Cloud Native Application Development
Event
Hub
Vault
Function
IAM
Container
Engine
Web
App
Function
Cache
Mail
Monitor
ScalingAlert &
Notify
• Managed
• Quick Provision &
Decommission
• Distributed
• Automated
• Pay per use/TCO
• Vendor takes runtime
responsibility
• Enterprise grade platform
accessible to startups
• Rapid Innovation
• Security & Availability
• Scale
Logging
Data
Integration
5. Characteristics of Cloud Native Applications
• Modern – born in the cloud (era)
• Emphasis on interaction (and decoupling) points:
• Between custom components: APIs, Events / message format
• From custom to PaaS Services: Service APIs, Formats and Protocols
• Less emphasis on [and insulation from?] implementation of PaaS (platform) services
• Key are industry standard APIs that are implemented by many different services
• Custom code in small, simple units
• No explicit Infrastructure (serverless, software defined) & managed Platform components
• Stateless and Dynamically Scalable (Elastic) – expand and contract
• Pay per Use
• Automation – no hands on ACC and Production (speed, repeatability, dynamic adjustment)
• Infrastructure/Platform as Code & Application CI/CD [pipelines]
DOAG 2020| Oracle Cloud Native Application Development
6. Some (PaaS) Cloud Capabilities
for Cloud Native development
• Functions (serverless application code)
• API Gateway
• Events / Queued Messages
• Data Persistence
• Cache
• Security
• Network access
• Web Application Firewall (DDOS & more)
• Authentication & Authorization
• Credentials & Key Management, En|Decrypt
• Automation – CI/CD & DevOps Pipelines
• Monitoring
• Metrics, notification, logging
• Auditing
• Cost Control
DOAG 2020| Oracle Cloud Native Application Development
API
Gateway
Function
Event Topic
Data
Persistence Cache
Monitoring
Security(WAF,Network,Authorization
CI/CD & DevOps Pipelines
8. Oracle Cloud Gen2
• First generation is now called Classic Cloud
• Mainly On Premises software, running in
Oracle’s data centers – with some (small)
degree of operational management (mainly
provisioning and patching)
• Not Cloud Native
• Gen2
• (largely) Designed from the ground up for the
cloud
• Applying lessons learned – from classic cloud as
well as AWS and Azure
• Some focused acquisitions (Dyn, Corente,
Palerra, Wercker, DataScience.com)
DOAG 2020| Oracle Cloud Native Application
Development
9. Oracle Cloud
Infrastructure
• Generation 2 Cloud
• Main themes:
• Competitive Pricing
• Secure by Design
• Open
• Enterprise (Technology)
• Trials – 30 days, $300 credits
• An always-free tier
• With Autonomous Database and
Autonomous Data Warehouse
DOAG 2020| Oracle Cloud Native Application
Development
10. Oracle Cloud Infrastructure Regions – current and planned
DOAG 2020| Oracle Cloud Native Application Development
16. Traditional Application:
Tweets to NoSQL, Streaming & Email
Streaming
NoSQL
Database
#doag2020
DOAG 2020| Oracle Cloud Native Application Development
17. Traditional Application:
Tweets to NoSQL, Streaming & Email
Streaming
#doag2020
DOAG 2020| Oracle Cloud Native Application Development
WebApp
Database
(RDBMS?)JVM
Java EE
Operating System
Hardware
3rd party libraries &
frameworks
NoSQL
Database
log
Logging? Monitoring?
A single unit to build | test |
deploy | scale | fail
Synchronous (internal)
interactions
No (built-in) monitoring |
logging
Platform and Infrastructure
to install | configure | secure |
operate | maintain
18. Demonstration of Cloud Native Application on OCI:
Tweets to NoSQL, Streaming & Email
Streaming
NoSQL
Database
#doag2020
DOAG 2020| Oracle Cloud Native Application Development
19. Cloud Native design
Tweets to NoSQL,
Streaming & Email
Healthcheck
Streaming
Events
Object
Storage
Notifications
API
Gate
way
NoSQL
Database
Function
Function
Write JSON
document with
Tweets
Retrieve X
minutes worth
of tweets
Store JSON
file with all
tweets in time
period Trigger
function with
event
Create
Tweet
Records
in NoSQL
Send
Tweet
Report as
email
Invoke Tweet
Aggregator every
X minutes
Cloud Event
for new
JSON file
Publish
each
Tweet to
Stream
Expose
function to
external callers
Publish to
Notification
Topic
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Get Twitter
credentials in
secure way
from vault
Custom code in multiple small
units that each build | test |
deploy | scale | fail
Asynchronous interactions
Managed Platform (and
underlying Infra) with built in
logging | monitoring | Ops
Configured through Infra as
Code
Auditing
Monitoring
Logging Container
Registry
Vault
21. Platform Preparation – Infrastructure as Code
using Resource Manager & Terraform plan
Object
Storage
Object
Storage
Bucket
Compartments
DOAG 2020| Oracle Cloud Native Application Development
Vault
Secure Vault
with Keys for
Twitter
credentials
Compartment
for Application
resources
Configuration
of Logging
Service for
Functions
Virtual Cloud
Network
Internet
Gateway
Route Table
Security
Lists
Virtual Cloud Network
with private and public
subnet and access from
and to public internet
API
Gate
way
Skeleton API
Gateway for
deploying API
routing rules to
Groups
Dynamic
Groups for API
Gateway and
for Functions
Streaming
Stream for
publishing
messages to
NoSQL Databas
Table for
persisting record
Resource
Manager
Auditing
Logging
Container
Registry
22. Serverless Function
& Object Storage
Object
Storage
Function
Write JSON
document with
Tweets
Store JSON
file with all
tweets in time
period
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Vault
Get Twitter
credentials in
secure way
from vault
Retrieve X
minutes worth
of tweets
Container
Image Registry
Central Logging
Service
Function
Tweet
Summarizer
Auditing
Logging
Container
Registry
23. Introducing Serverless Functions
• The Container Native Serverless Framwork
• Project Fn is an open source FaaS platform
that you can run anywhere -- any cloud or
on-premise.
• It implements serverless Functions through
Docker containers
• Out of the box support for many runtimes
•
• also support for custom Docker
Container (white box)
• Project Fn is the foundation for Serverless
Functions on OCI
DOAG 2020| Oracle Cloud Native Application Development
24. Fn Functions – request handling
DOAG 2020| Oracle Cloud Native Application Development
Function
Implementation
(Node, Java, Go,
Python, custom
Docker, …)
FDK
Function as a Service framework
HTTP Request
With method, headers, query
parameters, url path, body
input – derived from body
ctx – raw HTTP request components
26. Function tweet-summarizer – source code in GitHub
DOAG 2020| Oracle Cloud Native Application Development
Sources: bit.ly/cloud-native-latam2020
27. Configuration Settings for Function tweet-summarizer
result in environment variables accessible within the function
DOAG 2020| Oracle Cloud Native Application Development
Vault
28. Application Design
Object
Storage
Function
Write JSON
document with
Tweets
Store JSON
file with all
tweets in time
period
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Get Twitter
credentials in
secure way
from vault
Retrieve X
minutes worth
of tweets
Function
Tweet
Summarizer
Container
Image Registry
Central Logging
Service
Logging
Container
Registry
Vault
29. DOAG 2020| Oracle Cloud Native Application Development
Functions vs Containers vs VMs
• Serverless
• No IaaS DevOpS
• Stepless Pay per use
• Free-Tier
• Auto-Scale
• Automatic Fail Over
• Integration with OCI
• Triggered by OCI
• Call OCI services
• Leverage OCI for
Audit, Log, Monitor
• Restricted, simple
programming model
• Short running
Functions Containers on OKE
• Serverless runtime
• Initial IaaS Setup
• Pay per VM,
scale per VM
• Auto-Scale by K8S
• Some auto Fail Over
• Little integration with OCI
• Kubernetes for Ops
• Flexible programming
model – any container goes
VMs
• Fully Servered
• DIY IaaS Op
• Pay per VM, scale per
VM, cheap at large scale
• Manual Scaling
• Manual Fail Over
• Little integration with OCI
• Flexible programming
model
Virtual
Machine
Functions Container Engine
For Kubernetes
30. Object Storage – Bucket for twitter reports
DOAG 2020| Oracle Cloud Native Application Development
31. Serverless Function
& Object Storage
Object
Storage
Function
Tweet
Summarizer
Write JSON
document with
Tweets
Store JSON
file with all
tweets in time
period
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Get Twitter
credentials in
secure way
from vault
Retrieve X
minutes worth
of tweets
Logging
Container
Registry
Vault
32. Function execution is triggered by
DOAG 2020| Oracle Cloud Native Application Development
API Gateway
Notifications
Events
Healthcheck
Resource
Manager
Service
Connector
Alarming
Functions
33. Function execution is triggered by
DOAG 2020| Oracle Cloud Native Application Development
API Gateway
Notifications
Events
Healthcheck
Resource
Manager
Service
Connector
Alarming
Functions
Ops automation
Application
App
Ops
Ops
Ops
Ops
Ops
Ops
App
App
34. Public and Managed
Access to Function Tweet Summarizer
Object
Storage
API
Gate
way
Function
Expose
function to
external callers
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Function
Tweet
Summarizer
Container
Image Registry
Central Logging
Service
Logging
Container
Registry
Vault
35. API Gateway
• Handle HTTP calls:
• Authorize
• Route
• Map
• Transform
• Observer
• Monitor
• Audit
• Log
DOAG 2020| Oracle Cloud Native Application Development
API
Gateway
Oracle Cloud Infrastructure
Any 3rd party
HTTP endpoint
Function
Stock
Any HTTP
Endpoint
on OCI
OIC
OKE
VM
…
Any
Service
Invoker
Expose private endpoints publicly
Stable (public) endpoints
Routing (verb & wildcard based)
Parameter and Header mapping
Request Policies
Rate limiting (per client & general)
CORS (for direct web client access)
Metrics Monitoring & Logging
Authentication & Validation
Transform headers & parameters
in requests & headers in responses
AuditingMonitoringLogging
API Gateway
36. API Gateway Configuration for public access to Function
DOAG 2020| Oracle Cloud Native Application Development
37. Leverage OCI PaaS Services and Facilities from Functions
DOAG 2020| Oracle Cloud Native Application Development
API Gateway
Notifications
Functions
Autonomous
Database
Streaming
Object
Storage
Vault
NoSQL
Database
Autonomous
JSON
Email Delivery
AuditingMonitoringLogging
synchronous
asynchronous
39. [how to] Trigger Email Notification
Object
Storage
API
Gate
way
Function
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Vault
Function
Tweet
Summarizer
Expose
function to
external callers
Send
Tweet
Report as
email
?
40. Produce CloudEvent,
Publish Notification & Send Mail
Events
Object
Storage
Notifications
API
Gate
way
Function
#doag2020
Send
Tweet
Report as
email
Cloud Event for
new JSON file
on Object
Storage
Publish to
Notification
Topic
DOAG 2020| Oracle Cloud Native Application Development
Vault
44. Send email upon Notification
DOAG 2020| Oracle Cloud Native Application Development
45. Email Subscription is Pending until confirmed
DOAG 2020| Oracle Cloud Native Application Development
46. When confirmed, the subscription is active
DOAG 2020| Oracle Cloud Native Application Development
47. When the Tweet Summarizer is invoked…
DOAG 2020| Oracle Cloud Native Application Development
Events
Object
Storage
Notification
API
Gate
way
Function
Oracle Groundbreakers Tour 2020 LATAM | Oracle Cloud Native Application Development
Vault
48. … the email informs of the new file
DOAG 2020| Oracle Cloud Native Application Development
Events
Object
Storage
Notification
API
Gate
way
Function
Oracle Groundbreakers Tour 2020 LATAM | Oracle Cloud Native Application Development
Vault
49. Produce CloudEvent &
Publish Notification
Events
Object
Storage
Notifications
API
Gate
way
Function
Send
Tweet
Report as
email
Cloud Event
for new
JSON file
Publish to
Notification
Topic
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Vault
50. Trigger Function &
Store NoSQL Records
Events
Object
Storage
Notifications
API
Gate
way
NoSQL
Database
Function
Function
Create
Tweet
Records
in NoSQL
Trigger
function with
event
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Vault
Retrieve
Tweet
Report
51. Introducing NoSQL Database Cloud Service
• Launched in February 2020 – a fully managed cloud service
• Based on Oracle NoSQL Database
• Which itself is based on Berkeley DB as storage engine
• Schemaless JSON and SQL enabled on relational data
• ACID, consistent, relational, structured, joins
• Partial JSON Updates
• Server-side updates to JSON documents
• Time-To-Live – auto-expire on records
• Built-in Data Redundancy
• Multi-region architecture
• read-anywhere/write-anywhere in “WAN” cluster
• REST APIs, OCI CLI and SDKs in Java, Python, Node and Go
• NoSQL Database Cloud Simulator for local development and testing
• Note: August 13th 2020 – launch of “Autonomous JSON Database
DOAG 2020| Oracle Cloud Native Application Development
53. Tweet Records in TWEETS_TABLE
DOAG 2020| Oracle Cloud Native Application Development
54. Trigger Function &
Store NoSQL Records
Events
Object
Storage
Notifications
API
Gate
way
NoSQL
Database
Function
Function
Create
Tweet
Records
in NoSQLTrigger
function with
event
DOAG 2020| Oracle Cloud Native Application Development
#doag2020
Vault
Retrieve
Tweet
Report
56. OCI Streaming Service
• Kafka Like – Event Queue
• Support for partitions and consumer groups
• Events are Published and Consumed through
REST API calls
• Java SDK available
• No Push to Consumer!
• Consumer has to poll
Oracle Cloud Infrastructure
Listener
OCI
Streaming
Stream
Publisher
On prem
Listener
Listener
3rd Party
Cloud
DOAG 2020| Oracle Cloud Native Application Development
58. Tweet Messages Published by Function
consumed from the Stream
DOAG 2020| Oracle Cloud Native Application Development
59. Monitoring | Health Checks
• Verify through the eyes of an external client
if endpoints are available
and respond quick and well
• Periodic or Adhoc call to an endpoint
• HTTP(S) or Ping
• Specify Headers
• Specify Interval (check frequency)
• From selected Vantage Points
• 3rd party clouds, geographic locations around the world
• Health Check results can be inspected through the Monitor in the Console
• Use health checks for live function smoke tests
DOAG 2020| Oracle Cloud Native Application Development
Functions
Healthcheck
API Gateway
60. [tricks for] Scheduling Functions
• OCI does not currently have a way to schedule jobs
• Health Checks are scheduled, periodic HTTP(S) requests
• Once every 30 | 60 seconds or 5 | 10 | 15 minutes
• Monitoring Alarms are evaluated and repeated on the
indicated specific interval
DOAG 2020| Oracle Cloud Native Application Development
Functions
Healthcheck
API Gateway
Configuration
Parameters
CRON expression to run
once every hour | day |
Sunday | …
When triggered evaluate
CRON expression and
decide whether to run
CRON expression passed as
header or parameter in
healthcheck configuration
Notifications
Alarming
Monitoring
Define an Alarm that will be
triggered; set repeat interval at
desired function execution interval
61. Drawing OCI Solution Designs on draw.io
DOAG 2020| Oracle Cloud Native Application Development
Download draw.io graphics from https://docs.cloud.oracle.com/en-
us/iaas/Content/General/Reference/graphicsfordiagrams.htm… , unzip and upload as library to draw.io
65. Summary
Healthcheck
StreamingEvents
Object
Storage
Notifications
API
Gate
way
NoSQL
Database
Function
Function
Compartments
DOAG 2020| Oracle Cloud Native Application Development
Email Delivery
#doag2020 Auditing
Monitoring
Logging Container
Registry
Vault
DDoS
Protection
WAF
Cloud Guard
Identity & Access
Management
Cloud Native:
Platform Services
Managed => Serverless
Quick Provision & Decommission
Distributed
Automated
Pay per use/TCO
Vendor takes runtime responsibility
Enterprise grade platform for all
Security & Availability
Scale
Ops facilities (monitor, log, audit,
events, failover)
Rapid Innovation (new services, new
features, new pricing)
66. Thank you
for your attention
I hope
this was
useful
DOAG 2020| Oracle Cloud Native Application Development
lucas.jellema@amis.nl | technology.amis.nl | @lucasjellema | lucas-jellema
Editor's Notes
Out of the box PaaS services and facilities
persistence, routing/throttling/buffer/queue/retry
IDM, key mgt, encryption/decryption
Distribution, CDN, regional failover
Monitor, log, alert/notification
Scalability (up & down)
Quick Rampup
Emphasis on interaction points: APIs, Events/ message format
Less emphasis on implementation of PaaS (platform) services
MongoDB API – implemented by Azure Cosmos DB, MongoDB, Minimongo and AWS Document DB and soon Oracle Autonomous JSON
Kafka API – Azure Event Hub, OCI Streaming, AWS Managed Streaming
JDBC/SQL – Relational Database (Oracle, SQL Server, PostgreSQL, MySQL) and even NoSQL (Oracle NoSQL, Cassandra, Hadoop)
MongoDB API – implemented by Azure Cosmos DB, MongoDB, Minimongo and AWS Document DB and soon Oracle Autonomous JSON
Kafka API – Azure Event Hub, OCI Streaming, AWS Managed Streaming
JDBC/SQL – Relational Database (Oracle, SQL Server, PostgreSQL, MySQL) and even NoSQL (Oracle NoSQL, Cassandra, Hadoop)