Lucas Jellema presented on the evolution of the database and its role in application architecture. Key points include: - The database has evolved from decentralized silos to a centralized, consolidated layer providing data, business logic and interfaces. - The Oracle database introduced capabilities like PL/SQL, high availability, compression, and flashback queries. - Recent trends include multitenant databases, in-memory processing, and temporal/versioned data in the cloud. - Engineered systems optimize hardware and software to improve performance.

1. Lucas Jellema (CTO, AMIS)
10 Mei 2016
The evolution of the Database – role of
the database in application architecture

2. 2
Four years ago – here at Fontys

3. 3
Agenda
• Wie of wat is AMIS?
• Hoe zit het met die Database?
• Vragen & Opmerkingen

4. AMIS
• Leuke organisatie!
• Oracle & Java specialisten (pakweg 90 man & vrouw)
• Gevestigd in Nieuwegein
• Opgericht in 1991 door studenten van de Universiteit Twente
– A M I S
• Kennisdeling: SIGs, conferenties,
blog (sinds 2004)
– http://technology.amis.nl
• Opdrachten in
– Heerlen, Joure, Almelo, Nijmegen, Deventer
– Utrecht, Den Haag, Amsterdam, Rotterdam
– New Caledonia, Mongolië, Canada,
Denemarken, België, Koeweit,
Faroër eilanden, Duitsland, Verenigde Staten,
…
• Stage-opdrachten
– Mail: eva.van.der.kleij@amis.nl

5. 5

6. The Presenter:
Lucas Jellema
• Lives in The Netherlands
(close to Amsterdam)
• Started doing Oracle in 1994 with
Oracle Consulting (Oracle Designer, Forms, Database)
• Joined AMIS in 2002 – now working as CTO,
Consultant (Solution Architect, Technical Lead,
Programmer, Product Owner) and Trainer
• Oracle ACE (2005) & ACE Director (2006)
• Author of ‘Oracle SOA Suite 11g Handbook’
(Oracle Press, 2010), ‘Oracle SOA Suite 12c Handbook’ (2015)
• Presenter at Oracle OpenWorld, JavaOne and
many Oracle and Java User Group Conferences
• Frequent blogger at http://technology.amis.nl
• Active with SQL & PL/SQL, Java EE & ADF,
SOA, BPM & more Fusion Middleware

7. 7
AMIS
• Projecten rond (enterprise) web applicaties en portalen, B2B integratie,
“mobilisering”, 24/7 beschikbaarheid, User Experience, business process
management, digitale communicatie, self service, STP
• Klanten waaronder:
– Politie Nederland, Mediq, Gezondheidsdienst voor Dieren, Raad voor de Rechtspraak,
NS, ProRail, Eneco, Pensioenfondsen (APG, MN), financiële instellingen, logistieke
bedrijven, software producenten en een heel groot Social Network
• Technische kreten
– Database, Middleware
SOA, APIs, BI, IoT,
Big Data, UX, HTML 5,
(No)SQL, Real Time,
Java, grid, IdM, XML,
Cloud, BPM, VM, Docker,
Node.js, Angular, Scrum
Provisioning, Scalability
Engineered Systems

8. 8
De Nieuwegein-Eindhoven
connectie

9. THE EVOLUTION OF THE DATABASE –
ROLE OF THE DATABASE IN
APPLICATION ARCHITECTURE
9

10. Overview
• Role of the Database in Modern Architecture
• More than a container for data
• Evolution of the Oracle Database
• Recent Trends around the Database
• Q & A

11. 11
Decentralized Silos to central,
consolidated Layers

12. 12
Decentralized Silos to central,
consolidated Layers (2)

13. 13

14. 14
Consolidated, Layered
Architecture
Data
Business
Interface

15. 15
Consolidated, Layered
Architecture
Data
Business
Interface

16. 16
Cloud Power
PaaS

17. More than a container for data
• “We could also do that in the database”
• in the database? Huh?
RDBMS
≈>>

18. Brief history of Oracle
Database
• 1970: Paper “A Relational Model of Data for Large Shared Data Banks” by
Ted Codd (IBM)
• 1977: Software Development Laboratories was founded
• Late ‘70s – Project “Oracle” – executed by SDL for the CIA
• 1979: SDL became Relational Software, Inc. and introduced Oracle V2
(built in PDP-11)
– Basic SQL functionality (query, join) but no transactions
• 1982: RSI became Oracle Corporation
• 1983: Oracle 3 - rewritten in C, ported to UNIX platforms and extended
with COMMIT and ROLLBACK
• 1984: Oracle 4 – read consistency
• 1985-1990: Oracle 5, 5.1 and 6: client/server, distributed queries, row
level locking & hot back ups
• 1993: Oracle7… more than just a database

20. PL/SQL VM
(Oracle7, 1993)
PL/SQL VM

21. PL/SQL Virtual Machine
• Ever since Oracle7, the Oracle RDBMS contains a PL/SQL Virtual
Machine
• That means: write [PL/SQL] once, run anywhere (as long as the RDBMS
is running)
• This portability of PL/SQL across platforms is pretty extraordinary!
PL/SQL

22. 993 2009
10g7.0 8.0 8i 9i
internet
xml
JVM inside
ANSI/standards
interMedia
Spatial
OLAP/BI/Analytics
Data Warehouse/ETL
The Grid
11g
PL/SQL:
Triggers,
Packages
SOA
SOX
Constraints
20011997 2004
Data Mining
Semantic Query
OO
RegExp
‘13
XE
NoSQL
Evolution of the Oracle Database
12c
CBO
RAC
Flashback & Total Recall
Data Guard
R
24/7 EBR
Cloud
Multi-platform porting
Materialized Views & Query Rewrite
SQL2Hadoop
Pattern Matching
Temporal DB
Multitenant
In Memory DB
JSON
Data Masking
‘15

23. Backup and recovery
• Back up is not relevant….
• .. unless you have an established and test recovery strategy!
• Backup should be done ‘hot and on line’
• Recovery should be
– Unnecessary
– Online
– Focused
– Quick
• Oracle Backup & Recovery facilities
– Data Guard – (remote)
synchronized Stand By
Database
– Flashback Recovery
– RMAN

24. 24/7 – High Availability
• Unplanned Downtime
– Real Application Clusters (RAC)
– Data Guard Standby Database for fail over
– Hot Patching
– Back-up Air Conditioning & Power Generator
• Planned Downtime
– Online Redefinition
– Edition Based Redefinition for zero down time database application upgrades
– NoSQL “read ahead & write behind” cache layer

25. Compression
• When the system’s performance is I/O bound
– Reading/writing data from/to storage cannot keep up at the same rate as the CPUs
• and there is spare capacity in the CPUs to alleviate the I/O burden
– Through a reduction in the volume of data to be read and written
– by zipping before write and unzipping after read
• The same information gets across in fewer bytes and therefore with less
I/O
• It even reduces storage requirements

26. Flashback and Total Recall
• Oracle Database has long had two capabilities for ‘remembering’ the old
situation regarding records
– Uncommitted transactions whose changes are local
– Read consistency, even for long running queries
• In 9i, 10g and 11g this feature
has been extended to remember
‘the old situation’ for far longer
– Undo Data => Flashback Archive
• This allows for
– Historical queries
– Trend analysis over time
– Point-in-time recovery
• At database, schema, table
or even record level
• 12c adds Valid Time Modeling (aka Temporal
Validity): database keeps track of begin and end
date of the valid period of records

27. Valid time aware flashback
queries
• Select all employees
• Select all employees who were employed at a certain moment in time
• Perform all queries as if we traveled back in time
• Example: Run an end-of-year report at the end of January
SELECT *
FROM EMP AS OF TIMESTAMP TO_TIMESTAMP('01-JUN-2014 12.00.01 PM')
EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_time
( 'ASOF'
, TO_TIMESTAMP('29-MAR-2014 13:14:15')
);
SELECT *
FROM EMP

28. 28
Flash Forward:
Looking into the future…
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products

29. 29
Looking further into the
future…
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products
begin
DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME (
level => 'ASOF'
, query_time => TO_TIMESTAMP('01-10-2018', 'DD-MM-YYYY')
);
end;

30. 30
Current situation …
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products
begin
DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME (
level => 'CURRENT'
);
end;

31. 31
All data in the table
(the default setting)
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products
begin
DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME (
level => 'ALL'
);
end;

32. Role of the Database in
Modern Architecture
• Single Point of …
• What goes where …

33. Reaching out from the
database [platform]
Database

34. Java Class processing an
HTTP request: The servlet
• The JVM can handle HTTP requests via the Servlet Container
• HTTP requests are routed to a custom Java Class that writes the (usually
HTML response)
• The Servlet infrastructure handles the actual HTTP response to the
invoker
JVM
Servlet
WebLogic Server

35. PL/SQL Package processing an HTTP
request: The embedded PL/SQL gateway
• The Database (the PL/SQL VM) can handle HTTP requests via the
Embedded PL/SQL Gateway
• HTTP requests are routed to a custom PL/SQL Package that writes the
(usually HTML response)
• The EPG infrastructure handles the actual HTTP response to the invoker
Custom
PL/SQL
package
E
P
G
htp

36. Reaching out from the
database
Database
dbms_epg
(mod_plsql)
http
XMLDB
http
ftp
webdav
utl_http
http
Advanced
Queuing
AQ
jms
Queue
REST
Web
Service
File, O/S
JDBC,
SQL Net,
DB Link
APEX
GUIhttp

37. Database: single point of …
Scalability | Availability | Consistency
• Single Point of …
• What goes where …

38. 38
Relational Databases
• Based on relational model of data (E.F. Codd), a mathematical
foundation
• Uses SQL for query, DML and DDL
• Transactions are ACID (Atomicity, Consistency, Isolation, Durability)
– All or nothing
– Constraint Compliant
– Individual experience [in a multi-session environment] (aka concurrency)
– Down does not hurt

39. 39
ACID comes at a cost
• Transaction results have to be persisted in order to guarantee D
• Concurrency requires some degree of locking (and multi-versioning) in
order to have I
• Constraint compliance (unique key, foreign key) means all data hangs
together (as do all transactions) in order to have C
• ACID means:
– All data closely together (and sometimes far from
applications and users)
– All transactions centrally organized
– All data persisted as part of the transaction
• ACID means:
– Hard to scale out (horizontally)
• Oracle offers scale up with Real Application Clusters

40. 40
Databases re-evaluated
• Not all use cases require ACID (or can afford it)
– Read only (product catalog for web shops)
– Inserts only and no (inter-record) constraints
– Big Data collected and “dumped” in Data Lake (Hadoop) for subsequent processing
– High performance demands
• Not all data needs structured formats or structured querying and JOINs
– Entire documents are stored and retrieved based on a single key
• Sometimes – scalabile availability is more important than Consistency –
and ACID is sacrificed
– CAP-theorem states: Consistency [across nodes], Availability and Partition tolerance
can not all three be satisfied

41. 41
NoSQL and BASE
• NoSQL is a label for a wide variety of databases that lack some aspect of
a true relational database
– ACID-ness, SQL, relational model, constraints
• Cassandra, MongoDB, Redis, CouchDB, …
• BASE as alternative to ACID:
– basically available, soft state, eventually consistent

42. 42
Oracle Database 12.2.
Sharding for huge scale-up
• Multiple (potentially distributed) partitions
– Individually consistent
– Mutually joinable

43. Recent Trends around the
Database
• Self managing => role of DBA
• Cloud
• Appliances & Engineered Systems (the Oracle Exadata machine)
– From I/O bound back to CPU bound
– Software in Silicon
• Oracle Database Release 12c
– Multitenant, In Memory Database, Temporal Validity, Data Masking, JSON, Node.js

44. Cloud

45. 45
The Cloud Pyramid

46. 46
HaaS?

47. 47
Why is the cloud hot?
• Web Scale for the rest of us
– Availability, Scalability, Security, …
• No specific expertise or even effort for consuming services
– No need to acquire skills, build up experience and hire special resources
• No physical facilities – room, cooling, cleaning,
physical security, power
• No/Low initial investment (low entry level)
• Pay per use (no gain, no pain) (CAPEX => OPEX)
• Flexible up and down scaling (capacity on demand)
• Quick Ramp-Up, Rapid Start
• Enterprise functionality for small businesses
– Out of reach in the on premises world
• Accessibility from anywhere
• Global market place
• Lifting the burden from your shoulders
SaaS
PaaS
IaaS

48. 48
Cloud adoption in
bottom-up steps
IaaS/PaaS
Self study
PoC
Training
Load Test
Func Test
Peak, Failover
Peripheral Applications
Backup
BI
Edge Systems
Core Systems & Secure Data
[Distributed] Development
BPO

49. Database Cloud
• Automated (self-service) provisioning of database resources
• Multi-tenant
• Metering and per-usage billing
• 24/7 uptime – hot patching, fail-over
• Fine grained recovery, upgrade, authorizations
• Scalable
• ‘The Oracle Public Cloud’ IaaS
PaaS
SaaS

50. 50

51. 51
You need a grid
Nah, you don’t need a
cloud
What about just a cute
little private cloud?
Are you sure you wouldn’t
prefer an engineered
system?
Today, we
announce the
Oracle PubliC
Cloud
A previewof the
beta of the
prototypeis almost
…
Nimbus, Cirrus, Stratus, Incus, Pannus,…
anycloudunder the sun. And yes that
takes a while!
The boss is off sailing. I’m
hereto launchsome new
cloud services

52. 52

53. 53

54. Bring Oracle’s leading
Infrastructure, Technology,
Business Applications, and
Information to customers and
partners anywhere in the
World through the Oracle
Cloud
Oracle Cloud: Mission

55. 55
Four Cloud Tier approach

56. 56
=≈

57. 57
Oracle Public Cloud Stack

58. 58
Oracle Public PaaS Platform

59. 59
Hardware and Software
Engineered to work together

60. Appliances & Engineered
Systems
• Pre-configured, installed, plug’n’play
• One stop solution in case of issues
• Software/hardware mutually optimized (*
– Infini-band, Storage Cells, Flash-memory (between RAM & Disk)

61. 61
Software in Silicon

62. 62
Hardware and Software
Engineered to work together
• Leveraging tailored hardware optimizations for best performance and
security

63. Data Masking
• Gartner reports that: data masking should be mandatory for enterprises
using copies of sensitive production data for application development,
analytics or training.
• They also believe the market is
expanding into production and
unstructured data protection.

64. Data Redaction
• At runtime, you can optionally have the query results modified to
reset/scramble/randomize sensitive data
– Through ‘data redaction’ policies associated with tables and view and applied at
query time
• Because the data is masked in real-time, Data Redaction is well suited to
environments in which data is constantly changing.
• You can create the Data Redaction policies in one central location and
easily manage them from there.
SQL
engine SQL
POLICY
POLICY
RESULTS

65. Oracle 12c (released 2013, first
update Summer 2014, R2 in 2016)
C is for…
Complete
Complementary
Cloud
Consolidation
ContainerCrowd
Control
Core
Central
Cool Carefree
Classified

66. DBA is resource intensive

67. Managing dozens of databases
means…
– Installing
– Configuring
– Securing
– Monitoring
– Patching
– Upgrading
– Backing up
• many database instances on potentially a large number of machines
• Using dedicated resources for each individual database instance
– 20 processes
– Memory for SGA
– Disk space for generic objects such as most of the SYS schema

68. Multitenant Architecture

69. Oracle Database Architecture
• Requires memory, processes and database files
ERP
Database
Files
Background
Processes
Memory
Utilized

70. Oracle Database Architecture
• Separate memory and processes required for each database
ERP
Database
Files
Memory
Utilized
Background
Processes
CRM
Database
Files
Background
Processes
Memory
Utilized
DW
Database
Files
Background
Processes
Memory
Utilized

71. Pluggable Database
Architecture
ERP
Database
Files
Memory
Utilized
Background
Processes
CRM
Database
Files
Memory
Utilized
Background
Processes
DW
Database
Files
Memory
Utilized
Background
Processes
• Memory and processes required at container level only

72. New Pluggable Database
Architecture
• Memory and processes required at container level only
ERP CRM DW
Background
Processes
Memory
Utilized Container Database

73. Multitenant Architecture

74. 78
Big News…

75. 79
Oracle Database
In Memory Option

76. 80
Hybrid, dual-format data storage
transparent to application

77. 81
Free Software!!!
Oracle Technology Network
http://otn.oracle.com
• Gratis download van vrijwel alle Oracle producten
• Gebruiken voor studie, onderzoek, prototype development, …
• Geen beperking in tijd of functionaliteit
• Betalen als je een
systeem in productie
neemt
• Handig: Pre-Built
Virtual Machines
– PHP
– BigData
– SOA & BPM
– Java EE
– BI
– Solaris
– Oracle Database EE

78. 82
Oracle XE Database – free!
http://www.oracle.com/technetwork/database/database-technologies/express-edition/overview/index.html

79. 83
Summary
• Database is core of enterprise IT
– Persistent when the plug is pulled
• Consolidation is important theme
– More efficient usage of hardware resources and
of human resources
– Agile scaling – quick, simple, cheap
– Cloud as ultimate target
– Also: “logical consolidation” – centralize data
• Database is bottleneck – a single point of failure
• Logical consistency demands require transactions to use locks on data –
scalability at database level is reduced as a result
– CAP theorem – consistency, availability and partition tolerance can not all be
achieved
• Evolution of databases continues rapidly – the relevance of the Database
Platform is quite strong today (and what about tomorrow?)
• IT is a smart career choice – and AMIS is a fun company

80. • Blog: technology.amis.nl
– On Oracle, SQL, Java, SOA, BPM & more
• Email: lucas.jellema@amis.nl
• : lucasjellema
• : www.amis.nl, info@amis.nl
+31 306016000
Edisonbaan 15,
Nieuwegein