This document discusses Oracle NoSQL Database. It provides an overview of key-value, table, and JSON data models supported. It describes the simple architecture with data automatically sharded across nodes. Examples of using the key-value model via API calls to put and get data are also presented.

1. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted
11th Oracle
Developer
MeetupOracle NoSQL Database
김태완(Taewan Kim)
Oracle Korea Solution Engineering
2019.05.18

2. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing, and pricing of any
features or functionality described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.
Confidential – Oracle Internal/Restricted/Highly Restricted 2

3. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
About: 김 태완(@taewanme)
• taewanme@gmail.com
• 1999 ~ 2006 Java (SI, Developer)
• 2007 ~ 2013 Middleware Engineer
– WAS, Memory Cache, CEP, DB Replication
• 2013 ~ 2015 CouchDB, Erlang (SE)
• 2015 ~ Big Data, ML, Deep Learning(SC)
• ML, Cloud, Data Engineerning, DevOps
– http://taewan.kim/
http://taewan.kim

4. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 좋아요 in 2012~2015
Confidential – Oracle Internal/Restricted/Highly Restricted 4

5. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 5

6. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 6

7. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 이외의 데이터 저장소
Confidential – Oracle Internal/Restricted/Highly Restricted 7
Oracle Coherence
Oracle NoSQL Database
Oracle BDA

8. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
•오라클에도 NoSQL이 있나요?
8
OMZ

9. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL?
No SQL
Not only SQL
Non-relational database SQL
9
NOSQL DATABASES [>225]
- http://nosql-database.org
데이터의 특성에 맞추어 적절한 아키텍쳐에 저장하는 것이 TCO 절감의 출발점

10. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Database의 - CAP 이론
10
Consistency
Availability
Partition Tolerance

11. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 유형
11

12. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 유형
12
Key Value Store

13. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 유형
13
Key Value Store Document DB

14. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 유형
14
Key Value Store Document DB Columnar DB

15. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 유형
15
Key Value Store Document DB Columnar DB Graph DB

16. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 유형
16
Key Value Store Document DB Columnar DB Graph DB
데이터의 특성에 맞는 적절한 데이터 관리 기법을 사용하자.

17. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL 유형
17
Key Value Store Document DB Columnar DB Graph DB
데이터의 특성에 맞는 적절한 데이터 관리 기법을 사용하자.
범용적인 NoSQL

18. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL에 대한 이해
Confidential – Oracle Internal/Restricted/Highly Restricted 18

19. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL에 대한 이해 – Oracle RAC
19
Consistency
Availability
Partition Tolerance
O
O
X
C
A P
Oracle
RAC

20. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL에 대한 이해 – MongoDB
20
Consistency
Availability
Partition Tolerance
O
X
O
C
A P
CP 계열 NoSQL

21. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL에 대한 이해 – CouchDB
21
Consistency
Availability
Partition Tolerance
O
O
C
A PAP 계열 NoSQL

22. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
NoSQL에 대한 이해 – Coherence
Confidential – Oracle Internal/Restricted/Highly Restricted 22
• Abstraction of DHT(Distributed Hash Table)
• Abstraction of Consistent Hashing
• Supporting of Virtual Node (Partitioning)
• No Name Node
• Minimal Engineering

23. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
•Oracle NoSQL 주요 특징
Confidential – Oracle Internal/Restricted/Highly Restricted 23

24. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL
Confidential – Oracle Internal/Restricted/Highly Restricted 24
10110010100100100
11010101010111001
01010100100101
ReliableFlexibleFastSimple
less is more

25. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL
25
유연한 데이터 관리 체계
분산 기반 가용성
Fast Data
TCO 효율성 극대화
ERP
EAM
Inventory
Control
Accting &
Payroll
Process
Mgmt
Business
Analytics
CRM
…
Driver
Application
Real Time Event
Processing
Distributed, Web-
scale Applications
Online Gaming
…
Mobile Data
Management
Time Series &
Sensor Data Mgmt

26. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 26
Oracle NoSQL의 시작

27. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
• 특징
– 선형적인 확장성
– BASE Operations
– 테이블 / JSON / Binary
– 온라인 관리 체계
– C, C#, Java, Python, JS, Ruby
• 차별성
– ACID 트랜잭션
– 온라인 롤링 업그레이드
– 대용량 파일에 대한 스트리밍 API
– 오라클 솔루션과 통합성
– Engineered Systems & Commodity HW
Oracle NoSQL 특성
27
Application
Storage Nodes
Datacenter B
Storage Nodes
Datacenter A
Application
NoSQL DB Driver
Application
NoSQL DB Driver
Application
– 데이터 센터 지원
– 이차 인텍스
– 보안
– 유연한 스키마

28. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 28
Oracle NoSQL 아키텍처

29. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 29
Oracle NoSQL 아키텍처

30. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL 아키텍처
30
M1
M2
M3
R2
R2R1
R1 R3
R3
Shard1
Shard2
Shard3
Agents
A
A
A
Machine1
Machine2
Machine3
Application
NoSQL Driver
D
DD
D
DD

31. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL 아키텍처
• 탄력적 Auto Sharding
Store
• 유연한 데이터
영속성을 위해서
선출된 노드에
쓰기
• 읽기; 모든
노드에서 수행
Shard
M
R
R
Shard
M
Shard
R
R
R R
Application
NoSQL Driver
M
Shard
R
R
M
• Shard 추가시 데이터
조정(재분배)
31
Expand

32. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 32
Oracle NoSQL 아키텍처

33. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL Database – distributed, shared nothing
33
스케일 아웃
Shard1
NoSQL DB client driver
Shard2
Hash(primary key) -> shard_Id

34. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 34
Oracle NoSQL Database – physical organization
NoSQL DB client driver
Shard1 Shard2
D
M R R R R M
마스터
선출
+
HA를
위한 복제
Replication nodes
DD
D
DD
• 스케일 아웃

35. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 35
Oracle NoSQL Database – cluster expansion
NoSQL DB client driver
Shard1 Shard3 Shard2
M R R R M R R R M
재분배
• 스케일 아웃

36. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
•Oracle NoSQL: Simple
Confidential – Oracle Internal/Restricted/Highly Restricted 36

37. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
• 라이브러리 다운로드
• kvstore 시작
java -jar lib/kvstore.jar kvlite
• 저장소 상태 확인
java -jar lib/kvstore.jar ping -host kvhost -port 5000
• 관리 CLI
java -jar lib/kvstore.jar runadmin -host kvhost -port 5000
KVLite

38. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL Data Model
Confidential – Oracle Internal/Restricted/Highly Restricted 38
Key-Value
Data Model
Table
(Nested Tabe)
Data Model
JSON
Data Model

39. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Key-Value Data Model
Confidential – Oracle Internal/Restricted/Highly Restricted 39

40. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Key-Value Data Model – Data Type
Confidential – Oracle Internal/Restricted/Highly Restricted 40
• 단순 데이터 타입
– Boolean
– Binary
– Double
– Float
– Integer
– Long
– String
– Enum
• 복합 데이터 타입
– Array
– Map
– Recode

41. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 41
Key-Value Data Model: 구성
advanced key-value database
Primary key & Shard key
Automatic Data sharding
Secondary indexing
1. Key-value: Application specific values
2. Table API, Tables
3. JSON API, JSON data type

42. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 42
Key-Value Data Model: 예제 코드
// 키 생성
ArrayList<String> majorC = new ArrayList<String>();
majorC.add(“Note”)
Key myKey = Key.createKey(majorC);
// 데이터 변환
String myData = “value 부분의 데이터”
Value myValue = Value.createValue(myData.getBytes());
// 키/값 쌍 저장
kvstore.put(myKey, myValue);
Key Value

43. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 43
Oracle NoSQL Database – Table Data Model
ID
1234 JSON
5678 JSON
1000 JSON
TableName: PeopleData
{
Shipping: Truck
Depart: 1:00 AM
EstArrive: 6:00 PM
Weight: 1000
ColdOrHot: Cold
Shipper: Company 1
Signed: Yes
}
SQL 지원{
Shipping: Train
Depart: 5:00 AM
EstArrive: 6:00 PM
Weight: 2000
Shipper: Company 2
Signed: No
}
{
Item: Socks
Color: Blue
Qty: 10000
Size: S,M,L
Shipper: Company 2
Signed: No
}

44. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 44
Oracle NoSQL Database – Table Data Model
ID Shipper Method Vessel ID Weight GPS Location
1234 Com1 Ship 66666 Yes
5678 Com2 Car 1 200 Yes
1000 Com3 Train # 543 2,000 No
Integer
Float
String
Arrays
…
X Coordinate Y Coordinate
85.432 101.663
111.43 -34.3
66.788 -45.666

45. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 45
Parent Table & Child Table

46. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 46
Parent Table & Child Table

47. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 47
Table model: 테이블과 인덱스 생성
Fleet
vin: string
make: string
model: string
year: string
fuelType: enum
vehicleType: enum
Fleet.Mileage
date : string
driverID : string
startOdometer : integer
avgSpeed : float
totalDistance : integer
totalFuel : float
mpg : float
counter : integer
feeds {
currentTime : long
longtitude : float
latitude : float
odometer : integer
fuelUsed : float
speed : integer
}

48. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 48
SQL Query using Command Line
Connect Store
• java -jar $KVHOME/lib/sql.jar -helper-hosts kvhost:5000 -store mystore
NoSQL SQL Query Demo
• SELECT * FROM fleet;
• SELECT make FROM fleet;
• SELECT * FROM fleet WHERE make = "HONDA";
• SELECT vin, make, model, year FROM fleet WHERE make = "HONDA";
• SELECT * FROM fleet.mileage;
• SELECT * FROM fleet.mileage WHERE driverID = "E9824103";
• SELECT vin, date, feeds FROM fleet.mileage WHERE vin = "7IT86TDVSO8T146WP0R1";
• SELECT vin, date, feeds FROM fleet.mileage WHERE (vin = "7IT86TDVSO8T146WP0R1"
and date="0221");
• SELECT vin, driverID, date, mpg, totalDistance FROM fleet.mileage WHERE driverID = "
E9824103" ORDER BY date DESC;
KV Store Command Line

49. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Data Center 지원
• Availability Zones
• 유연한 구서을 지원
• Primary Zones
– 데이터 영속성 지워
– Low latency writes, HA
• Secondary Read-Only Zones
– 비동기 복제
– Analytic workloads
– Report generation
• 클라이언트 드라이버는 토폴로지 인식
• 부하 분산과 고 가용성에 용이
49
DC1 DC2 DC3
PrimaryZones
Reports
Batch Analytics

50. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
CAP 설정
• NoSQL 유연성 강화
– 설정 유연성 극대화
• 오퍼레이션 별로 Durability 설정
• 오퍼레이션별 데이터 일관성 설정
50
C
A P

51. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Large Object Support
• 병렬 스트리밍 인터페이스
– 대용량 오브젝트에 대한 저장/조회
효율성 강화
– 클라이언트 측 스트리밍 인터페이스
의 메모리 소비 최소함
– 서버측은 오브젝트 분할과 분산을 담
당, 읽기/쓰기 지연시간 최소화가 목
표
51
Large
Object
NoSQL DB Driver
Application
Shard 2
Shard N
Shard 1

52. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
보안
• 데이터 접근 보호
– 설정 기반
– 인증
• User/Password
• 클라이언트 time-out 설정
• Oracle Wallet integration
– 통신 채널 암호화
• 모든 채널은 SSH 암호화
– 권한제어
• Read/Write 권한 제어
• Administration
52
Store
Shard
M
Shard
R
R
Shard
R
R R
NoSQL Driver
M R
M
Username
Password
SSL
SSLSSL
Application
Read/Write
or Admin

53. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
온라인 롤링 업그레이드
• Linear time impact for upgrade
– 안전한 업그레이드 순서를 기술하는
관리자 명령 제공
– 업그레이드 자동화를 위한 스크립트
지원
– 업그레이드 프로세스 중 읽기/쓰기
가용성 보장
200 노드 버전 업그레이드 15분 소요
53
0
2.5
5
7.5
10
12.5
15
17.5
72 (24x3) 144 (48x3) 216 (72x3)
Time to Upgrade (min)
Total Nodes (Shards x Rep. Factor)
Online Rolling Upgrade

54. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
개발자 & 관리자 툴
• Oracle NoSQL 툴
– SNMP / JMX metrics
– Oracle Enterprise Manager
– Cloud ready HTML5 browser admin
– Command line interface
• Scripting
• Query prototyping
• Data load
– Easy to use developer API
• Java, C, REST, Javascript, Python, C#
• R, JRuby, Jython community drivers
54

55. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
YCSB on SSD-backed commodity servers
• 성능 테스트 환경
– Twitter : ~500M tweets/day
• 350K /min
• 3 ~ 10 set commodity server에 저장
– 1.25M ops/sec
– 2 billion 레코드
– 2 TB of data
– 95% 읽기, 5% 저장
– Low latency, High Scalability
55
0
1
2
3
4
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
1,400,000
6 (2x3) 12 (4x3) 24 (8x3) 30 (10x3)
Average Latency (ms)
Throughput (ops/sec)
Cluster Size ( 3->15 servers )
Mixed Throughput
Throughput (ops/sec) Write Latency (ms)
Read Latency (ms)

56. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Oracle Database와 통합
56
SQL – External Table
CREATE TABLE nosql_data (
email VARCHAR2(30),
name VARCHAR2(30),
gender CHAR(1),
address VARCHAR2(40),
phone VARCHAR2(20))
ORGANIZATION EXTERNAL
(type oracle_loader
default directory ext_tab
access parameters (
records delimited by newline 10
preprocessor nosql_bin_dir:'nosql_stream'
fields terminated by '|' missing field values
are null)
LOCATION ('nosql.dat')) 13 REJECT LIMIT
UNLIMITED 14 PARALLEL;

57. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 57
+

58. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Summary: Oracle NoSQL Database
58
Driver
Application
External Table
M/R Source
c
JPA

59. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 59
Oracle Big Data SQL
Summary: Oracle NoSQL Database
One fast SQL query, on all your data, in parallel

60. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 60