Oracle no sql release 3 4 overview

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL Database
Release 3.4 – What’s new
Anand Chandak
Principal Product Manager

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Oracle Confidential – Internal/Restricted/Highly Restricted 3

NoSQL Background
Release 3.4 Highlights
Querying Oracle NoSQL Database
4
1
2
3
Agenda

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted 5
101100101001001
001101010101011
100101010100100
101
ReliableFlexibleFastSimple
advanced Key-Value database designed as cost effective, high performance solution for simple operations on
collections of data with built in high availability and elastic scale-out.
less is more
What is Oracle NoSQL Database

Enterprise NoSQL Characteristics
• Simple Data Model
• Distributed, Replicated data
• Transparent load balancing
• Elastic configuration
• Simple administration
• Enterprise-ready Integration
• Commercial grade software and
support
Application
Storage Nodes
Datacenter B
Storage Nodes
Datacenter A
Application
NoSQL DB Driver
Application
NoSQL DB Driver
Application

Features
Application
Storage Nodes
Datacenter B
Storage Nodes
Datacenter A
Application
NoSQL DB Driver
Application
NoSQL DB Driver
Application
• Key-value, JSON & RDF data
• Large Object API
• BASE & ACID Transactions
• Primary and Secondary Data Centers
• Online Rolling Upgrade
• Online Cluster Management
• Table data model
• Secondary Indices
• Security
• Business Continuity and Disaster Recovery
• Off Heap Cache
• Bulk Get
• Complex Types for Hive Storage Handler

1 NoSQL Background
Release 3.4 Highlights
Querying Oracle NoSQL Database
8
2
3
Agenda

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 9
Ease of Adoption
Improved Performance
Business Continuity
Oracle NoSQL DB Release 3.4

Release 3.4 Feature Summary
• Data Centers –Failover and Switchover Operations
– Continue business operations in event of zone failure that results in loss of quorum
– Perform Maintenance and planned shutdown
• Complex data type support for Apache Hive and Oracle Big Data SQL
– Map Hive Types with Oracle NoSQL Database Types.
• Bulk Get
– Ability to search using multiple primary keys in a single operation
– Get results from multiple rows across the shards in parallel
• Off Heap Cache
– Use memory outside of the object heap to support large datasets
– Improve performance – avoid Java GC problems
– Leverage the growing memory sizes per machine to be cost effective.
10

Data Center Failover and Switchover
• Failover is unplanned change to one or more primary zones
– In-flight, non-replicated updates may be lost
– Adds the ability to redefine primary VS secondary zones to maintain write quorum
– Failed zones part of topology but made offline. Can be brought back once the fault is recovered
• Switchover is a planned configuration change to take one or more zones on- or offline,
or change their type
– Zero data loss
– Switchover waits until in-flight updates have been replicated
• New Admin CLI commands to handle these operations
• Requires intervention by cluster administrator
11

12
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
Zone 1 Primary RF =2 Zone 2 Primary RF =1
Shard 1
Shard 2
Shard 3
 Zone1 hosts two copies; Zone2
hosts 1 copy
 Application accesses all nodes
 Zone 2 goes down. Still read and
write can be done.
 Admin CLI can only be used when
a quorum of admin nodes is
available
Data Center A Data Center B
Failover and Switchover
2 Zones. RF=2

13
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
Shard 1
Shard 2
Shard 3
 Zone 1 goes down. Quorum Lost.
Read Only
 Admin available on zone2 only
 Diagnostic to zone1 unavailable
 Cannot perform Failover- Admin
quorum needed.
X
Zone1 Down

14
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
Shard 1
Shard 2
Shard 3
 repair-admin-quorum -zn zn2
 Modify admin quorum to 1
X
Failover and SwitchOver
Failover Repair Admin Quorum

15
Store
M R
R
R R
Application
NoSQL Driver
M
M
M
M
Zone 1 secondary and status
offline
Zone 2 Primary RF =1
Shard 1
Shard 2
Shard 3
 Plan failover –zn zn1 –type offline
secondary -wait
 Changes the zonetype of failed
zone to secondary
 Zone is offline.
Failover - Zone1 Secondary and Offline

16
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
Zone 1 secondary Zone 2 Primary RF =1
Shard 1
Shard 2
Shard 3
 New flag disable-services added to
the start and stop command for
kvstore
 Avoids Hard Rollback
 Storage Nodes starts without
starting their hosted service.
Switchover –Restore Zone1 as primary

17
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
Zone 1 primary RF =2 Status
Reachable
Zone 2 Primary RF =1
Shard 1
Shard 2
Shard 3
 plan repair-topology –wait
 Repair the topology to update the
topology of newly restarted Storage
Nodes with changes made by the
failover.
 topology change-zone-type -name
original-topo -zn zn1 -type primary
 plan deploy-topology -name
original-topo -wait
Switchover – repair and change topology

Integration with Apache Hive and Oracle Big Data SQL
• Oracle Big Data SQL
– Common SQL access layer to data stored in Hadoop HDFS,
Hive and Oracle NoSQL Database
– Defines new external table access driver types.
ORACLE_HIVE for accessing data stored in or referenced by
Hive Tables
– Query Oracle NoSQL Data from Hive or Oracle Database
– Runs Map Reduce Jobs against data stored in Oracle NoSQL
Database
• Support for primitive and complex data types
• Common data definition between Oracle NoSQL
DB, Hive and Oracle Database
18
BIG DATA SQL

Hive External Table Type Mapping
19
Oracle NoSQL Hive Type
FieldDef.Type.ENUM
STRING
TIMESTAMP
DATE
FieldDef.Type.MAP MAP<STRING, data_type>
FieldDef.Type.RECORD
STRUCT<col_name : data_type, ...>
UNIONTYPE<data_type, data_type, ...>
19
Hive> SELECT SSN FROM rmvTable WHERE ADDRESS.CITY SELECT COUNT(SSN) … GROUP BY GENDER
hive> CREATE EXTERNAL TABLE IF NOT EXISTS
rmvTable (ZIPCODE STRING, LASTNAME STRING, FIRSTNAME STRING,
SSN BIGINT,
GENDER STRING,
LICENSE BINARY,
PHONEINFO MAP<STRING, STRING>,
ADDRESS STRUCT<NUMBER:INT,
STREET:STRING,
UNIT:INT,
CITY:STRING,
STATE:STRING,
ZIP:INT>,
VEHICLEINFO ARRAY<STRUCT<TYPE:STRING, MAKE:STRING, MODEL:STRING, CLASS:STRING,
COLOR:STRING, VALUE:FLOAT, TAX:DOUBLE, PAID:BOOLEAN>>)
COMMENT 'Hive table rmvTable <---> KVStore table rmvTable'
STORED BY 'oracle.kv.hadoop.hive.table.TableStorageHandler'
TBLPROPERTIES ("oracle.kv.kvstore" = "example-store",
"oracle.kv.hosts" = "kv-host-1:5000",
"oracle.kv.tableName" = "rmvTable",
"oracle.kv.hadoop.hosts" = "dn-host-1,dn-host-2,dn-host-3");

• Performant API to retrieve records matching
multiple primary keys in a single API call.
– Takes a list of keys as input
– Similar to IN clause in SQL
– Outputs an iterator with the matching results
• Parallel scan across the shards
• Supports Key/Value and Table Interfaces
• All the primary keys should belongs the same
table
• The primary key must contain all of the fields
defined for the table‘s shard key
20
Bulk Get()
Getting 30M rows goes from 420ms to 149ms.
2-3X+ performance improvement.
get bulk-9 bulk-18 bulk-72
Time(ms) 438,487 219,842 171,081 143,937
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
500,000
KV3*3, client:1, thread:90, read
OPs:30,000,000, fields:10*10
Time(ms)

Read rows matching the condition that “country
is US or IND or CN or UK and userId in [0,100]”
from table User and its child tables User.
Address and Users. Email .
Tables definition:
User (country string,
userid integer,
name string,
…
primary key(shard(country), id))
User.Address (addrId integer,
…,
primary key(addrId))
User.Email (emailAddress string,
…
primary key(emailAddress))
Bulk Get Example
final String[] countries = {"US", "IND", "CN”, “UK”};
final List<PrimaryKey> keys = new ArrayList<PrimaryKey>(countries .length);
for (String country : countries) {
final PrimaryKey pk = userTable.createPrimaryKey();
pk.put("Country", "US");
keys.add(pk);
}
/* Initialize multiRowOption: userId in [0, 100]. child tables email, address */
final FieldRange range = userTable.createFieldRange("userid");
range.setStart(0, true).setEnd(100, true);
final MultiRowOptions mro =
new MultiRowOptions(range, null /* ancestor */,
Arrays.asList(emailTable, addressTable));
/* Initialize TableIteratorOptions */
final int batchResultsSize = 200;
final int parallalism = 9;
final TableIteratorOptions tio =
new TableIteratorOptions(null /* Direction */,
null /* Consistency */,
0 /* RequestTimeOut */,
null /*TimeUnit*/,
parallalism,
batchResultsSize);
TableIterator<Row> itr = null;
try {
itr = tableAPI.tableIterator(keys.iterator(), mro, tio);
while (itr.hasNext()) {
final Row user = itr.next();
/* ... */
}
} catch (StoreIteratorException sie) {
/* Handle exception.. */
} finally {
itr.close();
}

• Useful for Replication Nodes with 64GB+ of available memory
• Allocate memory outside of limits of Java Heap
– Avoid long GC pauses
– Mitigate Java “compressed OOPs” overhead
• Native platform memory allocator
– Oracle and IBM JDK
• Contains records evicted from main cache (LRU)
• Configurable
– Based on available memory
– Optional Performance tuning
• change-policy -params systemPercent=10
– Setting 100 will turn OFF off-heap cache
• change-policy -params rnHeapMaxMB=8192
22
Performance: Off-heap Cache
File System Cache
Newly Written log files, Sequentially read log files,
Log file fragments.
NoSQL DB Objects
Off Heap Cache
Java Heap (min (32G, 85% of memory))
System Use (10%)
(100 – rnHeapPercent – systemPercent)
Managed by
GC
Not
Managed by
GC

Thank You
23

Join NoSQL Database Community
Twitter
https://twitter.com/#!/OracleNoSQL
LinkedIn
http://www.linkedin.com/groups?gid=4147754
Oracle’s NoSQL DB blog
https://blogs.oracle.com/nosql
Oracle Technology Network
http://bit.ly/1f0d8wU
Developer Webcast Series
http://bit.ly/1doV2jl
Oracle.com/BigData
24

Oracle no sql release 3 4 overview

Oracle no sql release 3 4 overview

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Oracle no sql release 3 4 overview

Similar to Oracle no sql release 3 4 overview (20)

Recently uploaded

Recently uploaded (20)

Oracle no sql release 3 4 overview