More Related Content
Similar to Oracle NoSQL Database release 3.0 overview (20)
Oracle NoSQL Database release 3.0 overview
- 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
Agenda
Oracle NoSQL DB Overview
Oracle NoSQL DB Release 3.0
Best Practices
- 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
Big Data Architecture
Data WarehouseData Reservoir +
Oracle Big Data
Connectors
Oracle Data
Integrator
Oracle
Advanced
Analytics
Oracle
Database
Oracle Spatial
& Graph
Oracle NoSQL
Database
Cloudera Hadoop
Oracle R Distribution
Oracle Industry
Models
Oracle
GoldenGate
Oracle Data
Integrator
Oracle Event
Processing
Oracle Event
Processing
Apache
Flume
Oracle
GoldenGate
Oracle Advanced
Analytics
Oracle Database
Oracle Spatial
& Graph
Oracle Industry
Models
Oracle Data
Integrator
Oracle NoSQL
Database
Where does NoSQL fit?
- 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
Simple Data Model
Distributed, Replicated data
Transparent load balancing
Elastic configuration
Simple administration
Enterprise-ready Integration
Commercial grade software and
support
Characteristics
Oracle NoSQL Database
Scalable, Highly Available, Key-Value Database
Application
Storage Nodes
Datacenter B
Storage Nodes
Datacenter A
Application
NoSQL DB Driver
Application
NoSQL DB Driver
Application
- 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
Features
Release
3.0
Oracle NoSQL Database
Scalable, Highly Available, Key-Value Database
Application
Storage Nodes
Datacenter B
Storage Nodes
Datacenter A
Application
NoSQL DB Driver
Application
NoSQL DB Driver
Application
Key-value, JSON & RDF data
Large Object API
BASE & ACID Transactions
Data Center Support
Online Rolling Upgrade
Online Cluster Management
Table data model
Secondary Indices
Secondary Zones (Data Centers)
Security
- 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
Scalability Architecture – Applications View
Elastic Shards
(split, add, contract)
Store
Shard
M
Shard
M
R
Shard
M
R R
Application
NoSQL Driver
R R
R
Writes to elected
node
Reads from any
node in system
- 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
Oracle NoSQL Database
Predictability
Reliability & Support
Integration
When you need:
Web-Scale Transactions, Personalization
Sensor Data Management
Real-Time Event Processing
For Applications that do:
- 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Agenda
Oracle NoSQL DB Overview
Oracle NoSQL DB Release 3.0
Best Practices
- 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
Oracle NoSQL DB Release 3.0
Enterprise Ready
Ease of Adoption
Security
Business Continuity
- 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Oracle NoSQL DB Release 3.0
Table data model support
– Simplifies data modeling & leverages existing “table” expertise
Secondary indexing
– Allow indexing on any component of the record
– Huge performance benefit due to parallel, indexed data access
Data centers
– Metro-area zones for disaster recovery and business continuity
– Secondary zones for read-only workloads
Security
– Authentication and network-level encryption
Feature Summary
- 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Oracle NoSQL DB Release 3.0
Lower barrier to adoption, shorter time to market
Simplified application modeling
– Uses familiar table concepts
– Introduces strongly typed fields
– Easy to/from JSON with full type support and AVRO serialization
– Allows secondary indexing of non-key data
Sets foundation for SQL query access from Oracle DB
Previous Key/Value and JSON schema APIs still supported
Table Data Model Benefits
- 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Oracle NoSQL DB Release 3.0
Layered on top of distributed key-value model
Compatible with most Release 2.0 JSON schemas
Defines JSON schema automatically
Supports table evolution
New access API’s with table terminology
Retains flexible client access
Administrative CLI for schema creation and evolution
Table Data Model Characteristics
- 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Oracle NoSQL DB Release 3.0
Distributed Table records or “JSON Documents” – you choose
Records grouped locally (by shard-key)
ACID & BASE transactions
Automatic mapping of Major/Minor key structure
Enables future External Table improvements
Rich queries
Predicate push down
SQL query language
Table Data Model Advantages
- 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
Oracle NoSQL DB Release 3.0
table create -name Users
add-field -name userid -type integer
add-field -name lname -type string
add-field -name fname -type string
add-field -name email -type string
primary-key -field userid
shard-key -field userid
exit
plan add-table -name Users -wait
Simple Table Example
Can be specified as a JSON string
Must be proper subset of primary-key
By default shard-key == primary-key
- 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Oracle NoSQL DB Release 3.0
Simple Table Example
userId lname fname email
Table
Shard Key
Users
ValuePrimary Key
- 17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
Oracle NoSQL DB Release 3.0
table create -name Users
add-field -name userid -type integer
add-field -name lname -type string
add-field -name fname -type string
add-field -name email -type string
primary-key -field userid
exit
plan add-table -name Users –wait
table create -name Users.Folders
add-field -name foldername -type string
add-field -name msgcount -type integer
add-field -name favorite –type boolean
-default 'F'
primary-key -field foldername
exit
plan add-table -name Users.Folders -wait
Nested Table Example
- 18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
Oracle NoSQL DB Release 3.0
Table Data Model – Nested Tables
UserId lname fname email
Table
Users
Users.
Folders
[Value]Primary Key
…
[Value]
UserId Foldername msgcount favorite
Primary KeyShard Key
- 19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
Oracle NoSQL DB Release 3.0
Broader low latency use case support
– Primary and Secondary Indexes
– Indexes automatically maintained
– Provides sorted results
Low resource cost with high performance
– Low cardinality matching, guaranteed consistent
– Shard-local indexing, low latency record access
– B-trees locally ordered, scanned in parallel, merged result set
– Improved index cache eviction
Secondary Index support
- 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Get() API specifies index
Equality or range searches
Low order multi-value indexes
– Composite index on values
– Parallel Index Scan
– Ordered results
– Indexable single element arrays
Oracle NoSQL DB Release 3.0
Secondary Indexes
Index on (lname, fname)
UserId fname lname email
Table
Users
Equality search on lname + fname
Equality search on lname and range on fname
Range or Equality search on lname by itself
Can’t search on just fname
- 21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
Oracle NoSQL DB Release 3.0
plan add-index –table Users -name Users_idx1 –field lname –field fname
1. Defines index on all shards
2. In parallel (per shard) starts table scan via primary key, populates
index
3. Updates get populated into index automatically
4. When all shards are complete, index is available for access
Simple Example
- 22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
Secondary Index Storage
Pkey A Value A
F1 F2 F3 V1 V2 V3
Pkey B Value B
F1 F2 F3 V1 V2 V3
Skey A Pkey A
V2 V3 F1 F2 F3
Skey B Pkey B
V2 V3 F1 F2 F3
Skey A Pkey A
V1 F1 F2 F3
Skey B Pkey B
V1 F1 F2 F3
Primary Records
Secondary Indexes
Shard 1
Writes
Reads
Shard 2
Shard N
Index on (V1)
Index on (V2, V3)
- 23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
Oracle NoSQL DB Release 3.0
Metro-Local Quorum
– Low latency writes, HA
2nd’ary Read-Only Zones
– Analytic workloads
– Report generation
– Asynchronous replication
New Read Consistency: No Master
Topology Aware Client Driver
Provides business continuity and distributed workload management
Zones
DC1 DC2 DC3
Metropolitan Zones
Reports
Batch Analytics
- 24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
Oracle NoSQL DB Release 3.0
Protected Access to Data
– Enforcement is configurable
Authentication
– OS independent, password based user access
– Internal cluster components self authenticate (HA password distribution)
– Oracle Wallet integrated
– Admin utility to generate self-signed certificates
Wire level data encryption
– All client-server, server-server channels SSL encrypted
Security Overview
- 25. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
Oracle NoSQL DB Release 3.0
Port Restrictions for client and intra server communications
SSL Certificates stored, protected and obfuscated at the server
Client configured login-file or truststore for SSL credential management
Authenticated client sessions have admin configurable time-out
Auth Exceptions: AuthRequired, AuthFailed, AccessDenied
Supports 2 roles: Admin & User
System Admin manages security via Admin CLI
Security Features
- 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
Agenda
Oracle NoSQL DB Overview
Oracle NoSQL DB Release 3.0
Best Practices
- 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
Choosing a Data Model
Tables
– Highest level abstraction, simple to model, familiar to developers
– System managed Secondary indices, Table evolution support
– Future: Security per table, Query Language
JSON
– Medium level abstraction, need to model keys (strings), appealing to JSON-centric applications
– Application managed Index Views, Schema evolution support
– Future: Limited security
Raw Key-Value
– Lowest level abstraction, need to model keys (strings), application serialized data, maximum
flexibility
– Application managed Index Views, record evolution and security – roll your own
- 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
Metrics and Best Practices
No overhead, performs same as JSON schemas with AVRO
serialization
Use nested tables to encapsulate record types in a hierarchy
Use Arrays and Arrays of Records to store self contained sets
Record types can’t be indexed -> flatten into simple types if indexing is
required
Tables
- 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
Metrics and Best Practices
Same rationale as an RDBMS
– Reduces time for lookups/range scans
– Increases overhead for updates
– Provides sorted results
– Add indexes when required
No optimizer -- application picks index to be used
Remember to calculate cache size requirement
New CLI option for get: -reportsize tells you the size of a key. Use output with
DBCacheSize
Indexes
- 30. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
Metrics and Best Practices
Data Centers may add write latency, depending on durability policy and
inter-data center latency
Options for tuning:
– Consider reducing durability policy ACK requirement if latency is high
– Consider adding processing threads to use additional CPU
1. Allow Client Driver to perform load balancing
2. Use Secondary Zones for read-only, batch, analytical workloads
3. Use No-Master read consistency in the application
Data Centers/Zones