Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL Database
Release 3.4 – What’s new
Anand...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to o...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
NoSQL Background
Release 3.4 Highlights
Querying Ora...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Res...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL Database
Enterprise NoSQL Characteristi...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle NoSQL Database
Features
Application
Storage N...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
1 NoSQL Background
Release 3.4 Highlights
Querying O...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 9
Ease of Adoption
Improved Performance
Business Con...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Release 3.4 Feature Summary
• Data Centers –Failover...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Data Center Failover and Switchover
• Failover is un...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
12
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
13
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
14
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
15
Store
M R
R
R R
Application
NoSQL Driver
M
M
M
M
...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
16
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
17
Store
M R
R
R R
Application
NoSQL Driver
M
R
R
M
...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Integration with Apache Hive and Oracle Big Data SQL...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Hive External Table Type Mapping
19
Oracle NoSQL Hiv...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Performant API to retrieve records matching
multip...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Read rows matching the condition that “country
is US...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Useful for Replication Nodes with 64GB+ of availab...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Thank You
23
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Join NoSQL Database Community
Twitter
https://twitte...
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle no sql release 3 4 overview
Oracle no sql release 3 4 overview
Upcoming SlideShare
Loading in …5
×

Oracle no sql release 3 4 overview

643 views

Published on

Overview of new features in the 3.4 Oracle NoSQL Databse release

Published in: Software
  • Be the first to comment

Oracle no sql release 3 4 overview

  1. 1. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle NoSQL Database Release 3.4 – What’s new Anand Chandak Principal Product Manager
  2. 2. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Oracle Confidential – Internal/Restricted/Highly Restricted 3
  3. 3. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | NoSQL Background Release 3.4 Highlights Querying Oracle NoSQL Database 4 1 2 3 Agenda
  4. 4. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted 5 101100101001001 001101010101011 100101010100100 101 ReliableFlexibleFastSimple advanced Key-Value database designed as cost effective, high performance solution for simple operations on collections of data with built in high availability and elastic scale-out. less is more What is Oracle NoSQL Database
  5. 5. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle NoSQL Database Enterprise NoSQL Characteristics • Simple Data Model • Distributed, Replicated data • Transparent load balancing • Elastic configuration • Simple administration • Enterprise-ready Integration • Commercial grade software and support Application Storage Nodes Datacenter B Storage Nodes Datacenter A Application NoSQL DB Driver Application NoSQL DB Driver Application
  6. 6. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle NoSQL Database Features Application Storage Nodes Datacenter B Storage Nodes Datacenter A Application NoSQL DB Driver Application NoSQL DB Driver Application • Key-value, JSON & RDF data • Large Object API • BASE & ACID Transactions • Primary and Secondary Data Centers • Online Rolling Upgrade • Online Cluster Management • Table data model • Secondary Indices • Security • Business Continuity and Disaster Recovery • Off Heap Cache • Bulk Get • Complex Types for Hive Storage Handler
  7. 7. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 1 NoSQL Background Release 3.4 Highlights Querying Oracle NoSQL Database 8 2 3 Agenda
  8. 8. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 9 Ease of Adoption Improved Performance Business Continuity Oracle NoSQL DB Release 3.4
  9. 9. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Release 3.4 Feature Summary • Data Centers –Failover and Switchover Operations – Continue business operations in event of zone failure that results in loss of quorum – Perform Maintenance and planned shutdown • Complex data type support for Apache Hive and Oracle Big Data SQL – Map Hive Types with Oracle NoSQL Database Types. • Bulk Get – Ability to search using multiple primary keys in a single operation – Get results from multiple rows across the shards in parallel • Off Heap Cache – Use memory outside of the object heap to support large datasets – Improve performance – avoid Java GC problems – Leverage the growing memory sizes per machine to be cost effective. 10
  10. 10. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Data Center Failover and Switchover • Failover is unplanned change to one or more primary zones – In-flight, non-replicated updates may be lost – Adds the ability to redefine primary VS secondary zones to maintain write quorum – Failed zones part of topology but made offline. Can be brought back once the fault is recovered • Switchover is a planned configuration change to take one or more zones on- or offline, or change their type – Zero data loss – Switchover waits until in-flight updates have been replicated • New Admin CLI commands to handle these operations • Requires intervention by cluster administrator 11
  11. 11. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 12 Store M R R R R Application NoSQL Driver M R R M Zone 1 Primary RF =2 Zone 2 Primary RF =1 Shard 1 Shard 2 Shard 3  Zone1 hosts two copies; Zone2 hosts 1 copy  Application accesses all nodes  Zone 2 goes down. Still read and write can be done.  Admin CLI can only be used when a quorum of admin nodes is available Data Center A Data Center B Failover and Switchover 2 Zones. RF=2
  12. 12. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 13 Store M R R R R Application NoSQL Driver M R R M Zone 1 Primary RF =2 Zone 2 Primary RF =1 Shard 1 Shard 2 Shard 3  Zone 1 goes down. Quorum Lost. Read Only  Admin available on zone2 only  Diagnostic to zone1 unavailable  Cannot perform Failover- Admin quorum needed. Data Center A Data Center B X Failover and Switchover Zone1 Down
  13. 13. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 14 Store M R R R R Application NoSQL Driver M R R M Zone 1 Primary RF =2 Zone 2 Primary RF =1 Shard 1 Shard 2 Shard 3  repair-admin-quorum -zn zn2  Modify admin quorum to 1 Data Center A Data Center B X Failover and SwitchOver Failover Repair Admin Quorum
  14. 14. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 15 Store M R R R R Application NoSQL Driver M M M M Zone 1 secondary and status offline Zone 2 Primary RF =1 Shard 1 Shard 2 Shard 3  Plan failover –zn zn1 –type offline secondary -wait  Changes the zonetype of failed zone to secondary  Zone is offline. Data Center A Data Center B Failover and SwitchOver Failover - Zone1 Secondary and Offline
  15. 15. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 16 Store M R R R R Application NoSQL Driver M R R M Zone 1 secondary Zone 2 Primary RF =1 Shard 1 Shard 2 Shard 3  New flag disable-services added to the start and stop command for kvstore  Avoids Hard Rollback  Storage Nodes starts without starting their hosted service. Data Center A Data Center B Failover and SwitchOver Switchover –Restore Zone1 as primary
  16. 16. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 17 Store M R R R R Application NoSQL Driver M R R M Zone 1 primary RF =2 Status Reachable Zone 2 Primary RF =1 Shard 1 Shard 2 Shard 3  plan repair-topology –wait  Repair the topology to update the topology of newly restarted Storage Nodes with changes made by the failover.  topology change-zone-type -name original-topo -zn zn1 -type primary  plan deploy-topology -name original-topo -wait Data Center A Data Center B Failover and Switchover Switchover – repair and change topology
  17. 17. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Integration with Apache Hive and Oracle Big Data SQL • Oracle Big Data SQL – Common SQL access layer to data stored in Hadoop HDFS, Hive and Oracle NoSQL Database – Defines new external table access driver types. ORACLE_HIVE for accessing data stored in or referenced by Hive Tables – Query Oracle NoSQL Data from Hive or Oracle Database – Runs Map Reduce Jobs against data stored in Oracle NoSQL Database • Support for primitive and complex data types • Common data definition between Oracle NoSQL DB, Hive and Oracle Database 18 BIG DATA SQL
  18. 18. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Hive External Table Type Mapping 19 Oracle NoSQL Hive Type FieldDef.Type.ENUM STRING TIMESTAMP DATE FieldDef.Type.MAP MAP<STRING, data_type> FieldDef.Type.RECORD STRUCT<col_name : data_type, ...> UNIONTYPE<data_type, data_type, ...> 19 Hive> SELECT SSN FROM rmvTable WHERE ADDRESS.CITY SELECT COUNT(SSN) … GROUP BY GENDER hive> CREATE EXTERNAL TABLE IF NOT EXISTS rmvTable (ZIPCODE STRING, LASTNAME STRING, FIRSTNAME STRING, SSN BIGINT, GENDER STRING, LICENSE BINARY, PHONEINFO MAP<STRING, STRING>, ADDRESS STRUCT<NUMBER:INT, STREET:STRING, UNIT:INT, CITY:STRING, STATE:STRING, ZIP:INT>, VEHICLEINFO ARRAY<STRUCT<TYPE:STRING, MAKE:STRING, MODEL:STRING, CLASS:STRING, COLOR:STRING, VALUE:FLOAT, TAX:DOUBLE, PAID:BOOLEAN>>) COMMENT 'Hive table rmvTable <---> KVStore table rmvTable' STORED BY 'oracle.kv.hadoop.hive.table.TableStorageHandler' TBLPROPERTIES ("oracle.kv.kvstore" = "example-store", "oracle.kv.hosts" = "kv-host-1:5000", "oracle.kv.tableName" = "rmvTable", "oracle.kv.hadoop.hosts" = "dn-host-1,dn-host-2,dn-host-3");
  19. 19. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | • Performant API to retrieve records matching multiple primary keys in a single API call. – Takes a list of keys as input – Similar to IN clause in SQL – Outputs an iterator with the matching results • Parallel scan across the shards • Supports Key/Value and Table Interfaces • All the primary keys should belongs the same table • The primary key must contain all of the fields defined for the table‘s shard key 20 Bulk Get() Getting 30M rows goes from 420ms to 149ms. 2-3X+ performance improvement. get bulk-9 bulk-18 bulk-72 Time(ms) 438,487 219,842 171,081 143,937 0 50,000 100,000 150,000 200,000 250,000 300,000 350,000 400,000 450,000 500,000 KV3*3, client:1, thread:90, read OPs:30,000,000, fields:10*10 Time(ms)
  20. 20. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Read rows matching the condition that “country is US or IND or CN or UK and userId in [0,100]” from table User and its child tables User. Address and Users. Email . Tables definition: User (country string, userid integer, name string, … primary key(shard(country), id)) User.Address (addrId integer, …, primary key(addrId)) User.Email (emailAddress string, … primary key(emailAddress)) Bulk Get Example final String[] countries = {"US", "IND", "CN”, “UK”}; final List<PrimaryKey> keys = new ArrayList<PrimaryKey>(countries .length); for (String country : countries) { final PrimaryKey pk = userTable.createPrimaryKey(); pk.put("Country", "US"); keys.add(pk); } /* Initialize multiRowOption: userId in [0, 100]. child tables email, address */ final FieldRange range = userTable.createFieldRange("userid"); range.setStart(0, true).setEnd(100, true); final MultiRowOptions mro = new MultiRowOptions(range, null /* ancestor */, Arrays.asList(emailTable, addressTable)); /* Initialize TableIteratorOptions */ final int batchResultsSize = 200; final int parallalism = 9; final TableIteratorOptions tio = new TableIteratorOptions(null /* Direction */, null /* Consistency */, 0 /* RequestTimeOut */, null /*TimeUnit*/, parallalism, batchResultsSize); TableIterator<Row> itr = null; try { itr = tableAPI.tableIterator(keys.iterator(), mro, tio); while (itr.hasNext()) { final Row user = itr.next(); /* ... */ } } catch (StoreIteratorException sie) { /* Handle exception.. */ } finally { itr.close(); }
  21. 21. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | • Useful for Replication Nodes with 64GB+ of available memory • Allocate memory outside of limits of Java Heap – Avoid long GC pauses – Mitigate Java “compressed OOPs” overhead • Native platform memory allocator – Oracle and IBM JDK • Contains records evicted from main cache (LRU) • Configurable – Based on available memory – Optional Performance tuning • change-policy -params systemPercent=10 – Setting 100 will turn OFF off-heap cache • change-policy -params rnHeapMaxMB=8192 22 Performance: Off-heap Cache File System Cache Newly Written log files, Sequentially read log files, Log file fragments. NoSQL DB Objects Off Heap Cache Java Heap (min (32G, 85% of memory)) System Use (10%) (100 – rnHeapPercent – systemPercent) Managed by GC Not Managed by GC
  22. 22. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Thank You 23
  23. 23. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Join NoSQL Database Community Twitter https://twitter.com/#!/OracleNoSQL LinkedIn http://www.linkedin.com/groups?gid=4147754 Oracle’s NoSQL DB blog https://blogs.oracle.com/nosql Oracle Technology Network http://bit.ly/1f0d8wU Developer Webcast Series http://bit.ly/1doV2jl Oracle.com/BigData 24
  24. 24. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

×