Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Node.js and Oracle Database: New Development Techniques

2,885 views

Published on

These slides are from the AUSOUG webinar viewable at https://www.ausoug.org.au/event/node-js-and-oracle-database-new-development-techniques/

The session covered the best node-oracledb data access features for building great Node.js applications with Oracle Database. Spanning topics from the latest connection pooling advances, right through to efficient ways to access your data, all the best tips are demonstrated. After another busy year of node-oracledb releases, don’t miss the latest on this rapidly growing ecosystem.
This is a technical talk with code snippets demonstrating efficient use of the Node.js node-oracledb driver for Oracle DB. There have been several key releases of node-oracledb over the last year so there is plenty to talk about.

Published in: Software
  • Login to see the comments

Node.js and Oracle Database: New Development Techniques

  1. 1. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Node.js and Oracle Database Christopher Jones Data Access Development Oracle Database 22 May 2019 christopher.jones@oracle.com @ghrd New Development Techniques
  2. 2. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 5 About Me Christopher Jones Product Manger for Oracle Database, Scripting Language Drivers (and related technology) Contact information christopher.jones@oracle.com @ghrd blogs.oracle.com/opal
  3. 3. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. 6
  4. 4. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 8 Introduction It Starts and Ends With Connections Getting Data Out of the Database Putting Data Into the Database JSON and Simple Oracle Document Access What’s coming in node-oracledb 4 1 2 3 4 5 6
  5. 5. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 9 Introduction1 2 3 4 5 6
  6. 6. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Node.js • Middle-tier JavaScript runtime – Built on Chrome’s V8 JavaScript engine – Lightweight and efficient – Event-driven, non-blocking I/O model • Allows one language, front-end and back-end • npm package ecosystem – World’s largest repo of open-source libraries • Open Source – Under Node.js Foundation 10
  7. 7. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb • Node.js module for accessing Oracle Database • Development is under Apache 2.0 license – GitHub repository for source code – Ongoing features and maintenance by Oracle • Installable from npm registry Users can contribute under the Oracle Contributor Agreement. Thanks to all who have contributed code, documentation and ideas 11
  8. 8. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 12 Under the Covers app.js node-oracledb ODPI-C Oracle Call Interface in Oracle Client libraries Oracle Net Provides: • Connectivity, Failover, Encryption, and more • Configure with tnsnames.ora, sqlnet.ora Oracle Client Provides: • Data Access, Performance, High Availability, and more • Configure with oraaccess.xml Node.js
  9. 9. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb Features • Async/Await, Promises, Callbacks and Streams • SQL and PL/SQL Execution / Fetching of large result sets / REF CURSORs • Binding using JavaScript objects or arrays / DML RETURNING / Batch Statement Execution • Large Objects: CLOBs and BLOBs and Strings/Buffers or Streams • Smart mapping between JavaScript and Oracle types with manual mapping also available • Query results as JavaScript objects or arrays • Connection Pooling (Hetero & Homogeneous) with Aliasing, Queuing, Caching, Tagging, Draining • External Authentication / Privileged Connections / Proxy Connections / Password Changing • Row Prefetching / Statement Caching / Client Result Caching • End-to-End Tracing / Edition-Based Redefinition • Continuous Query Notification • Simple Oracle Document Access (SODA) • Oracle Database High Availability Features and Oracle Net Features 13
  10. 10. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 14 node-oracledb Classes Base class • Get connections or create pools • Set configuration parameters Connection Pooling • Dramatically increases performance • Built-in pool cache for convenience SQL and PL/SQL Execution • Transaction support w/data type conversion • Bind using JavaScript objects or arrays Read-consistent, pageable cursor • Used for large result sets • Callbacks, Promises or Node.js streams Large object support • Stream large LOBs with this class • Can fetch smaller LOBs as string/buffer Oracledb Pool Connection ResultSet Lob
  11. 11. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Starting with node-oracledb 1. Install Node.js 2. Install Oracle Instant Client 3. Create package.json 4. Run npm install Confidential – Oracle Internal/Restricted/Highly Restricted 16
  12. 12. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb fine print • Prebuilt node-oracledb 3.1 binary modules exist for: – Windows, Linux and macOS – Node.js 6, 8, 10, 11 • Upcoming node-oracledb 4.0 will be for Node.js 8+ • You can also build node-oracledb from source • LTS strategy Confidential – Oracle Internal/Restricted/Highly Restricted 17
  13. 13. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | oracledb.getConnection( {user:"hr", password:"welcome", connectString:"localhost/orclpdb1"}, function(err, connection) { if (err) { console.error(err.message); return; } connection.execute( `SELECT department_id, department_name FROM departments WHERE manager_id < :id`, [110], { outFormat: oracledb.ARRAY }, function(err, result) { if (err) { console.error(err.message); return; } console.log(result.rows); connection.close(function(err) { if (err) { console.error(err.message); } }); }); }); 18 node-oracledb – Node.js Callback Style
  14. 14. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 19 node-oracledb - Node.js 8’s Async/Await async function getEmployee(empid) { let conn; try { connection = await oracledb.getConnection( {user: "hr", password: "welcome", connectString: "localhost/orclpdb1" }); const result = await connection.execute( `SELECT department_id, department_name FROM departments WHERE manager_id < :id`, [empid] ); console.log(result.rows); } catch (err) { console.error(err); } finally { if (connection) { try { await connection.close(); } catch (err) { console.error(err); } } } } $ node select.js [ [ 60, 'IT' ], [ 90, 'Executive' ], [ 100, 'Finance' ] ]
  15. 15. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: • Using a fixed Oracle time zone helps avoid machine and deployment differences process.env.ORA_SDTZ = 'UTC'; Confidential – Oracle Internal/Restricted/Highly Restricted 20 TIP
  16. 16. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 21 It Starts and Ends With Connections 1 2 3 4 5 6
  17. 17. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Standalone Connections const oracledb = require("oracledb"); let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE"}; let connection = await oracledb.getConnection(dbConfig); // ... Use connection await connection.close(); 22 Node.js DB
  18. 18. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pooled Connections // Application initialization let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1, poolTimeout:60}; let pool = await oracledb.createPool(dbConfig); // General runtime let connection = await pool.getConnection(); // ... Use connection await connection.close(); // Application termination await pool.close(); 23 Node.js DB
  19. 19. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 24 Node.js Architecture (not entirely accurate) Timers TCP/UDP Evented File I/O DNS User Code Thread Pool Async API CallsMain Thread Event / Callback Queue Callback Functions
  20. 20. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • oracledb.createPool({ . . ., poolMin:0, poolMax:10, poolIncrement:1, poolTimeout:60 }, . . . – 10 users access the app • Result: slower than expected throughput. May see deadlocks – In-use connections wait for DB responses so they hold a thread in use – Node.js has maximum 4 worker threads by default (upper limit is 128) • Solution: increase Node.js thread limit before Node.js threadpool starts: export UV_THREADPOOL_SIZE=10 – May want more threads than connections, to do non-database work 25 Scenario 1: Want to Allow 10 Concurrent DB Users
  21. 21. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • Scenario: – UV_THREADPOOL_SIZE=4 – App opens one connection – promise.all on 1 x long running SELECT and 3 x INSERTs • Result – Each execute() gets thread from worker thread pool – But the connection can only do one thing • So the query will block the inserts, blocking the threads from doing other work • Solution – Keep control in the JavaScript layer 26 Scenario 2: promise.all
  22. 22. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • Scenario: – Each iteration of promise.all gets its own connection – Each connection used to insert some data • Can the database even handle (lots of) multiple connections? – You have to balance workload expectations with actual resources available • Transactional consistency not possible • Data load solution shown in later slides! 27 Scenario 3: Parallelizing Data Load
  23. 23. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Avoid Connection Storms on DB Server • Set poolMin = poolMax: oracledb.createPool( { user: "hr", password: "welcome", connectString: "localhost/XE”, poolMin: 10, poolMax: 10, poolIncrement: 0, poolTimeout: 60 }) • Or use Oracle Net Listener rate limit settings – CONNECTION_RATE_listener name – RATE_LIMIT 28 Node.js DB TIP
  24. 24. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Basic High Availability • Configure operating system network TCP timeouts • Configure Oracle Net options on the DB and/or node-oracledb side e.g. – SQLNET.OUTBOUND_CONNECT_TIMEOUT, SQLNET.RECV_TIMEOUT, SQLNET.SEND_TIMEOUT, EXPIRE_TIME,(ENABLE=BROKEN)etc. • connection.break() – may need: DISABLE_OOB=on if firewall blocks out-of-band breaks • connection.callTimeout with 18c Oracle Client libraries 29 TIP
  25. 25. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Immediately Usable Connections • Connections in pools can become unusable due to network dropouts etc • Use 19, 18, or 12.2 Oracle Client libraries – these have an always-on, network check during pool.getConnection() • Tune createPool()’s pingInterval setting – Round-trip ping for detecting session issues Confidential – Oracle Internal/Restricted/Highly Restricted 31 c = p.getConnection() c.execute() // error c = p.getConnection() c.execute() // error X X
  26. 26. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | poolPingInterval poolPingInterval Behavior at pool.getConnection() < 0 Never check for connection aliveness = 0 Always check for each pool.getConnection() > 0 (default is 60) Checks aliveness with round-trip ping if the connection has been idle in the pool (not "checked out" to the application by getConnection()) for at least poolPingInterval seconds. Confidential – Oracle Internal/Restricted/Highly Restricted 32 If a ping detects an unusable connection, it is dropped and a new one created before p.getConnection() returns
  27. 27. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Usable Connections • Always do error checking and recovery after execute() – network failures could occur anytime after getConnection() – Could disable poolPingInterval to get ultimate scalability • Avoid firewall timeouts and DBAs killing “idle” sessions – node-oracledb ping features may mask these problems • scalability impact – Use AWR to check connection rate Confidential – Oracle Internal/Restricted/Highly Restricted 33 TIP
  28. 28. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 34 Use the pool cache to access pool by name Pro Tip: No need to pass a pool around dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1, alias:"mypool"}; /* pool = */ await oracledb.createPool(dbConfig); connection = await oracledb.getConnection("mypool"); dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1}; await oracledb.createPool(dbConfig); connection = await oracledb.getConnection(); TIP
  29. 29. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection State • Many applications set connection state – e.g. using ALTER SESSION commands to set the date format, or a time zone – For all practical purposes, a 'session' is the same as a 'connection’ • Pooled connections retain state after being released back to the pool – Next user of the connection will see the same state – But will the next user get the same connection or a brand new one? Confidential – Oracle Internal/Restricted/Highly Restricted 35 Node.js DB
  30. 30. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Avoid unnecessary ALTER SESSIONs connection = await oracledb.getConnection(dbConfig); connection.execute(`ALTER SESSION . . . `); Use a connection callback function to set session state Confidential – Oracle Internal/Restricted/Highly Restricted 36 TIP X
  31. 31. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool sessionCallback await oracledb.createPool( {user: un, password: pw, connectString: cs, sessionCallback: initSession}); // Called only when the pool selects a brand new, never-before used connection. // Called before pool.getConnection() returns. function initSession(connection, requestedTag, cb) { connection.execute(`ALTER SESSION SET TIME_ZONE = 'UTC'`, cb); } . . . connection = await oracledb.getConnection(); // get connection in UTC let result = await connection.execute(sql, binds, options); await connection.close(); See examples/sessionfixup.js Confidential – Oracle Internal/Restricted/Highly Restricted 37 Scenario 1: when all connections should have the same state
  32. 32. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool sessionCallback poolMax Service Invoked getConnection() calls ALTER SESSION calls SELECT calls Total SQL Calls Without a callback 4 1000 1000 1000 1000 2000 Confidential – Oracle Internal/Restricted/Highly Restricted 38 Scenario 1: when all connections should have the same state Micro-service where each service invocation does one query poolMax Service Invoked getConnection() calls ALTER SESSION calls SELECT calls Total SQL Calls Without a callback 4 1000 1000 1000 1000 2000 With a callback 4 1000 1000 4 1000 1004
  33. 33. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Connection Pool sessionCallback Avoid round-trips function initSession(connection, requestedTag, cb) { connection.execute( `begin execute immediate 'alter session set nls_date_format = ''YYYY-MM-DD'' nls_language = AMERICAN'; execute immediate ' . . . '; -- other SQL end;`, cb); } Confidential – Oracle Internal/Restricted/Highly Restricted 39 TIP
  34. 34. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Set a connection tag to represent connection state connection = await pool.getConnection(); // Set state and update the tag connection.execute(`ALTER SESSION . . . `, cb); connection.tag = 'NAME1=VALUE1;NAME2=VALUE2’; await connection.close(); Confidential – Oracle Internal/Restricted/Highly Restricted 40 Scenario 2: when connections need differing state
  35. 35. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Requesting an already tagged pooled connection conn = await oracledb.getConnection({poolAlias:'default', tag:'USER_TZ=UTC'}); When using a sessionCallback this will: – Select an existing connection in the pool that has the tag USER_TZ=UTC • sessionCallback is NOT called – Or, select a new, previously unused connection in the pool (which will have no tag) • sessionCallback is called – Or, select a previously used connection with a different tag • The existing session state and tag are cleared • sessionCallback is called Confidential – Oracle Internal/Restricted/Highly Restricted 41 Scenario 2: when connections need differing state
  36. 36. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging conn = await oracledb.getConnection({poolAlias:'default’, tag:'USER_TZ=UTC', matchAnyTag: true}); • This will: – Select an existing connection in the pool that has the requested tag • sessionCallback is NOT called – Or, select another connection in the pool • Any existing connection state and tag are retained • sessionCallback is called Confidential – Oracle Internal/Restricted/Highly Restricted 42 Scenario 2: when connections need differing state
  37. 37. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Confidential – Oracle Internal/Restricted/Highly Restricted 43 Scenario 2: when connections need differing state function initSession(connection, requestedTag, cb) { console.log(`requested tag: ${requestedTag}, actual tag: ${connection.tag}`); // Parse requestedTag and connection.tag to decide what state to set . . . // Set the state connection.execute(`ALTER SESSION SET . . .`, (err) => { connection.tag = requestedTag; // Update to match the new state cb(err); // Call cb() last }); } See examples/sessiontagging1.js and examples/sessiontagging2.js
  38. 38. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | End Connections When No Longer Needed • Releasing connections is: – Required for pooled connections – Best practice for standalone connections • Don’t release connections until you are done with them, otherwise: – “NJS-003: invalid connection”, “DPI-1010: not connected”, “ORA-12537: TNS:connection closed”, etc. – Example scenario: promise.all error handler releasing the connection • Handler is called on first error – Example scenario: streaming from a Lob instance 45
  39. 39. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Draining and Force Closing • Closing the pool closes connections cleanly • May need: DISABLE_OOB=on if firewall blocks out-of-band breaks 46 // close pool only if no connections in use await pool.close(); // close pool when no connections are in use, or force it closed after 10 seconds // No new connections can be created but existing connections can be used for 10 seconds await pool.close(10); // force the pool closed immediately await pool.close(0);
  40. 40. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 47 Getting Data Out of the Database 1 2 3 4 5 6
  41. 41. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Query Methods • Direct Fetches: execute("SELECT . . .", . . . ) – All rows are returned in one big memory-limited array, or limited to maxRows • ResultSet : execute("SELECT . . .", . . ., { resultSet: true }, . . . ) – getRow(): Returns one row on each call until all rows are returned – getRows(numRows): Returns batches of rows in each call until all rows are returned • Stream: queryStream("SELECT . . .", . . . ) – Streams rows until all rows are returned 48
  42. 42. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Direct Fetches • Default query behavior – Easy to use • Tune network transfer performance with fetchArraySize – Memory can incrementally grow when the number of query rows is unknown, or varies from execution to execution – A single large chunk of memory doesn't need to be pre-allocated to handle the 'worst case' of a large number of rows • Drawbacks of direct fetches: – One big array of results is needed – Concatenation of record batches can use more memory than the final array requires 49
  43. 43. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 50 ResultSet Fetches const result = await connection.execute( `SELECT * FROM bigtable`, [], // no bind variables { resultSet: true } ); const rs = result.resultSet; let row; while ((row = await rs.getRow())) { console.log(row); } await rs.close(); // always close the ResultSet
  44. 44. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Querying Data • Use row-limiting SQL clauses: SELECT last_name FROM employees ORDER BY last_name OFFSET :offset ROWS FETCH NEXT :maxnumrows ROWS ONLY • Use Direct Fetches for known small number of rows – set fetchArraySize to number of rows, if known – E.g for single row fetches set fetchArraySize to 1 • Use ResultSets or queryStream() for data sets of unknown or big size – Always close ResultSets • Cast date and timestamp bind variables in WHERE clauses: – . . . WHERE cast(:ts as timestamp) < mytimestampcol – . . . WHERE cast(:dt as date) < mydatecol 51 TIP
  45. 45. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Fetching LOBs • Fetch and bind CLOB and BLOB as String and Buffer for performance – Only use Lob Stream class for huge LOBs or if memory is limited const result = await connection.execute( `SELECT c FROM mylobs WHERE id = :idbv`, [1], { fetchInfo: {"C": {type: oracledb.STRING}} } ); const clob = result.rows[0][0]; console.log(clob); 52
  46. 46. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 53 Putting Data Into the Database 1 2 3 4 5 6
  47. 47. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Single Row DML is Easy SQL> CREATE TABLE mytable (key NUMBER(9) NOT NULL, fruit VARCHAR2(40)); result = await connection.execute( "INSERT INTO mytable VALUES (:k, :f)", { k: 1, f: 'apple' }, // Bind values { autoCommit: true }); // Or use connection.commit() later console.log("Rows inserted: " + result.rowsAffected); // 1 54
  48. 48. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Use Batch Execution for DML • Executes one DML statement (e.g INSERT, UPDATE) with many data values • Reduces round trips: "A server round-trip is defined as the trip from the client to the server and back to the client." 55 executeMany() TIP
  49. 49. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 56 Basic Array DML sql = "INSERT INTO mytable VALUES (:k, :f)"; data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" } ]; options = { autoCommit: true, bindDefs: { k: { type: oracledb.NUMBER }, f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result); { rowsAffected: 2 }
  50. 50. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 57 data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" }, { k: null, f: "cherry" }, { k: 3, f: "pear" }, { k: null, f: "damson" } ]; options = { batchErrors: true, bindDefs: { k: { type: oracledb.NUMBER }, f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result); BatchErrors { rowsAffected: 3, batchErrors: [ { Error: ORA-01400: cannot insert NULL into ("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 2 }, { Error: ORA-01400: cannot insert NULL into ("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 4 } ] }
  51. 51. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Batch Errors • Batch errors are in the result, not the error – Some classes of error will always trigger real errors • Any autoCommit will be ignored if there are DML errors – Valid rows will have been inserted and can be explicitly committed • Attribute rowsAffected shows 3 rows were inserted • Array of errors, one for each problematic record – The offset is the data array position (0-based) of the problem record 58
  52. 52. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | {rowsAffected: 7, dmlRowCounts: [ 5, 2 ]} 59 DML Row Counts // assume 5 apples and 2 bananas exist in the table sql = "DELETE FROM mytable WHERE fruit = :f"; data = [ { f: "apple" }, { f: " banana" } ]; options = { dmlRowCounts: true, bindDefs: { f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result);
  53. 53. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 60 sql = "INSERT INTO tab VALUES (:1) RETURNING ROWID INTO :2"; binds = [ ["One"], ["Two"], ["Three"] ]; options = { bindDefs: [ { type: oracledb.STRING, maxSize: 5 }, { type: oracledb.STRING, maxSize: 18, dir: oracledb.BIND_OUT } ] }; result = await connection.executeMany(sql, data, options); console.log(result.outBinds); DMLRETURNING [ [ [ 'AAAmWkAAMAAAAnWAAA' ] ], [ [ 'AAAmWkAAMAAAAnWAAB' ] ], [ [ 'AAAmWkAAMAAAAnWAAC' ] ] ]
  54. 54. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | executeMany() Benchmark • Data: for (var i = 0; i < numRows; i++) data.push([i, "Test " + i]); • SQL: sql = "INSERT INTO mytable VALUES (:1, :2)"; • Multiple inserts: for (var i = 0; i < numRows; i++) { await connection.execute(sql, data[i], {autoCommit: (i < numRows-1 ? false : true)}); } • Single insert: await connection.executeMany(sql, data, {autoCommit: true}); 61
  55. 55. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | executeMany() Benchmark – Scenario 1 • 1 number, 1 short string • Local DB • YMMV 1 10 100 1000 10000 100000 execute() 10 38 265 2,323 23,914 227,727 executeMany() 16 9 20 16 39 361 62
  56. 56. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 63 executeMany() Benchmark – Scenario 2 • 1 x NUMBER, • 3 x VARCHAR2(1000)) • Remote DB • YMMV 1 10 100 1,000 execute() 57 510 5,032 50,949 executeMany() 57 155 368 2,537
  57. 57. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: executeMany() • For huge data sets may need to call executeMany() multiple times with batches of records – autoCommit: false for first batches – autoCommit: true for the last batch • Use SQL*Loader or Data Pump instead, where appropriate – These were added to Instant Client 12.2 64 TIP
  58. 58. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 66 PL/SQL
  59. 59. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Calling PL/SQL is easy const result = await connection.execute( `BEGIN myproc(:i, :io, :o); END;`, { i: 'Chris', // type found from the data. Default dir is BIND_IN io: { val: 'Jones', dir: oracledb.BIND_INOUT }, o: { type: oracledb.NUMBER, dir: oracledb.BIND_OUT } }); console.log(result.outBinds); Confidential – Oracle Internal/Restricted/Highly Restricted 67
  60. 60. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 68 SQL> CREATE OR REPLACE PROCEDURE testproc (p_in IN NUMBER, p_out OUT NUMBER) AS BEGIN p_out := p_in * 2; END; plsql = "BEGIN testproc(:1, :2); END;"; binds = [ [1], [2], [3] ]; options = { bindDefs: [ { type: oracledb.NUMBER }, { type: oracledb.NUMBER, dir: oracledb.BIND_OUT } ] }; result = await connection.executeMany(sql, data, options); console.log(result.outBinds); CallingPL/SQL [ [ 2 ], [ 4 ], [ 6 ] ]
  61. 61. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | PL/SQL Collection Associative Array (Index-by) Binds TYPE numtype IS TABLE OF NUMBER INDEX BY BINARY_INTEGER; PROCEDURE myinproc(p_id IN NUMBER, p_vals IN numtype) IS BEGIN FORALL i IN INDICES OF p_vals INSERT INTO sometable (id, numcol) VALUES (p_id, p_vals(i)); END; await connection.execute( "BEGIN myinproc(:idbv, :valbv); END;", { idbv: 1234, valbv: { type: oracledb.NUMBER, dir: oracledb.BIND_IN, val: [1, 2, 23, 4, 10] } }; 69
  62. 62. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 70 JSON and Simple Oracle Document Access 1 2 3 4 5 6
  63. 63. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | JSON with Oracle Database 12.1.0.2 SQL> CREATE TABLE myjsontab ( c CLOB CHECK (c IS JSON)) LOB (c) STORE AS (CACHE); myContent = {name: "Sally", address: {city: "Melbourne"}}; json = JSON.stringify(myContent); result = await connection.execute( 'insert into myjsontab (c) values (:cbv)', { cbv: json } ); 71 Inserting
  64. 64. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | JSON with Oracle Database 12.1.0.2 result = await connection.execute( `select c from myjsontab t where t.c.name = :cbv`, // 12.2 syntax { cbv: 'Sally' }, { fetchInfo: {"C": {type: oracledb.STRING } }}); js = JSON.parse(result.rows[0]); console.log('Name is: ' + js.name); console.log('City is: ' + js.address.city); //sql = "select c FROM myjsontab where json_exists(c, '$.address.city')"; 72 Fetching
  65. 65. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Simple Oracle Document Access (SODA) • Set of NoSQL-style APIs to create and access documents – Documents are often JSON – Oracle maps SODA calls to managed tables, indexes and sequences – Query-by-example access makes data operations easy • SQL is not necessary – But could be used for advanced analysis • SODA APIs also exist in Python, PL/SQL, C and Java 73 node-oracledb support requires DB 18.3 and Oracle Client 18.5/19.3
  66. 66. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 74 node-oracledb SODA Classes Base SODA class • Get SODA database • Manipulate SODA collections Set of Documents • Create and find documents Operation Builder • Filters and QBE • Retrieve, replace or remove documents Document Cursor • Iterate over retrieved documents SODA Document • Document content • Access as JSON, object or buffer SodaDatabase SodaCollection SodaOperation SodaDocumentCursor SodaDocument Oracledb Pool Connection ResultSet Lob
  67. 67. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 75 Creating Collections Using SODA in node-oracledb SQL> grant SODA_APP to cj; conn = await oracledb.getConnection({user:"cj",....}); soda = await conn.getSodaDatabase(); collection = await soda.createCollection("mycollection");
  68. 68. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 76 Inserting Documents Using SODA in node-oracledb content = {name: "Anthony", address: {city: "Edmonton"}}; await collection.insertOne(content); doc = await collection.insertOneAndGet(content); console.log("Key is:", doc.key); // JSON (or Object or Buffer) doc2 = soda.createDocument('{"name":"Venkat","city":"Bengaluru"}'); await collection.insertOne(doc2);
  69. 69. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Using SODA in node-oracledb • Recommendation is to turn on oracle.autocommit – Like with SQL, avoid auto commit when inserting lots of documents • SODA’s DDL-like operations occur in autonomous transactions 77 Commit Behavior
  70. 70. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 78 Operation Builder – find() Using SODA in node-oracledb doc = await collection.find().key(key).getOne(); // A SodaDocument content = doc.getContent(); // A JavaScript object console.log('Retrieved SODA document as an object:'); console.log(content); content = doc.getContentAsString(); // A JSON string console.log('Retrieved SODA document as a string:'); console.log(content);
  71. 71. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 79 Operation Builder – filter() Query-by-example (QBE) Using SODA in node-oracledb // Find all documents with city names starting with 'S' console.log('Cities starting with S'); documents = await collection.find() .filter({"address.city": {"$like": "S%"}}) .getDocuments(); for (let i = 0; i < documents.length; i++) { content = documents[i].getContent(); console.log(' city is: ', content.address.city); }
  72. 72. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 80 What’s coming in node-oracledb 4 1 2 3 4 5 6
  73. 73. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | What’s coming in node-oracledb 4.0 • Install (compile) current code snapshot with: npm install oracle/node-oracledb#dev-4.0 • Already landed: – N-NAPI code refactor improving Node.js version compatibility – Advanced Queuing for “RAW” queues, ie. Node.js String and Buffers – Implicit Results – SODA bulk insert • Being investigated (no promises!): – SQL and PL/SQL object binding & queries • When? Planned CY2019 Confidential – Oracle Internal/Restricted/Highly Restricted 81
  74. 74. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 82 AQ ‘RAW’ Queue Demo Oracle Advanced Queuing Dequeue: queue = connection.queue('MYQUEUE'); msg = await queue.deqOne(); await connection.commit(); console.log(msg.payload.toString()); Enqueue: queue = connection.queue('MYQUEUE'); messageString = 'This is my message'; await queue.enqOne(messageString); await connection.commit(); BEGIN dbms_aqadm.create_queue_table('DEMO_RAW_QUEUE_TAB', 'RAW'); dbms_aqadm.create_queue('MYQUEUE', 'DEMO_RAW_QUEUE_TAB'); dbms_aqadm.start_queue('MYQUEUE'); END; /
  75. 75. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 83 Wrap up
  76. 76. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb • Connection management is the basis for good scalability • Utilize the best query method for the task, and tune it • Use executeMany() to insert/update/delete multiple records efficiently • Use SODA’s NoSQL-style data access to simplify applications 84
  77. 77. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Resources • Mail: christopher.jones@oracle.com • Twitter: @ghrd, @AnthonyTuininga, @dmcghan • Blog: https://blogs.oracle.com/opal • Office Hours: https://tinyurl.com/NodeHours • Facebook: https://www.facebook.com/groups/oraclescripting/ • Home: https://oracle.github.io/node-oracledb/ • Code: https://github.com/oracle/node-oracledb 85

×