Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2014 EnterpriseDB Corporation. All rights reserved. 1
© 2014 EnterpriseDB Corporation. All rights reserved. 2
NoSQL on ACID: Meet Unstructured
Postgres
Marc Linster – SVP, Prod...
© 2014 EnterpriseDB Corporation. All rights reserved. 3
• EDB Introduction
• NoSQL & ACID DBMS –
Common Misconceptions
• P...
© 2014 EnterpriseDB Corporation. All rights reserved. 4
POSTGRES
innovation
ENTERPRISE
reliability
24/7
support
Services
&...
© 2014 EnterpriseDB Corporation. All rights reserved. 5
Postgres Plus
Advanced Server Postgres Plus
Cloud Database
High Av...
© 2014 EnterpriseDB Corporation. All rights reserved. 6
Why EDB?
More than 3,000 Enterprises and Governments
© 2014 EnterpriseDB Corporation. All rights reserved. 7
EnterpriseDB
is a Market
Leader
The Gartner report, Magic
Quadrant...
© 2014 EnterpriseDB Corporation. All rights reserved. 8
EDB is an Open Source Community
Leader
Amit Kapila
Ashesh Vashi
Br...
© 2014 EnterpriseDB Corporation. All rights reserved. 9
NoSQL & ACID DBMS – Common
Misconceptions
• ACID Compliant DBMS do...
© 2014 EnterpriseDB Corporation. All rights reserved. 10
• Data Types
− Postgres has JSON, JSONB, Key-Value Pair,
plus arr...
© 2014 EnterpriseDB Corporation. All rights reserved. 11
• HSTORE
− Key-value pair
− Simple, fast and easy
− Postgres v 8....
© 2014 EnterpriseDB Corporation. All rights reserved. 12
• JSON is the most popular
data-interchange format on the web
• D...
© 2014 EnterpriseDB Corporation. All rights reserved. 13
• Wherever there is JavaScript you will find JSON
• Most language...
© 2014 EnterpriseDB Corporation. All rights reserved. 14
• Creating a table with a JSONB field
CREATE TABLE json_data (dat...
© 2014 EnterpriseDB Corporation. All rights reserved. 15
SELECT DISTINCT
data->>'name' as products
FROM json_data;
product...
© 2014 EnterpriseDB Corporation. All rights reserved. 16
SELECT data FROM json_data;
data
--------------------------------...
© 2014 EnterpriseDB Corporation. All rights reserved. 17
• 1. Number:
− Signed decimal number that may contain a fractiona...
© 2014 EnterpriseDB Corporation. All rights reserved. 18
{
"firstName": "John", -- String Type
"lastName": "Smith", -- Str...
© 2014 EnterpriseDB Corporation. All rights reserved. 19
• BSON – stands for
‘Binary JSON’
• BSON != JSONB
− BSON cannot r...
© 2014 EnterpriseDB Corporation. All rights reserved. 20
JSONB and Node.js - Easy as π
• Simple Demo of Node.js to Postgre...
© 2014 EnterpriseDB Corporation. All rights reserved. 21
• JSON is naturally integrated with
ANSI SQL in Postgres
• JSON a...
© 2014 EnterpriseDB Corporation. All rights reserved. 22
SELECT DISTINCT
product_type,
data->>'brand' as Brand,
data->>'av...
© 2014 EnterpriseDB Corporation. All rights reserved. 23
Bridging between SQL and NoSQL
Simple ANSI SQL Table Definition
C...
© 2014 EnterpriseDB Corporation. All rights reserved. 24
• Start unstructured, and become
structured as you learn more
− U...
© 2014 EnterpriseDB Corporation. All rights reserved. 25
• Goal
− Help our customers understand when to choose Postgres an...
© 2014 EnterpriseDB Corporation. All rights reserved. 26
Postgres NoSQL Performance Evaluation
Generate JSON Documents
10 ...
© 2014 EnterpriseDB Corporation. All rights reserved. 27
NoSQL Performance Evaluation
276% 295%
465%
208%
0%
50%
100%
150%...
© 2014 EnterpriseDB Corporation. All rights reserved. 28
• Tests confirm that Postgres can handle many NoSQL
workloads
• E...
© 2014 EnterpriseDB Corporation. All rights reserved. 29
• FDW implements SQL/MED ("SQL
Management of External Data")
• Po...
© 2014 EnterpriseDB Corporation. All rights reserved. 30
CREATE EXTENSION mongo_fdw;
CREATE SERVER mongo_server
FOREIGN DA...
© 2014 EnterpriseDB Corporation. All rights reserved. 31
SELECT * FROM mongo_data WHERE brand='ACME' limit 10;
name | bran...
© 2014 EnterpriseDB Corporation. All rights reserved. 32
INSERT INTO mongo_data(name, brand, type)
VALUES('iphone6 phone',...
© 2014 EnterpriseDB Corporation. All rights reserved. 33
• Structures and standards emerge!
• Data has references (product...
© 2014 EnterpriseDB Corporation. All rights reserved. 34
• Postgres has many NoSQL features without
the drawbacks:
− Schem...
© 2014 EnterpriseDB Corporation. All rights reserved. 35
• Postgres is Not Only SQL (NoSQL is No SQL only)
• Fully ACID co...
© 2014 EnterpriseDB Corporation. All rights reserved. 36
© 2014 EnterpriseDB Corporation. All rights reserved. 37
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
Deep Dive on Elastic Load Balancing
Next
Download to read offline and view in fullscreen.

1

Share

Download to read offline

NoSQL on ACID: Meet Unstructured Postgres

Download to read offline

This presentation reviews Postgres’ unsurpassed NoSQL capabilities, which include a very rich JSON data type, a key-value pair data type, and a SQL/MED-based virtual data integration capability to bridge between Postgres and many other databases. Marc Linster discusses NoSQL syntax, NoSQL performance, and SQL/NoSQL integration and implementation recommendations.

To listen to the recording visit EnterpriseDB.com > Resources > Webcasts > On Demand Webcasts.

If you have any questions, please email sales@enterprisedb.com

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

NoSQL on ACID: Meet Unstructured Postgres

  1. 1. © 2014 EnterpriseDB Corporation. All rights reserved. 1
  2. 2. © 2014 EnterpriseDB Corporation. All rights reserved. 2 NoSQL on ACID: Meet Unstructured Postgres Marc Linster – SVP, Products and Services
  3. 3. © 2014 EnterpriseDB Corporation. All rights reserved. 3 • EDB Introduction • NoSQL & ACID DBMS – Common Misconceptions • Postgres – NoSQL for the Enterprise • Performance Evaluation • FDW – Coexistence of NoSQL only and SQL • Postgres as a NoSQL/SQL Integration Platform Agenda
  4. 4. © 2014 EnterpriseDB Corporation. All rights reserved. 4 POSTGRES innovation ENTERPRISE reliability 24/7 support Services & training Enterprise-class features & tools Indemnification Product road-map Control Thousands of developers Fast development cycles Low cost No vendor lock-in Advanced features Enabling commercial adoption of Postgres
  5. 5. © 2014 EnterpriseDB Corporation. All rights reserved. 5 Postgres Plus Advanced Server Postgres Plus Cloud Database High Availability PerformanceManagement REMOTE DBA 24x7 SUPPORT PROFESSIONAL SERVICES TRAINING EDB Serves All Your Postgres Needs PostgreSQL Security
  6. 6. © 2014 EnterpriseDB Corporation. All rights reserved. 6 Why EDB? More than 3,000 Enterprises and Governments
  7. 7. © 2014 EnterpriseDB Corporation. All rights reserved. 7 EnterpriseDB is a Market Leader The Gartner report, Magic Quadrant for Operational Database Management Systems, by Donald Feinberg, Merv Adrian and Nick Heudecker, was published October 16, 2014.
  8. 8. © 2014 EnterpriseDB Corporation. All rights reserved. 8 EDB is an Open Source Community Leader Amit Kapila Ashesh Vashi Bruce Momjian Dave Page Devrim Gunduz Jan Wieck Kevin Grittner Korry Douglas Muhammad Usama Robert M Haas Thom Brown
  9. 9. © 2014 EnterpriseDB Corporation. All rights reserved. 9 NoSQL & ACID DBMS – Common Misconceptions • ACID Compliant DBMS don’t support JSON and KVP • Traditional DBMS can’t handle the volume or the speed • Web 2.0 Applications rely on JSON – traditional DBMS focus on text, integer, etc. • Traditional DBMS don’t work well with Web 2.0 development languages
  10. 10. © 2014 EnterpriseDB Corporation. All rights reserved. 10 • Data Types − Postgres has JSON, JSONB, Key-Value Pair, plus arrays, ranges, timezones, dates, integer, floating point, etc. • Performance Benchmarks − Postgres is very fast and can handle huge amounts of data − Postgres can selectively relax key ACID features to increase performance • Proven track record − ACID compliant − Open source − ANSI SQL − Large developer and vendor community Postgres – NoSQL for the Enterprise
  11. 11. © 2014 EnterpriseDB Corporation. All rights reserved. 11 • HSTORE − Key-value pair − Simple, fast and easy − Postgres v 8.2 – pre-dates many NoSQL-only solutions • JSON − Hierarchical document model − Introduced in Postgres 9.2, perfected in 9.3 • JSONB − Binary version of JSON − Faster, more operators and even more robust − Postgres 9.4 NoSQL Data in Postgres
  12. 12. © 2014 EnterpriseDB Corporation. All rights reserved. 12 • JSON is the most popular data-interchange format on the web • Derived from the ECMAScript Programming Language Standard (European Computer Manufacturers Association). • Supported by virtually every programming language • New supporting technologies continue to expand JSON’s utility − PL/V8 JavaScript extension − Node.js • Postgres native JSON data type (v9.2) and a JSON parser and a variety of JSON functions (v9.3) • Postgres JSONB data type with binary storage and indexing (v9.4) Postgres: Document Store
  13. 13. © 2014 EnterpriseDB Corporation. All rights reserved. 13 • Wherever there is JavaScript you will find JSON • Most languages support it • Node.Js is becoming popular. • Lighter and more compact than XML. • Most application don't need the rich structure of XML • Flexible Structure • Due to its flexible structure, JSON is a good fit for NoSQL. Why JSON
  14. 14. © 2014 EnterpriseDB Corporation. All rights reserved. 14 • Creating a table with a JSONB field CREATE TABLE json_data (data JSONB); • Simple JSON data element: {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1} • Inserting this data element into the table json_data INSERT INTO json_data (data) VALUES (’ { "name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1 } ') JSON Examples
  15. 15. © 2014 EnterpriseDB Corporation. All rights reserved. 15 SELECT DISTINCT data->>'name' as products FROM json_data; products ------------------------------ Cable TV Basic Service Package AC3 Case Black Phone Service Basic Plan AC3 Phone AC3 Case Green Phone Service Family Plan AC3 Case Red AC7 Phone A simple query for JSON data This query does not return JSON data – it returns text values associated with the key ‘name’
  16. 16. © 2014 EnterpriseDB Corporation. All rights reserved. 16 SELECT data FROM json_data; data ------------------------------------------ {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1} A query that returns JSON data This query returns the JSON data in its original format
  17. 17. © 2014 EnterpriseDB Corporation. All rights reserved. 17 • 1. Number: − Signed decimal number that may contain a fractional part and may use exponential notation. − No distinction between integer and floating-point • 2. String − A sequence of zero or more Unicode characters. − Strings are delimited with double-quotation mark − Supports a backslash escaping syntax. • 3. Boolean − Either of the values true or false. • 4. Array − An ordered list of zero or more values, − Each values may be of any type. − Arrays use square bracket notation with elements being comma-separated. • 5. Object − An unordered associative array (name/value pairs). − Objects are delimited with curly brackets − Commas to separate each pair − Each pair the colon ':' character separates the key or name from its value. − All keys must be strings and should be distinct from each other within that object. • 6. null − An empty value, using the word null JSON Data Types JSON is defined per RFC – 7159 For more detail please refer http://tools.ietf.org/html/rfc7159
  18. 18. © 2014 EnterpriseDB Corporation. All rights reserved. 18 { "firstName": "John", -- String Type "lastName": "Smith", -- String Type "isAlive": true, -- Boolean Type "age": 25, -- Number Type "height_cm": 167.6, -- Number Type "address": { -- Object Type "streetAddress": "21 2nd Street”, "city": "New York”, "state": "NY”, "postalCode": "10021-3100” } "phoneNumbers": [ -- Object Array { -- Object "type": "home”, "number": "212 555-1234” }, { "type": "office”, "number": "646 555-4567” } ], "children": [], "spouse": null -- Null } JSON Data Type Example
  19. 19. © 2014 EnterpriseDB Corporation. All rights reserved. 19 • BSON – stands for ‘Binary JSON’ • BSON != JSONB − BSON cannot represent an integer or floating-point number with more than 64 bits of precision. − JSONB can represent arbitrary JSON values. • Caveat Emptor! − This limitation will not be obvious during early stages of a project! JSON and BSON
  20. 20. © 2014 EnterpriseDB Corporation. All rights reserved. 20 JSONB and Node.js - Easy as π • Simple Demo of Node.js to Postgres cnnection
  21. 21. © 2014 EnterpriseDB Corporation. All rights reserved. 21 • JSON is naturally integrated with ANSI SQL in Postgres • JSON and HSTORE are elegant and easy to use extensions of the underlying object- relational model • JSON and SQL queries use the same language, the same planner, and the same ACID compliant transaction framework JSON and ANSI SQL – A Great Fit
  22. 22. © 2014 EnterpriseDB Corporation. All rights reserved. 22 SELECT DISTINCT product_type, data->>'brand' as Brand, data->>'available' as Availability FROM json_data JOIN products ON (products.product_type=json_data.data->>'name') WHERE json_data.data->>'available'=true; product_type | brand | availability ---------------------------+-----------+-------------- AC3 Phone | ACME | true JSON and ANSI SQL Example ANSI SQL JSON No need for programmatic logic to combine SQL and NoSQL in the application – Postgres does it all
  23. 23. © 2014 EnterpriseDB Corporation. All rights reserved. 23 Bridging between SQL and NoSQL Simple ANSI SQL Table Definition CREATE TABLE products (id integer, product_name text ); Select query returning standard data set SELECT * FROM products; id | product_name ----+-------------- 1 | iPhone 2 | Samsung 3 | Nokia Select query returning the same result as a JSON data set SELECT ROW_TO_JSON(products) FROM products; {"id":1,"product_name":"iPhone"} {"id":2,"product_name":"Samsung"} {"id":3,"product_name":"Nokia”}
  24. 24. © 2014 EnterpriseDB Corporation. All rights reserved. 24 • Start unstructured, and become structured as you learn more − Use the quick-to-get-started capabilities of NoSQL − Complete the initial sprints without a DBA − Move data between unstructured and structured − Embrace corporate data standards as you move from the stand-alone application towards integrated applications with a bigger value proposition Postgres Provides Great Flexibility By 2017, 50% of data stored in NoSQL DBMSs will be damaging to the business due to lack of applied information governance policies and programs. Gartner, December 2013
  25. 25. © 2014 EnterpriseDB Corporation. All rights reserved. 25 • Goal − Help our customers understand when to choose Postgres and when to chose a specialty solution − Help us understand where the NoSQL limits of Postgres are • Setup − Compare Postgres 9.4 to Mongo 2.6 − Single instance setup on AWS M3.2XLARGE (32GB) • Test Focus − Data ingestion (bulk and individual) − Data retrieval Postgres NoSQL Performance Evaluation Load Generator Postgres 9.4 MongoDB 2.6
  26. 26. © 2014 EnterpriseDB Corporation. All rights reserved. 26 Postgres NoSQL Performance Evaluation Generate JSON Documents 10 M documents & 50 M documents Load into MongoDB 2.6 (IMPORT) Load into Postgres 9.4 (COPY) 10 Million individual INSERT commands 10 Million individual INSERT commands Multiple SELECT statements Multiple SELECT statements T1 T2 T3
  27. 27. © 2014 EnterpriseDB Corporation. All rights reserved. 27 NoSQL Performance Evaluation 276% 295% 465% 208% 0% 50% 100% 150% 200% 250% 300% 350% 400% 450% 500% Data Load Insert Select Size Mongo DB 2.4/Postgres 9.4 Relative Performance Comparison (50 Million Documents) Postgres MongoDB Postgres MongoDB Data Load (s) 4,732 13,046 Insert (s) 29,236 86,253 Select (s) 594 2,763 Size (GB) 69 145 Correction to earlier versions: MongoDB console does not allow for INSERT of documents > 4K. This lead to truncation of the MongoDB size by approx. 25% of all records in the benchmark.
  28. 28. © 2014 EnterpriseDB Corporation. All rights reserved. 28 • Tests confirm that Postgres can handle many NoSQL workloads • EDB is making the test scripts publically available • EDB encourages community participation to better define where Postgres should be used and where specialty solutions are appropriate • Download the source at https://github.com/EnterpriseDB/pg_nosql_benchmark Postgres NoSQL Performance Evaluation
  29. 29. © 2014 EnterpriseDB Corporation. All rights reserved. 29 • FDW implements SQL/MED ("SQL Management of External Data") • PostgreSQL 9.1 - read-only support • PostgreSQL 9.3 – read/write support • FDW − Makes data on other servers (or services) look like tables in Postgres − available for databases (MongoDB, MySQL, Oracle, …), files, services (Twitter, …) • MongoDB FDW: https://github.com/EnterpriseDB Foreign Data Wrappers – Co-Existence Platform
  30. 30. © 2014 EnterpriseDB Corporation. All rights reserved. 30 CREATE EXTENSION mongo_fdw; CREATE SERVER mongo_server FOREIGN DATA WRAPPER mongo_fdw OPTIONS (address '172.24.39.129', port '27017'); CREATE USER MAPPING FOR enterprisedb SERVER mongo_server OPTIONS (username 'mongo', password 'mongo'); CREATE FOREIGN TABLE mongo_data( name text, brand text, type text) SERVER mongo_server OPTIONS ( database 'benchmark', collection 'json_tables'); MongoDB FDW Example
  31. 31. © 2014 EnterpriseDB Corporation. All rights reserved. 31 SELECT * FROM mongo_data WHERE brand='ACME' limit 10; name | brand | type -------------+-------+------- AC7553 Phone | ACME | phone AC7551 Phone | ACME | phone AC7519 Phone | ACME | phone AC7565 Phone | ACME | phone AC7555 Phone | ACME | phone AC7529 Phone | ACME | phone AC7528 Phone | ACME | phone AC7547 Phone | ACME | phone AC7587 Phone | ACME | phone AC7541 Phone | ACME | phone (10 rows) MongoDB FDW Example
  32. 32. © 2014 EnterpriseDB Corporation. All rights reserved. 32 INSERT INTO mongo_data(name, brand, type) VALUES('iphone6 phone','Apple Inc','phone'); SELECT* FROM mongo_data WHERE brand='Apple Inc'; _id | name | brand | type --------------------------+----------------+-----------+------- 53ea4f59fe5586a15714881d | iphone6 phone | Apple Inc | phone UPDATE mongo_data SET brand='Apple Product' WHERE brand='Apple Inc’; SELECT * FROM mongo_data WHERE brand='Apple Product’; _id | name | brand | type --------------------------+----------------+---------------+------- 53ea4f59fe5586a15714881d | iphone6 phone | Apple Product | phone MongoDB FDW Example
  33. 33. © 2014 EnterpriseDB Corporation. All rights reserved. 33 • Structures and standards emerge! • Data has references (products link to catalogues; products have bills of material; components appear in multiple products; storage locations link to ISO country tables) • When the database has duplicate data entries, then the application has to manage updates in multiple places – what happens when there is no ACID transactional model? “No SQL Only” or “Not Only SQL”?
  34. 34. © 2014 EnterpriseDB Corporation. All rights reserved. 34 • Postgres has many NoSQL features without the drawbacks: − Schema-less data combined with structured data − High performance with predictable transaction model − Durable by default, but configurable per-table or per-transaction − Standards based with very low technology risk − Foreign Data Wrappers (FDW) for co-existence − Highly available skill set Postgres: The Best of Both Worlds
  35. 35. © 2014 EnterpriseDB Corporation. All rights reserved. 35 • Postgres is Not Only SQL (NoSQL is No SQL only) • Fully ACID compliant • Proven track record • Fully capable of handling the variety, velocity and volume requirements of most applications • Tackle NoSQL projects without leaving the capabilities of the relational model behind you • Combine Oracle compatibility, JSON and PostGIS to migrate applications onto more cost-effective platforms, make the app NoSQL capable and geo-location aware. Say ‘Yes’ to ‘Not Only SQL’
  36. 36. © 2014 EnterpriseDB Corporation. All rights reserved. 36
  37. 37. © 2014 EnterpriseDB Corporation. All rights reserved. 37
  • kjsphoto

    Sep. 25, 2015

This presentation reviews Postgres’ unsurpassed NoSQL capabilities, which include a very rich JSON data type, a key-value pair data type, and a SQL/MED-based virtual data integration capability to bridge between Postgres and many other databases. Marc Linster discusses NoSQL syntax, NoSQL performance, and SQL/NoSQL integration and implementation recommendations. To listen to the recording visit EnterpriseDB.com > Resources > Webcasts > On Demand Webcasts. If you have any questions, please email sales@enterprisedb.com

Views

Total views

1,415

On Slideshare

0

From embeds

0

Number of embeds

3

Actions

Downloads

47

Shares

0

Comments

0

Likes

1

×