NoSQL on ACID - Meet Unstructured Postgres


Published on

The proliferation of data from new data sources has generated greater demand for technologies that can handle and harvest value from unstructured data. Postgres is leading the movement of integrating unstructured data with the relational environment.

Postgres first added JSON and then enhanced it with new data types, functions and operators in recent releases. Now in beta is the JSONB “binary JSON” type. These advances follow the longstanding HStore data type added in 2006 to support key/value stores in Postgres. Now Postgres users can learn how to harness these capabilities to master unstructured data challenges with Postgres.

The presentation also covers:

* An overview of JSON data types and operators
* Examples of SELECT, UPDATE, etc
* An examination of performance considerations

For more information, please email

Published in: Technology
1 Comment
  • This finally convinced me that for agile development NoSQL/Json is usefull for avoiding DBAs to be a constraint for developper. But I see this usefull only during the initial development phase. When data structures emerge, then one should seriously consider to model them using pure relational modeling. Just because flexibility has allready been proven as being incompatible with scalability (think of EAV model which while being very flexible absolutly fails when it's about scaling on large volumes of data).

    Another thing : Acidity in the world of Big Data is also about acidity with distributed transactions. Because Big Data is also about scaling on multiple nodes. And acidity with distributed transactions means the more node you have the more latency you get. As far as i know except postgres XC (which is far from having proven it can scale despite of the increase of latency required to ensure acidity in the whole cluster) there's no solution to perform horizontal scalability in postgresql. But i may be wrong.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

NoSQL on ACID - Meet Unstructured Postgres

  1. 1. © 2014 EnterpriseDB Corporation. All rights reserved. 1 NoSQL on ACID - Meet Unstructured Postgres Marc Linster – SVP, Products and Services
  2. 2. © 2014 EnterpriseDB Corporation. All rights reserved. 2 •  Why NoSQL: Use Cases and Business Drivers •  Postgres’ response: JSON and HSTORE −  Examples −  More Examples −  Even More Examples •  Performance Evaluation •  No SQL Only or Not Only SQL? •  Next Steps Objectives
  3. 3. © 2014 EnterpriseDB Corporation. All rights reserved. 3 •  Where did NoSQL come from? −  Where all cool tech stuff comes from – Internet companies •  Why did they make NoSQL? −  To support huge data volumes and evolving demands for ways to work with new data types •  What does NoSQL accomplish? −  Enables you to work with new data types: email, mobile interactions, machine data, social connections −  Enables you to work in new ways: incremental development and continuous release •  Why did they have to build something new? −  There were limitations to most relational databases Let’s Ask Ourselves, Why NoSQL?
  4. 4. © 2014 EnterpriseDB Corporation. All rights reserved. 4 NoSQL: Real-world Applications •  Emergency Management System −  High variability among data sources required high schema flexibility •  Massively Open Online Course −  Massive read scalability, content integration, low latency •  Patient Data and Prescription Records −  Efficient write scalability •  Social Marketing Analytics −  Map reduce analytical approaches Source: Gartner, A Tour of NoSQL in 8 Use Cases, by Nick Heudecker and Merv Adrian, February 28, 2014
  5. 5. © 2014 EnterpriseDB Corporation. All rights reserved. 5 •  Traditional relational databases −  Don’t offer enough flexibility −  Every change in the table requires DBA support −  Data models evolve over time −  Incremental iterative development does not allow for upfront design and extensive data normalization −  New data types (documents, key value pairs, graphs) are not available •  Many projects are −  Developer driven −  Use cases and data access patterns emerge as the application matures Why not use SQL?
  6. 6. © 2014 EnterpriseDB Corporation. All rights reserved. 6 •  HSTORE −  Key-value pair −  Simple, fast and easy −  Postgres v 8.2 – pre-dates many NoSQL-only solutions −  Ideal for flat data structures that are sparsely populated •  JSON −  Hierarchical document model −  Introduced in Postgres 9.2, perfected in 9.3 •  JSONB −  Binary version of JSON −  Faster, more operators and even more robust −  Postgres 9.4 Postgres’ Response
  7. 7. © 2014 EnterpriseDB Corporation. All rights reserved. 7 •  Supported since 2006, the HStore contrib module enables storing key/ value pairs within a single column •  Allows you to create a schema-less, ACID compliant data store within Postgres Postgres: Key-value Store •  Combines flexibility with ACID compliance •  Create single HStore column and include, for each row, only those keys which pertain to the record •  Add attributes to a table and query without advance planning
  8. 8. © 2014 EnterpriseDB Corporation. All rights reserved. 8 •  Create a table with HSTORE field CREATE TABLE hstore_data (data HSTORE);! •  Insert a record into hstore_data INSERT INTO hstore_data (data) VALUES (’! ! !"cost"=>"500", ! ! !"product"=>"iphone", ! ! !"provider"=>"apple"');! •  Select data from hstore_data SELECT data FROM hstore_data ; ------------------------------------------! "cost"=>"500”,"product"=>"iphone”,"provider"=>"Apple"! (1 row) ! HSTORE Examples
  9. 9. © 2014 EnterpriseDB Corporation. All rights reserved. 9 •  Convert HSTORE data to JSON SELECT hstore_to_json(data) FROM hstore_data ;! hstore_to_json ! ------------------------------------------------ {"cost": "500", "product": "iphone", "provider": "Apple"}! SELECT hstore_to_json(data)->>'cost' as price ! FROM hstore_data ;! price ! -------! 500! HSTORE Examples
  10. 10. © 2014 EnterpriseDB Corporation. All rights reserved. 10 •  JSON is the most popular data-interchange format on the web •  Supported by virtually every programming language •  New supporting technologies continue to expand JSON’s utility −  PL/V8 JavaScript extension −  PL/Coffee V8 extension −  Node.js •  Postgres has a native JSON data type (v9.2) and a JSON parser and a variety of JSON functions (v9.3) •  Postgres will have a JSONB data type with binary storage and indexing (coming – v9.4) Postgres: Document Store
  11. 11. © 2014 EnterpriseDB Corporation. All rights reserved. 11 •  Creating a table with a JSONB field CREATE TABLE json_data (data JSONB);! •  Simple JSON data element: {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1}! •  Inserting this data element into the table json_data INSERT INTO json_data (data) VALUES ! !(’ { !"name": "Apple Phone", ! ! !"type": "phone", ! ! !"brand": "ACME", ! ! !"price": 200, ! ! !"available": true, ! ! !"warranty_years": 1 ! !! !} ')! JSON Examples
  12. 12. © 2014 EnterpriseDB Corporation. All rights reserved. 12 •  JSON data element with nesting: {“full name”: “John Joseph Carl Salinger”,! “names”: ! ![! !{"type": "firstname", “value”: ”John”},! !{“type”: “middlename”, “value”: “Joseph”},! !{“type”: “middlename”, “value”: “Carl”},! !{“type”: “lastname”, “value”: “Salinger”} !! !]! }! JSON Examples
  13. 13. © 2014 EnterpriseDB Corporation. All rights reserved. 13 SELECT DISTINCT ! !data->>'name' as products ! FROM json_data;
 ! products ! ------------------------------! Cable TV Basic Service Package! AC3 Case Black! Phone Service Basic Plan! AC3 Phone! AC3 Case Green! Phone Service Family Plan! AC3 Case Red! AC7 Phone! A simple query for JSON data This query does not return JSON data – it returns text values associated with the key ‘name’
  14. 14. © 2014 EnterpriseDB Corporation. All rights reserved. 14 SELECT data FROM json_data;! data ! ------------------------------------------! {"name": "Apple Phone", "type": "phone", "brand": "ACME", "price": 200, "available": true, "warranty_years": 1}! A query that returns JSON data This query returns the JSON data in its original format
  15. 15. © 2014 EnterpriseDB Corporation. All rights reserved. 15 •  BSON – stands for ‘Binary JSON’ •  BSON != JSONB −  BSON cannot represent an integer or floating-point number with more than 64 bits of precision. −  JSONB can represent arbitrary JSON values. •  Caveat Emptor! −  This limitation will not be obvious during early stages of a project! JSONB and BSON
  16. 16. © 2014 EnterpriseDB Corporation. All rights reserved. 16 •  JSON is naturally integrated with ANSI SQL in Postgres •  JSON and SQL queries use the same language, the same planner, and the same ACID compliant transaction framework •  JSON and HSTORE are elegant and easy to use extensions of the underlying object-relational model JSON and ANSI SQL - PB&J for the DBA
  17. 17. © 2014 EnterpriseDB Corporation. All rights reserved. 17 SELECT DISTINCT product_type, data->>'brand' as Brand, data->>'available' as Availability FROM json_data JOIN products ON (>>'name') WHERE>>'available'=true; product_type | brand | availability ---------------------------+-----------+-------------- AC3 Phone | ACME | true JSON and ANSI SQL Example ANSI SQL JSON No need for programmatic logic to combine SQL and NoSQL in the application – Postgres does it all
  18. 18. © 2014 EnterpriseDB Corporation. All rights reserved. 18 Bridging between SQL and JSON Simple ANSI SQL Table Definition CREATE TABLE products (id integer, product_name text );! Select query returning standard data set SELECT * FROM products;
 ! id | product_name ! ----+--------------! 1 | iPhone! 2 | Samsung! 3 | Nokia! Select query returning the same result as a JSON data set SELECT ROW_TO_JSON(products) FROM products; {"id":1,"product_name":"iPhone"} {"id":2,"product_name":"Samsung"} {"id":3,"product_name":"Nokia”}
  19. 19. © 2014 EnterpriseDB Corporation. All rights reserved. 19 •  Goal −  Help our customers understand when to chose Postgres and when to chose a specialty solution −  Help us understand where the NoSQL limits of Postgres are •  Setup −  Compare Postgres 9.4 to Mongo 2.6 −  Single instance setup on AWS M3.2XLARGE (32GB) •  Test Focus −  Data ingestion (bulk and individual) −  Data retrieval JSON Performance Evaluation Load Generator Postgres 9.4 MongoDB 2.6
  20. 20. © 2014 EnterpriseDB Corporation. All rights reserved. 20 Performance Evaluation Generate 10 Million JSON Documents Load into MongoDB 2.6 (IMPORT) Load into Postgres 9.4 (COPY) 10 Million individual INSERT commands 10 Million individual INSERT commands Multiple SELECT statements Multiple SELECT statements T1 T2 T3
  21. 21. © 2014 EnterpriseDB Corporation. All rights reserved. 21 Performance Evaluations - Results MongoDB 2.6   Postgres 9.4   Data load (s)   2,624   1,193   Inserts (s)   16,491   5,637   Selects (s)   187   67   DB Size (GB)   20.05   14.89  
  22. 22. © 2014 EnterpriseDB Corporation. All rights reserved. 22 •  Initial tests confirm that Postgres’ can handle many NoSQL workloads •  EDB is making the test scripts publically available •  EDB encourages community participation to better define where Postgres should be used and where specialty solutions are appropriate •  Download the source at •  Join us to discuss the findings at Performance Evaluations – Next Steps
  23. 23. © 2014 EnterpriseDB Corporation. All rights reserved. 23 •  Postgres has many NoSQL features without the drawbacks: −  Schema-less data types, with sophisticated indexing support −  Schema changes with rapid additional and removal of columns −  Durable by default, but controllable per-table or per-transaction •  Postgres was originally architected to be an object- relational database −  Extensible (e.g. PostGIS extension for geospatial) −  Supports objects and classes in the schemas and query language −  Supports custom data types and methods •  Postgres: The Best of Both Worlds
  24. 24. © 2014 EnterpriseDB Corporation. All rights reserved. 24 •  Structures and standards emerge! •  Data has references (products link to catalogues; products have bills of material; components appear in multiple products; storage locations link to ISO country tables) •  When the database has duplicate data entries, then the application has to manage updates in multiple places – what happens when there is no ACID transactional model? “No SQL Only” or “Not Only SQL”?
  25. 25. © 2014 EnterpriseDB Corporation. All rights reserved. 25 •  Postgres is Not Only SQL (NoSQL is No SQL) •  Fully ACID compliant •  Proven track record •  Fully capable of handling the variety, velocity and volume requirements of most applications •  Tackle NoSQL projects without leaving the capabilities of the relational model behind you Postgres is the best of both worlds, and it’s open source! Say ‘Yes’ to ‘Not Only SQL’
  26. 26. © 2014 EnterpriseDB Corporation. All rights reserved. 26 •  More evaluation scenarios −  EDB will continue publishing evaluation scenarios for new workloads −  EDB encourages participation, discussion and debate •  More examples −  EDB will continue publishing code samples to facilitate adoption of Postgres as the NoSQL Solution for Enterprise •  More webinars −  Keep your eyes open for the series of NoSQL-focused webinars throughout the summer •  Review the Postgres NoSQL resources − − What’s Next?
  27. 27. © 2014 EnterpriseDB Corporation. All rights reserved. 27 POSTGRES innovation ENTERPRISE reliability 24/7 support Services & training Enterprise-class features & tools Indemnification Product road-map Control Thousands of developers Fast development cycles Low cost No vendor lock-in Advanced features Enabling commercial adoption of Postgres
  28. 28. © 2014 EnterpriseDB Corporation. All rights reserved. 28