New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Carl Olofson


Published on

Presentation slides by Carl Olofso, Research Vice President, Database Management and Data Integration Software for IDC (International Data Corporation).

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Carl Olofson

  1. 1. Graph Computing and Relationship Discovery in the Enterprise<br />Carl Olofson<br />Research Vice President<br />IDC<br />
  2. 2. Agenda<br />The New Generation of DBMS<br />Overview of the movement toward a new generation of DBMS.<br />New demands of emerging Cloud-based applications.<br />“NoSQL” technologies use cases<br />Graph databases<br />What they are, and the role they play in social media and various analytical functions.<br />List of requirements for a graph database.<br />Limitations of RDBMS for handling graph database workloads.<br />How an object-oriented DBMS is well suited to use in deploying a graph database.<br />Conclusions and Recommendations<br />2<br />Source:/Notes:<br />
  3. 3. A New Generation of DBMS<br />Why a new generation of DBMS?<br />DBMS users are no longer insisting that one DBMS can handle all workloads.<br />New Cloud-based workloads demand different database architectures and technologies.<br />Requirements vary:<br />Large, informal data collections for ad hoc analysis<br />Large, complex data collections for ongoing analysis<br />Data sharing for clusters of application servers<br />Very large databases (VLDB)<br />Extreme transaction throughput<br />Rapid execution of complex relational database queries.<br />3<br />
  4. 4. Third Generation DBMS Technologies<br />Relational<br />Columnar<br />Cellular (or modular)<br />In-memory RDBMS<br />Non-Relational<br />Key-value pair<br />List-oriented (Hadoop oriented)<br />Graph<br />4<br />
  5. 5. DBMS Types<br />5<br />Graph<br />Database<br />Object<br />Database<br />Multi-value (Pick style)<br />Database<br />Data Complexity<br />HadoopDatabase<br />Relational<br />Database<br />Network<br />Database<br />Key-Value Pair<br />Database<br />Hierarchical<br />Database<br />Inverted List<br />Database<br />Model Complexity<br />
  6. 6. Web Apps<br />Pure OO Apps<br />OLTP & DW MF & Dist. Apps<br />Graph<br />Database<br />Object<br />Database<br />Multi-value (Pick style)<br />Database<br />MF Transactions, Complex Schema<br />DBMS Types and Workloads<br />6<br />Data Complexity<br />HadoopDatabase<br />Relational<br />Database<br />Network<br />Database<br />High Throughput Mainframe Transactions<br />Key-Value Pair<br />Database<br />Hierarchical<br />Database<br />Inverted List<br />Database<br />Model Complexity<br />
  7. 7. Most Flexible Relationship Management<br />7<br />Graph<br />Database<br />Object<br />Database<br />Multi-value (Pick style)<br />Database<br />Data Complexity<br />HadoopDatabase<br />Relational<br />Database<br />Network<br />Database<br />Key-Value Pair<br />Database<br />Hierarchical<br />Database<br />Inverted List<br />Database<br />Model Complexity<br />
  8. 8. DBMS Architectures and Workloads<br />Different kinds of DBMS being employed to take on workloads required by emerging Cloud-based applications.<br />Cloud demands virtualized, limitless data space<br />New kinds of data relationships introduced by website constructs and social media, not supported by conventional DBMS<br />Fluctuating user demand requires flexible resource scalability in terms of server capacity<br />Sometimes, data to be analyzed is dynamically gathered; users chafe at the requirement that they build a schema and map out storage space before they load it.<br />8<br />
  9. 9. The NoSQL Movement<br />9<br />
  10. 10. Graph Databases<br />A graph database is used to trace relationships among entities, most commonly people, to any depth. Its characteristics are:<br />Very simple, fixed schema<br />Very complex data relationships<br />Used to support complex associations among like entities.<br />10<br />Jeff Smith<br />Attribute(s)<br />Nancy Jones<br />Jim Smith<br />Edge<br />Node<br />Paul Jones<br />Jane Jones-Smith<br />Doris Smith<br />John Jones<br />Meta-Model<br />Instance Example (simplified)<br />
  11. 11. Requirements for a Graph DBMS<br />These are the core requirements of a graph DBMS:<br />Must be capable of supporting recursive relationships.<br />Must be scalable to arbitrary sizes with minimal administration.<br />Provides rapid means of traversing complex relationship structures.<br />Can rapidly search many objects based on their relationships and properties.<br />For applications such as law enforcement, must have ACID (atomicity, consistency, isolation, durability) properties.<br />11<br />
  12. 12. Limitations of Relational DBMS<br />Why RDBMS does not support graph databases.<br />The relational model is flat (2 dimensional).<br />A relational database cannot properly capture nested or recursive data.<br />Data based on multiple modes of relationship between entities<br />Data based on collections of entities of a type under another entity of the same type<br />Examples<br />Parts database and other “bill of materials” structures<br />Employee reporting hierarchy<br />Personal relationships captured in social media<br />12<br />Person-X-Ref<br /><ul><li>Knows-PID (PK)
  13. 13. Known-By-PID (PK)
  14. 14. Interpersonal-Detail</li></ul>Person<br /><ul><li>PID (PK)
  15. 15. Description
  16. 16. … </li></ul>Requires a separate SELECT for each level of query.<br />
  17. 17. Object-Oriented DBMS and Graph Databases<br />An object-oriented DBMS is well suited to use in deploying a graph database because it can handle the demands of very complex object relationships.<br />In order to support the various types of relationships demanded by object-oriented models, OODBMS supports<br />Container relationships<br />Recursion<br />Type hierarchies<br />The underlying building blocks needed to support these constructs also enable<br />Rapid relationship traversal and search<br />Efficient storage of databases of arbitrary size<br />13<br />
  18. 18. Conclusions and Recommendations<br />Conclusions<br />A new generation of DBMS is emerging, prompted by new problems posed by Web applications and scalability options inherent in faster, cheaper, more scalable hardware.<br />Users are more open to the idea of heterogeneous DBMS; choosing the right tool for the job.<br />Graph database is just such a technology, and object technology provides an excellent platform for its management.<br />Recommendations<br />Now is the time to put appropriate functionality above DBMS uniformity: keep an open mind regarding DBMS products.<br />RDBMS does not adequately address all data needs: choose the technology that’s right for the task at hand.<br />Graph databases offer relationship tracking and analysis capability that can’t be done by other means. Consider such technology as a solution for such problems in areas such as customer relationship management, social network management, supply chain management, etc.<br />14<br />