PowerOfRelationshipsInBigData_SVNoSQL

719 views

Published on

In this security solution demo, we have integrated Oracle NoSQL DB with InfiniteGraph to demonstrate the power of using the right tools for the solution. By integrating the key value technology of Oracle with the InfiniteGraph distributed graph database, we are able to create new views of existing Call Detail Record (CDR) details to enable discovery of connections, paths and behaviors that may otherwise be missed.

Discover how to add value to your existing Big Data to increase revenues and performance!

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
719
On SlideShare
0
From Embeds
0
Number of Embeds
80
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

PowerOfRelationshipsInBigData_SVNoSQL

  1. 1. The Database The Power of Relationships in Big Data Leon Guzenda - Objectivity, Inc. Silicon Valley NoSQL Meetup - 1/23/14 © Objectivity, Inc. 2014
  2. 2. Overview • The Problem
 • Current Big Data Analytics
 • Relationship Analytics
 • Leveraging NoSQL
 • Big Data Connection Platform
 • Solution Use Case Demo © Objectivity, Inc. 2014 !2
  3. 3. Objectivity, Inc. • Headquartered in San Jose, CA • Over two decades of NoSQL and Big Data experience • Enables complex data virtualization and Big Data solutions for the enterprise • Software products: • Objectivity/DB • InfiniteGraph • InfiniteGraph Social App • Embedded in hundreds of enterprises, government organizations and products, with millions of deployments. © Objectivity, Inc. 2014 !3
  4. 4. A Typical Deployment © Objectivity, Inc. 2014 !4
  5. 5. Current Big Data Analytics © Objectivity, Inc. 2014 !5
  6. 6. The Problem Information Overload!
 • Making sense of it all takes time and $$$
 • Which lead to a rush to Big Data Analytics © Objectivity, Inc. 2014 !6
  7. 7. Current Big Data Analytics © Objectivity, Inc. 2014 !7
  8. 8. Leveraging NoSQL © Objectivity, Inc. 2014 !8
  9. 9. Not Only SQL - Four Main Technologies Simple © Objectivity, Inc. 2014 Highly Interconnected !9
  10. 10. Hadoop? Hadoop:
 Objectivity/DB & InfiniteGraph
 • Parallel processing using a divide and conquer or split and merge paradigm
 • Distributed processing with multithreading client processes and simple servers*
 • Sharded, distributed file system
 
 
 
 • Distributed, segmented Federated Database with a Single Logical View down to fine grain objects
 • Tuned for sequential scans and simple queries
 • Tuned for random access and powerful parallel queries
 • Not suitable for highly interconnected data sets (graphs) • Excel at handling very large graph structures with built-in relationship analytics * Process workflow could be driven using MapReduce © Objectivity, Inc. 2014 !10
  11. 11. Incremental Analytic Improvements Aren’t Enough • All current solutions use the same basic architectural model. • None of the current solutions has an efficient way to store connections between entities in different silos. • Most analytic technology focuses on the content of the data nodes, rather than the many kinds of connections between the nodes and in those connections. • Why? Because most DBMSs are bad at handling relationships. • Object and Graph Databases can efficiently store, manage and query the many kinds of relationships hidden in the data. © Objectivity, Inc. 2014 !11
  12. 12. Relationship Analytics… A SQL Shortcoming Table_A Table_B Table_C Table_D Table_E Table_F Table_G There are some kinds of complex relationship handling problems that SQL wasn't designed for. © Objectivity, Inc. 2014 !12
  13. 13. …Relationship Analytics A SQL Shortcoming Table_A Table_B Table_C Table_D Table_E Table_F Table_G InfiniteGraph - The solution can be found with a few lines of code A3 © Objectivity, Inc. 2014 G4 !13
  14. 14. Graph Terminology ● VERTEX: A single node in a graph data structure
 ● EDGE: A connection between a pair of VERTICES
 ● PROPERTIES: Data items that belong to a particular Vertex or Edge
 ● WEIGHT: A quantity associated with a particular Edge
 ● GRAPH: A network of linked Vertex and Edge objects
 
 Vertex 1 City: San Francisco
 Pop: 812,826 © Objectivity, Inc. 2014 Edge 1 Road: I-101
 Miles: 47.8 Vertex 2 City: San Jose
 Pop: 967,487 !14
  15. 15. Example 1 - Relationship Analytics MARKET ANALYSIS SOCIAL NETWORK ANALYSIS LOGISTICS HEALTHCARE INFORMATICS © Objectivity, Inc. 2014 !15
  16. 16. Finding The Links… Events/Places People/Orgs Situation X Combatant A A Called P A Seen Near X P Emailed S Situation Y Bank X P Called Q Q Seen Near T X Paid S Target T Civilian P P Called R R Seen Near T Cafe C Civilian Q A Banks at X S Seen Near T Facts Civilian R Civilian S © Objectivity, Inc. 2014 A Seen At Y A Eats At !16
  17. 17. …Finding The Links… EDGES VERTICES Events/Places People/Orgs Facts Situation X Combatant A A Called P A Seen Near X P Emailed S Situation Y Bank X P Called Q Q Seen Near T X Paid S Target T Civilian P P Called R R Seen Near T Cafe C Civilian Q A Banks at X S Seen Near T Civilian R Civilian S © Objectivity, Inc. 2014 A Seen At Y A Eats At !17
  18. 18. …Finding The Links… Situation X Seen Near Eats At Cafe C Combatant A Seen At Called Banks At Bank X Civilian P Called Civilian Q Called Emailed Civilian R Seen Near Seen Near Situation Y Paid Civilian S Seen Near Target T © Objectivity, Inc. 2014 !18
  19. 19. …Finding The Links… Situation X Seen Near Combatant A Seen At Called Situation Y Banks At SUSPECTS Bank X Civilian P Called Civilian Q Called Civilian R Seen Near Seen Near Target T © Objectivity, Inc. 2014 Emailed Paid Civilian S Seen Near NEEDS PROTECTION !19
  20. 20. …Finding The Links OTHER DATABASE(S) GRAPH DATABASE © Objectivity, Inc. 2014 !20
  21. 21. Example 2 - Finding Patterns in Open Source Data The Challenges ● Data Volumes
 ● Fast-Changing Data
 ● Sensitivity of Data
 ● Significance of Data © Objectivity, Inc. 2014 !21
  22. 22. Example 3 - Cybersecurity © Objectivity, Inc. 2014 !22
  23. 23. Big Data Connection Platform © Objectivity, Inc. 2014 !23
  24. 24. Objectivity’s Disruptive Big Data Architecture Uses Data Virtualization to hide the nodes and focus on the connections © Objectivity, Inc. 2014 !24
  25. 25. InfiniteGraph Distributed Parallel Load and Queries Powerful Graph Queries X Start Start X Computational and Visualization Plugins Distributed Parallel Link Finding Latency Exceeded Start Start © Objectivity, Inc. 2014 Finish Custom Visualizer !25
  26. 26. Solution Use Case Demo… Let’s see InfiniteGraph coupled with Oracle’s NoSQL Solution… © Objectivity, Inc. 2014 !26

×