• Share
  • Email
  • Embed
  • Like
  • Private Content
Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.
 

Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data.

on

  • 944 views

On August 17, 2011, the InfiniteGraph team hosted a local Meetup attended by dozens of senior developers working on large scale enterprise and startup projects. Big Data problems are quickly ...

On August 17, 2011, the InfiniteGraph team hosted a local Meetup attended by dozens of senior developers working on large scale enterprise and startup projects. Big Data problems are quickly presenting themselves in almost every area of computing from Social Network Analysis to File Processing. Many technologies, such as those in the NoSQL space were developed in response to the limitations of current storage systems as an effective mechanism to deal with these mountains of data. And much of that data is interconnected in ways that, when organized properly, gives interesting and often valuable information. InfiniteGraph was designed specifically to traverse complex relationships in big data, and provide the framework for products built to provide real-time network analysis, business decision support and relationship analytics. Speakers: Thomas Krafft, Director of Marketing, InfiniteGraph. Darren Wood, Chief Architect, InfiniteGraph. Mark Maagdenberg, Senior Field Engineer, InfiniteGraph.

Statistics

Views

Total Views
944
Views on SlideShare
943
Embed Views
1

Actions

Likes
1
Downloads
13
Comments
0

1 Embed 1

http://blog.infinitegraph.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data. Meetup: An Introduction to InfiniteGraph, and Connecting the Dots in Big Data. Presentation Transcript

    • Graph Database Overview and Feature Update Darren Wood Chief Architect, InfiniteGraph
    • History
      • Objectivity – Massively scalable, distributed object oriented database
        • Used in Government (DoD, Intelligence)
          • Machine generated data such as sensor, acoustic…
        • OEM Markets
          • Either complex data models, or high ingest or both
      • Significant technical advantage in highly connected (many-to-many) data models
      Copyright © InfiniteGraph
    • Graph Databases
      • Key technical attributes
      • How Infinite Graph addresses these
      • Query and navigation
      • Challenges/Requirements of Distribution
      • Practical applications
      Copyright © InfiniteGraph
    • Graph Databases
      • Optimized around data relationships
        • Relationships as first class citizens
        • Super fast traversal between entities
        • Rich/flexible annotation of connections
      • Small focused API (typically not SQL)
        • Natively work with concepts of Vertex/Edge
        • SQL has no concept of “navigation”
        • Most attempts based in SQL are convoluted
      Copyright © InfiniteGraph
    • Distributed Graph Must Haves
      • High performance distributed persistence
      • Ability to deal with remote data reads (fast)
      • Intelligent local cache of subgraphs
      • Distributed navigation processing
      • Distributed, multi-source concurrent ingest
      • Write modes supporting both strict and eventual consistency
      Copyright © InfiniteGraph
    • Some Code Copyright © InfiniteGraph Vertex alice = myGraph.addVertex(new Person(“Alice”)); Vertex bob = myGraph.addVertex(new Person(“Bob”)); Vertex carlos = myGraph.addVertex(new Person(“Carlos”)); Vertex charlie = myGraph.addVertex(new Person(“Charlie”)); alice.addEdge(new Meeting(“Denver”, “5-27-10”), bob); bob.addEdge(new Call(timestamp), carlos); carlos.addEdge(new Payment(100000.00), charlie); bob.addEdge(new Call(timestamp), charlie); Alice Carlos Charlie Bob Meets Calls Pays Calls
    • Physical Storage Comparison Copyright © InfiniteGraph Meetings P1 Place Time P2 Alice Denver 5-27-10 Bob Calls From Time Duration To Bob 13:20 25 Carlos Bob 17:10 15 Charlie Payments From Date Amount To Carlos 5-12-10 100000 Charlie Met 5-27-10 Alice Called 13:20 Bob Payed 100000 Carlos Charlie Called 17:10 Rows/Columns/Tables Relationship/Graph Optimized
    • Query and Navigation
      • Queries – but not as you know them
      • More like a rules based search and discovery
      • Asynchronous Results
      Copyright © InfiniteGraph Alice Carlos Charlie Bob Meets Calls Pays Calls “ Find all paths between Alice and Charlie” “ Find all paths between Alice and Charlie – within 2 degrees” “ Find all paths between Alice and Charlie – events in May 2010”
    • Navigation Example Copyright © InfiniteGraph // Create a qualifier that describes the target vertex Qualifier findCharliePredicate = new VertexPredicate(personType, "name == ’Charlie'" ); // Construct a navigator which starts with Alice and uses a result qualifier // to find all paths in the graph to Charlie Navigator charlieFinder = alice.navigate( Guide.SIMPLE_BREADTH_FIRST, // default guide Qualifier.ANY, // no path constraints findCharliePredicate , // find paths ending with Charlie myResultHandler); // fire results to supplied handler // Start the navigator charlieFinder.start();
    • Management of Large Data Graphs
      • Graphs grow quickly
        • Billions of phone calls / day in US
        • Emails, social media events, IP Traffic
        • Financial transactions
      • Some analytics require navigation of large sections of the graph
      • Each step (often) depends on the last
      • Must distribute data and go parallel
      Copyright © InfiniteGraph
    • Basic Architecture Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Blueprints User Apps Objectivity/DB Distributed Database Session / TX Management Placement
    • Feature Update Copyright © InfiniteGraph 2.0
    • Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement Standard Blocking Ingest/Placement (MDP Plugin) Objectivity/DB App-1 (Ingest V 1 ) App-2 (Ingest V 2 ) App-3 (Ingest V 3 ) V 1 V 2 V 3 App-1 (E 1 2 { V 1 V 2 }) App-2 (E 23 { V 2 V 3 }) App-3 E 12 E 23
    • Accelerated Ingest Copyright © InfiniteGraph IG Core/API Configuration Navigation Execution Management Extensions Session / TX Management Placement (Standard) Placement (Accelerated) V 1 V 2 V 3 E 12 E 23 Distributed Pipelines Staging Containers Pipeline Containers E(1->2) E(3->1) E(2->3) E(2->1) E(2->3) E(3->1) E(1->2) E(3->2) E(1->2) E(2->3) E(3->1) E(2->1) E(2->3) E(3->1) E(3->2) E(1->2)
    • InfiniteGraph Visualizer
      • Really nice flexible graph viewer
      • Browser style navigation and history
      • Full index support – search your data
      • Display connections around a selected point
      • Fully customize display to your data model
      • Full data view via selection
      Copyright © InfiniteGraph
    • InfiniteGraph Visualizer Copyright © InfiniteGraph
    • InfiniteGraph Visualizer Copyright © InfiniteGraph
    • Indexing Framework
      • Focused on providing choice !
      • Manual Indexes for grouping data
      • Automatic Indexes for cross population
      • Query interface with qualification language
      • Pluggable query operators
      • External index support (Lucene)
      Copyright © InfiniteGraph
      • Automated Distributed Navigation
      • Stored Loadable Navigators
      • Visualizer Navigation Plugins
      • More Visualizer Enhancements
      • More Import/Export support
      Copyright © InfiniteGraph >> next
    • Thankyou ! Copyright © InfiniteGraph [email_address]