0
Neo4j
And TelCo goodness
                 Some usecases


                                 #neo4j
Peter Neubauer          ...
NOSQL data models
            Key-value stores
Data size




                          Bigtable clones


                 ...
The Neo4j model: Property Graph
 Core abstractions:
                                              name = “Emil”

   Nodes
...
Agenda
 The NoSQL landscape
 Neo4j intro
 Examples for TelCos
   CDR
   Routing
   Social graphs
   Master Data Management...
Neo4j?
 Most widely deployed graph db in the world
   ACID, persistent, embedded/server
   Robust: 24/7 production since 2...
Neo4j – Nodes, Relationships, Properties




 Nodes have different properties
   Matrix characters: People vs. Programs
 B...
Building a node space
GraphDatabaseService graphDb = ... // Get factory


// Create Thomas 'Neo' Anderson
Node mrAnderson ...
Building a node space
GraphDatabaseService graphDb = ... // Get factory
Transaction tx = graphdb.beginTx();

// Create Tho...
Code (2): Traversing a node space
// Instantiate a traverser that returns Mr Anderson's friends
Traverser friendsTraverser...
Ruby
gem install neo4j

require ”rubygems”
require 'neo4j'

class Person
  include Neo4j::NodeMixin
  property :name, :age...
Spatial and social data
                                                                        name = ...

              ...
Financial data – fraud detection
                                                                                name = .....
CDR analysis
                                                                               name = flat_business

        ...
Call Data Records (CDR)
  Forming a graph
  Location based
  Possible uses:
    Find clusters (better plans)
    Build soc...
Routing
Routing with Neo4j and A*
Routing
Connecting and Splitting


 Routing
Social graphs

 Recommendations
 Location based
 services
 Influencers
 Shortest path
Recommendations and big graphs
 Global heuristics
   Page rank
 Local recommendations
    Shortest paths
    Hammock funct...
Grammar based Random Walks




     Pic from gremlin.tinkerpop.com / Marko Rodriguez
Impact, Dependency Analysis
Master Data Management
Spatial
  Complex data
    Multiple indexing (domain, Spatial, temporal)
  Location entering many domains
  GIS going main...
Current challenges in Spatial
  Domain and Spatial interconnections
  Unstructured domain data
  Routing
  Topology handli...
Multiple indexes - GIS
OpenStreetMap
The OpenStreetMap dataset
 Wiki for Spatial info
 Freely available data
 Very unstructured, free tagging
   Points, Ways, ...
OpenStreetMap
OpenStreetMap
Web exposure of Spatial Data

 Davide Savazzi
 Geotools & GeoServer
 Routing
 uDig
GSoC 2010 - uDig
Neo4j dynamic layers

                         Geometry                     Layer1
                         Encoder
      ...
OpenStreetMap




Dynamic                 OSM
 Laye rs




     Inde x (RTre e )
Dynamic Layers
Network Topology analysis

 Analytics of network coverage and frequencies
   Cell towers
   Drive data
   Infrastructure
 ...
Neo4j and Android
  Small footprint
  Running almost unmodified
  Interesting scenarios
     Semantic homescreen
     Conn...
Connected devices



                                 Emil

Emils PC
                                                     ...
Interesting scenarios
  Semantic homescreen
  Connected devices
  Local caches
  Disconnected databases
How ego are you? (aka other impls?)
  Franz’ Alle groGraph (http://agraph.franz.com)
     Proprietary, Lisp, RDF-oriented ...
API References
 Wiki, Code, API references
   http://wiki.neo4j.org/content/Neo4j_Spatial
   http://github.com/neo4j/neo4j...
Questions?




             Image credit: lost again! Sorry :(
http://neotechnology.com
Upcoming SlideShare
Loading in...5
×

2010 09-neo4j-deutsche-telekom

8,275

Published on

In the Telecommunications sector, there are a lot of complex data sets and problems that are well suited for graph models and the use of graph databases like Neo4j.

This talk gives just some ideas on where Neo4j currently is used within the TelCo sector. If you recognize problem areas that you have, don't hesitate to contact me at peter at neotechnology dot com, we are eager to learn more and help!

Published in: Technology

Transcript of "2010 09-neo4j-deutsche-telekom"

  1. 1. Neo4j And TelCo goodness Some usecases #neo4j Peter Neubauer @peterneubauer peter@neotechnology.com Neo Technology
  2. 2. NOSQL data models Key-value stores Data size Bigtable clones Document databases Graph databases Data complexity
  3. 3. The Neo4j model: Property Graph Core abstractions: name = “Emil” Nodes age = 29 sex = “yes” Relationships between nodes Properties on both 1 2 type = KNOWS time = 4 years 3 type = car vendor = “SAAB” model = “95 Aero”
  4. 4. Agenda The NoSQL landscape Neo4j intro Examples for TelCos CDR Routing Social graphs Master Data Management Spatial and LBS Network topology analysis Neo4j and Android
  5. 5. Neo4j? Most widely deployed graph db in the world ACID, persistent, embedded/server Robust: 24/7 production since 2003 Mature: lots of production deployments Scalable: High Availability, Master failover Community: ecosystem of tools, bindings, frameworks Product: OSGi, Spatial, RDF, languages Available under AGPLv3 and as commercial product But the f rst one is free! For ALL use-cases. i Links http://neo4j.org http://lists.neo4j.org
  6. 6. Neo4j – Nodes, Relationships, Properties Nodes have different properties Matrix characters: People vs. Programs Build structure as you go Who loves Neo?
  7. 7. Building a node space GraphDatabaseService graphDb = ... // Get factory // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly
  8. 8. Building a node space GraphDatabaseService graphDb = ... // Get factory Transaction tx = graphdb.beginTx(); // Create Thomas 'Neo' Anderson Node mrAnderson = graphDb.createNode(); mrAnderson.setProperty( "name", "Thomas Anderson" ); mrAnderson.setProperty( "age", 29 ); // Create Morpheus Node morpheus = graphDb.createNode(); morpheus.setProperty( "name", "Morpheus" ); morpheus.setProperty( "rank", "Captain" ); morpheus.setProperty( "occupation", "Total bad ass" ); // Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS ); // ...create Trinity, Cypher, Agent Smith, Architect similarly tx.commit();
  9. 9. Code (2): Traversing a node space // Instantiate a traverser that returns Mr Anderson's friends Traverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING ); // Traverse the node space and print out the result System.out.println( "Mr Anderson's friends:" ); for ( Node friend : friendsTraverser ) { System.out.printf( "At depth %d => %s%n", friendsTraverser.currentPosition().getDepth(), friend.getProperty( "name" ) ); }
  10. 10. Ruby gem install neo4j require ”rubygems” require 'neo4j' class Person include Neo4j::NodeMixin property :name, :age, :occupation index :name has_n :friends end Neo4j::Transactoin.run do neo = Person.new :name=>'Neo', :age=>29 morpheus = Person.new :name=>'Morpheus', :occupation=>'badass' neo.friends << morpheus end neo.friends.each {|p|...}
  11. 11. Spatial and social data name = ... name = “The Tavern” lat = 1295238237 name = “Omni Hotel” long = 234823492 42 lat = 3492848 long = 283823423 length = 7 miles AD RO ROAD ROAD ROO 1 7 3 OAD RO 13 AD name = ... ROAD lat, long = ... name = “Swedland” lat = 23410349 length = 3 miles long = 2342348852 2 name = ...
  12. 12. Financial data – fraud detection name = ... name = “The Tavern” lat = 1295238237 long = 234823492 42 name = “Mr Godfather” AW karma = veeeery-low HDR cash = more-than-you amount = $1000 IT W OWNS TRANSFER WIT 1 7 3 HDR AW 13 S FE R DE name = “Emil” P OS cash = always-too-li'l IT TRAN title = “ATM @ Wall St” id = 230918484233 amount = $1000 cash_left = 384204 2 name = ...
  13. 13. CDR analysis name = flat_business name = “Bob” lat = 1295238237 long = 234823492 N 42 LA name = “Mr Godfather” karma = veeeery-low _ P cash = more-than-you time = 2min TA DA CALLED CALLED CAL 1 7 3 L ED CA 13 L name = “Emil” LE Y D cash = always-too-li'l FAMIL Name= customer_support time =13min 2 name = Alice
  14. 14. Call Data Records (CDR) Forming a graph Location based Possible uses: Find clusters (better plans) Build social connections Find influencers
  15. 15. Routing
  16. 16. Routing with Neo4j and A*
  17. 17. Routing
  18. 18. Connecting and Splitting Routing
  19. 19. Social graphs Recommendations Location based services Influencers Shortest path
  20. 20. Recommendations and big graphs Global heuristics Page rank Local recommendations Shortest paths Hammock functions Random walks Dijkstra, A*, Shooting star etc
  21. 21. Grammar based Random Walks Pic from gremlin.tinkerpop.com / Marko Rodriguez
  22. 22. Impact, Dependency Analysis
  23. 23. Master Data Management
  24. 24. Spatial Complex data Multiple indexing (domain, Spatial, temporal) Location entering many domains GIS going mainstream, topologies explode No good systems out there Proprietary stacks rule (ESRI, Oracle) Open Government Data Shapefiles suck.
  25. 25. Current challenges in Spatial Domain and Spatial interconnections Unstructured domain data Routing Topology handling No good OSS full GIS stack
  26. 26. Multiple indexes - GIS
  27. 27. OpenStreetMap
  28. 28. The OpenStreetMap dataset Wiki for Spatial info Freely available data Very unstructured, free tagging Points, Ways, Relations, Tags, Changesets Changes can be pushed back Used for other purposes Great coverage in interesting places (towns, disasters etc)
  29. 29. OpenStreetMap
  30. 30. OpenStreetMap
  31. 31. Web exposure of Spatial Data Davide Savazzi Geotools & GeoServer Routing uDig
  32. 32. GSoC 2010 - uDig
  33. 33. Neo4j dynamic layers Geometry Layer1 Encoder Dynamic Query Layer2 Dynamic Styles Layer3 Dynamic Meta-Inf Connected domain data Neo4j Spatial GIS and Spatial stacks
  34. 34. OpenStreetMap Dynamic OSM Laye rs Inde x (RTre e )
  35. 35. Dynamic Layers
  36. 36. Network Topology analysis Analytics of network coverage and frequencies Cell towers Drive data Infrastructure Analytics Spatial signal strength Antenna placement and azimuth Frequency planning Network differences over time Reporting and charting
  37. 37. Neo4j and Android Small footprint Running almost unmodified Interesting scenarios Semantic homescreen Connected devices Local caches Disconnected databases
  38. 38. Connected devices Emil Emils PC Marcus Leigh Johan “Link all pictures that any of my friends has taken of me into my PicturesOfMe folder and update this every hour”
  39. 39. Interesting scenarios Semantic homescreen Connected devices Local caches Disconnected databases
  40. 40. How ego are you? (aka other impls?) Franz’ Alle groGraph (http://agraph.franz.com) Proprietary, Lisp, RDF-oriented but real graphdb Sones graphDB (http://sones.com) .NET, Deutsche Telekom VC backed Twitter's Floc kDB (http://github.com/twitter/flockdb) Twitter's (graph) database for large and shallow graphs Google Pre ge l (http://bit.ly/dP9IP) We are oh-so-secret Objectivity's Infnite Graph (http://infinitegraph.com) New, closed OODB with Graph Layer on top
  41. 41. API References Wiki, Code, API references http://wiki.neo4j.org/content/Neo4j_Spatial http://github.com/neo4j/neo4j-spatial http://components.neo4j.org/neo4j-spatial Mailing list: neo4j@lists.neo4j.org http://neo4j.org/community/list/
  42. 42. Questions? Image credit: lost again! Sorry :(
  43. 43. http://neotechnology.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×