Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Neo4j and Python: Playing with graph data - PyCon India 2014 Talk.

5,163 views

Published on

Neo4J and Python: Playing with graph data.

This talk introduces the world of graphs, their utility and the efficient use of the Neo4J graph database for some super cool day to day applications with the help of py2neo.

Follow the development of the talk at http://www.sonalraj.com/pycon14.html

Check out the details at the PyCon India 2014 website: http://in.pycon.org/funnel/2014/252-neo4j-and-python-playing-with-graph-data

Published in: Technology
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Neo4j and Python: Playing with graph data - PyCon India 2014 Talk.

  1. 1. PyCon India 2014• • created by Sonal Raj • Neo4j and Python Playing with graph data Graph Everything Sonal Raj
  2. 2. PyCon India 2014• • created by Sonal Raj • The Plan for today Graphs and NOSQL Step One Neo4j and Cypher Step Two 4 Step Two Use Cases Py2neo and REST Step Two
  3. 3. PyCon India 2014• • created by Sonal Raj • Once upon a time..
  4. 4. PyCon India 2014• • created by Sonal Raj • Once upon a time.. • Relational databases ruled the earth . . • Data was stored in Tables, Rows and Columns • Connections using Primary keys, Foreign keys . . • That’s all that is relational about then  • No on-the-fly structural (schema) changes • Horrible for Interconnected data ( joins, really? )
  5. 5. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space
  6. 6. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space
  7. 7. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space
  8. 8. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space
  9. 9. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space RDBMS
  10. 10. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space Elastic scaling – Scale out, not up
  11. 11. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space Elastic scaling – Scale out, not up Big data, Transaction Friendly
  12. 12. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space Elastic scaling – Scale out, not up Big data, Transaction Friendly Economical, can run on commodity hardware
  13. 13. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space Elastic scaling – Scale out, not up Big data, Transaction Friendly Economical, can run on commodity hardware End of the DBA rule
  14. 14. PyCon India 2014• • created by Sonal Raj • In the NOSQL Space Elastic scaling – Scale out, not up Big data, Transaction Friendly Economical, can run on commodity hardware End of the DBA rule Flexible Data models
  15. 15. Graph Trivia
  16. 16. PyCon India 2014• • created by Sonal Raj • Where are Graphs . . .
  17. 17. PyCon India 2014• • created by Sonal Raj • Where are Graphs . . .
  18. 18. PyCon India 2014• • created by Sonal Raj • Where are Graphs . . .
  19. 19. PyCon India 2014• • created by Sonal Raj • Where are Graphs . . .
  20. 20. PyCon India 2014• • created by Sonal Raj • Some Graphs we overlook . .
  21. 21. PyCon India 2014• • created by Sonal Raj • Some Graphs we overlook . .
  22. 22. PyCon India 2014• • created by Sonal Raj • Some Graphs we overlook . .
  23. 23. PyCon India 2014• • created by Sonal Raj • Apart from that Fraud Analyses Investment securities & debt analysis Recommendation Engines Impact Analysis in networks
  24. 24. PyCon India 2014• • created by Sonal Raj • So, Why Graphs ? • Increasing Connectivity of Data • Increasing Semi-Structredness • Rising Complexity
  25. 25. PyCon India 2014• • created by Sonal Raj • So, Why Graphs ? • Increasing Connectivity of Data • Increasing Semi-Structredness • Rising Complexity Seven Bridges of Königsberg Leonhard Euler in 1735
  26. 26. PyCon India 2014• • created by Sonal Raj • Property Graphs
  27. 27. PyCon India 2014• • created by Sonal Raj • Property Graphs - Has nodes
  28. 28. PyCon India 2014• • created by Sonal Raj • Property Graphs - Has nodes - Has properties for each node
  29. 29. PyCon India 2014• • created by Sonal Raj • Property Graphs - Has nodes - Has properties for each node - Has Relationships
  30. 30. PyCon India 2014• • created by Sonal Raj • Property Graphs - Has nodes - Has properties for each node - Has Relationships - Has properties for each relationship
  31. 31. PyCon India 2014• • created by Sonal Raj • Building Blocks Nodes Relationships Labels Graph Database Properties
  32. 32. PyCon India 2014• • created by Sonal Raj • Data Models Native Graphs Inherently store data as nodes and relationships.
  33. 33. PyCon India 2014• • created by Sonal Raj • Data Models Native Graphs Inherently store data as nodes and relationships. The Other ones . . . Data stored in tables, joins and aggregates to simulate a graph
  34. 34. PyCon India 2014• • created by Sonal Raj • Data Models Native Graphs Inherently store data as nodes and relationships. The Other ones . . . Data stored in tables, joins and aggregates to simulate a graph
  35. 35. PyCon India 2014• • created by Sonal Raj • Why Neo4j ? Schema-less property graph
  36. 36. PyCon India 2014• • created by Sonal Raj • Why Neo4j ? Schema-less property graph Handles complex connected data efficiently
  37. 37. PyCon India 2014• • created by Sonal Raj • Why Neo4j ? Schema-less property graph Handles complex connected data efficiently Fully ACID Transactions
  38. 38. PyCon India 2014• • created by Sonal Raj • Why Neo4j ? Schema-less property graph Handles complex connected data efficiently Fully ACID Transactions Highly Scalable, High Availability Clusters
  39. 39. PyCon India 2014• • created by Sonal Raj • Why Neo4j ? Schema-less property graph Handles complex connected data efficiently Fully ACID Transactions Highly Scalable, High Availability Clusters REST API for servers. Can be embedded to applications on JVM.
  40. 40. PyCon India 2014• • created by Sonal Raj • Why Neo4j ? Schema-less property graph Handles complex connected data efficiently Fully ACID Transactions Highly Scalable, High Availability Clusters REST API for servers. Can be embedded to applications on JVM. Cypher – a declarative querying solution Graph DB with good native python bindings . .
  41. 41. PyCon India 2014• • created by Sonal Raj • Cypher in action • Highly expressive query language • Cares about ‘what’ rather than ‘how’ to retrieve from the graph. • Uses pattern matching expressions. 1 2 (1) – [ :label ] - (2) label
  42. 42. PyCon India 2014• • created by Sonal Raj • Cypher in action • Highly expressive query language • Cares about ‘what’ rather than ‘how’ to retrieve from the graph. • Uses pattern matching expressions. 1 2 START n=(1), m=(2) MATCH n – [r:label] – m RETURN r label
  43. 43. PyCon India 2014• • created by Sonal Raj • Cypher in action • Highly expressive query language • Cares about ‘what’ rather than ‘how’ to retrieve from the graph. • Uses pattern matching expressions. • To make life easy for some, it is inspired by SQL. 1 2 START n=(1), m=(2) MATCH n – [r:label] – m RETURN r label
  44. 44. PyCon India 2014• • created by Sonal Raj • Cypher in action Create Read CREATE (n:Person { name : ‘Chuck Norris', title : ‘Analyst' }) RETURN n MATCH (a:Person),(b:Person) WHERE a.name = ‘Chuck' AND b.name = ‘Rajani' CREATE (a)-[r:RELTYPE { name : ‘cannot_find’ }]->(b) RETURN r MATCH (n) RETURN n #everything is returned MATCH (n:Label) RETURN n #all with specific label MATCH (Titanic { title:‘Titanic' })<-[:ACTED_IN|:DIRECTED]-(person) RETURN person
  45. 45. PyCon India 2014• • created by Sonal Raj • Cypher in action Update Delete MATCH (n { name: 'Andres' }) SET n.surname = 'Taylor' RETURN n MATCH (peter { name: 'Peter' }) SET peter += { hungry: TRUE , position: 'Entrepreneur' } MATCH (n { name: 'Peter' }) REMOVE n.title REMOVE n:German RETURN n SET n.name = NULL
  46. 46. PyCon India 2014• • created by Sonal Raj • REST in peace !! Create POST http://localhost:7474/db/data/node { "foo" : "bar" } POST http://localhost:7474/db/data/node/1/relationships { "to" : "http://localhost:7474/db/data/node/10", "type" : "LOVES", "data" : { "foo" : "bar" } } POST http://localhost:7474/db/data/schema/index/person { "property_keys" : [ "name" ] }
  47. 47. PyCon India 2014• • created by Sonal Raj • REST in peace !! Read Update Delete GET http://localhost:7474/db/data/node/144 GET http://localhost:7474/db/data/relationship/65 GET http://localhost:7474/db/data/relationship/61/properties GET http://localhost:7474/db/data/schema/index/user PUT http://localhost:7474/db/data/relationship/66/properties { "happy" : false } PUT http://localhost:7474/db/data/relationship/60/properties/cost "deadly" DELETE http://localhost:7474/db/data/node/308 DELETE http://localhost:7474/db/data/relationship/58 DELETE http://localhost:7474/db/data/schema/index/SomeLabel/name
  48. 48. Beauty of py2neo
  49. 49. PyCon India 2014• • created by Sonal Raj • For the pythonistas As simple as that! from py2neo import neo4j graph_db = neo4j.GraphDatabaseService("http://localhost:7474/db/data/") from py2neo import node, rel die_hard = graph_db.create( node(name="Bruce Willis"), node(name="John McClane"), node(name="Alan Rickman"), node(name="Hans Gruber"), node(name="Nakatomi Plaza"), rel(0, "PLAYS", 1), rel(2, "PLAYS", 3), rel(1, "VISITS", 4), rel(3, "STEALS_FROM", 4), rel(1, "KILLS", 3), )
  50. 50. PyCon India 2014• • created by Sonal Raj • For the pythonistas graphdb • clear() • create(*abstracts) • delete(*entities) • delete_index(content_type, index_name) • find(label, property_key=None, property_value=None) • get_index(content_type, index_name) • get_indexed_node(index_name, key, value) • ...
  51. 51. PyCon India 2014• • created by Sonal Raj • For the pythonistas • get_indexed_relationship(index_name, key, value) • get_properties(*entities) • match(start_node=None, rel_type=None, end_node=None, bidirectional=False, limit=None) • match_one(start_node=None, rel_type=None, end_node=None, bidirectional=False) • node(id_) • get_or_create_index(content_type, index_name, config=None) • get_or_create_indexed_node(index_name, key, value, properties=None) graphdb
  52. 52. PyCon India 2014• • created by Sonal Raj • Complexity Handling “ A graph database without traversals is just a persistent graph ”
  53. 53. PyCon India 2014• • created by Sonal Raj • Paths with py2neo #Create Paths from py2neo import neo4j, node a, b, c = node(name="Alice"), node(name="Bob"), node(name="Carol") abc = neo4j.Path(a, ’KNOWS’, b, ’KNOWS’, c) d, e = node(name=“Doctor”), node(name=“Easter”) de = neo4j.Path(d, ‘KNOWS’, e) #Join paths abcde = neo4j.Path.join(abc, ‘KNOWS’, de) #commit to the db abcde.get_or_create(graph_db)
  54. 54. PyCon India 2014• • created by Sonal Raj • Schema, Indices with py2neo #The class py2neo.neo4j.Schema py2neo.neo4j.Index #Join paths create_index(label, property_key) drop_index(label, property_key) get_indexed_property_keys(label) add_if_none(key, value, entity) #Apache Lucene Query people = graph_db.get_or_create_index(neo4j.Node, "People") s_people = people.query("family_name:S*")
  55. 55. neo4j.Node
  56. 56. neo4j.Relationship
  57. 57. PyCon India 2014• • created by Sonal Raj • Cypher with py2neo #Create transaction object from py2neo import cypher Session = cypher.Session(“http://localhost:7474/”) tx = session.create_transaction() #Add transactions, execute or commit tx.append(“some cypher query”) tx.append(“some cypher query”) tx.execute() tx.append(“some cypher query”) tx.commit() #The classical way from py2neo import neo4j graph_db = neo4j.GraphDatabaseSercice() query = neo4j.CypherQuery(graph_db, ‘your cypher query’) query.execute() #query.stream()
  58. 58. PyCon India 2014• • created by Sonal Raj • Command Line neotool #Syntax of operation neotool [<option>] <command> <args> Or python –m py2neo.tool .. #Some serious examples neotool clear neotool cypher "start n=node(1) return n, n.name?“ neotool cypher-csv "start n=node(1) return n.name, n.age?" neotool cypher-tsv "start n=node(1) return n.name, n.age?" #Guess what, you can also access the shell neotool shell
  59. 59. PyCon India 2014• • created by Sonal Raj • Neo4j level 2 • Batch Inserter • High Availability • Built-in online backup tools • HTTPS support
  60. 60. PyCon India 2014• • created by Sonal Raj • Neo4j level 2 • Batch Inserter • High Availability • Built-in online backup tools • HTTPS support Neo4J Framework. • GraphUnit, for unit testing neo4j • Libraries for performance and API testing • Batch Transaction tools • Transaction Event tools • Some other utilities . .
  61. 61. Use Cases
  62. 62. PyCon India 2014• • created by Sonal Raj • Recommendation Engines Complex pattern matching
  63. 63. PyCon India 2014• • created by Sonal Raj • Social Network Data Many entities, highly interconnected
  64. 64. PyCon India 2014• • created by Sonal Raj • Map Data Traversals and routing
  65. 65. PyCon India 2014• • created by Sonal Raj • Python Family Relatives for Neo4j Neo4
  66. 66. • •PyCon India 2014 Thank YouNeo appreciates your patience. • Sonal Raj • http://www.sonalraj.com/ • http://github.com/sonal-raj/ • sonal@enfoss.org

×