Start Flying with Python & Apache TinkerPop

Start Flying with Python &
Apache TinkerPop
Graph Day SF
Jason Plurad • pluradj@us.ibm.com • @pluradj
IBM Open Technology • Apache TinkerPop • JanusGraph
Saturday, June 17, 2017
©2017 IBM Corporation 19 June 20172
@pluradj
Open source • Graph computing
Dev @IBM • PMC @ApacheTinkerPop • TSC @JanusGraph
Food truck chaser
Jason Plurad
©2017 IBM Corporation 19 June 20173
Agenda
Apache TinkerPop
Gremlin + Python
- Jupyter
- Flask
- IBM Cloud
House of Mirrors: Ketrina Yim @KetrinaYim
©2017 IBM Corporation 19 June 20174
Founded 2009 • Marko Rodriguez @twarko
Apache Top-Level Project 2016 • PMC Chair: Stephen Mallette @spmallette
Committers: 16 • Contributors: 67 • Graph Systems: 20
https://tinkerpop.apache.org
©2017 IBM Corporation 19 June 20175
Apache TinkerPop: Graph Computing Framework
§ Defines graph structure and process
§ Core framework is written in Java
§ Provider implementations
– Azure Cosmos DB
– Blazegraph
– DataStax Enterprise Graph
– GRAKN.AI
– HGraphDB
– IBM Graph
– JanusGraph
– Neo4j
– OrientDB
– Sqlg
– Stardog
©2017 IBM Corporation 19 June 20176
Gremlin Traversal Language
sql2gremlin: Daniel Kuppitz @dkuppitz
Recursively show all employees and their supervisors
©2017 IBM Corporation 19 June 20177
Graph Application Architecture
Gremlin Server Graph DatabaseApplication
©2017 IBM Corporation 19 June 20178
Gremlin Language Drivers
gremlin-javascript: Jean-Baptiste Musso @jbmusso
Gremlin Bindings
©2017 IBM Corporation 19 June 20179
Gremlin Language Variants (GLV)
§ Elevate Gremlin to a top-level citizen in the programming language of choice
§ GLV can work with any modern language
– Function composition
– Function nesting
§ Java and Groovy (native)
§ Python is the first non-JVM GLV
§ Others are coming soon (JavaScript, C#)
§ SPARQL-Gremlin and SQL-Gremlin
http://tinkerpop.apache.org/docs/current/tutorials/gremlin-language-variants
©2017 IBM Corporation 19 June 201710
Gremlin Bytecode
http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine
[ ['V'], ['range', 0, 5],
['out', 'created'],
['values', 'name'] ]
g.V().range(0, 5).
out('created').
values('name')
©2017 IBM Corporation 19 June 201711
©2017 IBM Corporation 19 June 201712
Programming Language Popularity
TIOBE Index
PYPL Popularity RedMonk Rankings
©2017 IBM Corporation 19 June 201713
Example Python Application Architecture
Graph DatabaseClients
Web browser
Data notebook
Gremlin Bytecode
Gremlin
Bytecode
HTTP
HTTP
ipython-gremlin
Gremlin-Python
Application Server
©2017 IBM Corporation 19 June 201714
©2017 IBM Corporation 19 June 201715
Gremlin-Python (GLV)
§ Directly incorporated and tested by Apache TinkerPop
§ Implements core pieces found in Gremlin-Java
– Graph Traversal
– Graph Traversal Source
– __
§ Install via pip package manager
§ Gremlin-Python sugar
– g.V().both()[1:3].toList()
– g.V().both().name.toList()
§ Bindings
– g.V(('id',4)).out('created').name.toList()
http://tinkerpop.apache.org/docs/current/reference/#gremlin-python
©2017 IBM Corporation 19 June 201716
©2017 IBM Corporation 19 June 201717
ipython-gremlin: Gremlin Magic
ipython-gremlin: David Michael Brown @davebshow
Line Magic CellMagic
©2017 IBM Corporation 19 June 201718
ipython-gremlin: Python integrations
ipython-gremlin: David Michael Brown @davebshow
pandas, matplotlib NetworkX
©2017 IBM Corporation 19 June 201719
Air Routes Graph
Air Routes Graph: Kelvin Lawrence @gfxman
©2017 IBM Corporation 19 June 201720
Air Routes Web Application
§ Build a small set of APIs as a microservice to power the app
§ APIs can be used to
– Serve data to the app
– Serve data to other services
– Serve data to the Jupyter notebook
§ Example API for routes
– GET /routes/<orig>
– GET /routes/<orig>/<dest>
– GET /routes/<orig>/<dest>?maxhops=2
§ Graph (TinkerPop + Gremlin) is an implementation detail
©2017 IBM Corporation 19 June 201721
©2017 IBM Corporation 19 June 201722
©2017 IBM Corporation 19 June 201723
©2017 IBM Corporation 19 June 201724
Flying into the cloud
§ Graphs are most useful when they are large
§ Operating distributed clusters
– Security
– Availability
– Scalability
§ Sharing is caring
©2017 IBM Corporation 19 June 201725
Python Application Architecture (Local)
Graph DatabaseClients
Web browser
Data notebook
Gremlin Bytecode
Gremlin
Bytecode
HTTP
HTTP
ipython-gremlin
Gremlin-Python
Application Server
©2017 IBM Corporation 19 June 201726
Python Application Architecture (IBM Cloud)
Graph Database
Application Server
Clients
Web browser
Data notebook
Gremlin Bytecode
Gremlin
Bytecode
HTTP
HTTP
ipython-gremlin
Gremlin-Python
©2017 IBM Corporation 19 June 201727
https://www.compose.com/janusgraph
©2017 IBM Corporation 19 June 201728
Thank You: Open Source Foundations
Start Flying with Python & Apache TinkerPop
1 of 29

More Related Content

Similar to Start Flying with Python & Apache TinkerPop(20)

Webinar - Analyzing VideoWebinar - Analyzing Video
Webinar - Analyzing Video
Turi, Inc.914 views
Loading data into Apache IgniteLoading data into Apache Ignite
Loading data into Apache Ignite
Stephen Darlington973 views
Northwestern 20181004 v9Northwestern 20181004 v9
Northwestern 20181004 v9
ISSIP186 views
Supercharge your app with Cloud Functions for FirebaseSupercharge your app with Cloud Functions for Firebase
Supercharge your app with Cloud Functions for Firebase
Bret McGowen - NYC Google Developer Advocate1.3K views
Electron - Build cross platform desktop appsElectron - Build cross platform desktop apps
Electron - Build cross platform desktop apps
Priyaranjan Mohanty3.2K views

Start Flying with Python & Apache TinkerPop

  • 1. Start Flying with Python & Apache TinkerPop Graph Day SF Jason Plurad • pluradj@us.ibm.com • @pluradj IBM Open Technology • Apache TinkerPop • JanusGraph Saturday, June 17, 2017
  • 2. ©2017 IBM Corporation 19 June 20172 @pluradj Open source • Graph computing Dev @IBM • PMC @ApacheTinkerPop • TSC @JanusGraph Food truck chaser Jason Plurad
  • 3. ©2017 IBM Corporation 19 June 20173 Agenda Apache TinkerPop Gremlin + Python - Jupyter - Flask - IBM Cloud House of Mirrors: Ketrina Yim @KetrinaYim
  • 4. ©2017 IBM Corporation 19 June 20174 Founded 2009 • Marko Rodriguez @twarko Apache Top-Level Project 2016 • PMC Chair: Stephen Mallette @spmallette Committers: 16 • Contributors: 67 • Graph Systems: 20 https://tinkerpop.apache.org
  • 5. ©2017 IBM Corporation 19 June 20175 Apache TinkerPop: Graph Computing Framework § Defines graph structure and process § Core framework is written in Java § Provider implementations – Azure Cosmos DB – Blazegraph – DataStax Enterprise Graph – GRAKN.AI – HGraphDB – IBM Graph – JanusGraph – Neo4j – OrientDB – Sqlg – Stardog
  • 6. ©2017 IBM Corporation 19 June 20176 Gremlin Traversal Language sql2gremlin: Daniel Kuppitz @dkuppitz Recursively show all employees and their supervisors
  • 7. ©2017 IBM Corporation 19 June 20177 Graph Application Architecture Gremlin Server Graph DatabaseApplication
  • 8. ©2017 IBM Corporation 19 June 20178 Gremlin Language Drivers gremlin-javascript: Jean-Baptiste Musso @jbmusso Gremlin Bindings
  • 9. ©2017 IBM Corporation 19 June 20179 Gremlin Language Variants (GLV) § Elevate Gremlin to a top-level citizen in the programming language of choice § GLV can work with any modern language – Function composition – Function nesting § Java and Groovy (native) § Python is the first non-JVM GLV § Others are coming soon (JavaScript, C#) § SPARQL-Gremlin and SQL-Gremlin http://tinkerpop.apache.org/docs/current/tutorials/gremlin-language-variants
  • 10. ©2017 IBM Corporation 19 June 201710 Gremlin Bytecode http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine [ ['V'], ['range', 0, 5], ['out', 'created'], ['values', 'name'] ] g.V().range(0, 5). out('created'). values('name')
  • 11. ©2017 IBM Corporation 19 June 201711
  • 12. ©2017 IBM Corporation 19 June 201712 Programming Language Popularity TIOBE Index PYPL Popularity RedMonk Rankings
  • 13. ©2017 IBM Corporation 19 June 201713 Example Python Application Architecture Graph DatabaseClients Web browser Data notebook Gremlin Bytecode Gremlin Bytecode HTTP HTTP ipython-gremlin Gremlin-Python Application Server
  • 14. ©2017 IBM Corporation 19 June 201714
  • 15. ©2017 IBM Corporation 19 June 201715 Gremlin-Python (GLV) § Directly incorporated and tested by Apache TinkerPop § Implements core pieces found in Gremlin-Java – Graph Traversal – Graph Traversal Source – __ § Install via pip package manager § Gremlin-Python sugar – g.V().both()[1:3].toList() – g.V().both().name.toList() § Bindings – g.V(('id',4)).out('created').name.toList() http://tinkerpop.apache.org/docs/current/reference/#gremlin-python
  • 16. ©2017 IBM Corporation 19 June 201716
  • 17. ©2017 IBM Corporation 19 June 201717 ipython-gremlin: Gremlin Magic ipython-gremlin: David Michael Brown @davebshow Line Magic CellMagic
  • 18. ©2017 IBM Corporation 19 June 201718 ipython-gremlin: Python integrations ipython-gremlin: David Michael Brown @davebshow pandas, matplotlib NetworkX
  • 19. ©2017 IBM Corporation 19 June 201719 Air Routes Graph Air Routes Graph: Kelvin Lawrence @gfxman
  • 20. ©2017 IBM Corporation 19 June 201720 Air Routes Web Application § Build a small set of APIs as a microservice to power the app § APIs can be used to – Serve data to the app – Serve data to other services – Serve data to the Jupyter notebook § Example API for routes – GET /routes/<orig> – GET /routes/<orig>/<dest> – GET /routes/<orig>/<dest>?maxhops=2 § Graph (TinkerPop + Gremlin) is an implementation detail
  • 21. ©2017 IBM Corporation 19 June 201721
  • 22. ©2017 IBM Corporation 19 June 201722
  • 23. ©2017 IBM Corporation 19 June 201723
  • 24. ©2017 IBM Corporation 19 June 201724 Flying into the cloud § Graphs are most useful when they are large § Operating distributed clusters – Security – Availability – Scalability § Sharing is caring
  • 25. ©2017 IBM Corporation 19 June 201725 Python Application Architecture (Local) Graph DatabaseClients Web browser Data notebook Gremlin Bytecode Gremlin Bytecode HTTP HTTP ipython-gremlin Gremlin-Python Application Server
  • 26. ©2017 IBM Corporation 19 June 201726 Python Application Architecture (IBM Cloud) Graph Database Application Server Clients Web browser Data notebook Gremlin Bytecode Gremlin Bytecode HTTP HTTP ipython-gremlin Gremlin-Python
  • 27. ©2017 IBM Corporation 19 June 201727 https://www.compose.com/janusgraph
  • 28. ©2017 IBM Corporation 19 June 201728 Thank You: Open Source Foundations