Managing Connected Big Data in Art with Neo4j Graph Database - Lorenzo Speranzoni - Codemotion Amsterdam 2016

NEO4ART
Managing Big Connected Data in Art
with Neo4j Graph Database
Amsterdam | 11,12 May 2016
Lorenzo Speranzoni (@inserpio)
http://www.thethirdline.com/media/16111/TTL-121205181357-9528.jp

CodeMotion Amsterdam - 11,12 May 2016
AGENDA
Let’s say HI
Managing Modern Data
GRAPH DATABASES (NEO4J)
NEO4ART
Graphs and Art
Importing WIKIPEDIA in Neo4j
Next Steps
Q&A
VINCENT VAN GOGH - Still Life: Vase with Twelve Sunflowers, 1888
http://en.wikipedia.org/wiki/Vincent_van_Gogh#/media/File:Vincent_Willem_van_Gogh_128.jpg

LET’S SAY HI
CLAUDE MONET - Impression, soleil levant, 1872
http://en.wikipedia.org/wiki/Impression,_Sunrise#/media/File:Claude_Monet,_Impression,_soleil_levan

LORENZO SPERANZONI
Founder, CEO @ LARUS Business Automation
lorenzo.speranzoni@larus-ba.it
@inserpio
inserpio.wordpress.com
it.linkedin.com/in/lorenzosperanzoni
I like studying, learning, talking, sharing, teaching
INNOVATIVE OPEN SOURCE TECHNOLOGIES
I love CYCLING, ART and NATURE
And - of course - I’m a NEO4J EVANGELIST

LARUS BUSINESS AUTOMATION
Founded in 2004
Based in Venice - ITALY
Our Specialties:
bringing innovation in the market
custom software design and development
coaching and training new technologies
and methodologies

… this is a typical
day in Venice …
George Clooney
Wedding
BY THE WAY … … NOT THIS ONE DURING …
#*@!#!!

LARUS BUSINESS AUTOMATION
NEO4J LEADING PARTNER IN ITALY

NEO4J 3.0 JDBC Driver w/ BOLT
Neo4j 3.0
APPLICATION
JDBC

NEO4J - COUCHBASE CONNECTOR

MANAGING
MODERN DATA
EDOUARD MANET - In The Conservatory, 1879
http://en.wikipedia.org/wiki/%C3%89douard_Manet#/media/File:In_the_Conservatory_-_edite

MANAGING MODERN DATA
HAVE YOU EVER HEARD ABOUT THESE GUYS?

DO YOU THINK THOSE GUYS
USE A RELATION DATABASE?

[volume, velocity] + variety = 3 Vs of BIG DATA
data complexity =
f (size, uniformity, connectedness)

data complexity = f (size, uniformity, connectedness)

http://www.couchbase.com/nosql-resources/what-is-no-sql

CONNECTED DATA
ARE EVERYWHERE

SOCIAL NETWORKS

NETWORK IMPACT ANALYSIS

ROUTE FINDING

RECOMMENDATION

LOGISTICS

ACCESS CONTROL

FRAUD DETECTION

UNLOCKING VALUE FROM DATA
RELATIONSHIPS
“Without a doubt, the ability to connect the dots is rare,
prized and valuable. Connecting dots, solving the problem
that hasn't been solved before, seeing the pattern before it is
made obvious, is more essential than ever before”.
SETH GODIN
https://en.wikipedia.org/wiki/Seth_Godin

RELATIONSHIPS
Stop merely collecting data points
Start connecting them.
“RELATIONSHIPS BETWEEN DATA POINTS
MATTER ALMOST MORE THAN
THE INDIVIDUAL POINTS THEMSELVES”

RELATIONAL DBMS
CAN’T HANDLE RELATIONSHIP WELL
Ironically, legacy relational database management systems
(RDBMS) are poor at handling relationships between data points.
… making them inappropriate when data relationships are valuable
in real-time

RELATIONAL DBMS
CAN’T HANDLE RELATIONSHIP WELL
Their tabular data models and rigid schemas make it difficult to
add new or different kinds of connections.
Performance degrades with number and levels of relationships,
and database size
Query complexity grows with need for JOINs

NO-SQL ALTERNATIVES TO RDBMS
KEY-VALUES
Riak, Redis
COLUMN-ORIENTED
Cassandra
DOCUMENT
Couchbase, MongoDB
GRAPH
Neo4j
NO-SQL
Aggregated Data Model
DENORMALISE
NORMALISE
Connected Data Model

AGGREGATED NO-SQL DATABASES
DON’T HANDLE RELATIONSHIP WELL
No data structures to model or store relationships
No query constructs to support data relationships
Relating data requires “JOIN logic” in the application
No ACID support for transactions
… making them inappropriate when data relationships are valuable
in real-time

RELATIONSHIPS
In order to leverage data relationships, organizations
needs a database technology that stores relationship
information as a first-class entity:
THAT TECHNOLOGY IS A GRAPH DATABASE

THE WORLD’S LEADING
GRAPH DATABASE

GRAPH DATABASES ARE DIFFERENT
(i.e. when to use them)
Other NoSQL Relational DBMS Neo4j Graph DB
Discrete Data
Minimally  
connected data
Connected Data
Focused on
Data Relationships
Neo4j is designed for data relationships

GRAPH DATABASES ARE DIFFERENT
(i.e. when to use them)
Connectedness and Size of Data Set
ResponseTime
0 to 2 hops 
0 to 3 degrees 
Thousands of connections
Tens to hundreds of hops 
Thousands of degrees 
Billions of connections
Relational and 
Other NoSQL 
Databases
NEO4J IS
1000x FASTER 
Reduces minutes  
to milliseconds

NEO4J ARCHITECTURAL PRINCIPLE
Index-free adjacency property: a graph database can find
the neighbours of any given node without having to consider
the full set of relationships in the graph.
A result of this is that, as dataset size increases, the time it
takes to get the relationships off any given node stays
constant.

DATABASE POPULARITY TREND
Graph Databases
http://db-engines.com/en/ranking_categories

ABOUT GRAPH DATABASES
“Forrester estimates that over 25% of
enterprises will be using graph databases
by 2017”
“Graph analysis is possibly the single most
effective competitive differentiator
for organizations pursuing data-driven
operations and decisions after the design of
data capture.”

http://neo4j.com/news/neo4j-powers-panama-papers-investigation/
http://neo4j.com/blog/analyzing-panama-papers-neo4j/
http://www.reuters.com/article/us-panama-tax-neotechnology

https://offshoreleaks.icij.org/pages/database
https://gist.github.com/jexp/4a9dfd20cff6e270ea92dd3397ffb4ba

GRAPH
DATABASES
GUSTAVE CAILLEBOTTE - The Floor Planers, 1875

WHAT IS A GRAPH?
NO! THESE ARE CHARTS!

WHAT IS A GRAPH?
VERTEX
EDGE

GRAPH THEORY ORIGINS
LEONHARD EULER
Swiss Mathematician
Inventor of Graph Theory (1735)

GRAPH THEORY ORIGINS
SEVEN BRIDGES OF KÖNIGSBERG
A historically notable problem in mathematics. Its negative
resolution by Leonhard Euler in 1735 laid the foundations of
graph theory and prefigured the idea of topology.

LABELED PROPERTYGRAPH MODEL
4 BUILDING BLOCKS
NODES
RELATIONSHIPS
PROPERTIES
LABELS

Nodes: used to represent entities and complex types

Relationships: defined with regard to node instances

Labels: used to represent roles

Properties: on nodes and relationships as well

CAR
DRIVES
name: “Lorenzo”
born: Nov 20, 1974
twitter: “@inserpio”
name: “Olimpia”
born: Oct 23, 1974
since:  
Apr 10, 2008
brand: “Merceded”
model: “VITO”
LOVES
LOVES
LIVES WITH
OW
NS
PERSON PERSON

RELATIONAL vs GRAPH MODELS
Relational Model Graph Model
KNOWS
KNOWS
KNOWS
ANDREAS
TOBIAS
MICA
DELIA
Person FriendPerson-Friend
ANDREAS
DELIA
TOBIAS
MICA

NEO4J
Labels property graph model
Scalable NO-SQL database
Schema free
Fast JOIN operations
Support complex queries
Good for flexible, rich domains
Transactional | ACID
Query Engine
All Languages Drivers + REST API

QUERYING THE
GRAPH WITH 
OPEN-CYPHER
BERTHE MORISOT - The Cradle, 1872
http://en.wikipedia.org/wiki/Berthe_Morisot#/media/File:Berthe_Morisot,_Le_berceau_(The_Cradle),_18

OPEN-CYPHER: 
PATTERN QUERY LANGUAGE
Cypher is an expressive, compact, easy graph database query language
Other graph databases have other means of querying data. Many,
including Neo4j, support the RDF query language SPARQL and the
imperative, path-based query language Gremlin

OPEN-CYPHER: 
Cypher enables a user to ask the database to find data that matches a
specific pattern:

OPEN-CYPHER: 
MATCH (:Person { name:“Lorenzo”} ) -[:LOVES]-> (:Person { name:“Olimpia”} )
LOVES
Lorenzo Olimpia
LABEL PROPERTY
NODE NODE
LABEL PROPERTY

CONGRATS!
(CODEMOTION participants)

INTRODUCING
NEO4ART
CLAUDE MONET - Woman with a Parasol, facing left, 1886
http://en.wikipedia.org/wiki/Woman_with_a_Parasol_-_Madame_Monet_and_Her_Son#/media/File:Claude_Monet_0

Well, you might think that
sometimes (always!? )
ART is boring
http://images.sodahead.com/polls/003290665/5214752928_Waking_up_tired_xlarge.jpeg
http://www.everydayfamily.com/wp-content/uploads/2013/10/and-the-secret-to-raising-a-well-behaved-child-is.jpg
http://adhdspecialists.com/wp-content/uploads/2011/07/Fotolia_37623911_Subscription_Monthly_M.jpg
http://i.huffpost.com/gen/1804555/thumbs/o-KID-YAWNING-facebook.jpg

NEO4ART TEAM FEEL LIKE THAT
http://images.sodahead.com/polls/003290665/5214752928_Waking_up_tired_xlarge.jpeg
http://www.everydayfamily.com/wp-content/uploads/2013/10/and-the-secret-to-raising-a-well-behaved-child-is.jpg
http://adhdspecialists.com/wp-content/uploads/2011/07/Fotolia_37623911_Subscription_Monthly_M.jpg
http://i.huffpost.com/gen/1804555/thumbs/o-KID-YAWNING-facebook.jpg

ART & GRAPH
we strongly believe graphs
can help improving knowledge
about art

That’s why we decided to
start developing

HELPING YOU
TO GET IN TOUCH WITH THE ARTS
TO LET YOU FINALLY
EXPERIMENT NEW EMOTIONS

ART & GRAPHS

NEO4ART
DOMAIN
VINCENT VAN GOGH - Self-Portrait, 1887
http://en.wikipedia.org/wiki/Vincent_van_Gogh#/media/File:Vincent_van_Gogh_-_Self-Portrait_-_Google_Art_Project_(4540

DOMAIN
Artist
Artwork
Art
Movement
Quite simple indeed…
Museum
City
Monument
Technique

DOMAIN
Artist
Art
Movement
Museum
Technique
Artwork
City
Monument
Document
Letter Book
Colour
Colour
Analysis
Polarity
Sentiment
Analisys
CIDOC
CRM
Wikipedia
Open
Data

IMPORTING
DATA FROM
WIKIPEDIA
EDGAD DEGAS - The Dance Class, 1873-1876
http://en.wikipedia.org/wiki/Edgar_Degas#/media/File:Edgar_Degas_-_The_Ballet_Class_-_Google_Art_Projec

WIKIPEDIA XML DUMP
<page>
<title>Berthe Morisot</title>
<ns>0</ns>
<id>4233</id>
<revision>
<id>641658342</id>
<parentid>641590552</parentid>
<timestamp>2015-01-09T01:08:00Z</timestamp>
<model>wikitext</model>
<format>text/x-wiki</format>
<text xml:space=“preserve”> </text>
</revision>
</page>
Lot’s of interesting stuffs here:
- Links
- Topics we’re interested in
…
<groupId>info.blili.wiki</groupId>
<artifactId>bliki-core</artifactId>

IMPORTING DATA FROM WIKIPEDIA
NOT FROM DB-PEDIA AND FREEBASE
( …as of now… maybe better?? )
MORE FLEXIBILITY
MORE WORK

WIKIPEDIA -[:loaded into]-> GRAPH
1 PAGE == 1 NODE
1 LINK (from a PAGE to another) == 1 RELATIONSHIP
1 NAMESPACE == 1 LABEL
Wikipedia
Page
Wikipedia
Project
Wikipedia
Template
Wikipedia
Category
Wikipedia
Portal
Wikipedia
File

WIKIPEDIA INFOBOXES
{{Infobox artist
| name = Berthe Morisot, Madame Eugène Manet
| image = Berthe Morisot, 1875.jpg
| birth_name = Berthe Marie Pauline Morisot
| birth_date = {{birth date|1841|1|14|mf=y}}
| birth_place = [[Bourges]], [[Cher (département)|Cher]], [[France]]
| spouse = Eugène Manet
| death_date = {{death date and age|1895|3|2|1841|1|14|mf=y}}
| death_place = [[Paris]], France
| nationality = [[France|French]]
| field = [[Painting]]
| movement = [[Impressionism]]
}}

IMPORTING DATA FROM WIKIPEDIA
1 INFOBOX == 1 “ART DOMAIN” LABEL +
1 LINK (from Neo4ART Domain to 
WIKIPEDIA Domain)
Wikipedia
Artist
Page
Wikipedia
Artwork
Page
Wikipedia
ArtMovement
Page
Artist
Artwork
Art
Movement

NEXT RELEASES WILL IMPORT
FROM OTHER DATA SOURCES
MUSEUMS’
APIs
CIDOC
CRM
Artist
Art
Movement
Museum
Technique
Artwork
City
Monument

CURRENT GRAPH DIMENSION

NEO4ART v.0.9.3-beta
- - LIVE DEMO - -

NEXT STEPS
PIERRE AUGUSTE RENOIR - The Swing, 1876
http://en.wikipedia.org/wiki/Pierre-Auguste_Renoir#/media/File:Auguste_Renoir_-_The_Swing_-_Google_Art_Proj

NEXT STEPS
RENDER WIKIPEDIA DATA
IMPORTING OTHER DATA SOURCES
RECOMMENDATION
GEOSPATIAL SEARCH
IMPROVE ALL KIND OF ANALYSES (colour,
sentiment, etc)

WE NEED YOUR SUPPORT
FORK THE PROJECT
AND START CODING WITH US
https://github.com/neo4art/neo4art

THANK YOU
ALFRED SISLEY - Boat in the Flood at Port Marly, 1876
http://en.wikipedia.org/wiki/Alfred_Sisley#/media/File:Alfred_Sisley_-_Boat_in_the_Flood_at_Port_Marly_-_Google_Art_Pro

Managing Connected Big Data in Art with Neo4j Graph Database - Lorenzo Speranzoni - Codemotion Amsterdam 2016

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Managing Connected Big Data in Art with Neo4j Graph Database - Lorenzo Speranzoni - Codemotion Amsterdam 2016

Similar to Managing Connected Big Data in Art with Neo4j Graph Database - Lorenzo Speranzoni - Codemotion Amsterdam 2016 (20)

More from Codemotion

More from Codemotion (20)

Recently uploaded

Recently uploaded (20)

Managing Connected Big Data in Art with Neo4j Graph Database - Lorenzo Speranzoni - Codemotion Amsterdam 2016