Bootstrapping Your Graph Project with Neo4j Data Importer and Browser

1. © 2022 Neo4j, Inc. All rights reserved.
Bootstrapping Your Graph
Project with Neo4j Data Importer
and Browser
Anurag Tandon,
Sr. Director, Product Management
Junxiang Chen,
Software Engineer, Neo4j DevTools

2. © 2022 Neo4j, Inc. All rights reserved.
2
About Us
Junxiang Chen
Software Engineer
Neo4j DevTools
Greg King ✈️🌴
Product Manager
Neo4j DevTools
Anurag Tandon
Sr. Director, PM
Neo4j User Tools

3. © 2022 Neo4j, Inc. All rights reserved.
Agenda
• Get to know Neo4j DevTools
◦ Our mission
◦ DevTools in context
• Live Demo! Bootstrapping your graph project
◦ Unlocking insights from UK parliamentary data
◦ How we use DevTools to build our graph project
• Data Importer - Initial modelling and loads
• Browser - Initial queries, hypothesis testing and and enrichment of the graph
• Reviewing insights with our complementary sister product, Neo4j Bloom
• Future evolution of DevTools
3

4. © 2022 Neo4j, Inc. All rights reserved.
4
Get to know Neo4j DevTools

5. © 2022 Neo4j, Inc. All rights reserved.
Initial data loads and
queries
Realising the benefits of
Neo4j with your own data
and queries.
5
DevTools mission
“To serve data practitioners on
their journey with Neo4j”
New to Graph and
Neo4j
Educate and get started
with tutorials and sample
projects
Neo4j Expert
Neo4j as a seamless part
of the development
workflow
Building skills and
knowledge
Using more advanced
features and exploring
new uses

6. © 2022 Neo4j, Inc. All rights reserved.
6
DevTools in context

7. © 2022 Neo4j, Inc. All rights reserved.
Desktop
• Maintenance releases, fixing various bugs and updating underlying dependencies
Browser
• Update of visualisation library that underpins Browser
• Parameter Assistance - making it easier to construct a params object in the editor
• Graph Visualisation Zoom - New zoom to fit option and scroll-to-zoom support
• Centrally-hosted Browser in Aura for rapid updates and fixes
Data Importer
• Beta launch to AuraDB Free/Professional and AuraDS tiers
• Improved surfacing of partial / incomplete mappings and better empty node
id handling
Recent Highlights

8. © 2022 Neo4j, Inc. All rights reserved.
8
Bootstrap a graph project

9. © 2022 Neo4j, Inc. All rights reserved.
9
UK Parliament
• Two house system
◦ House of Commons
◦ House of Lord
◦ Check-and-balance system
• People are elected as members (MPs) to
◦ Sit in a house
◦ Belong to respective parties and on
constituencies behalf
◦ Vote and debate on divisions, which
contributes to make laws and policies,
approve bills, etc
◦ Scrutinise government
◦ …
• Open-sourced data
◦ UK parliament data resources
◦ Public Whip – a non profit community
Source: https://commonslibrary.parliament.uk/data-tools-and-
resources/
https://www.publicwhip.org.uk/

10. © 2022 Neo4j, Inc. All rights reserved.
© 2022 Neo4j, Inc. All rights reserved.
10
Sample flat (relational) data
From Public Whip:
• Flatten data: *.csv
◦ Peoples (MPs /
Constituencies /
Parties / Houses)
metadata
◦ Offices metadata
◦ Divisions
◦ MPs votes
◦ Policies
◦ Wiki edit
• Nested data: *.json
◦ Policy set
“Find out how your MP
votes on the issues you
care about”
mp_id first_name last_name party … constituency person house …
1 Diane Abbott Lab … Hackney North and Stoke Newington 10001 commons …
pw_mp.csv
moffice_i
d
dept position person …
54045826 Treasury Committee Member 10001 …
pw_moffice.csv
division_id division_date division_name division_number house …
12373 1997-05-22 Referendums (Scotland and Wales) Bill 5 commons …
pw_division.csv
wiki_id division_number house text_body …
3395 5 common --- DIVISION TITLE ---
Business of the House -- Programme motion
…
…
pw_dyn_wiki_motion.csv
dream_id name …
91 PRO-EURO "ONE NATION" CONSERVATIVE …
pw_dyn_dreammp.csv
division_id mp_id vote
12373 1 no
pw_vote.csv
Primary Key
Index
Foreign Key
Example:
https://www.theyworkforyou.com/mp/25827/angela_richardson/guildford/

11. © 2022 Neo4j, Inc. All rights reserved.
© 2022 Neo4j, Inc. All rights reserved.
11
Sketch graph schema
Next: load *.csv data via Data Importer
From Public Whip:
• Flatten data: *.csv
◦ Peoples (MPs /
Constituencies /
Parties / Houses)
metadata
◦ Offices metadata
◦ Divisions
◦ MPs votes
◦ Policies
◦ Wiki edit
• Nested data: *.json
◦ Policy set
“Find out how your MP
votes on the issues you
care about”

12. © 2022 Neo4j, Inc. All rights reserved.
© 2022 Neo4j, Inc. All rights reserved.
14
Cypher solutions
Limitations on loading
• // Multi-key index
CREATE INDEX division FOR
(d:Division) ON
(d.division_number,
d.division_date, d.house)
• // Special unicode tokens
LOAD CSV WITH HEADERS FROM
'file:///pw_dyn_dreammp.csv'
AS row
FIELDTERMINATOR 'u001e'
• // *.json file handling
:param (policies.json)
UNWIND $policySets AS
policySet
• …
• Multi-key index
• Multiple labels/types
• Various normalised files for
one graph entity
• Special unicode tokens
• JSON file import
• …

13. © 2022 Neo4j, Inc. All rights reserved.
© 2022 Neo4j, Inc. All rights reserved.
15
Create shortcut path
• Create shortcut
relationships
between people
and the policies
they support
POSITION_AGAINST
{vote:no} {vote:aye}
POSITION_FOR
{vote:no} {vote:no}

14. © 2022 Neo4j, Inc. All rights reserved.
© 2022 Neo4j, Inc. All rights reserved.
16
Data aggregation
• Summarise the
overall number of
FOR and AGAINST
votes for political
parties on different
policies
• Work out a
polarisationScore
that shows how
polarised a party is
on a particular topic
MPs typically follow the
party line, but which
policies divide them and
how does that vary by
party?
policy party position numPeople overallPos polarisationScore
Iraq 2003 - For the invasion Lab POSITION_AGAINST 537 For 0.6010072747621713
Iraq 2003 - For the invasion Lab POSITION_FOR 1250
Iraq 2003 - For the invasion Con POSITION_AGAINST 81 For 0.20275344180225274
Iraq 2003 - For the invasion Con POSITION_FOR 718
… … … … …
Homosexuality - Equal rights Lab POSITION_AGAINST 1700 For 0.8865710560625815
Homosexuality - Equal rights Lab POSITION_FOR 2135
Homosexuality - Equal rights Con POSITION_AGAINST 1546 Against 0.975819807883405
Homosexuality - Equal rights Con POSITION_FOR 1473
propFor = positionFor / (positionFor + positionAgainst)
// > 0.5 —> For, = 0.5 —> Divided, < 0.5 —> against
polarisationScore = 1 - (2 * ABS(propFor - 0.5))
// 0 = completely aligned, 1 = completely divided

15. © 2022 Neo4j, Inc. All rights reserved.
17
DevTools key takeaways

16. © 2022 Neo4j, Inc. All rights reserved.
18
Future roadmap
Neo4j Workspace

17. © 2022 Neo4j, Inc. All rights reserved.
A Unified Neo4j Workspace to Bring Tools Together
To learn more, please
attend:
Introducing Workspaces,
a New Experience for
Neo4j Developer Tools
Wednesday, June 8
1:45 - 2:30 pm CT
Iris (4th floor)

18. © 2022 Neo4j, Inc. All rights reserved.
Resources and acknowledgements
• Session keynotes
◦ Please refer to https://dev.neo4j.com/devtools
• Special thanks
◦ UK parliament and Public Whip for providing the open-sourced data
◦ Gregory King, our Product Manager, for preparing the query examples and data insights
20

19. © 2022 Neo4j, Inc. All rights reserved.
21
Q & A

20. © 2022 Neo4j, Inc. All rights reserved.
22
Thank you!
Contact us at
anurag.tandon@neo4j.com
junxiang.chen@neo4j.com
gregory.king@neo4j.com