Network construction and
visualization
Francisco Restivo
frestivo@ucp.pt
slideshare.net/frestivo
favinks.com/board/FranciscoRestivo
Thank you
•
I appreciate very much the invitation and
your presence
•
I enjoy these dialogues
•
Ask, interrupt, don't let doubts persist
•
And don't forget, the teacher always
learns...
2
Escola das Artes 2023
Advance notice
•
My field is Informatics, that is, Automatic
Information
•
Basically, it's about teaching machines to
solve problems
•
I'm interested in problems related to
– Data → Information → Knowledge
•
Think of me as a translator...
3
Escola das Artes 2023
About me
4
Escola das Artes 2023
Lic. EE (6 anos) D.Phil. Prof. Assoc. Director Prof. Assoc
Signals DSP Multiprocessor systems Biomedical Systems Complex systems
Manufacturing systems Multi-agent systems Social systems Networks
Lic. EE (6 anos) D.Phil. Prof. Assoc. Director Prof. Assoc
Lic. EE (6 anos) D.Phil. Prof. Assoc. Director Prof. Assoc.
1970
Internet timeline
5
Escola das Artes 2023
IBM PC 1981
6
Escola das Artes 2023
Social media timeline
7
Escola das Artes 2023
htps://www.broadbandsearch.net/
7
7
Where are we?
•
The explosion of Big Data
•
Networks everywhere
•
Social Networks
•
Technologies
•
Tools
•
Projects
•
Digitization
8
Escola das Artes 2023
9
Escola das Artes 2023
An Internet minute
And heritage?
•
Documents
•
Pictures, etc
•
Digitized?
•
A quick tour
10
Escola das Artes 2023
A quick tour
•
Some projects
•
Basic ideas
•
How does a 'data scientist' think?
•
Collaborative platforms
•
Digital Humanities
11
Escola das Artes 2023
12
Escola das Artes 2023
slideshare.net/frestvo
13
Escola das Artes 2023
htp://www.martngrandjean.ch/
14
Escola das Artes 2023
15
Escola das Artes 2023
16
Escola das Artes 2023
htps://www.familysearch.org/
17
Escola das Artes 2023
18
Escola das Artes 2023
19
Escola das Artes 2023
20
Escola das Artes 2023
Challenges and opportunities
•
Existing data and changing management
•
Data discovery, access and reuse
•
Digital skills
•
There is an enormous potential in the
network approach
21
Escola das Artes 2023
Data science approach
•
Regularities
•
Correlations
•
Causality
•
Relations/networks/distances/clusters
•
Co-occurrences
•
Deep learning, etc
22
Escola das Artes 2023
The plan for today
•
A realistic plan
– Part 1: some work hands on with real
data
– Part 2: very basic ideas on data
structures and networks
•
Please feel free to question, interrupt,
etc
23
Escola das Artes 2023
Escola das Artes 2023 24
Questions?
Part 1
25
Escola das Artes 2023
Datasets
26
Escola das Artes 2023
Where to obtain?
•
Make or get!
•
Usually, we know
our subject
•
There are many
sites you can visit
– kaggle
– data.world
27
Escola das Artes 2023
Co-authors, in a Department
28
Escola das Artes 2023
APIs
•
Application Programming Interfaces
•
All major platforms offer APIs
– ChatGPT offers an API
•
Python is friendly
Escola das Artes 2023 29
Escola das Artes 2023 30
polsys.net
Coldplay musical universe
31
Escola das Artes 2023
Escola das Artes 2023 32
Escola das Artes 2023 33
Escola das Artes 2023 34
35
Escola das Artes 2023
36
Escola das Artes 2023
dimensions.ai
37
Escola das Artes 2023
Limits?
•
Data ownership
•
Privacy
•
Business
Escola das Artes 2023 38
Tools
39
Escola das Artes 2023
40
Escola das Artes 2023
41
Escola das Artes 2023
Characters, Les Misérables
42
Escola das Artes 2023
43
Escola das Artes 2023
44
Escola das Artes 2023
45
Escola das Artes 2023
46
Escola das Artes 2023
nocodefunctons.com
People
47
Escola das Artes 2023
Escola das Artes 2023 48
Escola das Artes 2023 49
Escola das Artes 2023 50
Escola das Artes 2023 51
Books
52
Escola das Artes 2023
Escola das Artes 2023 53
Escola das Artes 2023 54
Questions?
Part 2
55
Escola das Artes 2023
Computer programs
•
Algorithms + Data Structures
•
Many programming language (your choice)
•
Programming is a communication task
– Shared concepts
– Well defined actions
56
Escola das Artes 2023
Data structures
•
Lists
•
Stacks
•
Queues
•
Linked lists
•
Trees
•
Graphs
– Objects + attributes + relations
57
Escola das Artes 2023
Databases
•
Dataframes
•
Relational databases
•
Graph databases
58
Escola das Artes 2023
Graph database
59
Escola das Artes 2023
htps://neo4j.com/
Algorithms
•
Sort
•
Search
•
Insert/delete
•
Data analysis, data mining, etc
•
Graph algorithms
60
Escola das Artes 2023
Networks
•
Social, biological, financial, etc
•
Complex networks
•
Communities
•
Contagion
•
Controversies
•
Society!
61
Escola das Artes 2023
Social interactions
•
Like, comment, share, cite
•
Dating
•
e-Commerce
•
Payments
•
Digital marketing
•
Political marketing
•
Crime
62
Escola das Artes 2023
Metaverse?
•
Virtual
•
Augmented
•
Fake
•
Holograms
•
New,
or more of the same?
63
Escola das Artes 2023
SO?!
Escola das Artes 2023 64
•
Let's have a look at networks and graphs
Euler 1707 - 1783
Escola das Artes 2023 65
Actors and interactions
•
Networks are made of actors and
interactions
•
Actors from one or more sets
– People and movies, eg
•
Interactions of one or more types
– Follow and retweet, eg
•
Actors and interactions may have other
atributes
Escola das Artes 2023 66
Basics of graphs and networks
•
G = (V, E) vertices and edges
•
O(G) = |V| order
•
S(G) = |E| size
•
A adjacency matrix
• Ki
degree of vertex i
•
Directed/undirected
Escola das Artes 2023 67
●
Directed/undirected?
●
Usually not transitive
– a likes b and b likes c does not imply a likes c
●
Directed > Order (total/partial)
– Rankings, proeminence indexes, etc
●
Undirected > Equivalence
– Communities, clusters, etc
Escola das Artes 2023 68
Network edges
Representations of networks
•
Matrixes, graphs, edge lists, etc
A B
A C
A D
B A
B C
B E
C D
D B
D C
E A
E B
Escola das Artes 2023 69
A B C D E
A 0 1 1 1 0
B 1 0 1 0 1
C 0 0 0 1 0
D 0 1 1 0 0
E 1 1 0 0 0
Global metrics
•
Number of vertices 5
•
Number of edges 11
•
Number of components 1
•
Diameter 2
•
Density 0.55
Escola das Artes 2023 70
Centrality Measures
•
Degree centrality
– Edges per node
•
Closeness centrality
– How close the node is to every other node
•
Betweenness centrality
– How many shortest paths go through the node
•
Bibliometric + Internet style (quality of edges)
– PageRank, eigenvector
71
Escola das Artes 2023
Which metrics?
•
Depend of the answers we need
– Key nodes
– Ranking
– Influence, contagion
– Communities, etc
•
That's what network science is all about
•
You have to know the problem
72
Escola das Artes 2023
Which networks?
•
Identifying networks may be not intuitive
•
But at the end is all about actors and
interactions
– Vertices = nodes = actors
– Edges = relations = interactions
•
Going through examples is helpful
73
Escola das Artes 2023
My FB, Lost Circles
74
Escola das Artes 2023
Countries, FIFA World Cup
75
Escola das Artes 2023
New Testament
76
Escola das Artes 2023
Reccomendation systems
•
Bipartite networks: two types of nodes
•
Two similarity networks
Escola das Artes 2023 77
Bipartite networks
Escola das Artes 2023 78
Dynamics
•
Networks may have a temporal dimension
•
Interactions – follow, like, share,
mention, retweet, hashtag, etc – occur in
sequence
•
Network properties evolve in time
Escola das Artes 2023 79
Thank you!
Escola das Artes 2023 80

Network Construction and Visualization.pdf