SlideShare a Scribd company logo
From
Power Chords
to
the Power of
Models
@aliostad
Ali Kheyrollahi
8
Local pop music
?
9
Local pop music
“Cheelee pom!”
10
Boney M
“Rasputin”
11
Blondie
“Heart of Glass”
Short Wave Radio (SW2)
Data
Acquisition
Data Source - Wiki
4,990,2794,990,279 English Articles
37,583,879 Articles
Data Source - What about …
2012
http://money.cnn.com/2012/03/13/technology/encyclopedia-britannica-books/
Data Source - Wiki vs Britannica
Feng Zhu (assistant prof at Harvard):
There has been lots of research on the accuracy of
Wikipedia, and the results are mixed—some studies
show it is just as good as the experts, others show
[that] Wikipedia is not accurate at all.
… the editors [of Britannica] are still not
found to be more objective than the crowd
in articles that are sufficiently revised.
Data Source - Wikipedia in scholar papers
0
45000
90000
135000
180000
2005 2006 2007 2008 2009 2010 2011 2012 2013 2014
Source: Google Scholar
Data Source - Dark side of Wiki
The Doggs: A mid-1960s British rock band, noted for
barking while playing music
Lord Byron kept a crocodile and a honey badger as a pet
Ted and the Treble Tones, a rock band from Fresno with a
1959 hit single that appeared nowhere on the Billboard
charts
The Shaggs all female (sister) band Hacked-at drumbeats,
whacked-around chords, songs that seem to have little or
no meter to them… Zappa’s favourite…
Data Source - Content vs. Data
… or … Alt-country
Alternative country
… or … Alt.country
… or … Alt. Country
[mind the gap]
Data Source - Content vs. Data
comma
slash
new line
period?
no, greek middle dot U+00B7
Data Source - Content vs. Data
Hyphen
U+002D
figure dash
U+2012
minus sign
U+2015
em dash
U+2014
en dash
U+2013
Data Source - DBpedia
• http://wiki.dbpedia.org/
• bziped N-Triples or N-Quads RDF entries
• Different datasets:
• Titles
• Categories
• Infobox data
• Short Abstracts
• In the end was not used
Data Acquisition - Wiki
List of Rock
Genres
Rock Genres Rock Artists
Store
Store
HTML
Capture
Links
Store
HTML
Python scripts
Postgres
Data
Exploration
Data Exploration
“I personally … literally just look at the screen,
just like the matrix”
Claudia Perlich, multi-award winner Data
Scientist
Data Exploration
“… the dirty little secret that I have won all of them
because I have found something wrong with the
data… I would like to play around with dataset and
get initimately familiar with dataset and its
properties.“
Claudia Perlich
Album Genre
Album Genre
50 Years of
Rock
(1965-1989)
50 Years of Rock 1965
1. rock (23)
2. folk rock (15)
3. rock and roll (13)
4. folk (12)
5. rhythm and blues (12)
6. blues rock (10)
7. country (9)
The Who
My Generation
“Rock” as genre was king with The
likes of The Who and Beatles
Rock and Roll, popular then, soon to
disappear from the list
50 Years of Rock 1966
1. folk rock (25)
2. garage rock (21)
3. psychedelic rock (20)
4. rock (19)
5. folk (14)
6. protopunk (11)
7. pop (11)
Bob Dylan
Blonde on Blonde
Folk Rock was popular all of the
sixties with a resurgence in the early
70s
This is obviously named
retrospectively but along with
Garage rock to influence punk
50 Years of Rock 1967
1. psychedelic rock (56) ↑
2. folk rock (30)
3. pop (19)
4. folk (19)
5. rock (18)
6. garage rock (12)
7. psychedelic pop (12) The Beatles
Sgt. Pepper's Lonely Hearts
Club Band
67, a big year with Monterey Pop
Festival, was all about psychedelia
Even for psychedelic pop
50 Years of Rock 1968
1. psychedelic rock (70) ←
2. rock (29)
3. folk rock (26)
4. blues rock (25)
5. folk (18)
6. acid rock (18)
7. progressive rock (15)
Jimi Hendrix
Electric Ladyland
No comment, listen to Jimi!
Note Acid Rock and emergence of
progressive rock
50 Years of Rock 1969
1. psychedelic rock (60) ←
2. rock (40)
3. folk rock (33)
4. hard rock (27)
5. folk (26)
6. progressive rock (25)
7. country (22)
Led Zeppelin
Led ZeppelinNot much changed… progrock
moving up
50 Years of Rock 1970
1. rock (57)
2. folk rock (47) ↑
3. psychedelic rock (44) !!
4. hard rock (40) ↑
5. progressive rock (31) ↑
6. country rock (25)
7. folk (24) The Velvet Underground
Loaded
Artists leaving psychedelia for a
harder sound
50 Years of Rock 1971
1. rock (47)
2. hard rock (34) ↑
3. folk rock (33) !
4. progressive rock (33) ↑
5. psychedelic rock (26) !
6. folk (24)
7. country rock (18) CAN
Tago Mago
Progressive rock and hard rock
moving up
50 Years of Rock 1972
1. rock (54)
2. folk rock (34)
3. progressive rock (29)
4. hard rock (29)
5. country rock (27)
6. folk (25)
7. country (20)
Jethro Tull
Thick as a Brick
50 Years of Rock 1973
1. rock (60)
2. hard rock (44) ↑
3. progressive rock (40)
4. folk rock (29)
5. country (24)
6. glam rock (22)
7. country rock (22)
Flashy dresses and make-up, bring it
on!
Pink Floyd
Dark Side of the Moon
50 Years of Rock 1974
1. rock (58)
2. hard rock (52)
3. progressive rock (40)
4. glam rock (29) ↑
5. folk rock (28)
6. blues rock (20)
7. country rock (20)
Supertramp
Crime of the Century
50 Years of Rock 1975
1. rock (53)
2. hard rock (52)
3. progressive rock (37)
4. folk rock (23)
5. country (20)
6. blues rock (20)
7. country rock (18)
Patti Smith
Horses
50 Years of Rock 1976
1. rock (71)
2. hard rock (56)
3. progressive rock (28)
4. heavy metal (19)
5. pop (19)
6. folk (18)
7. country (18)
Did you notice?! Heavy Metal is now
a thing…
AC/DC
High Voltage
50 Years of Rock 1977
1. rock (77)
2. hard rock (43)
3. progressive rock (29)
4. punk rock (21)
5. pop (19)
6. folk rock (15)
7. country rock (14)
Oh yeah, punk rock in full swing and
pretty much mainstream.
Sex Pistols
Never Mind the Bollocks
50 Years of Rock 1978
1. rock (77)
2. hard rock (40)
3. progressive rock (24)
4. new wave (23)
5. pop (21)
6. country (20)
7. punk rock (19)
New wave with a more layered
sound replacing punk rock
Talking Heads
More Songs About Buildings and
Food
50 Years of Rock 1979
1. rock (68)
2. new wave (49) ↑
3. post-punk (42)
4. hard rock (38) !
5. country (22)
6. punk rock (20)
7. progressive rock (15) !
Only after two years of punk, we
have post-punk… phew
Gary Numan
Replicas
50 Years of Rock 1980
1. rock (66)
2. new wave (57)
3. post-punk (49)
4. hard rock (38)
5. country (24)
6. heavy metal (22)
7. punk rock (21)
Joy Division
Closer
50 Years of Rock 1981
1. new wave (75) ↑
2. rock (71)
3. post-punk (68)
4. hard rock (41)
5. synthpop (25)
6. heavy metal (21)
7. country (19)
Another movement is already in
motion with softer keyboard-based
sound
Klaus Nomi
Klaus Nomi
50 Years of Rock 1982
1. new wave (79)
2. post-punk (61)
3. rock (61)
4. hard rock (37)
5. heavy metal (28) ↑
6. synthpop (28)
7. hardcore punk (21)
Iron Maiden
Number of the Beast
Other than new wave and post-punk,
two completely different sounds
growing, heavy metal on one hand
and synth pop on the other…
50 Years of Rock 1983
1. rock (74)
2. new wave (71)
3. post-punk (45) !
4. synthpop (39) ↑
5. heavy metal (36)
6. hard rock (35)
7. pop rock (24) The Police
Synchronicity
50 Years of Rock 1984
1. new wave (73)
2. rock (61)
3. post-punk (44)
4. heavy metal (37)
5. synthpop (37)
6. hard rock (26)
7. alternative rock (24)
In case you haven’t noticed, New
Wave has been on the top spot for a
few years
Echo and the Bunnymen
Ocean Rain
50 Years of Rock 1985
1. new wave (66)
2. rock (56)
3. post-punk (45)
4. heavy metal (33)
5. pop rock (33)
6. alternative rock (28) ↑
7. synthpop (28) !
Alternative Rock appearing… soon
to label much of the rock music we
have now…
The Fall
This Nation’s Saving Grace
50 Years of Rock 1986
1. rock (68)
2. new wave (48) !
3. alternative rock (45) ↑
4. heavy metal (42)
5. post-punk (40)
6. hard rock (31)
7. synthpop (28)
Synthpop is over… only to come
back after 25 years.
The Smiths
The Queen Is Dead
50 Years of Rock 1987
1. rock (62)
2. alternative rock (50) ↑
3. hard rock (36)
4. thrash metal (35)
5. heavy metal (31)
6. new wave (29) !
7. pop rock (24)
Wow… where did this come from?!
Testament
The Legacy
50 Years of Rock 1988
1. alternative rock (59) ↑
2. rock (58)
3. heavy metal (42)
4. hard rock (40)
5. thrash metal (33)
6. synthpop (24)
7. new wave (23) !
New wave is out of favour after
almost 10 years…
Sonic Youth
Daydream Nation
50 Years of Rock 1989
1. rock (80)
2. alternative rock (67)
3. hard rock (41)
4. thrash metal (39)
5. heavy metal (34)
6. new wave (26)
7. pop (22)
The Cure
Disintegration
Data
Models
Data Models Model?!
Data Models Model
Mathematical representation of a concept
based on parameters that impact that
concept
• Rating of a native app
• Stackoverflow score
• Credit score
• Fraud check
Data Models Model Example
(Average)
Data Models Graph 101
Social Network Analysis
and Graph Theory
• Nodes/vertices and edges/lines
• Directedness:
• Directed
• Undirected
• Degree, InDegree/OutDegree
• Weight
A B
Data Models Centrality
API - Related artists
1
2
4
2
2
1
Same degree
Different betweenness
Degree
Graph Codez
import networkx as nx
g = nx.Graph()
g.add_edge(‘a’, ‘b’)
g.add_edge(‘b’, ‘c’)
…
print len(g[‘b’]) # degree
c = nx.betweenness_centrality(g, normalized=True)
# c -> dictionary of node names and their score
DiGraph()
Modelling
Influence
using
Wiki
Data Models Cited Influence
Howlin’
Wolf
Captain
Beefheart
1940 1964
Data Models Cited Influence
Most influential Rock Artists Based on out-degree
The Beatles => 188
Black Sabbath => 127
Led Zeppelin => 118
Jimi Hendrix => 114
Bob Dylan => 94
Pink Floyd => 86
Iron Maiden => 77
Metallica => 77
The Rolling Stones => 66
The Beach Boys => 65
Neil Young => 63
Nirvana => 62
Slayer => 60
Queen => 59
Data Models Cited Influence
Most influential Rock Artists Based on Betweenness Centrality
Jimi Hendrix => 53476.2014921
The Beatles => 47511.7957531
Bob Dylan => 38107.0298185
Led Zeppelin => 32701.7223273
Nirvana => 29733.9066836
Metallica => 29356.6009213
Queen => 28989.2844223
Robert Smith => 28880.670718
Elvis Presley => 28463.2891497
Slade => 27656.487307
Iron Maiden => 22449.6697023
Ramones => 22437.6112965
Rush => 21125.9481602
Neil Young => 19913.887522
Data Models Cited Influence
Most influential Artists Based on Betweenness Centrality
Metallica => 566.06
Iron Maiden => 419.21
Corey Taylor => 146.0
Led Zeppelin => 122.73
Slipknot => 116.58
King Diamond => 94.7
Machine Head => 85.12
Rush => 70.41
Black Sabbath => 68.0
Van Halen => 54.56
Deep Purple => 53.5
Megadeth => 42.63
Guns N' Roses => 24.25
Heavy Metal
Nirvana => 490.08
Muse => 114.5
Weezer => 97.33
Pixies => 94.17
Sonic Youth => 78.5
Rivers Cuomo => 69.5
Siouxsie and the Banshees => 51.67
The Smiths => 51.5
Jeff Buckley => 46.17
The Offspring => 43.0
Placebo => 42.0
My Chemical Romance => 34.0
The Smashing Pumpkins => 32.33
Alternative Rock
Rush => 54.0
Marillion => 34.0
Pink Floyd => 33.0
Yes => 20.0
Porcupine Tree => 19.5
Dream Theater => 19.0
Chris Squire => 16.5
Primus => 15.0
Tool => 12.0
Mahavishnu Orchestra => 8.0
Geddy Lee => 7.0
Neil Peart => 5.0
Keith Emerson => 5.0
Progressive Rock
Shaping
a Model
Shaping a Model
•Start with Literature survey
•Draw parallels with similar domains
•If you still cannot find one, define a
new model
•Try to refine your model with a
quantitative measure if available
Shaping Visionaries Model
Visionary: Influencing vs. being influenced
V: visionary factor
deg+: out-degree
deg-: in-degree
e: the exponential function
Shaping Visionaries Model
Joni Mitchell => 44.0
Roxy Music => 31.0
Leonard Cohen => 31.0
Josh White => 31.0
Robert Johnson => 24.0
Etta James => 23.0
Discharge => 22.0
Sam Cooke => 21.0
Duke Ellington => 20.0
Marc Bolan => 19.0
Hank Marvin => 17.0
Jonathan Richman => 16.0
Scott Walker => 16.0
Testament => 16.0
The Police => 16.0
Skip James => 15.0
Albert King => 15.0
The Cure => 14.7151776469
Nico => 14.0
Data Models Visionaries Model
Most influential Artists in genres
Radiohead => 15.0
Depeche Mode => 12.0
Morrissey => 11.0
Robert Smith => 11.0
The Replacements => 10.0
The Cure => 8.83
XTC => 8.0
Stone Temple Pilots => 6.0
Duran Duran => 6.0
Nick Cave => 6.0
Wipers => 5.0
Johnny Marr => 4.0
Jane's Addiction => 4.0
Alternative Rock
The Stooges => 6.0
The Jam => 6.0
Sex Pistols => 4.05
Jawbreaker => 4.0
The Runaways => 4.0
The Dickies => 4.0
Circle Jerks => 3.0
Big Black => 3.0
Patti Smith => 3.0
Elvis Costello => 3.0
The Only Ones => 3.0
Kevin Seconds => 3.0
Descendents => 2.94
Punk Rock
Captain Beefheart => 7.0
The Magic Band => 6.0
Genesis => 5.0
Scott Walker => 5.0
Scott Engel => 3.0
Talk Talk => 3.0
Sparks => 2.0
Richard Wright => 2.0
Roxy Music => 1.84
Pink Floyd => 1.49
Mark Hollis => 1.0
Nico => 1.0
Lou Reed => 1.0
Art Rock
Dave Lombardo => 11.0
Napalm Death => 8.0
Morbid Angel => 3.68
Possessed => 3.31
Carcass => 2.21
Immolation => 2.0
Vulcano => 2.0
Hellhammer => 2.0
Autopsy => 1.47
Repulsion => 1.1
Entombed => 1.1
Impetigo => 1.0
Nihilist => 1.0
Death Metal
Shaping Impact Model
Imact: Influencing many in a short span
Imp: impact factor
deg+: out-degree
L: longevity (initial continuous years active)
Shaping Impact Model
Imact: Influencing many in a short span
Cream => 3.33
Sex Pistols => 2.19
Joy Division => 2.0
Jimi Hendrix => 1.83
The Beatles => 1.59
The Smiths => 1.19
Big Star => 1.19
Parliament => 1.11
Nirvana => 1.0
Ritchie Valens => 0.78
Led Zeppelin => 0.73
Traffic => 0.67
Shaping Impact Model
Imact: Influencing many in a short span in a specific genre
Nirvana => 0.5
The Smiths => 0.5
Pixies => 0.28
Happy Mondays => 0.11
Jeff Buckley => 0.1
Jane's Addiction => 0.08
Soundgarden => 0.08
Violent Femmes => 0.06
The Replacements => 0.06
A Perfect Circle => 0.06
Sublime => 0.05
The Hush Sound => 0.04
The Get Up Kids => 0.03
Alternative Rock
Sex Pistols => 0.69
Fuel => 0.22
Descendents => 0.16
The Runaways => 0.16
The Clash => 0.16
Hot Water Music => 0.13
The Stooges => 0.09
Buzzcocks => 0.08
Misfits => 0.06
Ramones => 0.05
The Jam => 0.05
Dead Kennedys => 0.05
The Replacements => 0.05
Punk Rock
Cream => 1.67
Jimi Hendrix => 0.56
Thin Lizzy => 0.39
Led Zeppelin => 0.33
Deep Purple => 0.27
Ram Jam => 0.25
The Stooges => 0.11
Jim Morrison => 0.09
The Runaways => 0.08
Alice Cooper => 0.08
The Doors => 0.06
The Who => 0.04
Judas Priest => 0.02
Hard Rock
Joy Division => 0.56
Bauhaus => 0.028
The Police => 0.02
Siouxsie and Banshees => 0.02
Robert Smith => 0.006
Ultravox => 0.005
New Order => 0.005
My Bloody Valentine => 0.004
Sonic Youth => 0.004
Talking Heads => 0.003
Nick Cave => 0.002
The Cure => 0.001
David Byrne => 0.001
Post-punk
Other
Models
Weighted graph Album Genres
Krautrock
Psychedelic Rock
Experimental
Rock
1
1
1
Genre Affinity
Alternative Rock
Art Rock
Experimental Rock
Progressive Rock
29
47
31
27
Indie Rock
Genre Affinity
Indie Rock
Shoegazing
Alternative Rock
Dream Pop
22
25
24
12
Post-rock
Genre Affinity
Gothic Metal
Doom Metal
Black Metal
Heavy Metal
13
34
27
12
Stoner Metal
Clustering in Networks
Clustering in Networks
u1 u2 u3 u4 u5
u1 2
u2 3
u3 2
u4 2
u5 3
Spectral Clustering:
Using Eigenvectors of the Laplacian Matrix
−
u1 u2 u3 u4 u5
u1 1 0 0 1
u2 1 1 1 0
u3 0 1 0 1
u4 0 1 0 1
u5 1 0 1 1
=
u1 u2 u3 u4 u5
u1 2 -1 0 0 -1
u2 -1 3 -1 -1 0
u3 0 -1 2 0 -1
u4 0 -1 0 2 -1
u5 -1 0 -1 -1 3
Degree Matrix
Adjacency Matrix
(Similarity Matrix)
Laplacian Matrix
Clustering in Networks
Eigenvector: a vector (v) that by getting multiplied in matrix A
does not result in changing its direction (similar to being
multiplied by scalar λ)
u1 u2 u3 u4 u5
-0.7 0.3 -0.2 -0.1 0.7
-0.7 0.3 -0.2 -0.1 0.7
Spectral Clustering Codez
from sklearn.cluster import spectral_clustering
import numpy as np
A = [[0.0 for x in n] for x in n]
… # build adjacency matrix
res = spectral_clustering(np.matrix(A),
n_clusters)
# res -> list of cluster indices e.g. [1,1,0,5,…]
Spectral Clustering Results
Folk Rock
Country Rock
Blues
Folk
Country
Americana
Roots Rock
Blues Rock
Southern Rock
Power Metal
Progressive Metal
Symphonic Metal
Black Metal
Melodic Death Metal
Groove Metal
Nu Metal
Thrash Metal
Death Metal
Metalcore
Industrial Metal
Gothic Metal
Christian Metal
Doom Metal
Speed Metal
Alternative Rock
Indie Rock
New Wave
Synthpop
Electronica
Rock
R&B
Pop
Pop Rock
Funk
Soul
Heavy Metal
Hard Rock
Alternative Metal
50 Years of
Rock
(1990-2014)
50 Years of Rock 1990
1. alternative rock (69)
2. rock (59)
3. hard rock (48)
4. thrash metal (45)
5. heavy metal (36)
6. punk rock (25)
7. country (23)
This will stay at this spot pretty much
most of the remaining 25 years…
The Black Crowes
Shake Your Money Maker
50 Years of Rock 1991
1. alternative rock (82)
2. rock (68)
3. heavy metal (44)
4. hard rock (33)
5. thrash metal (30)
6. country (23)
7. pop (19)
Nirvana
Nevermind
50 Years of Rock 1992
1. alternative rock (83)
2. rock (56)
3. heavy metal (37)
4. hard rock (35)
5. thrash metal (29)
6. death metal (27)
7. indie rock (25)
A new genre is born….
Rage Against The Machine
Rage Against The Machine
50 Years of Rock 1993
1. alternative rock (125)
2. rock (72)
3. indie rock (39)
4. pop (28)
5. hard rock (28)
6. punk rock (28)
7. grunge (26)
It is all about the West Coast now,
especially Seattle…
Tool
Undertow
50 Years of Rock 1994
1. alternative rock (123)
2. rock (81)
3. indie rock (52)
4. punk rock (46) ↑
5. heavy metal (45)
6. hard rock (34)
7. country (24)
It is popular again and will be for the
next 10 years…
Soundgarden
Superknown
50 Years of Rock 1995
1. alternative rock (139)
2. rock (68)
3. punk rock (49)
4. heavy metal (46)
5. indie rock (44)
6. hard rock (37)
7. grunge (28)
The Smashing Pumpkins
Mellon Collie and the Infinite
SadnessAs it is clear here, grunge was really
undercurrent of what was going at
the time although first half of the 90s
known to be the grunge half-decade
50 Years of Rock 1996
1. alternative rock (132)
2. rock (98)
3. indie rock (60) ↑
4. punk rock (44)
5. heavy metal (42)
6. hard rock (28)
7. country (27)
Another generic label not fully
describing a genere…
Nick Cave and the Bad Seeds
Murder Ballads
50 Years of Rock 1997
1. alternative rock (119)
2. rock (78)
3. indie rock (60)
4. punk rock (45)
5. heavy metal (33)
6. alternative metal (23)
7. pop punk (22)
Radiohead
OK Computer
50 Years of Rock 1998
1. alternative rock (120)
2. rock (73)
3. indie rock (69)
4. punk rock (41)
5. heavy metal (37)
6. hard rock (29)
7. black metal (28)
Harder sound is more popular and
we have a new genre in the list
Neutral Milk Hotel
In the Aeroplane Over the Sea
50 Years of Rock 1999
1. alternative rock (146)
2. indie rock (73)
3. rock (72)
4. heavy metal (40)
5. hard rock (40)
6. alternative metal (34)
7. punk rock (33)
The Beta Band
The Beta Band
We really did not see Britpop making
an appearance… although the
impact and influence would exist
50 Years of Rock 2000
1. alternative rock (103)
2. indie rock (88)
3. rock (86)
4. punk rock (45)
5. heavy metal (41)
6. pop punk (37)
7. alternative metal (37)
Grandaddy
The Sophtware Slump
50 Years of Rock 2001
1. alternative rock (117)
2. rock (94)
3. indie rock (83)
4. alternative metal (49)
5. nu metal (45)
6. hard rock (44)
7. punk rock (36)
Another metal subgenre appearing
and will not be the last.
System of a Down
Toxicity
50 Years of Rock 2002
1. alternative rock (126)
2. indie rock (107)
3. rock (73)
4. heavy metal (54)
5. alternative metal (50)
6. hard rock (48)
7. punk rock (45)
Interpol
Turn on the Bright Lights
50 Years of Rock 2003
1. alternative rock (169)
2. indie rock (117)
3. rock (91)
4. alternative metal (63)
5. punk rock (53)
6. hard rock (51)
7. power metal (46)
The White Stripes
Elephant
50 Years of Rock 2004
1. alternative rock (155)
2. indie rock (130)
3. rock (96)
4. hard rock (64)
5. alternative metal (54)
6. punk rock (51)
7. pop punk (49)
Not much changing in the list…
Arcade Fire
Funeral
50 Years of Rock 2005
1. alternative rock (168)
2. indie rock (164)
3. rock (107)
4. hard rock (69)
5. heavy metal (57)
6. pop punk (55)
7. alternative metal (53)
Sufjan Stevens
Illinoise
50 Years of Rock 2006
1. alternative rock (188)
2. indie rock (179)
3. rock (105)
4. hard rock (70)
5. pop punk (58)
6. progressive metal (55)
7. power metal (46)
And yet another metal subgenre
hitting mainstream…
MUSE
Black Holes and Revelations
50 Years of Rock 2007
1. alternative rock (208)
2. indie rock (201)
3. rock (109)
4. hard rock (67)
5. alternative metal (64)
6. heavy metal (62)
7. pop punk (59)
Radiohead
In Rainbows
50 Years of Rock 2008
1. alternative rock (203)
2. indie rock (197)
3. rock (101)
4. hard rock (60)
5. pop (56)
6. metalcore (51)
7. heavy metal (50)
And we have another metal variant
that is here to stay…
Underoath
Lost in the Sound of Separation
50 Years of Rock 2009
1. alternative rock (200)
2. indie rock (194)
3. rock (72)
4. hard rock (68)
5. indie pop (59)
6. pop rock (59)
7. metalcore (56)
Softer sounds come back…
Animal Collective
Merriweather Post Pavilion
50 Years of Rock 2010
1. alternative rock (176)
2. indie rock (163)
3. rock (86)
4. hard rock (77)
5. indie pop (64)
6. heavy metal (62)
7. metalcore (61)
… and will be around… Arcade Fire
The Suburbs
50 Years of Rock 2011
1. alternative rock (169)
2. indie rock (162)
3. metalcore (63)
4. indie pop (60)
5. alternative metal (58)
6. heavy metal (57)
7. rock (57)
Tinariwen
Tassili
50 Years of Rock 2012
1. alternative rock (145)
2. indie rock (127)
3. rock (71)
4. hard rock (69)
5. alternative metal (50)
6. metalcore (49)
7. indie pop (45)
Goat
World Music
50 Years of Rock 2013
1. alternative rock (126)
2. indie rock (124)
3. rock (73)
4. hard rock (57)
5. synthpop (56)
6. metalcore (51)
7. indie pop (51)
And synthpop makes a come back
after 30 years or so…
Queens of the Stone Age
...Like Clockwork
50 Years of Rock 2014
1. alternative rock (108)
2. indie rock (106)
3. rock (50)
4. hard rock (48)
5. pop (41)
6. metalcore (41)
7. pop rock (40)
Mac DeMarco
Salad Days
Intelligent
Models
What is an Intelligent Model?
•Intelligent model is actually not a technically accurate term (but

we use it to denote shift in technique)
•Models learn from data instead of arbitrary definition
•Can sometimes even distinguish and filter noise
•Can represent a far more complex data relationships
•Examples include HMM, SVM and recently Deep Learning
•We review word2wec which uses Neural Network 

technique but not as deep as “Deep Learning”
word2vec Model
A very powerful model representing words (tokens) as vectors
based on their proximity to other words (tokens).
Once represented as vectors, all algebraic operations can be applied
to the vector representations.
word2vec Model
Skip-gram: a proximity-based probability model trained
using Neural Networks (Deep Learning)
Pink Floyd were an English rock band formed in London
X XX
word2vec Model
Break the
text to
sentences
Tokenise
each
sentence to
word/
phrases
Feed the
data to
engine
Store
the
Model
Training
Load the
model
Construct
the query
Run against
the model
(Filter noise)
Querying
word2vec Codz (Training)
import gensim
def train_and_save(multiword_dic_path, stopword_path,
corpus_files_glob_pattern, mode_file_name = ‘rock_music.w2v'):
tknsr =
dictionary_tokenization.DictionaryBasedMultiwordFinder(dictionary_path
=multiword_dic_path)
filtr = stopword_filtering.StopwordFilter(stopword_path)
tokenizze = lambda text: filtr.filter(tknsr.tokenise(text))
wvc =
gensim.models.Word2Vec(EnglishSentences(corpus_files_glob_pattern,
tokenizze, True), min_count=2)
wvc.save(mode_file_name)
word2vec Codz (Querying)
sims = word_sim.most_similar(positive=pos, negative=neg, topn=topn)
# pos is an array of positive e.g. [“Nick Mason”, “The Rolling Stones”]
# neg is an array of negative e.g. [“Pink Floyd”]
# result is an array of tuples of results for
# “Nick Mason” + “The Rolling Stones” - “Pink Floyd”
# the first entry of which is “Charlie Watts”
word2vec Demo
Wrap-up
Mystery missing artist
?
References
•All pictures from wikipedia.org used under Creative Commons
•Source of all data is from wikipedia.org collected online using a single call and then stored and processed
•Efficient Estimation of Word Representations in Vector Space. Mikolov et. al. http://arxiv.org/abs/1301.3781
•Gensim's word2vec
•networkx lib
•word2vec blog post (500K docs): Five crazy abstractions my Deep Learning word2vec model just did
•word2vec on Rock music blog: Daft Punk+Tool=Muse: word2vec model trained on a small Rock music corpus
•code for word2vec on wiki data
•Highcharts: highcharts
•word2vec paper: PDF
•Automatic real-time road marking recognition using a feature-driven approach PDF
•Video of the road marking recognition: here and here and here
•Future of Programming - Rise of the Scientific Programmer (and fall of the craftsman)

More Related Content

Similar to From power chords to the power of models

Classic rock bw
Classic rock bwClassic rock bw
Classic rock bwbwallwork
 
Rock Music - HubPages.com
Rock Music - HubPages.comRock Music - HubPages.com
Rock Music - HubPages.com
noxiouslaziness02
 
From Power Chord to the Power of Models - Oredev
From Power Chord to the Power of Models - OredevFrom Power Chord to the Power of Models - Oredev
From Power Chord to the Power of Models - Oredev
Ali Kheyrollahi
 
History of rock n roll answear all.docx
History of rock n roll answear all.docxHistory of rock n roll answear all.docx
History of rock n roll answear all.docx
write4
 
Rock genre research task
Rock genre research taskRock genre research task
Rock genre research task
bethhupchurchh
 
Synthwave
SynthwaveSynthwave
Synthwave
thorntonemily
 
Rock research powerpoint
Rock research powerpointRock research powerpoint
Rock research powerpoint
Heywoodmedia
 
PROG ROCK PRESENTATION.pptx
PROG ROCK PRESENTATION.pptxPROG ROCK PRESENTATION.pptx
PROG ROCK PRESENTATION.pptx
Christopher Baker
 
History of Rock Music
History of Rock MusicHistory of Rock Music
History of Rock Music
asmediac15
 
Music history 1970s
Music history 1970sMusic history 1970s
Music history 1970s
LisaKay Morton
 
Birth of Korean rock
Birth of Korean rockBirth of Korean rock
Birth of Korean rock
Anna Anya
 
Rock presentation
Rock presentationRock presentation
Rock presentation
heathergornall
 
Research into dubstep music
Research into dubstep musicResearch into dubstep music
Research into dubstep musictss000037
 
Pop and rock. nahernandez
Pop and rock. nahernandezPop and rock. nahernandez
Pop and rock. nahernandez
NathalyHernndez4
 
The history of rock music
The history of rock musicThe history of rock music
The history of rock musicjosepha0
 
Synthwave
SynthwaveSynthwave
Synthwave
shannon180
 
Pop And Rock. rafaeldieste
Pop And Rock. rafaeldiestePop And Rock. rafaeldieste
Pop And Rock. rafaeldiesteguest195a03
 

Similar to From power chords to the power of models (20)

Classic rock bw
Classic rock bwClassic rock bw
Classic rock bw
 
Rock Music - HubPages.com
Rock Music - HubPages.comRock Music - HubPages.com
Rock Music - HubPages.com
 
From Power Chord to the Power of Models - Oredev
From Power Chord to the Power of Models - OredevFrom Power Chord to the Power of Models - Oredev
From Power Chord to the Power of Models - Oredev
 
Punk rock
Punk rockPunk rock
Punk rock
 
History of rock n roll answear all.docx
History of rock n roll answear all.docxHistory of rock n roll answear all.docx
History of rock n roll answear all.docx
 
Rock genre research task
Rock genre research taskRock genre research task
Rock genre research task
 
Synthwave
SynthwaveSynthwave
Synthwave
 
Rock research powerpoint
Rock research powerpointRock research powerpoint
Rock research powerpoint
 
PROG ROCK PRESENTATION.pptx
PROG ROCK PRESENTATION.pptxPROG ROCK PRESENTATION.pptx
PROG ROCK PRESENTATION.pptx
 
History of Rock Music
History of Rock MusicHistory of Rock Music
History of Rock Music
 
100vinyls newkeynote
100vinyls newkeynote100vinyls newkeynote
100vinyls newkeynote
 
Music history 1970s
Music history 1970sMusic history 1970s
Music history 1970s
 
Birth of Korean rock
Birth of Korean rockBirth of Korean rock
Birth of Korean rock
 
Rock presentation
Rock presentationRock presentation
Rock presentation
 
Research into dubstep music
Research into dubstep musicResearch into dubstep music
Research into dubstep music
 
Rock 2
Rock 2Rock 2
Rock 2
 
Pop and rock. nahernandez
Pop and rock. nahernandezPop and rock. nahernandez
Pop and rock. nahernandez
 
The history of rock music
The history of rock musicThe history of rock music
The history of rock music
 
Synthwave
SynthwaveSynthwave
Synthwave
 
Pop And Rock. rafaeldieste
Pop And Rock. rafaeldiestePop And Rock. rafaeldieste
Pop And Rock. rafaeldieste
 

More from Ali Kheyrollahi

Autonomous agents with deep reinforcement learning - Oredev 2018
Autonomous agents with deep reinforcement learning - Oredev 2018Autonomous agents with deep reinforcement learning - Oredev 2018
Autonomous agents with deep reinforcement learning - Oredev 2018
Ali Kheyrollahi
 
Buildstuff - what do you need to know about RPC comeback
Buildstuff - what do you need to know about RPC comebackBuildstuff - what do you need to know about RPC comeback
Buildstuff - what do you need to know about RPC comeback
Ali Kheyrollahi
 
Deep learning for developers - oredev
Deep learning for developers - oredevDeep learning for developers - oredev
Deep learning for developers - oredev
Ali Kheyrollahi
 
Microservice Architecture at ASOS - DevSum 2017
Microservice Architecture at ASOS - DevSum 2017Microservice Architecture at ASOS - DevSum 2017
Microservice Architecture at ASOS - DevSum 2017
Ali Kheyrollahi
 
5 must have patterns for your microservice - techorama
5 must have patterns for your microservice - techorama5 must have patterns for your microservice - techorama
5 must have patterns for your microservice - techorama
Ali Kheyrollahi
 
Real time monitoring-alerting: storing 2Tb of logs a day in Elasticsearch
Real time monitoring-alerting: storing 2Tb of logs a day in ElasticsearchReal time monitoring-alerting: storing 2Tb of logs a day in Elasticsearch
Real time monitoring-alerting: storing 2Tb of logs a day in Elasticsearch
Ali Kheyrollahi
 
5 must-have patterns for your microservice - buildstuff
5 must-have patterns for your microservice - buildstuff5 must-have patterns for your microservice - buildstuff
5 must-have patterns for your microservice - buildstuff
Ali Kheyrollahi
 
From Hard Science to Baseless Opinions - Oredev
From Hard Science to Baseless Opinions  - OredevFrom Hard Science to Baseless Opinions  - Oredev
From Hard Science to Baseless Opinions - Oredev
Ali Kheyrollahi
 
5 must have patterns for your microservice
5 must have patterns for your microservice5 must have patterns for your microservice
5 must have patterns for your microservice
Ali Kheyrollahi
 
From hard science to baseless opinions
From hard science to baseless opinionsFrom hard science to baseless opinions
From hard science to baseless opinions
Ali Kheyrollahi
 
Microservice architecture at ASOS
Microservice architecture at ASOSMicroservice architecture at ASOS
Microservice architecture at ASOS
Ali Kheyrollahi
 
Us Elections 2016 - Iran Elections 2005
Us Elections 2016 - Iran Elections 2005Us Elections 2016 - Iran Elections 2005
Us Elections 2016 - Iran Elections 2005
Ali Kheyrollahi
 
5 Anti-Patterns in Api Design - NDC London 2016
5 Anti-Patterns in Api Design - NDC London 20165 Anti-Patterns in Api Design - NDC London 2016
5 Anti-Patterns in Api Design - NDC London 2016
Ali Kheyrollahi
 
5 Anti-Patterns in Api Design - buildstuff
5 Anti-Patterns in Api Design - buildstuff5 Anti-Patterns in Api Design - buildstuff
5 Anti-Patterns in Api Design - buildstuff
Ali Kheyrollahi
 
5 Anti-Patterns in API Design - DDD East Anglia 2015
5 Anti-Patterns in API Design - DDD East Anglia 20155 Anti-Patterns in API Design - DDD East Anglia 2015
5 Anti-Patterns in API Design - DDD East Anglia 2015
Ali Kheyrollahi
 
5 Anti-Patterns in API Design
5 Anti-Patterns in API Design5 Anti-Patterns in API Design
5 Anti-Patterns in API Design
Ali Kheyrollahi
 
Topic Modelling and APIs
Topic Modelling and APIsTopic Modelling and APIs
Topic Modelling and APIs
Ali Kheyrollahi
 
Http caching 101 and a bit of CacheCow
Http caching 101 and a bit of CacheCowHttp caching 101 and a bit of CacheCow
Http caching 101 and a bit of CacheCow
Ali Kheyrollahi
 

More from Ali Kheyrollahi (18)

Autonomous agents with deep reinforcement learning - Oredev 2018
Autonomous agents with deep reinforcement learning - Oredev 2018Autonomous agents with deep reinforcement learning - Oredev 2018
Autonomous agents with deep reinforcement learning - Oredev 2018
 
Buildstuff - what do you need to know about RPC comeback
Buildstuff - what do you need to know about RPC comebackBuildstuff - what do you need to know about RPC comeback
Buildstuff - what do you need to know about RPC comeback
 
Deep learning for developers - oredev
Deep learning for developers - oredevDeep learning for developers - oredev
Deep learning for developers - oredev
 
Microservice Architecture at ASOS - DevSum 2017
Microservice Architecture at ASOS - DevSum 2017Microservice Architecture at ASOS - DevSum 2017
Microservice Architecture at ASOS - DevSum 2017
 
5 must have patterns for your microservice - techorama
5 must have patterns for your microservice - techorama5 must have patterns for your microservice - techorama
5 must have patterns for your microservice - techorama
 
Real time monitoring-alerting: storing 2Tb of logs a day in Elasticsearch
Real time monitoring-alerting: storing 2Tb of logs a day in ElasticsearchReal time monitoring-alerting: storing 2Tb of logs a day in Elasticsearch
Real time monitoring-alerting: storing 2Tb of logs a day in Elasticsearch
 
5 must-have patterns for your microservice - buildstuff
5 must-have patterns for your microservice - buildstuff5 must-have patterns for your microservice - buildstuff
5 must-have patterns for your microservice - buildstuff
 
From Hard Science to Baseless Opinions - Oredev
From Hard Science to Baseless Opinions  - OredevFrom Hard Science to Baseless Opinions  - Oredev
From Hard Science to Baseless Opinions - Oredev
 
5 must have patterns for your microservice
5 must have patterns for your microservice5 must have patterns for your microservice
5 must have patterns for your microservice
 
From hard science to baseless opinions
From hard science to baseless opinionsFrom hard science to baseless opinions
From hard science to baseless opinions
 
Microservice architecture at ASOS
Microservice architecture at ASOSMicroservice architecture at ASOS
Microservice architecture at ASOS
 
Us Elections 2016 - Iran Elections 2005
Us Elections 2016 - Iran Elections 2005Us Elections 2016 - Iran Elections 2005
Us Elections 2016 - Iran Elections 2005
 
5 Anti-Patterns in Api Design - NDC London 2016
5 Anti-Patterns in Api Design - NDC London 20165 Anti-Patterns in Api Design - NDC London 2016
5 Anti-Patterns in Api Design - NDC London 2016
 
5 Anti-Patterns in Api Design - buildstuff
5 Anti-Patterns in Api Design - buildstuff5 Anti-Patterns in Api Design - buildstuff
5 Anti-Patterns in Api Design - buildstuff
 
5 Anti-Patterns in API Design - DDD East Anglia 2015
5 Anti-Patterns in API Design - DDD East Anglia 20155 Anti-Patterns in API Design - DDD East Anglia 2015
5 Anti-Patterns in API Design - DDD East Anglia 2015
 
5 Anti-Patterns in API Design
5 Anti-Patterns in API Design5 Anti-Patterns in API Design
5 Anti-Patterns in API Design
 
Topic Modelling and APIs
Topic Modelling and APIsTopic Modelling and APIs
Topic Modelling and APIs
 
Http caching 101 and a bit of CacheCow
Http caching 101 and a bit of CacheCowHttp caching 101 and a bit of CacheCow
Http caching 101 and a bit of CacheCow
 

Recently uploaded

Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
NYGGS Automation Suite
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
Google
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
TheSMSPoint
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
Fermin Galan
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
lorraineandreiamcidl
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
Adele Miller
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
Roshan Dwivedi
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
Philip Schwarz
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
Aftab Hussain
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)
Boni García
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket ManagementUtilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Mind IT Systems
 
AI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website CreatorAI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website Creator
Google
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Łukasz Chruściel
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Neo4j
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Aftab Hussain
 

Recently uploaded (20)

Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)APIs for Browser Automation (MoT Meetup 2024)
APIs for Browser Automation (MoT Meetup 2024)
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket ManagementUtilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
 
AI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website CreatorAI Genie Review: World’s First Open AI WordPress Website Creator
AI Genie Review: World’s First Open AI WordPress Website Creator
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
 

From power chords to the power of models

  • 1. From Power Chords to the Power of Models @aliostad Ali Kheyrollahi
  • 2. 8 Local pop music ? 9 Local pop music “Cheelee pom!” 10 Boney M “Rasputin” 11 Blondie “Heart of Glass”
  • 4.
  • 6. Data Source - Wiki 4,990,2794,990,279 English Articles 37,583,879 Articles
  • 7. Data Source - What about … 2012 http://money.cnn.com/2012/03/13/technology/encyclopedia-britannica-books/
  • 8. Data Source - Wiki vs Britannica Feng Zhu (assistant prof at Harvard): There has been lots of research on the accuracy of Wikipedia, and the results are mixed—some studies show it is just as good as the experts, others show [that] Wikipedia is not accurate at all. … the editors [of Britannica] are still not found to be more objective than the crowd in articles that are sufficiently revised.
  • 9. Data Source - Wikipedia in scholar papers 0 45000 90000 135000 180000 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Source: Google Scholar
  • 10. Data Source - Dark side of Wiki The Doggs: A mid-1960s British rock band, noted for barking while playing music Lord Byron kept a crocodile and a honey badger as a pet Ted and the Treble Tones, a rock band from Fresno with a 1959 hit single that appeared nowhere on the Billboard charts The Shaggs all female (sister) band Hacked-at drumbeats, whacked-around chords, songs that seem to have little or no meter to them… Zappa’s favourite…
  • 11.
  • 12. Data Source - Content vs. Data … or … Alt-country Alternative country … or … Alt.country … or … Alt. Country [mind the gap]
  • 13. Data Source - Content vs. Data comma slash new line period? no, greek middle dot U+00B7
  • 14. Data Source - Content vs. Data Hyphen U+002D figure dash U+2012 minus sign U+2015 em dash U+2014 en dash U+2013
  • 15. Data Source - DBpedia • http://wiki.dbpedia.org/ • bziped N-Triples or N-Quads RDF entries • Different datasets: • Titles • Categories • Infobox data • Short Abstracts • In the end was not used
  • 16. Data Acquisition - Wiki List of Rock Genres Rock Genres Rock Artists Store Store HTML Capture Links Store HTML Python scripts Postgres
  • 18. Data Exploration “I personally … literally just look at the screen, just like the matrix” Claudia Perlich, multi-award winner Data Scientist
  • 19. Data Exploration “… the dirty little secret that I have won all of them because I have found something wrong with the data… I would like to play around with dataset and get initimately familiar with dataset and its properties.“ Claudia Perlich
  • 23. 50 Years of Rock 1965 1. rock (23) 2. folk rock (15) 3. rock and roll (13) 4. folk (12) 5. rhythm and blues (12) 6. blues rock (10) 7. country (9) The Who My Generation “Rock” as genre was king with The likes of The Who and Beatles Rock and Roll, popular then, soon to disappear from the list
  • 24. 50 Years of Rock 1966 1. folk rock (25) 2. garage rock (21) 3. psychedelic rock (20) 4. rock (19) 5. folk (14) 6. protopunk (11) 7. pop (11) Bob Dylan Blonde on Blonde Folk Rock was popular all of the sixties with a resurgence in the early 70s This is obviously named retrospectively but along with Garage rock to influence punk
  • 25. 50 Years of Rock 1967 1. psychedelic rock (56) ↑ 2. folk rock (30) 3. pop (19) 4. folk (19) 5. rock (18) 6. garage rock (12) 7. psychedelic pop (12) The Beatles Sgt. Pepper's Lonely Hearts Club Band 67, a big year with Monterey Pop Festival, was all about psychedelia Even for psychedelic pop
  • 26. 50 Years of Rock 1968 1. psychedelic rock (70) ← 2. rock (29) 3. folk rock (26) 4. blues rock (25) 5. folk (18) 6. acid rock (18) 7. progressive rock (15) Jimi Hendrix Electric Ladyland No comment, listen to Jimi! Note Acid Rock and emergence of progressive rock
  • 27. 50 Years of Rock 1969 1. psychedelic rock (60) ← 2. rock (40) 3. folk rock (33) 4. hard rock (27) 5. folk (26) 6. progressive rock (25) 7. country (22) Led Zeppelin Led ZeppelinNot much changed… progrock moving up
  • 28. 50 Years of Rock 1970 1. rock (57) 2. folk rock (47) ↑ 3. psychedelic rock (44) !! 4. hard rock (40) ↑ 5. progressive rock (31) ↑ 6. country rock (25) 7. folk (24) The Velvet Underground Loaded Artists leaving psychedelia for a harder sound
  • 29. 50 Years of Rock 1971 1. rock (47) 2. hard rock (34) ↑ 3. folk rock (33) ! 4. progressive rock (33) ↑ 5. psychedelic rock (26) ! 6. folk (24) 7. country rock (18) CAN Tago Mago Progressive rock and hard rock moving up
  • 30. 50 Years of Rock 1972 1. rock (54) 2. folk rock (34) 3. progressive rock (29) 4. hard rock (29) 5. country rock (27) 6. folk (25) 7. country (20) Jethro Tull Thick as a Brick
  • 31. 50 Years of Rock 1973 1. rock (60) 2. hard rock (44) ↑ 3. progressive rock (40) 4. folk rock (29) 5. country (24) 6. glam rock (22) 7. country rock (22) Flashy dresses and make-up, bring it on! Pink Floyd Dark Side of the Moon
  • 32. 50 Years of Rock 1974 1. rock (58) 2. hard rock (52) 3. progressive rock (40) 4. glam rock (29) ↑ 5. folk rock (28) 6. blues rock (20) 7. country rock (20) Supertramp Crime of the Century
  • 33. 50 Years of Rock 1975 1. rock (53) 2. hard rock (52) 3. progressive rock (37) 4. folk rock (23) 5. country (20) 6. blues rock (20) 7. country rock (18) Patti Smith Horses
  • 34. 50 Years of Rock 1976 1. rock (71) 2. hard rock (56) 3. progressive rock (28) 4. heavy metal (19) 5. pop (19) 6. folk (18) 7. country (18) Did you notice?! Heavy Metal is now a thing… AC/DC High Voltage
  • 35. 50 Years of Rock 1977 1. rock (77) 2. hard rock (43) 3. progressive rock (29) 4. punk rock (21) 5. pop (19) 6. folk rock (15) 7. country rock (14) Oh yeah, punk rock in full swing and pretty much mainstream. Sex Pistols Never Mind the Bollocks
  • 36. 50 Years of Rock 1978 1. rock (77) 2. hard rock (40) 3. progressive rock (24) 4. new wave (23) 5. pop (21) 6. country (20) 7. punk rock (19) New wave with a more layered sound replacing punk rock Talking Heads More Songs About Buildings and Food
  • 37. 50 Years of Rock 1979 1. rock (68) 2. new wave (49) ↑ 3. post-punk (42) 4. hard rock (38) ! 5. country (22) 6. punk rock (20) 7. progressive rock (15) ! Only after two years of punk, we have post-punk… phew Gary Numan Replicas
  • 38. 50 Years of Rock 1980 1. rock (66) 2. new wave (57) 3. post-punk (49) 4. hard rock (38) 5. country (24) 6. heavy metal (22) 7. punk rock (21) Joy Division Closer
  • 39. 50 Years of Rock 1981 1. new wave (75) ↑ 2. rock (71) 3. post-punk (68) 4. hard rock (41) 5. synthpop (25) 6. heavy metal (21) 7. country (19) Another movement is already in motion with softer keyboard-based sound Klaus Nomi Klaus Nomi
  • 40. 50 Years of Rock 1982 1. new wave (79) 2. post-punk (61) 3. rock (61) 4. hard rock (37) 5. heavy metal (28) ↑ 6. synthpop (28) 7. hardcore punk (21) Iron Maiden Number of the Beast Other than new wave and post-punk, two completely different sounds growing, heavy metal on one hand and synth pop on the other…
  • 41. 50 Years of Rock 1983 1. rock (74) 2. new wave (71) 3. post-punk (45) ! 4. synthpop (39) ↑ 5. heavy metal (36) 6. hard rock (35) 7. pop rock (24) The Police Synchronicity
  • 42. 50 Years of Rock 1984 1. new wave (73) 2. rock (61) 3. post-punk (44) 4. heavy metal (37) 5. synthpop (37) 6. hard rock (26) 7. alternative rock (24) In case you haven’t noticed, New Wave has been on the top spot for a few years Echo and the Bunnymen Ocean Rain
  • 43. 50 Years of Rock 1985 1. new wave (66) 2. rock (56) 3. post-punk (45) 4. heavy metal (33) 5. pop rock (33) 6. alternative rock (28) ↑ 7. synthpop (28) ! Alternative Rock appearing… soon to label much of the rock music we have now… The Fall This Nation’s Saving Grace
  • 44. 50 Years of Rock 1986 1. rock (68) 2. new wave (48) ! 3. alternative rock (45) ↑ 4. heavy metal (42) 5. post-punk (40) 6. hard rock (31) 7. synthpop (28) Synthpop is over… only to come back after 25 years. The Smiths The Queen Is Dead
  • 45. 50 Years of Rock 1987 1. rock (62) 2. alternative rock (50) ↑ 3. hard rock (36) 4. thrash metal (35) 5. heavy metal (31) 6. new wave (29) ! 7. pop rock (24) Wow… where did this come from?! Testament The Legacy
  • 46. 50 Years of Rock 1988 1. alternative rock (59) ↑ 2. rock (58) 3. heavy metal (42) 4. hard rock (40) 5. thrash metal (33) 6. synthpop (24) 7. new wave (23) ! New wave is out of favour after almost 10 years… Sonic Youth Daydream Nation
  • 47. 50 Years of Rock 1989 1. rock (80) 2. alternative rock (67) 3. hard rock (41) 4. thrash metal (39) 5. heavy metal (34) 6. new wave (26) 7. pop (22) The Cure Disintegration
  • 50. Data Models Model Mathematical representation of a concept based on parameters that impact that concept • Rating of a native app • Stackoverflow score • Credit score • Fraud check
  • 51. Data Models Model Example (Average)
  • 52. Data Models Graph 101 Social Network Analysis and Graph Theory • Nodes/vertices and edges/lines • Directedness: • Directed • Undirected • Degree, InDegree/OutDegree • Weight A B
  • 53. Data Models Centrality API - Related artists 1 2 4 2 2 1 Same degree Different betweenness Degree
  • 54. Graph Codez import networkx as nx g = nx.Graph() g.add_edge(‘a’, ‘b’) g.add_edge(‘b’, ‘c’) … print len(g[‘b’]) # degree c = nx.betweenness_centrality(g, normalized=True) # c -> dictionary of node names and their score DiGraph()
  • 56. Data Models Cited Influence Howlin’ Wolf Captain Beefheart 1940 1964
  • 57. Data Models Cited Influence Most influential Rock Artists Based on out-degree The Beatles => 188 Black Sabbath => 127 Led Zeppelin => 118 Jimi Hendrix => 114 Bob Dylan => 94 Pink Floyd => 86 Iron Maiden => 77 Metallica => 77 The Rolling Stones => 66 The Beach Boys => 65 Neil Young => 63 Nirvana => 62 Slayer => 60 Queen => 59
  • 58. Data Models Cited Influence Most influential Rock Artists Based on Betweenness Centrality Jimi Hendrix => 53476.2014921 The Beatles => 47511.7957531 Bob Dylan => 38107.0298185 Led Zeppelin => 32701.7223273 Nirvana => 29733.9066836 Metallica => 29356.6009213 Queen => 28989.2844223 Robert Smith => 28880.670718 Elvis Presley => 28463.2891497 Slade => 27656.487307 Iron Maiden => 22449.6697023 Ramones => 22437.6112965 Rush => 21125.9481602 Neil Young => 19913.887522
  • 59. Data Models Cited Influence Most influential Artists Based on Betweenness Centrality Metallica => 566.06 Iron Maiden => 419.21 Corey Taylor => 146.0 Led Zeppelin => 122.73 Slipknot => 116.58 King Diamond => 94.7 Machine Head => 85.12 Rush => 70.41 Black Sabbath => 68.0 Van Halen => 54.56 Deep Purple => 53.5 Megadeth => 42.63 Guns N' Roses => 24.25 Heavy Metal Nirvana => 490.08 Muse => 114.5 Weezer => 97.33 Pixies => 94.17 Sonic Youth => 78.5 Rivers Cuomo => 69.5 Siouxsie and the Banshees => 51.67 The Smiths => 51.5 Jeff Buckley => 46.17 The Offspring => 43.0 Placebo => 42.0 My Chemical Romance => 34.0 The Smashing Pumpkins => 32.33 Alternative Rock Rush => 54.0 Marillion => 34.0 Pink Floyd => 33.0 Yes => 20.0 Porcupine Tree => 19.5 Dream Theater => 19.0 Chris Squire => 16.5 Primus => 15.0 Tool => 12.0 Mahavishnu Orchestra => 8.0 Geddy Lee => 7.0 Neil Peart => 5.0 Keith Emerson => 5.0 Progressive Rock
  • 61. Shaping a Model •Start with Literature survey •Draw parallels with similar domains •If you still cannot find one, define a new model •Try to refine your model with a quantitative measure if available
  • 62. Shaping Visionaries Model Visionary: Influencing vs. being influenced V: visionary factor deg+: out-degree deg-: in-degree e: the exponential function
  • 63. Shaping Visionaries Model Joni Mitchell => 44.0 Roxy Music => 31.0 Leonard Cohen => 31.0 Josh White => 31.0 Robert Johnson => 24.0 Etta James => 23.0 Discharge => 22.0 Sam Cooke => 21.0 Duke Ellington => 20.0 Marc Bolan => 19.0 Hank Marvin => 17.0 Jonathan Richman => 16.0 Scott Walker => 16.0 Testament => 16.0 The Police => 16.0 Skip James => 15.0 Albert King => 15.0 The Cure => 14.7151776469 Nico => 14.0
  • 64. Data Models Visionaries Model Most influential Artists in genres Radiohead => 15.0 Depeche Mode => 12.0 Morrissey => 11.0 Robert Smith => 11.0 The Replacements => 10.0 The Cure => 8.83 XTC => 8.0 Stone Temple Pilots => 6.0 Duran Duran => 6.0 Nick Cave => 6.0 Wipers => 5.0 Johnny Marr => 4.0 Jane's Addiction => 4.0 Alternative Rock The Stooges => 6.0 The Jam => 6.0 Sex Pistols => 4.05 Jawbreaker => 4.0 The Runaways => 4.0 The Dickies => 4.0 Circle Jerks => 3.0 Big Black => 3.0 Patti Smith => 3.0 Elvis Costello => 3.0 The Only Ones => 3.0 Kevin Seconds => 3.0 Descendents => 2.94 Punk Rock Captain Beefheart => 7.0 The Magic Band => 6.0 Genesis => 5.0 Scott Walker => 5.0 Scott Engel => 3.0 Talk Talk => 3.0 Sparks => 2.0 Richard Wright => 2.0 Roxy Music => 1.84 Pink Floyd => 1.49 Mark Hollis => 1.0 Nico => 1.0 Lou Reed => 1.0 Art Rock Dave Lombardo => 11.0 Napalm Death => 8.0 Morbid Angel => 3.68 Possessed => 3.31 Carcass => 2.21 Immolation => 2.0 Vulcano => 2.0 Hellhammer => 2.0 Autopsy => 1.47 Repulsion => 1.1 Entombed => 1.1 Impetigo => 1.0 Nihilist => 1.0 Death Metal
  • 65. Shaping Impact Model Imact: Influencing many in a short span Imp: impact factor deg+: out-degree L: longevity (initial continuous years active)
  • 66. Shaping Impact Model Imact: Influencing many in a short span Cream => 3.33 Sex Pistols => 2.19 Joy Division => 2.0 Jimi Hendrix => 1.83 The Beatles => 1.59 The Smiths => 1.19 Big Star => 1.19 Parliament => 1.11 Nirvana => 1.0 Ritchie Valens => 0.78 Led Zeppelin => 0.73 Traffic => 0.67
  • 67. Shaping Impact Model Imact: Influencing many in a short span in a specific genre Nirvana => 0.5 The Smiths => 0.5 Pixies => 0.28 Happy Mondays => 0.11 Jeff Buckley => 0.1 Jane's Addiction => 0.08 Soundgarden => 0.08 Violent Femmes => 0.06 The Replacements => 0.06 A Perfect Circle => 0.06 Sublime => 0.05 The Hush Sound => 0.04 The Get Up Kids => 0.03 Alternative Rock Sex Pistols => 0.69 Fuel => 0.22 Descendents => 0.16 The Runaways => 0.16 The Clash => 0.16 Hot Water Music => 0.13 The Stooges => 0.09 Buzzcocks => 0.08 Misfits => 0.06 Ramones => 0.05 The Jam => 0.05 Dead Kennedys => 0.05 The Replacements => 0.05 Punk Rock Cream => 1.67 Jimi Hendrix => 0.56 Thin Lizzy => 0.39 Led Zeppelin => 0.33 Deep Purple => 0.27 Ram Jam => 0.25 The Stooges => 0.11 Jim Morrison => 0.09 The Runaways => 0.08 Alice Cooper => 0.08 The Doors => 0.06 The Who => 0.04 Judas Priest => 0.02 Hard Rock Joy Division => 0.56 Bauhaus => 0.028 The Police => 0.02 Siouxsie and Banshees => 0.02 Robert Smith => 0.006 Ultravox => 0.005 New Order => 0.005 My Bloody Valentine => 0.004 Sonic Youth => 0.004 Talking Heads => 0.003 Nick Cave => 0.002 The Cure => 0.001 David Byrne => 0.001 Post-punk
  • 69. Weighted graph Album Genres Krautrock Psychedelic Rock Experimental Rock 1 1 1
  • 70. Genre Affinity Alternative Rock Art Rock Experimental Rock Progressive Rock 29 47 31 27 Indie Rock
  • 71. Genre Affinity Indie Rock Shoegazing Alternative Rock Dream Pop 22 25 24 12 Post-rock
  • 72. Genre Affinity Gothic Metal Doom Metal Black Metal Heavy Metal 13 34 27 12 Stoner Metal
  • 74. Clustering in Networks u1 u2 u3 u4 u5 u1 2 u2 3 u3 2 u4 2 u5 3 Spectral Clustering: Using Eigenvectors of the Laplacian Matrix − u1 u2 u3 u4 u5 u1 1 0 0 1 u2 1 1 1 0 u3 0 1 0 1 u4 0 1 0 1 u5 1 0 1 1 = u1 u2 u3 u4 u5 u1 2 -1 0 0 -1 u2 -1 3 -1 -1 0 u3 0 -1 2 0 -1 u4 0 -1 0 2 -1 u5 -1 0 -1 -1 3 Degree Matrix Adjacency Matrix (Similarity Matrix) Laplacian Matrix
  • 75. Clustering in Networks Eigenvector: a vector (v) that by getting multiplied in matrix A does not result in changing its direction (similar to being multiplied by scalar λ) u1 u2 u3 u4 u5 -0.7 0.3 -0.2 -0.1 0.7 -0.7 0.3 -0.2 -0.1 0.7
  • 76. Spectral Clustering Codez from sklearn.cluster import spectral_clustering import numpy as np A = [[0.0 for x in n] for x in n] … # build adjacency matrix res = spectral_clustering(np.matrix(A), n_clusters) # res -> list of cluster indices e.g. [1,1,0,5,…]
  • 77. Spectral Clustering Results Folk Rock Country Rock Blues Folk Country Americana Roots Rock Blues Rock Southern Rock Power Metal Progressive Metal Symphonic Metal Black Metal Melodic Death Metal Groove Metal Nu Metal Thrash Metal Death Metal Metalcore Industrial Metal Gothic Metal Christian Metal Doom Metal Speed Metal Alternative Rock Indie Rock New Wave Synthpop Electronica Rock R&B Pop Pop Rock Funk Soul Heavy Metal Hard Rock Alternative Metal
  • 79. 50 Years of Rock 1990 1. alternative rock (69) 2. rock (59) 3. hard rock (48) 4. thrash metal (45) 5. heavy metal (36) 6. punk rock (25) 7. country (23) This will stay at this spot pretty much most of the remaining 25 years… The Black Crowes Shake Your Money Maker
  • 80. 50 Years of Rock 1991 1. alternative rock (82) 2. rock (68) 3. heavy metal (44) 4. hard rock (33) 5. thrash metal (30) 6. country (23) 7. pop (19) Nirvana Nevermind
  • 81. 50 Years of Rock 1992 1. alternative rock (83) 2. rock (56) 3. heavy metal (37) 4. hard rock (35) 5. thrash metal (29) 6. death metal (27) 7. indie rock (25) A new genre is born…. Rage Against The Machine Rage Against The Machine
  • 82. 50 Years of Rock 1993 1. alternative rock (125) 2. rock (72) 3. indie rock (39) 4. pop (28) 5. hard rock (28) 6. punk rock (28) 7. grunge (26) It is all about the West Coast now, especially Seattle… Tool Undertow
  • 83. 50 Years of Rock 1994 1. alternative rock (123) 2. rock (81) 3. indie rock (52) 4. punk rock (46) ↑ 5. heavy metal (45) 6. hard rock (34) 7. country (24) It is popular again and will be for the next 10 years… Soundgarden Superknown
  • 84. 50 Years of Rock 1995 1. alternative rock (139) 2. rock (68) 3. punk rock (49) 4. heavy metal (46) 5. indie rock (44) 6. hard rock (37) 7. grunge (28) The Smashing Pumpkins Mellon Collie and the Infinite SadnessAs it is clear here, grunge was really undercurrent of what was going at the time although first half of the 90s known to be the grunge half-decade
  • 85. 50 Years of Rock 1996 1. alternative rock (132) 2. rock (98) 3. indie rock (60) ↑ 4. punk rock (44) 5. heavy metal (42) 6. hard rock (28) 7. country (27) Another generic label not fully describing a genere… Nick Cave and the Bad Seeds Murder Ballads
  • 86. 50 Years of Rock 1997 1. alternative rock (119) 2. rock (78) 3. indie rock (60) 4. punk rock (45) 5. heavy metal (33) 6. alternative metal (23) 7. pop punk (22) Radiohead OK Computer
  • 87. 50 Years of Rock 1998 1. alternative rock (120) 2. rock (73) 3. indie rock (69) 4. punk rock (41) 5. heavy metal (37) 6. hard rock (29) 7. black metal (28) Harder sound is more popular and we have a new genre in the list Neutral Milk Hotel In the Aeroplane Over the Sea
  • 88. 50 Years of Rock 1999 1. alternative rock (146) 2. indie rock (73) 3. rock (72) 4. heavy metal (40) 5. hard rock (40) 6. alternative metal (34) 7. punk rock (33) The Beta Band The Beta Band We really did not see Britpop making an appearance… although the impact and influence would exist
  • 89. 50 Years of Rock 2000 1. alternative rock (103) 2. indie rock (88) 3. rock (86) 4. punk rock (45) 5. heavy metal (41) 6. pop punk (37) 7. alternative metal (37) Grandaddy The Sophtware Slump
  • 90. 50 Years of Rock 2001 1. alternative rock (117) 2. rock (94) 3. indie rock (83) 4. alternative metal (49) 5. nu metal (45) 6. hard rock (44) 7. punk rock (36) Another metal subgenre appearing and will not be the last. System of a Down Toxicity
  • 91. 50 Years of Rock 2002 1. alternative rock (126) 2. indie rock (107) 3. rock (73) 4. heavy metal (54) 5. alternative metal (50) 6. hard rock (48) 7. punk rock (45) Interpol Turn on the Bright Lights
  • 92. 50 Years of Rock 2003 1. alternative rock (169) 2. indie rock (117) 3. rock (91) 4. alternative metal (63) 5. punk rock (53) 6. hard rock (51) 7. power metal (46) The White Stripes Elephant
  • 93. 50 Years of Rock 2004 1. alternative rock (155) 2. indie rock (130) 3. rock (96) 4. hard rock (64) 5. alternative metal (54) 6. punk rock (51) 7. pop punk (49) Not much changing in the list… Arcade Fire Funeral
  • 94. 50 Years of Rock 2005 1. alternative rock (168) 2. indie rock (164) 3. rock (107) 4. hard rock (69) 5. heavy metal (57) 6. pop punk (55) 7. alternative metal (53) Sufjan Stevens Illinoise
  • 95. 50 Years of Rock 2006 1. alternative rock (188) 2. indie rock (179) 3. rock (105) 4. hard rock (70) 5. pop punk (58) 6. progressive metal (55) 7. power metal (46) And yet another metal subgenre hitting mainstream… MUSE Black Holes and Revelations
  • 96. 50 Years of Rock 2007 1. alternative rock (208) 2. indie rock (201) 3. rock (109) 4. hard rock (67) 5. alternative metal (64) 6. heavy metal (62) 7. pop punk (59) Radiohead In Rainbows
  • 97. 50 Years of Rock 2008 1. alternative rock (203) 2. indie rock (197) 3. rock (101) 4. hard rock (60) 5. pop (56) 6. metalcore (51) 7. heavy metal (50) And we have another metal variant that is here to stay… Underoath Lost in the Sound of Separation
  • 98. 50 Years of Rock 2009 1. alternative rock (200) 2. indie rock (194) 3. rock (72) 4. hard rock (68) 5. indie pop (59) 6. pop rock (59) 7. metalcore (56) Softer sounds come back… Animal Collective Merriweather Post Pavilion
  • 99. 50 Years of Rock 2010 1. alternative rock (176) 2. indie rock (163) 3. rock (86) 4. hard rock (77) 5. indie pop (64) 6. heavy metal (62) 7. metalcore (61) … and will be around… Arcade Fire The Suburbs
  • 100. 50 Years of Rock 2011 1. alternative rock (169) 2. indie rock (162) 3. metalcore (63) 4. indie pop (60) 5. alternative metal (58) 6. heavy metal (57) 7. rock (57) Tinariwen Tassili
  • 101. 50 Years of Rock 2012 1. alternative rock (145) 2. indie rock (127) 3. rock (71) 4. hard rock (69) 5. alternative metal (50) 6. metalcore (49) 7. indie pop (45) Goat World Music
  • 102. 50 Years of Rock 2013 1. alternative rock (126) 2. indie rock (124) 3. rock (73) 4. hard rock (57) 5. synthpop (56) 6. metalcore (51) 7. indie pop (51) And synthpop makes a come back after 30 years or so… Queens of the Stone Age ...Like Clockwork
  • 103. 50 Years of Rock 2014 1. alternative rock (108) 2. indie rock (106) 3. rock (50) 4. hard rock (48) 5. pop (41) 6. metalcore (41) 7. pop rock (40) Mac DeMarco Salad Days
  • 105. What is an Intelligent Model? •Intelligent model is actually not a technically accurate term (but
 we use it to denote shift in technique) •Models learn from data instead of arbitrary definition •Can sometimes even distinguish and filter noise •Can represent a far more complex data relationships •Examples include HMM, SVM and recently Deep Learning •We review word2wec which uses Neural Network 
 technique but not as deep as “Deep Learning”
  • 106. word2vec Model A very powerful model representing words (tokens) as vectors based on their proximity to other words (tokens). Once represented as vectors, all algebraic operations can be applied to the vector representations.
  • 107. word2vec Model Skip-gram: a proximity-based probability model trained using Neural Networks (Deep Learning) Pink Floyd were an English rock band formed in London X XX
  • 108. word2vec Model Break the text to sentences Tokenise each sentence to word/ phrases Feed the data to engine Store the Model Training Load the model Construct the query Run against the model (Filter noise) Querying
  • 109. word2vec Codz (Training) import gensim def train_and_save(multiword_dic_path, stopword_path, corpus_files_glob_pattern, mode_file_name = ‘rock_music.w2v'): tknsr = dictionary_tokenization.DictionaryBasedMultiwordFinder(dictionary_path =multiword_dic_path) filtr = stopword_filtering.StopwordFilter(stopword_path) tokenizze = lambda text: filtr.filter(tknsr.tokenise(text)) wvc = gensim.models.Word2Vec(EnglishSentences(corpus_files_glob_pattern, tokenizze, True), min_count=2) wvc.save(mode_file_name)
  • 110. word2vec Codz (Querying) sims = word_sim.most_similar(positive=pos, negative=neg, topn=topn) # pos is an array of positive e.g. [“Nick Mason”, “The Rolling Stones”] # neg is an array of negative e.g. [“Pink Floyd”] # result is an array of tuples of results for # “Nick Mason” + “The Rolling Stones” - “Pink Floyd” # the first entry of which is “Charlie Watts”
  • 114. References •All pictures from wikipedia.org used under Creative Commons •Source of all data is from wikipedia.org collected online using a single call and then stored and processed •Efficient Estimation of Word Representations in Vector Space. Mikolov et. al. http://arxiv.org/abs/1301.3781 •Gensim's word2vec •networkx lib •word2vec blog post (500K docs): Five crazy abstractions my Deep Learning word2vec model just did •word2vec on Rock music blog: Daft Punk+Tool=Muse: word2vec model trained on a small Rock music corpus •code for word2vec on wiki data •Highcharts: highcharts •word2vec paper: PDF •Automatic real-time road marking recognition using a feature-driven approach PDF •Video of the road marking recognition: here and here and here •Future of Programming - Rise of the Scientific Programmer (and fall of the craftsman)