The case study discusses the challenges of combining data from various music-related APIs and the importance of storytelling and shared understanding in achieving this integration. It emphasizes the role of the Semantic Web, specifically the use of MusicBrainz IDs and SPARQL queries, to extract precise data about artists from multiple sources. Additionally, the document outlines lessons learned regarding performance optimization in Drupal and server configuration for better data management.

1. Case Study – ABC Dig Music
David Peterson @davidseth #ddu2011 http://www.flickr.com/photos/soyignatius/

2. David Peterson
@davidseth

3. Challenge
Create a snapshot of an artist

4. • Known Data
Combining • Data in the Wild

5. Problem
<xml>
<track>
<title>Purple Rain</title>
<artistName>Prince</artistName>
</track>
</xml>

6. Into

7. It’s all about Storytelling…

8. Shared Understanding
• Can’t tell a story if the other
person doesn’t get what we
mean
• Or even speak the same
language

9. • The story matters
• ... but ...
• You never really have all the information you
need, whether big or small

10. You Just don’t Always Know
• Someone else knows more than you
• How to find it?

11. One Exception

12. Semantic Web
• Core idea
– you never really know the entire picture
• This is a “good thing”
• Freedom

13. Open World
Closed World
http://www.flickr.com/photos/almasryalyoum_e/

18. “If the graph of people is
cool, imagine a graph of
everything”
- Dries Buytaert

19. Open Data

20. Facebook?
• A little late to the party ;)

21. Finding a Solution
• Which APIs to use
• Which APIs can we use
• How can we combine data from multiple
sources
• How can we automate it

22. The Curse of too Much
• There are over 50 APIs listed on
programmableweb.com
• Too many to look into
• Each has its own API methods and return data
formats
– JSON, XML, RSS, RDF !!!

23. Take your Pick
• APIs everywhere
– BBC Music
– Discogs
– Last.fm
– MusicBrainz
– Yahoo Music
– Flickr
– Youtube
– The Hype Machine

24. Finding the Key
• One common feature was the usage of a
MusicBrainz ID
– Last.fm
– Discogs
– Freebase
– Wikipedia/Dbpedia
– BBC

25. Eureka!
• Great, now all I had to do was use the
MusicBrainz API to look up the ID and I was
done. Easy...
• :(
• The search API sucked. It returned too many
fuzzy results
• crap

26. Back to the Future
• This is where the Semantic Web enters the
picture
– All that stuff about story telling
– Shared understanding
– URIs (web links)

27. SPARQL
Think of it as Google with a WHERE clause

28. SELECT ?artist WHERE {
?artist foaf:name "Prince"@en .
?artist a <http://dbpedia.org/ontology/MusicalArtist>.
}

29. SELECT ?artist ?bio ?url ?album WHERE {
?artist foaf:name "Prince"@en .
?artist a <http://dbpedia.org/ontology/MusicalArtist> .
?artist dbpedia2:abstract ?bio .
?artist foaf:page ?url .
OPTIONAL {
?album <http://dbpedia.org/ontology/artist> ?artist .
?album rdfs:label "Purple Rain"@en .
}
}
LIMIT 1

30. Pinpoint Results
• This returns ONE result
• “exactly” what we are looking for (or nothing!)

31. {170d193a-845c-479f-980e-bef15710653e}
http://www.flickr.com/photos/riseofphoenix/

32. {070d193a-845c-479f-980e-bef15710653e}
http://www.flickr.com/photos/angeldew/

33. Raw Data
• Not too pretty to look at
• But computers LOVE this stuff

34. So, what do we get
• Disambiguation
• MusicBrainz ID
• Discography
• Related Artists
• Official homepage
• Bio
• Credit card details (sometime in 2012)

35. The Rosetta Stone
• MusicBrainz ID is our key to the wild web of
APIs
• Wikipedia URL is the key to Semantic Web
• One happy family :)
http://www.flickr.com/photos/vportals/

37. Take a look
[browser]

38. Hindsight is 20/20
... or lessons learned

39. Drupal Sucks
• Drupal performance, what performance?

40. Don’t use Drupal
• To get the best performance out of Drupal 6,
don’t use Drupal 6!

41. Pressflow
• Key patches and enhancements
• Releases mirror official Drupal releases
• Big players are using it
– Drupal.org
– ABC
– Music labels
– Newspapers

42. Start your Engines
MySQL base install is ... lacking
• MyISAM == slow
• Use Percona XtraDB
• ... or ... InnoDB

43. Reduce your footprint
• APC
– PHP app is compiled & cached in memory
• Memcached

44. Search
• Drupal’s built in search can be a dawg
• Solr
– Much faster search
– Offers faceting
– Can become a platform in its own right

45. A Fresh Coat of Paint
• Varnish
– Last but certainly not least
– Up to millions of hits per hour

46. Performance Optimisations
• Switch host to Linode
• Two-server architecture - db server and app
server
• Master-slave relationship for mysql
• Migrated Drupal to Pressflow
• Changed tables to InnoDB
• Varnish for serving pages
• memcached for caching
• Setup munin to monitor servers

47. An Alternate Future
RDFaViewEntitFielMediStreaMongo

48. An Alternate Future
• Drupal 7
– RDFa
– Views 3
– Entities
– Fields
– Media Module
– Stream Wrappers
– MongoDB