Maintaining a community based music website with open source software

2,945 views

Published on

The Newcastle Music Directory is a community based music website that covers the music scene in and around Newcastle, NSW. This covers a region roughly from the
Hawkesbury River in the South to Taree in the North, and West to Merriwa in the upper Hunter Valley. The site publishes over 1500 local events per month on its gig guide and lists details of over 4000 local bands and artists and over 650 venues from the region. The site has constantly had to be upgraded and updated to reflect the changing internet technology over its 10 year history.

  • Be the first to comment

  • Be the first to like this

Maintaining a community based music website with open source software

  1. 1. Maintaining a Community Based Music Website with Open Source Software Open Source Developers Conference 27 th November 2009
  2. 2. In the beginning… <ul><li>Newcastle has a thriving music scene </li></ul><ul><li>Members of the community lacked cohesion </li></ul><ul><li>No support from commercial radio </li></ul><ul><li>.com excitement in 1999 </li></ul>
  3. 5. Newcastle Music Directory www.newcastlemusic.com <ul><li>Online since 1999 </li></ul><ul><li>Over 4500 local artists listed </li></ul><ul><li>Focuses on music from the Hunter Valley and Central Coast </li></ul><ul><li>Site has over 8000 pages </li></ul><ul><li>Updated frequently </li></ul><ul><li>Sections include: </li></ul><ul><ul><li>News </li></ul></ul><ul><ul><li>Links </li></ul></ul><ul><ul><li>Videos </li></ul></ul><ul><ul><li>Gig Guide </li></ul></ul>
  4. 6. TE Archive <ul><li>Online since mid 2002 </li></ul><ul><li>Site is located at 15min.org (ie. 15 minutes of fame) </li></ul><ul><li>Features articles from the TE liftout from May 1998 to March 2006 </li></ul><ul><li>Over 13,000 pages </li></ul><ul><li>Over 6,500 images </li></ul>
  5. 7. Geographical Focus <ul><li>Focuses solely on musical artists and websites from Newcastle and the surrounding regions </li></ul><ul><li>Allows the site to be comprehensive </li></ul>
  6. 8. Geographical Focus
  7. 9. Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul>
  8. 10. Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul>
  9. 11. Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul><ul><li>CDs are cheaper </li></ul>
  10. 12. Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul><ul><li>CDs are cheaper </li></ul><ul><li>Support the economy of your local region </li></ul>
  11. 13. Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul><ul><li>CDs are cheaper </li></ul><ul><li>Support the economy of your local region </li></ul><ul><li>Local musicians don’t mime </li></ul>
  12. 15. Why Do This For Free? <ul><li>From Wednesday’s keynote talk: </li></ul><ul><ul><li>Volunteers are “not motivated by financial or material gain” </li></ul></ul><ul><ul><li>Volunteers are “motivated by ideology” </li></ul></ul>
  13. 16. Volunteering <ul><li>Many local musos effectively volunteer their time for their passion </li></ul><ul><li>Much of the local music industry is run on goodwill </li></ul>
  14. 17. Aims <ul><li>Focus strictly on the region around Newcastle, to the point of actively excluding news from other areas </li></ul>
  15. 18. Aims <ul><li>Provide the Newcastle region with a musical identity of its own that bands and artists are proud to identify with. </li></ul>
  16. 19. Aims <ul><li>Provide the region with a comprehensive list of current gigs and events. </li></ul>
  17. 20. Aims <ul><li>Provide a comprehensive list of links to locally based websites that relate to the music scene. </li></ul>
  18. 21. Aims <ul><li>Provide up-to-date and informative profiles for as many local bands and artists as possible. </li></ul>
  19. 22. Aims <ul><li>Provide maps and information about local pubs, clubs and musical venues. </li></ul>
  20. 23. Aims <ul><li>Provide an archive historical information about the local music scene for long term reference. </li></ul>
  21. 24. Aims <ul><li>Provide a unified voice to highlight the musical talent of the region to people from other areas. </li></ul>
  22. 25. Aims <ul><li>Meet and collaborate with other people interested in the local music scene </li></ul>
  23. 26. Looking Back … <ul><li>Started compiling links and information about the music scene in 1999 </li></ul><ul><li>Have published over 20,000 pages about the local music industry on the internet </li></ul><ul><li>Have volunteered over 7000 hours to the local music scene </li></ul><ul><li>Received an award from in October 2005 for volunteer services to the local music industry </li></ul><ul><li>Involvement in local music festivals and competitions </li></ul><ul><li>All while working full time as an Engineer </li></ul>
  24. 27. Volunteers Award in 2005
  25. 29. Effects of the Regional Focus <ul><li>Never any argument about what should be included on the site </li></ul><ul><li>Restricted market – only really of relevance to the people of Newcastle (~1million people) </li></ul><ul><li>Allows the website to become vwery comprehensive (~80% coverage?) </li></ul>
  26. 30. Types of Information <ul><li>Bands and Artists </li></ul><ul><li>Radio </li></ul><ul><li>Larger Bands and Orchestras - Marching Bands, Orchestras, Choirs </li></ul><ul><li>Services </li></ul><ul><li>Music Shops </li></ul>
  27. 31. Types of Information <ul><li>Recording Studios Mobile Discos </li></ul><ul><li>Music Organisations </li></ul><ul><li>Record Labels </li></ul><ul><li>Production </li></ul><ul><li>Tuition </li></ul>
  28. 32. Types of Information <ul><li>Media </li></ul><ul><li>Events </li></ul><ul><li>Managers and Agents </li></ul><ul><li>Sound and Lighting </li></ul><ul><li>Promotion </li></ul><ul><li>Photography </li></ul>
  29. 33. Sources of Information <ul><li>Free webhosting services such as Geocities etc. </li></ul><ul><li>Mp3.com </li></ul><ul><li>Their own fully hosted domains </li></ul><ul><li>mp3.com.au </li></ul><ul><li>www.abc.net.au/newcastle as part of the ABC Newcastle Music Awards </li></ul><ul><li>PureVolume.com </li></ul><ul><li>MySpace.com </li></ul><ul><li>Blogs on services such as blogger.com </li></ul><ul><li>Press releases </li></ul>
  30. 34. Sources of Information <ul><li>Newspapers </li></ul><ul><li>flickr.com </li></ul><ul><li>Triple J Unearthed </li></ul><ul><li>YouTube.com </li></ul><ul><li>Booking agents and promoters websites </li></ul><ul><li>Facebook </li></ul><ul><li>Google Calendars </li></ul><ul><li>Twitter </li></ul><ul><li>ABC Contribute </li></ul><ul><li>Word of mouth </li></ul>
  31. 35. Link to External Websites <ul><li>Often the artist will maintain a bio/gig guide etc on another website </li></ul><ul><li>newcastlemusic.com has no login system </li></ul>
  32. 36. Automated Aggregation of Information <ul><li>More recently websites have allowed access to information via: </li></ul><ul><ul><li>RSS </li></ul></ul><ul><ul><li>API </li></ul></ul><ul><ul><li>Copying information from external websites (with the artists permission) </li></ul></ul>
  33. 37. Embedded YouTube Video <ul><li>Bandwidth supplied by YouTube </li></ul><ul><li>Information retrieved from YouTube via RSS </li></ul><ul><ul><li>Playtime </li></ul></ul><ul><ul><li>Author </li></ul></ul><ul><ul><li>Upload date </li></ul></ul><ul><li>About 2000 videos are embedded in the site </li></ul>
  34. 38. RSS <ul><li>Really Simple Syndycation </li></ul><ul><li>Many websites automatically generate RSS “feeds” when you update them </li></ul><ul><ul><li>Livejournal </li></ul></ul><ul><ul><li>MySpace </li></ul></ul><ul><ul><li>Blogspot </li></ul></ul><ul><ul><li>Youtube </li></ul></ul><ul><ul><li>Flickr </li></ul></ul><ul><ul><li>etc. </li></ul></ul><ul><li>RSS allows people to subscribe to your blog </li></ul><ul><li>Alerts people that you have updated your website without them visiting the site. </li></ul><ul><li>Is a form of XML (eXtensible Markup Language) </li></ul>
  35. 39. Artists Blog Entries <ul><li>Automatically brought into the site daily via RSS feeds </li></ul><ul><li>Subscribed to over 3500 myspace blogs from the Newcastle music industry </li></ul><ul><li>Provides an excellent source of up to date news about the local music scene </li></ul>
  36. 40. The Gig Guide <ul><li>Published around 1300 local events per month </li></ul><ul><li>An additional 500 events from outside the region on the artists profiles </li></ul><ul><li>Significant effort is made to filter out the local gigs </li></ul>
  37. 41. Number of Local Gigs Published
  38. 43. The Gig Guide Database <ul><li>Required a complete list of local venues and artists </li></ul><ul><li>Compilation of such lists took over 5 years </li></ul><ul><li>Required significant local knowledge </li></ul><ul><li>Currently contains: </li></ul><ul><ul><li>650 venues </li></ul></ul><ul><ul><li>4500 artists </li></ul></ul>
  39. 44. Venues from the Newcastle Region Indexed using Google Maps
  40. 45. Venue Locations
  41. 46. Venue Locations
  42. 47. Sources of Gig Information <ul><li>Booking agents </li></ul><ul><li>Venues </li></ul><ul><li>Band Homepages </li></ul><ul><li>Google Calendar </li></ul><ul><li>Various other sources </li></ul>
  43. 48. Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul>
  44. 49. Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul>
  45. 50. Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul><ul><li>Deleting cancelled gigs from the database </li></ul>
  46. 51. Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul><ul><li>Deleting cancelled gigs from the database </li></ul><ul><li>Ensuring that the correct band and venue are linked to a particular event in the database </li></ul>
  47. 52. Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul><ul><li>Deleting cancelled gigs from the database </li></ul><ul><li>Ensuring that the correct band and venue are linked to a particular event in the database </li></ul><ul><li>Specifying the time of an event if not otherwise stated. </li></ul>
  48. 53. Updating Gig Information <ul><li>Identify source </li></ul><ul><li>Disable all future gigs from that source </li></ul><ul><li>For each gig: </li></ul><ul><ul><li>Re-enable if already found in database </li></ul></ul><ul><ul><li>Add new event if not found </li></ul></ul><ul><li>This allows for cancellations/modifications </li></ul>
  49. 54. Maintaining the Gig Guide <ul><li>Database maintenance is not performed on the webserver </li></ul><ul><li>Uploaded twice a week </li></ul><ul><li>Much more stable and maintainable as a result </li></ul>
  50. 55. Gig Archive <ul><li>Over 50000 gigs from around the region </li></ul><ul><li>All gigs link to artist profile and map of venue </li></ul><ul><li>Archiving started in November 2006 </li></ul>
  51. 56. Open Information <ul><li>Local booking agents and promoters are protective about their business practices </li></ul><ul><ul><li>Which venues they represent </li></ul></ul><ul><ul><li>How often venues have entertainment </li></ul></ul><ul><ul><li>Contact details for venues and artists </li></ul></ul>
  52. 57. Open Information <ul><li>Local booking agents and promoters are protective about their business practices </li></ul><ul><ul><li>Which venues they represent </li></ul></ul><ul><ul><li>How often venues have entertainment </li></ul></ul><ul><ul><li>Contact details for venues and artists </li></ul></ul>
  53. 58. Bringing the Community Together
  54. 59. Open Source Software <ul><li>All websites built on open source software </li></ul><ul><ul><li>Apache </li></ul></ul><ul><ul><li>MySQL </li></ul></ul><ul><ul><li>Wordpress </li></ul></ul><ul><ul><li>Various libraries (Magpie RSS etc etc) </li></ul></ul>
  55. 60. Open Source Software <ul><li>Allowed the management these websites and the data behind them on a shoestring </li></ul>
  56. 61. In for the long haul … <ul><li>Open Source technologies allow flexibility and continuity </li></ul>
  57. 62. The Stickybeak <ul><li>In use on all websites since 2002 </li></ul><ul><li>Released in 2004 </li></ul><ul><li>Operates in a similar way to Google Analytics </li></ul><ul><li>Available from www.stickybeak.com </li></ul>
  58. 63. Newcastle Music Directory Page Requests Per Day
  59. 64. Silverchair Profile Requests
  60. 65. Artists Profile Requests
  61. 66. Section Usage
  62. 67. Sitemap
  63. 68. Web vs Print Media <ul><li>Internet and print media very different types of content </li></ul><ul><li>Advertising in music print media often considered content on music websites </li></ul><ul><li>Case Study: Reverb Street Press </li></ul>
  64. 69. Reverb Street Press Content <ul><li>Street Press content was analysed during late 2007 </li></ul><ul><li>Approximately 10% of the space was devoted to local artists </li></ul><ul><li>Over 40% of the page space contained advertisements </li></ul>
  65. 72. Advertising
  66. 73. Conclusion <ul><li>Open source technologies have allowed the Newcastle Music Directory to grow in an adaptive and economical manner over a sustained period of development. </li></ul>
  67. 74. Conclusion <ul><li>The result is a website which provides relevant information about the music scene to the people of the Newcastle region. </li></ul>
  68. 75. <ul><li>http://www.newcastlemusic.com/ </li></ul><ul><li>twitter.com/newcastlemusic </li></ul>
  69. 77. Local Music on the Internet <ul><li>Library of over 13,000 songs </li></ul><ul><li>Sourced from: </li></ul><ul><ul><li>JJJ Unearthed </li></ul></ul><ul><ul><li>Mp3.com.au </li></ul></ul><ul><ul><li>MySpace </li></ul></ul><ul><ul><li>ABC 1233 Awards </li></ul></ul><ul><li>All are indexed by: </li></ul><ul><ul><li>File type </li></ul></ul><ul><ul><li>Stream/download </li></ul></ul><ul><ul><li>Stereo/mono </li></ul></ul><ul><ul><li>Play Time </li></ul></ul><ul><ul><li>File Size </li></ul></ul><ul><ul><li>Sample Rate </li></ul></ul><ul><ul><li>Bit Rate </li></ul></ul>
  70. 78. Newcastle Survey February 2007 515 Download MySpace 2033 Stream MySpace 1598 Download mp3.com.au 846 Download Triple J 1508 Stream ABC Count Type Source
  71. 79. Bit Rates For Files in Newcastle Survey Feb 2007
  72. 80. ID3 Tags <ul><li>Allows attributes to be embedded in sound files </li></ul><ul><li>Possible attributes include: </li></ul><ul><ul><li>Artist name </li></ul></ul><ul><ul><li>Song name </li></ul></ul><ul><ul><li>Genre </li></ul></ul><ul><ul><li>Filesize, bit rate, sample rate </li></ul></ul><ul><li>Important for players to identify songs </li></ul><ul><li>Important for listeners managing mp3 libraries </li></ul><ul><li>Search for “id3 tag editor” on the internet and you will find many freely available programs to manipulate these tags </li></ul><ul><li>More information at http://www.id3.org/ </li></ul>
  73. 81. ID3 Tags in the Local Music Survey <ul><li>81% of songs did not have the “title” tag set </li></ul><ul><li>78% of songs did not have the “artist” tag set </li></ul><ul><li>85% of songs did not set the “genre” tag </li></ul>
  74. 82. ABC Music Awards Archive <ul><li>ABC Local Radio changed their website in March 2008 </li></ul><ul><li>Deleted their 2500 page archive of previous music awards entries </li></ul><ul><li>They have allowed me to republish the information </li></ul>
  75. 83. Referrals From Google

×