1.
Maintaining a Community Based Music Website with Open Source Software Open Source Developers Conference 27 th November 2009

2.
In the beginning… <ul><li>Newcastle has a thriving music scene </li></ul><ul><li>Members of the community lacked cohesion </li></ul><ul><li>No support from commercial radio </li></ul><ul><li>.com excitement in 1999 </li></ul>

5.
Newcastle Music Directory www.newcastlemusic.com <ul><li>Online since 1999 </li></ul><ul><li>Over 4500 local artists listed </li></ul><ul><li>Focuses on music from the Hunter Valley and Central Coast </li></ul><ul><li>Site has over 8000 pages </li></ul><ul><li>Updated frequently </li></ul><ul><li>Sections include: </li></ul><ul><ul><li>News </li></ul></ul><ul><ul><li>Links </li></ul></ul><ul><ul><li>Videos </li></ul></ul><ul><ul><li>Gig Guide </li></ul></ul>

6.
TE Archive <ul><li>Online since mid 2002 </li></ul><ul><li>Site is located at 15min.org (ie. 15 minutes of fame) </li></ul><ul><li>Features articles from the TE liftout from May 1998 to March 2006 </li></ul><ul><li>Over 13,000 pages </li></ul><ul><li>Over 6,500 images </li></ul>

7.
Geographical Focus <ul><li>Focuses solely on musical artists and websites from Newcastle and the surrounding regions </li></ul><ul><li>Allows the site to be comprehensive </li></ul>

8.
Geographical Focus

9.
Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul>

10.
Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul>

11.
Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul><ul><li>CDs are cheaper </li></ul>

12.
Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul><ul><li>CDs are cheaper </li></ul><ul><li>Support the economy of your local region </li></ul>

13.
Why “Local” Music? <ul><li>You can talk to the musicians after the gig </li></ul><ul><li>The musicians will play weekly/monthly in your area </li></ul><ul><li>CDs are cheaper </li></ul><ul><li>Support the economy of your local region </li></ul><ul><li>Local musicians don’t mime </li></ul>

15.
Why Do This For Free? <ul><li>From Wednesday’s keynote talk: </li></ul><ul><ul><li>Volunteers are “not motivated by financial or material gain” </li></ul></ul><ul><ul><li>Volunteers are “motivated by ideology” </li></ul></ul>

16.
Volunteering <ul><li>Many local musos effectively volunteer their time for their passion </li></ul><ul><li>Much of the local music industry is run on goodwill </li></ul>

17.
Aims <ul><li>Focus strictly on the region around Newcastle, to the point of actively excluding news from other areas </li></ul>

18.
Aims <ul><li>Provide the Newcastle region with a musical identity of its own that bands and artists are proud to identify with. </li></ul>

19.
Aims <ul><li>Provide the region with a comprehensive list of current gigs and events. </li></ul>

20.
Aims <ul><li>Provide a comprehensive list of links to locally based websites that relate to the music scene. </li></ul>

21.
Aims <ul><li>Provide up-to-date and informative profiles for as many local bands and artists as possible. </li></ul>

22.
Aims <ul><li>Provide maps and information about local pubs, clubs and musical venues. </li></ul>

23.
Aims <ul><li>Provide an archive historical information about the local music scene for long term reference. </li></ul>

24.
Aims <ul><li>Provide a unified voice to highlight the musical talent of the region to people from other areas. </li></ul>

25.
Aims <ul><li>Meet and collaborate with other people interested in the local music scene </li></ul>

26.
Looking Back … <ul><li>Started compiling links and information about the music scene in 1999 </li></ul><ul><li>Have published over 20,000 pages about the local music industry on the internet </li></ul><ul><li>Have volunteered over 7000 hours to the local music scene </li></ul><ul><li>Received an award from in October 2005 for volunteer services to the local music industry </li></ul><ul><li>Involvement in local music festivals and competitions </li></ul><ul><li>All while working full time as an Engineer </li></ul>

27.
Volunteers Award in 2005

29.
Effects of the Regional Focus <ul><li>Never any argument about what should be included on the site </li></ul><ul><li>Restricted market – only really of relevance to the people of Newcastle (~1million people) </li></ul><ul><li>Allows the website to become vwery comprehensive (~80% coverage?) </li></ul>

30.
Types of Information <ul><li>Bands and Artists </li></ul><ul><li>Radio </li></ul><ul><li>Larger Bands and Orchestras - Marching Bands, Orchestras, Choirs </li></ul><ul><li>Services </li></ul><ul><li>Music Shops </li></ul>

31.
Types of Information <ul><li>Recording Studios Mobile Discos </li></ul><ul><li>Music Organisations </li></ul><ul><li>Record Labels </li></ul><ul><li>Production </li></ul><ul><li>Tuition </li></ul>

32.
Types of Information <ul><li>Media </li></ul><ul><li>Events </li></ul><ul><li>Managers and Agents </li></ul><ul><li>Sound and Lighting </li></ul><ul><li>Promotion </li></ul><ul><li>Photography </li></ul>

33.
Sources of Information <ul><li>Free webhosting services such as Geocities etc. </li></ul><ul><li>Mp3.com </li></ul><ul><li>Their own fully hosted domains </li></ul><ul><li>mp3.com.au </li></ul><ul><li>www.abc.net.au/newcastle as part of the ABC Newcastle Music Awards </li></ul><ul><li>PureVolume.com </li></ul><ul><li>MySpace.com </li></ul><ul><li>Blogs on services such as blogger.com </li></ul><ul><li>Press releases </li></ul>

34.
Sources of Information <ul><li>Newspapers </li></ul><ul><li>flickr.com </li></ul><ul><li>Triple J Unearthed </li></ul><ul><li>YouTube.com </li></ul><ul><li>Booking agents and promoters websites </li></ul><ul><li>Facebook </li></ul><ul><li>Google Calendars </li></ul><ul><li>Twitter </li></ul><ul><li>ABC Contribute </li></ul><ul><li>Word of mouth </li></ul>

35.
Link to External Websites <ul><li>Often the artist will maintain a bio/gig guide etc on another website </li></ul><ul><li>newcastlemusic.com has no login system </li></ul>

36.
Automated Aggregation of Information <ul><li>More recently websites have allowed access to information via: </li></ul><ul><ul><li>RSS </li></ul></ul><ul><ul><li>API </li></ul></ul><ul><ul><li>Copying information from external websites (with the artists permission) </li></ul></ul>

37.
Embedded YouTube Video <ul><li>Bandwidth supplied by YouTube </li></ul><ul><li>Information retrieved from YouTube via RSS </li></ul><ul><ul><li>Playtime </li></ul></ul><ul><ul><li>Author </li></ul></ul><ul><ul><li>Upload date </li></ul></ul><ul><li>About 2000 videos are embedded in the site </li></ul>

38.
RSS <ul><li>Really Simple Syndycation </li></ul><ul><li>Many websites automatically generate RSS “feeds” when you update them </li></ul><ul><ul><li>Livejournal </li></ul></ul><ul><ul><li>MySpace </li></ul></ul><ul><ul><li>Blogspot </li></ul></ul><ul><ul><li>Youtube </li></ul></ul><ul><ul><li>Flickr </li></ul></ul><ul><ul><li>etc. </li></ul></ul><ul><li>RSS allows people to subscribe to your blog </li></ul><ul><li>Alerts people that you have updated your website without them visiting the site. </li></ul><ul><li>Is a form of XML (eXtensible Markup Language) </li></ul>

39.
Artists Blog Entries <ul><li>Automatically brought into the site daily via RSS feeds </li></ul><ul><li>Subscribed to over 3500 myspace blogs from the Newcastle music industry </li></ul><ul><li>Provides an excellent source of up to date news about the local music scene </li></ul>

40.
The Gig Guide <ul><li>Published around 1300 local events per month </li></ul><ul><li>An additional 500 events from outside the region on the artists profiles </li></ul><ul><li>Significant effort is made to filter out the local gigs </li></ul>

41.
Number of Local Gigs Published

43.
The Gig Guide Database <ul><li>Required a complete list of local venues and artists </li></ul><ul><li>Compilation of such lists took over 5 years </li></ul><ul><li>Required significant local knowledge </li></ul><ul><li>Currently contains: </li></ul><ul><ul><li>650 venues </li></ul></ul><ul><ul><li>4500 artists </li></ul></ul>

44.
Venues from the Newcastle Region Indexed using Google Maps

45.
Venue Locations

46.
Venue Locations

47.
Sources of Gig Information <ul><li>Booking agents </li></ul><ul><li>Venues </li></ul><ul><li>Band Homepages </li></ul><ul><li>Google Calendar </li></ul><ul><li>Various other sources </li></ul>

48.
Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul>

49.
Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul>

50.
Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul><ul><li>Deleting cancelled gigs from the database </li></ul>

51.
Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul><ul><li>Deleting cancelled gigs from the database </li></ul><ul><li>Ensuring that the correct band and venue are linked to a particular event in the database </li></ul>

52.
Maintaining the Gig Guide <ul><li>Identifying and removing duplicate entries </li></ul><ul><li>Catching misspelled band or venue names </li></ul><ul><li>Deleting cancelled gigs from the database </li></ul><ul><li>Ensuring that the correct band and venue are linked to a particular event in the database </li></ul><ul><li>Specifying the time of an event if not otherwise stated. </li></ul>

53.
Updating Gig Information <ul><li>Identify source </li></ul><ul><li>Disable all future gigs from that source </li></ul><ul><li>For each gig: </li></ul><ul><ul><li>Re-enable if already found in database </li></ul></ul><ul><ul><li>Add new event if not found </li></ul></ul><ul><li>This allows for cancellations/modifications </li></ul>

54.
Maintaining the Gig Guide <ul><li>Database maintenance is not performed on the webserver </li></ul><ul><li>Uploaded twice a week </li></ul><ul><li>Much more stable and maintainable as a result </li></ul>

55.
Gig Archive <ul><li>Over 50000 gigs from around the region </li></ul><ul><li>All gigs link to artist profile and map of venue </li></ul><ul><li>Archiving started in November 2006 </li></ul>

56.
Open Information <ul><li>Local booking agents and promoters are protective about their business practices </li></ul><ul><ul><li>Which venues they represent </li></ul></ul><ul><ul><li>How often venues have entertainment </li></ul></ul><ul><ul><li>Contact details for venues and artists </li></ul></ul>

57.
Open Information <ul><li>Local booking agents and promoters are protective about their business practices </li></ul><ul><ul><li>Which venues they represent </li></ul></ul><ul><ul><li>How often venues have entertainment </li></ul></ul><ul><ul><li>Contact details for venues and artists </li></ul></ul>

58.
Bringing the Community Together

59.
Open Source Software <ul><li>All websites built on open source software </li></ul><ul><ul><li>Apache </li></ul></ul><ul><ul><li>MySQL </li></ul></ul><ul><ul><li>Wordpress </li></ul></ul><ul><ul><li>Various libraries (Magpie RSS etc etc) </li></ul></ul>

60.
Open Source Software <ul><li>Allowed the management these websites and the data behind them on a shoestring </li></ul>

61.
In for the long haul … <ul><li>Open Source technologies allow flexibility and continuity </li></ul>

62.
The Stickybeak <ul><li>In use on all websites since 2002 </li></ul><ul><li>Released in 2004 </li></ul><ul><li>Operates in a similar way to Google Analytics </li></ul><ul><li>Available from www.stickybeak.com </li></ul>

63.
Newcastle Music Directory Page Requests Per Day

64.
Silverchair Profile Requests

65.
Artists Profile Requests

66.
Section Usage

67.
Sitemap

68.
Web vs Print Media <ul><li>Internet and print media very different types of content </li></ul><ul><li>Advertising in music print media often considered content on music websites </li></ul><ul><li>Case Study: Reverb Street Press </li></ul>

69.
Reverb Street Press Content <ul><li>Street Press content was analysed during late 2007 </li></ul><ul><li>Approximately 10% of the space was devoted to local artists </li></ul><ul><li>Over 40% of the page space contained advertisements </li></ul>

72.
Advertising

73.
Conclusion <ul><li>Open source technologies have allowed the Newcastle Music Directory to grow in an adaptive and economical manner over a sustained period of development. </li></ul>

74.
Conclusion <ul><li>The result is a website which provides relevant information about the music scene to the people of the Newcastle region. </li></ul>

75.
<ul><li>http://www.newcastlemusic.com/ </li></ul><ul><li>twitter.com/newcastlemusic </li></ul>

77.
Local Music on the Internet <ul><li>Library of over 13,000 songs </li></ul><ul><li>Sourced from: </li></ul><ul><ul><li>JJJ Unearthed </li></ul></ul><ul><ul><li>Mp3.com.au </li></ul></ul><ul><ul><li>MySpace </li></ul></ul><ul><ul><li>ABC 1233 Awards </li></ul></ul><ul><li>All are indexed by: </li></ul><ul><ul><li>File type </li></ul></ul><ul><ul><li>Stream/download </li></ul></ul><ul><ul><li>Stereo/mono </li></ul></ul><ul><ul><li>Play Time </li></ul></ul><ul><ul><li>File Size </li></ul></ul><ul><ul><li>Sample Rate </li></ul></ul><ul><ul><li>Bit Rate </li></ul></ul>

78.
Newcastle Survey February 2007 515 Download MySpace 2033 Stream MySpace 1598 Download mp3.com.au 846 Download Triple J 1508 Stream ABC Count Type Source

79.
Bit Rates For Files in Newcastle Survey Feb 2007

80.
ID3 Tags <ul><li>Allows attributes to be embedded in sound files </li></ul><ul><li>Possible attributes include: </li></ul><ul><ul><li>Artist name </li></ul></ul><ul><ul><li>Song name </li></ul></ul><ul><ul><li>Genre </li></ul></ul><ul><ul><li>Filesize, bit rate, sample rate </li></ul></ul><ul><li>Important for players to identify songs </li></ul><ul><li>Important for listeners managing mp3 libraries </li></ul><ul><li>Search for “id3 tag editor” on the internet and you will find many freely available programs to manipulate these tags </li></ul><ul><li>More information at http://www.id3.org/ </li></ul>

81.
ID3 Tags in the Local Music Survey <ul><li>81% of songs did not have the “title” tag set </li></ul><ul><li>78% of songs did not have the “artist” tag set </li></ul><ul><li>85% of songs did not set the “genre” tag </li></ul>

82.
ABC Music Awards Archive <ul><li>ABC Local Radio changed their website in March 2008 </li></ul><ul><li>Deleted their 2500 page archive of previous music awards entries </li></ul><ul><li>They have allowed me to republish the information </li></ul>

83.
Referrals From Google