This document provides an introduction to the Last.fm API. It describes what Last.fm is and how it collects music listening data through scrobbling. It then outlines the features of the Last.fm API, including the types of data that can be accessed, common methods, authentication requirements, response formats, and error handling. Examples of real applications built using the API are also presented.
The document discusses the proposed Stop Online Piracy Act (SOPA) and Protect IP Act (PIPA) which aimed to expand the ability of US authorities and copyright holders to block online access to websites involved in copyright infringement and piracy. However, the acts faced widespread opposition from internet companies and users who argued they posed a serious threat to free speech, innovation, and the internet economy. Major websites conducted protests that helped lead to the bills being postponed and eventually abandoned in 2012 without passing into law.
Mine and Jason's talk from BarCampBrighton 5 about building brilliant APIs that bloody work.
Probably won't make sense without having been there..
Any questions give myself (@ben_c) or Jason (@fry15) a shout on twitter.
The document discusses a presentation on YUI's foundational CSS tools - the CSS Reset, CSS Fonts, and CSS Grids. It provides background on each tool, what they do to normalize styles across browsers, and how to use them. It encourages attendees to download exercises from the listed URL to follow along with hands-on examples of integrating the tools.
From 2008 but still relevant - setting Purpose, social sites, identity, reputation, roles, campaigns, etiquette, rituals/conversation diary and subgroups/swarms. Strategies ask the questions, so this is question based.
Social Tags and Music Information Retrieval (Part II)Paul Lamere
Part 2 of the slides for the Social Tags and Music Information Retrieval Tutorial - Abstract: Social Tags are free text labels that are applied to items such as artists, playlists and songs. These tags have the potential to have a positive impact on music information retrieval research. In this tutorial we describe the state of the art in commercial and research social tagging systems for music. We explore some of the motivations for tagging. We describe the factors that affect the quantity and quality of collected tags. We present a toolkit that MIR researchers can use to harvest and process tags. We look at how tags are collected and used in current commercial and research systems. We explore some of the issues and problems that are encountered when using tags. We present current MIR-related research centered on social tags and suggest possible areas of exploration for future resear
Last.fm provides a public API that allows developers to access information about music and social features on Last.fm. The API includes over 100 methods to retrieve data on artists, albums, tracks, tags, users and more. It outputs data in XML or JSON formats and supports authentication for private data. Common methods include artist.getImages, album.getTags, and track.search. Developers need an API key to access the API.
The document discusses the proposed Stop Online Piracy Act (SOPA) and Protect IP Act (PIPA) which aimed to expand the ability of US authorities and copyright holders to block online access to websites involved in copyright infringement and piracy. However, the acts faced widespread opposition from internet companies and users who argued they posed a serious threat to free speech, innovation, and the internet economy. Major websites conducted protests that helped lead to the bills being postponed and eventually abandoned in 2012 without passing into law.
Mine and Jason's talk from BarCampBrighton 5 about building brilliant APIs that bloody work.
Probably won't make sense without having been there..
Any questions give myself (@ben_c) or Jason (@fry15) a shout on twitter.
The document discusses a presentation on YUI's foundational CSS tools - the CSS Reset, CSS Fonts, and CSS Grids. It provides background on each tool, what they do to normalize styles across browsers, and how to use them. It encourages attendees to download exercises from the listed URL to follow along with hands-on examples of integrating the tools.
From 2008 but still relevant - setting Purpose, social sites, identity, reputation, roles, campaigns, etiquette, rituals/conversation diary and subgroups/swarms. Strategies ask the questions, so this is question based.
Social Tags and Music Information Retrieval (Part II)Paul Lamere
Part 2 of the slides for the Social Tags and Music Information Retrieval Tutorial - Abstract: Social Tags are free text labels that are applied to items such as artists, playlists and songs. These tags have the potential to have a positive impact on music information retrieval research. In this tutorial we describe the state of the art in commercial and research social tagging systems for music. We explore some of the motivations for tagging. We describe the factors that affect the quantity and quality of collected tags. We present a toolkit that MIR researchers can use to harvest and process tags. We look at how tags are collected and used in current commercial and research systems. We explore some of the issues and problems that are encountered when using tags. We present current MIR-related research centered on social tags and suggest possible areas of exploration for future resear
Last.fm provides a public API that allows developers to access information about music and social features on Last.fm. The API includes over 100 methods to retrieve data on artists, albums, tracks, tags, users and more. It outputs data in XML or JSON formats and supports authentication for private data. Common methods include artist.getImages, album.getTags, and track.search. Developers need an API key to access the API.
Lucene 4 was recently released with key features including improved language analysis support for over 30 languages, faster indexing and storage capabilities, and pluggable similarity models. The large and diverse Lucene community is always testing to improve performance and relevance. Lucene remains an open source option for text search in applications beyond traditional search engines.
This document summarizes Neville Li's work at Spotify developing real-time data streaming applications using Storm. It describes Spotify's large data volumes, how Storm is used to process streaming data at Spotify, details of a social listening topology, and lessons learned around development processes, language choices, and deployment.
Slides for the presentation I made at ClueCon 21 on the experimental RED support in WebRTC, and how we've started tinkering with it in Janus. The presentation also addresses a more generic overview on audio features in WebRTC.
In April 2015, Apache Geode (incubating) was born from Pivotal’s GemFire, the distributed in-memory database. However, the donation of over 1M LOC was just the beginning of the journey. In this talk we discuss how the GemFire engineering team has adapted their development infrastructure, processes, and culture to embrace the “Apache Way". We present lessons learned and best practices for new and incubating open source projects in areas of initial code submission, IP clearance, governance policies, code review, and community building. We discuss the challenges the team faced and how we changed internal communication and software design processes to a community-driven model. In particular, we highlight effective strategies for growing a project community and embracing new members. Finally, we show how changing to the open source model has increased both productivity and quality.
My (quite boring) slides on what we needed to do in Janus to support multiple streams of the same type (e.g., 3 video streams) on the same PeerConnection.
This document discusses scaling issues faced when launching a new feature for 24 million Xbox360 users. It describes profiling traffic, identifying unnecessary API calls, custom logging, and contingencies used such as disabling writes and suppressing calls. The launch was monitored closely with various tools and teams to handle any failures.
Downtown SF Lucene/Solr Meetup: Developing Scalable Search for User Generated...Lucidworks
This document summarizes Sony's development of a scalable search system using Apache Solr for user-generated content on the PlayStation platform. PlayStation users can easily share media like broadcasts, screenshots, and videos to third-party networks. Sony built a SolrCloud-based system to provide a central place to search for this content across millions of users. The system uses three Solr clusters to handle different media types and supports over 20 languages. It processes over 1 billion search requests per day with low latency.
This document provides a summary of various technology topics including:
1. Articles about PHP, JavaScript, and using web services.
2. Information about new versions of Visual Studio 2010, Office 2010, and their features.
3. Summaries of new APIs from Baidu and resources on web development.
Power to the People: Manipulating SharePoint with Client-Side JavaScriptPeterBrunone
First delivered at SharePoint Saturday Houston, this presentation provides a brief background of the use of JavaScript in SharePoint before forging ahead into the capabilities of jQuery and the Client-Side Object Model.
This document discusses HTML5 apps and different approaches for building cross-platform apps, including PhoneGap, W3C Widgets, and other runtimes. It notes that HTML5 apps can access device APIs like the camera and GPS through frameworks like PhoneGap. It also summarizes pros and cons of different approaches, such as PhoneGap providing access to many device APIs but not being a standard, and W3C Widgets being very simple to create but only supporting a few APIs.
Proof of concept description for music (track) recommendations with NoSQL graph database Neo4j. This is not a Machine Learning-related PoC using Collaborative Filtering since the latter is just a piece in the songs similarity puzzle (user preferences cooccurrence distance). This PoC demonstrates the speed of Neo4j, its secret: the data structure, no SQL "select" nor "joins" are needed. Neo4j uses native graph storage and native graph processing whcih benefits from traversal performance.
This document summarizes a last.fm web application project that analyzes music listening data and creates visual dashboards. It uses Python, Bottle framework, and MySQL to build APIs for track analysis, album analysis, user similarity, and network analysis. Database design started with MongoDB but switched to MySQL with partitioning for better performance of joins. Algorithms like cosine similarity and Hadoop/EMR were used for tasks like calculating user music compatibility and trends. The front-end was created with Foundation and visualizations with Canvas JS and HERE Maps API.
First Beat Media - Tehnologije na velikim projektima #tnt3SICEF
The document discusses the technology and trends at an online entertainment media company called FirstBeatMedia. It describes some of their large projects including operating 3 online dating platforms and managing 500 websites with 40 million page views. It provides details on their hardware infrastructure including 200 physical servers and 200 virtual machines. It also discusses the software they use such as PHP, MySQL, caching tools like MemcacheD and Redis, and development tools including Git, testing frameworks and Phabricator. It concludes by noting the growing importance of mobile and the need to develop API-centric mobile applications for Android and iOS.
The document provides an overview of a hackathon being led by Simon Bennetts on extending the OWASP Zed Attack Proxy (ZAP) tool. The plan is to give an overview of how to extend ZAP, discuss potential topics to cover such as implementing scripts, scan rules, and extensions, and then have hands-on hacking sessions with assistance from Simon. Simon outlines many possible topics for discussion, including the ZAP project structure, development environment, documentation, scripting, active and passive scan rules, extensions, and features or fixes to work on.
Crowdmix is a social network focused on music that runs on an event-driven architecture. It allows users to share music content from various streaming services across crowds. The system is designed for scalability to support millions of users. Key components include Kafka for event streaming, Spark/EMR for batch processing, Elasticsearch for search, and Cassandra/Redshift for materialized views and analytics. Music matching is required to identify tracks across different services. Performance testing showed the architecture could support the targeted user scale.
Lucene 4 was recently released with key features including improved language analysis support for over 30 languages, faster indexing and storage capabilities, and pluggable similarity models. The large and diverse Lucene community is always testing to improve performance and relevance. Lucene remains an open source option for text search in applications beyond traditional search engines.
This document summarizes Neville Li's work at Spotify developing real-time data streaming applications using Storm. It describes Spotify's large data volumes, how Storm is used to process streaming data at Spotify, details of a social listening topology, and lessons learned around development processes, language choices, and deployment.
Slides for the presentation I made at ClueCon 21 on the experimental RED support in WebRTC, and how we've started tinkering with it in Janus. The presentation also addresses a more generic overview on audio features in WebRTC.
In April 2015, Apache Geode (incubating) was born from Pivotal’s GemFire, the distributed in-memory database. However, the donation of over 1M LOC was just the beginning of the journey. In this talk we discuss how the GemFire engineering team has adapted their development infrastructure, processes, and culture to embrace the “Apache Way". We present lessons learned and best practices for new and incubating open source projects in areas of initial code submission, IP clearance, governance policies, code review, and community building. We discuss the challenges the team faced and how we changed internal communication and software design processes to a community-driven model. In particular, we highlight effective strategies for growing a project community and embracing new members. Finally, we show how changing to the open source model has increased both productivity and quality.
My (quite boring) slides on what we needed to do in Janus to support multiple streams of the same type (e.g., 3 video streams) on the same PeerConnection.
This document discusses scaling issues faced when launching a new feature for 24 million Xbox360 users. It describes profiling traffic, identifying unnecessary API calls, custom logging, and contingencies used such as disabling writes and suppressing calls. The launch was monitored closely with various tools and teams to handle any failures.
Downtown SF Lucene/Solr Meetup: Developing Scalable Search for User Generated...Lucidworks
This document summarizes Sony's development of a scalable search system using Apache Solr for user-generated content on the PlayStation platform. PlayStation users can easily share media like broadcasts, screenshots, and videos to third-party networks. Sony built a SolrCloud-based system to provide a central place to search for this content across millions of users. The system uses three Solr clusters to handle different media types and supports over 20 languages. It processes over 1 billion search requests per day with low latency.
This document provides a summary of various technology topics including:
1. Articles about PHP, JavaScript, and using web services.
2. Information about new versions of Visual Studio 2010, Office 2010, and their features.
3. Summaries of new APIs from Baidu and resources on web development.
Power to the People: Manipulating SharePoint with Client-Side JavaScriptPeterBrunone
First delivered at SharePoint Saturday Houston, this presentation provides a brief background of the use of JavaScript in SharePoint before forging ahead into the capabilities of jQuery and the Client-Side Object Model.
This document discusses HTML5 apps and different approaches for building cross-platform apps, including PhoneGap, W3C Widgets, and other runtimes. It notes that HTML5 apps can access device APIs like the camera and GPS through frameworks like PhoneGap. It also summarizes pros and cons of different approaches, such as PhoneGap providing access to many device APIs but not being a standard, and W3C Widgets being very simple to create but only supporting a few APIs.
Proof of concept description for music (track) recommendations with NoSQL graph database Neo4j. This is not a Machine Learning-related PoC using Collaborative Filtering since the latter is just a piece in the songs similarity puzzle (user preferences cooccurrence distance). This PoC demonstrates the speed of Neo4j, its secret: the data structure, no SQL "select" nor "joins" are needed. Neo4j uses native graph storage and native graph processing whcih benefits from traversal performance.
This document summarizes a last.fm web application project that analyzes music listening data and creates visual dashboards. It uses Python, Bottle framework, and MySQL to build APIs for track analysis, album analysis, user similarity, and network analysis. Database design started with MongoDB but switched to MySQL with partitioning for better performance of joins. Algorithms like cosine similarity and Hadoop/EMR were used for tasks like calculating user music compatibility and trends. The front-end was created with Foundation and visualizations with Canvas JS and HERE Maps API.
First Beat Media - Tehnologije na velikim projektima #tnt3SICEF
The document discusses the technology and trends at an online entertainment media company called FirstBeatMedia. It describes some of their large projects including operating 3 online dating platforms and managing 500 websites with 40 million page views. It provides details on their hardware infrastructure including 200 physical servers and 200 virtual machines. It also discusses the software they use such as PHP, MySQL, caching tools like MemcacheD and Redis, and development tools including Git, testing frameworks and Phabricator. It concludes by noting the growing importance of mobile and the need to develop API-centric mobile applications for Android and iOS.
The document provides an overview of a hackathon being led by Simon Bennetts on extending the OWASP Zed Attack Proxy (ZAP) tool. The plan is to give an overview of how to extend ZAP, discuss potential topics to cover such as implementing scripts, scan rules, and extensions, and then have hands-on hacking sessions with assistance from Simon. Simon outlines many possible topics for discussion, including the ZAP project structure, development environment, documentation, scripting, active and passive scan rules, extensions, and features or fixes to work on.
Crowdmix is a social network focused on music that runs on an event-driven architecture. It allows users to share music content from various streaming services across crowds. The system is designed for scalability to support millions of users. Key components include Kafka for event streaming, Spark/EMR for batch processing, Elasticsearch for search, and Cassandra/Redshift for materialized views and analytics. Music matching is required to identify tracks across different services. Performance testing showed the architecture could support the targeted user scale.
Similar to Music Hackday Boston - The Last.fm API (20)
1. Last.fm API
A brief introduction by David Singleton
Wednesday, 26 May 2010
2. What’s Last.fm?
• We do lots of things:
• charts, recommendations, catalogue, radio, metadata,
similar artists, images, playlists, events, groups, tags,
venues, friends, journals, shoutboxes
• The “Social music revolution”, almost
entirely user generated content, so we
don’t like to keep it locked up
• Long history of offering APIs
Wednesday, 26 May 2010
4. Scrobbling
• For anyone who doesn’t know...
• It’s what we do, fundementally
• Not Scribbling, Scobbing, Snowballing or
Scrabbling. Scrobbling, to scrobble
scrobble: skrob· bul (ˈskrɒbəll)
[verb] To automatically add the tracks you play to your Last.fm
profile with a piece of software called a Scrobbler
Wednesday, 26 May 2010
5. Some scrobbling stats
• 571 scrobbles in the last second
• 34,260 scrobbles in the last minute
• 2,055,600 scrobbles in the last hour
• 24,667,200 scrobbles in the last 12 hours
• 49,334,400 scrobbles in the last day
That's over 34,925,934,051 tracks scrobbled since 2003.
About 272,444 years of music in total.
Wednesday, 26 May 2010
6. API 2.0
• A REST-style HTTP API, that you (might)
know and love (modeled on Flickr)
• Full documentation at http://last.fm/api (far
more reliable than me...). Seriously, read
this, it’s pretty damn good.
• Clients written in PHP, Ruby, Python, C++
• Unrestricted non-commercial use
Wednesday, 26 May 2010
7. Documentation
• Complete and up to date documentation
can be found at http://last.fm/api
• Each method has a page that details is
required and optional parameters
• It covers everything here in greater depth,
and is generally extremely useful.
Wednesday, 26 May 2010
8. What does it offer?
• One hundred or so methods to access
information about:
• Artists, albums, tracks, tags, users,
playlists, events, venues, groups, radio
• Authenticated access to private data
• Export a users full listening history
Wednesday, 26 May 2010
10. Or in an organised fashion...
Wednesday, 26 May 2010
11. Who uses the API?
• 1,000s of 3rd party developers every day
• Checkout http://build.last.fm for
• From forum sigs, to complex apps
• Microsoft built an Xbox360 app using only
our public API
• Squeezebox, Sonos and other hardware
devices
Wednesday, 26 May 2010
12. Hundreds of Scrobblers
• Hype Machine, Spotify, Songbird, Winamp,
iTunes and many more
• People have found ways to scrobble
Youtube, and even their vinyl collection
• You? Check out our submissions API too
• Question: as a developer what would make
scrobbling more appealing to you?
Wednesday, 26 May 2010
16. (Not just iPhone; Android and Blackberry too)
Wednesday, 26 May 2010
17. Using the API
• That’s why you’re here, right?
• You’ll need...
• An API Key (a 32 char unique identifier)
• A way to call the API (use a supported
client, or plain old wget, up to you)
• A way to parse the response (this is
where clients can come in handy)
Wednesday, 26 May 2010
18. Getting an API Key
• To get one you’ll need a last.fm account,
so if you don’t have one you’ll need to join
• Set it up at http://last.fm/api/account (you
probably non-commercial use)
• Only one API Key per account
Please don’t use the example key
Wednesday, 26 May 2010
19. Anatomy of an API call
• http://ws.audioscrobbler.com/2.0/?
method=user.getinfo&api_key=XXX&user=
underpangs
• Root URL + Method + API Key + method
specific parameters (check the method
documentation)
Wednesday, 26 May 2010
20. Anatomy of a response
<lfm status="ok">
<user>
<id>1021212</id>
<name>underpangs</name>
<realname>David Singleton</realname>
<url>http://www.last.fm/user/underpangs</url>
<image>http://userserve-ak.last.fm/serve/126/13884831.jpg</
image>
<country>UK</country>
<age>25</age>
<gender>m</gender>
<subscriber>1</subscriber>
<playcount>45519</playcount>
<playlists>12</playlists>
<bootstrap>0</bootstrap>
<registered unixtime="1073173934">2004-01-03 23:52</registered>
</user>
</lfm>
Every response is wrapped in a root <lfm> node
Wednesday, 26 May 2010
21. Output Formats
• By default all services output XML are also
available as JSON
• To switch, add Add format=json to your call
• Some services offer even more output
formats, check the method docs
• Noteably some can output RSS, XSPF,
iCal and JSONP
Wednesday, 26 May 2010
22. JSON Output
{
"user": {
"id": "1021212",
"name": "underpangs",
"realname": "David Singleton",
[SNIP!]
"registered": {
"#text": "2004-01-03 23:52",
"unixtime": "1073173934"
}
}
}
Note that for elements with both a text child and attributes,
that the text is expressed as the #text attribute
Wednesday, 26 May 2010
23. Paginated methods
• Some methods can return a lot of results,
we limit them and let you page through
• These services will generally accept a limit
(amount per page) and page offset. The
root node will also give you the total
number of results and pages
<lfm status="ok" total="109" page="1" perPage="50" totalPages="3">
...
</lfm>
Wednesday, 26 May 2010
24. Errors
• The root lfm node has a status of “failed”
and will contain an error code and message
• The possible errors for a method are listed
on it’s documentation page
<lfm status="failed">
<error code="10">Invalid API Key</error>
</lfm>
Wednesday, 26 May 2010
25. Common Errors
• Invalid format - This service doesn't exist in
that format
• Invalid parameters - Your request is missing
a required parameter
• Invalid API key
• Invalid method signature supplied
Wednesday, 26 May 2010
26. Common Methods
• artist.getImages
• album.getTags
• track.search
• user.getTopArtists
• geo.getEvents
• event.getInfo
Wednesday, 26 May 2010
27. Method “Types”
• Public - Does what it says on the tin
• Private - Requires authentication (which i’ll
get to shortly)
• “Enhanced” - Public, but with extra
information if you supply a user,
artist.getInfo is a good example of this
Wednesday, 26 May 2010
28. Let’s look at some live
example...
Wednesday, 26 May 2010
29. Caveats
• Send an identifiable user-agent
• Don’t hammer the API (check the TOS)
• Respect HTTP caching headers
• It can go down, so be flexible
• UTF-8 encoding assumed
Wednesday, 26 May 2010
30. Authenticated Calls
• Some methods access private data and
require explicit permission from a user
• To do this we use an oAuth model, where a
user will grant permission to an
application, in the form of a session key
• Slightly different ways to get a session key
for web, mobile and desktop
Wednesday, 26 May 2010
31. Radio
• Radio is an authenticated call, you’ll need a
user session key
• API-based radio is only available to
subscribers, sorry :(
• Full documentation for using this is
available at http://last.fm/api/radio
• However, here’s a quick introduction...
Wednesday, 26 May 2010
32. Tuning
• POST call, to API root, method=radio.tune
• The station is in the form of a Last.fm URL,
eg; lastfm://artist/cher/similarartists
• Once tuned, you request a playlist of 5
tracks at a time, with radio.getPlaylist which
returns an XSPF
Wednesday, 26 May 2010
37. How can you use this?
• Some humble suggestions;
• Rich artist information, stats, bio, images
• Bootstrapping a users music taste
• Catalogue search, artists, albums, tracks
• The API is simple and flexible, so it’s really
up to you and your imagination
Wednesday, 26 May 2010
38. Fin. Questions?
• email: davids@last.fm
• twitter: dsingleton
• irc: dsingleton on #musichackday
• Or just come ask me in person
Wednesday, 26 May 2010