![Some applications ,[object Object],[object Object],[object Object],[object Object],[object Object],Django Beatty / Adub Drupal Solutions - adub.com](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
Similar to Drupal and Web Services
Similar to Drupal and Web Services (20)
Recently uploaded
Recently uploaded (20)
Drupal and Web Services
- 1. Drupal and Web Services A look at the cutting-edge of Drupal integration Django Beatty / Adub Drupal Solutions - adub.com
- 3. Services model Django Beatty / Adub Drupal Solutions - adub.com Site Drupal site Data Data Services XML-RPC REST JSON JSON-RPC SOAP AMF Drupal modules
- 4. Rating Django Beatty / Adub Drupal Solutions - adub.com
- 5. Rating Django Beatty / Adub Drupal Solutions - adub.com Site Drupal backend User rates Results Ratings sent Batch ratings Voting API XML-RPC
- 6. User Generated Content Django Beatty / Adub Drupal Solutions - adub.com
- 7. User Generated Content Django Beatty / Adub Drupal Solutions - adub.com Site Drupal backend User submits photo User profile and photo File storage XML-RPC CCK User profile Moderation
- 8. Content Delivery Django Beatty / Adub Drupal Solutions - adub.com
- 9. Content Delivery Django Beatty / Adub Drupal Solutions - adub.com
- 10. Content Delivery Django Beatty / Adub Drupal Solutions - adub.com Site Drupal backend Views XML-RPC View
- 12. The Future Django Beatty / Adub Drupal Solutions - adub.com
Editor's Notes
- Often when people consider web services and Drupal they may be thinking about getting external data onto their site from services like Flickr. There's plenty to talk about on that, but I want to focus on another side which is not so widely known. One of the most exciting Drupal modules is the Services module. This allows your Drupal site to act as a server of web services that can be consumed by other sites. It lets you expose pretty much any of Drupal's functionality via a web services layer. It comes with decent authentication options and there are a number of servers available. XML-RPC is built-in. There are also pluggable servers for REST, JSON, JSON-RPC, SOAP and AMF. This should at least be on the radar of anyone considering large or complex Drupal deployments. It has been around for a while - since 2007 - but is still pretty cutting edge in terms of its development approach. It is not very widely used (to be expected) and has only relatively recently got onto a stable enough development path to be used more generally. Having said that, we have used it on big deployments and have found it offers many excellent integration possibilities. There are a growing number of add-on modules which leverage it and there is clearly an emergent Services mini-ecosystem. That's a great sign we have seen with other important 'game-changer' modules like CCK, Views, Organic Groups and so on. I'm going to show some applications of the current version of the Services module. But I'd also like to touch on the future of this module to understand the longer-term direction this is taking. We'll have a quick peek at the next generation which offers some significant developments.
- The Services concept is quite abstract and so it can be hard to explain exactly why this is important, where it can be applied, and how far it can be taken. Probably the easiest way to get a feel for that is to look at some concrete examples. You'll notice that functionally these cover quite a wide range. We'll look at examples of: Rating - this allows users to rate content on a bespoke non-Drupal site which has very high traffic. This information is then stored on and fetched from a separate back office system that is Drupal-based. UGC - This application builds on the rating application and allows users to submit profiles and photos from a non-Drupal site and for the Drupal backend system to manage them. Content Delivery - Views from a Drupal site will be pulled in by a non-Drupal site where that content will be redisplayed and cached locally. Some other applications worth mentioning: Syndication - Internal. Content is shared internally between a number of federated sites. This is a very typical and useful application of Services. Syndication - External. Content is made available on a Drupal site for third party sites to be able to access in a secure way. This is particularly suited to commercial syndication. Flash - You can also use Services to have a Flash front-end for a site that runs Drupal under the hood. This way you get all the Drupal functionality for free and concentrate your work on the front-end experience. One example was Adobe's Flex community site, which used to work with a Flash front-end and Drupal back-end (before they switched to standard Drupal throughout). Another more current example is a flash-based site called Digital Dollhouse which is a social networking virtual world application aimed at girls where they get to do interior design and friends can gift each other digital goods like furniture and so on. At the heart of this they have built both a virtual points-based economy and a digital goods micropayment system . Obviously this kind of thing would normally be extremely ambitious, but it was actually assembled from a handful of widely used modules such as Ubercart and Userpoints.
- First we'll briefly look at the general services pattern. A site written using any technology can communicate securely with a Drupal site and do things like login, read and write content, search and so on. This is typically done using XML-RPC, or if you prefer, REST, JSON, JSON-RPC, SOAP, AMF. The Services module allows access to any Drupal functionality. It comes with most useful functionality built in - e.g. nodes, views, users, taxonomy, search. It is also very easy to extend to cover anything that is available within Drupal. So it's not a major job to services-enable your custom modules for example - you just build a very lightweight wrapper to expose whatever functions you want to the Services module. Now we'll look at some real-world implementations.
- The Nuts website uses a bespoke content delivery system (using Zend framework) for the front end that any visitor sees. Here we have used Drupal services to power a rating feature. This allows visitors to rate photos as they click through a gallery. When a user clicks on a rating, this triggers an AJAX call to services which registers the vote on the back-end application, which is using Drupal's Voting API module.
- Here is a diagram of how that fits together. First the user rates an item. Due to the high traffic level of this site, we batch these ratings and post them to the back end periodically. When this happens, the ratings are sent in a set over XML-RPC to a Drupal backend system. This uses the Voting API module to register ratings. Voting API has evolved to handle complexities that often wouldn't occur to many developers if they were to rig up this kind of feature on a commercial deadline. For example it has separate tables to store votes and results as this kind of application is particularly prone to database contention. It handles result calculation asynchronously on a schedule and deals with things like policies for multiple votes from the same IP - can the user vote once or daily, and so on. This is all easily configurable via the Voting API GUI on the backend system. Finally the site makes XML-RPC calls to the backend to get the result for a specific item or to get ranking tables. This system was assembled to replace existing custom-built code which lived on the site. This had huge performance issues and regularly slowed the entire site. So a central aim was to improve performance. Moving the functionality off-site immediately allowed us to uncouple the impact that a popular rating page would have across the rest of the site. We can see straight away that this style of development is inherently scalable and helps us divide our functionality in a natural way that can evolve as we grow our system. Interestingly, the Drupal system powering the back end in this case runs smoothly on a very modest server setup and fails gracefully in the event of a flood.
- Following the success of this approach to the Nuts rating feature, Drupal services was used again for a UGC photo feature. Here users are rating photos submitted by other users. It uses the same rating feature as we've just seen but it also uses services for uploading images and profiles.
- So here's the model of what happens in addition to the rating we've just seen. The user submits a photo and profile using a normal form on the site. Those details are then sent via XML-RPC to the backend. In this case we send that information to file storage and CCK modules on the Drupal system. The profiles are also moderated by site administrators before being set to a published status. Now the site makes XML-RPC calls to the backend to get a user profile (and can access the photo via NFS). We were very happy with this as it built on the rating feature and was assembled very quickly in a couple of days by one of the in-house dev team in response to an editorial concept. (It's maybe instructive to consider how long this kind of thing would take to do using traditional development methods and the competitive disadvantage there.) The maintenance burden is also much lower than we would typically expect as the custom code is mainly glue for the components which do the lifting. That's another key concept with this style of development.
- Finally here's a small example of a more obvious application - getting content from Drupal to another site. Like Nuts, the NME has a custom-built site delivery system for the front-end and we also use Drupal as a back-end content management system. Here we're looking at an album review I've truncated a bit to show the key elements.
- We also had some other content that we wanted to display within these pages on the main site. These were album review links we collected from other sites' RSS feeds and store on our Drupal back-end. We wanted to be able to relate them to reviews created by the NME. These review links are collected and categorised (tagged) in an automatic process which is completely separate to the workflow for writing a review. We needed the site delivery system to be able to automatically collect these items from the backend, linking them up with the specific review being displayed.
- Here the site makes a request to the Drupal Views module. On the backend we've created a a view which contains a list of the review links from external sites. The view takes an argument (the name of the release). When the site delivery system requests the view, it passes the release name as a parameter and gets the appropriate review links. This brings the full power of views to our site delivery system and is very generally applicable. Hopefully these examples give an idea of how this relatively simple pattern of development provides extremely powerful building blocks for a very broad range of applications.
- Looking at the wider picture, the Services module is now spawning a growing ecosystem of related modules which could each be the subject of presentations. I'll briefly describe a few of them to give an overview of what's out there. Unlike the integration examples we've just looked at, these are based on Drupal sites talking to other Drupal sites. The Content Distribution module is designed for having a central hub site as a content repository and allowing satellite sites to pull selected articles across automatically on a scheduled basis. So the application here could be described as internal syndication. Deploy is designed for pushing content and configuration from a staging server to a live server. This allows you to have no editorial access to a live system (which was the requirement of the client who commissioned it). It handles articles, taxonomy, users, views, content types, system settings and so on. It is very rich in terms of dependency checking content (e.g. checking that the author exists for an article being staged). It doesn't currently handle automation. Clients is a somewhat experimental module I wrote which provides pluggable clients to consume services from another Drupal site. Among other things this provides automatic updates and ETL capabilities. The main uses are syndication to internal and external sites (and the possibility to import or redisplay content from non-Drupal sites to Drupal sites).
- Hopefully it's clear why Services should be on the radar of anyone looking at enterprise applications of Drupal. As we know, enterprise typically has longer planning cycles so an important question is where Services is heading over the next few years.
- The Services module now has a third version in alpha. This is incomplete but stable and is being used in production by its author, connecting a network of over 500 sites. It has some very major changes, among them the ability for any module to package its own service. This means that one site can run multiple services with different authentication, paths, protocols. This resource-oriented model of discrete services is very important if you wanted to, for example, run a site that combined secured commercial syndication over XML-RPC, whilst allowing ordinary users to push photos from their Flickr accounts. Services 3 is also moving to OAuth which is discoverable. This means that we will be able to offer the kind of user-friendly app authentication that is found on sites like Flickr and Twitter. The services dev team sees a future in discoverable standard protocols and this has very interesting applications in social networking. There is an emergent set of cross-platform APIs that together form the components for rich social applications. There is already a development release of a module supporting the OpenID Connect standard - this uses OAuth 2 (which Facebook is moving to) for authentication and OpenID for profile information. We also expect to see modules for Salmon, which provides a standard protocol for comments and annotations, and Portable Contacts, which provides a protocol for user contacts. These are each very useful on their own but together provide a functional stack for a distributed social web - especially taken in conjunction with other emerging protocols like XRDS for autodiscovery and Google's Pubsubhubbub for push notification. (These being open cross-platform protocols, sites using them can of course communicate beyond the Drupal userbase.) There is also some serious talk of services becoming a key feature for Drupal 8 and being baked into core. The vision here being that the installed base of Drupal sites will come enabled to interoperate. So this is a very interesting future where we have tools which will enable many applications we haven't even dreamt of yet.