Exploring the REST and AtomPub as WS-* Stack Alternatives

Slide 1: Exploring the Restful Stack Kurt Cagle, Burton Group an hr-xml.org webinar 2008 - 03 - 14 Kurt Cagle Research Analyst, XML Technologies Burton Group kcagle@burtongroup.com All Contents © 2007 Burton Group. All rights reserved.

Slide 2: In Search of Simplicity 2 autumn deepens - my spider still weaves sunbeams Basho, a haiku The web is simple … and simply is. Accept that simplicity, and the web will reward that acceptance with profoundness. Reject it, and the web will punish without mercy.

Slide 3: SOAP’s Dirty Secrets 3 The Simple Object Access Protocol (SOAP) was to have revolutionized the web, making application development simpler across the web. SOAP only needed … • WSDL – to define the service endpoint functionality • XSD – to define the data types that WSDL would support • Transport Protocols – for agnostic transport across different media (HTTP, FTP, SMTP, …) most of which were in turn never utilized (yet HTTP GET and POST were only added very late) • WS-Security – to provide security of SOAP messages across all those transport protocols • UDDI – to provide web service discovery, not to mention to provide a phone book for contacting all your company’s employees • WS-Authentication and Federated Identity Management – to assure that the receiver of the SOAP message was in fact entitled to receive (or send) that message • Homogenous Processing Environments – as different vendors produced incompatible SOAP processing stacks • Messaging Queues and queue management systems … Um … this is simple?

Slide 4: SOAP Going Down the Drain? 4 After nearly ten years, SOAP-based web services remain behind corporate firewalls in fragile applications. While they have definite specialized niches, SOAP-based Web Services have largely failed as a general services architecture. Moreover, most industry taxonomy organizations are developing their own web services stacks rather than using SOAP based protocols. The question is … why?

Slide 5: The Failure of SOAP Web Services 5 The WS-* Stack … •Emulates OOP. SOAP Services treat server nodes as objects with methods, events and internal state, but these are not native to HTTP. •Works Poorly From Web Sites. SOAP Services require advanced scripting and detailed knowledge of protocol to work from a web page. •Suffers From Ambiguity. The WS-* specifications are complex, which opens the door to incompatibilities between implementations. •Encourages Fragility. SOAP Toolkits encourage the creation of RPC-oriented applications, with more points of failure in the pipeline. •Attempts Transactions. SOAP Services attempts to subsume transactions across a transactionless environment. In other words, SOAP Services try to make the web act in ways that it was not designed to act, introducing too much complexity.

Slide 6: So How Does the Web Act? 6 Roy Fielding, inventor of the Apache Web Server, identified a number of characteristics of the web in his doctoral thesis: •The web consists of an aggregation of resources, where a resource is a “thing” – a document, a picture, a video, etc. •The address of each resource is a Uniform Resource Locator, or URL. This addressability is universal. •There is a limited set of verbs that describe what can be done with those resources, as identified by HTTP. •Those verbs are semantically neutral – they do not know anything about the specific characteristics of those resources. •The “state” of a URL (a server) is a representation of the resource at that URL. •Thus, the “architecture” of the web is built around the transfer of state representations.

Slide 7: Understanding REST 7 Representational State Transfer has become known as REST, and has a number of implications for the web: •The REST verbs are “publishing” verbs: GET, PUT, POST, DELETE, HEAD + some collection verbs (described later) •A URL can either point to a resource or to a container (or collection) of resources. •A resource retains an internal record of when it was created and last modified and who created it, separate from the resource content. •GET operations are idempotent – retrieving a representation of the resource will not change it. •A PUT operation replaces a resource with another, while a POST operation adds a new resource to a collection of resources.

Slide 8: REST as (Primitive) CMS 8 Put another way: REST is the absolute minimal definition of a content management system, one that makes no distinction about what exactly that content is, save that a bundle of content in a given resource has the potential to link to another resource in some fashion.

Slide 9: Why REST failed … the first time … 9 REST seems almost too simple, and because it was an emergent architectural pattern, it wasn’t even identified as such until well after the fact. This caused problems: •GET was used to modify the server state as an RPC. •POST was used to send parametric values (and general content) from the client to the server, and the original use of POST – to add new objects to a collection - disappeared. •PUT disappeared from most web implementations and was replaced with POST. •Resource focused URLs soon became RPC calls instead. •Toolkits were developed that made web programming like standalone application programming, suppressing the metaphor.

Slide 10: REST Resurgence - Syndication 10 The next stage in REST’s evolution was to provide a way to retrieve a collection of resources: •Index page. The collection as table of contents of a web “folder” •Search engine. Retrieval of HTML web resources as summaries in a list of search-term containing entries, usually in order of popularity •WebDAV COL. Retrieval of the resource URLs in a web folder as an XML document, with basic publishing data •News Feeds. Retrieval of URLs of resources from a virtual group, typically ordered by publication update time, along with simple abstracts. •Data Feeds (Emergent). Retrieval of abstraction information for the feed and multiple links to different representations of some data or document entity.

Slide 11: All the News That’s Fit to Print 11 News (or syndication) feeds represented a significant shift in the way that the web worked. •Feeds as Link Collections. Feeds are distinct representations of collections of links to documents, with inclusion and ordering based not on folder content but relevance. •Content Optional. Feeds could contain content, but that wasn’t their purpose – instead, each entry in a feed was a link with a bundle of summarizing metadata about that link •Feeds as Events. An application that could read a feed could determine when a given resource was added to a collection, when a given resource was changed, when it was deleted. In other words, the feed became an event proxy for the resources in question. •Asynchronous. The interval of feed updates on the server is independent from the interval of feed queries on the client.

Slide 12: RSS 2.0 or Atom? 12 Syndication feeds went through a number of formats, most notably under the name RSS (the acronym of a thousand names), many of which are becoming obsolete in favor of either RSS 2.0 and Atom. •RSS 2.0 was developed by Dave Winer, and is licensed by Harvard Law School, and is static. While it is used heavily for web page and blog syndication, it has a number of limitations that make it unacceptable for working with XML. •Atom was developed as an alternative to RSS, is standardized under the IETF and the W3C, and is undergoing active development. As a specification, it is designed specifically for use with XML (although Atom formats built around JSON also exist) and it is more appropriate as a vehicle for XML web data services than RSS 2.0.

Slide 13: Paying Attention to Blogs 13 Beyond their social implications, blogging also represents a major change in the way that people interact with the web: •Dynamic Publishing. Content was published by posting a rich message to a web publishing API, rather than using FTP or other protocol to transfer the content. •Integrated Notification. Posting a blog served to update the relevant RSS feed automatically, rather than requiring manual intervention. •Aggregation. Syndication feeds could in turn be loaded into an aggregator, which generated a new feed based upon a filtered subset of the component feeds. •Consolidation. The initial large number of platforms (and subsequent publishing APIs) illustrated why a common publishing protocol made sense.

Slide 14: Publishing Via AtomPub 14 As work progressed on the Atom feed format, the Atom working group created a correlated open standards publishing protocol, AtomPub, notable for a number of reasons: •REST Based. AtomPub was built to take advantage of a pure REST resource-oriented architecture, and as such aligns well with the web. •Resource and Collection Based. AtomPub works by sending or receiving feeds or individual entries, which in turn can wrap content. •XML Namespace Aware. Atom uses namespaces to both encapsulate content and to extend the language. AtomPub respects this content, and makes no assumption about content type. •Platform Independent. AtomPub can be freely implemented on any platform. It’s a protocol, not a specific implementation. Moreover, the protocol itself is extensible.

Slide 15: From Documents to XML Data 15 AtomPub opens a major opportunity for “data publishing”: •Semantically Neutral Content. An Atom entry can just as readily contain an invoice, a resume, or an aircraft specification as it could a blog, so long as each was encoded in XML. •Decoupled Messaging Queue. The resource collection used to create the feeds is a form of message queue, one that can be processed independently of the publishing system. •Feeds as Event Notifiers. When a resource is published via AtomPub, a person with the right credentials could see the updated resource show up on a portal page, an RSS aware email system, or in a specialized helper app. •Data Aggregation. Aggregation can also be used to create feed “joins” for disparate document types.

Slide 16: Syndicated Views 16 AtomPub can also be used in conjunction with internally coupled views that present the data content in a user friendly (and frequently user editable) way: • XForms works especially well in this model, as each resource can contain both an XForms-based viewer/editor and an XForms based summary table view. • XForms as Report Engine. Additionally, XForms can be used to synthesize reports based upon a given resource referencing other resources. • XSLT as Generator. The XSLT transformation language can also be used to create more static views, or to generate print or PDF related output. • Additional “Viewers”. The same architecture can be used with Microsoft’s InfoPath, Adobe’s Air and other XML data model-based systems. • Alternative Representations. Extending AtomPub also makes it possible to target the generation of OOXML and ODF documents, as well as “non-standard” output representations such as SVG for chart representation and X3D for immersive world output, or VoiceML for converting data into spoken text.

Slide 17: XQuery and Data Syndication 17 XQuery opens up another dimension to Atom data publishing, including: •Extending AtomPub. Using XQuery to create AtomPub extensions to return resources that satisfy given complex criteria (find all invoices between two dates for amounts over $400, sorted by submitter’s last name) •Publishing XQueries. Submitting queries as documents themselves to be used to perform other queries. •Virtual Resources. Working with XQuery to create, parse and update “virtual” resources made up of other content resources. •Generating Views. Using XQuery (possibly in conjunction with XSLT) to create new dynamic views of both single entries and whole collections (such as generating a report with inline SVG based graphs of invoice amounts per month)

Slide 18: Conclusion 18 The conjunction of Atom(Pub), XQuery, and XForms (and related XML model- centric tools) promises to open up a new way for interacting with data systems, one that is less tied into a single application (or even application framework) and more oriented towards accessing and manipulating data as it moves across the cloud. This approach takes significant advantage of the RESTful nature of the web, creating applications that are robust, asynchronous, distributed and very loosely coupled, require a minimal investment in infrastructure and can be implemented in a number of different ways, and that can integrate cleanly into any system that can utilize an XML data abstraction layer. Given this, it’s likely that such architectures will prove to be very common over the course of the next couple of years, leading to data connectivity at any time and any place, in any form.

Slide 19: 19 Questions?