Oopsla 2007 - The Web: Distributed Objects Realized!

The Web: Distributed Objects Realized! Mark Baker, Coactus Consulting Stuart Charlton, BEA Systems

About Us

Mark Baker Coactus Consulting

CTO-level consulting and large-scale distributed systems architect

Leading proponent of “REST style web services” for over seven years

Prior work at Nortel, Sun Microsystems, and several startups

Co-developed internet, mobile, and web standards through the IETF , W3C , and OMA

Mark’s Blog http://www.markbaker.ca/blog/

mailto:mark@coactus.com

Stuart Charlton BEA Systems

Enterprise architect in BEA’s Consulting practice

Co-develops BEA’s global consulting strategy and service offerings

Specialized in distributed systems architecture, data warehousing, and object technology

Prior work in EA, consulting, and mentoring both the financial services & telecommunications industries

Stu Says Stuff http://stucharlton.com/blog/

mailto:stuart.charlton@bea.com

Agenda

Part 1

How We Got Here

Understanding Distributed Object Interoperability

Exploring the Design of REST

Identity and Reference

Handling Evolution

Multi-Organizational Interoperability

Control Flow through Hypermedia

Part 2

RESTful Web Services in Action

Exploring the Web’s Architecture

Applied Techniques for RESTful Web Services

Resources-Oriented Architecture

Security Approaches

Transactions & Reliability

How We Got Here The Rise of the Object Web

Where we’ve come from…

“ The next shift catalyzed by the Web will be the adoption of enterprise systems based on distributed objects and IIOP (Internet Inter-ORB Protocol). IIOP will manage the communication between the object components that power the system. Users will be pointing and clicking at objects available on IIOP-enabled servers.”

- Marc Andressen, Netscape Cofounder (October, 1996)

The Object Web Shall Arise!

“To move to the next step, the Web needs distributed objects. We call this next wave of Internet innovation the "Object Web.“

The HTTP form you submit is still the basic unit of client/server interaction. This clumsy work-around is not suitable for full-blown client/server applications that require highly interactive conversations between components. It also does not scale well.”

- CORBA, Java, and the Object Web BYTE Magazine (October 1997)

By 1999…. XML Web Services Shall Arise!

…Over the past decade, component technology has been the cause of many arguments, disagreements, and debates . This component-induced friction can be traced to two primary factors:

1. “My Glue is Better than Your Glue”

2. “I Can’t Believe You Program In That Language”

- Don Box Lessons from the Component Wars: An XML Manifesto (September 1999)

By the mid-2000s…. what about “just” the Web?

Our Claims:

The dream of the “Object Web” is largely here: it’s the Web!

Constraints in a systems architecture lead to emergent properties .

The Representational State Transfer (REST) architectural style defines constraints which helped guide the web towards a globally scalable, interoperable, and extensible system.

The REST architectural style is both an evolution of past approaches to networked computing, and a radical departure .

More work remains to be done!

The Distributed Object Request Broker (ORB)

The Goals of an ORB:

Encapsulate all of the mechanisms to:

Find the object for the request

Prepare the implementation to receive the request

Communicate the data making up the request

Ensure complete transparency of object location and implementation

Inside an Object Request Broker

Source: OMG CORBA 2.3 Architecture & Specification

How can a web of distributed objects span organizations?

Interoperability Domains

A distributed system, even if the goal is transparency, has several co-existing domains of interoperability

A domain is a set of objects sharing a common characteristic or abiding by common rules.

Types of Interoperability Domains

Referencing

The scope of a distributed object reference

Representation

The scope of a message transfer syntax & protocol

Network Connectivity

Potential scope of a network message

Network Addressing

Potential scope of a network address

Security

The extent of a particular security policy

Others…

Solutions to Multiple Interoperability Domains

Canonicalization

Simple and scalable, at the cost of reduced runtime performance

Low barrier to entry for new clients , high barrier for new servers

Point-to-Point Adaptation

Good runtime performance, not as scalable, increasing complexity

Low barrier to entry for new servers , high entry for new clients

Canonical Domain Domain A Domain B Domain A Domain C Domain B

Solutions for a “Global Object Web”?

Goal seems to be primarily about use and reuse: more consumption

Thus, lower the barriers to new clients!

Scope of Object References: Global

Scope of Representation: Global (but extensible)

Scope of Network Connectivity: Global

Scope of Security: Shared Agreement

Requires at least some level of Global Canonicalization

Solutions for a “Global Object Web”?

Further Concerns:

Unpredictable type, usage, and volume of information

Thus, shaped around extensible, streamable, cacheable, identifiable information

Both Human and Automated actors

Thus, simple, reusable, and extensible interfaces, appropriate for either

… How would you architect such a system?

This is atypical of most distributed object systems architectures in practice

Two Very Different Types of Architectural Domains

Client or Builder-focused

Identifiable Client or Customer(s)

Clearly identifiable builders

Clearly identifiable users

Control governance & control

Overall architectural control

System is a result of deliberate value judgement by the client

Examples

Most IT projects, software products, civil structures, etc.

Collaborative Systems

No identifiable client

Many builders

Many (unpredictable) users

No central control in terms of development, or operation

System is a result of the voluntary choices of the participants

Examples

Internet and WWW, electrical power systems, joint military operations, etc.

Source: Eberherdt Rechtin, The Art of Systems Architecting

Building Emergent Capability into an Architecture

Collaborative systems are about creating emergent capability

The architecture of the system is the constraints on interfaces

Components & implementations are out of scope and out of control

An example style of such constraints:

Client/Server Architecture

Constraints:

Active client, Reactive server, client initiates communication

Emergent properties:

Simplifies server, enables multiple organizational domains

Client Server

Composing Constraints into an Architectural Style

Constraints both enable and detract from emergent properties, and may be combined together

Example:

Client/Server + Layered System + Caching + Stateless Interaction

Enables:

Simplified Clients and Servers (due to statelessness & C/S)

Shared Caching (due to layering)

Improved Visibility & Reliability (due to statelessness)

Add in:

Code on Demand (COD)

Result:

Simplifies clients, Improves Extensibility, but Reduces Visibility

Source: Roy Fielding, Architectural Styles and the Design of Network-Based Software Architectures

The Representational State Transfer (REST) Style

Originally referred to as “The HTTP Object Model”

Source: Roy Fielding, Architectural Styles and the Design of Network-Based Software Architectures

Some Major Differences between REST and Distributed Objects

1. Most distributed architectures do not assume application requirements

Yet, the Web is an application with goals & requirements!

The overall goal: sharing information and capabilities at a global scale

REST assumes the requirement is to provide an architecture of participation for a globally networked information space

2. Most distributed architectures do not require globally scoped interoperability domains

REST insists on a uniform interface to ensure global scope interoperability

The uniform interface is what we explore next!

These differences apply elsewhere: SOAP Web Services, RPC, etc. are also just general toolkits

RESTful Distributed Objects in contrast to SOAP Web Services

Perceived advantages:

Communication occurs through XML document exchange

Agreement occurs in the network protocol

The XML InfoSet is the basis of interoperability

No worrying about object-oriented purity / language-oriented bindings

Disadvantages:

Prefers interface descriptions for specific (non-uniform) interfaces

All data must flow through the XML InfoSet, other types are secondary

Nearly eliminates the notions of:

Object lifecycle (creation, removal, activation)

Object identification & references (i.e. linking)

Assumes transfer protocol independence is possible and desirable

In practice, transfer protocol semantics leak into message exchanges

A Word on REST vs. The World Wide Web

Consider three levels of abstraction:

Architectural Styles : Representation State Transfer (REST)

Architecture : The World Wide Web

Implementation : “User Agents, Web Servers, Proxies, etc.”

REST is a model of constraints and emergent properties

The Web is the most widely used and understood implementation

A Global, Networked, Shared Information Space

Users Consume & Manipulate Information via Agents

Information Objects Must Be Identifiable

An Object’s Interface is separated from Implementation

State is exchanged in a stateless, self-describing manner

Handling Evolution

Meaning is Determined by the Context of Relationships

Content & Relationships May Change Over Time...

...But the Identity of Information Must Remain Stable

The Information Space consists of Identifiable Conceptual Mappings State (at a point in time) Object

Instead of Objects & State, REST calls these Resources & Representations Representation (at a point in time) Resource

The conceptual mapping should remain consistent over time! Concept Representation (at a point in time) Resource “ Customer 31319” Concept “ The state of Customer 31319 at a point in time” Server Client (stable)

Example: Different Representations Concept XML Document Resource “ Customer 31319” Concept “ The state of Customer 31319 at a point in time” Server Client (stable) CSV File

Multi-Organization Interoperability

An Organization Manages Information

Information Spans Organization Boundaries

Clients Span Organizational Boundaries

Clients & Infrastructure Evolve Independently

So, representations must be uniformly specified

What Domains are in a Representation?

The means and intent of communication

All clients and servers should at least be able to communicate, regardless of the particular resource

Thus, communication must be very generic and uniformly applicable to all resources

The information bits themselves, along with their self-descriptive interpretation

Information can be represented in many ways, even if it’s the same concept

The references for the requested or related resources

Relationships between a medium identifiers can denote the meaning of information

How Stable Should the Domains of a Representation Be? Data Element Level of Stability Authority over Change Means & Intent of Communication Extremely Stable Global Representation of Information Very Stable Widely Shared Agreement Information Resource Identifier Stable Organization

Control Flow

Embed Control Flow Inside the Representation Itself

As Information is Manipulated or Conveyed, So Are The References for “Next Steps”

Hyperlinks Define the Possibilities for Communication

Application State is Held By the Agent

Summary of the REST Uniform Interface

All valuable resources are identified by a globally scoped, uniform, resource identifier

Simple, consistent ability to (de)reference both information & services

Resources are manipulated through the exchange of representations

Stateless, visible communication to encapsulated resources

All exchanges are self-descriptive

Enables visibility and extensibility

All access methods mean the same for all resources

Communication semantics are universal

Hypermedia is the engine of application state

Information and controls are intertwingled

Agenda

Part 1

How We Got Here

Identity and Reference

Handling Evolution

Multi-Organizational Interoperability

Control Flow through Hypermedia

Part 2

Exploring the Web’s Architecture

Resources-Oriented Architecture

Security Approaches

Transactions & Reliability

RESTful Light Bulbs

Imagine any lightbulb, anywhere, is an HTTP-supported resource

Multiple representations might be supported

Application/lightbulb+xml

An HTML form

E.g. Turning on a lightbulb:

PUT http://example.com/kitchen/main HTTP/1.1

Content-Type: application/ lightbulb +xml

< lightbulb >

<state>on</state>

</ lightbulb >

How does this impact the scale of our lightbulb system?

A Hypermedia Purchase Order

Without URIs and Links

POST /purchaseOrders HTTP/1.1

<?xml version="1.0"?>

<updatePurchaseOrder>

<purchaseOrder id=“31811” orderDate="1999-10-20">

<shipTo country="US">

<name>Alice Smith</name>

<street>123 Maple Street</street>

<city>Mill Valley</city>

<state>CA</state>

<zip>90952</zip>

</shipTo>

<items>

<item partNum="872-AA"> <quantity>1</quantity>

<USPrice>148.95</USPrice>

<comment>Confirm this is electric

</comment>

</item>

</items>

</purchaseOrder>

</updatePurchaseOrder>

With URIs Hyperlinks

PUT /purchaseOrder/31811 HTTP/1.1

<?xml version="1.0"?>

<purchaseOrder orderDate="1999-10-20">

<shipTo

href=“ http:// alice.com /address ”/>

<items>

<item href=“ http://lawnsrus.com/872-AA ”>

<quantity>1</quantity>

<comment>Confirm this is electric

</comment>

</item>

</items>

</purchaseOrder>

How do either of these impact evolution over time?

Exposing Legacy Information as Resources

Reusing web Infrastructure to support the legacy

E.g. Cacheable, Searchable, Composable Customer Data

RESTful Web Service Gateway Legacy Customer Information System Composite App DB Web Cache Web Crawler http://company.com/customer/30133 Other Resources

Exploring the Web Architecture Highlights

A View of the Current Web Architecture BEA Confidential | Web Architecture Foundation Internet Protocol Foundation Hypertext Transfer Protocol (HTTP) Security & Authentication Management & Caching BGP Application Protocol Standards ARP DNS IP TCP Multipurpose Internet Mail Extensions (MIME) Media Types HTML XML RSS / Atom URI/IRI Unicode JavaScript Images Video JSON Form data Interoperability Crawlers User Agents Semantic Web Microformats Web Sites Resource-Oriented System Interfaces Content Syndication

Inside an Object Request Broker

The New ORB: A Web Server

Hypertext Transfer Protocol

GET /docs/2.2/ HTTP/1.1

Host: httpd.apache.org

Accept: text/html, text/*, */*

User-Agent: GET/7 libwww-perl/5.40

HTTP/1.1 200 OK

Date: Thu, 31 Jul 2007 15:40:09 GMT

Server: Apache/2.2.4

Content-Type: text/html

Content-Language: en

Transfer-Encoding: chunked

Etag: “b381faa81c”

Cache-control: max-age=3600

Vary: Accept-Language

4090

<HTML><HEAD>

…

Highlights of the HTTP Interface

Methods

GET

Retrieve a resource’s state

PUT

Store/replace a resource’s state

POST

Create/append to a resource’s state

DELETE

Remove a resource’s state

HEAD

Retrieve resource & control metadata

OPTIONS

Retrieve what methods are supported

TRACE

Retrieve a loop-back of the request

CONNECT

Request that a proxy tunnel the request

Response Code Ranges

100…Informational

200…Success indicators

Request has succeeded

New resource created

300…Redirection

Content negotiation

Resources that have moved permanently or temporarily

Cacheable resources that have not changed

400…Bad Requests

Method not supported

Unauthorized access

Resource not found currently or permanently

500…Server Errors

Feature not implemented

Service temporarily unavailable

Internal server errors

How Stable Should a Web Representation Be? Data Element Level of Stability Authority over Change Means & Intent of Communication (e.g. identifiers, operations & metadata) Extremely Stable Global (HTTP & URI Syntax) Representation of Information Very Stable Widely Shared Agreement (MIME Types) Information Resource Identifier Stable Organization (Allocated URIs)

How does a Client Understand its Information?

Stable , standardized, streaming media formats

Multipurpose Internet Mail Extensions (MIME) Types

Generic containers for more specific information

The context of relationships to other identifiers

Information may embed links within a document

E.g. an HTML tag <IMG SRC=“http://images/customer/31319...”> implies:

A link is to an associated resource

This resource should respond to HTTP GET

This resource will be an image of some type

Code-on-demand that can interpret the information for the client

E.g. Java Applets, ActiveX Controls, Flash Plug-ins, JavaScript, AJAX

Intermediaries

Proxies are client-determined intermediaries

Gateways are server-determined intermediaries

Caching with HTTP Conditional GET

GET /docs/2.2/ HTTP/1.1

Host: httpd.apache.org

Accept: text/html, text/*, */*

If-None-Match: “38d8af8210bb”

HTTP/1.1 304 Not Modified

Content-Length: 0

ETag: “38d8af8210bb”

Proxy User Agent (Browser) Gateway Server

Some flaws in the Deployed Web Architecture

MIME Types

Benefit: reuse most of the metadata and media descriptions for email

But, designed for Email, not the web

Focus on packaging lots of information into a single package

Hypermedia prefers linking to subsets

Workarounds for layered content

e.g. GZIP transfer encoding of a text/html document

Assumes a lossy transport

Centralized type registry

Cookies

Site-wide opaque data

Reduced visibility, privacy, and security

Can lead to counter-intuitive behaviour

Some sites do not handle the browser’s back button

Persistent data should be a resource

Eg. A Shopping Cart URI

But, requires browser extensibility

Alternatives are emerging with the popularity of AJAX-like techniques to store & retrieve preferences

A Design Method for Resource-Oriented Architecture

Figure out the data set

Split the data set into resources

For each resource:

Name the resources with URIs

Expose a subset of the HTTP interface

Design the representation(s) accepted from the client

Design the representation(s) served to the client

Integrate this resource to others, using hypermedia links and forms

Consider use cases

Apply standard hypermedia types (e.g. HTML, Atom), where applicable

Consider error conditions

Apply standard hypermedia types (e.g. HTML, Atom), where applicable

Common & Emerging Hypermedia Types

The big six:

XHTML, HTML, Flash, PDF, RSS, and Atom

Emerging:

Atom Publishing Protocol, HTML Microformats

On the horizon :

HTML 5

RDF, GRDDL and Semantic Web standards

Media types usually define a general purpose control flow:

RESTful Security Approaches

Identity Assertion

OpenID

URI-based digital identity

Open ID 2.0 is adding attributes, secure messages

Kerberos / SPNEGO

Popular on intranets, requires a server

Not likely internet scalable

Current implementations require excessive round-trips

RESTful Message Confidentiality & Integrity

Mutual SSL

Robust, widely supported

Does not enable intermediary visibility or persistent signatures

XML Signature & Encryption

But XML Canonicalization is extremely complicated

S/MIME or PGP

Key negotiation is usually out of band

OpenID 2.0 may help

Widely deployed within the email system (perhaps not as widely used )

Fits within existing HTTP infrastructure

Not widely deployed within HTTP

Limitations: http://blog.jclark.com/2007/10/why-not-smime.html

Image: Mark Wahl, ldap.com

Transactions & Lost Updates

Conditional PUT

A RESTful approach to optimistic locking

PUT /test_resource HTTP/1.1

Host: bea.com

Accept: */*

Expect: 100-continue

If-None-Match: “381fa9311bc”

HTTP/1.1 412 Precondition Failed

Date: Thu, 31 Jul 2007 15:40:09 GMT

Server: Apache/2.2.4

RESTful Distributed Transactions

Design Sketch: Resource-Oriented Units of Work

Resource for the Transaction Manager & each Transaction

A new resource is created for each registered resource, enabling isolation

Transaction manager acts as an agent to register resources, prepare & commit

Reliable Messaging

Ordered messaging

Assumes a protocol will order stateful, asynchronous application exchange patterns

Whereas, REST assumes stateless, synchronous application exchange patterns

Each request requires a response

Requests may be pipelined, will be responded to in-order

Exactly-once processing

Legitimate need for resource state changes,

e.g. In HTML forms, eliminating the “double form submit” button

Two possible solutions:

Application-level check (e.g. unique ID in HTML hidden form field)

POST Exactly Once: http://www.mnot.net/drafts/draft-nottingham-http-poe-00.txt

Summary

The “Global Object Web” is largely here: it’s the Web!

The Web is similar to a Distributed Object System

Objects ~= Resources

Resources may be referenced, created, updated and/or deleted

Global-scale interoperability requires constraints on communication

Unpredictable evolution & configurations are the norm

Serendipitous reuse is more common than planned reuse

Must depend on a stable, but extensible, uniform interface to communicate

Many traditional distributed systems challenges can be solved in a RESTful manner

Reliability, optimistic locking, security, transactions

More work remains to be done – “Learn to love the Web”!

Thank you!

Alternative Architectures When is REST not appropriate?

Alternative Architectures

Transparent Remote Access

Developer transparency is paramount

Information sharing is unimportant

Evolution is federally controlled (e.g. producers & consumers are under the same control )

Distributed Objects or XML Web Services can be appropriate

Globally Consistent Transaction System

Tightly Coupled Single-System Image (e.g. a cluster)

Consistency trumps High Availability

Accomplished through language-level bindings (e.g. Java XA, MSDTC) or distributed object network protocols (e.g. TIP or IIOP)

Stateful, In-Order, Reliable Messaging

When you don’t have the time or inclination to share data

Most interactions are not safe or idempotent

The application does not lend itself well to statelessness

The physical scale (number of servers) is relatively small at each layer

Clients are trusted & controlled in the same federation as the servers

The network is controlled & predictable

Structured Data Analysis, Manipulation and Reporting

Remote Data Access (SQL, SPARQL) interfaces use general-purpose connectors

Requires extensive knowledge of the schema

Often requires per-client state maintenance

Stateful, real-time two-way communication

E.g. voice or video communication / conferencing, P2P file transfer

Requires a synchronous two-way streaming architecture

However, stream signaling and negotiation arguably can be RESTful

E.g. BitTorrent registries & trackers require the Web to function

High-speed Publish/Subscribe Event Notification

HTTP is limited to request/response semantics

May be significant protocol overhead

Arguably a limitation of HTTP 1.1, not REST

Examples of fitting publish/subscribe into HTTP:

Simulating an Event Stream as a long HTTP GET Request

Installing an HTTP Server on the Agent side

No obvious message correlation, hindering visibility

Events on the Web: Comet-Style AJAX Applications

Examples include:

Google Mail’s GTalk integration

Meebo instant messaging

Renkoo

Jot Live

Simulate events through long HTTP GET requests

References

Oopsla 2007 - The Web: Distributed Objects Realized!

0 comments

Notes on slide 1

4 Favorites