George James :: Querying The Web

Querying the Web Out of the Slipstream :: September 27, 2007

Querying the Web
“ Information wants to be free”
Stewart Brand, Whole Earth Catalogue
May 1985
“ If the new computer set up allowed folks inside to be more creative and independent, why not open it up to outsiders, too?”
Jeff Bezos, Amazon
March 2002
“ Data is the Next Intel Inside”
Tim O’Reilly
September 2005
Open Source has commoditized software
Creative Commons will commoditize information
Which leaves servers, services and service…

General Medical Council

Freebase

Freebase
Metaweb Query Language
Request:
{ "type" : "/medicine/physician",
"name" : “Michael Maher“ }
Response:
{ "code": "/api/status/ok", "result": { "type": "/medicine/physician", "name": “Michael Maher", “gender”: “Male”,
“ education”: “Leeds University”}
}
JSON

GMC
Authoritative
Website based search
Static
Restrictive license
Even if you pay for the data you still cannot use it, legally.
Periodic updates
GMC vs Freebase
Freebase
User sourced content
API
Extensible, dynamic
Creative Commons / PD
Automatic right to use
Stepwise refinement

REST
REpresentational State Transfer
Less rigourous equivalent of SOAP
Data are considered to be resources
Every resource has a unique address
Layered over http:
Client/Server separation
Stateless
Cacheable
Request:
GET http://rest.georgejames.com/product/Serenji/
Response:
Name=Serenji
Price=195.00
OrderCode=H1001

Amazon S3
S3 :: Simple Storage Service
Online storage space
$0.15 per Gbyte per month for storage
~ $0.20 per Gbyte data transfer
Storage request:
PUT http://s3.amazonaws.com/[bucket-name]/[key-name]
Retrieval request:
GET http://s3.amazonaws.com/[bucket-name]/[key-name]
EC2 :: Elastic Compute Clouds

Microformats

Microformats
Without Microformats:
<div class=‘opaque’>
Out of the Slipstream is a one-day conference on Thursday 27 September 2007 at Brooklands Museum, Surrey, UK.
</div>
With Microformats:
<div class=‘opaque vevent’>
<span class='summary'> Out of the Slipstream </span> is a one-day conference on
<abbr class="dtstart" title="20070927">
Thursday 27 September 2007
</abbr>
at Brooklands Museum, Surrey, UK.
</div>

Microformats

Astoria

Astoria in action
Request:
http://astoria.sandbox.live.com/northwind/northwind.rse/Categories
Response:

Astoria in action
Request:
http://astoria.sandbox.live.com/northwind/northwind.rse/Customers
Response:

Astoria in action
Request:
/Customers[FRANK]
Response:

Astoria in action
Request:
/Customers[FRANK]/Orders
Response:

Astoria in action
A variety of response formats:
POX
Web3S (Web, Structured, Schema’d and Searchable)
ATOM
JSON
JSON request:
/Customers[FRANK]?$format=json
Response:

Astoria is still evolving
Ongoing discussion about the format of requests:
/Customers!’FRANK’
/Customers!’FRANK’/Orders!10267
/Customers!CustomerID=‘FRANK’
/Customers(‘FRANK’)
/Customers(‘FRANK’)/Orders(10267)
Qualifiers control the response format:
/Customers(‘FRANK’)/CustomerName
/Customers(‘FRANK’)/CustomerName/$value
/Customers(‘FRANK’)/$format=json
/Customers/$skip=30&$take=10
Currently being Microsoftened…

Where is all this information going to come from?

Crowdsourcing
Jeff Howe, Wired Magazine, June 2006
Delegating an activity to a large number of unidentified individuals
Small finite tasks
Quantity more important than quality
The sum is greater than the parts
Examples:
Wikipedia

Crowdsourcing

Google Maps

Crowdsourcing
Jeff Howe, June 2006, Wired Magazine
Delegating an activity to a large number of unidentified individuals
Small finite tasks
Quantity more important than quality
The sum is greater than the parts
Examples:
Wikipedia
Galaxy Zoo
Amazon Mechanical Turk
Google route planner
Consequences:
Drives down the cost of data
Ownership may not be the traditional incubents
Client / user needs to discriminate

The Power of Information Review
Commissioned by the Cabinet Office, published in June 2007, to review and advise on the use of public sector information.
Recommendation 9:
By Budget 2008, government should commission
and publish an independent review of the
costs and benefits of the current trading
fund charging model for the re-use
of public sector information, including
the role of the five largest trading
funds, the balance of direct versus
downstream economic revenue,
and the impact on the quality of
public sector information.
US: Public Domain
UK: Crown Copyright

AND - Automotive Navigation Data

Press release:
July 4, 2007
Rotterdam - AND Automotive Navigation Data has
agreed ... to donate digital maps of the Netherlands,
China and India to the community.

More ways of querying the web
Google Search
Google Events
Google Base
Yahoo! Pipes
RSS – Really Simple Syndication
KML
BBC Backstage

The Internet is the Database

Thank you
Questions?

http://www.outoftheslipstream.com	70
http://www.outoftheslipstream.org	7
http://www.slideshare.net	2

George James :: Querying The Web

by george.james on Sep 28, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 79

Statistics

George James :: Querying The Web — Webinar Transcript