Diving and digital data
For example: Fotografie
Name all the Fish....
Talk about accomadations....
and the dangers for you and your data...
Clouds
Google Data APIs
Frank Mantek
“The benefit [of GData] is that you'd have a single
 API that could be used to query, update, and index
 structured data o...
Contrast this with the API efforts on Yahoo! Developer Network or
Windows Live Dev which are an inconsistent glop of incom...
Google’s APIs

     • Google Offers Many APIs
       – AJAX APIs – Presentation Oriented APIs
          • Maps, Search, Fe...
Building Web Services the REST way




                                     QCon
                                     2008
Building Web Services the REST way

• SOAP:
  – Arbitrary actions (verbs)
  – Arbitrarily data payloads
  – Complex servic...
Building Web Services the REST way

• SOAP:
  – Arbitrary actions (verbs)
  – Arbitrarily data payloads
  – Complex servic...
Building Web Services the REST way

• SOAP:
  – Arbitrary actions (verbs)
  – Arbitrarily data payloads
  – Complex servic...
Elements of the Data API

                  • Simple Format
        Atom
                     – Atom Syndication
     Publ...
Data Model: Standard Atom

     <entry xmlns:gd=quot;http://schemas.google.com/g/2005quot;>
         <category scheme=quot...
Data Model: Google Kinds and Extensions

     <entry xmlns:gd=quot;http://schemas.google.com/g/2005quot;>
         <catego...
Querying a Feed

• Full Text Search
   – http://www.example.com/feeds/friends?q=John
• Categories (Tags)
   – http://www.e...
Optimistic Concurrency

     GET /feeds/entry1

     200 OK
     <link rel=quot;editquot; href=quot;http://www.example.com...
Security and Authentication

     • Authentication for Desktop Apps
        – Users authenticates to Google Authentication...
Atom Publishing Everywhere?

     •   Google Apps data API
     •   Google Base data API
     •   Blogger data API
     • ...
Developer Support

• Open Source Client Libraries
                        – .NET
   – Java
                        – Pytho...
Google Apps data API

     • Functionality
       – Programmatically enable access to these Google Apps
          applicat...
Google Base data API

     • Functionality
       – Query Google Base data to create applications and
           mashups.
...
Blogger

     • Functionality
       – Create new blog posts, edit or delete existing posts, and
           query for post...
Google Calendar

     • Functionality
       – Create new events, edit or delete existing events, and query
           for...
Google Code Search

     • Functionality
       – Read Only - Query for public code snippets that match
           particu...
Picasa Web Albums

     • Functionality
       – Request a list of a user's albums, photos, comments, or
          tags, e...
Google Spreadsheets

     • Functionality
       – Request a list of a user's spreadsheets, edit or delete
           cont...
What Some Developers are Doing

                      Ambient Clock
                       (Calendar)
                    ...
Spanning Sync (www.spanningsync.com)

     • two way synchronization between iCal and Google
      Calendar
     • Client/...
Gumiyo (www.gumiyo.com)

     • Buyer/Seller platform using Google Base data api
     • Allows creation of ads completely ...
Bloghud (www.bloghud.com)

     • This is a Second Life “Heads Up” device (900 Linden for
      the Pro Version)
     • It...
Choosing REST over SOAP

• Technologies of REST are well understood
  • HTTP, URIs, XML, Atom
• Four actions are sufficien...
Is REST as flexible as SOAP?

• Some Operations are not obvious
  – eg. transactions
  – eg. send notification with reply
...
Using the Atom Publishing Protocol ...




                                              QCon
34
                         ...
Using the Atom Publishing Protocol ...

     • Good REST based design




                                              QC...
Using the Atom Publishing Protocol ...

     • Good REST based design
     • Missing Query:
       – Solution: query param...
Using the Atom Publishing Protocol ...

     • Good REST based design
     • Missing Query:
       – Solution: query param...
Using the Atom Publishing Protocol ...

     • Good REST based design
     • Missing Query:
       – Solution: query param...
Extending the Atom Publishing Protocol ...

     • Engineers care about efficiency

     • Forced Read-Modify-Write
      ...
Using Atom Data Format ...




                                  QCon
36
                                  2008
Using Atom Data Format ...

     • Originally designed for blogs not data




                                            ...
Using Atom Data Format ...

     • Originally designed for blogs not data

     • Drawbacks
       – syntax constraints (a...
Using Atom Data Format ...

     • Originally designed for blogs not data

     • Drawbacks
       – syntax constraints (a...
Using Atom Data Format ...

     • Originally designed for blogs not data

     • Drawbacks
       – syntax constraints (a...
Modeling Data in Atom ...




                                 QCon
37
                                 2008
Modeling Data in Atom ...

     • How to indicate extensions / semantics?
       – Solution: atom:category URI to indicate...
Modeling Data in Atom ...

     • How to indicate extensions / semantics?
       – Solution: atom:category URI to indicate...
Designing New Data Formats ...




                                      QCon
38
                                      2008
Designing New Data Formats ...

     • The Idea: GD-namespace
       – Create reusable extensions and kinds
       – Manag...
Designing New Data Formats ...

     • The Idea: GD-namespace
       – Create reusable extensions and kinds
       – Manag...
Designing New Data Formats ...

     • The Idea: GD-namespace
       – Create reusable extensions and kinds
       – Manag...
Designing New Data Formats ...

     • The Idea: GD-namespace
       – Create reusable extensions and kinds
       – Manag...
Care for your Developers

• Protocol Centric & Language Agnostic

• Open your Code

• Give Fair Warnings of Changes

• Mee...
Do you feel lucky?

     • Questions, comments?




                              QCon
40
                              20...
Google G Data Reading And Writing Data On The Web 1
Google G Data Reading And Writing Data On The Web 1
Upcoming SlideShare
Loading in...5
×

Google G Data Reading And Writing Data On The Web 1

1,216

Published on

Published in: Technology, News & Politics
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,216
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
20
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Google G Data Reading And Writing Data On The Web 1

  1. 1. Diving and digital data
  2. 2. For example: Fotografie
  3. 3. Name all the Fish....
  4. 4. Talk about accomadations....
  5. 5. and the dangers for you and your data...
  6. 6. Clouds
  7. 7. Google Data APIs Frank Mantek
  8. 8. “The benefit [of GData] is that you'd have a single API that could be used to query, update, and index structured data on the web--anywhere on the web. It's a pretty powerful vision and something I didn't expect to see for a couple more years.” Jeremy Zawodny, Yahoo
  9. 9. Contrast this with the API efforts on Yahoo! Developer Network or Windows Live Dev which are an inconsistent glop of incompatible RESTful protocols, SOAP APIs and XML-RPC methods all under the same roof. In the Google case, an app that can read and write data to Blogger can also do so to Google Calendar or Picasa Web Albums with minimal changes. This is not the case when using APIs provided by two Yahoo! services (e.g . Flickr and del.icio.us) or two Windows Live services (e.g. Live Search and Windows Live Spaces) which use completely different protocols, object models and authentication mechanisms even though provided by the same vendor.quot; Dare Obasanjo, Architect, Microsoft
  10. 10. Google’s APIs • Google Offers Many APIs – AJAX APIs – Presentation Oriented APIs • Maps, Search, Feeds – Google Data APIs – Full Fidelity Data APIs • Calendar, Blogger, Base, ... – AdWords API – SOAP based RPC API – Gadgets API – Portable JavaScript “Applets” – Checkout API – ReST based API – Desktop API – Extended Search – ... and many more ... QCon 11 2008
  11. 11. Building Web Services the REST way QCon 2008
  12. 12. Building Web Services the REST way • SOAP: – Arbitrary actions (verbs) – Arbitrarily data payloads – Complex service description and behavior (WSDL) QCon 2008
  13. 13. Building Web Services the REST way • SOAP: – Arbitrary actions (verbs) – Arbitrarily data payloads – Complex service description and behavior (WSDL) • REST: – Common actions (HTTP Get, Post, Put, Delete) – Operate on whole resources (XML documents) – Uniform resource names and links (URIs) QCon 2008
  14. 14. Building Web Services the REST way • SOAP: – Arbitrary actions (verbs) – Arbitrarily data payloads – Complex service description and behavior (WSDL) • REST: – Common actions (HTTP Get, Post, Put, Delete) – Operate on whole resources (XML documents) – Uniform resource names and links (URIs) • Why REST over SOAP? – It’s how the web works! (stateless, cachable, scalable) – It’s easy on the programmer QCon 2008
  15. 15. Elements of the Data API • Simple Format Atom – Atom Syndication Publishing Format Protocol • Simple Protocol – HTTP Google • Google Extensions Data – Data Model APIs – Query – Concurrency – Authentication QCon 13 2008
  16. 16. Data Model: Standard Atom <entry xmlns:gd=quot;http://schemas.google.com/g/2005quot;> <category scheme=quot;http://schemas.google.com/g/2005#kindquot; term=quot;http://schemas.google.com/g/2005#eventquot;/> <id>http://mycal.example.com/feeds/jo/home/full/e1a2af0</id> <published>2005-01-18T21:00:00Z</published> <updated>2006-01-01T00:00:00Z</updated> <title>Discuss BazMat API</title> <content>We will discuss integrating GData with BazMat.</content> <author> <name>Jo March</name> </author> <gd:when startTime='2005-01-18T21:00:00Z' endTime='2005-01-18T22:00:00Z'> <gd:reminder minutes='15'/> </gd:when> <gd:where valueString='Building 41, Room X'/> <gd:eventStatus value=quot;http://schemas.google.com/g/2005#event.confirmedquot;/> <gd:visibility value=quot;http://schemas.google.com/g/2005#event.publicquot;/> </entry> QCon 14 2008
  17. 17. Data Model: Google Kinds and Extensions <entry xmlns:gd=quot;http://schemas.google.com/g/2005quot;> <category scheme=quot;http://schemas.google.com/g/2005#kindquot; term=quot;http://schemas.google.com/g/2005#eventquot;/> <id>http://mycal.example.com/feeds/jo/home/full/e1a2af0</id> <published>2005-01-18T21:00:00Z</published> <updated>2006-01-01T00:00:00Z</updated> <title>Discuss BazMat API</title> <content>We will discuss integrating GData with BazMat.</content> <author> <name>Jo March</name> </author> <gd:when startTime='2005-01-18T21:00:00Z' endTime='2005-01-18T22:00:00Z'> <gd:reminder minutes='15'/> </gd:when> <gd:where valueString='Building 41, Room X'/> <gd:eventStatus value=quot;http://schemas.google.com/g/2005#event.confirmedquot;/> <gd:visibility value=quot;http://schemas.google.com/g/2005#event.publicquot;/> </entry> QCon 15 2008
  18. 18. Querying a Feed • Full Text Search – http://www.example.com/feeds/friends?q=John • Categories (Tags) – http://www.example.com/feeds/friends/-/work|play • Update Time – http://www.example.com/feeds/friends? updated-min=2005-08-09T10:57:00-08:00 • Custom ... – eg. spreadsheets supports min-row, max-row, ... • Output Format – http://www.example.com/feeds/friends?alt=json QCon 2008
  19. 19. Optimistic Concurrency GET /feeds/entry1 200 OK <link rel=quot;editquot; href=quot;http://www.example.com/feeds/entry1/version1/quot;/> PUT /feeds/entry1/version1/ 200 OK <link rel=quot;editquot; href=quot;http://www.example.com/feeds/entry1/version2/quot;/> DELETE /feeds/entry1/version1/ 409 CONFLICT QCon 17 2008
  20. 20. Security and Authentication • Authentication for Desktop Apps – Users authenticates to Google Authentication Service – Google returns Auth Token to Application • Authentication for Web Apps – Web Application redirects to Google Auth Service (1) – Login to Google, redirect back to app with Auth Token (2) – Web Application uses Token to proxy GData (3) – Password remains secure (1b) Accounts (2a) Authentication Client Browser (1a) Calendar Web App (3) data API (2b) QCon 18 2008
  21. 21. Atom Publishing Everywhere? • Google Apps data API • Google Base data API • Blogger data API • Google Calendar data API • Google Code Search data API • Google Notebook data API • Picasa Web Albums data API • Google Spreadsheets data API • Google Doclist data API • YouTube data API • ... and the list is growing ... QCon 19 2008
  22. 22. Developer Support • Open Source Client Libraries – .NET – Java – Python – PHP – JSON (JavaScript) – Objective C • Contributed Client Libraries – Ruby – Perl – Lisp • Documentation – Developer Guides – Knowledge Base • Active Developer Community (Google Groups) – Google Data API Group, ... – Google Groups Wiki ... QCon 2008
  23. 23. Google Apps data API • Functionality – Programmatically enable access to these Google Apps applications – Create, retrieve, update and delete user accounts, nicknames and email lists. • Supported Client Libraries – Basic HTTP – Java – .NET QCon 21 2008
  24. 24. Google Base data API • Functionality – Query Google Base data to create applications and mashups. – Input and manage Google Base items programmatically. • Supported Client Libraries – Basic HTTP – Java – .NET – PHP – Python QCon 22 2008
  25. 25. Blogger • Functionality – Create new blog posts, edit or delete existing posts, and query for posts that match particular criteria. • Supported Client Libraries – Basic HTTP – Java – .NET – PHP – Python – Objective C QCon 23 2008
  26. 26. Google Calendar • Functionality – Create new events, edit or delete existing events, and query for events that match particular criteria. • Supported Client Libraries – Basic HTTP – Java – .NET – PHP – Python – Objective C QCon 24 2008
  27. 27. Google Code Search • Functionality – Read Only - Query for public code snippets that match particular criteria. • Supported Client Libraries – Basic HTTP – Java – .NET – PHP – Python – Objective C QCon 25 2008
  28. 28. Picasa Web Albums • Functionality – Request a list of a user's albums, photos, comments, or tags, edit or delete content in an existing entry, and query the content in an existing entry • Supported Client Libraries – Basic HTTP – Java – .NET support soon QCon 26 2008
  29. 29. Google Spreadsheets • Functionality – Request a list of a user's spreadsheets, edit or delete content in an existing Spreadsheets worksheet, and query the content in an existing Spreadsheets worksheet • Supported Client Libraries – Basic HTTP – Java – .NET – Python QCon 27 2008
  30. 30. What Some Developers are Doing Ambient Clock (Calendar) Phixr Online Photo Editor (PicasaWeb) Timesheets with Google Office Tools (Spreadsheets, Calendar) QCon 2008
  31. 31. Spanning Sync (www.spanningsync.com) • two way synchronization between iCal and Google Calendar • Client/Server architecture, all GData operations happen on the server • Straight HTTP calls using PHP • around 6M GData API calls per day QCon 29 2008
  32. 32. Gumiyo (www.gumiyo.com) • Buyer/Seller platform using Google Base data api • Allows creation of ads completely on your mobile device (take picture, send MMS to your account) • Searching/buying on the phone • Rule based alert system that sends real time notifications to your phone QCon 30 2008
  33. 33. Bloghud (www.bloghud.com) • This is a Second Life “Heads Up” device (900 Linden for the Pro Version) • It allows you to post to your blog from inside the virtual world, using Google Authentication for Web Apps • /9 “your message” posts or drag text based notecards on the Hud QCon 31 2008
  34. 34. Choosing REST over SOAP • Technologies of REST are well understood • HTTP, URIs, XML, Atom • Four actions are sufficient • most of the time (90% solution) • Developers like it • Every API behaves the same • REST was a good choice QCon 2008
  35. 35. Is REST as flexible as SOAP? • Some Operations are not obvious – eg. transactions – eg. send notification with reply • Be creative • notifications as side effects of update • drop box feeds (insert only) • virtually infinite feeds (get & update only) • REST is sufficient, but you may have to be creative QCon 2008
  36. 36. Using the Atom Publishing Protocol ... QCon 34 2008
  37. 37. Using the Atom Publishing Protocol ... • Good REST based design QCon 34 2008
  38. 38. Using the Atom Publishing Protocol ... • Good REST based design • Missing Query: – Solution: query parameters – Not always sufficient ... an open issue QCon 34 2008
  39. 39. Using the Atom Publishing Protocol ... • Good REST based design • Missing Query: – Solution: query parameters – Not always sufficient ... an open issue • Missing Authentication: – Solution: Google Auth – More secure than Basic Auth, slowed integration QCon 34 2008
  40. 40. Using the Atom Publishing Protocol ... • Good REST based design • Missing Query: – Solution: query parameters – Not always sufficient ... an open issue • Missing Authentication: – Solution: Google Auth – More secure than Basic Auth, slowed integration • Concurrent Operations – Solution: Optimistic Concurrency via Edit URIs – We should have used strong eTags QCon 34 2008
  41. 41. Extending the Atom Publishing Protocol ... • Engineers care about efficiency • Forced Read-Modify-Write – Solution: URIs based on the data (ACL) • Batch Processing – Solution: GData batch extension • Incremental Changes / Updates – an open issue ... QCon 35 2008
  42. 42. Using Atom Data Format ... QCon 36 2008
  43. 43. Using Atom Data Format ... • Originally designed for blogs not data QCon 36 2008
  44. 44. Using Atom Data Format ... • Originally designed for blogs not data • Drawbacks – syntax constraints (atom:content, atom:updated) QCon 36 2008
  45. 45. Using Atom Data Format ... • Originally designed for blogs not data • Drawbacks – syntax constraints (atom:content, atom:updated) • Benefits – standardized (IETF) – extensible – useful fields (atom:id, atom:updated) – can use a feed reader (cool!) QCon 36 2008
  46. 46. Using Atom Data Format ... • Originally designed for blogs not data • Drawbacks – syntax constraints (atom:content, atom:updated) • Benefits – standardized (IETF) – extensible – useful fields (atom:id, atom:updated) – can use a feed reader (cool!) • Overall, a very good choice QCon 36 2008
  47. 47. Modeling Data in Atom ... QCon 37 2008
  48. 48. Modeling Data in Atom ... • How to indicate extensions / semantics? – Solution: atom:category URI to indicate “kind” – Should have been human readable string QCon 37 2008
  49. 49. Modeling Data in Atom ... • How to indicate extensions / semantics? – Solution: atom:category URI to indicate “kind” – Should have been human readable string • How to indicate related feeds and entries? – Solutions: ... • link@rel, gd:feedLink, gd:entryLink, content@href – No clear standard, try to be consistent QCon 37 2008
  50. 50. Designing New Data Formats ... QCon 38 2008
  51. 51. Designing New Data Formats ... • The Idea: GD-namespace – Create reusable extensions and kinds – Manage extensions in one namespace QCon 38 2008
  52. 52. Designing New Data Formats ... • The Idea: GD-namespace – Create reusable extensions and kinds – Manage extensions in one namespace QCon 38 2008
  53. 53. Designing New Data Formats ... • The Idea: GD-namespace – Create reusable extensions and kinds – Manage extensions in one namespace • Our Experience: – Almost every Google API has extended it – Used only by Google APIs (so far) QCon 38 2008
  54. 54. Designing New Data Formats ... • The Idea: GD-namespace – Create reusable extensions and kinds – Manage extensions in one namespace • Our Experience: – Almost every Google API has extended it – Used only by Google APIs (so far) • Alternatives: – Reuse existing XML schemas (GeoRSS, MediaRSS) – Create an ecology of ‘kinds’ (GData ACL) QCon 38 2008
  55. 55. Care for your Developers • Protocol Centric & Language Agnostic • Open your Code • Give Fair Warnings of Changes • Meet your Developers QCon 2008
  56. 56. Do you feel lucky? • Questions, comments? QCon 40 2008
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×