Web20 Tutorial Cts

Slide 1: Marlon Pierce, Geoffrey Fox Community Grids Lab Indiana University

Slide 2: Acknowledgements  The following people helped create the slides and material for this presentation:  Siddharth Maini  Joshua Rosen  Huapeng Yuan  Yu Deng  Fatih Mustacoglu

Slide 3: We discuss the impact of Web 2.0 on e-Science, Cyberinfrastructure, and Grids

Slide 4: Rich Client Interfaces  Concept of rich and interactive user interfaces that are locally deployed to the client machine  Designed to increase the usability of the interface  Increases the actual User Experience Source: http://www.vw.com/

Slide 5: AJAX  Stands for Asynchronous JavaScript + XML  Ajax is not a technology but a combination of:  Standards-based presentation using XHTML and CSS  Dynamic update and display content using DOM (Document Object Model)  Data communication using XMLHttpRequest  Asynchronous JavaScript  Being Asynchronous in nature  Make requests to the server without a page refresh.  Parse and work with XML documents  Extensively used by Google. E.g. Google Suggest  Used to developed a variety of Web Applications  Web page can communicate with web server online as user enters data into an HTML form Source: http://www.google.com/webhp?complete=1&hl=en

Slide 6: <script type=\"text/javascript\" language=\"javascript\"> var http_request ; function makeRequest(url) { Make an HTTP if (window.XMLHttpRequest) { // For Mozilla, Safari request using http_request = new XMLHttpRequest(); XMLHttpRequest class } else if (window.ActiveXObject) { // For IE http_request = new ActiveXObject(\"Msxml2.XMLHTTP\"); } Provide HTTP request object http_request.onreadystatechange = alertContents; the name of JavaScript object http_request.open('GET', url, true); which will process the response using onreadystatechange http_request.send(null); property function alertContents() { Once the state of if (http_request.readyState == 4) { request AND status code of server response if (http_request.status == 200) { is checked, we can alert(http_request.responseText); process the data } else { alert('There was a problem with the request.'); } } } </script> <span style=\"cursor: pointer; text-decoration: underline;\" onclick=\"makeRequest('test.html')\"> User makes the Make a request request in browser </span>

Slide 7: Output  User clicks the link “Make a Request in the Browser”  This calls the makerequest(“test.html”) with test.html in the same directory  Request is made and then (onreadystatechange) execution is passed to alertContents()  alertContents() verifies the response received and then alert()s the content of test.html file (as part of processing the data)

Slide 8: Limits of AJAX  XMLHttpRequest object lets JavaScript make GET, POST and other types of HTTP requests  But as a security feature you cannot call third party domains through latest web browsers  Exception IE 5 and 6 only under low security settings  You can only make requests back to the original server/hostname  Solution: Some hacks/methods Application proxies  Apache Proxy  JSON 

Slide 9: JSON: JavaScript Object Notation  Called as a “fat-free-alternative” to XML and a serialized JavaScript Object  It is an ultra-simple lightweight data interchange format  Based on subset of JavaScript syntax, array and object literals  Supported as an alternative output format by many Web 2.0 Services  All Yahoo APIs, Google Maps, Del.icio.us, etc.  Extremely fast and powerful  Built-in support in Browsers

Slide 10: JSON, Continued  Can make cross-domain requests unlike AJAX which uses XMLHttpRequest  To make cross-domain requests, just dynamically create your script tags using the DOM  Make web services requests using the dynamic script tag request, and  Web service lets you specify a JavaScript callback function  For more info: http://www.xml.com/pub/a/2005/12/21/json-dynamic-script-tag.html  Much simpler than XML  Mostly used in Ajax web application development  CSS can also be expressed in JSON (http://www.featureblend.com/css-json.html )  JavaScript's eval() method to convert the string into a real JavaScript object  Var data = eval('(' + req.responseText + ')');

Slide 11: JSON Example JSON Syntax Defining Arrays in JavaScript CSS JSON CSS Structure

Slide 12: XML -> JSON  XML can be converted to reversible JSON if:  all subelements names occur exactly once, or …  Subelements with identical names are in sequence.  XML can be converted to irreversible JSON if:  Sub elements are not unique  Element order doesn’t matter Source :Goessner, Stephen. \"XML.com: Converting Between XML and JSON.\" XML.com. 31 May 2006. O'REILLY. 16 May 2007 <http://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html?page=2>.

Slide 13: JSON Compared to XML XML String JSON String <classinfo> { \"classinfo\" : <students> { <student> \"students\" : [ <name>Michael Smith</name> { <average>99.5</average> \"name\" : \"Michael Smith\", <age>17</age> \"average\" : 99.5, <graduating>true</graduating> \"age\" : 17, </student> \"graduating\" : true <student> }, <name>Steve Johnson</name> { <average>34.87</average> \"name\" : \"Steve Johnson\", <age>17</age> \"average\" : 34.87, <graduating>false</graduating> \"age\" : 17, </students> \"graduating\" : false </classinfo> } ] } } Information repeated repeatedly Information isn’t repeated More bytes needed to store the same information Less Bytes needed to store the same information Have to convert string into JavaScript using: JavaScript's eval() method to convert the string into a real JavaScript var books = req.responseXML.getElementsByTagName(‘<element_name'); object E.g. var data = eval('(' + req.responseText + ')'); Source: Zakas, Nicholas C., Jeremy McPeak, and Joe Fawcett. Professional AJAX. 1st ed. WROX Press, 2006.

Slide 14: JavaScript/AJAX/JSON Toolkits  To make writing Web Sites or Web Applications easier to develop  GWT (Google Web Toolkit) http://code.google.com/webtoolkit/  YUI (Yahoo! User Interface Library) http://developer.yahoo.com/yui/  DWR http://getahead.org/dwr  script.aculo.us http://script.aculo.us/  Prototype http://www.prototypejs.org/

Slide 15: GWT (Google Web Toolkit)  Open source Java software development framework  Easier AJAX application development You write your front end in Java using any Java IDE available (e.g. Eclipse, 1. JProfiler, JUnit, IntelliJ…) GWT complier will automatically convert it into browser-complaint 2. JavaScript and HTML Confirm that the Web App. Runs successfully in each browser 3.  GWT Architecture GWT Java-to-JavaScript Compiler: Java-to-JavaScript compiler  GWT Hosted Web Browser: run and execute your GWT applications  JRE emulation library: contains JavaScript implementations of the most  widely used classes in the Java standard class library. GWT Web UI class library: set of custom interfaces and classes that let your  create web browser \"widgets” with use of CSS

Slide 16: Benefits of GWT  Static type checking in the Java language boosts productivity while reducing errors.  Common JavaScript errors are easily caught at compile time rather than at runtime.  Code prompting/completion is widely available.  Java-based OO designs are easier to communicate and understand  Makes your AJAX code base more comprehensible with less documentation.  Use GWT’s set of User Interface (UI) to build UI elements making an AJAX application

Slide 17: More GWT Benefits  Very low Compiler-generated JavaScript size (approx. 100K for a full GWT app.)  End-user performance: GWT apps are always as fast as hand- written JavaScript.  Development time:  Very little debugging time  More time can be spent on application functionality  Makes site more usable by adding “back button” functionality  Use of JavaScript Native Interface (JSNI)  Browser Interoperability  Internationalization: create internationalized libraries and apps.

Slide 18: GWT Example CODE package com.google.gwt.sample.kitchensink.client; import com.google.gwt.core.client.EntryPoint; import com.google.gwt.sample.kitchensink.client.Sink.SinkInfo; import com.google.gwt.user.client.History; import com.google.gwt.user.client.HistoryListener; import com.google.gwt.user.client.ui.DockPanel; import com.google.gwt.user.client.ui.HTML; /** Application that demonstrates all of the built-in widgets. */ public class KitchenSink implements EntryPoint, HistoryListener { protected SinkList list = new SinkList(); private SinkInfo curInfo; private Sink curSink; private HTML description = new HTML(); private DockPanel panel = new DockPanel(); private DockPanel sinkContainer;

Slide 19: GWT Example Source: http://code.google.com/webtoolkit/documentation/examples/kitchensink/

Slide 20: YUI (Yahoo! User Interface Library)  Collection of JavaScript and CSS resources  Makes RIA (Rich Internet Applications) easier to build  Open source  Yahoo! User Interface Library components fall into three groups:  Utilities  UI Controls, and  CSS resources  “Better documented than GWT” (Sidd’s personal opinion)  Can be used with Yahoo! Design Patterns Library to easily implement design patterns

Slide 21: Yahoo Components Utilities   Animation: to Create \"cinematic effects”  Browser History Manager: just like GWT’s Back Button functionality  Connection Manager: to manage XMLHttpRequest transactions in a cross-browser fashion  Data Source Utility: to provide an interface for retrieving data  Dom Collection:  Drag & Drop: Create draggable objects that can be picked up and dropped elsewhere on the page  Event: gives you easy and safe access to browser events  Controls AutoComplete:  Button Control  Calendar  DataTable Control  Logger: Provides easy way to write logs to an on-screen console  Menu  Slider  Tab View  Tree View   CSS Resources CSS Grids   CSS Fonts  CSS Reset

Slide 22: YUI ANIMATION Example http://developer.yahoo.com/yui/examples/animation/anim_basic.html

Slide 23: DWR (Direct Web Remoting)  Java open source library  Allows writing AJAX web sites  Two Main parts:  Java Servlet running on the server to process requests and sends responses back to the browser.  JavaScript running in the browser to send requests and can dynamically update the webpage.  It dynamically generates JavaScript 1. Feels like the execution is happening in the browser 2. But server is executing the code , and 3. DWR converts data back and forth using a callback function  In the example above, DWR dynamically generates the AjaxService class  It then converts all parameters and return values between Java and JavaScript

Slide 24: DWR features  Integrates with  Spring, Struts, WebWork, JSF, Hibernate, Beehive  Supported Environments and Application Servers  Tomcat, Weblogic, Websphere, Jboss, Jetty, Resin, Sun ONE, Glashfish  Ability to asynchronously send data from a web-server to a browser Source: http://getahead.org/dwr/integration

Slide 25: DWR Example (Wal-Mart) 1. Wal-Mart uses the DWR technology in the “Quick View” 2. Clicking “Quick View” will pop-up a dialog with more details, fetched asynchronously using DWR.

Slide 26: Features - Script.aculo.us  Features:  Visual Effects Library: To add animation  Drag and Drop JavaScript Library: enables drag-and-drop of elements in your Web App.  Dom utilities: Create DOM elements dynamically  Ajax Controls Drag and Drop  Draggables  Autocompletion  In Place Editing   Integration with development frameworks Ruby On Rails, Perl, Nitro, PHP, Java  Plone, DotNet, Symfony, Seaside  AIDA/Web, Open ACS, Django   Java Script Unit Testing Example Test Results Includes unit and functional tests for itself  Catch scripting and syntax errors and presenting them in a useful way without breaking the tests  Source: http://wiki.script.aculo.us/scriptaculous/show/UnitTesting 

Slide 27: EXAMPLE  Used animation and effects library to replace the Flash animation on the home page.  Allows easier maintenance of the drop-down menu

Slide 28: Summary GWT YUI DWR Script.aculo.us Code in Java and generate JavaScript/HTML YES NO YES NO automatically Internationalization YES NO NO NO (easily create international Libraries) Ruby on Rails, Perl, Eclipse, IntelliJ, Spring, Structs, WebWork, Nitro, Eclipse, Integration with frameworks NO JProfiler, JUnit JSF, Hibernate, beehive SeaSide, Django, Plone, Java, PHP

Slide 29: Google Gadgets & Widgets  Visually appealing mini-applications that work with Google Homepage, Google Desktop, or any page on the web E.g. Calendar, Mail Checker, NASA TV, Stock Prices, Weather Globes,  Free SMS etc. Enable a wide variety of visual effects and animation  Supports rich markup languages such as Flash, XML  Two Types of Google Gadgets   Universal Gadgets  Desktop Gadgets Date & Time (Universal Gadget) Google Gadget in Google Desktop (undocked and docked view

Slide 30: Universal Gadgets Desktop Gadgets Work in Google Desktop, Google Page Creator, any Works only with Google Desktop Web Page, Google Personalized Page, Blogger Easy to Create - No downloads necessary, No libraries to learn, No Web server required Anything that can be done on a WebPage Client Win32 Libraries Features HTML Forms Multi-user support through Google Talk Supported Easy integration with Google Maps/ Web e.g. Interactive Games Free form shapes and transparencies Search ActiveX Search files, emails etc across computers Easy to use standard UI elements React to user actions outside a gadgets Create Yes No without Download Offline No Yes Availability Languages HTML, JavaScript, generated HTML (eg PHP, JavaScript, C, C++, C#, and/or VB.Net Supported Java, Perl, ASP) \"Google Code - Google Gadgets.\" Google Code. Google Inc.. 17 May 2007 <http://code.google.com/apis/gadgets/>.

Slide 31: Creating Google Gadgets  Google Desktop SDK is used for creating Desktop Gadgets using Google Desktop Gadget API  Google Gadgets API is used to develop Universal Google Gadgets

Slide 32: How to Make a Desktop Gadget  Download the Google Desktop SDK and create the following files  gadget.gmanifest files specifies (required)  Metadata XML file describing Gadget  Components needed to be installed before gadget creation <gadget> element with minimumGoogleDesktopVersion parameter <about> element (required) <install> element (optional)  main.xml file (required)  Defines the <view> element to specify the overall appearance of the pane  main.js file to write the code for gadget functionality (required)  To handle the event  options.xml file (optional) to add options view (optional)  Strings. xml file  Contains all language-specific strings that will be visible in your gadget's UI  Contains variable assignments for the strings in a particular language.

Slide 33: gadget.gmanifest file  The <about> element can include these sub-elements: <id> : The gadget's CLSID.  <name> : Your gadget's name. (Required)  <description> : A short description of what the gadget does. (Required)  <version> : The gadget's version number.  <author> : Who wrote the gadget.  <authorEmail> : An email address for contacting the developer.  <authorWebsite> : A URL for your website.  <copyright> : A copyright notice.  <aboutText> : Text displayed in the gadget's About dialog.  <smallIcon> : A 24x24 pixels icon shown in your gadget's title bar.  <icon> : A 32x32 pixels icon shown in the gadget's About dialog and in the Alerts UI  <eventAPI> : the gadget can use the Google Desktop Event API.  <displayAPI usesNotifier=\"true\" /> : When set to true, enables Sidebar UI notifications.  <queryAPI allowModifyIndex=\"true\" /> : When set to true, allows the gadget to use the Query API and  index notifications.

Slide 34: main.xml file (example)  Specified that the gadget has main.js as the scripting file  Defines 250 X 150 pixel view for the label  Set the horizontal and vertical position  Set the alignment, size of text, width, horizontal and vertical pin of the label “HELLO_WORLD” (present in strings.xml file)  Retrieves the name “iulabs” for the label  Enable the element to fire mouse/keyboard events  Calls “onTextClick() function defined in main.js file

Slide 35: Main.js file (example)  Specify the code to spin the text clockwise  Taking 1500 milliseconds between 0 – 360 degrees  It also displays the caption “GADGET_COPYRIGHT” in the expanded view Google Gadget in Google Desktop (undocked and docked view

Slide 36: Strings.xml file (example)  Specify the metadata about the string  Here “HELLO_WORLD” element contains the string “IU Research” which is displayed when the gadget is run in Google Desktop

Slide 38: Social Bookmarking / Tagging  Users save a list of internet resources  They assign keywords or tags to each of these resources  This method of categorization is also called as folksonomy  Most bookmarking websites allow users to search for bookmarks on tags or keywords.

Slide 39: Advantages / Disadvantages  Advantages  You can get to your bookmarks from anywhere  You can share your bookmarks with your friends, coworkers, supervisor etc.  You can categorize or group/bundle your bookmarks according to your wish  You can search of other users registered on that website and vice versa  You also have the option to make your tags private  Disadvantages  No controlled vocabulary  No standard for structure of tags e.g. capitalization, singular, plural etc.  spelling mistakes in tags  Tags having multiple meanings  No hierarchical relationship  Spamming – bookmarking same page multiple times

Slide 40: Del.icio.us  Store links to your favorite internet resources  Share links / favorites with friends, family, coworkers, and the del.icio.us community.  Discover new things.  On del.icio.us, you can use tags to organize and remember your bookmarks, which is a much more flexible system than folders.  You can bundle the tags into groups  Example Uses  Research: keeping track of your research materials  Wish list: maintain a wish list  Collaboration: friends, coworkers and other groups can use a shared account

Slide 41: Ways to Use del.icio.us  Web interface  RSS Feeds  Application Programming Interfaces (API)  Embed JavaScript code inside your web page

Slide 42: Personal Bookmarks * Option to make bookmarks hidden Bookmarks of other users who used common tags * Option to add any user to your network Subscribe to interesting tags to be aggregated Add users to your network and browse their tags * Option to disable sharing of your network

Slide 43: Group / Network Individual Feed Feed  I can subscribe to my own feeds and feeds from any other use registered on del.icio.us

Slide 44: Application Programming Interfaces  All del.icio.us APIs are done over https and require HTTP-Auth  Example  https://api.del.icio.us/v1/tags/get  Returns a list of tags and number of times used by a user.  Output: XML  Code Output  <? require_once 'HTTP/Request.php'; -> require_once '/home/smaini/vals.php'; $req =& new HTTP_Request(\"https://api.del.icio.us/v1/tags/get\"); $req->setBasicAuth(delTagUser,delTagPass); $response = $req->sendRequest(); echo $req->getResponseBody(); >

Slide 45: API (continued)  Update  https://api.del.icio.us/v1/posts/update Returns the last update time for the user   https://api.del.icio.us/v1/posts/all? Returns to check if the data has changed since last fetch   Tags  https://api.del.icio.us/v1/tags/rename?old=horse&new=horses  Rename an existing tag with a new tag name Arguments required  &old (required) - tag to rename.  &new (required) - new name. 

Slide 46: DELICIOUS API  Posts  https://api.del.icio.us/v1/posts/get?  https://api.del.icio.us/v1/posts/recent?  https://api.del.icio.us/v1/posts/all?  https://api.del.icio.us/v1/posts/dates?  https://api.del.icio.us/v1/posts/add?  https://api.del.icio.us/v1/posts/delete?  Bundles  https://api.del.icio.us/v1/tags/bundles/all  https://api.del.icio.us/v1/tags/bundles/set?  https://api.del.icio.us/v1/tags/bundles/delete?

Slide 47: JavaScript Widget for Del.icio.us JavaScript can be embedded into your HTML code as a “Bookmarklet” This code can load a JavaScript Object that contains your latest set of tags <a href=\"javascript:location.href= 'http://del.icio.us/post?v=2&url='+encodeURIComponent(do cument.location.href)+' &title='+encodeURIComponent(document.title)+' '\"> Post to del.icio.us </a> Source: http://ekstreme.com/seo/socialbookmarkingcode.php

Slide 48: Connotea  Free online references management for scientists and clinicians  One can save and organize references  Step 1: Add the reference to Connotea’s library as a Bookmark (using browser button, add form or DOIs)  Step 2: Tag the reference and type in the title  Find users who used your tags  Find Bookmarks matching your tags  Entire library can be exported to the following formats:  RIS: Suitable for importing into Reference Manager and similar software  EndNote: Exporting to MS EndNote  BibTex: For LATEX  MODS: U.S. Library of Congress Metadata Object Description Schema (MODS) format  RSS Feeds, RDF (Resource Description Framework)  Can Import links / references using local file  RIS, BibTeX, EndNote, MODS, ISI Web of knowledge, Firefox bookmarks

Slide 49: Automatic Collection of Bibliographic Information  Connotea will add automatic bibliographic information for pages saved from the following sources  Nature.com, PubMed, PubMed Central, Science  PloS, BioMed Central, Supported Eprints repositories  Supported Highwire Press publications  Blackwell Synergy, Wiley Interscience, Scitation  arXiv.org, Smithsonian/NASA Astrophysics Data system  Amazon, HubMed, D-Lib Magazine

Slide 50: Web Interface 3) Type in the Display Title, Tags, Descriptions etc. 1) Find an interesting article to be tagged 2) Click on “Add to Connotea” button in the browser 4) This tags the Article of Interest in “My Library”

Slide 51: RSS Feeds Group RSS Feed Individual RSS Feed

Slide 52: Multiple Users and Tags  Clicking around on user and tag names allows you to view the articles for one user or one tag  But you can construct a URL for combining the tags using AND and OR operators  To see a list of articles for the users fdr and jfk, construct the URL as: http://www.connotea.org/user/fdr/jfk  This, in fact, filters for fdr OR jfk http://www.connotea.org/user/fdr+jfk  A plus signs means AND, whereas a forward slash means OR.

Slide 53: More Tag Searching Rules This works for tag names too, and you can combine user names, tag names, +'s and /'s in any combination. For example: http://www.connotea.org/user/fdr+hst/jfk+lbj/tag/topsecret/classified gives you a list of articles tagged as 'topsecret' or 'classified' by both fdr and hst, or by both jfk and lbj. After getting the results you have the option to export the list as any of the formats mentioned including RSS Feeds

Slide 54: Programming Interface  API Version 1 released wrapper libraries in Perl and Ruby  URL Structure: http://www.connotea.org/data /bookmarks or /tags or '' (empty string, which means 'posts') /user/ [username] /tag/ [tagname] /date/ [date of form YYYY-MM-DD ] /uri/ [uri or hash] ? q= [free text search string] & num= [number of results per] & start= [result number to start at]  All these are standard HTTP requests  Use GET to retrieve the URL  Output Format: RDF Format (Resource Descriptor Framework Format) Example: http://www.connotea.org/data/tags/user/sidds1601

Slide 55: Embedding Connotea JavaScript Widgets <a style=\"cursor:pointer;\" onclick=\"javascript: w=open('http://www.connotea.org/addpopup?'); <img src=\"images/connotea.png\" border=\"0“ alt=\"add bookmark to connotea\" align=\"absmiddle\">Add to Connotea</a> Will display “Add to Connotea” icon in your webpage.

Slide 56: CiteULike Purpose: To share, store, and organize the papers Provides “Bookmarklets” to extract information from current page Can manually post an article Can add URLs/DOIs and bibliographic metadata using its supported sites You can add tags to your own or other entries

Slide 57: CiteULike  It also provides extra bibliographic information from various databases  It is NOT open source  It can only import references from BibTex  It can export reference list to:  EndNote or  BibTex format  Supports Watch Lists: Page with papers that may be relevant to your field of study in the future.  Note: you can watch a page only on one tag and not more than one.  You only have the option to watch a page when you click on the active tags on the site.

Slide 58: Web Interface Click to save it as a reference

Slide 59: RSS Feeds Entire libraries can be exported as an RSS feed.

Slide 60: Inserting JavaScript Code  <script type=\"text/javascript\" src=\"http://static.citeulike.org/button.js\"> </script> Citeulike icon in your Web Page

Slide 61: Similarities / Comparison Connotea CiteUlike Bookmarklets Yes Yes Export Formats RIS EndNote: EndNote BibTex: BibTex MODS RSS Feeds RDF Tag Sorting Order of Entry Alphabetically OpenSource Yes No

Slide 62: A short guide to REST-style Web Services.

Slide 63: Representational State Transfer  REST is HTTP as a CS Ph. D. thesis.  Roy Fielding, co-author of HTTP specification  Standard Web Services have  WSDL is an API language.  SOAP is the network message envelop  REST proposes to use Internet as programming platform with only HTTP.  Use HTTP to ship machine process-able content, not just HTML.  Simple (simplistic) but scales.  Clients to REST services have one API for all occasions.  HTTP GET, PUT, POST, DELETE  Operations are performed on URLs.  Very suitable for AJAX and JSON programming  REST services are stateless (or idempotent).  Identical requests give identical responses.

Slide 64: REST Services, Continued  Content of URLs should be XML or similar encoded data (not just HTML).  No universal message format like SOAP, but RSS and Atom are commonly used.  You could use SOAP also.  Real implication is that there are no SOAP headers  No SOAP header processing  Quality of REST services are what you get with HTTP and TCP/IP. No additional SOAP QOS layer   No SOAP relay messaging, fault handling, RPC encoding, etc.

Slide 65: An April Fools Critique of REST  REST tends to attract passionate religious debates  Sanjiva Weerawarana sees the light:  http://www.bloglines.com/blog/sanjiva?id=196  Weerarana is co-author of WSDL specification, project leader of IBM Web Service efforts, Apache Axis 2, CEO of WSO2  See also his more serious comments at  http://www.infoq.com/articles/sanjiva-rest-myths echo “Groundskeeper Willie: It won't last. Brothers and sisters are natural enemies. Like Englishmen and Scots! Or Welshmen and Scots! Or Japanese and Scots! Or Scots and other Scots! Damn Scots! They ruined Scotland! Principal Skinner: You Scots sure are a contentious people. Groundskeeper Willie: You just made an enemy for life!” | sed “s/Scots/RESTifarians/”

Slide 66: We examine Amazon’s Simple Storage System as a REST case study.

Slide 67: • S3 issues two codes to each account. • An account access key: your identity • Shared secret key: used to digitally sign communications • In each HTTP request, you have to add an Authorization field. • It will use the account access key and a “Signature” which is a HMAC-SHA1 hash of the request (sans “Authorization” line) using the secret access key as the key. The authorization line is formed like so: “Authorization: AWS “+ AWSAccessKeyID + “:” + Signature An example : Authorization: AWS 1ATXQ3HHA59CYF1CVS02:SZf1cHmQ/nrZbsrC13hCZS061 yw=

Slide 68: Buckets: Grouping Your Files  Objects (stored files) are stored in buckets.  An account can have multiple buckets.  These bucket names are not user specific.  In other words, if aS3 user is already using a desired bucket name, that name is unavailable for everyone else.  This bucket name will be used in the url for your resources. An example would be : http://s3.amazonaws.com/bucket_name/sample.jpeg

Slide 69: Buckets (cont’d) Buckets are created with an http PUT formed like this : PUT /[bucket-name] HTTP/1.0 Date: Wed, 08 Mar 2006 04:06:15 GMT Authorization: AWS [aws-access-key-id]:[header-signature] Host: s3.amazonaws.com If properly formed it would return a response of HTTP/1.1 200 OK x-amz-id-2: VjzdTviQorQtSjcgLshzCZSzN+7CnewvHA+6sNxR3VRcUPyO5fmSmo8bWnIS52qa x-amz-request-id: 91A8CC60F9FC49E7 Date: Wed, 08 Mar 2006 04:06:15 GMT Location: /[bucket-name] Content-Length: 0 Connection: keep-alive Server: AmazonS3

Slide 70: Objects: Stored Files  There is more to these objects than the content of the file.  Metadata can be included with each object.  Name/value pair collections  The objects must have unique names (keys) in regards to their own bucket.  In other words, “s3.txt” can exist in multiple buckets, but only once in a single bucket.  There are no “sub-buckets” so many programmers decide to group files by prefixing. For instance, all pictures would start with “Pictures.”  This works well with the “list” operation.

Slide 71: Replacing/Overwriting Objects  If a file is uploaded that has a key that already exists, that file is then replaced.  This queue is based on when the file completes upload.  So if one user starts to upload a large file to foo.bar and another one starts a much smaller file to that same key, even thought the smaller one started last, it is quite possible the larger one will overwrite the smaller one when it finishes.  The firs S stands for “simple”

Slide 72: The HTTP request to upload a file will look like this: PUT /[bucket-name]/[key-name] HTTP/1.0 Date: Wed, 08 Mar 2006 04:06:16 GMT Authorization: AWS [aws-access-key-id]:[header-signature] Host: s3.amazonaws.com Content-Length: 14 x-amz-meta-title: my title Content-Type: text/plain this is a test This will give the following response: HTTP/1.1 200 OK x-amz-id-2: wc15E1LUrjDZhNtT4QZtsbtadnOMKGjw5QTxkRDVO1owwbA6YoiqJJEuKShopufw x-amz-request-id: 7487CD42C5CA7524 Date: Wed, 08 Mar 2006 04:06:16 GMT ETag: \"54b0c58c7ce9f2a8b551351102ee0938\" Content-Length: 0 Connection: keep-alive Server: AmazonS3

Slide 73: File Permissions  There are extensive rules to read and write access to objects and buckets.  These rights are stored in an ACL (access control list), which is an XML document.  Rights can be granted to users on a one to one basis, or to pre-defined groups.

Slide 74: • READ: For a bucket, allows listing of the objects in that bucket. For an object, allows reading of data and/or metadata • WRITE: For a bucket, allows the creation and deletion of all new and existing objects in this bucket. There is no support or WRITE on an object. • READ_ACP: Allows the reading of a bucket or object’s ACL. • WRITE_ACP: Allows the changing of a bucket or object’s ACL. • FULL_CONTROL: Grants all of the above permissions. No other rights are added with this type.

Slide 75: Permissions (Grantee Types)  User : Has to be a user of S3.  Can be identified either by e-mail address or by an id supplied by Amazon (canonical).  Owner : Always has full rights and is always the creator of the object.  Group : Currently there are only two groups: all users and authenticated users  Rights given by these groups do not overwrite other access control considerations.  http://acs.amazonaws.com/grouops/global/AllUsers: Represents everyone, anonymous or authenticated.  http://acs.amazonaws.com/groups/global/AuthenticatedUse rs: Represents non-anonymous users.

Slide 76: <AccessControlPolicy> <Owner> <ID>a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9</ID> <DisplayName>chriscustomer</DisplayName> Owner </Owner> <AccessControlList> <Grant> <Grantee xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:type=\"CanonicalUser\"> <ID>a9a7b886d6fd24a52fe8ca5bef65f89a64e0193f23000e241bf9b1c61be666e9</ID> <DisplayName>chriscustomer</DisplayName> Single User </Grantee> <Permission>FULL_CONTROL</Permission> Entry </Grant> <Grant> <Grantee xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:type=\"Group\"> <URI>http://acs.amazonaws.com/groups/global/AllUsers<URI> </Grantee> <Permission>READ</Permission> </Grant> Group Entry </AccessControlList> </AccessControlPolicy>

Slide 77: How to Use S3 With PHP programming examples

Slide 78: Define Constants • When a new account is created, it will have a key and “secret code” attached to it. • These should be placed in a separate file and be defined as constants. • For these purposes, they will be known as ‘amazonKey’ and ‘amazonSecret’, respectively. define (“amazonKey”,”15B4D3491F177624206A”); define (“amazonSecret”,”(secret code given by S3)”); define (“amazonURL”,”http://s3.amazonaws.com/” ); define (“amazonBucket”,”BucketName”);

Slide 79: S3 requires the use of HTTP Requests and RFC 2104 compliant hashes. Luckily, the Pear framework (which comes with PHP) has made packages for these purposes. Before using these packages, they must be added to PHP. Simply run these commands (pear is found in /bin under the /php directory). $ pear install HTTP_Request $ pear install Crypt_HMAC Any php script that will use these packages must include these two lines. require_once ‘Crypt/HMAC.php’; require_once ‘HTTP/Request.php’;

Slide 80: Two functions will need to be created to facilitate the use of S3. function hex2b64($str) Converts a string from { hex to base 64 $raw = ''; for ($i=0; $i < strlen($str); $i+=2) { Constructs the $raw .= chr(hexdec(substr($str, $i, 2))); “Signature” using the } return base64_encode($raw); secret key to hash the } given string and encode it function constructSig($str) { $hasher =& new Crypt_HMAC(amazonSecret, \"sha1\"); $signature = hex2b64($hasher->hash($str)); return($signature); }

Slide 81: function createBucket($bucket, $acl = 'private') { $httpDate = gmdate(\"D, d M Y G:i:s T\"); This section echo $httpDate; constructs the $stringToSign = \"PUT\\n\\n\\n$httpDate\\nx-amz- acl:$acl\\n/$bucket\"; signature $signature = constructSig($stringToSign); $req =& new HTTP_Request(amazonUrl . $bucket); $req->setMethod(\"PUT\"); $req->addHeader(\"Date\", $httpDate); $req->addHeader(\"Authorization\", \"AWS \" . amazonKey. \":\" . $signature); This section $req->addHeader(\"x-amz-acl\", $acl); constructs the $response = $req->sendRequest(); headers, and creates $responseCode=$req->getResponseCode(); the signature if ($responseCode == 200) { return true; } else { Send the request and return return false; whether or not it was successful } }

Slide 82: • Once that function is created, it’s pretty simple to create a bucket. • It is usually more desirable to keep the ACL private, so we’ll keep that blank. • The objects under it can still be made publicly readable, which allows browser access • This prevents others from searching the directory, and adding objects of their own. So in this instance, running this line (see previous slide): createBucket(amazonBucket); should create an HTTP request looking like : PUT / BucketName HTTP/1.0 Content-Length: 0 Authorization: AWS 15B4D3461F177624206A:YFhSWKDg3qDnGbV7JCnkfdz/IHY= Date: Thu, 17 Nov 2005 02:40:52 GMT

Slide 83: function putObject($bucket, $key, $filePath, $contentType, $contentLength, $acl, $metadataArray=array(), $md5=\"\"){ sort($metadataArray); Prepare the request $resource = \"$bucket/$key\"; $resource = urlencode($resource); $req = & new HTTP_Request($this->serviceUrl.$resource); Add the necessary Headers $req->setMethod(\"PUT\"); $httpDate = gmdate(\"D, d M Y G:i:s T\"); $req->addHeader(\"Date\", $httpDate); $req->addHeader(\"Content-Type\", $contentType); Includes an md5 if specified $req->addHeader(\"Content-Length\", $contentLength); $req->addHeader(\"x-amz-acl\", $acl); if($md5){ Add the contents of $MD5 = $this->hex2b64(md5_file($filePath)); $req->addHeader(\"Content-MD5\", $MD5); the file to the body of } $req->setBody(file_get_contents($filePath)); the request $stringToSign=\"PUT\\n$MD5\\n$contentType\\n$httpDate\\nx-amz-acl:$acl\\n\"; foreach($metadataArray as $current){ if($current!=\"\"){ $stringToSign.=\"x-amz-meta-$current\\n\"; $header = substr($current,0,strpos($current,':')); $meta = substr($current,strpos($current,':')+1,strlen($current)); $req->addHeader(\"x-amz-meta-$header\", $meta); Creates the signature, } } with metadata $stringToSign.=\"/$resource\"; $signature = $this->constructSig($stringToSign); $req->addHeader(\"Authorization\", \"AWS \" . $this->accessKeyId . \":\" . $signature); $response = $req->sendRequest(); $responseCode = $req->getResponseCode(); if (responseCode == 200) { Send the request return true; } else { and return the echo $req->getResponseBody(); response return false; } }

Slide 84: In this instance, a file is being uploaded that is simply called “Message”. This is a simple text file with the contents “Paper or Plastic”. putObject (amazonBucket, $_FILES[‘upFile’][‘name’], $_FILES[‘upFile’][‘tmp_name’], $_FILES[‘upFile’][‘type’], filesize($_FILES[‘upFile’][‘tmp_name’]), ‘public_read’); This will produce the HTTP request seen here : PUT /BucketName/Neo HTTP/1.0 Content-Length: 16 Authorization: AWS 15B4D3461F177624206A:xQE0diMbLRepdf3YB+FIc8F2Cy8= Date: Thu, 17 Nov 2005 07:48:33 GMT Content-Type: text/plain Paper or Plastic Which will upload this file to the S3 server which can be accessed either by the REST service or directly by accessing the link: http://s3.amazonaws.com/BucketName/Neo

Slide 85: A brief overview of news feeds, how to create, and how to consume.

Slide 86: Atomsphere  An Atom feed library written in Java http://www.colorfulsoftware.com/projects/atomsphere  Download packages  atomsphere  atomsphere-taglib  atomsphere-webapp  atomsphere-weblib  Add jar files included in the packages above to the project

Slide 87: Create Atom Feed for Bay1Temp  Create an atom feed document  add attribute “xmlns” to be “ http://www.w3.org/2005/Atom” to the feed  add elements “Author”, “Id”, “Title”, “Link” , “Updated”  Add an entry document to the feed  add elements “Author”, “Id”, “Title”, “Updated”  add element “Content” which contains parameters’ value of Bay1Temp

Slide 88: Java Code for creating an atom feed doc // New a feed Feed theFeed = new Feed(); // Add \"xmlns\" attribute to the feed theFeed.addAttribute(new Attribute(\"xmlns\",\"http://www.w3.org/2005/Atom\")); // Add Author theFeed.addAuthor(new Author(\"Yu(Carol) Deng\")); // Set a universally unique Id for the feed theFeed.setId(new Id(\"urn:uuid:\" + new UID().toString())); // Add Title Title feedTitle = new Title(\"text\"); // Set title type for the feed feedTitle.setText(\"Bay1Temp Atom Feed\"); // Set title content theFeed.setTitle(feedTitle); // Set the title // Add Link Link feedLink = new Link(); // New a Link in the feed feedLink.setRel(new Attribute(\"rel\", \"self\")); //Set \"rel\" attribute of the link feedLink.setType(new Attribute(\"type\", \"application/atom+xml\")); //Set \"type\" attribute of the link feedLink.setHref(new Attribute(\"href\", FeedHref)); //Set \"href\" attribute of the link theFeed.addLink(feedLink); // Set Updated to the entry theFeed.setUpdated(new Updated(new Date()));

Slide 89: Code for adding an entry doc to feed // New an Entry parcelEntry = new Entry(); // Add Author parcelEntry.addAuthor(new Author(\"Yu(Carol) Deng\")); // Add Title Title parcelTitle = new Title(\"text\"); // Set title type for the feed parcelTitle.setText(\"SensorName, TimeStamp, DoubleData\"); // Set title content parcelEntry.setTitle(parcelTitle); // Set the title // Set a universally unique Id for the entry parcelEntry.setId(new Id(\"urn:uuid:\" + new UID().toString())); // Set Updated to the entry Calendar cal = new GregorianCalendar(); parcelEntry.setUpdated(new Updated(cal.getTime())); // Set the current data to the Content parcelEntry.setContent(nodeSensorName + nodeTimeStamp + nodeDoubleData); // Add the Entry to the feed currentFeed.addEntry(parcelEntry);

Slide 90: Atom Feed for Bay1Temp <?xml version=\"1.0\" encoding=\"utf-8\" ?> <feed xmlns=\"http://www.w3.org/2005/Atom\"> <id>urn:uuid:29fefa08:112967cfbb9:-8000</id> <updated>2007-05-16T16:07:16.376-05:00</updated> <title type=\"text\">Bay1Temp Atom Feed</title> <author> <name>Yu(Carol) Deng</name> </author> <link rel=\"self\" type=\"application/atom+xml\" href=\"http://hagar.cs.indiana.edu:8181/foo.xml\" /> <entry> <id>urn:uuid:29fefa08:112967cfbb9:-7fff</id> <updated>2007-05-16T16:03:32.423-05:00</updated> <title type=\"text\">SensorName, TimeStamp, DoubleData</title> <author> <name>Yu(Carol) Deng</name> </author> <content type=\"html\">Bay1Temp 2007-05-16 20:06:35Z 25.2<br>Bay1Temp 2007-05-16 20:06:55Z 25.1<br>Bay1Temp 2007-05-16 20:07:15Z 25.1</content> </entry> </feed>

Slide 91: Bay1Temp Atom Feed in iGoogle

Slide 92: Making Your Own Feed Consumer  There are plenty of libraries for consuming feeds that you can embed in your own Web pages, blogs, wikis, etc.  I looked at two for PHP:  Magpie RSS: wasted an afternoon trying to get this to work.  SimplePie: worked in 5 minutes.  We’ll look at SimplePie for MediaWiki

Slide 93: Adding SimplePie to MediaWiki, Part I  These steps require access to MediaWiki’s directories under Apache  Download SimplePie and put the simplepie.inc file in your MediaWiki’s “extensions” folder.  Download the MediaWiki plugin and put it (simple_pie.php) in your extensions folder also.  Edit LocalSettings.php to add the line  include(\"./extensions/simplepie_mediawiki.php\");  Next steps can be done by anyone.

Slide 94: Adding a Feed, Part II  Let’s say you want to add your blog’s RSS feed to your team’s wiki.  Create a new Wiki page and edit it. To show just dates  Part of the art of a wiki... and titles, do this: <feed showdesc=\"false\" showdate=\"true\"> ... In the text area, add a line like </feed> <feed> http://myblog.host.org/rss/ </feed>

Slide 95: Use blog to create posts. Display blog RSS feed in MediaWiki.

Slide 96: Aggregating the Rich Internet Experiences and other buzz phrases

Slide 97: What Is a Portal?  Traditionally, a portal is personalized Web page that recognizes you.  You have to log in and set cookies.  You get customized content and layouts.  Typically newsfeeds but this is also a good model for science gateways.  Portals have a component model. Collections of components are managed by a portal container.  Enterprise portals are based on standards like  JSR 168 (Java)  Web Services for Remote Portlets (WSRP)  Web 2.0 equivalent is called a Start Page.

Slide 98: Science Portals  Science Portals are often built using portlet components.  Ex: TeraGrid Science Gateways  Portlets are a server side technology.  Can be built with enterprise technologies like JSF.  Users can select from available portlets and customize their layouts.

Slide 99: Web 2.0 Challenges for Science Portals  Client-side integration of components from multiple service providers.  Start Pages do this all the time with widget “ecosystems”  Multiple views of the same Web application.  Consider del.icio.us: Browser, JSON, REST/XML, JavaScript Widget interfaces all to the same application.  You don’t have to go to http://del.icio.us to use it.  Simple programming interfaces that encourage “do it yourself” science mash-ups.  JSON, AJAX compatible REST APIs.  Widgets/gadgets that allow portal capabilities to be exported to other Web sites and start pages.  Easily add stuff like Digg, YouTube, MySpaces, etc.

Slide 100: Netvibes Widgets Google Gadgets

Slide 101: Portals, Portlets Start Pages, Gadgets  Integrate content from  Containers only show content deployed on the portal server. anywhere.  Users can only choose from portlets  Content is under complete control deployed in the container. of user.  Portlets have Java programming  Universal widget modules with API. supporting JavaScript libraries.  Requires expertise in Java web  Any standalone HTML application programming (JSP, JSF, Struts, etc). can be converted into a  Portlets must be deployed on the widget/gadget. server that runs the portal  Javascript expertise needed to make container. sophisticated widgets.  Only the portal administrator can  Gadgets are easily created, add a new portlet published and shared.  No way to share running JSR 168  Anyone can write a gadget. portlets between portal containers.  Gadgets can be anywhere on the  WSRP is supposed to solve this. Web  Iframes are more practical  Netvibes ecosystem  But the portal administrators have  But you don’t have access to complete control over the Netvibes or Google container container and content. source code.  You download and run everything on your server.  Downloadable Start Page containers?

Slide 102: Writing a Google Gadget  We first create a simple XML description for the gadget and place in a URL.  For example, content of the gadget descriptor located will be located at http://hostname:8080/gridsphere/ogcegadget.html

Slide 103: Gadget XML Description Content of ogcegadget.html is <?xml version=\"1.0\" encoding=\"UTF-8\" ?> <Module> <ModulePrefs title=\"OGCE Portal\" /> <Content type=\"url\" href=\"http://hostname:8080/gridsphere/ogce1.html \" /> </Module>

Slide 104: Add it to your iGoogle page in the usual fashion (click “Add Stuff”). Gadget shows up in your layout.

Slide 105: Writing a Netvibes Widget  Basic steps are as follows:  Write an HTML page.  Decorate with Netvibes metatags (required).  Use Netvibes CSS style sheets (optional)  Use Netvibes JavaScript libraries (optional)  Add to your Netvibes content.

Slide 106: <?xml version=\"1.0\" encoding=\"utf-8\"?> <!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\"> Add required <html xmlns=\"http://www.w3.org/1999/xhtml\"> Netvibes meta tags. <head> <meta name=\"author\" content=\"Huapeng Yuan\" /> <meta name=\"description\" content=\"A Netvibes Widget for OGCE\" /> Use Netvibes <m