Incorporating Web Services in Mobile Applications April 3, 2009 – San Francisco Chuck Hudson Aduci.com – Web and Mobile Apps

April 3, 2009 – San Francisco

Chuck Hudson

Aduci.com – Web and Mobile Apps

Disclaimer Web services and APIs of third parties referenced in this presentation are owned by the third parties and governed by their agreements. Before using any of the APIs or code referenced in this presentation review the terms of use by the third party providing the API. These slides are available under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 License. You can blog, photograph, and share this information with others. Trademarks and brands are the property of their respective owners.

Web services and APIs of third parties referenced in this presentation are owned by the third parties and governed by their agreements. Before using any of the APIs or code referenced in this presentation review the terms of use by the third party providing the API.

These slides are available under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 License. You can blog, photograph, and share this information with others.

Trademarks and brands are the property of their respective owners.

Introduction - Convergence Available APIs are growing daily Currently 1229 (and growing) APIs registered at Programmable Web Shifts in Consumer Expectations Consumers now expect mobile devices to be connected to Internet Sites and Services Platform Development becoming easier and more standardized iPhone SDK – Greater than 50,000 registered developers Android – 5% of smart phone web usage in just three months 1 Seeing extension of enterprise applications to mobile devices Salesforce.com 1 Techcrunch, March 24, 2009, Erick Schonfeld

Available APIs are growing daily

Currently 1229 (and growing) APIs registered at Programmable Web

Shifts in Consumer Expectations

Consumers now expect mobile devices to be connected to Internet Sites and Services

Platform Development becoming easier and more standardized

iPhone SDK – Greater than 50,000 registered developers

Android – 5% of smart phone web usage in just three months 1

Seeing extension of enterprise applications to mobile devices

Salesforce.com

1 Techcrunch, March 24, 2009, Erick Schonfeld

Introduction Many Web Services and APIs were originally developed with server to server or server to browser in mind, not mobile applications Mobile platforms have their own set of challenges given: Bandwidth Memory and CPU Availability Storage Capacity Connectivity Options and Issues Security User Interaction and Display Web Service

Many Web Services and APIs were originally developed with server to server or server to browser in mind, not mobile applications

Mobile platforms have their own set of challenges given:

Bandwidth

Memory and CPU Availability

Storage Capacity

Connectivity Options and Issues

Security

User Interaction and Display

Mobile Integration Challenges

API Developer Programs Is mobile access allowed? Other considerations: Call Limitations (# per second, total per hour/day, pricing above) Caching and Storage of Data Persistence of Data – Length of time stored Freshness of Data – Length of time before refreshing

Is mobile access allowed?

Other considerations:

Call Limitations (# per second, total per hour/day, pricing above)

Caching and Storage of Data

Persistence of Data – Length of time stored

Freshness of Data – Length of time before refreshing

Persistence Limitations Refreshing Requirements Caching and Storage Limits

Persistence Limitations

Refreshing Requirements

Certification of your Mobile Application Some API sets require certification on top of distributor certification Determine Costs / Timeframe / Effort given a mobile app Prepare for testing – typically looking for error handling, API abuse Certification is a positive! Credibility, validation, marketing Keeps the neighborhood safe Application Verification

Certification of your Mobile Application

Some API sets require certification on top of distributor certification

Determine Costs / Timeframe / Effort given a mobile app

Prepare for testing – typically looking for error handling, API abuse

Certification is a positive!

Credibility, validation, marketing

Keeps the neighborhood safe

Mobile Techniques

Authentication vs. Authorization The Difference Authentication from the API provider – API Key Authorization from the user – authToken Session Key By providing combination of API Key and authToken can then receive the session key What is the shelf life of the session key Authorization will commonly affect user’s experience on mobile eBay authentication and authorization screens

The Difference

Authentication from the API provider – API Key

Authorization from the user – authToken

Session Key

By providing combination of API Key and authToken can then receive the session key

What is the shelf life of the session key

Authorization will commonly affect user’s experience on mobile

eBay authentication and authorization screens

Last.fm - Sample Flow http://www.lastfm.com/api/mobileauth Get an API key Request auth – auth.getMobileSession Inputs: username authToken (md5(username + md5(password))) api_key api_sig Returns session key Make API call with sk (session key) Note: Try to keep credentials in one area to facilitate later changes

http://www.lastfm.com/api/mobileauth

Get an API key

Request auth – auth.getMobileSession

Inputs:

username

authToken (md5(username + md5(password)))

api_key

api_sig

Returns session key

Make API call with sk (session key)

Note: Try to keep credentials in one area to facilitate later changes

Last.fm auth.getmobilesession auth.getmobilesession Result with new session key http://ws.audioscrobbler.com/2.0/?method=auth.getmobilesession&api_key=be8c2…&username=chudsonfm&authToken=4eb71…&api_sig=e93fb… <lfm status=&quot;ok”> <session> <name>chudsonfm</name> <key>142c0085138a4b72d1782215b1da770a</key> <subscriber>0</subscriber> </session> </lfm> username (Required) : The last.fm username. authToken (Required) : A 32-byte ASCII hexadecimal MD5 hash of the user's Last.fm credentials. api_key (Required) : A Last.fm API key. api_sig (Required) : A Last.fm method signature.

auth.getmobilesession

Result with new session key

Last.fm – Call Execution Now we can make the call with our sk (session key) – user.getinfo Response http://ws.audioscrobbler.com/2.0/?method=user.getinfo&api_key=be8c2…&sk=142c0…&api_sig=ccbd1…cc9dd <lfm status=&quot;ok”> <user> <id>17317105</id> <name>chudsonfm</name> <realname/> <url>http://www.last.fm/user/chudsonfm</url> <image/> <lang>en</lang> <country/> <age/> <gender>n</gender> <subscriber>0</subscriber> <playcount>0</playcount> <playlists>0</playlists> </user> </lfm> sk (Required) : The session key returned by auth.getMobileSession service. api_key (Required) : Your 32-character API key. api_sig (Required) : Your API method signature

Now we can make the call with our sk (session key) – user.getinfo

Response

<lfm status=&quot;ok”>

<user>

<id>17317105</id>

<name>chudsonfm</name>

<realname/>

<url>http://www.last.fm/user/chudsonfm</url>

<image/>

<lang>en</lang>

<country/>

<age/>

<gender>n</gender>

<subscriber>0</subscriber>

<playcount>0</playcount>

<playlists>0</playlists>

</user>

</lfm>

Speed Traps Could be on wifi, 3G or 1G based signal and bandwidth Segment functionality / calls to prevent issues Standard Apple test for approval is bandwidth usage when on cell network Polling Create subset of calls and functionality when on slower bandwidth Use notifications if possible Modify polling interval based on need Recovery from intermittent or lost connections Test the use of APIs via public vendor API test tools on various bandwidths

Could be on wifi, 3G or 1G based signal and bandwidth

Segment functionality / calls to prevent issues

Standard Apple test for approval is bandwidth usage when on cell network

Polling

Create subset of calls and functionality when on slower bandwidth

Use notifications if possible

Modify polling interval based on need

Recovery from intermittent or lost connections

Test the use of APIs via public vendor API test tools on various bandwidths

Large Data Set Handling Decreasing returns for XML with large data sets and repetitive XML sections Even more the case with mobile devices given: Communication challenges Memory and processing limitations Minimize large data sets: Use Just-in-Time (JIT) methodology Ask for only those elements that you require Ask for only those items that you require Store what you can locally instead of requesting the same data

Decreasing returns for XML with large data sets and repetitive XML sections

Even more the case with mobile devices given:

Communication challenges

Memory and processing limitations

Minimize large data sets:

Use Just-in-Time (JIT) methodology

Ask for only those elements that you require

Ask for only those items that you require

Store what you can locally instead of requesting the same data

Response Processing Leverage Stream Parsing for mobile environments Provides low memory overhead But, requires more heavy lifting iPhone SDK based on stream parsing – NSXMLParser class Android has Xstream as the streaming parser

Leverage Stream Parsing for mobile environments

Provides low memory overhead

But, requires more heavy lifting

iPhone SDK based on stream parsing – NSXMLParser class

Android has Xstream as the streaming parser

iPhone Parsing NSXMLParser delegate methods 5 key methods Parser:didStartElement: Parser:foundCharacters: Parser:didEndElement: Parser:didEndDocument: Parser:parseErrorOccurred: Method Purpose parser:didStartElement: Start of element tag occurred with attributes parser:foundCharacters: Element data located which may be all or some parser:didEndElement: End of element tag located parser:didEndDocument: End of XML document occurred parser:parseErrorOccurred: Unrecoverable parsing error occurred

NSXMLParser delegate methods

5 key methods

Parser:didStartElement:

Parser:foundCharacters:

Parser:didEndElement:

Parser:didEndDocument:

Parser:parseErrorOccurred:

Typical didStartElement, foundCharacters, and didEndElement

Typical didStartElement, foundCharacters, and didEndElement

Parsing In long parsing streams provide feedback when available Example: Control4 My House Home control from your iTouch/iPhone During initial project load describes the sections being processed Stores those items that will not change including house devices But when providing current data, requests from server current state Blends stored information with real time status information to provide display

In long parsing streams provide feedback when available

Example: Control4 My House

Home control from your iTouch/iPhone

During initial project load describes the sections being processed

Stores those items that will not change including house devices

But when providing current data, requests from server current state

Blends stored information with real time status information to provide display

Filtering Types of Request Filtering Selection criteria for narrowing data set returned In general determine how efficient are the calls – reduce data waste How much extra data, streaming, parsing is happening to access the data you use? How much data is being dropped on the floor? <?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?> <FindItemsAdvancedRequest xmlns=&quot;urn:ebay:apis:eBLBaseComponents&quot;> <!-- Standard Input Fields --> <MessageID> string </MessageID> <!-- Call-specific Input Fields --> <BidCountMax> int </BidCountMax> <BidCountMin> int </BidCountMin> <CategoryHistogramMaxChildren> int </CategoryHistogramMaxChildren> <CategoryHistogramMaxParents> int </CategoryHistogramMaxParents> <CategoryID> string </CategoryID> <CharityID> int </CharityID> <Condition> ItemConditionCodeType </Condition> <Currency> CurrencyCodeType </Currency> … <ShippingLocation> CountryCodeType </ShippingLocation> <ShippingPostalCode> string </ShippingPostalCode> <SortOrder> SortOrderCodeType </SortOrder> <StoreName> string </StoreName> <StoreSearch> StoreSearchCodeType </StoreSearch> </FindItemsAdvancedRequest>

Types of Request Filtering

Selection criteria for narrowing data set returned

In general determine how efficient are the calls – reduce data waste

How much extra data, streaming, parsing is happening to access the data you use? How much data is being dropped on the floor?

Paging Provide paging functionality to users Not all calls have paging available Inform user of current location in set, easy access to next and previous if applicable Cache key information depending on The user flow (detail page from results page) The aging of the information (when does it expire)

Provide paging functionality to users

Not all calls have paging available

Inform user of current location in set, easy access to next and previous if applicable

Cache key information depending on

The user flow (detail page from results page)

The aging of the information (when does it expire)

Page Requests Specify the page that you are requesting <PageNumber> How many items are on a page Set the number of entries per page returned <MaxEntries> http://developer.ebay.com/products/overview/

Specify the page that you are requesting

<PageNumber>

How many items are on a page

Set the number of entries per page returned

<MaxEntries>

http://developer.ebay.com/products/overview/

List Paging Example Must cache of course the current page Remember though the total pages can change dynamically so check total pages after subsequent requests <TotalItems> int </TotalItems> <TotalPages> int </TotalPages>

Must cache of course the current page

Remember though the total pages can change dynamically so check total pages after subsequent requests

<TotalItems> int </TotalItems>

<TotalPages> int </TotalPages>

Display Caching of information for future retrieval and navigation Be careful of data aging depending on what is being shown Execute web service parsing and display in background when possible to prevent UI blocking example – eBay item description and images When parsing large data blocks, progress indications can be provided from element name (sections of the XML) Synchronous versus Asynchronous calls Use JIT and Information on Demand to maximize small layouts and minimize web service calls

Caching of information for future retrieval and navigation

Be careful of data aging depending on what is being shown

Execute web service parsing and display in background when possible to prevent UI blocking

example – eBay item description and images

When parsing large data blocks, progress indications can be provided from element name (sections of the XML)

Synchronous versus Asynchronous calls

Use JIT and Information on Demand to maximize small layouts and minimize web service calls

Caching Options To minimize API round trips leverage caching and storage Cache in memory Cache in onboard light weight database or file Off device persistence

To minimize API round trips leverage caching and storage

Cache in memory

Cache in onboard light weight database or file

Off device persistence

Expanding Your Options

Blending Multiple Results What if you have to: Blend results from multiple calls in the same API Blend results from multiple API sources This could take the form of: Extra item information on top of item details call Note this is simplified so does not show authentication and authorization checking, encoding, or the like. API Service Mobile View Trigger Request 1 Mobile App Request 2 Result 1 Fulfill Fulfill Result 2 Blend Outlet

What if you have to:

Blend results from multiple calls in the same API

Blend results from multiple API sources

This could take the form of:

Extra item information on top of item details call

Note this is simplified so does not show authentication and authorization checking, encoding, or the like.

Blending Multiple Services Multiple Services Dependent on multiple services Example: Weather request from additional WS based on zip code returned by latitude and longitude from a geoposition service API Service 1 Mobile View Trigger Request 1 Mobile App Request 2 Result 1 Fulfill Result 2 Blend Outlet API Service 2 Fulfill

Multiple Services

Dependent on multiple services

Example:

Weather request from additional WS based on zip code returned by latitude and longitude from a geoposition service

Proxy Server Solution Provides layer of abstraction to consolidate or manage web services Proxy Server retrieves data on demand or on scheduled basis Allows for single consolidated call from the mobile device Processes data and returns subset or stores data for retrieval later Can also reduce API calls by caching similar results across user Allows for extending business logic layer

Provides layer of abstraction to consolidate or manage web services

Proxy Server retrieves data on demand or on scheduled basis

Allows for single consolidated call from the mobile device

Processes data and returns subset or stores data for retrieval later

Can also reduce API calls by caching similar results across user

Allows for extending business logic layer

Proxy Flow Provides for single round trip for mobile application Provides easy maintenance of web services business logic layer Easily modify Business logic API credentials Proxy Server Mobile View Trigger Request Mobile App Request Result Result Outlet API Services Fulfill Fulfill Request Result

Provides for single round trip for mobile application

Provides easy maintenance of web services business logic layer

Easily modify

Business logic

API credentials

Proxy Landscape Web Service Web Service Proxy Server

Designing your WS/API Destiny

Creating Web Services Try to serve lowest common device Balance flexibility with overhead in calls: Filtering of criteria and paging of data functionality Split out high traffic calls versus critical requests Easy but secure authentication and authorization for both the consumer and application Benefit for mobile consumers, application programmers and web service providers Decreased round trips Increased efficiency of calls and applications Better use of call volume restrictions User experience improves

Try to serve lowest common device

Balance flexibility with overhead in calls:

Filtering of criteria and paging of data functionality

Split out high traffic calls versus critical requests

Easy but secure authentication and authorization for both the consumer and application

Benefit for mobile consumers, application programmers and web service providers

Decreased round trips

Increased efficiency of calls and applications

Better use of call volume restrictions

User experience improves

Wrap Up Provide user preference options that translate into better tuned API calls - number of items to show on a page as an example This will translate to data efficiency, fewer callers and greater use of data retrieved Minimize the number of trips, combine in single calls when possible Leverage caching and data storage when allowed Balance data granularity with length of returns for parsing Final thought to ponder The mobile device that is typically serving as a web service client could in the now or future be a web service provider…

Provide user preference options that translate into better tuned API calls

- number of items to show on a page as an example

This will translate to data efficiency, fewer callers and greater use of data retrieved

Minimize the number of trips, combine in single calls when possible

Leverage caching and data storage when allowed

Balance data granularity with length of returns for parsing

Final thought to ponder

The mobile device that is typically serving as a web service client could in the now or future be a web service provider…

Thank you Chuck Hudson [email_address] aduci

Chuck Hudson

[email_address]

Appendix A - Resources API / Web Service Online Catalogues Programmable Web www.programmableweb.com/api Web Service List www.webservicelist.com

API / Web Service Online Catalogues

Programmable Web

www.programmableweb.com/api

Web Service List

www.webservicelist.com

Appendix B – Code Resources iPhone URL Connections http://developer.apple.com/documentation/Cocoa/Conceptual/URLLoadingSystem/Tasks/UsingNSURLConnection.html iPhone Rest Wrapper ihttp://github.com/akosma/iphonerestwrapper/tree/master iPhone XML Parsing – Cocoa Stream Parser http://developer.apple.com/documentation/Cocoa/Conceptual/XMLParsing/XMLParsing.html SOAP with iPhone examples http://icodeblog.com/2008/11/03/iphone-programming-tutorial-intro-to-soap-web-services/ http://developer.omniture.com/node/321 Façade Server landscape – O’REILLY – Windows platform http://www.ondotnet.com/pub/a/dotnet/2004/02/23/mobilewebserviceapps.html Connecting to Web Services – Sean Sullivan http://www.slideshare.net/sullis/connecting-to-web-services-on-android W3C – Mobile Web Application Best Practices http://www.w3.org/TR/2008/WD-mwabp-20081222/

iPhone URL Connections

http://developer.apple.com/documentation/Cocoa/Conceptual/URLLoadingSystem/Tasks/UsingNSURLConnection.html

iPhone Rest Wrapper

ihttp://github.com/akosma/iphonerestwrapper/tree/master

iPhone XML Parsing – Cocoa Stream Parser

http://developer.apple.com/documentation/Cocoa/Conceptual/XMLParsing/XMLParsing.html

SOAP with iPhone examples

http://icodeblog.com/2008/11/03/iphone-programming-tutorial-intro-to-soap-web-services/

http://developer.omniture.com/node/321

Façade Server landscape – O’REILLY – Windows platform

http://www.ondotnet.com/pub/a/dotnet/2004/02/23/mobilewebserviceapps.html

Connecting to Web Services – Sean Sullivan

http://www.slideshare.net/sullis/connecting-to-web-services-on-android

W3C – Mobile Web Application Best Practices

http://www.w3.org/TR/2008/WD-mwabp-20081222/

Appendix C - Efficient XML Interchange (EXI) W3C Working Draft 19 September 2008 - http://www.w3.org/TR/2008/WD-exi-20080919/ “ EXI is a very compact representation for the Extensible Markup Language (XML) Information Set that is intended to simultaneously optimize performance and the utilization of computational resources.” Components of XML Complexity 1. size 2. total number of elements 3. number of unique elements 4. tree-height (nesting) 5. number of unique external references 6. total number of attributes 7. number of unique attributes 8. regularity, or &quot;self-similarity&quot;

W3C Working Draft 19 September 2008 - http://www.w3.org/TR/2008/WD-exi-20080919/

“ EXI is a very compact representation for the Extensible Markup Language (XML) Information Set that is intended to simultaneously optimize performance and the utilization of computational resources.”

Components of XML Complexity

1. size

2. total number of elements

3. number of unique elements

4. tree-height (nesting)

5. number of unique external references

6. total number of attributes

7. number of unique attributes

8. regularity, or &quot;self-similarity&quot;

Facebook Notice the session key expiration http://wiki.developers.facebook.com/index.php/Authorizing_Applications Oauth MPOAuthMobile – compiled in project library - Karl Adam http://code.google.com/p/mpoauthconnection/wiki/MPOAuthMobile Appendix D - Other Auth Examples

Facebook

Notice the session key expiration

http://wiki.developers.facebook.com/index.php/Authorizing_Applications

Oauth

MPOAuthMobile – compiled in project library - Karl Adam

http://code.google.com/p/mpoauthconnection/wiki/MPOAuthMobile