Device APIs at TakeOff Conference


Published on

Most APIs being worked on at the W3C DAP WG

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Devices in this context include desktop computers, laptop computers, cellular phones, TVs, camera, etc.richer and more context-aware Web applications.
  • Joint work with the WebRTC Working Group. DAP not concerned with other APIs part of WebRTC such as PeerConnection or DataChannel API
  • window.requestAnimationFrame
  • window.requestAnimationFrame
  • My experience: the only mobile browser that supports this is FF for Android. Even though it “landed” on WebKit Chrome for Android doesn’t include it.
  • -If the user declines, the error callback is invoked…
  • -Browse buttons in the demos could be disabled if the service gets disconnect When requesting a service: onserviceavailable;onserviceunavailable;When comunicating with a onserviceonline;onserviceoffline;
  • Started as Inter-app communication mechanism; now “everything” can be done with Web Intents Has been strongly pushed by Google. They currently face a few issues with UI and UXNative support therefore removed from Chrome for now (o_o)Mozilla’s counterproposal: Web Activities; will feature in Firefox OS R1
  • This approach moved away from the original idea of providing read/write access to the device’s gallery.
  • Also: User contacts can be store in a variety of sources, such as cloud not just in the deviceThe scope for Web Intents have changed from a simple inter-app communication mechanism to doing a bit of everything, including seamless access to local and remote sensors, as well as local network services
  • What happens when you measure a bandwidth against the host of the web app but an image will actually be retrieved from a CDN…
  • What happens when you measure a bandwidth against the host of the web app but an image will actually be retrieved from a CDN…Metered is only really known by the provider and “some users”. Metered is also not a great name for the attribute…
  • What happens when you measure a bandwidth against the host of the web app but an image will actually be retrieved from a CDN…Metered is only really known by the provider and “some users”. Metered is also not a great name for the attribute…
  • Stronger integration with the host platform Typically platform-certified appsDAP: Web apps that run in the browser runtime, i.e. can cope with users visiting untrusted sites-> cautious approach narrows the scope of what this Apps/APIs can doEx: contacs picker vs. contacts manager
  • Device APIs at TakeOff Conference

    1. 1. W3C Device APIsWhat are they, how to use them and where are they goingDiana ChengTakeOff ConferenceLille, January 2013
    2. 2. Diana Cheng – Vodafone Group, R&D daianacheng W3C DAP and SysApps member, formerly also GeolocationJanuary 2013 Take Off Conf – Device APIs 2
    3. 3. Contents• Introduction: DAP Working Group and scope• DAP specs and APIs (with code and demos) • HTML Media Capture • getUserMedia • Battery Status • Vibration • Network Service Discovery • Pick Media Intent • Web Intents • Other Intents-based APIs • Network Information• Conclusions: What now and future work• Questions?January 2013 Take Off Conf – Device APIs 3
    4. 4. W3C DAP Working Group• Device APIs working group• Scope of the group and its specifications has changed over time: recharted in 2011• Edits a number of specs with different degrees of maturity• Focuses on APIs that run in the browser security context and are therefore sandboxed• Works also on „Task Forces“ together with the Web Intents and WebRTC WGsJanuary 2013 Take Off Conf – Device APIs 4
    5. 5. So what are Device APIs?• Browser-based Javascript APIs• Provide access to the device hardware and services• Allows us to build rich Web applications  Making the Web THE platform!• A lot of people seem to consider them part of HTML5 Such functionality was previously only available to native appsPhoneGap, now Apache Cordova, was started to fill this void in mobile devicesJanuary 2013 Take Off Conf – Device APIs 5
    6. 6. Device APIs Specification Status Desktop Mobile Implementations Implementations HTML Media Capture LC Draft ✔✔✔ ✔✔✔ getUserMedia (WebRTC) PW Draft ✔✔✔ Battery API CR ✔ ✔ Vibration API CR ✔ Network Service Discovery PW Draft ✔ Contacts: Pick Contacts Intent PW Draft ? “Gallery”: Pick Media Intent PW Draft ? Proximity API LC Draft ✔ Ambient Lights API LC Draft ✔ Network Information Pretty ? bad?  PW DraftJanuary 2013 Take Off Conf – Device APIs 6
    7. 7. HTML Media Capture (1/2) <input type="file" accept="image/*;capture=camera” multiple>• extends the HTML Input Element with a capture attribute• capture is a hint for the source of the media.• Possible values: camera, camcorder, microphone, filesystem• Media type could be: image, video, audio• multiple won‟t work on mobile devices due to the camera interfaceJanuary 2013 Take Off Conf – Device APIs 7
    8. 8. HTML Media Capture (2/2)Support: Desktop Safari, Firefox, Chrome, Opera latest versions Mobile Chrome for Android, Firefox for Android, Android 4.0 stock browser, Blackberry browserMy own experience:• Desktop versions tested on Mac• Opera Mobile seems to provide no real access to camera, only to device Gallery• Demo won‟t work in Opera Desktop (FileReader problem)* * Opera for Mac 12.1 has a bug in the FileReader‟s getDataAsURL and the demo for this API won‟t work there.January 2013 Take Off Conf – Device APIs 8
    9. 9. getUserMedia (WebRTC) (1/3) navigator.getUserMedia (constraints, successCallback, errorCallback);• Access to the device‟s camera and microphone on the Web today!• Farewell to plugins! • Call will prompt the user for permission and invoke the errorCallback if denied• Can request a LocalMediaStream with audio, video or both constraints = {video: true, audio: true}• Still prefixed in several implementationsJanuary 2013 Take Off Conf – Device APIs 9
    10. 10. getUserMedia (WebRTC) (2/3)Support: 2013 Take Off Conf – Device APIs 10
    11. 11. getUserMedia (WebRTC) (3/3)My own experience:• Support in Opera Mobile 12.10?• Making it work cross-browser requires considering the prefixes and careful testing• spec keeps on changing: e.g. MediaStreamRecorder out? currently no standard way to record a device‟s live stream: recorder = localMediaStream.recorder(); localMediaStream.stop(); recorder.getRecordedData(function(blob) { //do smth…}); (see recorder demo for a workaround)January 2013 Take Off Conf – Device APIs 11
    12. 12. Battery Status API (1/2) navigator.battery• Programmatic access to the device‟s battery level and events!• Avoid tasks in our Web apps that are not suitable for a low battery level: Power-efficient Web apps!• Example use cases: • avoid network operations like syncing with the server, or do them with a lower frequency • Avoid power-consuming, processor-intensive animationsJanuary 2013 Take Off Conf – Device APIs 12
    13. 13. Battery Status API (2/2)• Attributes: level (]0 - 1.0]), charging, chargingtime*, dischargingtime* chargingtime = Infinity (battery is discharging) chargingtime = 0 (battery is full) dischargingtime = Infinity (battery is charging)• Events: levelchange, chargingchange, chargingtimechange, dischargingtimechange• How often the update takes place (events fire) is left to the implementation• Support: Firefox and Firefox for Android• My own experience: • Couldn‟t be simpler to use! But it‟s not too easy to test  * in secondsJanuary 2013 Take Off Conf – Device APIs 13
    14. 14. Vibration API (1/1) navigator.vibrate(time); navigator.vibrate(pattern);• Programmatically trigger feedback in the form of vibration• Particularly nice for mobile games; not aimed for generic notification• time is in milliseconds• pattern is an array of consecutive vibration durations, e.g. [1000, 500, 2000]• navigator.vibrate(0) or navigator.vibrate([ ]) stop the vibrationJanuary 2013 Take Off Conf – Device APIs 14
    15. 15. Network Service Discovery API (1/2) navigator.getNetworkServices(type, success, error);• Asynchronous discovery of local HTTP-based services in Web Apps via a single API call!• Common discovery protocols: SSDP (UPnP), DNS-SD (ZeroConf)• Use cases: • Remotely controlling local network appliances from Web Apps, e.g. Web-based TV remote control • Browse content in a local network and select the appropriate renderer among those found • type: • „upnp:urn:schemas-upnp-org:service:ContentDirectory:1‟, urn:schemas-upnp-org:service:RenderingControl:1 • zeroconf:_boxee-jsonrpc._tcpJanuary 2013 Take Off Conf – Device APIs 15
    16. 16. Network Service Discovery API (2/2)• After discovery, the URL of a requested service is whitelisted by the browser for the given context; this allows to do cross-site requests via AJAX, Web Sockets, etc.• Events for changes in service reachability: onserviceonline, onserviceoffline• Only implementation at the moment is an experimental build by Opera supporting only UPnP and with no UI for user consent yet opera/ (two very cool demos there too)• My own experience: • The Opera build works as expected • Familiarity with the networking protocols (and XML/SOAP for UPnP) requiredJanuary 2013 Take Off Conf – Device APIs 16
    17. 17. Pick Media Intent“The Pick Media Intent defines a Web Intent …” Web Intents?January 2013 Take Off Conf – Device APIs 17
    18. 18. Android IntentsJanuary 2013 Take Off Conf – Device APIs 18 18
    19. 19. Web Intents The Problem in the Web today Sharing 1001 APIs to deal with…January 2013 Take Off Conf – Device APIs 19 19
    20. 20. Web Intents fixes that• Applications register their ability to handle an Intent (action). They express a certain capability <intent action="" type="image/*" />• A given app can delegate functionality it doesn‟t support to any of those apps that have registered it var intent = new Intent(); intent.action =; intent.type = image/*; = [imgData]; window.navigator.startActivity(intent);January 2013 Take Off Conf – Device APIs 20 20
    21. 21. Web Intents fixes that• Users select which app to use• The original site will then send the payload over to the handling app, and will get the result back in a callback window.intent.postResult([ /*... some data ...*/ ]);January 2013 Take Off Conf – Device APIs 21 21
    22. 22. Pick Media Intent (1/2) var intent = new Intent({ action: "", type: "", extras: { search: "olympic", filters: ["title", "description", "author”], limit: 100 }});• Defines a new Web Intent in order to access the user‟s media on the device or in the cloud• Intent action: pick• Intent type: media• Extras: hints for the Intents Service• Why: users store media on a variety of cloud services and Web apps should not require prior knowledge of them. Local or remote, the sources (services) just need to be registered with the browserJanuary 2013 Take Off Conf – Device APIs 22
    23. 23. Pick Media Intent (2/2)• The spec also defines a Media dictionary with the relevant Metadata• Other DAP specs which are based on Web Intents: • Pick Contacts Intent • Web Intents Addendum for Local Services: Local Services (UPnP and mDNS) register as Service Intents, i.e. their ability to perform e.g. media playbackJanuary 2013 Take Off Conf – Device APIs 23
    24. 24. Sensor APIs• Proximity Events API• Ambient Lights API• More coming soon: Temperature? Atmospheric Pressure?January 2013 Take Off Conf – Device APIs 24
    25. 25. And few issues with ….January 2013 Take Off Conf – Device APIs 25
    26. 26. Network Information API (1/4)January 2013 Take Off Conf – Device APIs 26
    27. 27. Network Information API (2/4) Premise: bandwidth on mobile is still a constraint• Provide an API that gives information about the speed of the connection the device is currently on• Use cases: • help us serve the right images to the device according to the available bandwidth (as it is done according to screen size and resolution) • avoid initiating network-hungry requests if they are unlikely to be completed• Interesting research by PPK: 2013 Take Off Conf – Device APIs 27
    28. 28. Network Information API (3/4)What information to provide:1. Connection type vs. bandwidth (actual speed) Connection type Bandwidth • can be obtained from the • tricky to measure underlying OS • power and data consuming • can be a poor indicator. e.g : a • highly dependent on both the crowded conference Wifi might client and the server offer slower connectivity than 3G • can change quickly as the user moves in/out/inside buildings, on a fast train, etc. • a single “sample” might not be representative so as to make a decisionJanuary 2013 Take Off Conf – Device APIs 28
    29. 29. Network Information API (4/4)2. roaming vs. metered roaming metered • user agent could obtain this from • in other words: is the user on a the underlying OS limited data plan? e.g: 1GB/month • definite indicator that networking • useful but this information is hard should be limited? the user might to obtain not care about costs: e.g. I have a • the user could be prompted but European flat rate from my they can provide wrong employer, other people might information have a lot of money• Should developers be making this kind of decisions?• Should we rely on the browser to do batched requests (if indicated by the developer) in order to preserve a low-power radio state? …..SUGGESTIONS?January 2013 Take Off Conf – Device APIs 29
    30. 30. And whatever happened to…• Messaging• Calendar• System Information• Gallery• Telephony …?January 2013 Take Off Conf – Device APIs 30
    31. 31. SysApps Working Group• A new scope: a different runtime environment and security model and APIs to build system-level Web Apps• Standardize and converge specifications inspired by projects such as Mozilla‟s Firefox OS and TizenJanuary 2013 Take Off Conf – Device APIs 31
    32. 32. What can you do?Provide your feedback!• DAP Mailing list:• SysApps Mailing list: join? January 2013 Take Off Conf – Device APIs 32
    33. 33. Questions?January 2013 Take Off Conf – Device APIs 33