Mobile Device APIs

(Mobile) Device APIs James Pearce @jamespearce http://tripleodeon.com

What are Device APIs? “Client-side APIs that enable the development of Web Applications that interact with device hardware” ...and other capabilities outside of the browser’s traditional remit

Why are they important?• They’re not... ...if you’re happy with a web of documents• They are... ... if you want the the web to be competitive with native app platforms

HistoryCurrent Work Use Cases

WTAI Make Call<a href="wtai://wp/mc;16505551234"> Call us</a><a href="call.wmls#makeCall()"> Call us</a>// call.wmlsextern function makeCall() { WTAPublic.makeCall("16505551234");} http://www.wapforum.org/what/technical/wtai-30-apr-98.pdf

WTAI Call Control<a href="calls.wmls#setupCall()">Call us!</a><a href="calls.wmls#acceptCall()">Brring!</a><a href="wtai://cc/sd;1*2*3456">Enter code</a>// calls.wmlsextern function setupCall() { WTACallCont.setup("16505551234", 1);}extern function acceptCall() { WTACallCont.accept("1", 1);}

WTAI Messaging// sms.wmlsextern function sendMsg() { WTANetText.send("16505551234", "WML rocks");}extern function readMsg(i) { var sms = WTANetText.read(i); return WTANetText.getFieldValue( sms, "body" );}

WTAI Contacts// contacts.wmlsextern function addFriend(i, num, name) { WTAPhoneBook.write(i, num, name);}// return structs of contact detailsextern function getFriendById(i) { return WTAPhoneBook.read("i", i);}extern function getFriendByName(name) { return WTAPhoneBook.read("t", name);}

I may be laboring the point But that was 1998... ... and you still can’t do any of this with a contemporary mobile browser

BONDI“enables web based content to access nativedevice capability, intermediated through arobust, but ﬂexible security framework” http://www.omtp.org/1.11

Messagingvar sms = bondi.messaging.createSMS({ to: ["16505551234"], body: "Home soon"});var mms = bondi.messaging.createMMS({ to: ["16505551234"], subject: "I will be...", body: "...home soon"});

Messagingbondi.messaging.sendSMS( function () {}, // success function () {}, // error sms, true);bondi.messaging.sendMMS( function () {}, // success function () {}, // error mms, true);

Messagingbondi.messaging.subscribeToSMS( function () {}, // success function () {}, // error function () { // listener alert("They replied!"); }, { from: "16505551234" // filter }, true // exclusive);

Contactsbondi.pim.contact.getAddressBooks( function(books) { // success var contact = books[0].createContact({ name: James Pearce, emails: [{email:j@tripleodeon.com}], phoneNumbers: [{number:16505551234}] }); }, function () {} // error);

Telephonybondi.telephonyLog.findLogEntries( function () {}, // success function () {}, // error, [ bondi.telephonyLog.MISSED_CALLS_FOLDER, bondi.telephonyLog.RECEIVED_CALLS_FOLDER ], { phoneNumber: "16505551234" }); * Yes, it’s true: you can’t actually make or receive calls

App Launcherbondi.applauncher.launchApplication( function () {}, // success function () {}, // error "tel:16505551234");// mailto:...// sms:...// file:...

On one hand...• Security-conscious architecture• Comprehensive API support• Consistent API patterns• Conformance speciﬁcations• Reasonable test coverage http://www.omtp.org/1.11/apis

On the other...

bondi.*deviceapis.*

WAC 2.0• Accelerometer • File system• Calendar • Geolocation• Camera • Messaging• Contacts • Orientation• Core • Tasks• Device interaction • Viewport• Device status • Webview http://specs.wacapps.net/

Which is all awesome Except...

Device APIs Working Group “to create client-side APIs that enable the development of Web Applications and Web Widgets that interact with devices services such as Calendar, Contacts, Camera, etc” Joint efforts with WebRTC Working Group, WebApps Working Group http://www.w3.org/2009/dap/

W3C Device APIs• Media Capture(HTML) • Vibration• Media Capture(gUM) • Calendar• Contact • Permissions• Messaging • Menu• Battery Status • Gallery• Network Information • System info• Sensor • Media Stream

Media Capture (HTML)<input type="file" accept="image/*" /><!-‐-‐ The accept attribute is a hint. Specification suggests that it can be used to offer alternative picker UIs.-‐-‐> Public working draft, http://www.w3.org/TR/2011/WD-html-media-capture-20110414

Brad Lassey, September 2008

Media Capture (HTML)<input type="file" accept="image/*" capture="camera"/><!-‐-‐ Again, a hint. Valid options are: camera, camcorder, microphone, filesystem-‐-‐>

Media Capture (HTML)<input type="file" accept="image/*" capture="camera" id="photo"/>var photo = document.getElementById(photo);photo.files[0].getFormatData( function (data) {}, // success function () {} // error);// file is a MediaFile, extending File// data is a MediaFileData

MediaFileDatainterface MediaFileData { attribute DOMString codecs; attribute unsigned long bitrate; attribute unsigned long height; attribute unsigned long width; attribute float duration;}; ...seems to be under discussion

Media capture (gUM)navigator.getUserMedia( { audio: true, video: false }, function (stream) {}, // success function () {} // error);// stream is a LocalMediaStream Editors draft, http://dev.w3.org/2011/webrtc/editor/getusermedia.html

LocalMediaStreamvar recorder = stream.record();// recorder is a MediaStreamRecorderrecorder.getRecordedData( function (file) {} // success);// file is a FileURL.createObjectURL(stream);// a Blob URI for, say <video src=... />

Messagingnavigator.sendMessage( "sms:16505551234?" + "body=JavaScript%20says%20hi", [], // attachments as File array function () {}, // success function () {} // error);// also mms: and mailto:// oh... were you looking for receipt? Editors draft, http://dev.w3.org/2009/dap/messaging/

Sensorsnavigator.findSensors().onsuccess = function() { this.result.forEach(function (sensor) { // ... });};var sensor = new SensorConnection(Temperature);sensor.addEventListener(sensordata, function(e) { if(e.data > 20.0) { window.console.log(Hawt); }});sensor.startWatch({ interval: 2000 }); Editors draft, http://dev.w3.org/2009/dap/system-info/Sensors.html

Sensors• Temperature, ºC• AmbientLight, Lux• AmbientNoise, dbA• MagneticField, uTesla• Proximity, cm• AtmPressure, kP• RelHumidity, %

Contactsnavigator.contacts.find( [ // fields name, emails ], function (contacts) {}, // success function () {}, // error { // find options filter: James, multiple: true }); Editors draft, http://w3c-test.org/dap/contacts/

Contactinterface Contact { readonly attribute DOMString id; attribute DOMString? displayName; attribute ContactName? name; attribute DOMString? nickname; attribute ContactField[]? phoneNumbers; attribute ContactField[]? emails; attribute ContactAddress[]? addresses; attribute ContactField[]? ims; attribute ContactOrganization[]? organizations; attribute Date? birthday; attribute DOMString? note; attribute ContactField[]? photos; attribute DOMString[]? categories; attribute ContactField[]? urls;};

Cool, but... “The ability to add and update contact information is not a function of the API provided in this speciﬁcation.”

Holy Contacts Editing WTFvar vcard = BEGIN:VCARDrn + VERSION:3.0rn + UID: + contact.id + rn + N:Pearce;Jamesrn + FN:James Pearcern + END:VCARD;document.getElementById(vcard).href = data:text/x-‐vcard;charset=utf-‐8, + encodeURIComponent(vcard);

WebAPI “We are aiming at providing all the necessary APIs to build a basic HTML5 phone experience within the next 3-6 months” https://wiki.mozilla.org/WebAPI

WebAPI• Telephony • Embedded browser• Messaging • Battery• Contacts • Device Status• Camera • Settings• Network • Mouse Lock• USB • Vibrator

Hopefully W3C++• “Our proposal might or might not look like it for the moment”

Telephony!var tcall = navigator.telephony.dial("16505551234");tcall.readyState; // dialing, ringing, busy, connecting, // connected, disconnecting, disconnected, // incomingtcall.onconnected = function () {}; // onbusy, ondisconnected etctcall2.answer(); http://www.youtube.com/watch?v=RuIQskGD3u0#t=19s

Contacts “We need read/write”

USBvar usb = new MozUSBManager();usb.addEventListener("attachdevice", function () {});usb.addEventListener("detachdevice", function () {});usb.claimDevice({ deviceClass: ..., deviceSubClass: ...});

We deserve Device APIs...• As forward-looking as WTAI• As well-structured as BONDI• As comprehensive as WAC• As respectable as W3C DAP• As agile as WebAPI

And we’ve got...

PhoneGap• Accelerometer • Events• Camera • File• Capture • Geolocation• Compass • Media• Connection • Notiﬁcation• Contacts • Storage• Device

Use Cases

Useless Demos https://github.com/jamesgpearce/compios5

Fixing layout bugs, FMLwindow.addEventListener("deviceorientation", function (e) { orientation = Math.abs(w.orientation); rotation = Math.abs(e.gamma); if (rotation > 8 && orientation === 0) { if (enabled) { disableZoom(); } else { restoreZoom(); } } }, false); http://vimeo.com/adactio/iosbug https://github.com/scottjehl/iOS-Orientationchange-Fix

Camera• Proﬁle pictures• Review sites• Photo sharing• Recognition • Bar-codes • Text • Faces

Filters!$(img).vintage({ vignette: { black: 0.8, white: 0.2 }, noise: 20, screen: { red: 12, ... }, desaturate: 0.05}); https://github.com/rendro/vintageJS

Augmented reality? http://fhtr.org/JSARToolKit/

Contacts & calendaring• Add business details to address book• Web-based messaging• Facial recognition again...• Presence• Context-sensitivity of content

Messaging• Sharing• Notiﬁcations• Ofﬂine transport fallback• Frictionless web sign-on

Expect more

I still want... • Audio processing • Bluetooth • RFID & NFC • Both cameras • Memory & power • Native recognition

window.addEventListener( appointmentdue, function(appt) { if (device.near(WORK)) { siri.remind( contacts.get(Peter), calendar.getDetails(appt) ); } }, false); PS: does not work

James Pearce @jamespearcehttp://tripleodeon.com

http://sd.csdn.net	7716
http://www.oschina.net	1953
http://css.dzone.com	733
http://functionsource.com	571
http://zubairov.posterous.com	182
http://a0.twimg.com	118
http://www.linuxeden.com	81
http://us-w1.rockmelt.com	35
http://www.newsblur.com	28
http://soa.dzone.com	16
http://architects.dzone.com	15
http://m.oschina.net	14
http://bitly.com	13
http://mobile.dzone.com	12
http://tweetedtimes.com	12
http://www.twylah.com	11
http://bitly.net	10
http://articles.csdn.net	8
http://127.0.0.1	2
http://localhost	2
http://haiman.org	1
http://hi.baidu.com	1
http://www.dzone.com	1
http://cmccapp7.hq.cmcc	1
http://www.shixue123.com	1
http://cnc.qzs.qq.com	1
http://bbs.ccidnet.com	1
http://bhyanyu.buaa.edu.cn	1
https://twimg0-a.akamaihd.net	1
http://sd.csdn.net.sixxs.org	1

Mobile Device APIs

by James Pearce on Jan 11, 2012

Accessibility

Categories

Tags

Upload Details

Usage Rights

30 Embeds 11,542

Statistics

Mobile Device APIs — Presentation Transcript