Mobile Device APIs

Mobile Device APIs

• 1.
  (Mobile) Device APIs James Pearce @jamespearce http://tripleodeon.com
• 2.
  What are DeviceAPIs? “Client-side APIs that enable the development of Web Applications that interact with device hardware” ...and other capabilities outside of the browser’s traditional remit
• 3.
  Why are theyimportant? • They’re not... ...if you’re happy with a web of documents • They are... ... if you want the the web to be competitive with native app platforms
• 4.
  History Current Work UseCases
• 7.
  WTAI Make Call <a  href="wtai://wp/mc;16505551234">    Call  us </a> <a  href="call.wmls#makeCall()">    Call  us </a> //  call.wmls extern  function  makeCall()  {    WTAPublic.makeCall("16505551234"); } http://www.wapforum.org/what/technical/wtai-30-apr-98.pdf
• 8.
  WTAI Call Control <a  href="calls.wmls#setupCall()">Call  us!</a> <a  href="calls.wmls#acceptCall()">Brring!</a> <a  href="wtai://cc/sd;1*2*3456">Enter  code</a> //  calls.wmls extern  function  setupCall()  {    WTACallCont.setup("16505551234",  1); } extern  function  acceptCall()  {    WTACallCont.accept("1",  1); }
• 9.
  WTAI Messaging //  sms.wmls extern  function  sendMsg()  {    WTANetText.send("16505551234",  "WML  rocks"); } extern  function  readMsg(i)  {    var  sms  =  WTANetText.read(i);    return  WTANetText.getFieldValue(        sms,  "body"    ); }
• 10.
  WTAI Contacts //  contacts.wmls extern  function  addFriend(i,  num,  name)  {    WTAPhoneBook.write(i,  num,  name); } //  return  structs  of  contact  details extern  function  getFriendById(i)  {    return  WTAPhoneBook.read("i",  i); } extern  function  getFriendByName(name)  {    return  WTAPhoneBook.read("t",  name); }
• 11.
  I may belaboring the point But that was 1998... ... and you still can’t do any of this with a contemporary mobile browser
• 13.
  BONDI “enables web basedcontent to access native device capability, intermediated through a robust, but flexible security framework” http://www.omtp.org/1.11
• 14.
  Messaging var  sms  =  bondi.messaging.createSMS({    to:  ["16505551234"],    body:  "Home  soon" }); var  mms  =  bondi.messaging.createMMS({    to:  ["16505551234"],    subject:  "I  will  be...",    body:  "...home  soon" });
• 15.
  Messaging bondi.messaging.sendSMS(    function  ()  {},  //  success    function  ()  {},  //  error    sms,  true ); bondi.messaging.sendMMS(    function  ()  {},  //  success    function  ()  {},  //  error    mms,  true );
• 16.
  Messaging bondi.messaging.subscribeToSMS(    function  ()  {},                      //  success    function  ()  {},                      //  error    function  ()  {                          //  listener        alert("They  replied!");    },  {                                      from:  "16505551234"          //  filter    },    true                                            //  exclusive );
• 17.
  Contacts bondi.pim.contact.getAddressBooks(    function(books)  {  //  success        var  contact  =  books[0].createContact({            name:  'James  Pearce',            emails:  [{email:'j@tripleodeon.com'}],            phoneNumbers:  [{number:'16505551234'}]        });    },      function  ()  {}        //  error );
• 18.
  Telephony bondi.telephonyLog.findLogEntries(    function  ()  {},  //  success    function  ()  {},  //  error,    [        bondi.telephonyLog.MISSED_CALLS_FOLDER,        bondi.telephonyLog.RECEIVED_CALLS_FOLDER    ],    {        phoneNumber:  "16505551234"    } ); * Yes, it’s true: you can’t actually make or receive calls
• 19.
  App Launcher bondi.applauncher.launchApplication(    function  ()  {},  //  success    function  ()  {},  //  error    "tel:16505551234" ); //  mailto:... //  sms:... //  file:...
• 20.
  On one hand... •Security-conscious architecture • Comprehensive API support • Consistent API patterns • Conformance specifications • Reasonable test coverage http://www.omtp.org/1.11/apis
• 21.
  On the other...
• 22.
  bondi.* deviceapis.*
• 23.
  WAC 2.0 • Accelerometer • File system • Calendar • Geolocation • Camera • Messaging • Contacts • Orientation • Core • Tasks • Device interaction • Viewport • Device status • Webview http://specs.wacapps.net/
• 24.
  Which is allawesome Except...
• 27.
  Device APIs WorkingGroup “to create client-side APIs that enable the development of Web Applications and Web Widgets that interact with devices services such as Calendar, Contacts, Camera, etc” Joint efforts with WebRTC Working Group, WebApps Working Group http://www.w3.org/2009/dap/
• 28.
  W3C Device APIs •Media Capture(HTML) • Vibration • Media Capture(gUM) • Calendar • Contact • Permissions • Messaging • Menu • Battery Status • Gallery • Network Information • System info • Sensor • Media Stream
• 29.
  Media Capture (HTML) <input  type="file"  accept="image/*"  /> <!-­‐-­‐      The  accept  attribute  is  'a  hint'.    Specification  suggests  that  it  can  be    used  to  offer  alternative  picker  UIs. -­‐-­‐> Public working draft, http://www.w3.org/TR/2011/WD-html-media-capture-20110414
• 30.
  Brad Lassey, September2008
• 31.
  Media Capture (HTML) <input  type="file"  accept="image/*"    capture="camera" /> <!-­‐-­‐      Again,  a  hint.  Valid  options  are:        camera,  camcorder,  microphone,  filesystem -­‐-­‐>
• 33.
  Media Capture (HTML) <input  type="file"  accept="image/*"    capture="camera"  id="photo" /> var  photo  =  document.getElementById('photo'); photo.files[0].getFormatData(    function  (data)  {},  //  success    function  ()  {}            //  error ); //  file  is  a  MediaFile,  extending  File //  data  is  a  MediaFileData
• 34.
  MediaFileData interface  MediaFileData  {        attribute  DOMString          codecs;        attribute  unsigned  long  bitrate;        attribute  unsigned  long  height;        attribute  unsigned  long  width;        attribute  float                  duration; }; ...seems to be under discussion
• 35.
  Media capture (gUM) navigator.getUserMedia(    {        audio:  true,        video:  false    },    function  (stream)  {},  //  success    function  ()  {}                //  error ); //  stream  is  a  LocalMediaStream Editor's draft, http://dev.w3.org/2011/webrtc/editor/getusermedia.html
• 36.
  LocalMediaStream var  recorder  =  stream.record(); //  recorder  is  a  MediaStreamRecorder recorder.getRecordedData(    function  (file)  {}    //  success ); //  file  is  a  File URL.createObjectURL(stream); //  a  Blob  URI  for,  say  <video  src='...'  />    
• 37.
  Messaging navigator.sendMessage(    "sms:16505551234?"  +        "body=JavaScript%20says%20hi",      [],                          //  attachments  as  File  array    function  ()  {},  //  success    function  ()  {}    //  error ); //  also  mms:  and  mailto: //  oh...  were  you  looking  for  receipt? Editor's draft, http://dev.w3.org/2009/dap/messaging/
• 38.
  Sensors navigator.findSensors().onsuccess  =  function()  {    this.result.forEach(function  (sensor)  {        //  ...    }); }; var  sensor  =  new  SensorConnection('Temperature'); sensor.addEventListener('sensordata',  function(e)  {    if(e.data  >  20.0)  {        window.console.log('Hawt');    } }); sensor.startWatch({  interval:  2000  }); Editor's draft, http://dev.w3.org/2009/dap/system-info/Sensors.html
• 39.
  Sensors • Temperature, ºC •AmbientLight, Lux • AmbientNoise, dbA • MagneticField, uTesla • Proximity, cm • AtmPressure, kP • RelHumidity, %
• 40.
  Contacts navigator.contacts.find(    [                                              //  fields        'name',          'emails'    ],      function  (contacts)  {},  //  success    function  ()  {},                  //  error    {                                              //  find  options          filter:  'James',            multiple:  true    } ); Editor's draft, http://w3c-test.org/dap/contacts/
• 41.
  Contact interface  Contact  {        readonly  attribute  DOMString                              id;                          attribute  DOMString?                            displayName;                          attribute  ContactName?                        name;                          attribute  DOMString?                            nickname;                          attribute  ContactField[]?                  phoneNumbers;                          attribute  ContactField[]?                  emails;                          attribute  ContactAddress[]?              addresses;                          attribute  ContactField[]?                  ims;                          attribute  ContactOrganization[]?    organizations;                          attribute  Date?                                      birthday;                          attribute  DOMString?                            note;                          attribute  ContactField[]?                  photos;                          attribute  DOMString[]?                        categories;                          attribute  ContactField[]?                  urls; };
• 42.
  Cool, but... “Theability to add and update contact information is not a function of the API provided in this specification.”
• 43.
  Holy Contacts EditingWTF var  vcard  =      'BEGIN:VCARDrn'  +    'VERSION:3.0rn'  +    'UID:'  +  contact.id  +  'rn'  +          'N:Pearce;Jamesrn'  +    'FN:James  Pearcern'  +    'END:VCARD'; document.getElementById('vcard').href  =    'data:text/x-­‐vcard;charset=utf-­‐8,'  +    encodeURIComponent(vcard);  
• 46.
  WebAPI “We areaiming at providing all the necessary APIs to build a basic HTML5 phone experience within the next 3-6 months” https://wiki.mozilla.org/WebAPI
• 47.
  WebAPI • Telephony • Embedded browser • Messaging • Battery • Contacts • Device Status • Camera • Settings • Network • Mouse Lock • USB • Vibrator
• 48.
  Hopefully W3C++ • “Ourproposal might or might not look like it for the moment”
• 49.
  Telephony! var  tcall  =    navigator.telephony.dial("16505551234"); tcall.readyState;    //  dialing,  ringing,  busy,  connecting,    //  connected,  disconnecting,  disconnected,      //  incoming tcall.onconnected  =  function  ()  {};    //  onbusy,  ondisconnected  etc tcall2.answer(); http://www.youtube.com/watch?v=RuIQskGD3u0#t=19s
• 50.
  Contacts “We needread/write”
• 51.
  USB var  usb  =  new  MozUSBManager(); usb.addEventListener("attachdevice",      function  ()  {} ); usb.addEventListener("detachdevice",      function  ()  {} ); usb.claimDevice({    deviceClass:  ...,    deviceSubClass:  ... });
• 53.
  We deserve DeviceAPIs... • As forward-looking as WTAI • As well-structured as BONDI • As comprehensive as WAC • As respectable as W3C DAP • As agile as WebAPI
• 54.
  And we’ve got...
• 55.
  PhoneGap • Accelerometer • Events • Camera • File • Capture • Geolocation • Compass • Media • Connection • Notification • Contacts • Storage • Device
• 56.
  Use Cases
• 57.
  Useless Demos https://github.com/jamesgpearce/compios5
• 58.
  Fixing layout bugs,FML window.addEventListener("deviceorientation",    function  (e)  {        orientation  =  Math.abs(w.orientation);        rotation  =  Math.abs(e.gamma);        if  (rotation  >  8  &&  orientation  ===  0)  {            if  (enabled)  {                disableZoom();            }  else  {                restoreZoom();            }        }    },  false ); http://vimeo.com/adactio/iosbug https://github.com/scottjehl/iOS-Orientationchange-Fix
• 59.
  Camera • Profile pictures •Review sites • Photo sharing • Recognition • Bar-codes • Text • Faces
• 60.
  Filters! $('img').vintage({    vignette:  {        black:  0.8,        white:  0.2    },    noise:  20,    screen:  {        red:  12,        ...    },    desaturate:  0.05 }); https://github.com/rendro/vintageJS
• 61.
  Augmented reality? http://fhtr.org/JSARToolKit/
• 62.
  Contacts & calendaring •Add business details to address book • Web-based messaging • Facial recognition again... • Presence • Context-sensitivity of content
• 63.
  Messaging • Sharing • Notifications •Offline transport fallback • Frictionless web sign-on
• 64.
  Expect more
• 65.
  I still want... • Audio processing • Bluetooth • RFID & NFC • Both cameras • Memory & power • Native recognition
• 66.
  window.addEventListener(    'appointmentdue',    function(appt)  {        if  (device.near(WORK))  {            siri.remind(                contacts.get('Peter'),                calendar.getDetails(appt)            );        }    },    false ); PS: does not work
• 67.
  James Pearce @jamespearce http://tripleodeon.com