In Touch with Things:
Programming NFC on Android with MORENA
Kevin Pinte
Andoni Lombide Carreton
Wolfgang De Meuter




Droidcon
April 10, 2013
Berlin, Germany
RFID in Android

• NFC (touch range).
                            NdefMessage: {
• Callback on activities                        ,
  to detect RFID tags
  with subscribed MIME-                         , NdefRecord
  type in memory.                                 byte array
                            }
• File access abstraction


                                read    write
Drawbacks of the Android NFC API

• Manual failure handling
  failures are the rule rather than the exception:
  NFC causes A LOT of failures because of its hardware characteristics


• Blocking communication
  Android documentation recommends to use a separate thread
  for many NFC operations


• Manual data conversion


• Tight coupling with activity-based architecture
Lessons Learnt from Previous Ambient-Oriented
Programming Research

• Using objects as first class software representations for RFID-tagged “things”
  is a nice abstraction.


• These objects can be directly stored in the RFID tags’ memory to minimize
  data conversion.


• Event-driven discovery (fortunately built-in into Android).


• Asynchronous, fault tolerant reads and writes.
                                                         experimental scripting
                                                       language for mobile apps
                                                          in ad hoc networks
       RFID tags as “mobile
        devices” and I/O as
      network communication
MORENA Middleware Architecture



           Application

           Thing level       One middleware
                             for Android 4.0
                             or higher (API 14)
            Tag level

            Android
Evaluation: Wi-Fi Sharing Application
Evaluation: Wi-Fi Sharing Application
Things

                                                      From now on, we don’t
public class WifiConfig extends Thing {               have to worry about the
	   public String ssid_;
                                                         activity anymore.
	   public String key_;

	   public WifiConfig(ThingActivity<WifiConfig> activity, String ssid, String key) {
	   	 super(activity);
	   	 ssid_ = ssid;
	   	 key_ = key;
	   }
	
	   public boolean connect(WifiManager wm) {
    	 // Connect to ssid_ with password key_	
	   };
                                                Supported serialization:
}
                                                 - JSON-serializable fields.
                                                 - Skipping transient fields.
                                                 - Deep serialization, no cycles.
Initializing Things
                           As soon as an empty
                              tag is detected
   @Override
   public void whenDiscovered(EmptyRecord empty) {
       empty.initialize(
           myWifiThing,
           new ThingSavedListener<WifiConfig>() {
   	 	 	       @Override
   	 	 	       public void signal(WifiConfig thing) {
   	 	 	 	         toast("WiFi joiner created!");
   	 	 	       }
   	 	     },
   	 	     new ThingSaveFailedListener() {
   	 	 	       @Override
   	 	 	       public void signal() {
   	 	 	 	        toast("Creating WiFi joiner failed, try again.");
   	 	 	       }
   	 	     },
           5000);
   }
Discovering and Reading Things


                       As soon as a WifiConfig
                           tag is detected
     @Override
     public void whenDiscovered(WifiConfig wc) {
        toast("Joining Wifi network " + wc.ssid_);
     	 wc.connect();
     }
                              Contains cached fields for
                                synchronous access.
                               Physical reads must be
                                   asynchronous.
Saving Modified Things
 myWifiConfig.ssid_ = "MyNewWifiName";
 myWifiConfig.key_ = "MyNewWifiPassword";

 myWifiConfig.saveAsync(
     new ThingSavedListener<WifiConfig>() {
         @Override
         public void signal(WifiConfig wc) {
 	 	 	       toast("WiFi joiner saved!");
 	 	 	 }
     },
 	   new ThingSaveFailedListener() {
 	       @Override
 	 	 	 public void signal() {
             toast("Saving WiFi joiner failed, try again.");
 	 	 	 }
     },
     5000);
Broadcasting Things to Other Phones

                         Will trigger whenDiscovered on the
                     receiving phone with the broadcasted thing

 myWifiConfig.broadcast(
       new ThingBroadcastSuccessListener<WifiConfig>() {
 	         @Override
 	 	 	    public void signal(WifiConfig wc) {
 	 	 	 	     toast("WiFi joiner shared!");
 	 	 	    }
 	 	 },
 	 	 new ThingBroadcastFailedListener<WifiConfig>() {
 	         @Override
 	 	 	    public void signal(WifiConfig wc) {
 	 	           toast("Failed to share WiFi joiner, try again.");
 	 	 	    }
       },
       5000);
Evaluation: WiFi Sharing Application
MORENA Middleware Architecture



           Application

           Thing level       One middleware
                             for Android 4.0
            Tag level        or higher (API 14)

            Android
Detecting RFID Tags

    private class TextTagDiscoverer extends TagDiscoverer {	 	
    	 @Override
    	 public void onTagDetected(TagReference tagReference) {
    	 	 readTagAndUpdateUI(tagReference);
    	 }
    	 	
    	 @Override
    	 public void onTagRedetected(TagReference tagReference) {
    	 	 readTagAndUpdateUI(tagReference);
    	 }
    }



    new TextTagDiscoverer(
       currentActivity,
       TEXT_TYPE,
       new NdefMessageToStringConverter(),
       new StringToNdefMessageConverter());
The Tag Reference Abstraction
Reading RFID Tags


  tagReference.read(
        new TagReadListener() {
             @Override
  	         public void signal(TagReference tagReference) {
  	             // tagReference.getCachedData()
  	 	      }
  	    },
  	    new TagReadFailedListener() {
  	         @Override
  	         public void signal(TagReference tagReference) {
  	             // Deal with failure
  	         }
  	   });
Writing RFID Tags

  tagReference.write(
        toWrite,
        new TagWrittenListener() {
  	         @Override
  	         public void signal(TagReference tagReference) {
  	             // Handle write success
  	 	      }
  	    },
  	    new TagWriteFailedListener() {
  	         @Override
  	 	      public void signal(TagReference tagReference) {
  	 	          // Deal with failure
  	 	      }
  	   });
Fine-grained Filtering

  private class TextTagDiscoverer extends TagDiscoverer {	 	
  	 @Override
  	 public void onTagDetected(TagReference tagReference) {
  	 	 readTagAndUpdateUI(tagReference);
  	 }
  	 	
  	 @Override
  	 public void onTagRedetected(TagReference tagReference) {
  	 	 readTagAndUpdateUI(tagReference);
  	 }

      @Override
  	   public boolean checkCondition(TagReference tagReference) {
          // Can be used to apply a predicate
          // on tagReference.getCachedData()
  	   }
  }
MORENA Conclusion

• Event-driven discovery.


• Non-blocking communication:


  • Things: cached copy, asynchronous saving of cached data.


  • TagReferences: first class references to RFID tags offering asynchronous
    reads and writes.


• Automatic data conversion


• Looser coupling from activity-based architecture
Current Research: Volatile Database

• Data structures over many tags?


• Querying collections of things?


• Stronger consistency guarantees?


• Transactions?


• Validations?


• ...                                Active Record (RoR)
                                           for RFID
Thing Associations
  defmodel: Shelf properties: {
  	 number: Number
  } associations: {
  	 hasMany: `books               a shelf contains many books
  };



  defmodel: Book properties: {
  	 title: Text;
  	 authors: Text;
  	 isbn: Text;
  } proto: {
  	 def isMisplaced(currentShelf) {
  	 	 shelf != currentShelf;
  	 };
  } associations: {
     belongsTo: `shelf         a book belongs to exactly
  };                             one shelf in the library
Instantiating a Model and Saving to Tag
  def newBook := Book.create: {
     title   := “Agile Web Development with Rails”;
     authors := “Dave Thomas, ...”;
     isbn    := “978-0-9776-1663-3”;
  };
  def saveReq := newBook.saveAsync(10.seconds);
  when: saveReq succeeded: {
     // the book was saved to a tag
  } catch: { |exc|
     // book could not be saved within 10 sec
  };

  newBook.shelf := shelf42;

                                    associate a book with a shelf:
  shelf42.books << newBook;
                                      foreign key in book thing
Working with Many Things: Reactive Queries

                 finding misplaced books in a library


  def shelf42 := Shelf.all.first: { |s| s.number == 42 };
  def currentShelf := Shelf.all.last;

                                          most recently scanned shelf

  def misplacedBooks := Book.all.where: { |b|
     b.isMisplaced(currentShelf)
  };                                     all books  not matching the
                                              last scanned shelf
  misplacedBooks.each: { |b|
     b.shelf := currentShelf;
     b.saveAsync(5.seconds)
  };
                                  synchronize misplaced books
                                        with current shelf
In Touch with Things:
Programming NFC on Android with MORENA

            tinyurl.com/morena-android


            tinyurl.com/kevinpinte
            kevin.pinte@vub.ac.be
            @bommasaurus



            code.google.com/p/ambienttalk
            soft.vub.ac.be/amop

            @ambienttalk


             Droidcon, April 10, 2013, Berlin, Germany

Pinte

  • 1.
    In Touch withThings: Programming NFC on Android with MORENA Kevin Pinte Andoni Lombide Carreton Wolfgang De Meuter Droidcon April 10, 2013 Berlin, Germany
  • 2.
    RFID in Android •NFC (touch range). NdefMessage: { • Callback on activities , to detect RFID tags with subscribed MIME- , NdefRecord type in memory. byte array } • File access abstraction read write
  • 3.
    Drawbacks of theAndroid NFC API • Manual failure handling failures are the rule rather than the exception: NFC causes A LOT of failures because of its hardware characteristics • Blocking communication Android documentation recommends to use a separate thread for many NFC operations • Manual data conversion • Tight coupling with activity-based architecture
  • 4.
    Lessons Learnt fromPrevious Ambient-Oriented Programming Research • Using objects as first class software representations for RFID-tagged “things” is a nice abstraction. • These objects can be directly stored in the RFID tags’ memory to minimize data conversion. • Event-driven discovery (fortunately built-in into Android). • Asynchronous, fault tolerant reads and writes. experimental scripting language for mobile apps in ad hoc networks RFID tags as “mobile devices” and I/O as network communication
  • 5.
    MORENA Middleware Architecture Application Thing level One middleware for Android 4.0 or higher (API 14) Tag level Android
  • 6.
  • 7.
  • 8.
    Things From now on, we don’t public class WifiConfig extends Thing { have to worry about the public String ssid_; activity anymore. public String key_; public WifiConfig(ThingActivity<WifiConfig> activity, String ssid, String key) { super(activity); ssid_ = ssid; key_ = key; } public boolean connect(WifiManager wm) { // Connect to ssid_ with password key_ }; Supported serialization: } - JSON-serializable fields. - Skipping transient fields. - Deep serialization, no cycles.
  • 9.
    Initializing Things As soon as an empty tag is detected @Override public void whenDiscovered(EmptyRecord empty) { empty.initialize( myWifiThing, new ThingSavedListener<WifiConfig>() { @Override public void signal(WifiConfig thing) { toast("WiFi joiner created!"); } }, new ThingSaveFailedListener() { @Override public void signal() { toast("Creating WiFi joiner failed, try again."); } }, 5000); }
  • 10.
    Discovering and ReadingThings As soon as a WifiConfig tag is detected @Override public void whenDiscovered(WifiConfig wc) { toast("Joining Wifi network " + wc.ssid_); wc.connect(); } Contains cached fields for synchronous access. Physical reads must be asynchronous.
  • 11.
    Saving Modified Things myWifiConfig.ssid_ = "MyNewWifiName"; myWifiConfig.key_ = "MyNewWifiPassword"; myWifiConfig.saveAsync( new ThingSavedListener<WifiConfig>() { @Override public void signal(WifiConfig wc) { toast("WiFi joiner saved!"); } }, new ThingSaveFailedListener() { @Override public void signal() { toast("Saving WiFi joiner failed, try again."); } }, 5000);
  • 12.
    Broadcasting Things toOther Phones Will trigger whenDiscovered on the receiving phone with the broadcasted thing myWifiConfig.broadcast( new ThingBroadcastSuccessListener<WifiConfig>() { @Override public void signal(WifiConfig wc) { toast("WiFi joiner shared!"); } }, new ThingBroadcastFailedListener<WifiConfig>() { @Override public void signal(WifiConfig wc) { toast("Failed to share WiFi joiner, try again."); } }, 5000);
  • 13.
  • 14.
    MORENA Middleware Architecture Application Thing level One middleware for Android 4.0 Tag level or higher (API 14) Android
  • 15.
    Detecting RFID Tags private class TextTagDiscoverer extends TagDiscoverer { @Override public void onTagDetected(TagReference tagReference) { readTagAndUpdateUI(tagReference); } @Override public void onTagRedetected(TagReference tagReference) { readTagAndUpdateUI(tagReference); } } new TextTagDiscoverer( currentActivity, TEXT_TYPE, new NdefMessageToStringConverter(), new StringToNdefMessageConverter());
  • 16.
    The Tag ReferenceAbstraction
  • 17.
    Reading RFID Tags tagReference.read( new TagReadListener() { @Override public void signal(TagReference tagReference) { // tagReference.getCachedData() } }, new TagReadFailedListener() { @Override public void signal(TagReference tagReference) { // Deal with failure } });
  • 18.
    Writing RFID Tags tagReference.write( toWrite, new TagWrittenListener() { @Override public void signal(TagReference tagReference) { // Handle write success } }, new TagWriteFailedListener() { @Override public void signal(TagReference tagReference) { // Deal with failure } });
  • 19.
    Fine-grained Filtering private class TextTagDiscoverer extends TagDiscoverer { @Override public void onTagDetected(TagReference tagReference) { readTagAndUpdateUI(tagReference); } @Override public void onTagRedetected(TagReference tagReference) { readTagAndUpdateUI(tagReference); } @Override public boolean checkCondition(TagReference tagReference) { // Can be used to apply a predicate // on tagReference.getCachedData() } }
  • 20.
    MORENA Conclusion • Event-drivendiscovery. • Non-blocking communication: • Things: cached copy, asynchronous saving of cached data. • TagReferences: first class references to RFID tags offering asynchronous reads and writes. • Automatic data conversion • Looser coupling from activity-based architecture
  • 21.
    Current Research: VolatileDatabase • Data structures over many tags? • Querying collections of things? • Stronger consistency guarantees? • Transactions? • Validations? • ... Active Record (RoR) for RFID
  • 22.
    Thing Associations defmodel: Shelf properties: { number: Number } associations: { hasMany: `books a shelf contains many books }; defmodel: Book properties: { title: Text; authors: Text; isbn: Text; } proto: { def isMisplaced(currentShelf) { shelf != currentShelf; }; } associations: { belongsTo: `shelf a book belongs to exactly }; one shelf in the library
  • 23.
    Instantiating a Modeland Saving to Tag def newBook := Book.create: { title := “Agile Web Development with Rails”; authors := “Dave Thomas, ...”; isbn := “978-0-9776-1663-3”; }; def saveReq := newBook.saveAsync(10.seconds); when: saveReq succeeded: { // the book was saved to a tag } catch: { |exc| // book could not be saved within 10 sec }; newBook.shelf := shelf42; associate a book with a shelf: shelf42.books << newBook; foreign key in book thing
  • 24.
    Working with ManyThings: Reactive Queries finding misplaced books in a library def shelf42 := Shelf.all.first: { |s| s.number == 42 }; def currentShelf := Shelf.all.last; most recently scanned shelf def misplacedBooks := Book.all.where: { |b| b.isMisplaced(currentShelf) }; all books not matching the last scanned shelf misplacedBooks.each: { |b| b.shelf := currentShelf; b.saveAsync(5.seconds) }; synchronize misplaced books with current shelf
  • 25.
    In Touch withThings: Programming NFC on Android with MORENA tinyurl.com/morena-android tinyurl.com/kevinpinte kevin.pinte@vub.ac.be @bommasaurus code.google.com/p/ambienttalk soft.vub.ac.be/amop @ambienttalk Droidcon, April 10, 2013, Berlin, Germany