Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multidevice Controls: A Different Approach to UX

10 views

Published on

Everybody knows boring form-based user interfaces. What if you could add mobile devices to improve the UX of desktop or web applications? A research project at the University of Applied Sciences and Arts Northwestern Switzerland has tried to address this topic in a totally different way. Usually you would expect to rework the UI and make it fancier, but here the idea is to enhance controls so they can reside on a mobile device. For example, you can think about a text input field that gets the focus, with the actual data input being done on a mobile phone. This session presents an overview of the concept and shows you some results of the research project in the form of demos based on Swift, JavaFX, and Polymer.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Multidevice Controls: A Different Approach to UX

  1. 1. Another approach to UX Controls Multi Device
  2. 2. ABOUT ME. Gerrit Grunwald | Engineer
  3. 3. University of Applied Sciences and Arts Northwestern Switzerland
  4. 4. The IDEA
  5. 5. Combining the Platforms
  6. 6. The Principle
  7. 7. Apple Touch Bar
  8. 8. Separate Controls
  9. 9. Separate Controls
  10. 10. The Requirements
  11. 11. Mobile AVAILABILITY
  12. 12. Mobile Availability 97.5% 14-29 Years 96.7% 30-49 Years 96.5% 50-64 Years 68.0% 65 and older (Bitkom, Germany 2013) No phone 9.6 % Phone 90.4 %
  13. 13. USAGE
  14. 14. 1AM 3AM 6AM 9AM 12PM 3PM 6PM 9PM 12AM PHONE DESKTOP Office hours Usage
  15. 15. PHONE DESKTOP 1AM 3AM 6AM 9AM 12PM 3PM 6PM 9PM 12AM Office hours Usage
  16. 16. The prerequisite
  17. 17. Synchronization
  18. 18. Synchronization Change Change Desktop Mobile
  19. 19. Synchronization Change Change Desktop Mobile
  20. 20. Synchronization Change Change Desktop Mobile Server
  21. 21. Version1.
  22. 22. ★ USing only java ★ Controls with additional functionality ★ same control on desktop and mobile ★ different VISUALIZATION for each platform 1. Version
  23. 23. Technology
  24. 24. One Control… 15.0+4
  25. 25. One Control… 15.0+4 333.0
  26. 26. Gluon Mobile
  27. 27. ★ Single Application ★ Multiple Platforms ★ Java end to end ★ Access hardware by API ★ Native apps for iOS and Android Gluon Mobile
  28. 28. Architecture
  29. 29. Architecture OpenDolphin
  30. 30. Open Dolphin
  31. 31. Open Dolphin Desktop Client Presentation Model Presentation Model Presentation Model EventBus Mobile Client Presentation Model Server Binding Binding Sync Sync Sync
  32. 32. ★ Binds controls to the presentation model ★ takes care about Synchronization Open Dolphin
  33. 33. Demo
  34. 34. CONSPROS
  35. 35. ConsPros ★Same code base ★Easy to develop ★Easy to test ★Easy to deploy ★One control on all environments ★All plain Java ★Overhead by OpenDolphin ★Performance loss ★Bound to Java ecosystem
  36. 36. Version2.
  37. 37. ★ platform independent ★ standard FRAMEWORK controls on desktop ★ ONE control ON MOBILE for each desktop control 2. Version
  38. 38. Technology
  39. 39. Technology
  40. 40. Architecture
  41. 41. Architecture MQTT Polymer
  42. 42. Using MQTT
  43. 43. Using MQTT Client Client Broker WE HAVE TO SYNCHRONIZE
  44. 44. Using MQTT Mobile Client Broker CLIENT SIDE VALIDATION Validation Validation JavaFX Client Validation Web Client
  45. 45. content ACROSS CTRLS Shared
  46. 46. ★ CTRL Type ★ Ctrl id ★ Numbers ( Double, Integer ) ★ Min- and Max-Values ★ Unit ★ Text Shared content
  47. 47. { }Json
  48. 48. { "ctrl" : "TextField", "id" : "numberField", "type" : "NUMBER", "text" : "", "min" : 0, "max" : 100, "value" : 4, "decimals" : 0, "unit" : "" "combo" : "" } JSON
  49. 49. Types…
  50. 50. Types… ★ TEXT ★ NUMBER ★ NUMERIC ★ DATE ★ COLOR ★ COMBO ★ SIGNATURE ★ IMAGE
  51. 51. public enum Type { // Types of Controls UNKNOWN, TEXT, NUMBER, NUMERIC, DATE, COLOR, COMBO, SIGNATURE, IMAGE, // Commands EXIT, NEXT, PREVIOUS, REQUEST } Types of Controls
  52. 52. The App…
  53. 53. The App… Polymer
  54. 54. Controls…
  55. 55. Controls… ★ TEXTFIELD ( Number, Numeric, Date, Color ) ★ SPINNER ★ Textarea ★ COMBOBOX ★ IMAGEVIEW ( photo, signature )
  56. 56. Representations…
  57. 57. NumberField
  58. 58. NumberField Polymer
  59. 59. NumericField
  60. 60. NumericField Polymer
  61. 61. DateField
  62. 62. DateField Polymer
  63. 63. ColorField
  64. 64. ColorField Polymer
  65. 65. Spinner
  66. 66. Spinner Polymer
  67. 67. TextArea
  68. 68. TextArea Polymer
  69. 69. ComboBox
  70. 70. ComboBox Polymer
  71. 71. Photo
  72. 72. Photo Polymer
  73. 73. Signature
  74. 74. Signature Polymer
  75. 75. flow The
  76. 76. The Flow Polymer Field gains focus
  77. 77. The Flow Polymer Field gains focus { } { "ctrl" : "TextField", "id" : "numberField", "type" : "NUMBER", "text" : "", "min" : 0, "max" : 100, "value" : 1, "decimals" : 0, "unit" : "", "combo" : "" } Generate JSON
  78. 78. The Flow Polymer { } { "ctrl" : "TextField", "id" : "numberField", "type" : "NUMBER", "text" : "", "min" : 0, "max" : 100, "value" : 1, "decimals" : 0, "unit" : "", "combo" : "" } Generate JSON Waiting… Send to Device via MQTT
  79. 79. The Flow Polymer Waiting… Send to Device via MQTT { } { "ctrl" : "TextField", "id" : "numberField", "type" : "NUMBER", "text" : "", "min" : 0, "max" : 100, "value" : 1, "decimals" : 0, "unit" : "", "combo" : "" }
  80. 80. The Flow Polymer Generate JSON and send to Desktop { } { "ctrl" : "TextField", "id" : "numberField", "type" : "NUMBER", "text" : "", "min" : 0, "max" : 100, "value" : 5, "decimals" : 0, "unit" : "", "combo" : "" }
  81. 81. The Flow Polymer file.io upload image
  82. 82. The Flow Polymer file.io upload imageget url back
  83. 83. The Flow Polymer file.io { } { "ctrl" : "ImageView", "id" : "photo", "type" : "NUMBER", "text" : "<URL>", "min" : 0, "max" : 100, "value" : 5, "decimals" : 0, "unit" : "", "combo" : "" }
  84. 84. Demo - - Connected to network - - Screen Mirroring activated - - Airserver running - - iPhone hooked to Airserver via airplay - - Airplay in window and not fullscreen mode - - gradle jproRun to run JPro demo - - polymer serve to run Polymer demo
  85. 85. CONSPROS
  86. 86. ★Ecosystem independent ★Better performance ★Easy to deploy ★One mobile client for different desktop platforms ★Different code base ★Platform knowledge needed ★One control for each environment ★Needs at least ios and Android client ConsPros General cons: Bad connection -> bad ux 100 ms -> instantaneously 200 ms -> dogged (esp. for coders) 1000 ms -> limit for user's flow of thought to stay uninterrupted 10000 ms -> limit to keep user focused on dialog
  87. 87. Thank U

×