SAFETY ENHANCEMENT

THROUGH SITUATION-AWARE
USER INTERFACES
Vincenzo De Florio

© PATS Research Group

vincenzo.deflorio@u...
Introduction
• GUI: contact point between two worlds
 U: User (Physical) world
 C: Cyber world

• These two worlds are v...
How are GUIs now (1/2)
• Adaptive, anticipative, personalized,
"intelligent"...
• ...but mostly focusing on functional
asp...
How are GUIs now (2/2)
• User activity is unquestioned
 “Does it make sense?”
 “Is it "logical" / "meaningful" / SAFE?”
...
Idea
• GUI as a usability sensor
• All U-activity reported dynamically to C
 Both functional and non-functional
 Actions...
Application (1/5)
• C makes use of the model to…
1. tell whether U's behavior is “in order"
rather than "abnormal"
• Safet...
Application (2/5)
• C makes use of the model to
4. reshape the GUI
• WYSIWYU: What-you-see-is-what-youunderstand and expec...
Application (3/5)
• C makes use of the model to
5. reshape itself
 Unneeded functionality is "unloaded" →
reduced complex...
Application (4/5)
• C makes use of the model to
6. "...tell Computers and Humans Apart“
 as in CAPTCHA: "Complete Automat...
Application (5/5)
• C makes use of the model to
7. provide the business end with usability
intelligence
 Big data about u...
Approach: Design of
Autonomic GUIs
• GUI publishes widget actions and times
 Simple Tcl/Tk toy GUI

• Interaction logger ...
Simple analysis: QoE

7-Dec-13

12
Visualization

7-Dec-13

13
Detection of discomfort

7-Dec-13

14
Discomfort
detected:
Burst (iRate = 212.7134279)
7-Dec-13

15
Dynamic adaptation
• Current (trivial) strategy: GUI executes
the interaction analyzer of its own
current interaction

7-D...
Dynamic adaptation
• No separation of design concerns
• Embedded in one GUI
• Solution: aspect orientation
 Future work

...
Multiple
detections
unravel
situations

After six
discomfort
detections, we
assume we are
in an unsafe
situation

 Identi...
Conclusions (1/3)
• Analyses of user interaction can tell us
much about the user
 Is s/he in command?
 Is s/he still the...
Conclusions (2/3)
• This knowledge may (should!) be used
to understand what went wrong / react
before things go wrong
• “G...
Conclusions (3/3)
• Here, a simple proof of concepts – with
many potential applications
• Embedded in one simple GUI
• No ...
Please let us know your
questions / ideas for
collaborations!
vincenzo.deflorio@ua.ac.be
jonas.buys@ua.ac.be
7-Dec-13

23
References
• [GD12a] Gui, N. and De Florio, V.:
"Transformer: an adaptation framework with
contextual adaptation behavior ...
Upcoming SlideShare
Loading in …5
×

SAFETY ENHANCEMENT THROUGH SITUATION-AWARE USER INTERFACESIet system safety 2012

342 views

Published on

Due to their privileged position halfway the physical and the cyber universe, user interfaces may play an important role in learning, preventing, and tolerating scenarios affecting the safety of the mission and the user's quality of experience. This vision is embodied here in the main ideas and a proof-of-concepts implementation of user interfaces combining dynamic profiling with context- and situation-awareness and autonomic software adaptation.

The corresponding paper may be downloaded from https://dl.dropboxusercontent.com/u/67040428/Articles/gg.pdf

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
342
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

SAFETY ENHANCEMENT THROUGH SITUATION-AWARE USER INTERFACESIet system safety 2012

  1. 1. SAFETY ENHANCEMENT THROUGH SITUATION-AWARE USER INTERFACES Vincenzo De Florio © PATS Research Group vincenzo.deflorio@ua.ac.be
  2. 2. Introduction • GUI: contact point between two worlds  U: User (Physical) world  C: Cyber world • These two worlds are very different  Different notion of time  Different notions of behavior, actions, evolution… • The GUI is the "medium" between these two distant realities 7-Dec-13 2
  3. 3. How are GUIs now (1/2) • Adaptive, anticipative, personalized, "intelligent"... • ...but mostly focusing on functional aspects  GUI is a way to "send commands to the other side"  → No interpretation of the user behavior 7-Dec-13 3
  4. 4. How are GUIs now (2/2) • User activity is unquestioned  “Does it make sense?”  “Is it "logical" / "meaningful" / SAFE?”  “Is it "normal?" >> (w.r.t. given user, given specs, given environment...) • No assessment of QoE • No assessment of the current situations 7-Dec-13 4
  5. 5. Idea • GUI as a usability sensor • All U-activity reported dynamically to C  Both functional and non-functional  Actions, mistakes, timing b/w actions... (big data!) • C then uses U-activity to build/refine a model of U  Stereotypes, rules, hidden Markov models, Bayesian intelligence...  (Currently, simple rules) 7-Dec-13 5
  6. 6. Application (1/5) • C makes use of the model to… 1. tell whether U's behavior is “in order" rather than "abnormal" • Safety, confidentiality... • E.g. switching off features when misbehaviours or erroneous “human-machine conversations” are detected. 2. tell whether the user has changed • GUI detects an unusual stereotype → device in different hands? • GUI as biometric sensor 3. Detect / react from lack of reactivity 7-Dec-13 6
  7. 7. Application (2/5) • C makes use of the model to 4. reshape the GUI • WYSIWYU: What-you-see-is-what-youunderstand and expect • I don't need it? It's not there  E.g.: Better management of screen space  E.g.: eInclusion 7-Dec-13 7
  8. 8. Application (3/5) • C makes use of the model to 5. reshape itself  Unneeded functionality is "unloaded" → reduced complexity → less faults, less resource consumption...  à la Transformer [GD12a,GD12b] 7-Dec-13 8
  9. 9. Application (4/5) • C makes use of the model to 6. "...tell Computers and Humans Apart“  as in CAPTCHA: "Complete Automatic Public Turing test ..."  Does the client interact as a human user? Gestalt psychology, morphisms,...  Enhanced robustness against attacks → safety in eBusiness 7-Dec-13 9
  10. 10. Application (5/5) • C makes use of the model to 7. provide the business end with usability intelligence  Big data about usability of devices / services  Data analysis may help tell what feature is most wanted / most hated in products  Etc. 7-Dec-13 10
  11. 11. Approach: Design of Autonomic GUIs • GUI publishes widget actions and times  Simple Tcl/Tk toy GUI • Interaction logger receives actions/times stream and creates a compact representation (iCode) • iCode is sent to an Interaction analyser  Context is gathered and situations analysed • Actions are then planned and executed  The GUI is adapted 7-Dec-13 11
  12. 12. Simple analysis: QoE 7-Dec-13 12
  13. 13. Visualization 7-Dec-13 13
  14. 14. Detection of discomfort 7-Dec-13 14
  15. 15. Discomfort detected: Burst (iRate = 212.7134279) 7-Dec-13 15
  16. 16. Dynamic adaptation • Current (trivial) strategy: GUI executes the interaction analyzer of its own current interaction 7-Dec-13 16
  17. 17. Dynamic adaptation • No separation of design concerns • Embedded in one GUI • Solution: aspect orientation  Future work 7-Dec-13 17
  18. 18. Multiple detections unravel situations After six discomfort detections, we assume we are in an unsafe situation  Identity of the user must be reconfirmed 7-Dec-13 18
  19. 19. Conclusions (1/3) • Analyses of user interaction can tell us much about the user  Is s/he in command?  Is s/he still the same who logged in?  Etc (re: crash of Air France 447) • They can tell us much about the interface  Did the interface behave as expected?  (re: Therac-25 accidents) 7-Dec-13 20
  20. 20. Conclusions (2/3) • This knowledge may (should!) be used to understand what went wrong / react before things go wrong • “Going wrong” ranges from usability to safety issues 7-Dec-13 21
  21. 21. Conclusions (3/3) • Here, a simple proof of concepts – with many potential applications • Embedded in one simple GUI • No actual experimentation • Many unanswered questions:  Analyzers may be too simple or simply wrong  Components may fail – what then?  Etc • Complex problems that call for multidisciplinary solutions 7-Dec-13 22
  22. 22. Please let us know your questions / ideas for collaborations! vincenzo.deflorio@ua.ac.be jonas.buys@ua.ac.be 7-Dec-13 23
  23. 23. References • [GD12a] Gui, N. and De Florio, V.: "Transformer: an adaptation framework with contextual adaptation behavior composition support," accepted for publication in Software: Practice & Experience, Wiley, 2012. • [GD12b] Gui, N., De Florio, V., "Towards Meta-Adaptation support with Reusable and Composable Adaptation Components," Sixth IEEE International Conference on SelfAdaptive and Self-Organizing Systems (SASO 2012), Lyon, France, 10-14 September 2012. 7-Dec-13 24

×