High-Confidence
Ubiquitous Computing Systems
           David S. Rosenblum
          School of Computing
     National University of Singapore
Some Facts
          Google Android Market


• The average price of the top 50 paid
  applications is US$3.79   [modymi.com]

• 79.3% of paid applications have been
  downloaded less than 100 times     [Distimo]

• Only 0.1% of paid applications have been
  downloaded 50,000 times or more          [Distimo]
Some Facts
             Google Android Market


   • The average price of the top 50 paid
     applications is US$3.79   [modymi.com]

   • 79.3% of paid applications have been
     downloaded less than 100 times     [Distimo]

   • Only 0.1% of paid applications have been
     downloaded 50,000 times or more          [Distimo]


There are many simplistic, low-quality apps!
CAAAs
Context-Aware Adaptive Applications
CAAAs
Context-Aware Adaptive Applications
CAAAs
Context-Aware Adaptive Applications
CAAAs
Context-Aware Adaptive Applications
CAAAs
Context-Aware Adaptive Applications
Adaptation in CAAAs

       Application




       Environment
Adaptation in CAAAs

                   Application




Physical Context   Environment
Adaptation in CAAAs

                    Application


                   Adaptation



                                Middleware
                    Manager


                    Context
                    Manager

Physical Context   Environment
Adaptation in CAAAs

                    Application


                   Adaptation



                                Middleware
                    Manager


                    Context
Sensed Context      Manager

Physical Context   Environment
Adaptation in CAAAs

                    Application


                   Adaptation



                                Middleware
Inferred Context    Manager


                    Context
Sensed Context      Manager

Physical Context   Environment
Adaptation in CAAAs

Presumed Context     Application


                    Adaptation



                                 Middleware
 Inferred Context    Manager


                     Context
  Sensed Context     Manager

 Physical Context   Environment
Adaptation in CAAAs

Presumed Context     Application


                    Adaptation



                                 Middleware
 Inferred Context    Manager


                     Context                  3rd-Party
  Sensed Context     Manager                  Libraries

 Physical Context   Environment
Adaptation in CAAAs

Presumed Context     Application


                    Adaptation                  Rule


                                 Middleware
 Inferred Context    Manager                   Engine


                     Context                  3rd-Party
  Sensed Context     Manager                  Libraries

 Physical Context   Environment
Validation of CAAAs

        Application


       Adaptation                 Rule


                    Middleware
        Manager                  Engine


        Context
        Manager

       Environment
Validation of CAAAs
  Rules are strongly
   interdependent        Application

 and have multiple
     priorities         Adaptation                 Rule


                                     Middleware
                         Manager                  Engine

  making reasoning
                         Context
 difficult even for a     Manager
small number of rules
                        Environment
Validation of CAAAs

        Application


       Adaptation



                    Middleware
        Manager


        Context                  3rd-Party
        Manager                  Libraries

       Environment
Validation of CAAAs
Context is sensed
  periodically       Application


  from multiple     Adaptation



                                 Middleware
                     Manager
     sources

                     Context                  3rd-Party
at varying rates     Manager                  Libraries

                    Environment
Approach
1. Derive Adaptation Finite-State Machine (A-
   FSM) from rule logic
2. Explore state space of A-FSM to discover
   potential faults
  ✓ Enumerative algorithms
  ✓ Symbolic algorithms
  ✓ Planner-based counterexample generation
3. (Confirm existence of discovered faults)
PhoneAdapter
PhoneAdapter
      silent, vibrate
                                       loud,
       normal,                       divert to
       vibrate                      hands-free




    silent, divert to voicemail   loud, vibrate
PhoneAdapter
      silent, vibrate
                                       loud,
       normal,                       divert to
       vibrate                      hands-free




    silent, divert to voicemail   loud, vibrate
PhoneAdapter A-FSM
 Jogging    Sync     Meeting




 Outdoor   General   Office




                     Driving
  Home     Driving
                      Fast
PhoneAdapter A-FSM
                     ActivateMeeting             DeactivateMeeting

 Jogging    Sync                       Meeting




 Outdoor   General                     Office




                                       Driving
  Home     Driving
                                        Fast
Example Faults in
 PhoneAdapter

       General   Office




Home
Example Faults in
          PhoneAdapter

                         General          Office




         Home




User’s phone discovers office PC at home (or vice versa)
Example Faults in
 PhoneAdapter

            General      Office




Home




       Nondeterminism!
Example Faults in
 PhoneAdapter

       General
Example Faults in
  PhoneAdapter

Outdoor        General




          User leaves home
Example Faults in
             PhoneAdapter

           Outdoor          General




                             Driving




User starts driving before Bluetooth detects hands-free system
Example Faults in
  PhoneAdapter
Jogging




Outdoor         General




                 Driving




          Activation hazard!
Example Faults in
  PhoneAdapter
Jogging




Outdoor         General




                 Driving




          Activation hazard!
Faults in CAAAs
• Behavioral Faults
    Nondeterminism    Unreachable state
    Dead rule         Activation race
    Dead state        Activation cycle
Faults in CAAAs
• Behavioral Faults
    Nondeterminism      Unreachable state
    Dead rule           Activation race
    Dead state          Activation cycle

• Hazards
    Hold hazard         Priority inversion
    Activation hazard   hazard
Why Not Use
       Model Checkers?
-   Difficult to encode fault patterns as
    temporal logic formulae
    ✴ Bisimilar models may fail differently

-   Difficult to encode rule logic as models in
    common model checkers
    ✴ Predicates and actions label the transitions

-   Difficult to interpret counterexamples as
    faults in adaptation behavior
Algorithms
               Basic Operation
    Jogging          Sync           Meeting




   Outdoor          General         Office




                                    Driving
    Home            Driving
                                     Fast



For each state
  Analyze rules & inputs, and search along
  transitions for instances of fault patterns
Algorithms
               Basic Operation
    Jogging          Sync           Meeting




   Outdoor          General         Office




                                    Driving
    Home            Driving
                                     Fast



For each state
  Analyze rules & inputs, and search along
  transitions for instances of fault patterns
Algorithms
               Basic Operation
    Jogging          Sync           Meeting




   Outdoor          General         Office




                                    Driving
    Home            Driving
                                     Fast



For each state
  Analyze rules & inputs, and search along
  transitions for instances of fault patterns
Algorithms
               Basic Operation
    Jogging          Sync           Meeting




   Outdoor          General         Office




                                    Driving
    Home            Driving
                                     Fast



For each state
  Analyze rules & inputs, and search along
  transitions for instances of fault patterns
PhoneAdapter Results          TABLE 2

           BehavioralConfigurations Reported for PhoneAdapter
            Faulty Input Faults: Enumerative, Symbolic


   State        Nondeterministic       Dead        Adaptation       Unreachable
                  Adaptations        Predicates   Races   Cycles      States
General                         37            1      45      13                0
Outdoor                         3             0     135      23                0
Jogging                         0             0      97      19                0
Driving                         0             0      36      13                0
DrivingFast                     0             0      58      19                0
Home                            0             0      76      19                0
Office                           0             0      29         1              0
Meeting                         0             0      32         1              0
Sync                            0             0      27         5              1
PhoneAdapter Results
                       Hazards: Enumerative
n PhoneAdapter                                     Table 2: Faults
aptation Races and Cycles
                 State      Vars. Context Hazards
                                   Nondet. Adaptation       Dead Pred
signments   Race    Cycle   Paths Assignments
                                   Hold   Activ. Faults
                                                   Prior. Assignments
     3968     General 13
              45            14085
                               7      0    12811    3182
                                                    37            128
     3968    135
              Outdoor 23      161
                               5      0     32 0      352          17
     3072     Jogging 19
              97               22     0      4 0      00            1
     2560     Driving 13
              36               316    2      8 2      04            7
     3072     DrivingFast
              58       19      22     0      4 0      00            2
     2816     Home
              76       19     104
                               4      8     16 0      013           9
     2848     O ce
              29        1   82634
                               7   1828    128
                                             368    2164
                                                      1            65
     2048     Meeting 1
              32               10     0      2 0      00            2
     1024     Sync
              27        5      22     2      4 0      00            1



 ned a formal model of a key Contextbehavioral char-
           6.4 Detecting complex Hazards
eristic, namely adaptation, of an increasingly sequences of asynchr
             This class of faults corresponds to large and
Conclusion
                 Comparison of Approaches

 Enumerative       Symbolic         Hybrid          Planner

  Local Search    Local Search    Local Search   Global Search

  Less Precise    Less Precise    Less Precise   More Precise
    Concrete        Symbolic        Symbolic        Concrete
Counterexamples Counterexamples Counterexamples Counterexamples
Handles Smaller   Handles Big    Handles Bigger    Sequential
  State Spaces    State Spaces    State Spaces       Search
      Fast           Faster         Fastest         Slowest
Future Work
                   Verification



• Continue the work on hazards and
   planners
•  Quantitative reasoning about faults
  • Battery level, movement timings, etc.
• Online analysis of rules and faults
Future Work
                  Design


• Alternatives to rule-based adaptation!
 • Machine learning approaches to context
    classification and adaptation selection
Future Work
                  Design


• Alternatives to rule-based adaptation!
 • Machine learning approaches to context
    classification and adaptation selection


• Felicitous Computing Institute
Thank You!
REFERENCES

Z. Wang, S. Elbaum and D.S. Rosenblum, Automated Generation of Context-Aware Tests, Proc. 2007 Int’l Conf. on
    Software Engineering (ICSE 2007), Minneapolis, MN, USA, May 2007, pp. 406–415.

M. Sama, D.S. Rosenblum, Z. Wang and S. Elbaum, Multi-Layer Faults in the Architectures of Mobile, Context-Aware Adaptive
    Applications: A Position Paper, Short Paper, Proc. ICSE 2008 Workshop on Software Architectures
    and Mobility (SAM 2008), Leipzig, Germany, May 2008, pp. 47–49.

M. Sama, F. Raimondi, D. Rosenblum and W. Emmerich, Algorithms for Efficient Symbolic Detection of Faults in Context-
    Aware Applications, Proc. 1st Int’l Workshop on Automated Engineering of Autonomous and
    Run-Time Evolving Systems (ARAMIS 2008), L’Aquila, Italy, Sep. 2008, pp. 1–8.

M. Sama, D.S. Rosenblum, Z. Wang and S. Elbaum, Model-Based Fault Detection in Context-Aware Adaptive Applications,
    Proc. 16th ACM SIGSOFT Int’l Symposium on the Foundations of Software Engineering
    (FSE 2008), Atlanta, GA, USA, Nov. 2008, pp. 261–271.

J. Cubo, F. Raimondi, M. Sama and D. Rosenblum, A Model to Design and Verify Context-Aware Adaptive Service
    Composition, Proc. IEEE Int’l Conf. on Services Computing (SCC 2009), Bangalore, India, Sep. 2009,
    pp. 184–191.

M. Sama, D.S. Rosenblum, Z. Wang and S. Elbaum, Multi-Layer Faults in the Architectures of Mobile, Context-Aware Adaptive
    Applications, Journal of Systems and Software, invited paper for Special Issue on Software Architecture
    and Mobility,Vol. 83, Issue 6, Jun. 2010, pp. 906–914.

M. Sama, S. Elbaum, F. Raimondi and D.S. Rosenblum, Context-Aware Adaptive Applications: Fault Patterns and Their
    Automated Identification, IEEE Transactions on Software Engineering, invited paper for Special Issue on
    the Best Papers of FSE 2008,Vol. 36, No. 5, Sep./Oct. 2010, pp. 644–661.

High-Confidence Ubiquitous Computing Systems (invited talk at ISHCS 2011)

  • 1.
    High-Confidence Ubiquitous Computing Systems David S. Rosenblum School of Computing National University of Singapore
  • 2.
    Some Facts Google Android Market • The average price of the top 50 paid applications is US$3.79 [modymi.com] • 79.3% of paid applications have been downloaded less than 100 times [Distimo] • Only 0.1% of paid applications have been downloaded 50,000 times or more [Distimo]
  • 3.
    Some Facts Google Android Market • The average price of the top 50 paid applications is US$3.79 [modymi.com] • 79.3% of paid applications have been downloaded less than 100 times [Distimo] • Only 0.1% of paid applications have been downloaded 50,000 times or more [Distimo] There are many simplistic, low-quality apps!
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
    Adaptation in CAAAs Application Environment
  • 10.
    Adaptation in CAAAs Application Physical Context Environment
  • 11.
    Adaptation in CAAAs Application Adaptation Middleware Manager Context Manager Physical Context Environment
  • 12.
    Adaptation in CAAAs Application Adaptation Middleware Manager Context Sensed Context Manager Physical Context Environment
  • 13.
    Adaptation in CAAAs Application Adaptation Middleware Inferred Context Manager Context Sensed Context Manager Physical Context Environment
  • 14.
    Adaptation in CAAAs PresumedContext Application Adaptation Middleware Inferred Context Manager Context Sensed Context Manager Physical Context Environment
  • 15.
    Adaptation in CAAAs PresumedContext Application Adaptation Middleware Inferred Context Manager Context 3rd-Party Sensed Context Manager Libraries Physical Context Environment
  • 16.
    Adaptation in CAAAs PresumedContext Application Adaptation Rule Middleware Inferred Context Manager Engine Context 3rd-Party Sensed Context Manager Libraries Physical Context Environment
  • 17.
    Validation of CAAAs Application Adaptation Rule Middleware Manager Engine Context Manager Environment
  • 18.
    Validation of CAAAs Rules are strongly interdependent Application and have multiple priorities Adaptation Rule Middleware Manager Engine making reasoning Context difficult even for a Manager small number of rules Environment
  • 19.
    Validation of CAAAs Application Adaptation Middleware Manager Context 3rd-Party Manager Libraries Environment
  • 20.
    Validation of CAAAs Contextis sensed periodically Application from multiple Adaptation Middleware Manager sources Context 3rd-Party at varying rates Manager Libraries Environment
  • 21.
    Approach 1. Derive AdaptationFinite-State Machine (A- FSM) from rule logic 2. Explore state space of A-FSM to discover potential faults ✓ Enumerative algorithms ✓ Symbolic algorithms ✓ Planner-based counterexample generation 3. (Confirm existence of discovered faults)
  • 22.
  • 23.
    PhoneAdapter silent, vibrate loud, normal, divert to vibrate hands-free silent, divert to voicemail loud, vibrate
  • 24.
    PhoneAdapter silent, vibrate loud, normal, divert to vibrate hands-free silent, divert to voicemail loud, vibrate
  • 25.
    PhoneAdapter A-FSM Jogging Sync Meeting Outdoor General Office Driving Home Driving Fast
  • 26.
    PhoneAdapter A-FSM ActivateMeeting DeactivateMeeting Jogging Sync Meeting Outdoor General Office Driving Home Driving Fast
  • 27.
    Example Faults in PhoneAdapter General Office Home
  • 28.
    Example Faults in PhoneAdapter General Office Home User’s phone discovers office PC at home (or vice versa)
  • 29.
    Example Faults in PhoneAdapter General Office Home Nondeterminism!
  • 30.
    Example Faults in PhoneAdapter General
  • 31.
    Example Faults in PhoneAdapter Outdoor General User leaves home
  • 32.
    Example Faults in PhoneAdapter Outdoor General Driving User starts driving before Bluetooth detects hands-free system
  • 33.
    Example Faults in PhoneAdapter Jogging Outdoor General Driving Activation hazard!
  • 34.
    Example Faults in PhoneAdapter Jogging Outdoor General Driving Activation hazard!
  • 35.
    Faults in CAAAs •Behavioral Faults Nondeterminism Unreachable state Dead rule Activation race Dead state Activation cycle
  • 36.
    Faults in CAAAs •Behavioral Faults Nondeterminism Unreachable state Dead rule Activation race Dead state Activation cycle • Hazards Hold hazard Priority inversion Activation hazard hazard
  • 37.
    Why Not Use Model Checkers? - Difficult to encode fault patterns as temporal logic formulae ✴ Bisimilar models may fail differently - Difficult to encode rule logic as models in common model checkers ✴ Predicates and actions label the transitions - Difficult to interpret counterexamples as faults in adaptation behavior
  • 38.
    Algorithms Basic Operation Jogging Sync Meeting Outdoor General Office Driving Home Driving Fast For each state Analyze rules & inputs, and search along transitions for instances of fault patterns
  • 39.
    Algorithms Basic Operation Jogging Sync Meeting Outdoor General Office Driving Home Driving Fast For each state Analyze rules & inputs, and search along transitions for instances of fault patterns
  • 40.
    Algorithms Basic Operation Jogging Sync Meeting Outdoor General Office Driving Home Driving Fast For each state Analyze rules & inputs, and search along transitions for instances of fault patterns
  • 41.
    Algorithms Basic Operation Jogging Sync Meeting Outdoor General Office Driving Home Driving Fast For each state Analyze rules & inputs, and search along transitions for instances of fault patterns
  • 42.
    PhoneAdapter Results TABLE 2 BehavioralConfigurations Reported for PhoneAdapter Faulty Input Faults: Enumerative, Symbolic State Nondeterministic Dead Adaptation Unreachable Adaptations Predicates Races Cycles States General 37 1 45 13 0 Outdoor 3 0 135 23 0 Jogging 0 0 97 19 0 Driving 0 0 36 13 0 DrivingFast 0 0 58 19 0 Home 0 0 76 19 0 Office 0 0 29 1 0 Meeting 0 0 32 1 0 Sync 0 0 27 5 1
  • 43.
    PhoneAdapter Results Hazards: Enumerative n PhoneAdapter Table 2: Faults aptation Races and Cycles State Vars. Context Hazards Nondet. Adaptation Dead Pred signments Race Cycle Paths Assignments Hold Activ. Faults Prior. Assignments 3968 General 13 45 14085 7 0 12811 3182 37 128 3968 135 Outdoor 23 161 5 0 32 0 352 17 3072 Jogging 19 97 22 0 4 0 00 1 2560 Driving 13 36 316 2 8 2 04 7 3072 DrivingFast 58 19 22 0 4 0 00 2 2816 Home 76 19 104 4 8 16 0 013 9 2848 O ce 29 1 82634 7 1828 128 368 2164 1 65 2048 Meeting 1 32 10 0 2 0 00 2 1024 Sync 27 5 22 2 4 0 00 1 ned a formal model of a key Contextbehavioral char- 6.4 Detecting complex Hazards eristic, namely adaptation, of an increasingly sequences of asynchr This class of faults corresponds to large and
  • 44.
    Conclusion Comparison of Approaches Enumerative Symbolic Hybrid Planner Local Search Local Search Local Search Global Search Less Precise Less Precise Less Precise More Precise Concrete Symbolic Symbolic Concrete Counterexamples Counterexamples Counterexamples Counterexamples Handles Smaller Handles Big Handles Bigger Sequential State Spaces State Spaces State Spaces Search Fast Faster Fastest Slowest
  • 45.
    Future Work Verification • Continue the work on hazards and planners • Quantitative reasoning about faults • Battery level, movement timings, etc. • Online analysis of rules and faults
  • 46.
    Future Work Design • Alternatives to rule-based adaptation! • Machine learning approaches to context classification and adaptation selection
  • 47.
    Future Work Design • Alternatives to rule-based adaptation! • Machine learning approaches to context classification and adaptation selection • Felicitous Computing Institute
  • 48.
    Thank You! REFERENCES Z. Wang,S. Elbaum and D.S. Rosenblum, Automated Generation of Context-Aware Tests, Proc. 2007 Int’l Conf. on Software Engineering (ICSE 2007), Minneapolis, MN, USA, May 2007, pp. 406–415. M. Sama, D.S. Rosenblum, Z. Wang and S. Elbaum, Multi-Layer Faults in the Architectures of Mobile, Context-Aware Adaptive Applications: A Position Paper, Short Paper, Proc. ICSE 2008 Workshop on Software Architectures and Mobility (SAM 2008), Leipzig, Germany, May 2008, pp. 47–49. M. Sama, F. Raimondi, D. Rosenblum and W. Emmerich, Algorithms for Efficient Symbolic Detection of Faults in Context- Aware Applications, Proc. 1st Int’l Workshop on Automated Engineering of Autonomous and Run-Time Evolving Systems (ARAMIS 2008), L’Aquila, Italy, Sep. 2008, pp. 1–8. M. Sama, D.S. Rosenblum, Z. Wang and S. Elbaum, Model-Based Fault Detection in Context-Aware Adaptive Applications, Proc. 16th ACM SIGSOFT Int’l Symposium on the Foundations of Software Engineering (FSE 2008), Atlanta, GA, USA, Nov. 2008, pp. 261–271. J. Cubo, F. Raimondi, M. Sama and D. Rosenblum, A Model to Design and Verify Context-Aware Adaptive Service Composition, Proc. IEEE Int’l Conf. on Services Computing (SCC 2009), Bangalore, India, Sep. 2009, pp. 184–191. M. Sama, D.S. Rosenblum, Z. Wang and S. Elbaum, Multi-Layer Faults in the Architectures of Mobile, Context-Aware Adaptive Applications, Journal of Systems and Software, invited paper for Special Issue on Software Architecture and Mobility,Vol. 83, Issue 6, Jun. 2010, pp. 906–914. M. Sama, S. Elbaum, F. Raimondi and D.S. Rosenblum, Context-Aware Adaptive Applications: Fault Patterns and Their Automated Identification, IEEE Transactions on Software Engineering, invited paper for Special Issue on the Best Papers of FSE 2008,Vol. 36, No. 5, Sep./Oct. 2010, pp. 644–661.

Editor's Notes