The document discusses the test case generation for software enforcers, particularly focusing on runtime enforcement actions for untrusted applications. It outlines a framework called test4enforcers that generates sequences of test cases to evaluate the behavior of enforcers, emphasizing issues like inaccurate enforcement models. Additionally, it showcases specific test sequences and provides insights into the effectiveness of enforcement in applications.

1. Test4Enforcers: Test Case
Generation for Software
Enforcers
Oliviero Riganelli
University of Milano Bicocca
joint work with
Michell Guzman, Daniela Micucci and Leonardo Mariani

2. Runtime Enforcement
Enforcer
Actions Valid
Actions
Android(API Guides) Sept 20, 2020
Call release() to release the camera for use
by other applications. Applications should
release the camera immediately in onPause()
Untrusted App System
Violation
Policy

3. Enforcement Model
Untrusted App
Enforcer
Untrusted App
Actions Valid
Actions
Untrusted App System
onPause()req?/
onPause()api!
onPause()req?/release()api!; onPause()api!
open()req?/open()api!
release()req?/release()api!
Android(API Guides) Sept 20, 2020
Call release() to release the camera for use
by other applications. Applications should
release the camera immediately in onPause()
Policy

4. From Model to Code
Untrusted App
Enforcement Model
Self-enforcing App
Code Instrumentation

5. Issues with Enforcers
Expected Behavior
Modeled Behavior
Inaccurate Enforcement Models

6. Inconsistent Enforcer Implementations
Implemented Behavior
Modeled Behavior
Issues with Enforcers
Expected Behavior
Modeled Behavior
Inaccurate Enforcement Models

7. Inconsistent Enforcer Implementations
Implemented Behavior
Modeled Behavior
Faulty Enforcer Code
Implemented Behavior
Issues with Enforcers
Expected Behavior
Modeled Behavior
Inaccurate Enforcement Models

8. Goal
Enforcer Model
Test CasesTest4Enforcers

9. Test4Enforcers
Enforcement
Model
Generation of
Test Sequences
Concrete
Test Case
Generation
Gui Ripping
with Monitoring
App
Test
Cases
Test4Enforcers Model
onPause(),
onPause(), onPause(),
open(), onPause()
open(), onPause(), onPause()
open(), release(), onPause()
Test Sequences

10. onPause()req?/
onPause()api!
onPause()req?/release()api!; onPause()api!
open()req?/open()api!
release()req?/release()api!
Generation of Test Sequences
1. onPause()req
2. onPause()req, onPause()req
3. open()req, onPause()req
4. open()req, onPause()req, onPause()req
5. open()req, release()req, onPause()req
HSI-Method*
*Petrenko et al.: Testing deterministic implementations from nondeterministic fsm specifications. In: Proceedings of the IFIP TC6 International Workshop on
Testing of Communicating Systems (1996)

11. GUI Ripping with Monitoring
Touch buttons
Type text
Press HOME key
…

12. GUI Ripping with Monitoring
Touch buttons
Type text
Press HOME key
…
Actions in the
enforcement model

13. Test4Enforcers Model
GrantPermissionsActivity
<FIRST>
GrantPermissionsActivity
TouchEvent(“3”)
MainActivity
MainActivity
Launcher
KeyEvent(HOME)
release();onPause()
open()
TouchEvent(“ALLOW”) TouchEvent(“ALLOW”)

14. GrantPermissionsActivity
TouchEvent(“3”)
MainActivity
MainActivity
Launcher
KeyEvent(HOME)
release();onPause()
open()
TouchEvent(“ALLOW”) TouchEvent(“ALLOW”)
GrantPermissionsActivity
TouchEvent(“ALLOW”)
TouchEvent(“3”)
MainActivity
KeyEvent(HOME)
release();onPause()
open()
Test4Enforcers Model
KeyEvent(HOME)
STATE
GUI:
GrantPermissionsActivity
<FIRST>

15. GrantPermissionsActivity
<FIRST>
GrantPermissionsActivity
TouchEvent(“3”)
MainActivity
MainActivity
Launcher
KeyEvent(HOME)
release();onPause()
open()
TouchEvent(“ALLOW”) TouchEvent(“ALLOW”)
GrantPermissionsActivity
<FIRST>
GrantPermissionsActivity
TouchEvent(“ALLOW”)
GrantPermissionsActivity
<FIRST>
GrantPermissionsActivity
TouchEvent(“ALLOW”)
Test4Enforcers Model
Launcher
KeyEvent(HOME)
GrantPermissionsActivity
<FIRST>
EVENT
Type: Touch
View:
Command: adb shell input …
Trace: release();onPause()

16. Concrete Test Case Generation
1. TouchEvent(“ALLOW”), TouchEvent(“ALLOW”), KeyEvent(HOME)
Test Case
1. open()req, release()req, onPause()req
Test Sequence

17. Concrete Test Case Generation
1. open()req, release()req, onPause()req
Test Sequence
GrantPermissionsActivity
<FIRST>
GrantPermissionsActivity
TouchEvent(“3”)
MainActivity
MainActivity
Launcher
KeyEvent(HOME)
release();onPause()
open()
TouchEvent(“ALLOW”) TouchEvent(“ALLOW”)

18. Concrete Test Case Generation
1. open()req, release()req, onPause()req
1. TouchEvent(“ALLOW”), TouchEvent(“ALLOW”), KeyEvent(HOME)
Test Sequence
Test Case
GrantPermissionsActivity
<FIRST>
GrantPermissionsActivity
TouchEvent(“3”)
MainActivity
MainActivity
Launcher
KeyEvent(HOME)
release();onPause()
open()
TouchEvent(“ALLOW”) TouchEvent(“ALLOW”)

19. Test Case Oracles
Transparent
Enforcement
Actual
Enforcement
App with enforcer App without enforcer
fooCam has stopped
Open app again

20. Proof of Concept
Configuration
Google Pixel 2
Android 8.0
Metrics
# of covered test sequencesSize of models
APPs
Correct fooCam app
https://play.google.com/store/apps/details?id=net.phunehehe.foocam2&hl=EN
Enforcer
Faulty fooCam app

21. Size of Test4Enforcers models
0
75
150
225
300
States Transitions
Faulty fooCam Correct fooCam

22. # of covered test sequences
APP
Test Sequence Coverage
Faulty fooCam
onPause()req INFEASIBLE
onPause()req, onPause()req INFEASIBLE
open()req, onPause()req INFEASIBLE
open()req, onPause()req, onPause()req INFEASIBLE
open()req, release()req, onPause()req COVERED
Correct fooCam
onPause()req INFEASIBLE
onPause()req, onPause()req INFEASIBLE
open()req, onPause()req COVERED
open()req, onPause()req, onPause()req INFEASIBLE
open()req, release()req, onPause()req INFEASIBLE

23. Conclusions