A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications
•Download as PPTX, PDF•
0 likes•760 views
Recommended
More Related Content
Similar to A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications
Similar to A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications (20)
A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications
- 1. A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications Wei Yang Mukul R. Prasad Tao Xie Department of Computer Science, Software Systems Innovation group Department of Computer Science, North Carolina State University, Fujitsu Labs. Of America North Carolina State University, Raleigh, North Carolina Sunnyvale, CA Raleigh, North Carolina
- 2. Growing popularity of Smartphone 2
- 3. Growing demand for high quality apps Solution? Automated Testing Tools! 3
- 4. GUI model • GUI model is essential for developing automated testing tools – What does it represent? – Why is such a model useful/important for testing? 4
- 5. Sample App: Simple TippyTipper Used to calculate tip amount for a meal MENU MENU 5
- 6. GUI Model for Simple TippyTipper Calculate 1 2 1 2 4 Menu Menu 3 About 4 Settings 5a 5b a1 10 11 a2 a2 01 a1 5c 5d 00 3 3 5 a1: Toggle exclude tax rate option. a2: Toggle round up option. 6
- 7. Applications of GUI Model • Classical Model-based testing applications – Automatic (system-level) test-case generation – Use as a driver for symbolic execution (SE) based testing • More recent “crawl-and-compare” applications – Cross-platform compatibility testing of an application – Regression testing across different versions of the application 7
- 8. Related Work • GUITAR: GUI Ripping for Windows, Java-based GUI (desktop) apps, Atif et al. A simplified event-flow model of simple tippytipper. Our GUI model of simple tippytipper. Calculate Click 3 Click 1 1 2 Click 2 Menu Menu Click 5 Click 4 3 About 4 Settings Click 6 5a 5b Click 7 a1 11 10 Click 8 a2 a2 Click 9 5c 01 a1 5d 00 Click Click 0 Click CLEAR DEL a1: Toggle exclude tax rate option. a2: Toggle round up option. Click Calculate 8
- 9. Related Work(Cont.) • Concolic Testing (SE) of Android Applications, Saswat et al. • Random Testing of Android Applications, Cuixiong et al. • Crawljax: Crawling of AJAX Web Applications, Ali et al. • AndroidRipper: Stress Testing of Android Applications, Domenico et al 9
- 10. ORBIT GUI Crawler: Overview GUI Model Android Automated Apps Analysis (including source code) 10
- 11. ORBIT GUI Crawler: Overview ORBIT GUI Model Action Android Action inference Mapping Dynamic Crawler AUT (source code) 11
- 12. Proposed GUI Model • Visual Observable State – Inspired by the UI-design principles espoused by the Android team. – Composition of the state • Model – A finite-state machine over visual observable states with the user actions constituting the transitions between these states 12
- 13. GUI Model for Simple TippyTipper Calculate 1 2 1 2 4 Menu Menu 3 About 4 Settings 5a 5b a1 10 11 a2 a2 01 a1 5c 5d 00 3 3 5 a1: Toggle exclude tax rate option. a2: Toggle round up option. 13
- 14. Illustration: State These two states differ in the state of this widget Two different states of Screen # 5b & 5d 5b 5d 14
- 15. Action Inference • Approach: – Locate all instances of event handler definitions in the code (sinks) – Associate each with the corresponding widget instantiations (sources) TippyTipper.java View btn_delete = findViewById(R.id.btn_delete); R.Id.java … Btn_delete.setOnClickListener(new onClickListener() { public void onClick(View v) { removeBillAmount(); FlurryAgent.onEvent(“Delete Button”); } }); … Btn_delete.setOnClickListener(new onLongClickListener() { public void onLongClick(View v) { Inference: Widget btn_delete with Id = 0x7f0000a clearBillAmount(); supports actions click and longClick return true; } }); 15
- 16. Simple TippyTipper: Actions Found • ScreenActionMapping – (net.mandaria.tippytipper.activities.TippyTipper, Menu) – (net.mandaria.tippytipper.activities.Total, Menu) • WidgetActionMapping – (R.id.btn_delete, click) – (R.id.btn_delete, longclick) – (R.id.btn_clear, click) – (R.id.btn_clear, longclick) 16
- 17. Dynamic Crawling: Illustration Activity Setting Activity TippyTipper Activity Total Toggle BACK Toggle Calculate Toggle Toggle BACK MENU BACK MENU Toggle Toggle Toggle SETTINGS Toggle 17
- 18. Tool implementation Action Detector WALA Partial Connected Call Graph source code Sub-CallGraph Inference Algorithm GUI Model Intent Passing Logic Action Mapping Android AUT deploy FwdCrawl Algorithm Robotium Android Runtime Dynamic Crawler ORBIT 18
- 19. Research question • RQ1: Is the proposed GUI crawling algorithm more efficient than a standard depth-first state-traversal algorithm? • RQ2: Are the widget and screen actions inferred by static analysis effective in enhancing the behavior covered by the generated model? • RQ3: Can our tool generate a higher-quality model, more efficiently, compared to other state-of-the-art techniques? 19
- 20. Subjects used in the evaluation Subject #LOC #A Category Purpose TippyTipper 2238 5 Tool Dining tip calculator OpenManager 1595 6 Business File manager for Android Notepad 322 3 Productivity Note creation and management TomDroid 3711 3 Business Online note-reading Aarddict 4518 4 Books Aard Dictionary for Android HelloAUT 234 1 Entertainment Color shapes ContactManager 497 2 Productivity Contacts manager ToDoManager 323 2 Productivity Create and manage task lists • Our evaluation is based on subjects drawn from existing related tools • We avoid bias by including all subjects used to evaluate Android GUITAR and GUI Ripper in previous work. 20
- 21. Crawling algorithm comparison Subjects FwdCrawl DFS Time Cov. #Fwd Back Time Cov. #Fwd Back (sec) (%) (sec) (%) TippyTipper 198 78 61 15 512 82 134 52 OpenManager 480 63 92 18 822 56 209 29 Notepad2 102 82 25 4 147 83 39 12 Notepad0 80 78 18 2 75 71 15 2 TomDroid 340 70 78 23 459 58 61 8 AardDict 173 65 15 2 397 60 20 8 HelloAUT 156 86 46 0 278 85 61 0 ContactManager 125 91 20 1 137 92 22 2 ToDoManager 178 75 60 2 294 74 84 4 • DFS takes 70% more time to traverse all 9 subjects together 21
- 22. Validity of action inference Subjects #clicks #longClicks #menu #States Coverage(%) C C+I C C+I C C+I C C+I C C+I TippyTipper 21 55 - 2 - 4 3 9 47 78 OpenManager 50 67 - 19 - 4 10 20 39 63 Notepad2 2 13 - 3 - 9 2 7 39 82 Notepad0 0 8 - 1 - 9 0 7 14 78 TomDroid 3 52 - 0 - 26 2 9 36 70 AardDict 4 15 - 0 - 7 3 7 43 64 HelloAUT 15 34 - 0 - 12 4 8 53 86 ContactManager 20 20 - 0 - 0 5 5 92 92 ToDoManager 60 60 - 0 - 0 7 7 76 76 • Firing non-click actions during crawling increases the coverage by 34% on average. 22
- 23. Comparison with other tools Subjects Monkey Android Android GUI ORBIT GUITAR Ripper Time Cov. Time Cov. Time Cov. Time Cov.(%) (sec) (%) (sec) (%) (sec) (%) (sec) TippyTipper 83 41 322 47 - - 198 78 OpenManager 90 29 - - - - 480 63 Notepad2 127 60 - - - - 102 82 Notepad0 122 59 - - - - 80 78 TomDroid 69 46 - - 529 40 340 70 AardDict 124 51 - - 694 27 173 65 HelloAUT 98 71 117 51 - - 156 86 ContactManager 90 53 247 61 - - 125 91 ToDoManager 115 71 194 71 - - 178 75 • Our crawler is 32%-75% faster while constructing a 5%-140% more complete model than Android GUITAR and Android GUI Ripper. 23
- 24. Conclusions & Future Work • Crawling algorithm – Incorporate the context of event call to prune the order of the sequence. • GUI model – Explore the possibility to generate a complete specification or test oracle by using our driver and partial specification. • Selection of subjects – Inter-application communication of Android application. • Manual effort 24
- 25. Thanks! *This project was supported in part by NSF grants CCF-0845272, CCF-0915400, CNF-0958235, CNS-1160603, and an NSA Science of Security Lablet Grant.
Editor's Notes
- Developers are required to develop high quality apps in order to be competitiveMobile apps are usually developed in relatively small-scale projects, which may not be able to support extensive and expensive manual testing
- Classical Model-based testing applicationsAutomatic (system-level) test-case generationEach path in the model is a potential test-caseUse as a driver for symbolic execution (SE) based testingThe model provides the event sequences while explores the space of data-valuesMore recent “crawl-and-compare” applicationsCross-platform compatibility testing of an applicationDifferent mobile platforms or versions, desktop vs. mobileRegression testing across different versions of the applicationOther potential applications (future research)To complement more rigorous (computationally expensive) approaches such as SE for testingProvided model extraction can be done cheaply (efficiently)
- GUITAR: GUI Ripping for Windows, Java-based GUI (desktop) apps.AtifMemon et al. U. MarylandCompletely event and component based modelNot suitable for capturing stateful behavior of Android apps.Completely black-box & dynamicCannot reverse engineer complex events other than simple tapsCrawljax: Crawling of AJAX Web ApplicationsAli Mesbah et al., T U Delft, Univ. of British ColumbiaFlat, state-based modelDoes not represent the structure of Android application: Very useful in testing & debuggingNOTE:In web-apps such information is not easily available/representableCompletely black-box and dynamic: Similar limitation as GUITARConcolic Testing (SE) of Android ApplicationsSaswatAnand et al., Georgia Tech., Stanford Univ.Uses Symb. Exec. to reverse engineer GUI action sequencesVery (compute) intensive and limited by length of sequencePath explosion since each sequence represented separately: No model !
- GUI Model: What should it represent?All/Many use-cases of the application available to the user through the GUIScreen viewable by the userAction (on widgets) sequences exercisable by the user (through the GUI)Why is such a model useful/important (for Android applications) ?Mobile applications strongly GUI-driven/GUI-centricNot designed to be exercised without the GUITypically GUI encodes all permissible use-cases of the applicationLooking at the GUI alone is mostly sufficient to extract these use-casesMostly not necessary to look at/analyze application implementationJust executing it (through GUI) sufficient !
- Visual Observable StateInspired by the UI-design principles espoused by the Android team. The Android User Experience Team suggests that developers should “make places in the app look distinct” to give users confidence that they know their way around the app.Composition of the stateThe hierarchy tree of the UI screen, as well as a vector of attribute values of each of the executable components.The structure of a GUI screen in Android is represented by a tree of different GUI components, called a hierarchy treeExecutable components are those GUI components support user actions (which are detected by our static analysis)ModelA finite-state machine over visual observable states with the user actions constituting the transitions between these states
- A state corresponds to a specific set of widgets (i.e. a screen) of a specific component, with each widget in a specific statei.e., a state is identified by the states of its constituent widgetsThe state of a widget is denoted by the value of its attributes such as: enabled (true/false), visible(true/false), checked(true/false)
- We make the observation that in the Android framework a user action is defined by either registering an appropriate event listener for it or, by inheriting the event-handling method of an Android-framework component. We term the former as registered action and the latter inherited action. We use the registered action as a demonstration here.For both these categories, identifying an action involves three basic steps: identify the place where an action is instantiated or registered; locate the component on which the action would be fired; extract an identifier of the component that the crawler can later use to recognize the corresponding object and fire the action.
- ScreenActionMapping(net.mandaria.tippytipper.activities.TippyTipper, Menu)(net.mandaria.tippytipper.activities.Total, Menu)WidgetActionMapping(R.id.btn_delete, click)(R.id.btn_delete, longclick)(R.id.btn_clear, click)(R.id.btn_clear, longclick)
- Action DetectorWALAstatic-analysis frameworkintent-passing logicCreated based on our knowledge of the Android SDK.Builds a mapping of intent-sending methods and intent filters by analyzing the app’s source code and manifest file.Connects the sub call graphs into a partial connected call graph.Dynamic CrawlerDynamically-created GUI componentsTypically appear in Android containers like ListView, as a list of dynamically-created child components.System-generated GUI componentsTypically have system-defined IDs and predefined actions.
- Crawling algorithmThe randomness in the choice of the next action to exploreSide effects of executionSelection of subjectsWe try to avoid bias by including all subjects used to evaluate Android GUITAR and GUI Ripper in previous workManual effortThe only manual work in our approach is to manually select attributes of executable components to compose the visual observable states for the GUI. This effort is a one-time effort for a mobile platform.