Automatically Discovering, Reporting and Reproducing Android Application CrashesKevin Moran
Mobile developers face unique challenges when detecting and reporting crashes in apps due to their prevailing GUI event-driven nature and additional sources of inputs (e.g., sensor readings). To support developers in these tasks, we introduce a novel, automated approach called CRASHSCOPE. This tool explores a given Android app using systematic input generation, according to several strategies informed by static and dynamic analyses, with the intrinsic goal of triggering crashes. When a crash is detected, CRASHSCOPE generates an augmented crash report containing screenshots, detailed crash reproduction steps, the captured exception stack trace, and a fully replayable script that automatically reproduces the crash on a target device(s).
We evaluated CRASHSCOPE’s effectiveness in discovering crashes as compared to five state-of-the-art Android input generation tools on 61 applications. The results demonstrate that CRASHSCOPE performs about as well as current tools for detecting crashes and provides more detailed fault information. Additionally, in a study analyzing eight real-world Android app crashes, we found that CRASHSCOPE’s reports are easily readable and allow for reliable reproduction of crashes by presenting more explicit information than human written reports.
“Writing code that lasts” … or writing code you won’t hate tomorrow.Rafael Dohms
As developers we write code everyday, only to frown at it a week after that. Why do we have such a hard time with code written by others and ourselves, this raging desire to rewrite everything we see? Writing code that survives the test of time and self judgment is a matter of clarity and simplicity. Let's talk about growing, learning and improving our code with calisthenics, readability and good design.
This is a talk I gave at Software Testing Atlanta Conference (STAC 2015).
ABSTRACT::
Learn about developing and maintaining automation tests for web and mobile applications. In this two part presentation, Shauvik will cover "Move Fast" - how to efficiently generate automation test scripts for manual test cases, and "Fix Tests" - strategies for maintaining tests between regressions or platforms.
This is the slide deck that my team used at the ACC School's Startup Madness Competition held at NCSU in Raleigh, NC. CheckDroid won the 1st place out of 16 ACC school teams.
Software robots like monkey provide a quick way to validate your application. With robots on new cloud testing services, it is easier than ever to get started testing your app without even having written any tests. In this talk, I will introduce a few tools from both academia and industry, and then cover the basics of how these tools work. You will learn about the strengths and limitations these tools and how to use them effectively to maximize code coverage and catching failures.
For the success of our apps, we often add end user documentation in the form of help notes, tutorials and app demos/tours. Such end user docs are often outdated leading to confused users who make noise on support channels and worse on the play store. Tests also have the same problem — they need to be updated after the app changes to keep them in sync. However, tests produce pass/fail reports to indicate out of sync problems. So, why not kill two birds with one stone? I’ll show how end-to-end UI tests can generate beautiful user docs for happy cases and make sure they're never out of date.
Automatically Discovering, Reporting and Reproducing Android Application CrashesKevin Moran
Mobile developers face unique challenges when detecting and reporting crashes in apps due to their prevailing GUI event-driven nature and additional sources of inputs (e.g., sensor readings). To support developers in these tasks, we introduce a novel, automated approach called CRASHSCOPE. This tool explores a given Android app using systematic input generation, according to several strategies informed by static and dynamic analyses, with the intrinsic goal of triggering crashes. When a crash is detected, CRASHSCOPE generates an augmented crash report containing screenshots, detailed crash reproduction steps, the captured exception stack trace, and a fully replayable script that automatically reproduces the crash on a target device(s).
We evaluated CRASHSCOPE’s effectiveness in discovering crashes as compared to five state-of-the-art Android input generation tools on 61 applications. The results demonstrate that CRASHSCOPE performs about as well as current tools for detecting crashes and provides more detailed fault information. Additionally, in a study analyzing eight real-world Android app crashes, we found that CRASHSCOPE’s reports are easily readable and allow for reliable reproduction of crashes by presenting more explicit information than human written reports.
“Writing code that lasts” … or writing code you won’t hate tomorrow.Rafael Dohms
As developers we write code everyday, only to frown at it a week after that. Why do we have such a hard time with code written by others and ourselves, this raging desire to rewrite everything we see? Writing code that survives the test of time and self judgment is a matter of clarity and simplicity. Let's talk about growing, learning and improving our code with calisthenics, readability and good design.
This is a talk I gave at Software Testing Atlanta Conference (STAC 2015).
ABSTRACT::
Learn about developing and maintaining automation tests for web and mobile applications. In this two part presentation, Shauvik will cover "Move Fast" - how to efficiently generate automation test scripts for manual test cases, and "Fix Tests" - strategies for maintaining tests between regressions or platforms.
This is the slide deck that my team used at the ACC School's Startup Madness Competition held at NCSU in Raleigh, NC. CheckDroid won the 1st place out of 16 ACC school teams.
Software robots like monkey provide a quick way to validate your application. With robots on new cloud testing services, it is easier than ever to get started testing your app without even having written any tests. In this talk, I will introduce a few tools from both academia and industry, and then cover the basics of how these tools work. You will learn about the strengths and limitations these tools and how to use them effectively to maximize code coverage and catching failures.
For the success of our apps, we often add end user documentation in the form of help notes, tutorials and app demos/tours. Such end user docs are often outdated leading to confused users who make noise on support channels and worse on the play store. Tests also have the same problem — they need to be updated after the app changes to keep them in sync. However, tests produce pass/fail reports to indicate out of sync problems. So, why not kill two birds with one stone? I’ll show how end-to-end UI tests can generate beautiful user docs for happy cases and make sure they're never out of date.
Front-End Test Fest Keynote: The State of the Union for Front End Testing.pdfApplitools
In this Front-End Test Fest Keynote by Developer Advocate Andrew Knight, learn more about the seven major trends in front end testing along with the best approaches.
Dipping Your Toes Into Cloud Native Application DevelopmentMatthew Farina
Presented at CloudDevelop 2016
Building cloud native applications in containers is a new hot topic. Netflix and Google are two prime examples that have been doing it successfully for some time. Some of the new exciting projects like Docker and Kubernetes are focused on cloud native applications in containers. There are supposed to be numerous benefits including the ability to scale applications out easily while doing development on small systems like laptops, the ability for the system to handle some operational problems, and the capability to safely deploy updates to production many times per day. But, what does this look like in practice and how do you start the move to cloud native and containerized applications? In this session we'll look at what makes up a cloud native application, how they work, and how you can start small. We'll look at applications from an architecture and process point of view along with how you can deploy them to AWS, Azure, or Google Cloud. You'll walk away ready to start development on a cloud native app.
ICSE17 - Tool Demonstration - CrashScope A Practical Tool for the Automated T...Kevin Moran
This presentation covers the CrashScope tool which uses a combination of static and dynamic analysis to practically and effectively test Android applications. For more information please visit https://www.android-dev-tools.com/crashscope-home/
JavaScript: The Machine Language of the Ambient Computing EraAllen Wirfs-Brock
Keynote Presentation at Front-Trends 2013
Warsaw, Poland
April 5, 2013
We are well into the transition away from PC centric computing. But the new order isn't just about mobile phones and tables. It's about all of us living in a world that is augments by the ambient presence of billions of computing devices. And they're all going to be programming using JavaScript! Well, at least a lot of them will be. In this presentation I'll be discussing how JavaScript is changing the world, and how the world is changing JavaScript..
Exposed! A case study on the vulnerability-proneness of Google Play AppsSebastiano Panichella
Title: Exposed! A case study on the vulnerability-proneness
of Google Play Apps
Authors:
Andrea Di Sorbo, Sebastiano Panichella
Venue:
ESEC/FSE - Journal First Presentation
14-18, November 2022, Singapore
Video:
https://www.youtube.com/watch?v=9lv3WGuNM0A&ab_channel=Sebastiano
Finding Concurrency Errors in Event-Driven Applications - Strangeloop'14Pavol Bielik
We present a new open-source analysis system called EventRacer (www.eventracer.org) developed at the Software Reliability Lab at ETH Zurich, that finds harmful concurrency errors in event-driven applications such as web pages and Android applications. We illustrate how these concurrency errors can occur in real-world applications as well as the key analysis mechanisms behind EventRacer.
Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mob...Kevin Moran
Mobile app development involves a unique set of challenges including device fragmentation and rapidly evolving platforms, making testing a difficult task. The design space for a comprehensive mobile testing strategy includes features, inputs, potential contextual app states, and large combinations of devices and underlying platforms. Therefore, automated testing is an essential activity of the development process. However, current state of the art of automated testing tools for mobile apps posses limitations that has driven a preference for manual testing in practice. As of today, there is no comprehensive automated solution for mobile testing that overcomes fundamental issues such as automated oracles, history awareness in test cases, or automated evolution of test cases.
In this perspective paper we survey the current state of the art in terms of the frameworks, tools, and services available to developers to aid in mobile testing, highlighting present shortcom- ings. Next, we provide commentary on current key challenges that restrict the possibility of a comprehensive, effective, and practical automated testing solution. Finally, we offer our vision of a comprehensive mobile app testing framework, complete with research agenda, that is succinctly summarized along three principles: Continuous, Evolutionary and Large-scale (CEL).
This BJUG presentation is focused on the unique inter-process communication (IPC) mechanism introduced by Android, namely Binder. As such, we will be exploring the entire Android software stack, starting from the Linux kernel, moving up to the Linux userspace, reaching the Android framework and ending our journey with some real-life usages of Binder in Android applications.
We will show how widely used Binder actually is in Android, thus proving that it is a cornerstone of the operating system. The main focus is on the programming model exposed towards developers and we will highlight when, how and why it should be used.
https://bjug.ro/binding-android-piece-by-piece/
Testing on Mobile Devices with Location ServicesSauce Labs
During this webinar we look into location services on Android and iOS real devices, as well as emulators and simulators.
We provide you with tips and tricks on when to change the mobile Geo-IP and when to change the mobile GPS, how to use ADB commands to enable the device location, and how to handle the location services permission alerts. We also discuss Appium capabilities for Location Services, using GeoLocation with Appium Desktop, and more.
Key takeaways:
- Understand what Location Services are
- Understand GPS compare to Geo-IP
- Know the Appium commands for Location Services
- The differences in the use of Location Services between Android and iOS
- Learn how to automate tests that involve Location Services
Training in Bangalore is one of the best Selenium Testing Training Institute in bangalore. We offer Online Training and Classroom Training. Our Trainers are very professional and more than 8 years of experience.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Front-End Test Fest Keynote: The State of the Union for Front End Testing.pdfApplitools
In this Front-End Test Fest Keynote by Developer Advocate Andrew Knight, learn more about the seven major trends in front end testing along with the best approaches.
Dipping Your Toes Into Cloud Native Application DevelopmentMatthew Farina
Presented at CloudDevelop 2016
Building cloud native applications in containers is a new hot topic. Netflix and Google are two prime examples that have been doing it successfully for some time. Some of the new exciting projects like Docker and Kubernetes are focused on cloud native applications in containers. There are supposed to be numerous benefits including the ability to scale applications out easily while doing development on small systems like laptops, the ability for the system to handle some operational problems, and the capability to safely deploy updates to production many times per day. But, what does this look like in practice and how do you start the move to cloud native and containerized applications? In this session we'll look at what makes up a cloud native application, how they work, and how you can start small. We'll look at applications from an architecture and process point of view along with how you can deploy them to AWS, Azure, or Google Cloud. You'll walk away ready to start development on a cloud native app.
ICSE17 - Tool Demonstration - CrashScope A Practical Tool for the Automated T...Kevin Moran
This presentation covers the CrashScope tool which uses a combination of static and dynamic analysis to practically and effectively test Android applications. For more information please visit https://www.android-dev-tools.com/crashscope-home/
JavaScript: The Machine Language of the Ambient Computing EraAllen Wirfs-Brock
Keynote Presentation at Front-Trends 2013
Warsaw, Poland
April 5, 2013
We are well into the transition away from PC centric computing. But the new order isn't just about mobile phones and tables. It's about all of us living in a world that is augments by the ambient presence of billions of computing devices. And they're all going to be programming using JavaScript! Well, at least a lot of them will be. In this presentation I'll be discussing how JavaScript is changing the world, and how the world is changing JavaScript..
Exposed! A case study on the vulnerability-proneness of Google Play AppsSebastiano Panichella
Title: Exposed! A case study on the vulnerability-proneness
of Google Play Apps
Authors:
Andrea Di Sorbo, Sebastiano Panichella
Venue:
ESEC/FSE - Journal First Presentation
14-18, November 2022, Singapore
Video:
https://www.youtube.com/watch?v=9lv3WGuNM0A&ab_channel=Sebastiano
Finding Concurrency Errors in Event-Driven Applications - Strangeloop'14Pavol Bielik
We present a new open-source analysis system called EventRacer (www.eventracer.org) developed at the Software Reliability Lab at ETH Zurich, that finds harmful concurrency errors in event-driven applications such as web pages and Android applications. We illustrate how these concurrency errors can occur in real-world applications as well as the key analysis mechanisms behind EventRacer.
Continuous, Evolutionary and Large-Scale: A New Perspective for Automated Mob...Kevin Moran
Mobile app development involves a unique set of challenges including device fragmentation and rapidly evolving platforms, making testing a difficult task. The design space for a comprehensive mobile testing strategy includes features, inputs, potential contextual app states, and large combinations of devices and underlying platforms. Therefore, automated testing is an essential activity of the development process. However, current state of the art of automated testing tools for mobile apps posses limitations that has driven a preference for manual testing in practice. As of today, there is no comprehensive automated solution for mobile testing that overcomes fundamental issues such as automated oracles, history awareness in test cases, or automated evolution of test cases.
In this perspective paper we survey the current state of the art in terms of the frameworks, tools, and services available to developers to aid in mobile testing, highlighting present shortcom- ings. Next, we provide commentary on current key challenges that restrict the possibility of a comprehensive, effective, and practical automated testing solution. Finally, we offer our vision of a comprehensive mobile app testing framework, complete with research agenda, that is succinctly summarized along three principles: Continuous, Evolutionary and Large-scale (CEL).
This BJUG presentation is focused on the unique inter-process communication (IPC) mechanism introduced by Android, namely Binder. As such, we will be exploring the entire Android software stack, starting from the Linux kernel, moving up to the Linux userspace, reaching the Android framework and ending our journey with some real-life usages of Binder in Android applications.
We will show how widely used Binder actually is in Android, thus proving that it is a cornerstone of the operating system. The main focus is on the programming model exposed towards developers and we will highlight when, how and why it should be used.
https://bjug.ro/binding-android-piece-by-piece/
Testing on Mobile Devices with Location ServicesSauce Labs
During this webinar we look into location services on Android and iOS real devices, as well as emulators and simulators.
We provide you with tips and tricks on when to change the mobile Geo-IP and when to change the mobile GPS, how to use ADB commands to enable the device location, and how to handle the location services permission alerts. We also discuss Appium capabilities for Location Services, using GeoLocation with Appium Desktop, and more.
Key takeaways:
- Understand what Location Services are
- Understand GPS compare to Geo-IP
- Know the Appium commands for Location Services
- The differences in the use of Location Services between Android and iOS
- Learn how to automate tests that involve Location Services
Training in Bangalore is one of the best Selenium Testing Training Institute in bangalore. We offer Online Training and Classroom Training. Our Trainers are very professional and more than 8 years of experience.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
1. Cross-Platform
Testing and Maintenance of
Web and Mobile Applications
PhD Thesis Defense
Shauvik Roy Choudhary
Advisor: Dr. Alessandro Orso
School of Computer Science, Georgia Tech
Committee:
Dr. Mayur Naik, Dr. Milos Prvulovic
Dr. Russ Clark, Dr. Mukul Prasad
1
11. Thesis
• Key Insight: Analyze cross-platform application
behavior to address the problems
• Challenge: Significant difference in cross-
platform behavior. Match behavior despite
legitimate differences.
• Thesis: Approximate matching algorithms with
the right abstractions can be used to automate
cross-platform testing and maintenance.
11
19. Evaluation Subjects
19
Name Type URL States Transitions
Organizer Productivity http://localhost/organizer 13 99
GrantaBooks Publisher http://grantabooks.com 9 8
DesignTrust Business http://designtrust.org 10 20
DivineLife Spiritual http://sivanandaonline.org 10 9
SaiBaba Religious http://shrisaibabasansthan.org 13 20
Breakaway Sport http://breakaway-adventures.com 19 18
Conference Information http://localhost/conference 3 12
Fisherman Restaurant http://fishermanslodge.co.uk 15 17
Valleyforge Lodge http://valleyforgeinn.net 4 12
UniMelb University http://www.economics.unimelb.edu.au/ACT/ 9 8
Konqueror Software http://www.konqueror.org 5 4
UBC Club http://www.ubcsororities.com 7 7
BMVBS Ministry http://m.bmvbs.de 5 20
StarWars Movie http://www.starwarsholidayspecial.com 10 9
Prior Art (6)
Example & Survey (4)
Random (4)
20. Evaluation
• RQ1: Can X-PERT find Cross-Browser
Issues in real web applications?
• RQ2: How does X-PERT compare to the
state-of-art?
45%↑ Precision and 14%↑ Recall
Yes!
77% Precision and 95% Recall
20
25. Overall Approach
Platforms
Trace
Extraction
Traces
1
Action
Recognition
A
X
B
Y
C
D
A
B
C
D
A
Q
R
S
A
Q
R
Q
R
S
M
N
O
P
U
V
W
Labeled Actions
A
B
C
D
A
Q
R
S
2 1. REQUEST: GET /index.php
2. RESPONSE: 200 OK, 'text/html'
3. REQUEST: GET /style.css
4. REPONSE: 200 OK, 'text/css'
5. REQUEST: GET /logo.png
6. REPONSE: 200 OK, 'image/png'
7. REQUEST: GET /script.js
8. REPONSE: 200 OK, 'text/javascript'
9. REQUEST: POST /login.php
user=user1&pass=..&sid=w2s31
10. RESPONSE: 200 OK, 'text/html'
....
11. REQUEST: POST /create_blog.php
title=..&content=..
12. RESPONSE: 200 OK, 'text/html'
Domain specific
vocabulary to identify
keywords in requests
Ignore requests
to stylistic resources
25
26. 1. REQUEST: GET /index.php
2. RESPONSE: 200 OK, 'text/html'
3. REQUEST: GET /style.css
4. REPONSE: 200 OK, 'text/css'
5. REQUEST: GET /logo.png
6. REPONSE: 200 OK, 'image/png'
7. REQUEST: GET /script.js
8. REPONSE: 200 OK, 'text/javascript'
9. REQUEST: POST /login.php
user=user1&pass=..&sid=w2s31
10. RESPONSE: 200 OK, 'text/html'
....
11. REQUEST: POST /create_blog.php
title=..&content=..
12. RESPONSE: 200 OK, 'text/html'
Overall Approach
Platforms
Trace
Extraction
Traces
1
Action
Recognition
A
X
B
Y
C
D
A
B
C
D
A
Q
R
S
A
Q
R
Q
R
S
M
N
O
P
U
V
W
Labeled Actions
A
B
C
D
A
Q
R
S
2 1. REQUEST: GET /index.php
2. RESPONSE: 200 OK, 'text/html'
3. REQUEST: GET /style.css
4. REPONSE: 200 OK, 'text/css'
5. REQUEST: GET /logo.png
6. REPONSE: 200 OK, 'image/png'
7. REQUEST: GET /script.js
8. REPONSE: 200 OK, 'text/javascript'
9. REQUEST: POST /login.php
user=user1&pass=..&sid=w2s31
10. RESPONSE: 200 OK, 'text/html'
....
11. REQUEST: POST /create_blog.php
title=..&content=..
12. RESPONSE: 200 OK, 'text/html'
(‘index’),
(‘script’),
(‘login’,‘user’,‘pass’,‘sid’),
(‘create’,‘blog’,‘title’,‘content’)
Keywords
26
31. Evaluation
• Tool: FMAP (Feature Mapping Across Platforms)
• Subjects: Popular web apps (6 open source + 3 public)
Subject Type Mobile plug-in
wordpress Blogging wordpress mobile pack
drupal Content Management nokia mobile theme
phpbb Forum artodia mobile style
roundcube Email mobilecube theme
elgg Social Networking elgg mobile module
gallery Photo Management imobile theme
wikipedia.org Wiki -
stackoverflow.com Q&A -
twitter.com Social Networking -
31
32. Evaluation
• RQ1: How effective is FMAP in recognizing
web application actions?
• RQ2: How effective is FMAP in matching
features between the desktop and mobile
versions of real web applications?
Accuracy: 86.3% (FMAP)
51.5% (Baseline)
Accuracy: 97.8% (Desktop)
99.6% (Mobile)
32
33. Evaluation
• RQ1: How effective is FMAP in recognizing
web application actions?
• RQ2: How effective is FMAP in matching
features between the desktop and mobile
versions of real web applications?
Accuracy: 86.3% (FMAP)
51.5% (Baseline)
Accuracy: 97.8% (Desktop)
99.6% (Mobile)
40
50
Missing features reported by FMAP
confirmed from user reports & software fixes
33
40. MyList App - Test Migration
View Items
Add Item
MyList
Items
MyList Item
MyList
TAP “MyList Item”TAP “View Items”
Items
Delete
MyList Item
Star
MyList
TAP “Star”
Items
MyList Item
MyList
AssertExists “Star”
MyList Item >
MyList +
TAP “MyList Item”
MyList Item
MyList +
Delete ItemStar Item
TAP “Star Item”
MyList Item >
MyList +
AssertExists “Star”
40
41. Challenges
Independently
developed
(In different languages
& frameworks)
Posts
tap
"New Post"
tap
"Refresh"
tap
"More Options"
tap
"Hello World!"
Menu
tap "Open drawer"
tap "Posts"
tap "Comments"
tap "Media"
tap "Pages"
tap
tap "Navigate up"
tap "Upload"
type "Title"
tap "Content"
tap
"Post Settings"
type "Content"
BACK button
tap
"Navigate up"
tap
"Navigate up"
tap "Delete Post"
tap
"Preview"
tap "Preview Post"
tap "Share URL"
tap "Add Comment"
tap "Edit Post"
tap "Sign out"
tap "Settings"
Automated
Behavior
Exploration
Same actions
Different Widgets
No structural
similarity
Partial
Consider all
possibilities
41
42. Assumptions
• Action correspondence:
1-1 correspondence between actions
• Action ordering:
Is preserved in matched use cases
• Functionality under test:
present on both platforms
42
43. Intuition
Test Case Model Generation
Init
State
State
A
State
B
State
C
Action 1
Action 2
Action 3
Oracle
Init
State
State
1
State
5
State
6
State
2
State
3
State
4
State
7
Init
State
State
1
State
5
State
6
State
2
State
3
State
4
State
7
PLATFORM 1 PLATFORM 2
43
44. Model Definition
Labeled Transition System (LTS)
(S, s0, A, δ)
Transition
Function
Initial state
Actions
States
δ(scurrent,a) = snext
44
View Items
Add Item
MyList
Items
MyList Item
MyList
45. Solving for maximizing matching
Given:
Constraint:
Where:
•
•
Subject to:
•
Model for Platform 2:
Test Trace for Platform 1:
a & b are from
platforms p1 & p2
45
46. Solving for maximizing matching
Given:
Constraint:
Subject to:
•
•
•
a & b are from
platforms p1 & p2
Model for Platform 2:
Test Trace for Platform 1:Ordering Constraint
on matched actions
ai aj
bx by
46
47. Solving for maximizing matching
Given:
Constraint:
Subject to:
•
•
•
a & b are from
platforms p1 & p2
Model for Platform 2:
Test Trace for Platform 1:No Invalid Ordering
between matched actions
ai aj
bx bybz
ak
47
48. Solving for maximizing matching
Given:
Constraint:
Where:
•
•
Subject to:
•
Model for Platform 2:
Test Trace for Platform 1:
a & b are from
platforms p1 & p2
48
49. Technique Overview
Platform 1
App
Platform 2
App
Test Trace
Generation
Platform 1
Tests
Incremental
Model
Generation
Platform 1
Test Trace
Test
Generation
Platform 2
Tests
Model-Trace
Mapping
1 2 3
49
51. Technique Overview
Platform 1
App
Platform 2
App
Test Trace
Generation
Platform 1
Tests
Incremental
Model
Generation
Platform 1
Test Trace
Test
Generation
Platform 2
Tests
Model-Trace
Mapping
1 2 3
51
52. 1. Trace Generation
MyList Item >
MyList +
TAP “MyList Item”
TAP
<“cell”, “”, “”, “MyList Item”, 0>
Action Widget Information:
• Class Name
• ID
• Name
• Value
• Position in Parent
52
53. 1. Trace Generation
TAP “MyList Item”
TAP “Star Item”
AssertExists “Star”
Test
TAP <“cell”, “”, “”, “MyList Item”, 0>
TAP <“button”, “”, “btn1”, “Star Item”, 0>
AssertExists <“icon”, “Star”, “”, “”, 0>
Test Trace
53
55. General Model Exploration
INPUT:
Application under Test
ALGO:
Model ← (S, s0, A, δ) | S = {s0}, A=ϵ, δ=ϵ
Worklist ← s0.actions()
While (Worklist.size > 0) and time() < TIMEOUT
(s, a) ← getNextAction(Worklist)
snext ← doAction(s, a)
if snext ∉ S
Worklist ← snext.actions()
Model.add(s, a, snext)
Return Model
View Items
Add Item
MyList
Items
MyList Item
MyList
55
56. MigraTest Model Exploration
INPUT:
Application under Test
Test Traceset = {<a1, a2, …, an>, …}
ALGO:
Model ← (S, s0, A, δ) | S = {s0}, A=ϵ, δ=ϵ
Worklist ← s0.actions()
While (Worklist.size > 0) and time() < TIMEOUT
Sol ← MaxMatchSolve(Model, Traceset)
(s, a) ← Sol.getNextAction(Worklist)
snext ← doAction(s, a)
if snext ∉ S
Worklist ← snext.actions()
Model.add(s, a, snext)
Return Model, Sol.Map
MigraTest decides
what to explore
56
57. getNextAction
• Intuition: Prioritize clicks based on action similarity
and scope of new potential assignments
Cross-platform action similarity
(widget similarity & action compatibility)
Number of unassigned
dependent actions
57
58. Solving for Matching
a1
a2
a3
b1 b2
∼
∼
∼ ∼
∼
∼
TAP “MyList Item”
TAP “Star Item”
TAP “Delete Item”
TAP
“View
Item
s”
TAP
“Add
Item
”
?
?✔
TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
58
59. TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
Items
MyList Item
MyList
Add Item
MyList
Item Name
Add
TAP
"MyList Item"
TAP
"Add"
Type
"Item Name"
b3
b4
b5
59
60. Solving for Matching
a1
a2
a3
b1 b2 b3 b4 b5
=
∼
∼
∼
∼ ∼
∼
∼ ∼
TAP “MyList Item”
TAP “Star Item”
TAP “Delete Item”
TAP
“M
yListItem
”
TAP
“View
Item
s”
TAP
“Add
Item
”
TYPE
“Item
Nam
e”
TAP
“Add”
✔
TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
Items
MyList Item
MyList
Add Item
MyList
Item Name
Add
TAP
"MyList Item"
TAP
"Add"
Type
"Item Name"
b3
b4
b5
60
? ?
? ?
61. TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
Items
MyList Item
MyList
Add Item
MyList
Item Name
Add
TAP
"Add"
Type
"Item Name"
b3
b4
b5
Items
Delete
MyList Item
Star
MyList
TAP
"MyList Item"
TAP
"Star"
TAP
"Delete"
b6 b7
TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
Items
MyList Item
MyList
Add Item
MyList
Item Name
Add
TAP
"MyList Item"
TAP
"Add"
Type
"Item Name"
b3
b4
b5
61
62. Solving for Matching
a1
a2
a3
b1 b2 b3 b4 b5 b6 b7
=
∼ ∼
∼
∼
∼
∼ ∼
∼
∼ ∼
✔
✔
TAP “MyList Item”
TAP “Star Item”
TAP “Delete Item”
TAP
“M
yListItem
”
TAP
“View
Item
s”
TAP
“Add
Item
”
TYPE
“Item
Nam
e”
TAP
“Add”TAP
“Star”
TAP
“Delete”
✔
62
? ?
? ?
Trace
Action
Model
Action
a1 b3
a2 b6
a3 b7
Trace-Model
Action Mapping
63. 3. Test Generation
Model
TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
Items
MyList Item
MyList
Add Item
MyList
Item Name
Add
TAP
"Add"
Type
"Item Name"
b3
b4
b5
Items
Delete
MyList Item
Star
MyList
TAP
"MyList Item"
TAP
"Star"
TAP
"Delete"
b6 b7
Trace
a2a1
a1 a3
Trace
Action
Model
Action
a1 b3
a2 b6
a3 b7
Trace-Model
Action Mapping63
64. 3. Test Generation
Trace
a2a1
a1 a3
Model
TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
Items
MyList Item
MyList
Add Item
MyList
Item Name
Add
TAP
"Add"
Type
"Item Name"
b3
b4
b5
Items
Delete
MyList Item
Star
MyList
TAP
"MyList Item"
TAP
"Star"
TAP
"Delete"
b6 b7
b3b1 b6
b3b1 b7
Trace
Action
Model
Action
a1 b3
a2 b6
a3 b7
Trace-Model
Action Mapping64
65. 3. Test Generation
Model
TAP
"View Items"
View Items
Add Item
MyList
TAP
"Add Item"
b1
b2
Items
MyList Item
MyList
Add Item
MyList
Item Name
Add
TAP
"Add"
Type
"Item Name"
b3
b4
b5
Items
Delete
MyList Item
Star
MyList
TAP
"MyList Item"
TAP
"Star"
TAP
"Delete"
b6 b7
b3b1 b6
b3b1 b7
Tap “View Items”
Tap “MyList Item”
Tap “Star”
Tap “View Items”
Tap “MyList Item”
Tap “Delete”
65
67. 3. Test Generation
(Oracle Migration)
AssertExists
selector
AssertText
selector ExpectedValue
AssertText
new_selector ExpectedValue
AssertExists
new_selector
new_selector :=
best_of(class, label, name, value, position)
Extra Checks
to catch potential migration issues
assertExists✔
Multiple elements exist
Element doesn’t exist yet (Flaky?)
Element exists but with different value
67
71. wikiHow: Test Cases
TEST CASE DESCRIPTION
Items loaded
When user selects category,
check if items in category are loaded
Page title
When user selects category,
check if page title matches category name
Back button
When user selects category & item, & presses button with
label “Back”, check if page title matches category name
Pop-up shown
When user selects category & item, & presses button with
label “Add”, check if pop-up is displayed to user
WebView loaded
When user selects category & item,
check if a WebView is added on the screen
71
72. wikiHow: Test Migration
TEST CASE MIGRATED CHALLENGE ISSUES FOUND
Items loaded Yes
Page title Yes Duplicates found
Back button Partial
Platform specific
functionality
Pop-up shown Partial
Missing
functionality
Missing
functionality
WebView loaded Yes
72
76. WhiteHouse: Test Cases
76
TEST CASE DESCRIPTION
Menu Loaded When the user taps on Menu, check if “BLOG” is present
Photo Loaded
When the user taps on Menu, and selects “PHOTOS”
check if the first photo is loaded
Video Loaded
When the user taps on Menu, and selects “VIDEOS” check
if the first photo is loaded
Search
When the user taps on Menu, and enters “A” in the search
bar, check if page contains “No Results”
77. WhiteHouse: Test Migration
77
TEST CASE MIGRATED CHALLENGE
POTENTIAL
ISSUES FOUND
Menu Loaded Yes
Photo Loaded Yes
Empty label on
Android
Video Loaded Yes
Empty label on
Android
Search Yes Text Input
79. Plans
• Submit unpublished work to ISSTA’15
• Capture-Replay project for “Testing Everywhere”
• NSF i-Corps Program to Commercialize Capture-
Replay technology for Business Intelligence
79