Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Detecting and Summarizing GUI Changes in Evolving Mobile Apps

146 views

Published on

Mobile applications have become a popular software development domain in recent years due in part to a large user base, capable hardware, and accessible platforms. However, mobile developers also face unique challenges, including pressure for frequent releases to keep pace with rapid platform evolution, hardware iteration, and user feedback. Due to this rapid pace of evolution, developers need automated support for documenting the changes made to their apps in order to aid in program comprehension. One of the more challenging types of changes to document in mobile apps are those made to the graphical user interface (GUI) due to its abstract, pixel-based representation. In this paper, we present a fully automated approach, called Gcat, for detecting and summarizing GUI changes during the evolution of mobile apps. Gcat leverages computer vision techniques and natural language generation to accurately and concisely summarize changes made to the GUI of a mobile app between successive commits or releases. We evaluate the performance of our approach in terms of its precision and recall in detecting GUI changes compared to developer specified changes, and investigate the utility of the generated change reports in a controlled user study. Our results indicate that Gcat is capable of accurately detecting and classifying GUI changes – outperforming developers – while providing useful documentation.

Published in: Software
  • //DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... //DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... //DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Detecting and Summarizing GUI Changes in Evolving Mobile Apps

  1. 1. Kevin Moran, Cody Watson, John Hoskins, George Purnell, & Denys Poshyvanyk Detecting and Summarizing GUI Changes in Evolving Mobile Apps 33rd International Conference on Automated Software Engineering (ASE’18) Thursday, September 6th, 2018
  2. 2. !3
  3. 3. !3
  4. 4. COMPREHENDING MOBILE GUIS
  5. 5. COMPREHENDING MOBILE GUIS
  6. 6. COMPREHENDING MOBILE GUIS <PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" android:key="main" android:title="@string/account_settings_title_fmt"> <PreferenceScreen android:key="account_settings" android:title="@string/account_settings_general_title"> <com.takisoft.fix.support.v7.preference.AutoSummaryEditTextPreference android:dialogTitle="@string/account_settings_description_label" android:key="account_description" android:title="@string/account_settings_description_label" /> <CheckBoxPreference android:key="account_default" android:summary="@string/account_settings_default_summary" android:title="@string/account_settings_default_label" /> <com.fsck.k9.ui.settings.account.HoloColorPickerPreference android:key="chip_color" android:summary="@string/account_settings_color_summary" android:title="@string/account_settings_color_label" /> </PreferenceScreen>
  7. 7. COMPREHENDING MOBILE GUIS <PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" android:key="main" android:title="@string/account_settings_title_fmt"> <PreferenceScreen android:key="account_settings" android:title="@string/account_settings_general_title"> <com.takisoft.fix.support.v7.preference.AutoSummaryEditTextPreference android:dialogTitle="@string/account_settings_description_label" android:key="account_description" android:title="@string/account_settings_description_label" /> <CheckBoxPreference android:key="account_default" android:summary="@string/account_settings_default_summary" android:title="@string/account_settings_default_label" /> <com.fsck.k9.ui.settings.account.HoloColorPickerPreference android:key="chip_color" android:summary="@string/account_settings_color_summary" android:title="@string/account_settings_color_label" /> </PreferenceScreen>
  8. 8. COMPREHENDING MOBILE GUIS
  9. 9. COMPREHENDING MOBILE GUIS
  10. 10. class GeneralSettingsActivity : K9Activity(), OnPreferenceStartScreenCallback { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) setContentView(R.layout.general_settings) initializeActionBar() if (savedInstanceState == null) { fragmentTransaction { add(R.id.generalSettingsContainer, GeneralSettingsFragment.create()) } } } private fun initializeActionBar() { val actionBar = supportActionBar ?: throw RuntimeException("getSupportActionBar() == null”) actionBar.setDisplayHomeAsUpEnabled(true) } COMPREHENDING MOBILE GUIS
  11. 11. COMPREHENDING MOBILE GUIS
  12. 12. COMPREHENDING MOBILE GUIS
  13. 13. Video Source: Google
  14. 14. Video Source: Google
  15. 15. COMPLICATIONS OF FRAGMENTATION https://opensignal.com/reports/2015/08/android-fragmentation/
  16. 16. Developers need support in identifying and reasoning about GUI related changes in mobile apps
  17. 17. EXISTING WORK
  18. 18. EXISTING WORK Detecting Presentation Failures in Web Apps
  19. 19. EXISTING WORK Detecting Presentation Failures in Web Apps Automated Source Code Summarization
  20. 20. EXISTING WORK Detecting Presentation Failures in Web Apps GUI Differencing Automated Source Code Summarization
  21. 21. EXISTING WORK Detecting Presentation Failures in Web Apps GUI Differencing Automated Source Code Summarization Cross Browser Testing
  22. 22. EXISTING WORK Detecting Presentation Failures in Web Apps Detecting Design Violations in Mobile Apps GUI Differencing Automated Source Code Summarization Cross Browser Testing
  23. 23. CURRENT LIMITATIONS
  24. 24. CURRENT LIMITATIONS • No approach combines:
  25. 25. CURRENT LIMITATIONS • No approach combines: • Automated dynamic analysis
  26. 26. CURRENT LIMITATIONS • No approach combines: • Automated dynamic analysis • CV-based GUI analysis
  27. 27. CURRENT LIMITATIONS • No approach combines: • Automated dynamic analysis • CV-based GUI analysis • Automated summarization
  28. 28. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY1 71Screens with DesignViolations 1Moran, K., Li, B., Bernal-Cárdenas, C., Jelf, D., and Poshyvanyk, D., “Automated Reporting of GUI Design Violations for Mobile Apps”, (ICSE’18)
  29. 29. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY1 71Screens with DesignViolations Authors 1Moran, K., Li, B., Bernal-Cárdenas, C., Jelf, D., and Poshyvanyk, D., “Automated Reporting of GUI Design Violations for Mobile Apps”, (ICSE’18)
  30. 30. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY1 71Screens with DesignViolations Authors Open Coding 1Moran, K., Li, B., Bernal-Cárdenas, C., Jelf, D., and Poshyvanyk, D., “Automated Reporting of GUI Design Violations for Mobile Apps”, (ICSE’18)
  31. 31. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY1 71Screens with DesignViolations Authors Open Coding 1Moran, K., Li, B., Bernal-Cárdenas, C., Jelf, D., and Poshyvanyk, D., “Automated Reporting of GUI Design Violations for Mobile Apps”, (ICSE’18)
  32. 32. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY1 71Screens with DesignViolations Authors Open Coding GUI-Change Taxonomy 25% 1Moran, K., Li, B., Bernal-Cárdenas, C., Jelf, D., and Poshyvanyk, D., “Automated Reporting of GUI Design Violations for Mobile Apps”, (ICSE’18)
  33. 33. GUI-CHANGE TAXONOMY Change Category Specific Change Text Changes Text Content Change Font Change Font Color Change Layout Changes VerticalTranslation HorizontalTranslation Horizontal Size Vertical Size Resource Changes Image Color Change Removed Component Added Component Image Change ComponentType Change
  34. 34. GUI-CHANGE TAXONOMY Change Category Specific Change Text Changes Text Content Change Font Change Font Color Change Layout Changes VerticalTranslation HorizontalTranslation Horizontal Size Vertical Size Resource Changes Image Color Change Removed Component Added Component Image Change ComponentType Change
  35. 35. GUI-CHANGE TAXONOMY Change Category Specific Change Text Changes Text Content Change Font Change Font Color Change Layout Changes VerticalTranslation HorizontalTranslation Horizontal Size Vertical Size Resource Changes Image Color Change Removed Component Added Component Image Change ComponentType Change
  36. 36. GUI-CHANGE TAXONOMY Change Category Specific Change Text Changes Text Content Change Font Change Font Color Change Layout Changes VerticalTranslation HorizontalTranslation Horizontal Size Vertical Size Resource Changes Image Color Change Removed Component Added Component Image Change ComponentType Change
  37. 37. GUI-CHANGE TAXONOMY Change Category Specific Change Text Changes Text Content Change Font Change Font Color Change Layout Changes VerticalTranslation HorizontalTranslation Horizontal Size Vertical Size Resource Changes Image Color Change Removed Component Added Component Image Change ComponentType Change
  38. 38. GUI-CHANGE TAXONOMY Change Category Specific Change Text Changes Text Content Change Font Change Font Color Change Layout Changes VerticalTranslation HorizontalTranslation Horizontal Size Vertical Size Resource Changes Image Color Change Removed Component Added Component Image Change ComponentType Change
  39. 39. GUI-CHANGE TAXONOMY Change Category Specific Change Text Changes Text Content Change Font Change Font Color Change Layout Changes VerticalTranslation HorizontalTranslation Horizontal Size Vertical Size Resource Changes Image Color Change Removed Component Added Component Image Change ComponentType Change
  40. 40. GUI CHANGE ANALYSIS TOOL (GCAT) Git/CI Integration Project History Virtual or Physical Device Pool N1 N2 N3 N4 N5T7 T3 T17 T35 T12 T25 T42 T59 T29 T21 N6 CrashScope Automated Execution Engine Screenshot & Metadata Pairs - Commit i Screenshot & Metadata Pairs - Commit i+n Screen Matching & Filtering Activity/ Window Comparator Bipartite Image Comparison GUI-Hierarchy for Commit i GUI-Hierarchy for Commit i+n Leaf Node Component Detection and Matching GUI Change Resolution Perceptual Image Differencing (PID) GUI-Change Report Layout Change Detection Text Change Detection Resource Change Detection NL Summary Generator Checks: <x>,<y> Checks: <width>,<height> Normalized String Comparison Image Comparison Analysis Commit i Commit i+1 Commit i+2 Commit i+n
  41. 41. GUI CHANGE ANALYSIS TOOL (GCAT) 1) CI/CD Integration Git/CI Integration Project History Virtual or Physical Device Pool N1 N2 N3 N4 N5T7 T3 T17 T35 T12 T25 T42 T59 T29 T21 N6 CrashScope Automated Execution Engine Screenshot & Metadata Pairs - Commit i Screenshot & Metadata Pairs - Commit i+n Screen Matching & Filtering Activity/ Window Comparator Bipartite Image Comparison GUI-Hierarchy for Commit i GUI-Hierarchy for Commit i+n Leaf Node Component Detection and Matching GUI Change Resolution Perceptual Image Differencing (PID) GUI-Change Report Layout Change Detection Text Change Detection Resource Change Detection NL Summary Generator Checks: <x>,<y> Checks: <width>,<height> Normalized String Comparison Image Comparison Analysis Commit i Commit i+1 Commit i+2 Commit i+n
  42. 42. GUI CHANGE ANALYSIS TOOL (GCAT) 1) CI/CD Integration 2) Automated GUI-based Exploration Git/CI Integration Project History Virtual or Physical Device Pool N1 N2 N3 N4 N5T7 T3 T17 T35 T12 T25 T42 T59 T29 T21 N6 CrashScope Automated Execution Engine Screenshot & Metadata Pairs - Commit i Screenshot & Metadata Pairs - Commit i+n Screen Matching & Filtering Activity/ Window Comparator Bipartite Image Comparison GUI-Hierarchy for Commit i GUI-Hierarchy for Commit i+n Leaf Node Component Detection and Matching GUI Change Resolution Perceptual Image Differencing (PID) GUI-Change Report Layout Change Detection Text Change Detection Resource Change Detection NL Summary Generator Checks: <x>,<y> Checks: <width>,<height> Normalized String Comparison Image Comparison Analysis Commit i Commit i+1 Commit i+2 Commit i+n
  43. 43. GUI CHANGE ANALYSIS TOOL (GCAT) 1) CI/CD Integration 2) Automated GUI-based Exploration 3) GUI Interpretation & Change Detection Git/CI Integration Project History Virtual or Physical Device Pool N1 N2 N3 N4 N5T7 T3 T17 T35 T12 T25 T42 T59 T29 T21 N6 CrashScope Automated Execution Engine Screenshot & Metadata Pairs - Commit i Screenshot & Metadata Pairs - Commit i+n Screen Matching & Filtering Activity/ Window Comparator Bipartite Image Comparison GUI-Hierarchy for Commit i GUI-Hierarchy for Commit i+n Leaf Node Component Detection and Matching GUI Change Resolution Perceptual Image Differencing (PID) GUI-Change Report Layout Change Detection Text Change Detection Resource Change Detection NL Summary Generator Checks: <x>,<y> Checks: <width>,<height> Normalized String Comparison Image Comparison Analysis Commit i Commit i+1 Commit i+2 Commit i+n
  44. 44. STAGE 1: CI/CD INTEGRATION Git/CI Integration Project History Virtual or Physical Device Po N1 N2 N3 N4 T7 T3 T17 T35 T12 T25 T42 T T29 T21 N6 CrashScope Automated Execution Engine Commit i Commit i+1 Commit i+2 Commit i+n
  45. 45. STAGE 1: CI/CD INTEGRATION Git/CI Integration Project History Virtual or Physical Device Po N1 N2 N3 N4 T7 T3 T17 T35 T12 T25 T42 T T29 T21 N6 CrashScope Automated Execution Engine Commit i Commit i+1 Commit i+2 Commit i+n
  46. 46. STAGE 1: CI/CD INTEGRATION Git/CI Integration Project History Virtual or Physical Device Po N1 N2 N3 N4 T7 T3 T17 T35 T12 T25 T42 T T29 T21 N6 CrashScope Automated Execution Engine Commit i Commit i+1 Commit i+2 Commit i+n
  47. 47. STAGE 2:AUTOMATED DYNAMIC ANALYSIS
  48. 48. STAGE 2:AUTOMATED DYNAMIC ANALYSIS CrashScope1 Execution Engine 1Moran, K., Linares-Vásquez, M., Bernal-Cárdenas, C., Vendome, C., and Poshyvanyk, D., “Automatically Discovering, Reporting and Reproducing Android Application Crashes”, ICST’16
  49. 49. STAGE 2:AUTOMATED DYNAMIC ANALYSIS CrashScope1 Execution Engine Execution Engine 1Moran, K., Linares-Vásquez, M., Bernal-Cárdenas, C., Vendome, C., and Poshyvanyk, D., “Automatically Discovering, Reporting and Reproducing Android Application Crashes”, ICST’16
  50. 50. STAGE 2:AUTOMATED DYNAMIC ANALYSIS CrashScope1 Execution Engine Execution Engine 1Moran, K., Linares-Vásquez, M., Bernal-Cárdenas, C., Vendome, C., and Poshyvanyk, D., “Automatically Discovering, Reporting and Reproducing Android Application Crashes”, ICST’16
  51. 51. STAGE 2:AUTOMATED DYNAMIC ANALYSIS CrashScope1 Execution Engine Execution Engine 1Moran, K., Linares-Vásquez, M., Bernal-Cárdenas, C., Vendome, C., and Poshyvanyk, D., “Automatically Discovering, Reporting and Reproducing Android Application Crashes”, ICST’16
  52. 52. STAGE 2: SCREEN MATCHING AND FILTERING Older Commit Newer Commit
  53. 53. STAGE 2: SCREEN MATCHING AND FILTERING GUI Color Histogram Analysis Spatial GUI Component Analysis Older Commit Newer Commit Discard Non-Unique Activity-Window Pairs
  54. 54. STAGE 2: SCREEN MATCHING AND FILTERING GUI Color Histogram Analysis Spatial GUI Component Analysis Older Commit Newer Commit Discard Non-Unique Activity-Window Pairs
  55. 55. STAGE 2: SCREEN MATCHING AND FILTERING GUI Color Histogram Analysis Spatial GUI Component Analysis Older Commit Newer Commit Matched Screen Pairs ≈ ≈ ≈ Discard Non-Unique Activity-Window Pairs
  56. 56. STAGE3: GUI COMPREHENSION Old Commit New Commit
  57. 57. STAGE3: GUI COMPREHENSION Old Commit New CommitSpatial Component Matching
  58. 58. = (|xm xr| + |ym yr| + |wm wr| + |hm hr|)<latexit sha1_base64="Ht7153Xmi/4IY79Asy9xCp7Yq18=">AAACKHicdVDLSgMxFM3UV62vqks3wSIo4pCpr3YhFN24VLBaaMuQSdNOMJkZkox2aPs7bvwVNwoqbv0S03EEFb0QOI97ubnHizhTGqE3KzcxOTU9k58tzM0vLC4Vl1cuVRhLQusk5KFseFhRzgJa10xz2ogkxcLj9Mq7Phn7VzdUKhYGFzqJaFvgXsC6jGBtJLdYa/WwEBgewc1h3xVwB/ZdOYTbcJikLMnYbcpuM+anzDdsyy2WkF2uOPsHDkT2LkKHCBlQrh44lTJ0bJRWCWR15hafWp2QxIIGmnCsVNNBkW4PsNSMcDoqtGJFI0yucY82DQywoKo9SC8dwQ2jdGA3lOYFGqbq94kBFkolwjOdAmtf/fbG4l9eM9bdSnvAgijWNCCfi7oxhzqE49hgh0lKNE8MwEQy81dIfCwx0Sbcggnh61L4P6iX7aqNzvdKteMsjTxYA+tgEzjgENTAKTgDdUDAHXgAz+DFurcerVfr7bM1Z2Uzq+BHWe8fdYKjUQ==</latexit><latexit sha1_base64="Ht7153Xmi/4IY79Asy9xCp7Yq18=">AAACKHicdVDLSgMxFM3UV62vqks3wSIo4pCpr3YhFN24VLBaaMuQSdNOMJkZkox2aPs7bvwVNwoqbv0S03EEFb0QOI97ubnHizhTGqE3KzcxOTU9k58tzM0vLC4Vl1cuVRhLQusk5KFseFhRzgJa10xz2ogkxcLj9Mq7Phn7VzdUKhYGFzqJaFvgXsC6jGBtJLdYa/WwEBgewc1h3xVwB/ZdOYTbcJikLMnYbcpuM+anzDdsyy2WkF2uOPsHDkT2LkKHCBlQrh44lTJ0bJRWCWR15hafWp2QxIIGmnCsVNNBkW4PsNSMcDoqtGJFI0yucY82DQywoKo9SC8dwQ2jdGA3lOYFGqbq94kBFkolwjOdAmtf/fbG4l9eM9bdSnvAgijWNCCfi7oxhzqE49hgh0lKNE8MwEQy81dIfCwx0Sbcggnh61L4P6iX7aqNzvdKteMsjTxYA+tgEzjgENTAKTgDdUDAHXgAz+DFurcerVfr7bM1Z2Uzq+BHWe8fdYKjUQ==</latexit><latexit sha1_base64="Ht7153Xmi/4IY79Asy9xCp7Yq18=">AAACKHicdVDLSgMxFM3UV62vqks3wSIo4pCpr3YhFN24VLBaaMuQSdNOMJkZkox2aPs7bvwVNwoqbv0S03EEFb0QOI97ubnHizhTGqE3KzcxOTU9k58tzM0vLC4Vl1cuVRhLQusk5KFseFhRzgJa10xz2ogkxcLj9Mq7Phn7VzdUKhYGFzqJaFvgXsC6jGBtJLdYa/WwEBgewc1h3xVwB/ZdOYTbcJikLMnYbcpuM+anzDdsyy2WkF2uOPsHDkT2LkKHCBlQrh44lTJ0bJRWCWR15hafWp2QxIIGmnCsVNNBkW4PsNSMcDoqtGJFI0yucY82DQywoKo9SC8dwQ2jdGA3lOYFGqbq94kBFkolwjOdAmtf/fbG4l9eM9bdSnvAgijWNCCfi7oxhzqE49hgh0lKNE8MwEQy81dIfCwx0Sbcggnh61L4P6iX7aqNzvdKteMsjTxYA+tgEzjgENTAKTgDdUDAHXgAz+DFurcerVfr7bM1Z2Uzq+BHWe8fdYKjUQ==</latexit> ≊ ≊ ≊ ≊ STAGE3: GUI COMPREHENSION Old Commit New CommitSpatial Component Matching
  59. 59. Perceptual Image Differencing (PID) STAGE 3: GUI-CHANGE DETECTION
  60. 60. Perceptual Image Differencing (PID) STAGE 3: GUI-CHANGE DETECTION
  61. 61. Perceptual Image Differencing (PID) STAGE 3: GUI-CHANGE DETECTION
  62. 62. ≊ ≊ ≊ ≊ ≊ STAGE 3: GUI-CHANGE DETECTION
  63. 63. ≊ ≊ ≊ ≊ ≊ STAGE 3: GUI-CHANGE DETECTION
  64. 64. ≊ ≊ ≊ ≊ ≊ STAGE 3: GUI-CHANGE DETECTION
  65. 65. List of Potential GUI Changes STAGE 3: GUI CHANGE RESOLUTION
  66. 66. List of Potential GUI Changes STAGE 3: GUI CHANGE RESOLUTION
  67. 67. List of Potential GUI Changes Layout Change Resolver • <x,y> • <height,width> Text Change Resolver • <image> • <text> CHA + CQ Resource Change Resolver • <image> • Missing/ Extra CHA + CQ STAGE 3: GUI CHANGE RESOLUTION
  68. 68. GCAT REPORT
  69. 69. GVT REPORT
  70. 70. GVT REPORT
  71. 71. EMPIRICAL STUDY
  72. 72. EMPIRICAL STUDY • RQ1: Performance of Screen Matching and Filtering?
  73. 73. EMPIRICAL STUDY • RQ1: Performance of Screen Matching and Filtering? • RQ2: GCAT Detecting and Classifying GUI-Changes?
  74. 74. EMPIRICAL STUDY • RQ1: Performance of Screen Matching and Filtering? • RQ2: GCAT Detecting and Classifying GUI-Changes?
  75. 75. EMPIRICAL STUDY • RQ1: Performance of Screen Matching and Filtering? • RQ2: GCAT Detecting and Classifying GUI-Changes? • RQ3: Developers Detecting and Classifying GUI-Changes?
  76. 76. EMPIRICAL STUDY • RQ1: Performance of Screen Matching and Filtering? • RQ2: GCAT Detecting and Classifying GUI-Changes? • RQ3: Developers Detecting and Classifying GUI-Changes? • RQ4: Developer Utility?
  77. 77. STUDY CONTEXT FDroid 31 Open Source Apps (62 Apks) Screen Collection Methodology
  78. 78. STUDY CONTEXT FDroid • Compilable Subsequent Commits? 31 Open Source Apps (62 Apks) Screen Collection Methodology
  79. 79. STUDY CONTEXT FDroid • Compilable Subsequent Commits? • Compiled Commits run on target Nexus 7 device? 31 Open Source Apps (62 Apks) Screen Collection Methodology
  80. 80. STUDY CONTEXT FDroid • Compilable Subsequent Commits? • Compiled Commits run on target Nexus 7 device? • GUI Changes exist on at least 2 screens? 31 Open Source Apps (62 Apks) Screen Collection Methodology
  81. 81. RQ1 & RQ2: EFFECTIVENESS AND PERFORMANCE 31 Open Source Apps (62 Apks)
  82. 82. RQ1 & RQ2: EFFECTIVENESS AND PERFORMANCE 31 Open Source Apps (62 Apks) CrashScope Execution Engine
  83. 83. RQ1 & RQ2: EFFECTIVENESS AND PERFORMANCE 31 Open Source Apps (62 Apks) CrashScope Execution Engine 3,854 Extracted Screens
  84. 84. RQ1 & RQ2: EFFECTIVENESS AND PERFORMANCE 3,854 Extracted Screens
  85. 85. RQ1 & RQ2: EFFECTIVENESS AND PERFORMANCE 3,854 Extracted Screens GUI Color Histogram Analysis Spatial Analysis Screen Matching & Filtering Discard Non-Unique Activity-Window Pairs
  86. 86. RQ1 & RQ2: EFFECTIVENESS AND PERFORMANCE 3,854 Extracted Screens Matched Screens • High # of Changes • Medium # of Changes • Low # of Changes GUI Color Histogram Analysis Spatial Analysis Screen Matching & Filtering Discard Non-Unique Activity-Window Pairs
  87. 87. RQ1 & RQ2: RESULTS % Filtered Screens Matching Precision (3,538 screens) 91.8% (134/151 pairs) 84.4% Screen Filtering & Matching
  88. 88. RQ1 & RQ2: RESULTS % Filtered Screens Matching Precision (3,538 screens) 91.8% (134/151 pairs) 84.4% Screen Filtering & Matching R CP DP 0 25 50 75 100 GUI Change Detection and Classification Detection Precision (DP) Classification Precision (CP) Recall (R) *Random Sample of 18 Screen Pairs
  89. 89. RQ2 & RQ3: DEVELOPER PERFORMANCE & UTILITY Layout Changes TextChanges Resource Changes 20 Academic and Industrial Developers 25% CP, DP, R Usability and User Preference Questions
  90. 90. RQ2 & RQ3: EFFECTIVENESS AND PERFORMANCE 31 Open Source Apps (62 Apks) Manual Author Screen Analysis 50 Screen Paris Split Among High, Medium and Low # of changes
  91. 91. RQ2 & RQ3: RESULTS R CP DP 0 25 50 75 100 Detection Precision (DP) Classification Precision (CP) Recall (R) Developer Performance
  92. 92. RQ2 & RQ3: RESULTS R CP DP 0 25 50 75 100 Detection Precision (DP) Classification Precision (CP) Recall (R) Developer Performance Easy to understand? Reports were Useful? Would Ike to use often? Hard to Read? Too Complex? 0 1.25 2.5 3.75 5 AgreeDisagree Developer Utility
  93. 93. RQ2 & RQ3: RESULTS
  94. 94. “The detailed changes [are best], since they show the status of the elements before/after the changes. ” RQ2 & RQ3: RESULTS
  95. 95. “The detailed changes [are best], since they show the status of the elements before/after the changes. ” “Seeing how components moved with the highlighted red boxes [was most helpful to me].” RQ2 & RQ3: RESULTS
  96. 96. “The detailed changes [are best], since they show the status of the elements before/after the changes. ” “Seeing how components moved with the highlighted red boxes [was most helpful to me].” “The spanning tree, I like the way it describes changes in tree structure to make it clear and straightforward. ” RQ2 & RQ3: RESULTS
  97. 97. “The detailed changes [are best], since they show the status of the elements before/after the changes. ” “A way to group the changes or a potential importance (e.g., a new component may be more important as compared to a 2px change).” “Seeing how components moved with the highlighted red boxes [was most helpful to me].” “The spanning tree, I like the way it describes changes in tree structure to make it clear and straightforward. ” RQ2 & RQ3: RESULTS
  98. 98. AVENUES FOR FUTURE WORK
  99. 99. • Develop Static AnalysisTechniques for integrating GUI code information into change reports AVENUES FOR FUTURE WORK
  100. 100. • Develop Static AnalysisTechniques for integrating GUI code information into change reports • Integrate GUI change information into IDEs to aid in program comprehension AVENUES FOR FUTURE WORK
  101. 101. • Develop Static AnalysisTechniques for integrating GUI code information into change reports • Integrate GUI change information into IDEs to aid in program comprehension • DevelopTechniques for effectively handling dynamic screen content AVENUES FOR FUTURE WORK
  102. 102. • Develop Static AnalysisTechniques for integrating GUI code information into change reports • Integrate GUI change information into IDEs to aid in program comprehension • DevelopTechniques for effectively handling dynamic screen content • Prioritizing the Importance of changes for more effective summarization AVENUES FOR FUTURE WORK
  103. 103. https://www.android-dev-tools.com/gcat/
  104. 104. Any Questions? Thank you! Kevin Moran Post-Doctoral Fellow College of William & Mary @kevpmo kpmoran@cs.wm.edu https://www.kpmoran.com
  105. 105. ADDITIONAL SLIDES
  106. 106. GUI CHANGE AMBIGUITIES a) Previous application menu bar b) New application menu bar The was replaced by the , the was replaced by the and the was added to the GUI. The and were shifted to the left, and the was added to the GUI.

×