Presentation for ICSE 2018 Technical paper entitled "Automated Reporting of GUI Design Violations for Mobile Apps".

1. Kevin Moran,
Boyang Li,
Carlos Bernal-Cárdenas,
Dan Jelf, and
Denys Poshyvanyk
Automated Reporting of GUI
DesignViolations for Mobile Apps
40th International Conference
on Software Engineering (ICSE’18)
Wednesday, May 30th, 2018

4. Video Source: Google

5. Video Source: Google

6. !5

7. !5

8. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

9. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

10. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam
Sketch

11. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

12. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

13. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

14. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

15. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

16. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

17. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

18. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

19. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

20. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

21. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

22. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

23. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

24. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

25. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

26. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

27. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

28. RESEARCH PROBLEM
UI/UX DesignTeam DevelopmentTeam

29. EXISTING WORK
• Utilizes Perceptual Image Differencing (PID) for detecting Differences
• Performs clustering of difference areas and detects potential faulty html elements
• Predicts Root Causes by Probabilistic modeling
Detecting Presentation Failures in Web Applications*
*Detection and Localization of HTML Presentation Failures Using Computer Vision-Based Techniques. Sonal Mahajan, William G. J. Halfond. In
Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation (ICST). April 2015.
*Using Visual Symptoms for Debugging Presentation Failures in Web Applications. Sonal Mahajan, Bailan Li, Pooyan Behnamghader, William G.J.
Halfond. In Proceedings of the 9th IEEE International Conference on Software Testing, Verification, and Validation (ICST). April 2016.

30. EXISTING LIMITATIONS

31. EXISTING LIMITATIONS
• Tailored for web pages and is not directly applicable to native mobile
app GUIs

32. EXISTING LIMITATIONS
• Tailored for web pages and is not directly applicable to native mobile
app GUIs
• Does not generate expressive reports to facilitate communication
between teams

33. EXISTING LIMITATIONS
• Tailored for web pages and is not directly applicable to native mobile
app GUIs
• Does not generate expressive reports to facilitate communication
between teams
• Predictive modeling of visual symptoms may not be precise enough

34. OUR SOLUTION

35. OUR SOLUTION
GUI Design Specifications
GUI Implementation

36. OUR SOLUTION

37. OUR SOLUTION

38. OUR SOLUTION

39. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY
71Screens with
DesignViolations

40. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY
71Screens with
DesignViolations
Authors

41. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY
71Screens with
DesignViolations
Authors
Open Coding

42. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY
71Screens with
DesignViolations
Authors
Open Coding

43. PRELIMINARY INDUSTRIAL EMPIRICAL STUDY
71Screens with
DesignViolations
Authors
Open Coding
DesignViolation
Taxonomy
25%

44. PRELIMINARY EMPIRICAL STUDY - RESULTS
H = Horizontal
V =Vertical
Trans = Translation

45. GUI VERIFICATION SYSTEM (GVT)

46. GUI VERIFICATION SYSTEM (GVT)
1) GUI-Collection

47. GUI VERIFICATION SYSTEM (GVT)
1) GUI-Collection
2) GUI
Comprehension

48. GUI VERIFICATION SYSTEM (GVT)
1) GUI-Collection
2) GUI
Comprehension
3) DesignViolation
Detection

49. STAGE 1: GUI COLLECTION

50. STAGE 1: GUI COLLECTION

51. STAGE 2: GUI COMPREHENSION
Mock-Up Implementation

52. STAGE 2: GUI COMPREHENSION
Mock-Up ImplementationSpatial
Component
Matching

53. = (|xm xr| + |ym yr| + |wm wr| + |hm hr|)<latexit sha1_base64="Ht7153Xmi/4IY79Asy9xCp7Yq18=">AAACKHicdVDLSgMxFM3UV62vqks3wSIo4pCpr3YhFN24VLBaaMuQSdNOMJkZkox2aPs7bvwVNwoqbv0S03EEFb0QOI97ubnHizhTGqE3KzcxOTU9k58tzM0vLC4Vl1cuVRhLQusk5KFseFhRzgJa10xz2ogkxcLj9Mq7Phn7VzdUKhYGFzqJaFvgXsC6jGBtJLdYa/WwEBgewc1h3xVwB/ZdOYTbcJikLMnYbcpuM+anzDdsyy2WkF2uOPsHDkT2LkKHCBlQrh44lTJ0bJRWCWR15hafWp2QxIIGmnCsVNNBkW4PsNSMcDoqtGJFI0yucY82DQywoKo9SC8dwQ2jdGA3lOYFGqbq94kBFkolwjOdAmtf/fbG4l9eM9bdSnvAgijWNCCfi7oxhzqE49hgh0lKNE8MwEQy81dIfCwx0Sbcggnh61L4P6iX7aqNzvdKteMsjTxYA+tgEzjgENTAKTgDdUDAHXgAz+DFurcerVfr7bM1Z2Uzq+BHWe8fdYKjUQ==</latexit><latexit sha1_base64="Ht7153Xmi/4IY79Asy9xCp7Yq18=">AAACKHicdVDLSgMxFM3UV62vqks3wSIo4pCpr3YhFN24VLBaaMuQSdNOMJkZkox2aPs7bvwVNwoqbv0S03EEFb0QOI97ubnHizhTGqE3KzcxOTU9k58tzM0vLC4Vl1cuVRhLQusk5KFseFhRzgJa10xz2ogkxcLj9Mq7Phn7VzdUKhYGFzqJaFvgXsC6jGBtJLdYa/WwEBgewc1h3xVwB/ZdOYTbcJikLMnYbcpuM+anzDdsyy2WkF2uOPsHDkT2LkKHCBlQrh44lTJ0bJRWCWR15hafWp2QxIIGmnCsVNNBkW4PsNSMcDoqtGJFI0yucY82DQywoKo9SC8dwQ2jdGA3lOYFGqbq94kBFkolwjOdAmtf/fbG4l9eM9bdSnvAgijWNCCfi7oxhzqE49hgh0lKNE8MwEQy81dIfCwx0Sbcggnh61L4P6iX7aqNzvdKteMsjTxYA+tgEzjgENTAKTgDdUDAHXgAz+DFurcerVfr7bM1Z2Uzq+BHWe8fdYKjUQ==</latexit><latexit sha1_base64="Ht7153Xmi/4IY79Asy9xCp7Yq18=">AAACKHicdVDLSgMxFM3UV62vqks3wSIo4pCpr3YhFN24VLBaaMuQSdNOMJkZkox2aPs7bvwVNwoqbv0S03EEFb0QOI97ubnHizhTGqE3KzcxOTU9k58tzM0vLC4Vl1cuVRhLQusk5KFseFhRzgJa10xz2ogkxcLj9Mq7Phn7VzdUKhYGFzqJaFvgXsC6jGBtJLdYa/WwEBgewc1h3xVwB/ZdOYTbcJikLMnYbcpuM+anzDdsyy2WkF2uOPsHDkT2LkKHCBlQrh44lTJ0bJRWCWR15hafWp2QxIIGmnCsVNNBkW4PsNSMcDoqtGJFI0yucY82DQywoKo9SC8dwQ2jdGA3lOYFGqbq94kBFkolwjOdAmtf/fbG4l9eM9bdSnvAgijWNCCfi7oxhzqE49hgh0lKNE8MwEQy81dIfCwx0Sbcggnh61L4P6iX7aqNzvdKteMsjTxYA+tgEzjgENTAKTgDdUDAHXgAz+DFurcerVfr7bM1Z2Uzq+BHWe8fdYKjUQ==</latexit>
≊
≊
≊
≊
STAGE 2: GUI COMPREHENSION
Mock-Up ImplementationSpatial
Component
Matching

54. Perceptual
Image
Differencing
(PID)
STAGE 3: DESIGN VIOLATION DETECTION

55. Perceptual
Image
Differencing
(PID)
STAGE 3: DESIGN VIOLATION DETECTION

56. Perceptual
Image
Differencing
(PID)
STAGE 3: DESIGN VIOLATION DETECTION

57. ≊
≊
≊
≊
STAGE 3: DESIGN VIOLATION DETECTION

58. ≊
≊
≊
≊
STAGE 3: DESIGN VIOLATION DETECTION

59. ≊
≊
≊
≊
STAGE 3: DESIGN VIOLATION DETECTION

60. List of Potential
DesignViolations
STAGE 3: DESIGN VIOLATION DETECTION

61. List of Potential
DesignViolations
STAGE 3: DESIGN VIOLATION DETECTION

62. List of Potential
DesignViolations
Layout DV Resolver
• <x,y>
• <height,width>
Text DV Resolver
• <image>
• <text> CHA + CQ
Resource DV Resolver
• <image>
• Missing/
Extra CHA + CQ
STAGE 3: DESIGN VIOLATION DETECTION

63. GVT REPORT

64. GVT REPORT

65. THE GVT INTERFACE

66. THE GVT INTERFACE

67. THE GVT INTERFACE

68. THE GVT INTERFACE

69. THE IMPROVED GVT INTERFACE
W&M Undergrads- Ashley Roten, Ivan Echevarria, Karen Coppage, Kelly Giddens,Thomas Roche

70. THE IMPROVED GVT INTERFACE
W&M Undergrads- Ashley Roten, Ivan Echevarria, Karen Coppage, Kelly Giddens,Thomas Roche

71. THE IMPROVED GVT INTERFACE
W&M Undergrads- Ashley Roten, Ivan Echevarria, Karen Coppage, Kelly Giddens,Thomas Roche

72. EMPIRICAL STUDY

73. EMPIRICAL STUDY
• RQ1: Detecting and Classifying Design Violations ?

74. EMPIRICAL STUDY
• RQ1: Detecting and Classifying Design Violations ?
• RQ2: Utility for Developers?

75. EMPIRICAL STUDY
• RQ1: Detecting and Classifying Design Violations ?
• RQ2: Utility for Developers?
• RQ3: Industrial Applicability?

76. STUDY 1: EFFECTIVENESS AND PERFORMANCE
Sketch

77. STUDY 1: EFFECTIVENESS AND PERFORMANCE
Runtime
Screenshots & XML

78. STUDY 1: EFFECTIVENESS AND PERFORMANCE
Synthetic DV
Injection
Runtime
Screenshots & XML

79. STUDY 1: EFFECTIVENESS AND PERFORMANCE
XML and
Image Manipulation
Synthetic DV
Injection
Runtime
Screenshots & XML

80. STUDY 1: EFFECTIVENESS AND PERFORMANCE
LayoutViolations
TextViolations
ResourceViolations
100 Screens
200 DVs
XML and
Image Manipulation
Synthetic DV
Injection
Runtime
Screenshots & XML

81. STUDY 1: RESULTS
80
85
90
95
100
FontColor(18)
FontStyle
(18)
Im
age
Color(22)
Im
age
(16)
M
issingG
C
(26)
Size
(18)
TextContent(14)
Translation
(68)
CP = Classification Precision DP = Detection Precision R = Recall

82. STUDY 2: GVT UTILITY
LayoutViolations
TextViolations
ResourceViolations
45 Screens
40 DesignViolations
10 Academic and
Industrial Android Developers
25%
CP, DP, R
Usability and
User Preference
Questions

83. STUDY 2: RESULTS
Easy to understand?
Reports were Useful?
Would Ike to use often?
Hard to Read?
Too Complex?
0 1.25 2.5 3.75 5
Developer UX/UP
Responses
Developer CP, DP and R
AgreeDisagree

84. STUDY 3: INDUSTRIAL APPLICABILITY
20 Industrial Android
Developers & Designers
Usability and
User Preference
Questions
GVTTool
Semi-Structured
Interviews with
Managers

85. STUDY 3: RESULTS
0 1 2 3 4 5
Industrial Developer/Designer Responses
AgreeDisagree
Using GVT tool helped to improve the quality of
mobile applications produced by Huawei
The GVT tool helped my team communicate with
other teams regarding GUI design violations
The GVT tool is able to accurately report existing
design violations in production applications
The GVT has helped you to reduce the time
required for verifying design violations
The GVT tool allowed for better transfer of
the design to the implementation of the app

86. STUDY 3: RESULTS

87. STUDY 3: RESULTS
“When the page is more complicated, the tool is more helpful.”

88. STUDY 3: RESULTS
“When the page is more complicated, the tool is more helpful.”
“[DVs] can be detected pretty well... [but the tool is] not very
flexible. For example, a switch component in the design is
open, but the switch is closed in the implementation"

89. STUDY 3: RESULTS
“When the page is more complicated, the tool is more helpful.”
“[DVs] can be detected pretty well... [but the tool is] not very
flexible. For example, a switch component in the design is
open, but the switch is closed in the implementation"
“Currently, most errors are layout errors, so [the] tool is
accurate. Static components are basically detected, [but]
maybe the next extension should focus on dynamic
components.”

90. STUDY 3: RESULTS
“When the page is more complicated, the tool is more helpful.”
“[DVs] can be detected pretty well... [but the tool is] not very
flexible. For example, a switch component in the design is
open, but the switch is closed in the implementation"
“Currently, most errors are layout errors, so [the] tool is
accurate. Static components are basically detected, [but]
maybe the next extension should focus on dynamic
components.”
“If you consider the time savings on
discussion and arguments between the two
teams, this tool saves us a lot of time”

91. GVT: COLLABORATION OUTCOME

92. GVT: COLLABORATION OUTCOME
GVT has impacted over 1,000 developers and designers!

93. FUTURE WORK

94. • Add support for detecting design violations for dynamic content
FUTURE WORK

95. • Add support for detecting design violations for dynamic content
• Improve the user interface and usability of the tool
FUTURE WORK

96. • Add support for detecting design violations for dynamic content
• Improve the user interface and usability of the tool
• Gather further feedback from developers and designers
FUTURE WORK

97. https://www.android-dev-tools.com/gvt/

103. Any Questions?
Thank you!
Kevin Moran
College of William & Mary
@kevpmo
kpmoran@cs.wm.edu
https://www.kpmoran.com

105. ADDITIONAL SLIDES

106. EXISTING WORK
Commercial Solutions

107. LIMITATIONS

108. LIMITATIONS
• Impose restrictions on the app design process

109. LIMITATIONS
• Impose restrictions on the app design process
• Cannot enforce the implementation of design specifications

110. LIMITATIONS
• Impose restrictions on the app design process
• Cannot enforce the implementation of design specifications
• Can be expensive or prohibitive to operate

111. PROBLEM FORMALIZATION
Screens and GUI-Components
DesignViolations
S = {GC1{GC2{GCi}, GC3}}<latexit sha1_base64="fUnIc/N9nW4lcCs00VSw9yHs4w0=">AAACCnicdVDLSgMxFM3UV62vqks3oUVwIUNmqn0shGIXuqxotdApJZOmbWjmQZIRyjB7N/6KGxcqbv0Cd/6N6bSCih7IzeGce0nucUPOpELow8gsLC4tr2RXc2vrG5tb+e2daxlEgtAWCXgg2i6WlDOfthRTnLZDQbHncnrjjhtT/+aWCskC/0pNQtr18NBnA0aw0lIvX7iEJ9CJzxo9K612WpmTHEJ9l5zESXr5IjLtqnVctiAySwhVENLErpWtqg0tE6Uogjmavfy70w9I5FFfEY6l7FgoVN0YC8UIp0nOiSQNMRnjIe1o6mOPym6c7pLAfa304SAQ+vgKpur3iRh7Uk48V3d6WI3kb28q/uV1IjWodmPmh5GiPpk9NIg4VAGcBgP7TFCi+EQTTATTf4VkhAUmSseX0yF8bQr/Jy3brJno4qhYP52nkQV7oAAOgAUqoA7OQRO0AAF34AE8gWfj3ng0XozXWWvGmM/sgh8w3j4BN/aY3g==</latexit><latexit sha1_base64="fUnIc/N9nW4lcCs00VSw9yHs4w0=">AAACCnicdVDLSgMxFM3UV62vqks3oUVwIUNmqn0shGIXuqxotdApJZOmbWjmQZIRyjB7N/6KGxcqbv0Cd/6N6bSCih7IzeGce0nucUPOpELow8gsLC4tr2RXc2vrG5tb+e2daxlEgtAWCXgg2i6WlDOfthRTnLZDQbHncnrjjhtT/+aWCskC/0pNQtr18NBnA0aw0lIvX7iEJ9CJzxo9K612WpmTHEJ9l5zESXr5IjLtqnVctiAySwhVENLErpWtqg0tE6Uogjmavfy70w9I5FFfEY6l7FgoVN0YC8UIp0nOiSQNMRnjIe1o6mOPym6c7pLAfa304SAQ+vgKpur3iRh7Uk48V3d6WI3kb28q/uV1IjWodmPmh5GiPpk9NIg4VAGcBgP7TFCi+EQTTATTf4VkhAUmSseX0yF8bQr/Jy3brJno4qhYP52nkQV7oAAOgAUqoA7OQRO0AAF34AE8gWfj3ng0XozXWWvGmM/sgh8w3j4BN/aY3g==</latexit><latexit sha1_base64="fUnIc/N9nW4lcCs00VSw9yHs4w0=">AAACCnicdVDLSgMxFM3UV62vqks3oUVwIUNmqn0shGIXuqxotdApJZOmbWjmQZIRyjB7N/6KGxcqbv0Cd/6N6bSCih7IzeGce0nucUPOpELow8gsLC4tr2RXc2vrG5tb+e2daxlEgtAWCXgg2i6WlDOfthRTnLZDQbHncnrjjhtT/+aWCskC/0pNQtr18NBnA0aw0lIvX7iEJ9CJzxo9K612WpmTHEJ9l5zESXr5IjLtqnVctiAySwhVENLErpWtqg0tE6Uogjmavfy70w9I5FFfEY6l7FgoVN0YC8UIp0nOiSQNMRnjIe1o6mOPym6c7pLAfa304SAQ+vgKpur3iRh7Uk48V3d6WI3kb28q/uV1IjWodmPmh5GiPpk9NIg4VAGcBgP7TFCi+EQTTATTf4VkhAUmSseX0yF8bQr/Jy3brJno4qhYP52nkQV7oAAOgAUqoA7OQRO0AAF34AE8gWfj3ng0XozXWWvGmM/sgh8w3j4BN/aY3g==</latexit>
(GCm
i ⇡ GCr
j ) ^ (GCm
i 6= GCr
j )
=) DV 2 {GCm
i , GCr
j }<latexit sha1_base64="xgaNEjkEl6iZoQFEvQTxuPwBpk4=">AAACTHicdVFNbxMxFPQGaEv4aKBHLk9ESEVCK2/oR3qrWiR6LBJpK8Vp5PW+pKa2d7G90Gi1f7AXJG78Cy4cKELCSRNREIxkaTQzT34ep4WSzlP6JWrcun1naXnlbvPe/QcPV1uPHh+5vLQCeyJXuT1JuUMlDfa89ApPCotcpwqP0/P9qX/8Aa2TuXnrJwUONB8bOZKC+yANWxmsv94fVrI+1cB4Udj8AqbCu/rUPgf2EbMx3owYfP/bZwyY1GFNdPDqKHADrFpkXyxirB622jTudJPNrQRo/JLSbUoD6exsJd0OJDGdoU3mOBy2PrMsF6VG44XizvUTWvhBxa2XQmHdZKXDgotzPsZ+oIZrdINq1kYNz4KSwSi34RgPM/XmRMW1cxOdhqTm/sz97U3Ff3n90o+6g0qaovRoxPVFo1KBz2FaLWTSovBqEggXVoZdQZxxy4UPH9AMJSxeCv8nvU68E9M3G+3dvXkbK+QJeUrWSUK2yS45IIekRwS5JF/Jd3IVfYq+RT+in9fRRjSfWSN/oLH0C+ISsuQ=</latexit><latexit sha1_base64="xgaNEjkEl6iZoQFEvQTxuPwBpk4=">AAACTHicdVFNbxMxFPQGaEv4aKBHLk9ESEVCK2/oR3qrWiR6LBJpK8Vp5PW+pKa2d7G90Gi1f7AXJG78Cy4cKELCSRNREIxkaTQzT34ep4WSzlP6JWrcun1naXnlbvPe/QcPV1uPHh+5vLQCeyJXuT1JuUMlDfa89ApPCotcpwqP0/P9qX/8Aa2TuXnrJwUONB8bOZKC+yANWxmsv94fVrI+1cB4Udj8AqbCu/rUPgf2EbMx3owYfP/bZwyY1GFNdPDqKHADrFpkXyxirB622jTudJPNrQRo/JLSbUoD6exsJd0OJDGdoU3mOBy2PrMsF6VG44XizvUTWvhBxa2XQmHdZKXDgotzPsZ+oIZrdINq1kYNz4KSwSi34RgPM/XmRMW1cxOdhqTm/sz97U3Ff3n90o+6g0qaovRoxPVFo1KBz2FaLWTSovBqEggXVoZdQZxxy4UPH9AMJSxeCv8nvU68E9M3G+3dvXkbK+QJeUrWSUK2yS45IIekRwS5JF/Jd3IVfYq+RT+in9fRRjSfWSN/oLH0C+ISsuQ=</latexit><latexit sha1_base64="xgaNEjkEl6iZoQFEvQTxuPwBpk4=">AAACTHicdVFNbxMxFPQGaEv4aKBHLk9ESEVCK2/oR3qrWiR6LBJpK8Vp5PW+pKa2d7G90Gi1f7AXJG78Cy4cKELCSRNREIxkaTQzT34ep4WSzlP6JWrcun1naXnlbvPe/QcPV1uPHh+5vLQCeyJXuT1JuUMlDfa89ApPCotcpwqP0/P9qX/8Aa2TuXnrJwUONB8bOZKC+yANWxmsv94fVrI+1cB4Udj8AqbCu/rUPgf2EbMx3owYfP/bZwyY1GFNdPDqKHADrFpkXyxirB622jTudJPNrQRo/JLSbUoD6exsJd0OJDGdoU3mOBy2PrMsF6VG44XizvUTWvhBxa2XQmHdZKXDgotzPsZ+oIZrdINq1kYNz4KSwSi34RgPM/XmRMW1cxOdhqTm/sz97U3Ff3n90o+6g0qaovRoxPVFo1KBz2FaLWTSovBqEggXVoZdQZxxy4UPH9AMJSxeCv8nvU68E9M3G+3dvXkbK+QJeUrWSUK2yS45IIekRwS5JF/Jd3IVfYq+RT+in9fRRjSfWSN/oLH0C+ISsuQ=</latexit>
(<x,y><height,width><text><image>)
Presentation Failures
{DV1, DV2, ...DVi} 2 {GCm
i , GCr
j }
PF 2 {Sm
, Sr
}<latexit sha1_base64="yXwZIsk0lnWWH3xnETRAGLTCsiI=">AAACVXicdVFdb9MwFHWyMUb5WIHHvVirkHioIifA1r1NDI09dhrdJtVd5LjOamY7wb5BVFH+JLzAT+FlmtO10kBwJVtH55x7bR9npZIOCPkVhGvrDzYebj7qPH7y9NlW9/mLM1dUlosRL1RhLzLmhJJGjECCEhelFUxnSpxn14etfv5VWCcL8wnmpZhodmVkLjkDT6VdTUF8A6trmTeYfqnYFNP6w1ka97Hfk34URS2Q1KvSeO3jYVrL5lL3W/C5ubS0obSzmgIzYVZzhkfLllPvxqetM+32SJQM4ne7MSbRG0L2CPEg2d+NBwmOI7KoHlrWMO1+p9OCV1oY4Io5N45JCZOaWZBciaZDKydKxq/ZlRh7aJgWblIvYmnwK89McV5YvwzgBXu/o2baubnOvFMzmLm/tZb8lzauIB9MamnKCoThdwfllcJQ4DZjPJVWcFBzDxi30t8V8xmzjIP/iY4PYfVS/H8wSqL9iJy87R28X6axibbRDnqNYrSHDtAxGqIR4ugH+h2EwVrwM7gJ18ONO2sYLHteoj8q3LoFJMWyDg==</latexit><latexit sha1_base64="yXwZIsk0lnWWH3xnETRAGLTCsiI=">AAACVXicdVFdb9MwFHWyMUb5WIHHvVirkHioIifA1r1NDI09dhrdJtVd5LjOamY7wb5BVFH+JLzAT+FlmtO10kBwJVtH55x7bR9npZIOCPkVhGvrDzYebj7qPH7y9NlW9/mLM1dUlosRL1RhLzLmhJJGjECCEhelFUxnSpxn14etfv5VWCcL8wnmpZhodmVkLjkDT6VdTUF8A6trmTeYfqnYFNP6w1ka97Hfk34URS2Q1KvSeO3jYVrL5lL3W/C5ubS0obSzmgIzYVZzhkfLllPvxqetM+32SJQM4ne7MSbRG0L2CPEg2d+NBwmOI7KoHlrWMO1+p9OCV1oY4Io5N45JCZOaWZBciaZDKydKxq/ZlRh7aJgWblIvYmnwK89McV5YvwzgBXu/o2baubnOvFMzmLm/tZb8lzauIB9MamnKCoThdwfllcJQ4DZjPJVWcFBzDxi30t8V8xmzjIP/iY4PYfVS/H8wSqL9iJy87R28X6axibbRDnqNYrSHDtAxGqIR4ugH+h2EwVrwM7gJ18ONO2sYLHteoj8q3LoFJMWyDg==</latexit><latexit sha1_base64="yXwZIsk0lnWWH3xnETRAGLTCsiI=">AAACVXicdVFdb9MwFHWyMUb5WIHHvVirkHioIifA1r1NDI09dhrdJtVd5LjOamY7wb5BVFH+JLzAT+FlmtO10kBwJVtH55x7bR9npZIOCPkVhGvrDzYebj7qPH7y9NlW9/mLM1dUlosRL1RhLzLmhJJGjECCEhelFUxnSpxn14etfv5VWCcL8wnmpZhodmVkLjkDT6VdTUF8A6trmTeYfqnYFNP6w1ka97Hfk34URS2Q1KvSeO3jYVrL5lL3W/C5ubS0obSzmgIzYVZzhkfLllPvxqetM+32SJQM4ne7MSbRG0L2CPEg2d+NBwmOI7KoHlrWMO1+p9OCV1oY4Io5N45JCZOaWZBciaZDKydKxq/ZlRh7aJgWblIvYmnwK89McV5YvwzgBXu/o2baubnOvFMzmLm/tZb8lzauIB9MamnKCoThdwfllcJQ4DZjPJVWcFBzDxi30t8V8xmzjIP/iY4PYfVS/H8wSqL9iJy87R28X6axibbRDnqNYrSHDtAxGqIR4ugH+h2EwVrwM7gJ18ONO2sYLHteoj8q3LoFJMWyDg==</latexit>

112. STAGE 2: GUI COMPREHENSION

113. STAGE 3: DESIGN VIOLATION DETECTION

114. GVT REPORT

115. GVT REPORT

116. HUAWEI COLLABORATION METHODOLOGY

117. HUAWEI COLLABORATION METHODOLOGY
• Bi-weekly meetings with designers and developers

118. HUAWEI COLLABORATION METHODOLOGY
• Bi-weekly meetings with designers and developers
• More than 12 releases of the tool, starting with a CLI

119. HUAWEI COLLABORATION METHODOLOGY
• Bi-weekly meetings with designers and developers
• More than 12 releases of the tool, starting with a CLI
• Acceptance tests designed by UI/UX experts

120. HUAWEI COLLABORATION METHODOLOGY
• Bi-weekly meetings with designers and developers
• More than 12 releases of the tool, starting with a CLI
• Acceptance tests designed by UI/UX experts
• Deployed to a controlled user base for feedback

121. THE GVT INTERFACE

122. THE GVT INTERFACE