Process, design, implementation and evaluation of a mobile collaboration layer

Process, design, implementation and
evaluation of a mobile collaboration layer
Mauro Carlos Pichiliani
PhD student
Prof. Dr. Celso Massaki Hirata
Advisor
Prof. Dr. Prasun Dewan
CoAdvisor
INSTITUTO TECNOLÓGICO DE AERONÁUTICA - ITA
Electronic Engineering and Computing - EEC/I
Department of Computer Science
Brazil

Monday, 7th March 2016
Process, design, implementation and evaluation of a mobile collaboration layer
2PhD presentation - Mauro Carlos Pichiliani
Goal
This work proposes a specification,
design, and evaluation of a software
layer that employs a combination of
techniques to prototype
collaborative mobile applications
The approach reduces the
development effort while keeping
the possibility for future ad hoc
customizations

Overview
– Introduction
– Related work
– Lacomo specification and design
– Implementation
– User study
– Results
– Discussion
– Conclusions & future work

Introduction

Introduction
 Online stores with thousands of applications
 Few apps allow synchronous collaboration
 Vendors provide SDK
 Implementation of synchronous ad hoc collaboration is
complex and costly (require specific application design)
 Thesis statement:
It is possible to create prototype
collaborative mobile applications with an
event notification layer that require less
development effort than previous
techniques that create or adapt
applications to support collaboration?

Motivation – Draw Something
 Simple collaborative drawing app
 Social features
 OMGPOP studio purchased by Zynga
 180 million dollars acquisition
 35 million registered users
 10 million active users
 1 billion views of advertisements

Motivation Scenarios
 MS1: Understanding calculator operations
 MS2: Form filling
 MS3: Sketching drawings

Research questions
 RQ1 (Problem contextualization): Is there a place on the design space
of previous work to support synchronous collaborative requirements
that overcome limitations and fulfills the requirements of the existing
approaches?
 RQ2 (Application adaptation): How to adapt existing mobile
applications so that they implement a subset of synchronous
collaborative requirements without changing source code?
 RQ3 (Effort comparison): How much effort is required to change a
mobile app to support collaborative requirements? Given our
approach is based on event notification, what is the effort to use it
compared to ad hoc implementation?

Contributions
 A model and a semi-automatic process to modify
existing applications
 A software engineering technique based on a layer
(Lacomo: Layer to develop collaborative mobile
applications)
 Implementation of Lacomo using the Android platform
 Four prototypes created using Lacomo
 A user study to evaluate the effort to use Lacomo

Related Work

Mobile Software Engineering
 SE techniques can be easily applied to the mobile application domain
 Survey of mobile app developers (WASSERMAN, 2010b):
 Adherence to best practices
 Rarely use formal development process
 Gather few metrics
 Novel factors of mobile application development (DEHLINGER; DIXON,
2011):
 User interface provides new human-computer interaction
 Divergent mobile platforms
 Challenges and opportunities of mobile computing
 Main challenges (JOORABCHI; MESBAH; KRUCHTEN, 2013):
 Platform fragmentation
 Monitoring
 Analysis and testing support
 Openness of development platform
 Frequent SDK changes
 Code reusability

Mobile prototyping
 Prototype: partial simulation of a product with respect to its final
appearance and behavior (HOUDE; HILL, 1997)
 Classification on interface and design detail:
 Low fidelity (pen and paper)
 High fidelity (supported by specialized tools)
 High fidelity prototypes enable more relative usability problems to be
explored and discovered
 Current prototype tools lack resources to mockup or prototype
collaboration requirements on top of existing applications
 Hypothesis: it is possible to experiment with collaboration
requirements with Lacomo with less development effort than
traditional development techniques

Colaboration requirements
 Automation
 How to automate the development process of an existing class of applications
 Constraints: Amount and complexity of code x Effort to multi-user behavior
 Flexibility
 Ability to support a variety of collaborative applications and scenarios (ROUSSEV,
2003)
 Trade off between Automation and Flexibility
 Code reuse
 Amount of code not newly written to support collaboration
 Affects Flexibility
 Extensibility
 Aspects associated with modifications (customizations and adaptations)
 Clarifies how a developer takes any part of the system and replaces it with
customizations

Architectures for collaboration
 “Generic collaborative architectures structure the components of a groupware
into a variable number of layers with functional aspects and impose few
assumptions on the nature of the applications they support” (DEWAN, 1995)
 Architectures for collaborative applications extend single-user architectures
(DEWAN, 1995)
 MVC (Model View Controller) divides the application into three parts:
 Controller: input handling
 View: output and user interface
 Model: underlying application and data
 Combination of MVC components in collaborative applications (SUTHERS,
2001):
 Centralized
 Hybrid
 Distributed
 Replicated

Design Space
Low level
systems
High level
systems

Low level systems
 Collaboration by sharing granular abstract applications elements
(documents, tuples, screen or windows)
 Repository, database, and document-based:
 DFS
 Bayou
 Lotus Notes
 Screen and windows sharing:
 XTV
 Shared X
 VNC
 ITA - Intelligent Transparent Adaptation

High level systems
 Collaboration at different levels
 Often require the application source code
 Component-based: JViews, Live, COCA, MoCA, CoCoWare,
Prospero, Flexible JAMM, Habanero, JCE
 Direct access to source code:
 Architecture: Rendevouz
 Toolkits: MAUI, GroupKit
 Frameworks: TouchSync, COCA, EXEC Framework, Flexible
Sharing
 Other: Collab, TACT, Coda, Sync
 Transparent Adaptation requires an API of the target application to
promote collaboration

Answer to RQ1
RQ1: The location on the design space of previous work
that supports collaborative requirements to overcome
limitations and fulfill the requirements further than
existing approaches is at the intersection of the Wide
range of apps value of the Flexibility axis with the Event
notification value of the App. Knowledge axis. This area
can be explored by employing event notification
resources available from the accessibility API of the
operating system combined with screen sharing
and UI testing technologies

Lacomo specification
and design

Model
 The Multi-user collaborative MVC Model

Process
 The MVC UI Component Modification Process

Problem domain formalization
 P = application; E = set of possible user interaction events; F = set of programming
abstractions; A = set of interactions captured by F; U = set of users interacting with P
 User interaction event sequence seq = [e1,e2,…,en], where each ei ϵ E is a user
interaction event. Set: SEQ
 Generic API I = [c1, c2,…,cn], where ci is on ordered pair (ej,fk) that represents a user
interaction event ej ϵ E captured by a programming abstraction fk ϵ F. Set: SEQF
 Accessible user interaction event sequence seqa = [a1, a2,…,an], where ai ϵ A is a
user interaction event captured by the API I. Set: SEQA
 Multi user interaction event sequence seqam = [m1, m2,…,mn], where mi is on ordered
pair (aj,uk) that represent an event aj ϵ A performed by a user uk ϵ U. Set: SEQAM
 Operations:
 len(seqam) returns the length of the sequence seqam
 head(seqam,k) returns a subsequence with the first k events in seqam.
 Execution t = exec(P,seqam). Set T = {exec(P,seqam) | seqam ϵ SEQAM}.
 Initial state s0. State s’ = 〈s0,seqam〉. Space state explored:
St = {〈s0,head(seqam,k)〉 | 1 ≤ k ≤ len(seqam)}

Specification
 Design recommendations for APIs (LIN et al., 2007):
 DR1: Event interception
 DR2: Event access
 DR3: Event differentiation
 DR4: Event generation
 DR5: Event action origin
 Items of the Lacomo specification:
 Direct read access to the OS System log
 Read/Write access to the current user screen
 Interfaces to capture events by a trigger model (DR1, DR2, DR5)
 Navigation and read access to UI hierarchy of elements
 High-level mechanism to inject user interactions
 Resources for application rewriting
 Private rules to control data sharing (DR3, DR4)

Design
 Almost all UI components found on mobile platforms are accessible by
default
 95% of apps are built with default SDK components (ZHOU et al.,
2012)
 Lacomo design is based on an OS layer that has low-level hooks to
capture events
 The design combines existing techniques (accessibility, screen
sharing, UI testing, and application rewriting)

Anwser to RQ2
 Subset of synchronous collaborative requirements: sharing of UI
control’s data to prototype collaboration requirements on existing
mobile apps
 RQ2: To adapt existing mobile applications so that they implement a
subset of synchronous collaborative requirements without changing
the source code we propose Lacomo, a software layer that uses an
API that capture user events, screen sharing technology, and UI
testing techniques that communicate with an existing application by
capturing event data and replaying it at the UI controls of the users

Implementation

Accessibility API
 Get low-level information about targets. E.g.: MSAA API
 Operating systems have accessibility applications (screen readers,
magnification glass)
 Android platform provides a complete accessibility API:
 Low-level hooks that capture events
 Complete identification of the element
 Reconstruction of the UI View hierarchy
 Default textual description
 New accessibility service creation
 Integration with external devices (e.g. braille keyboards)
 Developers can create accessibility services that users must activate
on the device

Accessibility Evaluation
 Evaluation of 10 most popular applications (Feb. 16th, 2015)
 Event trigger Rate and Content Rate

Technology
 Android accessibility service created from the class AcessibilityService
 Method onAccessbilityEvent() fired at every event
 Access to UI hierarchy, app and device state
 ClientCapture and ClientReceive classes with the SendEvent() method
 Screen sharing
 Read pixels on the screen
 Transparent “window” on top of the UI to write pixels
 UI Testing
 Navigate on UI widgets
 Simulate local user interaction
 Application rewriting
 Manually change source code
 Decompile, find source code areas, inject code, recompile

Prototypes
 CoPortals
 CoCalculator
 CoSudoku
 CoTomdroid

User study

Design and setup
 EH1: Lacomo requires less effort than the ad hoc approach
 Within participant design (18 male, 2 female). Avg. age: 28.6±7
 Requirement: >= six months of experience with Android development
 $5 reward + ebook + 2 on-line course access + gift raffle
 Procedure:
 Consent form
 Pre and pos-experiment questionnaires
 Training with the approach (Lacomo or ad hoc) for UI sharing
 Task session (max 60 min.) to change calculator app
 Debriefing interview
 Instrumentation: face, desktop and audio recording, custom Eclipse plug-in,
BCI, cardiac monitor (BPM)
 Environment: Windows 8.1 + Eclipse + 2 Android tablets + WiFi network (no
internet access)

Participants

Threats to validity
 Representativeness of application:
 Open source app available at Google Play store
 Generalization of participants:
 Experienced industry developers
 Wide range of expertise
 Data collection and inspection:
 Independently inspected by experienced researcher
 Questions to the monitor:
 Limit the content of the answer
 Artificial motivations:
 Explicitly mention the evaluation of the tool, not the person
 Contribution regardless of recompenses

Demonstration (UI sharing)

Results

Quantitative results
 Profile: 65% have app on store; 70% developed multi-
user app; 100% prototype
 Task completion: 90% (Lacomo) x 20% (ad hoc)
 Time (min.) average: 27.7±18.9 (Lacomo) x 59±3.1 (ad
hoc)
 LOC = physical lines added + modified + removed
 Effort: LOC, LOC divided by time, calories, distance
covered by the mouse pointer, save events

Statistics (α=0.05)
 Confirm the EH1 (95% confidence level) on all five variables
Metric Mean
Value
Normality
(Kolmogoro
v-Smirnov)
Distribution
(Two-tailed
Mann-Whitney)
Variance
(F-Test)
Average
(Non pared
T-Test)
Avg. LOC µlacomo=11.3±9.2
µadhoc==158.1±134.6
Normal
distribution
placomo=0.48
padhoc=0.35
Different
distributions
min(u1,u2) = 0
Same
variance
(f=0.001)
Different
average
t=-3.44
p=0.0029
Avg.
LOC/Time
µlacomo=0.51±0.3
µadhoc==2.6±2.2
Normal
distribution
placomo=0.41
padhoc=0.33
Different
distributions
min(u1,u2) = 7
Same
variance
(f=0.02)
Different
Average
t=-2.99
p=0.0078
Avg.
Calories
µlacomo=97.1±86
µadhoc==243.3±135
Normal
distribution
placomo=0.11
padhoc=0.46
Different
distributions
min(u1,u2) = 16
Different
variance
(f=0.40)
Different
average
t=-2.88
p=0.0112
Avg.
Mouse
Movement
µlacomo=224,841.1
±164,467.2
µadhoc==580,471.6
±222,677.1
Normal
distribution
placomo=0.17
padhoc=0.82
Different
distributions
min(u1,u2) = 10
Different
variance
(f=0.54)
Different
average
t=-4.06
p=0.0008
Avg. Save
events
µlacomo=6.7±3.6
µadhoc==29.1±26.7
Normal
distribution
placomo=0.13
padhoc=0.25
Different
distributions
min(u1,u2) = 22
Different
variance
(f=0.02)
Different
average
t=-2.49
p=0.0235

Normalized means

Line edits

Line edit operations

Attention, meditation, status messages

IDE events

Qualitative results

Subjective mental effort
 Responses to mental effort are significantly different (p<0.05)
 High level of correlation with task completion:
 Strong positive correlation (r=0.76) with difficulty
 Strong positive correlation (r=0.75) with mental effort
 Very strong positive correlation (r=0.90) with concentration

PU and PEOU

Answer to RQ3
 RQ3: The effort required to change a mobile app to
support collaborative requirements is, on average, 27.7
minutes, 11.3 lines of code, 0.51 lines per minute, 97.1
calories, 224,841.1 pixels traveled by the mouse, and
6.7 save events. The effort to use an event notification
approach compared to ad hoc implementation required,
on average, 64% less time, 95% less lines of code,
78% less lines per minute, 64% less calories, 69% less
pixels traveled by the mouse, and 85% less save events

Discussion

Design comparison
 Lacomo’s design is similar to previous work at some abstraction level
 Variation on how to acquire and replay data: event based
 Handle collaboration as an operating system-oriented approach
 Phases of update handling:
 Flexible Sharing (ROUSSEV, 2003) and the Exec Framework (LI; LI, 2002 )
listen to property changes and uses value assignment. No context for:
 User interactions
 System event notification
 Cascading property changes
 Lacomo can be complemented by previous work

Feasibility and convenience
 Consider resources and effort to adapt legacy applications
 Source code modifications may insert faults and bugs
 Complete re-design may be error-prone and time-consuming
 Lacomo allows the fast creation of prototypes with synchronous
collaborative features
 How much effort savings depends on context, language, and
development environment
 Non-functional requirements must also be addressed:
 User privacy
 Data security
 Adoption
 Social elements

Implementation usage & comparison
 Previous work seldom provides implementation effort
 Lack of empirical effort to mobile collaboration
development in the literature
 Challenges to compare techniques to implement
synchronous collaboration
 Basic LOC values
 Context of effort comparison
 LOC values only to create new applications
 No current OS addressed (iOS, Android, Windows
Phone)

Lacomo limitations
 Lacomo is limited by its underlying technologies:
 Accessibility API
 Screen Sharing
 UI testing
 Application rewriting
 Lacomo does not have automated or semi-automated features to
share Model’s data
 No current accessibility API allow the complete implementation of the
Lacomo design
 The Android accessibility API is under active development by Google
 Changes in technology highlights the advantages of model and
implementation separation of Lacomo
 Future changes in base technologies potentially improve the features
of Lacomo

Conclusions &
future work

Conclusions
 An event notification technique can be explored to create high fidelity
mobile prototype applications with collaboration features
 A design that combines accessibility API, screen sharing, UI testing,
and application rewriting techniques
 The implementation of Lacomo on the Android platform can create
prototypes on top of existing applications without source code
changes
 A user study proved that Lacomo required less effort compared to the
ad hoc implementation, when effort is measured by LOC (lines of
code), LOC divided by time, calories, mouse movement, and save
events metrics
 20 variables observed on +/- 30 hours of experiments with application
developers over 8 months

Future work
 Refine the Lacomo design and implement it on Android
or other mobile OS (CyanogemMod or Tizen)
 Creation of other prototypes with Lacomo (map
navigation, music player)
 Further analyze the quantitative and qualitative data
gathered in the user study for future Software
Engineering research

Acknowledgements
Thank you!
e-mail mauro@pichiliani.com.br

Process, design, implementation and evaluation of a mobile collaboration layer

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (9)

Similar to Process, design, implementation and evaluation of a mobile collaboration layer

Similar to Process, design, implementation and evaluation of a mobile collaboration layer (20)

Recently uploaded

Recently uploaded (20)

Process, design, implementation and evaluation of a mobile collaboration layer