This document discusses the development of a tool that allows non-programmers to create "mashups" by using natural language and drag-and-drop interfaces. The tool was evaluated in two iterations with participants completing tasks like combining map and video APIs. Feedback was positive and most participants were able to complete given and self-generated integration tasks. The tool aims to balance expressiveness with intuitiveness through features like auto-completion, example categories, and live programming assistance. Future work includes standardizing component descriptions and exploring additional application domains.

1. Natural End-User Development of Mashups
Saeed Aghaee, Cesare Pautasso, and Antonella De Angeli
Email: saeed.aghaee@usi.ch
Twitter: @mr_wersa, @naturalmash
Website: http://naturalmash.com
VL/HCC 2013, San Jose, CA, USA
1
Faculty of Informatics
University of Lugano (USI)
Switzerland
DISI
University of Trento
Italy

2. Situational Applications
2

3. Situational Applications
3
Small group
of users
Good
enough
Short
life-span
Rapid
development

4. 4
Web 2.0 Mashups
Situational Applications

5. 5
Web-based
Mix reusable
components
Cost-
effective
Useful
Web 2.0 Mashups

6. 6

7. Develop-it-yourself
9
End-user mashup development for
everyone on the Web

8. 10

9. 11
Too specialized/complex for
non-professional users

10. 12
Too simplified

11. 13

12. 14

13. 15
“Intuitive
ness”
Expressive
ness

14. 16
Mashup development using natural
language and visual interaction

15. Description
Output mashup
Available ingredients
Used Ingredients
17

16. 18
Event-based (“when the map is clicked, ..”) and
sequential (“find youtube videos about title, and …”)
integration styles
Natural Language based Integration

17. 19
Every API has a short description with natural
language (autocompletion helps with discovery)
Natural Language Autocompletion

18. 20
Move and resize widgets
User Interface Design

19. Description
Direct manipulation
21
Programming by Demonstration (PbD)

20. 22
Live Programming Style

21. 23
Requirements /
Features
Expressiveness Intuitiveness
Natural Language
Programming
X X
Autocompletion X
UI Design (WYSIWYG) X X
PbD X X
Live Programming X
Main Features

22. 24
Demo
Shopping Assistant Mashup

23. 25
Architecture

24. 26
Compiler
Scanner
Formal parser + NL Parser
Semantic Extraction (Control Flow)
Semantic Extraction (Data Flow)
Code Emitter (JOpera)

25. User-centered Design and
Development
27
Two iterations are completed
Design
ImplementationEvaluation
Analysis

26. 28
Design
ImplementationEvaluation
Analysis
Two iterations are completed
User-centered Design and
Development

27. 29
Exploratory Evaluation: Goals
Main Goal:
Identify usability problems to help correctly drive
the design.
Sub Goal:
Assess the success of the design in meeting the
requirements (expressiveness and naturalnesss)

28. 30
Exploratory Evaluation: Participants
Sampling:
36 participants (iteration 1: 12, iteration 2: 24)
were selected by convenience method from high
school students, first year BSc students, and
university staff.
Background:
diverse background (20 non-programmers and
16 novice programmers)

29. 31
Exploratory Evaluation: Methods
Task-based
Questionnaire
Screen and session recording (Iteration 2)
Interview (Iteration 2)

30. 32
Exploratory Evaluation: Tasks
Pre-constructed task:
4 different (given) mashups
Open (self-generated) task
A mashup proposed by the individual participant

31. 33
Exploratory Evaluation: Process
Background
Assessment
Tutorial Tasks
End-of-
session
questionnaire
Interview

32. 34
Iteration 1 version

33. 35
Iteration 2 version

34. 36
Semi-structured Editor
completely eliminates the possibility of making
syntactical errors
Punctuations are added automatically, while they can also be typed by the user

35. 37
Ingredients Toolbar
shows a searchable list of available components
(ingredients)

36. 38
Component Dock
Shows which components (ingredients) are
used in the mashup

37. 39
Drag & Drop
Allows to add ingredients to the mashup being
edited by dragging them from the toolbar and
dropping it in the text field or visual field.

38. 40
Requirements /
Decisions
Expressiveness Intuitiveness
Semi-Structured editor X
Ingredients Toolbar X X
Component Dock X
Drag & Drop X
Feature Additions (Iteration 1)

39. 41
Requirements /
Decisions
Expressiveness Intuitiveness
Composition suggestion X X
Categorizes examples for
mashups
X X
Stateful live programming X
Feature Additions (Iteration 2)

40. 42
Exploratory Evaluation: Results
Task completion:
The majority (90%) of the participants (in both
iterations) completed the tasks
Satisfaction and user experience:
The majority of the participants felt satisfied and
positively about the tool.

41. 43
Exploratory Evaluation: Results
- A mashup that measures the level of online presence in tourism, using
social network APIs such as Twitter and Facebook.
- A mashup that enhances online shopping experience using eBay, Flickr,
and Facebook.
- A mashup that delivers location-based news with related media content
(video and image) using Google News, Youtube, Flickr, and Twitter.
Self-generated tasks:

42. 44
can

43. Natural End-User Development of Mashups
Saeed Aghaee, Cesare Pautasso, and Antonella De Angeli
Email: saeed.aghaee@usi.ch
Twitter: @mr_wersa, @naturalmash
Website: http://naturalmash.com
VL/HCC 2013, San Jose, CA, USA
45
Faculty of Informatics
University of Lugano (USI)
Switzerland
DISI
University of Trento
Italy

44. 46
Component Development
Technical part:
API protocol standards: REST, SOAP, etc.
Input/output data types
Technical-independent part:
Natural Language annotation
Semantic annotation of input/output data

45. 47
Mashup Use-cases
Business and financial analysis
Crisis management
Internet of Things
E-learning

46. 48
- HTTP+POX
- OAuth (1.0, 1.0a, 2.0)
- JavaScript+HTML Widgets
- SOAP
- JSON
- XML
Web API Technologies
User-based API ranking