Specification of a Visual Programming Language by Example

Institute for Software Technology – Maximilian Fellner 2013 1
Specification of a
Visual Programming
Language by Example
Behavior-Driven Development with Cucumber for Catrobat
Master’s Thesis by Maximilian Fellner

Institute for Software Technology – Maximilian Fellner 2013
Question:
What are the advantages of
standards and speciﬁcations?
2

Standardization
‣ Competitive advantages
‣ Strategic alliances
‣ Reduction of costs
‣ Safety & quality
‣ Public interest
3
Economic Beneﬁts of Standardization, Deutsches Institut für Normung, Beuth Verlag

Speciﬁcation
‣ Understand the requirements
‣ Describe a product abstractly
‣ Formulate binding contracts
‣ Quality assurance
4
Prerequisite for standardization

Existing approaches to
language speciﬁcation
‣ Natural language
‣ Formal semantics
‣ Reference implementations
‣ Test suites
‣ Syntax speciﬁed by context-free grammars and regular expressions
5

Existing approaches to language specification
‣ Specification documents, 
e.g., ECMA-262 (JavaScript)
‣ Readily understood by most
‣ Free choice of content and format
‣ Cannot be automatically verified
6
Natural language

‣ Can be difﬁcult to read and write
‣ Rarely used for consumer software
‣ Most precise, unambiguous, provable
‣ Tool support, e.g, VDM-SL
7
Formal semantics

‣ Are self-sufﬁcient
‣ Written in a programming language
‣ Possible bugs & unexpected behavior
‣ Dependency on implementation details
8
Reference implementations

‣ Specification consists of tests written in the
specified language
‣ Can define positive and negative test cases
‣ Should be executable by any implementation
‣ Nondeterministic behavior difficult to specify
9
Test suites

‣ Languages with greater abstraction
‣ No tool support for speciﬁcation
‣ Visual tests not feasible
10
Visual programming languages

Can we combine the
advantages of natural language
and those of test suites?
11
Question:

Speciﬁcation by Example
12
From business goals to executable speciﬁcation
‣ Introduced by Gojko Adzic
‣ Agile methodology
‣ Acceptance Test-Driven Development
‣ Behavior-Driven Development

Right Product
Built Right
Business
Failure
Success
Useless
Software
Maintenance
Difﬁculties
test-driven development
programming languages
frameworks & tools
speciﬁcation by example
project management
Success of a software product, adapted from Adzic [1]
business intent

Question:
How can tests better specify
the business intent?
14

Customer Tests
(or Acceptance Tests)
Business Intent
Usability Testing
Is it pleasurable?
Component Tests
Architect Intent
Exploratory Testing
Is it self-consistent?
Unit Tests
Developer Intent
Property Testing
Is it responsive, secure, scalable?
Per Functionality Cross-Functional
Business
Facing
Technology
Facing
The testing matrix, adapted from Meszaros [2]

Behavior-Driven
Development
‣ Derived from test-driven development
‣ Strong community interest & many tools
‣ Ubiquitous language
16
Speciﬁcation by example concretized

What is a ubiquitous language?
A common language for a
shared understanding of the
domain.
17
E. Evans, Domain-Driven Design: Tackling Complexity in the Heart of Software [3]

Title (one line describing the story)
!
Narrative:
As a [role]
I want [feature] So that [benefit]
!
Acceptance Criteria: (presented as Scenarios)
Scenario: Title
Given [context]
And [some more context]…
When [event]
Then [outcome]
And [another outcome]…
D. North. What’s in a story? [4]

Cucumber
‣ Open-source project
‣ Ubiquitous language Gherkin
‣ Available for Ruby and JVM languages
19
Behavior-driven development tool

Describing a Calculator with Gherkin
Feature: Addition
Scenario: Adding two numbers
Given I enter 1 into the calculator
And I enter 2 into the calculator
When I press add
Then I should see 3 on the display
20

Given /Î enter (d+) into the calculator$/ do |n|
@stack.push n.to_i
end
21
Then /Î should see (d+) on the display$/ do |n|
@result.should == @result
end
When /Î press add^$/ do
while not @stack.emtpy? do
@result += @stack.pop
end
end

Project
Features
Scenarios
Steps
Step Deﬁnitions
Support Code
Automation Library
System
Business Facing
Technology Facing
The Cucumber testing stack, adapted from Hellesoy and Wynne [5]

Cucumber
‣ Gherkin is simple but effective
‣ Steps are platform independent
‣ Deﬁnition methods are reusable
‣ Redundant code is reduced
23
Advantages of steps & step deﬁnitions

However, there is a problem…
They do not support Ruby or the JVM!
®
®

Existing approaches to
Cucumber on mobile
‣ Open-source software by Xamarin Inc.
‣ Extends Ruby variant of Cucumber
‣ Supports Google Android & Apple iOS
25
E.g., Calabash

developer’s computer
or build server
Ruby client library
& step deﬁnitions
Cucumber features
device or emulator
Calabash HTTP server
& native test runner
Mobile Application
Calabash system architecture

Existing approaches to Cucumber on mobile
‣ Ruby installation required
‣ Only pre-deﬁned operations available
‣ Step-deﬁnitions are written in Ruby
‣ No integration with native libraries
27
Disadvantages

Cucumber JVM on Android
‣ Use Java implementation of Cucumber
‣ Integrate with Android test framework
‣ Run Cucumber directly on the device
28
A different approach

Overview of the Android test framework
application package (.apk)
InstrumentationTestRunner
test package (.apk)
test tools
monkey runner
test case classes
Instrumentation JUnit
mock objects
process

Class diagram of the Cucumber-Android module
Instrumentation
CucumberInstrumenation Runtime
RuntimeOptions
AndroidResourceLoader
ClassLoader
JavaBackend1..* DexClassFinder
AndroidObjectFactory
ResourceLoader
ClassFinder
ObjectFactory1
1
1
1
1
1
1

Cucumber JVM on Android
‣ Still no solution for Windows Phone and iOS
‣ Best option is to improve Cucumber
‣ Gherkin parser is based on Ragel
‣ Cucumber runtime can be ported to C
31
Remaining issue

The visual language
Catrobat
‣ Android, Windows Phone, iOS, Web
‣ Phones, tablets & desktop browsers
‣ Inspired by Scratch
‣ Sophisticated bricks (blocks)
32
Cross-platform, mobile, visual

The visual language Catrobat
33
Pocket Code (Android) and Scratch 2.0

From requirements to
ﬁnished product
‣ Team meetings and discussions
‣ Write requirements as story cards
‣ Implement stories using Kanban & TDD
34
Before

weekly sub-
team meeting
bi-weekly team
meeting
planning game create new stories
discuss implementation
get feedback
write unit
tests

From requirements to ﬁnished product
‣ Stories are imprecise
‣ Story cards are archived or thrown away
‣ No automatic validation
‣ Android implementation serves as the
model for other teams
36
Problems

‣ Translate stories into features and scenarios
‣ Share feature ﬁles with everybody
‣ Each team implements their own step-
deﬁnitions independently
37
With Cucumber

weekly sub-
team meeting
bi-weekly team
meeting
planning gamecreate new stories
discuss implementation
get feedback
write unit
tests
write
cucumber
features

‣ Features and scenarios are precise
‣ Feature ﬁles are stored in SCM system (Git)
‣ Automated validation of speciﬁcations
‣ Implementation-agnostic model for all
teams
‣ Better synchronization between teams
39
Advantages

Specifying semantics of a
visual language
‣ Android implementation in Java
‣ Furthest developed variant
‣ Existing set of unit tests
40
Catrobat & Pocket Code

Creating a new Catrobat program
Background:
Given I have a program
And this program as an object ‘Object’
41

@Given(“^I have a Program$”)
public void I_have_a_program() throws IOException {
ProjectManager pm = ProjectManager.getInstance();
pm.initializeNewEmptyProject(“Cucumber”,
getContext());
}
42
Step deﬁnitions should be self-contained

@And(“^this program has an Object ‘(w+)’$”)
public void program_has_object(String name) {
ProjectManager pm = ProjectManager.getInstance();
Sprite sprite = new Sprite(name);
pm.getCurrentProject().addNewObject(sprite);
Cucumber.put(Cucumber.KEY_CURRENT_OBJECT, sprite);
}
43
Step deﬁnitions can require a mechanism to share state

Behavior-Driven Development asks:
How will the system behave
when a user interacts with it?
44

How will a Catrobat program
behave when it is executed?
45
The language speciﬁcation should answer:

Specifying a loop (Repeat brick)
Feature: Repeat brick
A Repeat Brick should repeat another set of bricks
a given number of times.
!
Background:
...
!
Scenario: Increment variable inside loop
Given ‘Object’ has a Start script
And this script has a set ‘i’ to 0 brick
And this script has a Repeat 8 times brick
And this script has a change ‘i’ by 1 brick
And this script has a Repeat end brick
When I start the program
And I wait until the program has stopped
Then the variable ‘i’ should be equal 8
46

Specifying a loop (Repeat brick)
Feature: Repeat brick
A Repeat Brick should repeat another set of bricks
a given number of times.
!
Background:
...
!
Scenario: Increment variable inside loop
Then the variable ‘i’ should be equal 8
47
Add script to Object
for 8 times do
i = i + 1;
end

@Then(“^the variable ‘(w+)’ should be equal (d+.?d*)$”)
public void var_should_equal_float(String name, float expected) {
Sprite object = (Sprite) Cucumber.get(Cucumber.KEY_CURRENT_OBJECT);
Project project = ProjectManager.getInstance().getCurrentProject();
UserVariable var = project.getUserVariables()
.getUserVariable(name, object);
assertNotNull(var);
float actual = var.getValue().floatValue();
assertThat(actual, equalTo(expected));
}
48
Step deﬁnition for equality assertion

Automatically generated report

Using the loop
(Repeat brick)
Altering
position and
transparency

Using the loop
(Repeat brick)
Question of
performance

Specifying performance of a loop
Scenario: No more than 100 iterations in 2 seconds
And this script has a set ‘k’ to 0 brick
And this script has a Wait 2 seconds brick
And this script has a set ‘k’ to ‘i’ brick
!
!
Then the variable ‘k’ should be less than or equal 100
52

Deﬁning an upper bound for iterations per second
Scenario: No more than 100 iterations in 2 seconds
And this script has a set ‘k’ to 0 brick
And this script has a Wait 2 seconds brick
And this script has a set ‘k’ to ‘i’ brick
!
!
Then the variable ‘k’ should be less than or equal 100
53
ﬁrst Start script
second Start script
store value of i after 2 s.
increment i by 1
1 loop should take at
least 20 ms

Specifying concurrency control (Broadcast Wait)
Scenario: Broadcast Wait brick sends message
in program with two When scripts
And this script has a BroadcastWait ‘hello’ brick
And this script has a Print brick with ‘-S1-’
Given ‘Object’ has a When ‘hello’ script
And this script has a Wait 100 milliseconds brick
And this script has a Print brick with ‘-W1-’
Given ‘Object’ has a When ‘hello’ script 
Then I should see the printed output ‘-W1--W2--S1-’
54

Specifying concurrency control (Broadcast Wait)
Scenario: Broadcast Wait brick sends message
in program with two When scripts
And this script has a BroadcastWait ‘hello’ brick
And this script has a Print brick with ‘-S1-’
Given ‘Object‘ has a When ‘hello’ script
Given ‘Object’ has a When ‘hello’ script 
Then I should see the printed output ‘-W1--W2--S1-’
55
Start script initiates
two When scripts
ﬁrst When script
runs ≥ 100 ms
second When script
runs ≥ 200 ms
output shows
execution order of
scripts

‣ Functional
‣ Expected output of a program
‣ Object-oriented
‣ Properties of objects (attributes & methods)
‣ Hardware-oriented
‣ Comparable performance
‣ Measured values from sensors
56
Specifying semantics of a visual language

Conclusion
‣ Working implementation on Android
‣ Cross-platform with Cucumber is feasible
57
Cucumber for the Catrobat language

‣ Cucumber extends present test-driven
approach
‣ New features can be developed inside
existing project workﬂow
‣ Features and steps can be shared between
teams working on different platforms
58
Advantages
Conclusion

‣ Features and scenarios need to be well-
formulated
‣ When does a certain property correspond
to a feature and when to a scenario?
‣ Step deﬁnitions need to be implemented
per-platform and depend on existing code
59
Issues with Cucumber features
Conclusion

‣ Adopt Cucumber throughout all Catrobat
sub-teams
‣ Port Cucumber to C or C++
‣ Enhance development process with
continuous integration & cucumber.pro
60
Future work
Conclusion

Bibliography
1. G. Adzic. Speciﬁcation by Example: How Successful Teams Deliver the Right Software. Manning Pubs
Co Series. Manning, 2011. isbn: 9781617290084.
2. G. Meszaros. XUnit Test Patterns: Refactoring Test Code. A Martin Fowler signature book. Addison
Wesley Professional, 2007. isbn: 9780131495050.
3. E. Evans. Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison Wesley
Professional, 2004. isbn: 9780321125217.
4. D. North. What’s in a story? Online; accessed 2013-10-05. url: http: //dannorth.net/whats-in-a-story.
5. A. Hellesoy and M. Wynne. The Cucumber Book: Behaviour-Driven Development for Testers and
Developers. Pragmatic Programmers. Pragmatic Book- shelf, 2012. isbn: 9781934356807.

Specification of a Visual Programming Language by Example

More Related Content

What's hot

Viewers also liked

Similar to Specification of a Visual Programming Language by Example

Recently uploaded

Specification of a Visual Programming Language by Example