This document proposes extending algorithmic skeletons with event-driven programming to address the inversion of control problem in skeleton frameworks. It introduces event listeners that can be registered at event hooks within skeletons to access runtime information. This allows implementing non-functional concerns like logging and performance monitoring separately from the core parallel logic. The approach is implemented in the Skandium skeleton library, and examples are given of a logger and online performance monitor built using it. An analysis shows the overhead of processing events is negligible, at around 20 microseconds per event.

1. Tackling Algorithmic Skeleton’s Inversion of Control
Gustavo Pab´on
Global Business Services, IBM Chile,
Av. Providencia 655, Santiago, Chile.
gpabon@cl.ibm.com
Mario Leyton
NIC Chile Research Labs, Universidad de Chile,
Miraflores 222, Piso 14, 832-0198, Santiago, Chile.
mleyton@niclabs.cl
Abstract
Algorithmic Skeletons are a high-level parallel program-
ming model which suffers from inversion of control. This pa-
per argues that extending Algorithmic Skeletons with Event-
Driven Programming (EDP) can reduce the negative side
effects produced by the inversion of control by: i) provid-
ing online information about the runtime execution without
lowering the skeleton’s high-level abstraction, and ii) fur-
ther improve the separation between functional code (mus-
cles) and non-functional concerns. The proposed approach
is used to implement a logger and an online performance
monitoring tool, and it is shown that the proposed exten-
sion incurs a small performance overhead.
1 Introduction
Algorithmic Skeletons[9] (skeletons for short) are a
high-level programming model for parallel and distributed
computing, introduced by Cole [4]. Skeletons take advan-
tage of recurrent parallel programming patterns to hide the
complexity of parallel and distributed applications. Start-
ing from a basic set of patterns (skeletons), more complex
patterns can be built by nesting the basic ones.
Event-driven Programming (EDP) is a programming
model where the program flow is determined by events.
EDP has been mainly used on interactive applications, like
graphical user interfaces, and has been successfully used
on real-time and embedded systems [12, 7] thanks to its
lightweight implementation and its signal (event) process-
ing nature.
This paper proposes a combination of both programming
models: Event-Driven Programming (EDP) and skele-
tons, with two objectives: i) provide online information
about the runtime execution without lowering the skele-
ton’s high-level of abstraction and ii) improve the separation
of concerns between functional code (muscles) and non-
functional concerns, in a high-level and efficient manner.
Skeletons use inversion of control [8] where the ac-
tual parallel execution flows are hidden from programmers
yielding a batch-like processing. Working with intermedi-
ate results and information about execution, to implement
non-functional concerns, lowers the programming model’s
high-level abstraction. This paper propose that by trigger-
ing events during skeleton execution, intermediate results
and runtime information can be sent to implement non-
functional concerns using event listeners.
Aspect Oriented Programming (AOP) [10] is the pre-
ferred and widely used programming model for the design
of a clear separation of concerns. This paper shows how
AOP concepts are introduced into Algorithmic Skeletons
using EDP. Both AOP and EDP allow the addition of cus-
tom code (advice code in AOP and event handlers in EDP)
in specific points on the control flow of a program (join
points in AOP and event hooks in EDP). We have chosen
to apply AOP concepts using EDP: i) because there is no
need to weave non-functional code as we can statically cre-
ate event hooks as part of the Skeleton Framework; ii) in
order to minimize programmer conceptual disruption (one
of the principles presented by Cole in its manifesto [5]), by
avoiding the necessity of adding another framework, and in
some cases another language.
Separation of concerns is essentially software engineer-
ing matter. The proposed separation of concerns recognizes
three different roles: i) skeleton provider: whose job it is
to materialize parallelism patterns into skeletons, ii) non-
functional provider: who implements other, non-parallel,
non-functional features, and iii) business programmer:
who uses both skeleton patterns and non-functional features
to implement the final business application.
We have implemented our proposal in Skandium [13,
11]. Skandium is a Java based Algorithmic Skeleton library
for high-level parallel programming of multicore architec-
tures. Skandium provides basic nestable parallelism pat-
terns, which can be composed to program more complex
applications.
This paper is organized as follows. In Section 2 related
work is presented. Section 3 discusses algorithmic skele-

2. ton’s inversion of control. The proposed approach is pre-
sented in Section 4. Results are presented in section 5 with
a logger and an online performance monitor, including an
overhead analysis. The paper finally concludes in section 6.
2 Related Work
In [1], Aldinucci et al, introduce muskel, a full Java
library providing a skeleton based parallel-and-distributed
programming environment implemented exploiting macro
data flow (MDF) technology, rather than implementation
templates. The predefined skeleton set in Muskel is com-
posed by farm and pipeline skeletons. Aldinucci et al pro-
pose annotations and AOP, to extend the possibility for
users to control some non-functional features in a high-level
way. The non-functional concerns discussed includes auto-
nomic managers (optimization), portability, security, paral-
lelism exploitation properties, and expandability.
The approach we present in this paper considers a wider
range of skeleton patterns and does not require the intro-
duction of another programming framework, like AspectJ,
which could potentially introduce conceptual disruption to
the programmer while we still apply the main concepts of
AOP.
Another Skeleton Framework that incorporates AOP is
SBASCO [6]. SBASCO includes the notion of aspect in
conjunction with the original paradigms. Aspects are im-
plemented as components and connectors, achieving in
this way an homogeneous and clean implementation since
skeletons are based on the same model. In the proposed
approach, however, we provide a Skeleton pattern specific
based extension rather than a tertiary generic abstraction.
In [3], such as Eden’s Trace Viewer [2], have focused
on post-mortem performance analysis and visualisation of
the skeleton’s execution. The approach presented in this pa-
per provides a high-level way of achieving online execution
analysis instead of a post-mortem one, as detailed in Sec-
tion 5.
3 Inversion of Control Problem
Inversion of control is a design pattern [8] where the
“main” code does not control the program’s execution flow.
Instead the framework (caller) receives the business code as
parameter and decides when and where it is executed. This
allows common and reusable code being developed inde-
pendently from problem-specific code, producing valuable
abstraction layers.
Algorithmic Skeleton Frameworks use these abstraction
layers to provide high-level parallelism exploitation pat-
terns, hiding the primitives and explicit scheduling of paral-
lel and distributed applications.
The cost of this high-level parallel programming abstrac-
tion is paid in control over the execution flow. This pro-
duces batch-like processing where intermediate results and
information about execution are hidden from programmers,
which can make handling non-functional concerns difficult.
Listing 1 shows an example of a common Skandium pro-
gram. Once execution starts there is no mechanism to get
information of the current execution or to get partial results.
// Skeleton definition
DaC<P,R> skeleton = new DaC<P,R>(
new SpecificCondition(...),
new SpecificSplit(...),
new SpecificExec(...),
new SpecificMerge(...));
// Input parameters
Future<R> future = skeleton.input(new P(...));
// do something else ...
// Wait for results
R result = future.get();
Listing 1. Batch-like Skeleton Processing
4 Events for Algorithmic Skeletons
This section describes the proposed events extension for
algorithmic skeletons. We begin by describing the sup-
ported Event Listener types, then we present the Event
Hooks where events listeners can be registered and finally
we describe how events are implemented in the execution-
level instructions.
4.1 Event Listeners
Programmers must implement any of the following in-
terfaces to create a new Event Listener. Each listener type
provides a different set of information, lowering the abstrac-
tion level on a need-to-know basis.
• SimpleListener<P> is the most high-level lis-
tener which specifies a guard/handler receiving a pa-
rameter P passed between skeletons and muscles at
some point in the computation.
• TraceListener<P> additionally specifies a
Skeleton[] trace containing as its first element the
root skeleton of a nested skeleton composition, and as
its last element the currently executed element. This
is useful when the event listener is registered in more
than one part of the skeleton nesting tree.
• IndexListener<P> additionally specifies an inte-
ger parameter as a result of context inferred indexes
such as in pipe stages; or map, fork and d&c splits.

3. Parameters Received Skeletons
Simple Listener
Trace Listener
Index Listener
Condition Listener
Generic Listener
Parameter P All Skeletons
+ Skeleton trace All Skeletons
+ Context inferred
indexes
pipe, for, while,
map, fork, d&c
+ Condition result if, while, d&c
+ Event hook identifiers
(when and where)
All Skeletons
Tabla 1. Versión 1
ay, November 18, 2011
Table 1. Comparison of Listener’s type
• ConditionListener<P> additionally specifies a
boolean parameter as a result of Condition muscles in
if, while and d&c skeletons
Table 1, listener’s type comparison, shows the different
parameters sent by type and shows the Skeletons supported
by each type.
Listing 2 provides an example of how a programmer may
implement an Event Listener using the given interfaces.
class MyListener implements TraceListener<Integer
>, ComparableListener{
@Override
public int compareTo(SkandiumListener l) {
// do something
return 0;
}
@Override
public boolean guard(Integer param, Skeleton[]
strace) {
if(...) { return true;}
return false;
}
@Override
public Integer handler(Integer param, Skeleton
[] strace) {
// do something
return param;
}
}
Listing 2. Example of Event Listener
implementation
Programmers also have the possibility of registering/re-
moving Generic Listeners. A Generic Listener can be regis-
tered in any Event Hook, and can be used to create common
functionality for several events, just like point cuts in AOP,
at the price of sacrificing type verification. Where the lis-
tener is applied on all nested skeletons, registered/removed
on all the event hooks that respect filter settings defined by
the following parameters:
Before After
Skeleton !
farm(!)
pipe(!1,!2)
seq(fe)
if(fc,!true,!false)
while(fc,!)
for(n,!)
map(fs,!,fm)
fork(fs,{!}, fm)
d&c(fc, fs,!, fm)
!: P !: R
!: P !: R
!i: Pi , i !i: Ri , i
fe: P fe: R
fc: P
!x: Px , x
fc: P , fc(P)
!x: Rx , x
fc: P , index
!: P , index
fc: P , fc(P) , index
!: R , index
!: P , index !: R , index
fs: P
!: Pi , i
fm: {Ri}
fs: {Pi}
!: Ri , i
fm: R
fs: P
!i: Pi , i
fm: {Ri}
fs: {Pi}
!i: Ri , i
fm: R
fc: P , index
fs: P , index
!: Pi , i , index
fm: {Ri} , index
fc: P , fc(P) , index
fs: {Pi} , index
!: Ri , i , index
fm: R , index
Friday, November 18, 2011
Table 2. Event hooks
• pattern filters by the Skeleton dimension (e.g.
Map.class, so the listener will be registered/removed
on all the Map events), the wild card is Skeleton.class.
• when can take the values before or after, the wild card
is null.
• where filters event hooks related to an specific skeleton
element, for example a condition muscle, split muscle,
nested skeleton, etc; the wild card is null.
4.2 Event Hooks
Event Hooks define the skeleton pattern based interest-
ing locations where additional code could be integrated.
Each pattern provides a different set of hooks, and each
hook applies to a specific set of Event Listener types.
Hooks are presented in a before/after semantics. For
example, all skeletons provide addBefore(...) and
addAfter(...) hooks for SimpleListener and
TraceListener, which are triggered before and after
the execution of the skeleton.
More specific hooks are also available depend-
ing on the pattern. For example, the if skele-
ton provides addBeforeCondition(...) and
addAfterCondition(...) hooks before and after
the execution of a condition. Note however that in this
particular case only the addAfterCondition(...)
hook accepts a ConditionListener<P> hook as the
boolean parameter only exists after the condition has
been evaluated.

4. Figure 1. Online Performance Monitor
Hooks for before and after nested skeletons are more
specific on each pattern. For example, pipe has two sub-
skeletons while if has two and the hooks definition differ
accordingly. However, the only type unsafe hooks corre-
spond to pipe’s nested parameter. This could be resolved,
however, by extending pipe’s definition from Pipe<P,R>
to Pipe<P,X,R> to account for the intermediary type.
The semantics of IndexListener’s index parameter
are skeleton sensitive. For example, in the for skeleton the
index corresponds to i-th iteration while in the map skeleton
the index corresponds to the i-th element of split’s result.
Table 2 shows the current set of event hooks imple-
mented in Skandium. The second and third columns shows
what parameters are sent for each stage of the skeleton’s ex-
ecution. The first row implies that all skeletons has a before-
and-after event hook. The parameters P and R corresponds
to the inpunt and output of each stage.
5 Results
By means of the event hooks and event listeners pre-
sented in section 4, the user has access to runtime execution
information on the fly, including intermediate results; and
non-functional concerns can be implemented with a clear
separation from the functional code (muscles).
5.1 Logger and Online Performance Mon-
itoring
As proof of concept of the approach proposed in this pa-
per, a logger and a visual performance monitor have been
developed.
The logger is a generic tool that could be activated
and deactivated on the fly during a skeleton’s execution.
The logger traces relevant information about the execution
which could be used to identify performance bottlenecks.
Figure 2. Absolute ovehead processing time.
The visual execution monitor is an extension of our pre-
vious work [3] which now provides online performance
monitor rather than post-mortem. As shown in Figure 1, for
each muscle and skeleton, the number of invocations and
processing time spent is updated continuously.
5.2 Overhead Analysis
In this section we show that the ovehead introduced by
the support of events is negligible.
Overhead analysis was made using a parallel QuickSort
algorithm implemented using one divide and conquer skele-
ton. Divide and Conquer skeleton generates a variable num-
ber of event hooks that is linearly proportional to the size of
the input. All tests were run on a dual core processor of
2.40GHz with 2GB of RAM.
The analysis compares the execution of the same prob-
lem with two different versions of Skandium, the one be-
fore the events support, and the one that includes events.
This analysis calculates the overhead caused only by the in-
clusion of the event instructions (event hooks) during the
transformation and reduction steps. Time presented is an
approximation of CPU time.
Figure 2 shows the result of the analysis; The x-axis cor-
responds to the number (in millions) of event hooks, while
y-axis shows the overhead calculated (CPU time with events
minus CPU time of without events). As result we can see
a linear increment with a slope of 19,80 seconds for each
million of event hooks created. Therefore the overhead is
19,80 microseconds per event hook.
Figure 3 shows the relative overhead processing time. X-
axis correspond to the problem size in MB. The light gray
part of each bar shows the percentage of total CPU time
used to solve the problem, and the dark gray part corre-
sponds to the percentage used to interpret the empty event
hooks.
This is the worst case behaviour expected since DaC is
the skeleton that generates the most event hooks propor-

5. Figure 3. Relative overhead processing time.
tional to the input size. Furthermore the implementation
of events in Skandium is itself an embarrassingly parallel
problem and thus in actual parallel execution this overhead
is linearly parallelized with respect to the wallclock over-
head time.
6 Conclusions
We have extended Algorithmic Skeletons with Event-
Driven Programming (EDP), by means of event hooks and
event listeners to i) provide access to users of runtime ex-
ecution information on the fly without lowering the skele-
ton’s high-level abstraction, and ii) improve on a clear sep-
aration of concerns between the non-functional aspects, im-
plemented using event handlers, and functional code (mus-
cles).
The proposed approach has been verified by implement-
ing a logger and a visual performance tool, and our imple-
mentation has been measured to have a negligent perfor-
mance overhead.
6.1 Future Work
The proposed approach could be used to implement a
wide range of non-functional concerns such as:
• Audit, control, and reporting. As well as the logger al-
ready implemented another controlling and reporting
tools could be built in order to address an online qual-
ity goals analysis (e.g. processor speed, memory, disk
space, network bandwidth, etc.)
• Error analysis. A full skeleton debugger could be im-
plemented. Building a tracer that increases/decreases
trace level online is possible by using the ability of reg-
ister and remove listeners dynamically.
• Security. Using the ability of manipulate intermediate
results, algorithms of encryption/decryption could be
used during communication.
References
[1] M. Aldinucci, M. Danelutto, and P. Dazzi. Muskel: an ex-
pandable skeleton environment. Scalable Computing: Prac-
tice and Experience, 8(4):325–341, Dec. 2007.
[2] J. Berthold and R. Loogen. Visualizing parallel functional
program runs: Case studies with the eden trace viewer. In
PARCO, volume 15 of Advances in Parallel Computing,
pages 121–128. IOS Press, 2007.
[3] D. Caromel and M. Leyton. Fine tuning algorithmic skele-
tons. In 13th International Euro-Par Conference: Parallel
Processing, volume 4641 of Lecture Notes in Computer Sci-
ence, pages 72–81. Springer-Verlag, 2007.
[4] M. Cole. Algorithmic skeletons: structured management of
parallel computation. MIT Press, Cambridge, MA, USA,
1991.
[5] M. Cole. Bringing skeletons out of the closet: a pragmatic
manifesto for skeletal parallel programming. Parallel Com-
puting, 30(3):389–406, 2004.
[6] M. D´ıaz, S. Romero, B. Rubio, E. Soler, and J. M. Troya.
Adding aspect-oriented concepts to the high-performance
component model of sbasco. In D. E. Baz, F. Spies, and
T. Gross, editors, PDP, pages 21–27. IEEE Computer Soci-
ety, 2009.
[7] B. P. Douglass. Doing hard time: developing real-time
systems with UML, objects, frameworks, and patterns.
Addison-Wesley Longman Publishing Co., Inc., Boston,
MA, USA, 1999.
[8] E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design
Patterns: Elements of Reusable Object-Oriented Software.
Addison-Wesley Professional, 1995.
[9] H. Gonz´alez-V´elez and M. Leyton. A survey of algorith-
mic skeleton frameworks: high-level structured parallel pro-
gramming enablers. Software: Practice and Experience,
40(12):1135–1160, 2010.
[10] G. Kiczales, J. Lamping, A. Mendhekar, C. Maeda,
C. Lopes, J.-M. Loingtier, and J. Irwin. Aspect-oriented
programming. In M. Aksit and S. Matsuoka, editors,
ECOOP’97 — Object-Oriented Programming, volume 1241
of Lecture Notes in Computer Science, pages 220–242.
Springer Berlin / Heidelberg, 1997. 10.1007/BFb0053381.
[11] M. Leyton and J. M. Piquer. Skandium: Multi-core pro-
gramming with algorithmic skeletons. In Proceedings of the
18th Euromicro PDP, pages 289–296, Pisa, Italy, Feb. 2010.
IEEE CS Press.
[12] M. Samek. Practical UML Statecharts in C/C++, Second
Edition: Event-Driven Programming for Embedded Sys-
tems. Newnes, Newton, MA, USA, 2 edition, 2008.
[13] Skandium. http://skandium.niclabs.cl/.