Doctoral degree (PhD) proposal submitted back in 2014 [...] While the underlying theme and idea remained the same, some aspects of the proposal were substantially changed during the course of the Ph.D.

1. Software Tools for Orchestration
Doctoral Research Proposal
Lighton Phiri, Graduate Student
lphiri@cs.uct.ac.za
Prof. Dr. Christoph Meniel, Supervisor
christoph.meinel@hpi.de
Assoc. Prof. Hussein Suleman, Supervisor
hussein@cs.uct.ac.za
Department of Computer Science
Faculty of Science
University of Cape Town

2. Software Tools for Orchestration
Lighton Phiri‡
Supervised by
Prof. Dr. Christoph Meinel††
Assoc. Prof. Hussein Suleman‡‡
August 2014
1. INTRODUCTION
This proposed research is aimed at exploring technology-driven orchestration—the teacher-centric
real time management of in-classroom activities (Roschelle et al., 2013)—, in order to understand
whether/how the organising of orchestrated activities affects educators’ effectiveness. Specifically, the
research seeks to investigate whether the streamlining of well-organised in-classroom activities can
potentially result in the increase in effectiveness of teachers within the constantly evolving modern-day
classroom environment. By its nature, orchestration involves a number of multi-constrained activities
which if appropriately managed could potentially result in the increase in effectiveness of educators in
these complex spaces.
It is premised that by comprehensively understanding classroom orchestration and indeed the unique
challenges faced in formal learning environments, it becomes possible to appropriately structure
in-classroom activities, effectively making it possible for educators to be more efficient and effective
in the classroom.
To this end, a software orchestration workbench is proposed as a potential alternative technology for
organising and streamlining in-classroom activities, in order to overcome the challenges associated
with classroom orchestration. Envisioned as a framework with a layered suite of software tools and
services, the orchestration workbench is aimed at providing a single unified environment, capable of
integrating and centralising access to different tools and services.
1.1 Problem statement
Supporting educators within the classroom is considered one of the most effective ways of making
formal learning spaces effective learning environments. One way of achieving this is through the
facilitation of classroom orchestration. However, contemporary orchestration is considered complex,
challenging (Dillenbourg, 2013) and, arguably, ad hoc due to the variety of tools and services used to
orchestrate activities. The complexity and challenge is the direct result of the multi-faceted nature of
orchestration and, additionally, the constraints associated to orchestration. Furthermore, the varying
orchestration levels, shown in Figure 1, present additional challenges.
A key concern in facilitating orchestration is how best to handle the core orchestration aspects outlined
below.
‡
lphiri@cs.uct.ac.za
††
christoph.meinel@hpi.de
‡‡
hussein@cs.uct.ac.za
1

3. Opening
Phase
Instruction
Phase
Closing
Phase
Individual Level Orchestration
Group Level Orchestration
Class Level Orchestration
Individual
Group
Class
Figure 1. Organisation of classroom orchestration elements
1.1.1 Orchestration aspects
Activity management There is a range of activities that take place within the classroom, with the
majority of them involving display and visualisation of information. However, there are some activities
that might require additional actioning. Thus, there ought to be a reliable way of performing appropriate
actions associated with individual activities.
Activity management Orchestration typically takes place in a linear fashion, with planned activities
sequentially performed within set time-frames. While the sequence of activities is in most cases fixed,
the sequencing could possibly be performed dynamically; an example here would be swapping start
time-frames of a particular course. It is imperative, therefore, that all possible sequencing approaches
be taken into account.
Time management Time is a finite resource, and is the most critical constraint during orchestration.
Owing to the multiple activities that tend to be allocated individual time slices, teachers typically plan
ahead to ensure that time is appropriately allocated. A possible way of efficiently using the limited
time would be to devise a non-invasive technique of indicating the time-lapse.
1.1.2 Orchestration workbench
In helping resolve the issues highlighted in Section 1.1, and indeed taking into account orchestration
aspects mentioned in Section 1.1.1, the integrated orchestration workbench is envisaged to provide the
following benefits:
Flexibility
Easy to use
Agnostic to ICT integration challenges
Figure 2 shows the conceptual design of the orchestration workbench.
2

4. AppLauncher
Orchestration Tools& Services
ContentViewer Backchannel ...
Orchestration Workbench UI
Figure 2. The orchestration workbench platform conceptual design
1.2 Motivation
1.2.1 A thought experiment
An instructor, who lectures a first year university introductory Python programming course, arrives for
his 45 minute-long lecture session. During the course of the 45 minutes, the instructor initiates a range
of classroom activities, which include:
A walk-through of lecture slides
A practical programming session to illustrate lecture concepts
A structured assessment session
An open ended discussion session
The instructor employs a variety of software tools and services, in order to orchestrate each of the
activities. For instance, a PDF viewer is used to present the slides and a specialised programming
environment for the practical programming session. The tools and services are used and accessed
independent of each other. Suppose the instructor were to use a single unified platform—an orchestra-
tion workbench—capable of integrating and centralising access to the tools and services. Would the
orchestration of the activities be better than before?
1.2.2 Inspiration
With the prevalence of software tools and services, organised software environments have been success-
fully employed in a number of domains. Integrated Development Environments (IDEs) (Wasserman
and Pircher, 1987) are used during software development to facilitate integration and centrality of
typical tools used by programmers, subsequently increasing the productivity of programmers. Similarly,
Scientific Workbenches (Curcin and Ghanem, 2008) are commonly used within a number of science
fields in order to facilitate workflow-centric tasks.
The successful use of such organised environments naturally makes the application a similar approach,
to facilitate orchestration, a viable option. On one hand, the efficacy of scientific workbenches in
facilitating workflow-centric tasks (Curcin and Ghanem, 2008) makes this proposed approach desirable
as classroom activities are typically performed in sequential order. On the other hand, the ability of
IDEs in making programmers effective, through the integration and centralisation of various tools,
further reinforces the viability of the proposed approach. Fundamentally, this would make it easier to
sequence activities through the integration of tools and services.
3

5. 1.3 ICT4D context
The close link between education quality and economic well-being is well established (Hanushek and
Wößmann, 2007). Since 2000, a great deal of progress has been made to achieve Universal Primary
Education, the second goal in the United Nations Millennium Development Goal1
; the aim is to “ensure
that by 2015, children everywhere, boys and girls alike will be able to complete a full course of primary
schooling” (Sachs and McArthur, 2005). With the set 2015 target date drawing near, there are plans for
a post-20152
development agenda with a strong focus on the overall quality of education. However,
there are still a number of challenges (EFA Global Monitoring Report, 2014) that need to be overcome
in order to achieve this goal, particularly in developing countries.
Some of the prominent teacher-centric challenges affecting the quality of education in developing
countries include high pupil-teacher ratios; the use of unqualified teaching personnel; and the lack
of teacher professional development (Glewwe and Kremer, 2006). Overcoming these teacher-centric
challenges is desirable as a key focus on educators has been identified as a viable approach to greatly
improving the quality of education. This is, in part, due to the increasing evidence (EFA Global
Monitoring Report, 2014) identifying educators as key role players in improving the quality of education
and more significantly, the overwhelming literature linking student achievement in developing countries
to the quality of teaching (Haßler et al., 2011).
2. BACKGROUND
The use of technologies to enhance teaching and learning—both within and outside the confines of the
classroom—is widespread, and can be traced as far back as 1890, with the invention of the chalkboard.
Classrooms have largely been dominated by one-to-many technologies but, more recently, techno-
logical change—Moore’s law and indeed economies of scale—has made it possible for cutting edge
technologies to be integrated within classrooms. Most interestingly, however, is the prospect of
integrating one-to-one devices into the classroom.
2.1 Orchestration
The educational technology field uses the orchestration metaphor to describe teachers’ real-time
management of intrinsic and extrinsic activities within the classroom (Roschelle et al., 2013). A
particularly interesting aspect of orchestration is the unique focus it places on supporting teachers’
roles. The importance of orchestration is further amplified through the classification of orchestration
of learning, as being one of the grand challenges of technology enhanced learning (Orchestrating
learning - Stellar Deliverable 1.1 2010).
The discourse surrounding orchestration has only recently gained ground, however, available literature
has identified design considerations (Dillenbourg and Jermann, 2010; Dillenbourg, 2013) that ought to
be taken into account from an orchestration perspective. Guidelines are also available, specifying how
orchestration technologies for classrooms should be evaluated (Do-Lenh et al., 2012).
2.2 Related work
2.2.1 Classroom orchestration
The orchestration term is used within the Technology Enhanced Learning field to describe technology-
oriented techniques for teaching and learning that place significant emphasis on supporting teachers
within the classroom.
1
http://www.un.org/millenniumgoals
2
http://www.post2015hlp.org
4

6. Roschelle et al. (Roschelle et al., 2013) note that the support is generally focused on challenges
associated with technology use within the classroom.
Dillenbourg (Dillenbourg, 2013) contends that orchestration has more to do with how teachers manage
the complexities of the multi-layered activities conducted in learning space, and notes that multiple
constraints existing within the classroom are a major obstacle in effectively managing classrooms.
Furthermore, he comprehensively classifies activities and constraints in order to highlight fundamental
differences between orchestration and instructional design, with the latter being more concerned with
intrinsic activities.
2.2.2 Technology for orchestration
Integrating technology within the classroom is considered a vital strategy for 21st
century classrooms
(Saxena, 2013). However, with the exception of mainstream technologies such as interactive white-
boards, LCD projectors and mobile devices such as laptops and tablets, several of these proposed
classroom technologies are hardware-based, and generally targeted for learners.
Although GLUEPS-AR (Muñoz-Cristóbal et al., 2013) was designed to provide a potential solution to
the orchestration burden, it is specifically aimed at orchestration of across-spaces learning situations.
Similarly, GLUE!-PS (Prieto et al., 2014) is focused on computer-supported collaborative learning.
However, looking at evidence obtained from their experiments (Munoz-Cristobal et al., 2014), it is
evident that supporting orchestration activities presents opportunities necessary to support educators.
Further, there are some notable teacher-centric attempts to turn classrooms into effective learning
environments. Google Classroom (Google Inc., 2014a) is perhaps one of the most recent orchestration
software platforms. Classroom promises to increase teachers’ productivity by “weaving together”
existing productivity tools such as Docs, Drive and Gmail. Google Play for Education (Google Inc.,
2014b) is another Google initiative aimed at increasing innovation in education by facilitating easy
access to approved tools and content in order for teachers to meet individual student needs. While
meant to be a generic hub for educational resources, it is in part aimed at facilitating tablet use within
the classroom.
That being said, an important point to note is that the use of orchestration technology, though warranted,
poses the risk of making the already complex classroom ecosystem more difficult to manage. Sharples
(Sharples, 2013) notes that teachers in the modern day classroom grapple to manage its demands and
so adding an orchestration technology to this “volatile mix” could potential worsen the situation.
2.3 Conclusion
This proposed research is strongly rooted in Dillenbourg’s view of orchestration (Dillenbourg, 2013),
and also resonates with his notion of using technology to make formal learning spaces effective learning
environments. Furthermore, from what is known, based on available evidence, the conclusions are clear;
educational software solutions for classroom orchestration are either a direct result of appropriation
of general purpose software or specialised solutions aimed at achieving a single specific objective.
There is clearly a lack of a viable general purpose framework to comprehensively help facilitate the
orchestration process.
3. PROPOSED RESEARCH
The scientific goal of this research is to investigate the impact of the proposed orchestration workbench
on teaching learning. The aim is two-fold; first, to devise a generic orchestration workbench framework
for achieving streamlined technology driven orchestration and, secondly, to demonstrate its potential
applicability in real-world settings.
5

7. 3.1 Research questions
Three core research questions have been developed to scope and guide the proposed study.
3.1.1 Does an orchestration workbench enable educators to become more effective in the
classroom?
This question is aimed at assessing the potential impact of the orchestration workbench on teaching by
empirically determining whether the workbench would have a significant impact on the educator’s
effectiveness in the classroom. Some potentially viable indicators of assessing the effectiveness would
be derived through exploration on the following sub-questions.
Is the technology being used “just because it is there”?
Does COW enable teachers do old things in new ways or old ways?
3.1.2 What is the impact of an orchestration workbench on the teaching experience?
The perceived experience of using a software tool is associated to its ultimate adoption. This research
question is aimed at evaluating the perceived experience of using tools and services integrated within
the orchestration workbench.
3.1.3 What is the impact of an orchestration workbench on the students’ learning experience?
Technology can hinder the learning process if integrated and/or used inappropriately within the
classroom. In particular, it could potentially take away valuable learning it effectively disrupting the
overall student learning experience.
3.2 Research approach
Working with the principle research questions in Section 3.1, the orchestration workbench will be
designed and and deployed into a production environment in order to comprehensively evaluate it.
Sections 3.2.2 and 3.2.3 specify how the research questions will be addressed.
3.2.1 Preliminary work
A comprehensive understanding of the dynamic nature and complexity of the classroom environment
will be central to this proposed research, in order to gain insight into the problem domain. Thus, as a
first step, expert review sessions shall be conducted with a selection educators; this exercise will also
be used to recruit potential study participants.
3.2.2 Prototyping
What is not immediately clear are the specific architectural design and user requirements for the work-
bench. To that effect, the first task will be to develop a prototype orchestration tool and subsequently
run a pilot study in order to elicit the necessary requirements. This will form the basis for the set of
requirements needed to implement the orchestration workbench.
A number of orchestration workbench prototypes will then be implemented, to enable the data collection
from the range of planned experiments.
3.2.3 Experimentation
The specific details of planned experiments are outlined in Section 3.3. However, the assessment of
the applicability and generality of the proposed approach will be central to the overall evaluation. In
addition, looking at the nature of the potential study participants, ethical clearance will be sought prior
to undertaking any user studies, by following potential experimental settings guidelines and University
of Cape Town ethical clearance guidelines.
6

8. 3.3 Evaluation
The assessment criteria of this proposed work will encompass two core aspects. In the first instance,
data will need to be collected to establish a generic orchestration framework and, finally, the proposed
approach will need to be assessed in a real-world setting.
3.3.1 Controlled studies
In order to establish generic characteristics of the proposed approach, considerable data will need to be
gathered. Thus, due to timing and logistical constraints, controlled experiments shall be employed.
3.3.2 Case studies
Case studies form a crucial part of assessing the applicability of the proposed approach. As such, a
series of case studies will be conducted in real-world learning environments. In particular, the objective
will be to assess the influence of aspects such as learning models and class size on the proposed
approach.
3.4 Timeline
A timeline of the planned activities—with associated timescales—for the proposed research is presented
in form of a Gantt chart, shown in Appendix A.
4. ANTICIPATED OUTCOME
4.1 Contributions
The main contributions of this proposed research will be as follows:
1. Working prototypes of orchestration workbenches
2. Case studies demonstrating the use of the teacher orchestration workbench
3. Analysis and results of experiments conducted to demonstrate the effect of the teacher orches-
tration workbench on teaching and learning
4.2 Deliverables
The anticipated deliverables of the proposed research are as follows:
Orchestration workbench framework
Orchestration workbench prototypes
Conference and/or journal publications
Thesis manuscript
4.3 Milestones
Table 1 outlines a chronological ordered list of milestones anticipated during the course of the proposed
research.
7

9. Table 1. Anticipated milestones of the proposed research with associated timescales
Milestone Milestone description Timescale
1. Research proposal Proposed work plan—this document July 2014
2. Position paper A position paper based on the research idea October 2014
3. Background chapter Draft thesis background chapter December 2014
4. Orchestration framework Orchestration architectural framework April 2015
5. Orchestration workbench Prototype orchestration workbench August 2015
6. Conference paper Conference paper based on design September 2015
7. Orchestration plugins Prototype Orchestration tools June 2016
8. Journal article Journal article based on research December 2016
9. Thesis draft Draft thesis manuscript February 2017
10. Thesis manuscript Final version of thesis manuscript April 2017
BIBLIOGRAPHY
[1] Vasa Curcin and Moustafa Ghanem. “Scientific workflow systems - can one size fit all?” In:
IEEE, Dec. 2008, pp. 1–9. DOI: 10.1109/CIBEC.2008.4786077. URL: http://dx.doi.org/10.
1109/CIBEC.2008.4786077.
[2] Pierre Dillenbourg. “Design for classroom orchestration”. In: Computers & Education 69 (Nov.
2013), pp. 485–492. DOI: 10.1016/j.compedu.2013.04.013.
[3] Pierre Dillenbourg and Patrick Jermann. “Technology for classroom orchestration”. In: New
Science of Learning. Ed. by Myint Swe Khine and Issa M. Saleh. Springer New York, 2010.
Chap. Chapter 26, pp. 525–552. DOI: 10.1007/978-1-4419-5716-0_26.
[4] EFA Global Monitoring Report. Teaching and learning: Achieving quality for all. Tech. rep.
EFA Global Monitoring Report, 2014, p. 496. URL: http://unesdoc.unesco.org/images/0022/
002256/225660e.pdf (visited on May 1, 2014).
[5] Paul Glewwe and Michael Kremer. “Schools, teachers, and education outcomes in developing
countries”. In: 2.06 (2006). DOI: 10.1016/S1574-0692(06)02016-2.
[6] Google Inc. “Google for Education: Save time and stay connected”. In: (). URL: http://www.
google.com/edu/classroom (visited on Dec. 10, 2014).
[7] Google Inc. “Google Play for Education”. In: (). URL: http://developer.android.com/distribute/
googleplay/edu (visited on Dec. 10, 2014).
[8] Eric A Hanushek and Ludger Wößmann. The Role of Education Quality in Economic Growth.
Tech. rep. World Bank Policy Research, 2007. URL: http://hdl.handle.net/10986/7154 (visited
on May 1, 2014).
[9] Björn Haßler, Sara Hennessy, Tom Lord, Andrew Cross, Alan Jackson, and Matthew Simpson.
“An investigation of appropriate new technologies to support interactive teaching in Zambian
8

10. schools (ANTSIT)”. In: (2011). URL: http://www.educ.cam.ac.uk/centres/cce/initiatives/
projects/antsit (visited on May 1, 2014).
[10] Son Do-Lenh, Patrick Jermann, Amanda Legge, Guillaume Zufferey, and Pierre Dillenbourg.
“TinkerLamp 2.0: Designing and Evaluating Orchestration Technologies for the Classroom”. In:
21st Century Learning (2012). DOI: 10.1007/978-3-642-33263-0_6.
[11] Juan A. Munoz-Cristobal, Ivan M. Jorrin-Abellan, Juan I. Asensio-Perez, Alejandra Martinez-
Mones, Luis P. Prieto, and Yannis Dimitriadis. “Supporting Teacher Orchestration in Ubiq-
uitous Learning Environments: A Study in Primary Education”. In: Learning Technologies,
IEEE Transactions on PP.99 (2014), pp. 1–1. DOI: 10 . 1109 / TLT . 2014 . 2370634. URL:
http://dx.doi.org/10.1109/TLT.2014.2370634.
[12] Juan A. Muñoz-Cristóbal, Luis P. Prieto, Juan I. Asensio-Pérez, Iván M. Jorrín-Abellán, Alejan-
dra Martínez-Monés, and Yannis Dimitriadis. “GLUEPS-AR: A System for the Orchestration of
Learning Situations across Spaces Using Augmented Reality”. English. In: Scaling up Learning
for Sustained Impact. Ed. by Davinia Hernández-Leo, Tobias Ley, Ralf Klamma, and Andreas
Harrer. Vol. 8095. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2013,
pp. 565–568. DOI: 10.1007/978-3-642-40814-4_59. URL: http://dx.doi.org/10.1007/978-3-
642-40814-4_59.
[13] Orchestrating learning - Stellar Deliverable 1.1. 2010. URL: http://www.stellarnet.eu/d/1/1/
Orchestrating_learning (visited on May 1, 2014).
[14] Luis P. Prieto, Juan I. Asensio-Pérez, Juan A. Muñoz-Cristóbal, Iván M. Jorrín-Abellán, Yannis
Dimitriadis, and Eduardo Gómez-Sánchez. “Supporting orchestration of CSCL scenarios in
web-based Distributed Learning Environments”. In: Computers and Education 73 (2014), pp. 9–
25. DOI: 10.1016/j.compedu.2013.12.008. URL: http://dx.doi.org/10.1016/j.compedu.2013.12.
008.
[15] Jeremy Roschelle, Yannis Dimitriadis, and Ulrich Hoppe. “Classroom orchestration: Synthesis”.
In: Computers & Education 69 (Nov. 2013), pp. 523–526. DOI: 10.1016/j.compedu.2013.04.010.
[16] Jeffrey D. Sachs and JW McArthur. “The millennium project: a plan for meeting the millennium
development goals”. In: Lancet 365.9456 (Jan. 2005), pp. 347–353. DOI: 10.1016/S0140-
6736(05)17791-5.
[17] Saomya Saxena. “Top 10 Characteristics of a 21st Century Classroom”. In: EdTechReview (Dec.
2013). URL: http://edtechreview.in/news/862-top-10-characteristics-of-a-21st-century-
classroom (visited on Dec. 9, 2014).
[18] Mike Sharples. “Shared orchestration within and beyond the classroom”. In: Computers &
Education 69 (Nov. 2013), pp. 504–506. DOI: 10.1016/j.compedu.2013.04.014.
[19] Anthony I. Wasserman and Peter A. Pircher. “A Graphical, Extensible Integrated Environment
for Software Development”. In: SIGPLAN Not. 22.1 (Jan. 1987), pp. 131–142. DOI: 10.1145/
390012.24224. URL: http://dx.doi.org/10.1145/390012.24224.
9

11. APPENDIX A. PROPOSED WORK PLAN
10