Academic Paper Writing Service http://StudyHub.vip/An-Analysis-Instrument-For-Gameplay-Inf 👈

1. An Analysis Instrument for Gameplay Information Flows Supporting
Sustainability Complex Problem-Solving
Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
School of Computing and Engineering, University of Huddersfield, UK
dimitar.gyaurov@hud.ac.uk
carlo.fabricatore@gmail.com
ximelopez@gmail.com
DOI: 10.34190/GBL.19.171
Abstract: This article presents a game analysis instrument for the identification and evaluation of gameplay information
flows which may promote the development of complex problem-solving capabilities. Complex problem-solving is essential
to foster the sustainable development of our global world, addressing our needs while at the same time ensuring that future
generations will be able to address their own. Learning for sustainable development should be a central focus of
contemporary education, and games can be valuable learning environments for the advancement of complex problem-
solving capabilities. Players’ engagement in complex problem-solving processes depends on contents, function, timeliness
and amount of gameplay information available to them. Hence, the instrument presented in this article was developed to
capture functionality and properties of gameplay information flows key to engage in and interpret complex problem-solving
situations. The instrument is based on a model of gameplay as a contextualised activity process driven by meaning-making,
developed integrating perspectives from complex problem-solving theory and constructivism learning theory. An exploratory
test of the instrument was carried out through a case study based on the game Stop Disasters. Results suggest that the
instrument allows to comprehensively evaluate suitability of gameplay information flows to engage players in complex
problem-solving situations, and foster complex problem-solving capabilities. The instrument can support the analysis of
existing games, as well as the design of new games. Hence, it can be a comprehensive and valuable tool for researchers,
developers and educators involved in projects concerning sustainability and game-based learning.
Keywords: digital games, constructivist learning, complex problem-solving, sustainability, game analysis, gameplay
information flows
1. Introduction
Our world is a complex system defined by multiple interacting environmental, social and economic phenomena,
and characterised by continuous change, unpredictability and uncontrollability (Fabricatore and López, 2014).
The interplay of these phenomena generates complex problems that threaten the sustainable development of
our global society, such as climate change, poverty, and global healthcare. These sustainability problems
demand non-traditional solutions because they challenge people’s abilities to address immediate needs without
compromising the prosperity of future generations (Ozbekhan, Jantsch and Christakis, 1970). In order to
successfully manage complex sustainability problems, people need specific capabilities and attitudes, which
should be a central focus of contemporary education (Bowser et al., 2014, Fabricatore and López, 2014).
Complex problem-solving (CPS) is a central capability to focus on, and consists in the ability to bring about
desirable changes in real-world contexts, addressing ill-defined problems in changing, unpredictable and
uncontrollable environmental conditions (Frensch and Funke, 1995b, Fabricatore and López, 2014, Dörner and
Funke, 2017).
Developing CPS capabilities requires specific learning environments, since traditional problem-solving and CPS
are significantly different in defining and dealing with problems (Dörner and Funke, 2017). Microworlds have
been identified as environments suitable to foster CPS capabilities (Sterman, 1994). They are interactive
simulation systems which can offer the experience of complex real-world transformations in an experimental
setting, and promote sustainability skills through high-quality feedback loops, i.e. information flows (Dörner and
Wearing, 1995, Fabricatore, 2019). Games centred on simulation gameplay mechanics can function as
microworlds, providing in addition essential characteristics of intrinsically-motivating learning environments
(Rieber, 1996). Furthermore, games can engage players in complex meaning-making contexts mirroring real-
world scenarios (Fabricatore, 2019). Games can therefore be powerful learning environments suitable for the
development of CPS capabilities through engaging players in sustainability-related contexts.
To harness the potentialities that games offer to promote sustainability CPS learning, analysis tools are needed
to understand to what extent a game may elicit and support CPS processes contextualised in sustainability-
863

2. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
relevant scenarios. This paper presents an instrument developed to address this need through facilitating the
identification and analysis of functionality and properties of gameplay information flows provided to the player.
The instrument is based on a model of gameplay information flows integrating contributions from the fields of
CPS, constructivism, and gameplay theory. The paper is structured as follows: Section 2 outlines the conceptual
framework underpinning this model and the instrument. Section 3 presents the methodology for the
development of the instrument. Section 3.1 outlines the gameplay information flows model underpinning the
instrument. Section 3.2 and 3.3 present the instrument and findings from its testing, respectively. Section 4
presents a discussion of the findings and related conclusions.
2. Conceptual framework
2.1 CPS
Simply stated, a problem is a discrepancy between a knowable state of matters and a new, desirable goal state,
where the subject doesn’t know how to accomplish the new state (Duncker, 1945), or where there are barriers
between the given and the desired goal state (Dörner, 1976). A problem becomes complex when given state,
goal state, barriers and/or other salient environmental conditions are defined by multiple properties, highly
interconnected, dynamically changing, intransparent, and only partially known (Dörner and Wearing, 1995,
Fischer, Greiff and Funke, 2012, Dörner and Funke, 2017). All this makes a problem situation uncertain and
uncontrollable, generating the need for specific approaches to address problems.
Traditional problem-solving can be defined as discovering a linear sequence of steps to transform the given state
into the goal state (Newell and Simon, 1972). Instead, the characteristics of complex problems make CPS an
iterative process that requires multiple interactions with a dynamically changing environment, involving
exploring, evaluating and predicting its state (Frensch and Funke, 1995b).
Recent research has emphasized the importance of considering CPS as a contextualised process (Dörner and
Funke, 2017). Real-world CPS tasks require problem solvers to construct knowledge through interacting with
real-world problem spaces. While transforming the environment, problem solvers continuously interpret
circumstances and consequently reflect, criticise, structure, and modify their own behaviours (Jonassen and
Rohrer-Murphy, 1999; Fischer, Greiff and Funke, 2017). This places high demands on problem solvers, who are
required to self-regulate their cognition and think creatively to continuously produce and adapt varied and
incomplete solutions in highly challenging situations (Dörner and Funke, 2017). Taking into account the
relationship between the attributes of complex problems as well as the processes involved in solving them, the
new definition proposed by Dörner and Funke (2017) conceptualises CPS as: i) a set of self-regulated processes
and activities to achieve ill-defined goals; ii) situated in dynamic environments; iii) requiring creativity and wide
range of strategies; iv) characterised by solutions that are mingled and incomplete; v) involving the problem
solver’s cognition, affection and motivation through high stake-challenges; and vi) frequently requiring
collaboration with others.
Contextualised CPS is especially relevant to foster sustainability learning, as developing sustainability capabilities
requires engaging learners cognitively, behaviourally and affectively in learning scenarios reflecting real-world
sustainability issues and dynamics (Juech and Michelson, 2011, Fabricatore and López, 2012, 2014).
Environments suitable to foster sustainability-relevant CPS learning should therefore allow players to: i) engage
in complex and challenging problem situations contextualised in sustainability scenarios; ii) explore and
transform the environment iteratively; and iii) understand multiple, opaque and interdependent variables
affecting sustainable development. For this, information flows are key. They should readily be available for
problem solvers to interpret the state of the environment, comprehend the effectiveness of their own strategies,
and adjust their behaviours accordingly (Frensch and Funke, 1995b, Guzdial et al., 1996, Engelhart, Funke and
Sager, 2017). Hence, their specific properties and functionalities can significantly influence CPS performance
(Engelhart, Funke and Sager, 2017).
2.2 Constructivist learning and CPS
Constructivist learning may be especially valuable to foster the development of CPS skills, because it exposes
the learner to experiencing the world and building knowledge through engaging in the management of complex
problems driven by meaning-making (Jonassen and Rohrer-Murphy, 1999, Tam, 2000). According to the
constructivist paradigm, learners learn by actively engaging in a contextualised process of constructing new
864

3. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
knowledge using present and past real-world experiences, reflecting on those experiences, forming and testing
hypotheses through interactions with the learning environment, and evaluating outcomes of these interactions
(Jonassen 1994, Honebein, 1996). Accordingly, Jonassen and Rohrer-Murphy (1999) suggest that constructivist
learning can be conceptualised as a meaning-making process entailed by purposeful and contextualised activity.
Through such activity, a subject interacts with the environment, transforming it in order to bring about a
desirable state (goal). As the subject interacts with and transforms the environment, they continuously interpret
it and consequently construe knowledge which is then used to progress the activity (Jonassen and Rohrer-
Murphy, 1999, Jonassen, 2002). Thus, constructivist learning can be regarded as a meaning-making process
(Jonassen and Rohrer-Murphy, 1999, Jonassen, 2002), which is embedded in activity, and entails the
construction of knowledge regarding the activity process, context, the involved elements and their relationships.
In order to promote this process, learning environments should provide sufficient opportunities and means for
learners to explore and transform their surrounding environment, and information to interpret and predict its
state (Jonassen, 1994, Jonassen and Rohrer-Murphy, 1999). Constructivist learning environments are
characterised by key components including: i) a problem-project space based on real-world contexts; ii)
information resources that support the problem-solving process; iii) tools to facilitate material interactions with
the environment; iv) cognitive tools to support exploration, reflection and interpretation of the environment;
and v) conversation and collaboration tools promoting learning through social experiences (Jonassen and
Rohrer-Murphy, 1999). Therefore, it can be argued that sustainability-relevant CPS can be fostered through
learning environments that present these characteristics, contextualise the problem space and tools in scenarios
reflecting sustainability complex problems, and provide to the learner information sufficient to make sense of
the environment and hypothesize possible states.
2.3 Gameplay activity
Gameplay can be conceptualised as a contextualised problem-solving activity consisting in a system of
hierarchical and iterative goal-oriented tasks, mediated by objects and social entities, and driven by meaning-
making (Fabricatore, 2019). Through gameplay tasks players attempt to transform the game environment in
order to achieve goal states. As they do, they focus on transforming target entities (e.g. collecting an object),
while interacting with enabling and/or hindering gameplay entities (i.e. tools and collaborators that may
facilitate transformative processes, or barriers and opponents that may hinder them). Gameplay tasks are
always contextualised in fictional spatio-temporal and socio-cultural settings that define the functioning and
meaning of elements, events and relationships involved in gameplay activities (Fabricatore, 2019). Gameplay
settings may have varying complexity and, in the case of simulation games, reproduce real-world complex
dynamics (Fabricatore, 2019).
Meaning-making is a central component of the gameplay activity. As they act upon the gameplay environment,
players continuously make sense of it, interpreting situations, involved elements and their relationships in order
to understand what needs to be done, how and why (Fabricatore, Gyaurov and López, 2019). Through meaning-
making, players construct knowledge that they use to set gameplay goals, plan approaches for their attainment,
and evaluate outcomes of their actions (Fabricatore, 2019). Meaning-making integrates indissolubly players’
affection, cognition and behaviour, engaging players holistically in the situations that they face (Fabricatore and
López, 2014; Fabricatore, 2019).
Players’ meaning-making is fed by gameplay information flows made available by the game. These are necessary
for players to understand their role, and what, how and why things (could) happen in the game environment
(Fabricatore, Gyaurov and López, 2019). Fabricatore (2019) suggests that effective gameplay information flows
should: i) help players to identify, accept and evaluate a task; ii) allow players to plan methods to achieve a task
goal; iii) support players to evaluate contextual conditions which may affect tasks performance; iv) enable
players to understand aspects of the local game context; v) enable players to understand aspects of the global
game context; vi) help players to establish connections between things and events in the game space; and vii)
be provided as much as possible in response to the player’s active engagement with the game space.
It can be argued from the above that games can be effective learning environments to develop sustainability-
relevant CPS capabilities. Games can function as constructivist learning environments which are complex in
nature (Rieber, 1996). Furthermore, games can be particularly effective to promote sustainability-relevant CPS,
because of: i) the iterative problem-solving nature of the gameplay activity (Fabricatore, 2019); ii) the role that
865

4. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
meaning-making has in it, and its ability to engage players cognitively, affectively and behaviourally in gameplay
scenarios (Fabricatore, Gyaurov and López, 2019); and iii) the possibility of contextualising the gameplay activity
in scenarios simulating complex real-world situations related to sustainability (Fabricatore and López, 2014,
Fabricatore, 2019).
Gameplay information flows are crucial to define the extent to which games can foster CPS. The properties and
functionalities of information flows define CPS situations and affect the performance of problem-solvers in any
context (Frensch and Funke, 1995b, Engelhart, Funke and Sager, 2017). In the case of games, information flows
are the key input for the meaning-making process that drives and motivates the gameplay activity (Fabricatore,
Gyaurov and López, 2019). Therefore, an instrument for the formal analysis of contextualised CPS information
flows in games can be key to examine the extent to which a game can elicit CPS capabilities and support their
development.
3. Development of an instrument for the analysis of gameplay CPS information flows
3.1 A model of gameplay CPS information flows
Establishing a model of CPS information flows in games was crucial to define the aspects to be evaluated and
guide the creation of the instrument items. We elaborated the Information Flows Model (Figure 1 and Figure 2)
integrating CPS components (Dörner and Wearing, 1995, Frensch and Funke, 1995b, Dörner and Funke, 2017),
constructivist learning components (Honebein, 1996, Jonassen and Rohrer-Murphy, 1999) and gameplay theory
components (Fabricatore, 2019). Gameplay is modelled as a contextualised, meaning-making-driven activity
process. The model combines into a unified system: i) the problem solver, learner and player; ii) the contextual
aspects of complex, constructivist learning and gameplay environments; iii) the elements of complex, learning
and gameplay tasks; and iv) the process phases of CPS, constructivist learning and gameplay activity.
The model particularly focuses on the functionality and properties of information flows emerging from the
engagement of players with the gameplay activity system. Structural components of this interaction are
represented in Figure 1. The core components of the model are the player, the gameplay environment, the
gameplay process, and the gameplay tasks.
Figure 1: Information flows model – structural components
The player’s thoughts, feelings and behaviour influence the gameplay interactions and are affected by them.
The gameplay environment is based on real-world contexts and provides problems with unknown, unpredictable
and uncontrollable elements and relationships. Elements from the environment can have different degrees of
perceived participation in gameplay tasks: those in the micro-context as explicitly perceived by the player and
directly affect the task; the meso-context contains elements which are perceived by the player but affect only
866

5. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
indirectly the current task; elements of the macro-context cannot be directly perceived but may indirectly affect
the ongoing task. The environment also includes player-triggered and non-player-triggered events, some of
which can be traced back to their source and others cannot.
The interaction with the game environment consists of sub-phases. The player iteratively identifies a goal,
explores the environment, forms hypothesis, predicts possible side effects, regulates their intentions
accordingly, plans and executes actions, evaluates the outcomes of the actions and self-reflects based on their
performance. Reaching the goal of a gameplay task requires a sequence of meaningful actions to be performed.
The actions are directed towards gameplay target objects and may be enabled or hindered by elements of the
game. Elements are designed with a specific purpose but the player may generate new meanings based on
dependencies and dynamics of the properties of elements.
Figure 2: Information flows model – relationships and dynamics between components
In addition to the relationships and dynamics between components of the model, Figure 2 also focuses on the
information flows of the gameplay activity. Information flows emerge from the game context, gameplay process,
gameplay tasks and gameplay elements. Information flows have the following properties and functionalities: i)
“Timeliness” refers to the time that passes between an action or an event and the associated information flow;
ii) “Reiteration” is related to the duration or the number of repetitions of the information flow; iii) “Response”
differentiates between information flows of player actions and non-player-triggered events; iv) “Relation”
describes the clarity or ambiguity in the connection between an act or an event and the information flow; v)
“Relevance” is about the appropriateness of the type of information flow based on the given act or event; vi)
“Strength” identifies the capability of the information flow to shift player attention from the ongoing task.
3.2 Gameplay CPS information flows analysis instrument
The instrument was aimed at capturing the contents, properties and functionalities of gameplay information
flows which have been identified to engage players in the CPS process, constructivist learning and gameplay
activity. The instrument was developed based on the method by Moore and Benbasat (1991), which consists of
three stages: creation of instrument items; organisation of items in categories, based on the topics they address;
testing of the instrument.
The objective of the first stage was to evaluate and confirm the clarity of created items. They closely correspond
to individual elements of the core components of the model. Two game analysis experts, familiar with the
867

6. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
required theoretical background, conceptually pilot tested the existing items with a variety of games. After this,
concepts were defined in order for the instrument to be more self-explanatory.
After another testing iteration of the instrument, we decided that the items of the instrument should be self-
contained. Therefore, examples were added to each item to support the use of the instrument with a diverse
number of games. For instance, the item “real-world context” was clarified through “weather patterns; political
systems; city layouts; population behaviours; geographical features; historical events”. In another case,
“exploring environment” was explained with “searching for characters to talk with; observing the patterns of a
puzzle mechanism”. The final modifications of items happened during the second stage when they were
organised in categories based on topics they address (e.g. “Identifying Goal” from the gameplay process category
and “Task Goal State” from the gameplay task category were redundant items and the “Task Goal State” was
removed).
The objective of the second stage of the instrument development process was to categorise the confirmed items.
Instrument scales were developed based on the four core components of the model. In addition to “game
context structure”, “gameplay process”, “gameplay tasks” and “player impacts”, a fifth scale was included for
information flows and their general properties and functionalities. “Information flows general properties” was
added in order to remove confusion between features of the information and its causes and effects.
The five scales and 34 items were finally reworded to a questionnaire form in preparation for testing. For
example, in the scale for “player impacts” it is investigated if “information flows may impact on: i) player’s
thoughts while engaged in game tasks; ii) player’s feelings while engaged in game tasks; and iii) player’s
behaviour while engaged in game tasks”. The perceived sources, features and impacts of information flows are
assessed through a single item on a five-point Liker scale (1: “strongly disagree; 5: “strongly agree”). There is a
“not applicable for this game” point as well.
The full version of the questionnaire is available at:
https://drive.google.com/open?id=1txi2kAMsOcR9T9jlLzmnDGszFb3EvnwR
3.3 Exploratory testing of the gameplay CPS information flows analysis instrument
The exploratory testing was conducted to assess the reliability of the instrument. Three game analysis expert
tested the instrument independently. One of them was blind to the process of creation of the instrument. All
testers were familiar the gameplay theory concepts involved in the instrument and the underpinning model.
Stop Disasters, an online sustainability-based game, was used for the testing. The game simulates five natural
hazards and requires players to organise urbanised areas applying effective methods of prevention and
mitigation of risks for the population and the environment. Stop Disasters was selected because it: i) focuses on
simulation gameplay mechanics; ii) challenges players through real-world-based scenarios; and iii) requires
players to learn how to deal with natural disasters through managing problems with multiple variables and
interdependencies. Additionally, Stop Disasters provides to players a broad variety of information flows worth
exploring in relation to CPS, constructivist learning and gameplay activity.
The experts were asked to play the Tsunami scenario for at least ten minutes, and then complete the online
questionnaire. The experts were also invited to provide additional comments on the game and the instrument.
Data analysis was aimed at calculating the inter-rater reliability between the testers, using intraclass correlation
coefficients (ICC). The commonly used standards provided by Cicchetti (1994) for cut-offs qualifying values are
< 0.4 as poor, between 0.4 and 0.59 as fair, between 0.6 and 0.74 as good, and > 0.75 as excellent. Analysis was
performed including all 34 items from the questionnaire. Inter-rater reliability between the three testers was
calculated as “good” (ICC=0.69). Figure 3 and Figure 4 present the item scores from each expert, and their mean.
868

7. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
Figure 3: “Game context structure” and “gameplay process” scale items scores
Figure 4: “Gameplay tasks”, “information flows general properties”, and “player impacts” scale items scores
4. Discussion
This article presented an instrument to evaluate the properties and functionalities of gameplay information
flows which may support player engagement in the gameplay CPS process. This instrument is relevant to
examine the potential that information flows may have to elicit the development and application of CPS
capabilities. Hence, it addresses the recognised need for new tools to test and investigate CPS in simulated
environments (Greiff et al., 2014). The results of the inter-rater reliability analysis suggest that the instrument
may be a robust tool for this purpose. However, the test was merely exploratory.
Further tests involving a larger number of testers are needed for more conclusive results. Players’ cultural
background might also limit the effectiveness and reliability of instruments designed to examine CPS-related
issues. Depending on cultures, people make sense of things and solve problems in different ways. The literature
suggest that research is needed to explore the impacts of cultural frameworks on gameplay meaning-making
(Fabricatore, 2019) and problem-solving (Dörner and Funke, 2017). Hence, it would be appropriate to test the
instrument with subjects from diverse cultural backgrounds. It would also be appropriate to test the instrument
with other games, of the same category of Stop Disaster, but also representative of other categories (e.g.
adventure and role-playing games). Furthermore, comments from testers suggested that “reiteration of
information flows” (item 27) should be clarified in regards to player-triggered or game-triggered reiteration.
The game Stop Disasters was selected because it challenges players to learn how to manage real-world-based
problems. The testing results suggest that in this game CPS processes may be predominantly influenced by
information flows regarding: i) gameplay features related to the number of variables that players manage and
their relationships (item 24); ii) the emerging information flows (items 26 to 31); and iii) core elements of
gameplay tasks (items 18 to 20). The results suggest that the game information flows were highly impactful at
869

8. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
emotional, cognitive and affective levels (items 32 to 34). The untimeliness of information flows (item 26) in the
game Stop Disasters may have affected the analysis of other components such as “gameplay tasks” and
“gameplay process” (Figure 3 and Figure 4). Evaluations concerning transformable and transformative functions
of gameplay elements (item 21), their enabling and hindering role (item 22), the dependencies and dynamics
among their properties (item 24 and item 25) may have also been compromised by delayed and unresponsive
information flows in the game. Similar reasons may have penalised the identification of sub-phases of the
gameplay process (items 9 to 17). Stop Disasters provides the majority of information flows only at the end of
the game. This may have compromised the prediction of side effects (item 12), the regulation of intentions (item
13), the evaluation of outcomes (item 16) and self-reflection (item 17). Unreliable information may have affected
the interpretation of the game context (items 1 to 8). Overall, testing results suggest that in Stop Disasters
information flows may promote CPS processes. However, the information flow analysis alone is not sufficient to
infer that proper CPS processes can actually take place in the game.
Even though the way CPS unfolds is heavily dependent on the available information flows, CPS can only take
place in complex problem scenarios that problem solvers can iteratively explore and transform. Hence, game-
based CPS learning depends on actual complexity features implemented in gameplay scenarios (e.g.
unpredictable non-player controlled events), means that players have to explore and act upon such scenarios
(e.g. tools to build structures or modify terrains), as well as on information flows available to interpret and
anticipate scenarios. To promote sustainability-relevant CPS learning, sustainability contextualisation of
gameplay scenarios is also necessary (e.g. a dwelled tropical island exposed to tsunamis). The instrument
presented in this article has been designed to analyse aspects of game information flows relevant to foster CPS.
It should therefore be complemented by other instruments specifically designed to investigate game features
suitable to generate complex gameplay dynamics contextualised in sustainability settings, and permit iterative
transformations of the game environment. These should be based on game models specifically developed to
support the analysis of game systems to identify sustainability learning affordances, such as the one proposed
by Fabricatore e López (2014). Using complementary specialised tools for the comprehensive analysis of CPS-
related issues is a recommended approach to tackle the challenges stemming from the multi-dimensional nature
of CPS (Funke, Fischer and Holt, 2018).
In conclusion, the proposed instrument has the potential to support game analysis for the identification of games
that could promote sustainability-relevant CPS, as well as for the design of games aimed at this. Therefore, this
instrument can be a comprehensive and valuable tool for researchers, developers and educators involved in
projects concerning sustainability and game-based learning.
References
Bowser, G., Gretzel, U., Davis, E., & Brown, M. (2014). Educating the Future of Sustainability. Sustainability, 6(2), 692-701.
Cicchetti, D.V., 1994. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment
instruments in psychology. Psychol. Assess. 6 (4), 284–290.
Dörner, D. (1976). Problemlösen als Informationsverarbeitung [Problem solving as information processing]. Stuttgart:
Kohlhammer.
Dörner D and Funke J (2017) Complex Problem Solving: What It Is and What It Is Not. Front. Psychol. 8:1153.
Dörner, D., & Wearing, A. T. (1995). Complex Problem-Solving: Towards a (computersimulated) theory. In J. Funke & P.
Frensch (Eds.), Complex Problem Solving – The European perspective (pp. 65–99). Hillsdale, NJ: Erlbaum.
Duncker, K. (1945). On problem solving. Psychological Monographs, 58. American Psychological Association.
Engelhart, M., Funke, J., and Sager, S. (2017). A web-based feedback study on optimization-based training and analysis of
human decision making. J. Dynamic Dec. Mak. 3, 1–23.
Fabricatore, C. (2019) ‘Underneath and beyond mechanics: An activity-theoretical perspective on meaning-making in
gameplay’, in Suter, B., Kocher, M., and Bauer, R. (eds) Games and Rules: Game Mechanics for the “Magic Circle”.
Bielefeld, DE: Transcript Verlag, pp. 87–112.
Fabricatore, C., & López, X. (2014). A Model to Identify Affordances for Game-Based Sustainability Learning. In C. Busch
(Ed.), Proceedings of the 8th European Conference on Game Based Learning (pp. 99– 109). Reading, UK: Academic
Conferences and Publishing International Limited.
Fabricatore, C. and López, X. (2012) “Sustainability Learning through Gaming: An Exploratory Study”, Electronic Journal of
E-Learning, Vol10, No.2, pp 209–222.
Fabricatore, C., Gyaurov, D. and Lopez, X. (2019) ‘An exploratory study of the relationship between meaning-making and
quality in games’, Multimedia Tools and Applications, 78(10), pp. 13539–13564.
Fischer, A., Greiff, S., and Funke, J. (2017). “The history of complex problem solving,” in The Nature of Problem Solving:
Using Research to Inspire 21st Century Learning, eds B. Csapó and J. Funke (Paris: OECD Publishing), 107–121.
Fischer, A., Greiff, S., & Funke, J. (2012). The process of solving complex problems. Journal of Problem Solving, 4, 19–42.
870

9. Dimitar Gyaurov, Carlo Fabricatore and Ximena Lopez
Frensch, P. A., and Funke, J. (1995b). “Definitions, traditions, and a general framework for understanding complex problem
solving,” in Complex Problem Solving: The European Perspective, eds P. A. Frensch and J. Funke (Hillsdale, NJ:
Lawrence Erlbaum), 3–25.
Funke, J., Fischer, A., and Holt, D. V. (2018). “Competencies for complexity: problem solving in the 21st century,” in
Assessment and Teaching of 21st Century Skills, eds E. Care, P. Griffin, and M. Wilson (Dordrecht: Springer), 3.
Greiff, S., Fischer, A., Stadler, M. and Wüstenberg, S. (2014). Assessing complex problem- solving skills with multiple
complex systems. Thinking & Reasoning, 21(3), pp.356-382.
Guzdial, M., Kolodner, J., Hmelo, C., Narayanan, H., Carlson, D., Rappin, N., Hübscher, R., Turns, J. and Newstetter, W.
(1996). Computer support for learning through complex problem solving. Communications of the ACM, 39(4), pp.43-
45.
Honebein, P.C. (1996). Seven goals for the design of constructivist learning environments.in Constructivist Learning
Environments: Case Studies in Instructional Design. Brent G. Wilson (Ed.). Englewood Cliffs: Educational Technology
Publications: 11-24.
Jonassen, D. H. (1994). Toward a Constructivist Design Model. Educational Technology, April, 34-37.
Jonassen, D. H. (2002). Learning as Activity. Educational Technology, 42(2), 45–51.
Jonassen, D. H., and L. Rohrer-Murphy. 1999. “Activity Theory as a Framework for Designing Constructivist Learning
Environments.” Educational Technology Research and Development 47: 61–79.
Juech, C., & Michelson, E. (2011). Rethinking the Future of Sustainability: From silos to systemic resilience. Development,
54(2), 199-201.
Moore, G. C., & Benbasat, I. (1991). Development of an instrument to measure the perceptions of adopting an information
technology innovation. Information Systems Research, 2, 192–222.
Newell, A. & Simon, H.A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
Ozbekhan, H., Jantsch, E. and Christakis, A.N. (1970). The predicament of mankind: A quest for structured responses to
growing world-wide complexities and uncertainties. Proposal to the Club of Rome.
Rieber, L. P. (1996). Seriously considering play: Designing interactive learning environments based on the blending of
microworlds, simulations, and games. Educational Technology Research & Development, 44(2), 43-58
Sterman, J. (1994). Learning in and about complex systems. System Dynamics Review, 10(2-3), pp.291-330.
Tam, M. (2000). Constructivism, Instructional Design, and Technology: Implications for Transforming Distance Learning.
Educational Technology and Society, 3 (2).
871