Adaptarse a las nuevas formas de crear y compartir contenidos digitales constituye un reto para la preparación de profesionales en los perfiles emergentes de disciplinas ajenas a la informática y la computación. Los lenguajes y las herramientas de creación digital no están muchas veces pensados para su utilización por parte de usuarios de estos campos. Un reto en el campo de la computación creativa es la posibilidad de incorporar capacidades interactivas multimodales, junto con realidad virtual y realidad aumentada, en las herramientas de autoría con las que se elaboran los materiales y diseños de aprendizaje. El objetivo general de la charla es motivar la investigación sobre la computación creativa, así como mostrar desarrollos diversos alrededor de un marco de trabajo que aspira a fomentar las habilidades de diseño, creación y despliegue de experiencias educativas con capacidades analíticas para el aprendizaje y la evaluación en un contexto multidisciplinar.

1. Diseño de escenarios interactivos de
aprendizaje con realidad aumentada para
experiencias de computación creativa
Juan Manuel Dodero Beardo
Software Process Improvement Research Group
Departamento de Ingeniería Informática
Universidad de Cádiz
juanma.dodero@uca.esDoctorado en Ingeniería Informática 2016/17
5 Marzo 2017
Facultad de Informática
Universidad Complutense de Madrid

2. What is Creative Computing?
(Brennan, Balch & Chung – Harvard Grad School of Education)
CC is about Creativity
CS and computing-related fields
have long been introduced to
[young] people disconnected
from their interests–
emphasizing technical detail
over creative potential
CC is about Empowerment
Many [young] people
with access to computers
participate as consumers, rather
than designers or creators
CC is about… Computing
Creating computational
artifacts supports [young]
people’s development as
computational thinkers in all
aspects of their lives, across
disciplines and contexts

3. Antecedents
Computational
Thinking
> Seymour Papert (1981)
“programming the computer to
[…] establish an intimate contact
with some of the
deepest ideas
from science,
from maths, and
from the art of
intellectual
model building.”
> Jeannette M. Wing (2006)
> STEM
Computing in
Education
> Mark Guzdial (Georgia Tech)
“There is no reliable research
showing that computing makes
one more creative or more able
to problem-solve.”
“For the same reasons people
should understand biology,
chemistry or physics, it makes
a lot of sense to understand
computing in our lives”
> STEM+C
Computing and
Humanities (STEAM)
> S. Krishnamurthi (Brown U.)
“we are just overly intoxicated
with computer science”
> English student (Stanford):
“I don’t see myself
as having skills
missing […] When
you are analyzing
Melville, you have
to unpack that
language and synthesize it
back”

4. Program or be programmed
The debate over whether the Net is good or bad for us
fills the airwaves and the blogosphere. But for all the
heat of claim and counter-claim, the argument is
essentially beside the point: it’s here; it’s everywhere.
The real question is, do we direct technology, or do we
let ourselves be directed by it and those who have
mastered it? “Choose the former,” writes Rushkoff, “and
you gain access to the control panel of civilization.
Choose the latter, and it could be the last real choice you
get to make.”

5. But… in a mobile world context

6. with a mixed reality…

7. gestural interactions…
Leap motion

8. empowered by gadgets and wearables…

9. …in a cyber-physical brave new world

10. Where is the teacher?

11. What about assessment?

12. Learning Analytics: Who is going to build this?

13. Computer Scientists only?

14. Involve teachers in the creation of their own extended
{mobile, AR, gestural,…} learning scenarios using
creative computing
Motivation

15. Focus
Mobile
devices
Augmented
reality
Gestural
interactions
Learning
analytics

16. State of the art: e.g. augmented reality tools
For programmers For non programmers

17. Visual programming languages can help to develop
creative computing abilities to build such extended
learning scenarios
General hypothesis

18. State of the art: Visual Languages

19. Authoring framework
http://vedils.uca.es

20. Based on AppInventor

21. Existing components

22. Specific objectives
Augmented reality
> Marker, text and
image recognizing
3D Engine
> 3D interactive objects
(.OBJ, .3DS, .MD2, etc.)
Gesture
interaction
> Hand gestures
> Arm movements
Learning analytics
> Cloud store of
interaction data
> Queries + datatables +
charts

23. Extensions
Augmented reality
> Marker, text and
image recognizing
3D Engine
> 3D interactive objects
(.OBJ, .3DS, .MD2, etc.)
Gesture
interaction
> Hand/arm gestures
Learning analytics
> Cloud store of
interaction data
> Queries + datatables +
charts

24. Extension of the visual language to build
augmented-reality mobile apps
Specific objective 1

25. VEDILS AR components

26. Use machines and tools for a torque test lab task
assignment, preventing damage to the equipment
and uncompromising the student’s safety
Case study: material resistance lab

27. Mobile App development

28. Image recognition

29. Creating 3D models

30. App behavior programming

31. Lab development

32. Extension of the visual language to build
learning analytics mobile apps
Specific objective 2

33. VEDILS LA components

34. ActivityTracker selectable automatic tracking
Global Actions Focused action (e.g. button)

35. ActivityTracker blocks
http://www.eclipse.org/aspectj/

36. ActivityTracker tracked data

37. ActivitySimpleQuery design
DistinctResults:
To avoid duplicated results.
FieldsToRetrieve:
Data to recover from the data
storage
TableId:
Google Fusion Tables id

38. ActivityAggregationQuery design
Group By: Fields to group the results
MetricsToRetrieve: aggregations

39. DataTable & Chart design
Query: Query from which obtain data
Height/width: dimensions in screen
Visible: To show/hide the table/chart
ChartType: (currently) line/bar chart
IndexFor[Category/Value]Axis: data index
[Category/Value]AxisTitle: label

40. Teacher implementation of Who am I? game in a
German language course including augmented reality
Case study: Foreign language learning

41. Learning analytics capabilities
Google Fusion Tables data
> Ejemplo: Wir Ben Ich
Pentaho dashboard
> Projects
> Users

42. Interactions
> Compilation of students’ interactions relevant
for assessment : e.g. number of attempts to
guess a character, characters that are more
difficult to guess, etc.
Learning analytics results
Survey
> Inside the app to know learners’ opinion

43. University of Algarve, Mechanical
Engineering, “Design I” class
Learn foundations of technical drawing
including perspectives
Case study: Technical Drawing

44. APP BASED ON AUGMENTED REALITY
Welcome screen Rendering 3D models Survey on useful of
the application.

45. Different display types

46. Extended information
Touching the graph you obtain
more detailed information

47. Extension of the visual language to build
gadget-enriched mobile apps
Specific objective 3

48. VEDILS HMI components

49. http://emphasys.uca.es/
Case study: e-Health project

50. Ease the creation of enriched mobile apps by non-
programmers
Create and visualize assessment information from
the app
Conclusions

51. Integration with data mining libraries to ease assessment
interpretation
Version of the authoring framework to develop eHealth
gadget-enriched mobile apps
http://emphasys.uca.es/
Not only low cost devices: HTC Vive, Hololens, etc.
Future work

52. VEDILS LA Demo: simple query + data table

53. VEDILS LA Demo: aggregation query + chart

54. Thanks
http://vedils.uca.es

55. Images from:
http://elearningindustry.com/augmented-future-elearning-augmented-reality-elearning
http://www.educatorstechnology.com/2013/06/20-ways-to-use-augmented-reality-in.html