1. Think back on your life… What story has most influenced you?
One that you’ve…
• participated in as a child and still stays with you today.
• has offered different parts of the story through different mediums …whether
through a book, a movie, a toy.
• held up as perhaps a moral compass to guide you through life.-----------Tangible Storytelling
+ Play + Learning
…blending the digital and physical worlds to
expand children’s transmedia experiences
Erin Reilly
Managing Director + Research Fellow
USC Annenberg Innovation Lab
ereilly@usc.edu

2. Star Wars is one of the most influential story worlds in
my life.
I…
• Waited in line
• Collected the toys
• Dressed up as Princess Leia
• Passed this love onto my son
• And I’m counting down the days for the next movie----
--
Dallas, Texas – 1977 – 5 Years Old

3. A good transmedia story, like Star Wars,
• Works across media to expand the world
• Extends the timeline providing multiple plots and
white space for the fans to add their own stories
• Deepen our familiarity with the characters across
multiple platforms
• and in return deepen our engagement as fans.
It is telling and shaping stories across these different
mediums where imagination feeds upon the culture
around it and children show enormous capacity to re-
imagine the stories that enter their lives.
-------------
Transmedia
Stories…
• Expand the World
• Extend the Timeline
• Deepen the Characters
• Increase Engagement

4. As you can see here with my son who brought his two
favorite stories – Star Wars and Harry Potter – to make
his latest Halloween costume, Darth Potter.
Often re-imagination of our favorite stories is done
through play. The new media literacies’ definition of
Play is the ability to experiment with your surroundings
as a form of problem-solving and it is an important
mode of interacting with our world.
--------------

5. So you can understand my excitement when The
Museum Experience Star Wars: Where Science meets
Imagination was developed 10 years ago. This builds
off what we know of the six Star Wars films but it
creates a new kind of experience that takes advantage
of the distinctive affordances of the museum, which is
designed to feed the imagination by creating
connections to the culture and society that we live in
and offers a new lens to look at story from a different
point of view. ------------
• Opened at the Museum of
Science in 2005
• 20 venues
• Eight-year international tour
• More than 3 million visitors
worldwide experienced this
exhibition.

6. This exhibit was organized around two themes, "Getting Around" and
"Robots and People.”
The experience combined costumes and props from the Star Wars
films with interactive exhibits related to real-world technologies.
Take for example, Luke Skywalker's landspeeder. Museum-goers were
encouraged to find out how things move without touching the ground in
the real world, from models of flying cars to commercial space planes,
including jumping on an actual hovercraft. -----------

7. Visitors also had the opportunity to meet C-3PO and
R2-D2 and explore how people relate to the Droids in
Star Wars. The exhibition also featured real-world
robots that navigate and sense the world around them
while communicating in increasingly sophisticated
ways.
This exhibit was not just about bringing Entertainment
into the Museum space but a way to offer multi-modal
approaches to learning through a story.
Neither example in the exhibit builds on extensive
narrative information about Star Wars but it does
reward fans who apply what they learned in one context
to each new appearance of the story objects and
characters. -----------

8. So I found myself surprised when after visiting this
museum experience, my son and I wanted to further
explore these themes at home and we couldn’t find
anything online to enhance the experience. We found
an overview of the exhibit and a physical book
published at the end of the tour in 2011 that re-told what
we just experienced. But nothing for us to do beyond
the visit to the museum.
Here was a missed opportunity. The Star Wars Museum
extensions (the website and the book) only offered the
same content and replicated the museum information to
the website and the printed book. There were no
extensions of the Museum theme and stories to further
explore. We live in a digital age where are children
have never known a world without the internet, a world
without a mobile device where they can access their
interests through a variety of media, anytime --
anywhere. ----------
Buy the Book… that’s it?!
I went to the exhibit …Can I still play and learn?!

9. As educators, researchers, and designers at the
Annenberg Innovation lab, we are interested in the
ways in which trans media can be a resource for
learning through participation, experimentation,
expression—and, in particular, through play.------
The Flotsam Experiment
– expanding to Transmedia Play + Learning

10. So similar to the Star Wars Museum exhibit (a physical
space), we took David Wiesner’s wordless picture
book, Flotsam, (a physical book) and developed an
interactive children’s app for the iPad to experiment
with the goal of developing places to play both inside
the digital and physical realms allowing for multiple
points of entry for different types of learners. To do this
effectively, we need to build upon the key
characteristics of transmedia storytelling and expand
them to play and learning. ----------
Partners

11. For those of you unfamiliar with the Flotsam story…
This is a book that begins with a boy at the beach. He
finds a camera that washes up on shore. Has the film
developed and follows the stories that the pictures
reveal …whether it is the fantastical world under the
sea to the other children around the globe who have
handled the magical camera. ------------
A Wordless Children’s Picture Book
Flotsam is about boy at the beach, who finds a camera that washes up on shore.
He has the film developed and follows the stories that the pictures reveal.

12. What we found in looking at the Flotsam book is that at
the surface, this was a beautiful wordless picture book,
but throughout the printed pages were opportunities for
an engaging transmedia story – one that was ripe for
experimentation and offered a world where children
could not only explore fantastical worlds but their own
backyards.
Like the Star Wars story, Flotsam is a transmedia story
and embodies many of the key characteristics. The
story begs the reader to not be limited to the printed
page, but step outside the book and explore the world
for themselves. ---------
Transmedia
Stories…
• Expand the World
• Extend the Timeline
• Deepen the Characters
• Increase Engagement

13. One of the experiments we did with this book is to
explore new approaches to reading across media,
helping children develop broad literacy skills necessary
to navigate a media-saturated society.
Similar to the Star Wars Museum exhibit, Flotsam
blends science fact and science fiction as a way to
provoke curiosity. A recurring theme in the book was
geography and the images in the book incited us to ask
“Where are the giant starfish going? And, the other
children that found the camera, where do they live?”
The team enhanced these element by layering learning
about the oceans and geography depicted in the book’s
illustrations through the Interactive map features.
But the digital map was embedded inside the interactive
mobile app to expand on the reading experience. How
were we going to encourage young readers to step
outside the digital experience and explore the real world
for themselves?--------
Where do all these children live?
Where are the giant starfish and tiny
whales migrating to?

14. So we developed another experiment in the form of a
physical explorer kit, just like the one the curious boy
uses in the story. This additional tool encouraged
learners to explore their own backyards, construct
understanding and draw complex connections between
information, leading to learning that is deeply
meaningful.
However, this experiment didn’t work out quite the way
we had hoped and when we play tested this with 2nd
graders, we found it didn’t make the connections back
to the digital reading experience we were hoping for.
They ran through all the material we had provided in the
box and not enough guided instruction left them
thinking that was it.

15. …the most successful experiment that extended the
digital reading experience to a physical space was
enabled through collectable explorer cards. We
designed an actual explorer card game that had the
children use math to play a game that metaphorically
represented the oceans and had them physically up
and moving as part of the game play.
Some of the explorer cards were AR enabled and when
the card was held up in front of the iPad camera, it
unlocked additional digital assets in the book. For
example, when the whale and squid explorer cards
were brought together under the camera …one could
see on the table how realistically these 2 marine
animals would engage in the ocean… BUT even in our
efforts of truly blending the digital with physical play
…we were limited and we always had the tablet or the
mobile phone as an intermediary in the experience. -----
Game Rules
1. Your piece of Flotsam must cross the equator 3x to win the game
2. You can use your Dare card (one in the middle) if you get stuck in an “Eddy”
3. If your opponent doesn’t accept the Dare, then you have to do it in order to get unstuck from the “Eddy”
…so choose Dares that you’re willing to do too!

16. This brings us to my latest research where I’m working through questions like…
How can we offer transmedia stories and play experiences to enhance 21st century learning
that better blends our digital and physical worlds? An experience where the digital part of the
experience is not a distraction but an enhancement to the experience.----
Bridging the Physical and Digital Spaces
– Exploring “tangibility” in Transmedia Story + Play to
enhance 21st Century Learning

17. In entertainment, Disney
Infinity is an attempt at
bridging this gap. It’s very
good at pulling together
Disney’s ever-growing library
of characters into a sprawling,
shared world.
But part of blending the digital
and physical worlds together
is to encourage tangibility
where kids actually touch,
play and manipulate objects.

18. The notion of tangibility in this game is limited as they’re
figurines rather than toys.
Once you put the character on the portal to bring him
into the game, nothing’s done with that figurine until you
need to switch it out with another to advance in the
game. Furthermore, in talking with kids about bringing
their Infiniti characters into playing with other action
figures beyond the game, kids often responded “Why,
those are for the game.”
To address this problem, we need to consider
affordances of tangibility that will change the story, not
just support the story and encourage kids to play with
the objects in different ways beyond its original
intention.

19. Tangibility is something that is familiar to Children’s museums. However, what are the affordances of tangibility to consider when blending the digital
and physical spaces together.
Manipulability is where objects can be directly moved, changed or otherwise altered using one’s hands or other methods of physical interaction. By
default we think about moving the toy itself, but we have to remember we can move where we are in relation to the physical object.------
1. Manipulability
2. Physical
Connectivity
3. Performativity
Manipulability
Objects can be directly
moved, changed or
otherwise altered using
one’s hands or other
methods of physical
interaction.
Affordances of Tangibility

20. For example, take what the Anar Foundation did with
the "Only for Children” campaign.
Take a look… ---------
https://www.youtube.com/watch?v=6zoCDyQSH0o#t=50

21. Museums are known to foster co-learning among
children and adults In their lives. However, adults and
children start off inherently in different positions of
power. Manipulating viewpoints in public spaces
provides the type of flexibility for children to come to
a transmedia play experience on their own accord.
Thinking about this in a museum space, some of my
friends are working on combining learning and a fun
scavenger hunt at the Samuel Oschin Pavilion for the
Space Shuttle Endeavour. Their goal is to
manipulate the space from the eye level of a child
and provide hidden digital easter eggs for children to
collect and find throughout the experience and
discover personalized information through a trail
found only with a black light.
Blacklight and foot steps
Logo of California Science Center

22. Osmo is another good example of manipulability. Two
google engineers have recently created this mirror
device that goes onto the webcam of an ipad. This
mirror projects the play space in front of the iPad
allowing People to play with Real Objects and foster
creative thinking.
Their latest Osmo app is Masterpiece. Children can
actually pick any image (whether they take it with the
camera or find it via the galleries or web) and
Masterpiece will transform it into easy-to-follow lines to
help the child draw it, and share a time-lapse video of
the creation with friends and family.
Osmo is great way to show process and growth in fine
motor skills and muscle strength. Not to mention just a
great teaching tool to use as demonstrations and as a
critique.------
https://www.playosmo.com

23. Another affordance of Tangibility is physical connectivity
- Objects or the child’s physical body can directly
connect to, or otherwise interact with, the story, other
objects or location.-------
1. Manipulability
2. Physical
Connectivity
3. Performativity
Physical Connectivity
Objects or the child’s
physical body can directly
connect to, or otherwise
interact with, the story, other
objects or location.
Affordances of Tangibility

24. A lot of people are thinking about near field
communication, just being near something triggers a
result, like you can do with magic bands at Disney
World. Wearable devices are exploding and it opens
story possibilities to extend the timeline to our physical
location.
Right now, Disney uses this for marketing purposes but
wearables can shape and add story. For example,
imagine the child visiting Disney World waiting in a very
long line. While waiting, the child’s favorite character
could stop by for a visit or a video pops up on a screen
to further extend the story’s ride making the waiting
pass by faster than you think.

25. A wearable that I believe embodies physical
connectivity is Moff. Moff connect to a mobile app and
the wearable becomes the speaker to any object the
child picks up. For example, a child picks up a marker
and become the wizard with his wand or grabs a broom
and becomes the musician with his guitar.
How could something like this be used in a museum?
Not only can a child’s interactions with museum objects
come alive in their hands through sound, but using a
wearable like this could automate story narration if a
child stays longer in front of certain artifacts, the system
can adapt and lengthen the storyline.-----
www.moff.mobi

26. You don’t need to be Disney to tangibly blend a digital experiences with your
museum, nor do you need to give a wearable to every child that walks through
your doors.
There are start ups, like Pixie, that are digital tags where any physical object gets
an identity. Pixie offers an open technology platform for developers which
provides a world of possibilities for tangible storytelling in a public space.
Let me give you an example with one of my favorite museums. What if we placed
Pixie Tags all over the City Museum in St Louis. Data collected from the Pixies
throughout the museum can unlock different parts of a digital story and extend the
child’s museum experience when they go home, encouraging a more personal
and mobile connection to the museum. So if your child spent all of his time
exploring the dinosaur caves, he could receive digital stories on the largest
animals to ever walk the earth. While another child spent his time outside playing
in the airplane and school bus, and an exploration on travel and transportation is
shared instead.

27. The last characteristic of Tangibility is Performativity as
Objects can literally become storytellers by using visual,
auditory, or kinetic methods for the object to
communicate with the child.-----
1. Manipulability
2. Physical
Connectivity
3. Performativity
Performativity
Objects can literally become
storytellers by using visual,
auditory, or kinetic methods
for the object to
communicate with the play
partner.
Affordances of Tangibility

28. Augmented Reality is one of the lowest hanging fruit for
performativity and we saw this happen in the Flotsam
experiment when kids replayed over and over again the
squid and whale dancing or fighting (depending on who
you asked). AR adds context or content, via any
audio/visual means, to the current physical space of the
visitor. We’ve seen this done in museums where your
cell phone becomes the docent and brings the
inanimate to life. Or shifts time. Or adds additional
layers of knowledge.

29. This additional layer is great but what about having the
physical space and objects react and interact with the
people. I love this example from the Retail Design Expo
because it removes the need of taking out a mobile
device or always holding a screen in front of your
experience in order to participate in a public space.
Technology should enhance and not distract from the
museum experience.
SHOW VIDEO
https://vimeo.com/121878247

30. But what’s next…. Through examples I’ve provided that
combine IoT, Wearables, AR and sensors with story --
Hyper-personalization and customization are key
elements to the experience and encourage connections
to the DIY and maker space movement that are
popping up in communities across the US.----------
What’s next …
What role do hyper-personalization and customization play
in connecting Tangibility to Transmedia Story + Play +
Learning?

31. Products, like Lumo Play, could help to develop hyper-
personalization into a digital projection exhibit at your
museum. Not only does it offer a customized platform
for children to make their own digital projections but it
also interacts with objects turning the projection into a
reactive learning space.
SHOW VIDEO
https://www.youtube.com/watch?v=z6lf8slV7Ak

32. Moving forward, another area of creative participation
with physical objects is 3d printing.
The Maker Movement allows for more people to
customize their play + learning experience not only in
having a makerbot in your home (which is costly) but
especially with consumer access growing through
stores like Staples and even at Children’s museums
like the one in Pittsburgh.-----------
Print at Home Print at Service Provider
Print at Children’s Museums

33. Beyond developing a makerspace in your museum, one
recent example of incorporating 3d printing into an
exhibit comes from the collaboration between 3D
Printing 4 Everyone and the New York’s Children’s
Museum of the Arts. “Into the 3rd Dimension” exhibit
places Kids Creation Station prints alongside 2D
artworks from the museum’s permanent collection to
allow patrons to ponder the two forms.
This is an example of a more free form and open
concept of hyper personalization and customization by
taking any drawing that a child has created and turning
it into a tangible experience. A made to order one off..
However this has a limited audience due to the cost.

34. So when thinking about effectively incorporating 3d printing into a transmedia play + learning experience at a museum, we need to consider these questions.
• How do we make a 3D printed artifact change the story in an exhibit?
• How do we do something simple enough that it is not too costly?
• How do we design for the usual timed museum visit for a child to manipulate it and use it to augment a museum exhibit?
• And if it doesn’t need to happen during the visit, can it be something mailed as a way to extend the experience beyond the museum?------------
Questions to consider…
• How do we make a 3D printed artifact change the story
in an exhibit?
• How do we do something simple enough that it is not
too costly?
• How do we design for the usual timed museum visit for
a child to manipulate it and use it to augment a museum
exhibit?
• And if it doesn’t need to happen during the visit, can it be
something mailed as a way to extend the experience
beyond the museum?

35. Another experiment we did at the lab with FOX and the television
show, Sleepy Hollow, helps to address these questions. We
developed a 3d printed puzzle (small enough to print in 30 minutes)
where the configuration of the puzzle pieces triggers additional story
content of Sleepy Hollow using AR.
Though this doesn’t allow for hyper-personalization or customization,
it does afford the characteristics of tangibility to directly impact the
story itself.

36. To really consider tangibility in Transmedia Play + Learning, we were
Interested in how the artifact can drive the story.
This takes us to the Winklebeans experiment, a six week stint we started
last summer. With this research, we focused on what can be done with
physical objects, and what unique participatory affordances these objects
offer in storytelling.---------
The Winklebeans
Experiment
– Tangible Storytelling
+ Play + Learning
Team: Erin Reilly, Alisa Katz, Geoffrey Long, Aninoy Mahapatra, Daniel Burwen, Mitch Thompson, Shane Reilly

37. Winklebeans are little wooden monsters with magnetic interchangeable sensored pieces. Their story world
is about togetherness and through mixing and matching the Winklebean pieces, kids learn the skill of
sharing.-------
Meet the Winklebeans!
Toag – Neas – Caitir – Onchu – Aodh – Barra
www.winklebean.com

38. To get started, the card the comes with the toy provides
simple instructions to put your Winklebean together and
sync your tangible smart toy with the mobile story app.
Let me show you a quick demo of how manipulating the
toy changes the story.
https://vimeo.com/103361002

39. The combination of the built in audio recorder with the
Winklebean’s awareness of its location combines
physical and digital ways to play! You can interact with
your Winklebean 1-on-1 where the object become the
child’s play partner and encourages joint media
engagement in annotating our public spaces.
The combination of the built in audio recorder with the
Winklebean’s awareness of its location combines
physical and digital ways to play! You can interact with
your Winklebean 1-on-1 where the object become the
child’s play partner and encourages joint media
engagement in annotating our public spaces.

40. Another attribute in the Winklebean story world is the
actions of “mix and match” and “grab and swap.”
Transmedia play and storytelling involve exploration,
experimentation, and remix, all activities that are firmly
aligned with a constructivist approach to learning and
by adding opportunities through 3d printing, children
can create their own elements to add and change the
Winklebean story.
Children can 3d Print their own Winklebean elements to add to the Story World

41. In our project we’ve checked all the key characteristics
of Tangibility that are highly supportive of learning within
a complex story world and bridges us closer to blending
the physical and digital experiences without the digital
screen being a distraction.--------
Tangibility
 Manipulability
 Physical
Connectivity
 Performativity
Transmedia Story +
Play + Learning
Hyper-personalization
Customization
Constructivist learning

42. So imagine how your museum can really create a
paradigm shift in play where the child has as much
freedom to add to the story in your exhibits as the
original curator. Your exhibits and learning experiences
move beyond the public space into the complementary
digital space, allowing for museum extensions that
further enhance the story and in return makes our
cultural past interesting, engaging and relevant for the
visitor of the future.
Thinking back on the Star Wars exhibit … I know my
son and I would be jumping for joy if this had happened
and unlocked some further learning to explore through
a story we’re both passionate about.---------
Now Imagine...
Flotsam Experiment

43. Thank you.
Add contact information …next steps …twitter, email,
phone etc-------
For More Information
Erin Reilly
207-251-1617
ereilly@usc.edu
@ebreilly