Think back on your life… What story has most influenced you?
One that you’ve… participated in as a child and still stays with you today. has offered different parts of the story through different mediums …whether through a book, a movie, a toy. held up as perhaps a moral compass to guide you through life.
Star Wars is one of the most influential story worlds in my life. I… Waited in line Collected the toys Dressed up as Princess Leia Passed this love onto my son And I’m counting down the days for the next movie
A good transmedia story, like Star Wars, Works across media to expand the world Extends the timeline providing multiple plots and white space for the fans to add their own stories Deepen our familiarity with the characters across multiple platforms and in return deepen our engagement as fans.
It is telling and shaping stories across these different mediums where imagination feeds upon the culture around it and children show enormous capacity to re-imagine the stories that enter their lives.
As you can see here with my son who brought his two favorite stories – Star Wars and Harry Potter – to make his latest Halloween costume, Darth Potter.
Often re-imagination of our favorite stories is done through play. The new media literacies’ definition of Play is the ability to experiment with your surroundings as a form of problem-solving and it is an important mode of interacting with our world.
In transmedia play, children participate in both old and new media and often children’s stories and experimentation is done socially
So you can understand my excitement when The Museum Experience Star Wars: Where Science meets Imagination was developed 10 years ago. This builds off what we know of the six Star Wars films but it creates a new kind of experience that takes advantage of the distinctive affordances of the museum, which is designed to feed the imagination by creating connections to the culture and society that we live in and offers a new lens to look at story from a different point of view.
This exhibit was organized around two themes, "Getting Around" and "Robots and People.”
The experience combined costumes and props from the Star Wars films with interactive exhibits related to real-world technologies.
Take for example, Luke Skywalker's landspeeder. Museum-goers were encouraged to find out how things move without touching the ground in the real world, from models of flying cars to commercial space planes, including jumping on an actual hovercraft.
Visitors also had the opportunity to meet C-3PO and R2-D2 and explore how people relate to the Droids in Star Wars. The exhibition also featured real-world robots that navigate and sense the world around them while communicating in increasingly sophisticated ways.
This exhibit was not just about bringing Entertainment into the Museum space but a way to offer multi-modal approaches to learning through a story.
Neither example in the exhibit builds on extensive narrative information about Star Wars but it does reward fans who apply what they learned in one context to each new appearance of the story objects and characters.
So I found myself surprised when after visiting this museum experience, my son and I wanted to further explore these themes at home and we couldn’t find anything online to enhance the experience. We found an overview of the exhibit and a physical book published at the end of the tour in 2011 that re-told what we just experienced. But nothing for us to do beyond the visit to the museum.
Here was a missed opportunity. The Star Wars Museum extensions (the website and the book) only offered the same content and replicated the museum information to the website and the printed book. There were no extensions of the Museum theme and stories to further explore. We live in a digital age where are children have never known a world without the internet, a world without a mobile device where they can access their interests through a variety of media, anytime -- anywhere.
As educators, researchers, and designers at the Annenberg Innovation lab, we are interested in the ways in which transmedia can be a resource for learning through participation, experimentation, expression—and, in particular, through play.
So similar to the Star Wars Museum exhibit (a physical space), we took David Wiesner’s wordless picture book, Flotsam, (a physical book) and developed an interactive children’s app for the iPad to experiment with the goal of developing places to play both inside the digital and physical realms allowing for multiple points of entry for different types of learners. To do this effectively, we need to build upon the key characteristics of transmedia storytelling and expand them to play and learning.
For those of you unfamiliar with the Flotsam story… This is a book that begins with a boy at the beach. He finds a camera that washes up on shore. Has the film developed and follows the stories that the pictures reveal …whether it is the fantastical world under the sea to the other children around the globe who have handled the magical camera.
What we found in looking at the Flotsam book is that at the surface, this was a beautiful wordless picture book, but throughout the printed pages were opportunities for an engaging transmedia story – one that was ripe for experimentation and offered a world where children could not only explore fantastical worlds but their own backyards.
Like the Star Wars story, Flotsam is a transmedia story and embodies many of the key characteristics. The story begs the reader to not be limited to the printed page, but step outside the book and explore the world for themselves.
One of the experiments we did with this book is to explore new approaches to reading across media, helping children develop broad literacy skills necessary to navigate a media-saturated society.
Similar to the Star Wars Museum exhibit, Flotsam blends science fact and science fiction as a way to provoke curiosity. A recurring theme in the book was geography and the images in the book incited us to ask “Where are the giant starfish going? And, the other children that found the camera, where do they live?” The team enhanced these element by layering learning about the oceans and geography depicted in the book’s illustrations through the Interactive map features.
But the digital map was embedded inside the interactive mobile app to expand on the reading experience. How were we going to encourage young readers to step outside the digital experience and explore the real world for themselves?
So we developed another experiment in the form of a physical explorer kit, just like the one the curious boy uses in the story. This additional tool encouraged learners to explore their own backyards, construct understanding and draw complex connections between information, leading to learning that is deeply meaningful.
However, this experiment didn’t work out quite the way we had hoped and when we play tested this with 2nd graders, we found it didn’t make the connections back to the digital reading experience we were hoping for. They ran through all the material we had provided in the box and not enough guided instruction left them thinking that was it.
…the most successful experiment that extended the digital reading experience to a physical space was enabled through collectable explorer cards. We designed an actual explorer card game that had the children use math to play a game that metaphorically represented the oceans and had them physically up and moving as part of the game play.
Some of the explorer cards were AR enabled and when the card was held up in front of the iPad camera, it unlocked additional digital assets in the book. For example, when the whale and squid explorer cards were brought together under the camera …one could see on the table how realistically these 2 marine animals would engage in the ocean… BUT even in our efforts of truly blending the digital with physical play …we were limited and we always had the tablet or the mobile phone as an intermediary in the experience.
This brings us to my latest research where I’m working through questions like…
How can we offer transmedia stories and play experiences to enhance 21st century learning that better blends our digital and physical worlds? An experience where the digital part of the experience is not a distraction but an enhancement to the experience.
In entertainment, Disney Infinity is an attempt at bridging this gap. It’s very good at pulling together Disney’s ever-growing library of characters into a sprawling, shared world.
But part of blending the digital and physical worlds together is to encourage tangibility where kids actually touch, play and manipulate objects.
The notion of tangibility in this game is limited as they’re figurines rather than toys.
Once you put the character on the portal to bring him into the game, nothing’s done with that figurine until you need to switch it out with another to advance in the game. Furthermore, in talking with kids about bringing their Infiniti characters into playing with other action figures beyond the game, kids often responded “Why, those are for the game.”
To address this problem, we need to consider affordances of tangibility that will change the story, not just support the story and encourage kids to play with the objects in different ways beyond its original intention.
Tangibility is something that is familiar to Children’s museums. However, what are the affordances of tangibility to consider when blending the digital and physical spaces together.
Manipulability is where objects can be directly moved, changed or otherwise altered using one’s hands or other methods of physical interaction. By default we think about moving the toy itself, but we have to remember we can move where we are in relation to the physical object.
For example, take what the Anar Foundation did with the "Only for Children” campaign.
Take a look…
Museums are known to foster co-learning among children and adults In their lives. However, adults and children start off inherently in different positions of power. Manipulating viewpoints in public spaces provides the type of flexibility for children to come to a transmedia play experience on their own accord.
Thinking about this in a museum space, some of my friends are working on combining learning and a fun scavenger hunt at the Samuel Oschin Pavilion for the Space Shuttle Endeavour. Their goal is to manipulate the space from the eye level of a child and provide hidden digital easter eggs for children to collect and find throughout the experience and discover personalized information through a trail found only with a black light.
Osmo is another good example of manipulability. Two google engineers have recently created this mirror device that goes onto the webcam of an ipad. This mirror projects the play space in front of the iPad allowing People to play with Real Objects and foster creative thinking.
Their latest Osmo app is Masterpiece. Children can actually pick any image (whether they take it with the camera or find it via the galleries or web) and Masterpiece will transform it into easy-to-follow lines to help the child draw it, and share a time-lapse video of the creation with friends and family.
Osmo is great way to show process and growth in fine motor skills and muscle strength. Not to mention just a great teaching tool to use as demonstrations and as a critique.
Another affordance of Tangibility is physical connectivity - Objects or the child’s physical body can directly connect to, or otherwise interact with, the story, other objects or location.
A lot of people are thinking about near field communication, just being near something triggers a result, like you can do with magic bands at Disney World. Wearable devices are exploding and it opens story possibilities to extend the timeline to our physical location.
Right now, Disney uses this for marketing purposes but wearables can shape and add story. For example, imagine the child visiting Disney World waiting in a very long line. While waiting, the child’s favorite character could stop by for a visit or a video pops up on a screen to further extend the story’s ride making the waiting pass by faster than you think.
A wearable that I believe embodies physical connectivity is Moff. Moff connect to a mobile app and the wearable becomes the speaker to any object the child picks up. For example, a child picks up a marker and become the wizard with his wand or grabs a broom and becomes the musician with his guitar.
How could something like this be used in a museum? Not only can a child’s interactions with museum objects come alive in their hands through sound, but using a wearable like this could automate story narration if a child stays longer in front of certain artifacts, the system can adapt and lengthen the storyline.
You don’t need to be Disney to tangibly blend a digital experiences with your museum, nor do you need to give a wearable to every child that walks through your doors.
There are start ups, like Pixie, that are digital tags where any physical object gets an identity. Pixie offers an open technology platform for developers which provides a world of possibilities for tangible storytelling in a public space.
Let me give you an example with one of my favorite museums. What if we placed Pixie Tags all over the City Museum in St Louis. Data collected from the Pixies throughout the museum can unlock different parts of a digital story and extend the child’s museum experience when they go home, encouraging a more personal and mobile connection to the museum. So if your child spent all of his time exploring the dinosaur caves, he could receive digital stories on the largest animals to ever walk the earth. While another child spent his time outside playing in the airplane and school bus, and an exploration on travel and transportation is shared instead.
The last characteristic of Tangibility is Performativity as Objects can literally become storytellers by using visual, auditory, or kinetic methods for the object to communicate with the child.
Augmented Reality is one of the lowest hanging fruit for performativity and we saw this happen in the Flotsam experiment when kids replayed over and over again the squid and whale dancing or fighting (depending on who you asked). AR adds context or content, via any audio/visual means, to the current physical space of the visitor. We’ve seen this done in museums where your cell phone becomes the docent and brings the inanimate to life. Or shifts time. Or adds additional layers of knowledge.
This additional layer is great but what about having the physical space and objects react and interact with the people. I love this example from the Retail Design Expo because it removes the need of taking out a mobile device or always holding a screen in front of your experience in order to participate in a public space. Technology should enhance and not distract from the museum experience.
But what’s next…. Through examples I’ve provided that combine IoT, Wearables, AR and sensors with story -- Hyper-personalization and customization are key elements to the experience and encourage connections to the DIY and maker space movement that are popping up in communities across the US.
Products, like Lumo Play, could help to develop hyper-personalization into a digital projection exhibit at your museum. Not only does it offer a customized platform for children to make their own digital projections but it also interacts with objects turning the projection into a reactive learning space.
Moving forward, another area of creative participation with physical objects is 3d printing.
The Maker Movement allows for more people to customize their play + learning experience not only in having a makerbot in your home (which is costly) but especially with consumer access growing through stores like Staples and even at Children’s museums like the one in Pittsburgh.
Beyond developing a makerspace in your museum, one recent example of incorporating 3d printing into an exhibit comes from the collaboration between 3D Printing 4 Everyone and the New York’s Children’s Museum of the Arts. “Into the 3rd Dimension” exhibit places Kids Creation Station prints alongside 2D artworks from the museum’s permanent collection to allow patrons to ponder the two forms.
This is an example of a more free form and open concept of hyper personalization and customization by taking any drawing that a child has created and turning it into a tangible experience. A made to order one off.. However this has a limited audience due to the cost.
So when thinking about effectively incorporating 3d printing into a transmedia play + learning experience at a museum, we need to consider these questions. How do we make a 3D printed artifact change the story in an exhibit? How do we do something simple enough that it is not too costly? How do we design for the usual timed museum visit for a child to manipulate it and use it to augment a museum exhibit? And if it doesn’t need to happen during the visit, can it be something mailed as a way to extend the experience beyond the museum?
Another experiment we did at the lab with FOX and the television show, Sleepy Hollow, helps to address these questions. We developed a 3d printed puzzle (small enough to print in 30 minutes) where the configuration of the puzzle pieces triggers additional story content of Sleepy Hollow using AR.
Though this doesn’t allow for hyper-personalization or customization, it does afford the characteristics of tangibility to directly impact the story itself.
To really consider tangibility in Transmedia Play + Learning, we were Interested in how the artifact can drive the story.
This takes us to the Winklebeans experiment, a six week stint we started last summer. With this research, we focused on what can be done with physical objects, and what unique participatory affordances these objects offer in storytelling.
Winklebeans are little wooden monsters with magnetic interchangeable sensored pieces. Their story world is about togetherness and through mixing and matching the Winklebean pieces, kids learn the skill of sharing.
To get started, the card the comes with the toy provides simple instructions to put your Winklebean together and sync your tangible smart toy with the mobile story app.
Let me show you a quick demo of how manipulating the toy changes the story.
The combination of the built in audio recorder with the Winklebean’s awareness of its location combines physical and digital ways to play! You can interact with your Winklebean 1-on-1 where the object become the child’s play partner and encourages joint media engagement in annotating our public spaces.
Another attribute in the Winklebean story world is the actions of “mix and match” and “grab and swap.” Transmedia play and storytelling involve exploration, experimentation, and remix, all activities that are firmly aligned with a constructivist approach to learning and by adding opportunities through 3d printing, children can create their own elements to add and change the Winklebean story.
In our project we’ve checked all the key characteristics of Tangibility that are highly supportive of learning within a complex story world and bridges us closer to blending the physical and digital experiences without the digital screen being a distraction.
So imagine how your museum can really create a paradigm shift in play where the child has as much freedom to add to the story in your exhibits as the original curator. Your exhibits and learning experiences move beyond the public space into the complementary digital space, allowing for museum extensions that further enhance the story and in return makes our cultural past interesting, engaging and relevant for the visitor of the future.
Thinking back on the Star Wars exhibit … I know my son and I would be jumping for joy if this had happened and unlocked some further learning to explore through a story we’re both passionate about.
Add contact information …next steps …twitter, email, phone etc
Childrens Museums: Tangible Storytelling + Play + Learning
+ Play + Learning
…blending the digital and physical worlds to
expand children’s transmedia experiences
Managing Director + Research Fellow
USC Annenberg Innovation Lab
A Wordless Children’s Picture Book
Flotsam is about boy at the beach, who finds a camera that washes up on shore.
He has the film developed and follows the stories that the pictures reveal.
• Expand the World
• Extend the Timeline
• Deepen the Characters
• Increase Engagement
Where do all these children live?
Where are the giant starfish and tiny
whales migrating to?
1. Your piece of Flotsam must cross the equator 3x to win the game
2. You can use your Dare card (one in the middle) if you get stuck in an “Eddy”
3. If your opponent doesn’t accept the Dare, then you have to do it in order to get unstuck from the “Eddy”
…so choose Dares that you’re willing to do too!
Bridging the Physical and Digital Spaces
– Exploring “tangibility” in Transmedia Story + Play to
enhance 21st Century Learning
Objects can be directly
moved, changed or
otherwise altered using
one’s hands or other
methods of physical
Affordances of Tangibility
Objects or the child’s
physical body can directly
connect to, or otherwise
interact with, the story, other
objects or location.
Affordances of Tangibility
Objects can literally become
storytellers by using visual,
auditory, or kinetic methods
for the object to
communicate with the play
Affordances of Tangibility
Print at Home Print at Service Provider
Print at Children’s Museums
Questions to consider…
• How do we make a 3D printed artifact change the story
in an exhibit?
• How do we do something simple enough that it is not
• How do we design for the usual timed museum visit for
a child to manipulate it and use it to augment a museum
• And if it doesn’t need to happen during the visit, can it be
something mailed as a way to extend the experience
beyond the museum?
– Tangible Storytelling
+ Play + Learning
Team: Erin Reilly, Alisa Katz, Geoffrey Long, Aninoy Mahapatra, Daniel Burwen, Mitch Thompson, Shane Reilly
Meet the Winklebeans!
Toag – Neas – Caitir – Onchu – Aodh – Barra