What is it?
In A World is an AR (Augmented Reality) storybook for children which allows kids to reimagine and create their own story endings through play. It lets children play with physical characters in the real world, while those characters simultaneously react to virtual objects on a screen and play certain animations.
For this demonstration, we chose the story of The Three Little Pigs. As the child flips through the last page of the storybook, she finds detachable characters from the story. She can then move them around in her own environment as she narrates her own version of the story ending. When her friend points a camera-enabled device at the characters, she is able to view rich background scenes where the digital characters, controlled through the detached characters, interact with each other and play certain micro interactions and animations. These animations do not direct the story in anyway and are open to interpretation.
How does it work?
Each detachable character from the book has a unique marker called an AR marker or fiducial. These are visual cues which trigger the display of virtual information when seen through a camera. The positions of the fiducial inform the positions of the virtual characters.
Unique animations are programmed to trigger when the virtual characters interact with each other and with other objects in the virtual space.
Process
Starting with an open brief on exploring AR / VR / XR technologies, we researched various potentials and the downsides of Augmented Reality (AR). We decided to specifically focus on ideas to enhance the potentials of this technology and came up with the challenge:
How might we use Augmented Reality to inspire creativity in children?
The question made us think about what imagination means to kids. All kids love to play, we thought the best way to engage with them and inspire their creativity would be through play. After ideating on many ways to incorporate AR in children’s play, we decided to go with the concept of a storybook where kids can create unique narratives for the story ending while AR helps in stimulating their imagination by displaying virtual scenes and animations.
We generated fiducials for the main characters and for the scene. Character illustrations were overlaid onto the fiducials and cut into shapes that could be moved around. These fiducials were inserted into the last page of the storybook, with a set of instructions on how to detach and use.
The animations and the augmented reality scenes were created using Unity and Cinema 4D.
Value
Children love telling stories. This immersive experience aids in stimulating their already rich imagination with the help of colourful virtual environment and playful animations.
- The tactile interaction with the physical characters helps with sensory stimulation, thus building nerve connections in the brain and promoting normal development.
- When played as a group, it enables children to tell stories as well as listen to a story from others’ perspectives, thus enhancing their listening and language skills.
Future scope
For further development of this project, we see a lot of potential in exploring these ideas:
- Record and share stories in a classroom environment.
- Collaborate with friends on story endings.
- A never-ending story instead of a fixed ‘The End’.
- Richer play experience with sound, physical textures and shapes.
This project was done as part of the one-week course on Immersive Experiences at CIID, along with my teammate Chaeri Bong.