puppetreality

A virtual reality puppeteering experience.

• Project site: http://teeterclassic.com/puppetreality/ (has formatting problems at the moment!)
• Role: Concept, wireframes, storyboards, puppet fabrication, audio.
• Tools: DIY Virtual Reality headset, Google Hangouts.
• Deliverable: Physical-virtual reality installation (proof-of-concept).

PuppetReality is a virtual reality puppeteering experience. The puppeteer wears a VR headset giving a direct, live feed from the puppet's point of view as she directs the puppet through a custom-made set. Meanwhile, outside of the set, all other spectators watch a projection of the puppet(eer)'s view, which can be cross-cut with alternate views from security cameras hidden throughout the set, allowing for a more cinematic experience for the external spectators.

Ideation

This project began with a question: what would it be like for camera-equipped devices (i.e. our smartphones) themselves to be puppets? Some of the initial concepts were, admittedly, less than inspiring:

We narrowed the concept to a single handmade puppet (below right; see alternative below left with multiple puppets) containing a webcam, to be operated in a custom-built set. For this, we settled on a highly dramatic, well-known scene: the opening scene from Raiders of the Lost Ark, in which Indiana Jones steals a golden idol and is chased by a boulder. The familiarity of this scene, along with the visual and auditory immersion, would help the puppeteer lose herself in the scene.

Our core concept, ultimately, was a video game for which the input is reality.

Fabrication

We constructed several experimental puppets before our final design:

The final puppet design (see rightmost picture) has extensible arms and a magnet attached to its hand so that it can pick up the golden idol.

The easiest and most reliable way to set up live video feed from webcam to head-mounted device ended up being simply to make a Google Hangouts call, so this is what powers our DIY VR system. The security cameras hidden in the set run on proprietary third-party software.

The audio track was taken directly from the film, and spliced into discrete chunks corresponding to the onscreen action; during the performance, a helper manually triggers when to move onto the next spliced part, so that the audio matches what is happening inside the set -- including the eventual boulder chase.

Check the video to see how it all came together!