The field of Virtual Reality is still a new frontier, and the stories that come through our studio constantly surprise and delight us.

Lincoln in the Bardo is the debut novel from George Saunders, and it is already generating a ton of buzz. It’s a work of historical fiction, based on the story of Abraham Lincoln after the death of his son Willie. As the Civil War was in its infancy, Lincoln’s young son fell ill and died; Lincoln was rumored to be so grief-stricken that he went to the graveyard to hold his son’s corpse a few days after his passing.

Some novels lend themselves to be adaptable for the stage. Others make excellent transitions to scripts for film or television. Lincoln in the Bardo is a story that is all at once so haunting and heartbreaking that it adapts well to something as deeply immersive as a VR experience.

We had the pleasure of working with filmmaker Graham Sack, The New York Times VR, Sensorium Works, Plympton, and Silver Sound to create this VR experience in time for the novel’s release on February 14.

PRODUCTION

The Molecule on-set VFX supervised the filming of Lincoln in the Bardo in November of 2016.

In Greenwood Cemetery in Brooklyn, NY we filmed the background plate and the live-action plate of Abraham Lincoln and his son Willie’s body with a VR rig. (Right: Graham Sack on set in Greenwood Cemetery)

Later on we filmed the ghost actors individually and in small groups against a green screen on a sound stage with a single-lens camera. This strategy was helpful in post production, as we could have more control over how their placement in VR space.

Photo by Matthew Niederhauser, Sensorium Works

POST PRODUCTION AND VFX

Assembling this particular project was uniquely difficult because the final export was a continuous shot.

The finished piece was over 35,000 frames long. We had to find a unique way to break it down into workable pieces, so the solution became a combination of using compositing software (we use Nuke in-house) and Adobe Premiere.

VR Compositing Supervisor Selim Yang explains that it would have been problematic to render the entire scene from Nuke. “This piece was about 10 minutes without cuts,” she says. “Since Nuke is shot-based software, it could have been very difficult to composite hundreds of elements for a 10-minute sequence in Nuke.”

Our team stitched the background footage in Nuke and rendered it out. They rotoscoped Lincoln and his son in Nuke, and rendered out the full-length scene with an alpha layer. And finally, they applied the ghost effect to the actors and rendered them all out individually. VR Producer Drew Wood then integrated all of the layers in Adobe Premiere.

Sensorium Works edited the piece, and Silver Sound produced the sound.

THE POWER OF STORY

Sometimes the creative process can desensitize artists to the story itself. In VFX especially, because artists break down full scenes to individual frames, it can be difficult to remember to see the forest for the trees.

This wasn’t the case for our artists with Lincoln. Everyone on our team was touched by this story, and they were proud to be a part of sharing it with the world.

As poignant as the experience is, it also has its chilling moments (don’t forget, it takes place in a graveyard). “It’s really spooky,” Selim recalls. “The ghosts walk right up to you.”

Photo by Matthew Niederhauser, Sensorium Works

IMMERSE YOURSELF

The original novel by George Saunders is already receiving accolades far and wide, so if you’re the type of person who likes to read the book before seeing anything else, you should definitely start there.

Penguin Random House Audio released an audiobook along with the novel, and it features the voices of many familiar celebrities like Ben Stiller, Julianne Moore, and David Sedaris.

Lincoln in the Bardo is also on a country-wide book tour, where you can see our VR piece in action in cities including New York, Boston, Los Angeles, Chicago, Washington D.C., and Austin.

Finally, you can experience the story in VR with your smartphone, the VR App from The New York Times, and Google Cardboard (recommended), or as scrollable 360° video (below).

However you choose to experience Lincoln in the Bardo, we hope you enjoy it as much as we enjoyed being a part of sharing it with you.