In film production, the phrase “We’ll fix it in post” has always been synonymous with mistakes and failures. Just this month, I was on a commercial set calling for a live Bengal tiger to be shot in front of a greenscreen. Unfortunately, the spot’s director and production team were so enamored with filming the beautiful animal that they completely forgot to shoot plate images (i.e., empty shots of each scene to superimpose the tiger into later) until production had wrapped. This forced the agency to choose between two costly alternatives—either reshoot the scenes or find a workaround in postproduction.
Almost every filmmaker can relate to these experiences, be it lack of coverage, blurry takes or an ever-present boom mic suddenly entering the frame. But such impracticalities in production could soon be a thing of the past with the introduction of the Lytro Immerge light-field camera.
The camera system is built around capturing a 3D light-field volume that allows creators the freedom to manipulate and manage all aspects of their images in postproduction rather than at the point of capture. It can also be used on professional virtual reality (VR) productions with technology allowing for a level of immersion that simply hasn’t been possible from any other camera in existence.
THE PROMISE OF LIGHT FIELD
If you study the history of photography from the early daguerreotypes to today’s high-resolution digital images, it seems amazing how far we’ve come. But innovation in photography hasn’t accelerated as much as we may surmise. We still capture all of our images on a flat 2D plane, which is then relegated to a small rectangle. However, the move from analog to digital has allowed for an explosion in the technology of photography and video capture.
Now, through new research in light-field technology, it’s possible for extensive breakthroughs in optical and computational imaging, something Lytro and many of Silicon Valley’s brightest investors believe is the future of VR production.
Without a doubt, VR is a burgeoning medium for narrative content. One only need look at the early work of primo content creators such as Félix & Paul, VRSE and WEVR to recognize that creativity in the VR space is already outpacing the limits of current production technology.
Nothing is more important than “presence” in VR, the feeling of being completely immersed and transported into a new world. Current camera rig systems used in production have become an impediment to this presence, providing only static flat image capture with no solutions to significant problems including image calibration, resolution or parallax distortion, just to name a few.
The debut of spherical 360º video was an important step in the introduction of VR to the masses, but the promise of accessing worlds where people can move and interact with each other is the true potential of the medium, something VR consumers already understand.
The Immerge camera has a radical multi-tiered lens system that’s unlike anything ever witnessed on a camera. Its design allows for thousands of microlenses to capture a scene from numerous perspectives that are all built into a final capture model from a full light-field volume.
This allows users the freedom for more realistic head movement, as well as the ability to change perspective while moving around in space, something that has only been achieved in video games and CGI.
THE CAPABILITIES OF LIGHT FIELD
The Lytro Immerge is radically different from any other camera on the market. Its light-field technology uses an array of microlenses placed in front of an image sensor to sense the intensity, color and direction of all rays of light within a given space.
These thousands of tiny microlenses allow for capture of the entire light-field volume, along with light ray data for the Lytro Immerge to create a 3D model of the scene being viewed. The camera is always heavily oversampling in every direction to record as much information about the scene as possible. Along with that oversampling comes an obscene amount of data.
To handle that data, the camera comes with its own large server to store and process captured footage. Lytro is also developing other end-to-end solutions, such as postproduction tools that can be integrated into existing VFX tools such as NUKE, as well as a cloud-based video player for distribution.
Despite the obvious drawback of having to output immense sizes of data to a server, the camera does offer filmmakers a torrent of features that will enhance the filmmaking experience. Along with that comes extensive control in postproduction over the images they capture.
Here are a few more of the varied features of the Lytro Immerge light-field camera.
Depth of Field. One of the core benefits of the camera is infinite depth of field. Everything captured can be placed in focus, or focus points can be changed and resampled within the scene. The Immerge also offers a shallow depth of field far beyond any traditional camera system, offering capabilities far beyond any traditional or VR camera rig available on the market.
Another feature of this infinite depth-of-field scenario is that it plays into other features such as computational frame rates and shutter angles, allowing filmmakers to play with things such as frame rate and motion blur on an artistic level in post, rather than having to “bake in” these elements at the moment of capture.
Dynamic Range. The Immerge has, in essence, an aperture that never has to be stopped down, meaning it’s capable of shooting in extreme low light, even scenes with candlelight or moonlight using a wide depth of field. This isn’t even conceivable with other contemporary cameras.
Even more amazing is an ability to change exposure and dynamic range later in postproduction, possible due to the camera’s constant oversampling of data pulled from each image captured. This allows filmmakers an unprecedented level of freedom that no other 2D camera comes close to matching.
Lens Replication. Yet another massive benefit for filmmakers is the camera’s computational optics systems that can match the character of any predefined lens on the market. Since the oversampled master shot from the light-field camera has a flat response, the image can be augmented in postproduction to replicate the parameters of any selected lens. This means no more spending thousands of dollars on rare lenses or having to rent cases of priceless anamorphic lenses from a camera house.
Visual Effects. The Lytro Immerge lends itself especially well to productions requiring heavy VFX work. It can semiautomate several tedious processes while eliminating several others all together, leading to major cost-cutting benefits for those using it in VFX.
Because light field is always measuring the light, plus where the subjects are in relation to the camera in a 3D model, each object already exist as a layer. That means there’s no need to isolate an asset, or person, in front of a depth screen for VFX shots, meaning you can input VFX into the scene without the use of a greenscreen.
Implementing VFX into any footage shot on the Immerge should be seamless, as their postproduction tools have been built to be integrated easily with VFX programs such as the aforementioned NUKE.
Other incredible features include automated 3D camera tracking and the freedom to re-light a live-action scene later in post.
Time will tell if the Immerge can live up to its full potential and its lofty promises, but delivery of just one of these game-changing features is an incredible step toward easier production for 2D and immersive live-action experience.
With several major position-tracking headsets slated for release across 2016, Lytro could find its camera system in high demand. One thing is certain: Light field points to a world in which we no longer enter production with a goal to shoot for post, but rather to shoot in post after the production is done.
The Lytro Immerge will be released before the end of 2016. Check out Lytro.com for more news and information.