I’ve been in production on two documentaries for almost a year and a half. One of the documentaries has had a pretty diverse media menu that breaks down as follows:
A Camera – RED Epic Dragon, Canon C200, Canon C300 MKII
B Camera – Canon XF405 x 3
Gimbal Cameras – Canon 80D, iPhone 8 Plus
Go Pros – Between six and 10 GoPros for each shoot
Still Cameras – Canon 7D MKII, Canon 80D
While it’s obvious that each shoot has generated a considerable amount of media, what has been weighing on my mind is how I’m downloading, archiving and locating all of the media. Our strategy has been as follows. After each shoot, one of our producers takes all of the media shot to our office. This documentary is shot outdoor on boats, so managing media on-site was ruled out, as the beach is a lousy place to clone media if you can avoid it. Once back at our office, we immediately download all of the media using Shotput Pro to two drives simultaneously using ShotPut Pro’s checksum verification. This is a technology that actually will compare the transferred media itself with the original, not just comparing file size, but the actual data. It takes a bit longer, but when using it, you can be assured that your media has actually ALL transferred over to your drive copy.
Once the media lives on two storage drives, we’ve decided that it’s important to include geographic diversity in our media strategy. My own office is located in Ventura County. As you may or may not have recalled from last year and early this year, one of California’s largest and most costly wildfires on record, the Thomas Fire, occurred less than 15 miles from our office. We were packed up and ready to evacuate for over a week. A friend who I had recently transferred a lot of footage to a drive for actually had her office burn down in the same Thomas Fire, and the drives I had sent her were nothing but little blobs of molten metal afterward. This made me think about how the only truly safe and effective media management strategy needs to include at least two, preferably three, copies of media located in at least two or preferably three different geographic locations. Is this a pain? Yes, undoubtedly. Fires, floods and hurricanes are all actually increasing in frequency with the global climate changes. It makes sense to at least attempt to geographically locate at least two copies (three is better if you can afford it) of your media to protect it.
One solution that’s intriguing is online backup in the cloud. Most producers have two problems with this strategy. The first is the questionable security that’s present with putting your assets anywhere on the internet. The second is the practicality of uploading huge media files from most locations. If you’re lucky enough to have truly high-speed internet access, like 300 to 500 Mbps access or greater, it may be somewhat practical to upload large media files. But for most of us, we simply don’t have enough bandwidth to back up all of our media online, especially those of us shooting RAW formats. For now, for our needs, we’re simply using two and then eventually three sets of drives, one with each project producer in two different geographical location with a third set of media that will be cloned and sent on larger-sized drives to our editor who’s in the Bay area to reduce the amount of drives we have to send him.
This takes care of the day-to-day media management needs for a typical project. But what about long-term media archival storage? Unfortunately, storing all of your projects and their associated media on spinning hard drives is somewhat economically viable. Too bad that hard drives aren’t a reliable long-term storage medium. I have some media-filled hard drives that are 10 to 15 years old. I take them out twice a year, plug them in and spin them up to speed to make sure that they still work. Each time I do this, at least one and sometimes more drives fail to boot. There are many reasons why this happens, but it’s been a constant reminder that the digital media we shoot and make a living with these days is a temporary asset unless you archive it to a more reliable format than hard drives. This paper from SCART, a Belgian website, offers a good overview of the different file formats and containers.
The National Archives offers this handy guideline for which file formats and containers are thought to be best for long-term archival storage. The bottom line, though, is that there’s not a perfect, economically viable storage system for digital media yet. LTO Tape drives are known to have better long-term storage reliability than any spinning drive but are expensive and slow to write to and read from. Motion picture film has also been proven to last more than 100 years with proper storage conditions, but it’s a tad bit expensive to lay off your digital files to 35mm motion picture film. Realistically, the last generation of analog media lasts longer than the current state-of-the-art digital media. This leaves us in somewhat of a conundrum. How much of the media that you’re creating today will need to be viewable in 50 or 100 years? Kind of makes your head spin, no?
The great thing about digital media is that it can be cloned indefinitely. So, if you have material that you’re concerned about lasting more than about a decade, it’s viable to keep multiple copies of it and then continually clone the data to newer media as it becomes available. That’s expensive and a pain, but it increases the chances of your media being viable for more than a decade.
Good luck, and keep your ear out for the latest developments from technology companies who are still trying to crack the digital archival issue.