Reliable Data, Part 3

Over a series of posts (that you can find here and here), I have talked about keeping your footage safe during post. Most discussions on data safekeeping are about multiple location backups and long-term archiving. While all of that is very important, I have concentrated on thinking about data safety during an edit. What would happen if you lost footage during the edit process? How quickly could you get the data back and continue editing? Would you still make the deadline?

In part two, I talked about using RAID storage to keep working if you lose a drive…or two. But what if there’s a problem with the RAID hardware? Although I keep a second power supply on hand, I don’t keep a second full RAID system as a backup.

To guard against RAID failure, I back up the entire project to an externally connected drive (USB3 or Thunderbolt, depending on the workstation) using off-the-shelf backup software. This drive doesn’t have to be high performance because the first backup takes place overnight. Once the first full backup is complete, I use incremental backups. This means that only new data is backed up. Generally, I don’t back up things like preview renders unless they would take a long time to rebuild.

If I have a really tight deadline, once the full backup is complete I might run the incremental backup manually at lunch or during any long breaks. Since the original footage has already been backed up, the incremental backups don’t take too long. But to make sure everything is working the way it should, I configure both the RAID and the backup software to send me emails on a regular basis to confirm that systems are running properly.

While you can set up system configurations to email only when there are problems, this can break down if there’s a complete failure. For example, if you lose a power supply, it won’t send emails and you might not find out about a problem right away.

Lastly, my project file is also backed up to the cloud every 5 minutes. If the edit software doesn’t have a built-in feature to do this, I use third party tools to copy folders at the system level to cloud storage.

So what does this gain me? If I lose a drive in the array, I can still edit. If I lose two drives in the array, I can still edit. If I lose the entire array, I can edit with reduced performance and/or I can copy the project to a faster drive so I’m back up and editing without having to reingest or relink all the footage.

The above strategy isn’t the only way to keep your data safe during post. I hope, however, it gives you some ideas and makes you consider what could happen if you don’t pay attention to keeping your data safe.