Dealing With HEVC/H.265 Footage

Dealing With HEVC/H.265 Footage

The Panasonic Lumix DC-S1H is one of the newest generation mirrorless hybrids that utilizes the H.265 codec for internal recording.

Does your current primary camera support H.265 recording? Other than a preset in your camera’s codec/recording options, what do you know about the H.265 codec and how to most effectively use it in your production and post workflow? I thought it would be timely to explore H.265 usage in digital cinema and pro video since we’re just receiving word that there are already new options on the horizon that will update and compete with H.265. While this is by no means a complete list, popular cameras like the entire ZCam line, the Panasonic Lumix S1H and AU-EVA 1, Go Pro Hero 7 and 8 Black, Apple iPhone, Canon XF-705, as well as many others, currently or will support H.265 in the future.  

Dealing With HEVC/H.265 Footage
H.265 was originally known as High-Efficiency Video Coding.

High-Efficiency Video Coding (HEVC), also known as H.265 and MPEG-H Part 2, is a video compression standard designed as a successor to the widely used Advanced Video Coding (AVC, H.264 or MPEG-4 Part 10). I was amazed to discover that H.265 was started in 2013, but it’s only about the past two years that it seems to have reached critical mass with a lot of new cameras, from Go Pros to high-end digital cinema cameras, including it in their codec options. In comparison to AVC, HEVC offers from 25 percent to 50 percent better data compression at the same level of video quality or substantially improved video quality at the same bit rate. It supports resolutions up to 8192×4320, including 8K UHD, and unlike the primarily 8-bit AVC, HEVC’s higher fidelity Main10 profile has been incorporated into nearly all-supporting hardware.  

Dealing With HEVC/H.265 Footage
The Fujifilm X-T3 utilizes the H.265 codec for internal recording at resolutions up to 4K DCI at 60p.

My H.265-Capable Cameras

Out of my own two main digital cinema cameras, only one of them supports H.265 recording, the Fujifilm X-T3, but I also notice that my GoPro Hero 7 Black supports H.265, although GoPro refers to it only as HEVC. Like many other terms in our business, there are often two or even three commonly used names, but just to be clear, as of today, HEVC and H.265 are the same thing and are often used interchangeably. From here on out, I’ll refer to HEVC primarily as H.265, but just beware that as your read and research about how this codec works and how it related to your gear, you may see both of the names HEVC and H.265 in common and interchangeable use.

Dealing With HEVC/H.265 Footage
8-Bit capture or processing typically results in banding. What is image banding? This frame capture shows banding if you look at the gradations from the sun, left and right to the sky. You will see similar results on interiors lit with any bright sources on flat surfaces where the light falls off in intensity.

Getting Rid Of Banding?

For me, the main factor in H.265 is that its efficiency allows 10-bit video recording in a tiny mirrorless camera like the Fujifilm X-T3, although not all H.265-capable cameras support 10-bit recording (looking at you, Apple!), but most do. That alone makes it an interesting codec as the camera’s other recording choice, H.264, is limited to 8-bit recording. As you’re probably aware, with 8-bit video the maximum number of colors that can be displayed at any one time is only 256. The result is that 8-bit video can look very good but will be limited in color reproduction, resulting in color banding in gradients. If you shoot skies, you’ll often see banding, as well as in numerous interior situations. When performing color-intensive techniques like green screen, keying and compositing, it can be more difficult to pull a clean and precise key with 8-bit video.

Shooting 10-bit video, the maximum number of colors that can be displayed jumps from 256 to 1,024 levels per channel. The end result is that 10-bit video reproduces gradients and skies with little to no perceptible banding and for keying/green/blue screen, compositing and color grading, 10-bit gives you much more color resolution to work with, meaning that you can refine composites and push the signal around in color grading considerably more.

Dealing With HEVC/H.265 Footage
H.265 utilizes a different and more efficient blocking scheme in its compression than its predecessor H.264 did, resulting in greater compression efficiency and more accurate image reproduction with fewer macro blocking artifacts.

How does H.265 increase efficiency over say, its predecessor, H.264? H.265 achieves greater levels of compression, including better variable-block-size segmentation, improved deblocking and motion compensation filters, sample adaptive offset filtering and better motion vector prediction and precision. H.264 utilized same size blocks while H.265 advances the concept.

Now that we have established what H.265 is, let’s talk about how you actually can efficiently utilize it in your own work. Now that H.265 footage is becoming more commonplace in professional production workflows, I have a few observations about how it’s used and what some common issues are around H.265.

H.265 Is NOT An Editing Codec. It’s a Distribution Codec.

In our business, time is money, right? In production, shooting H.265 isn’t any more difficult or time consuming than any other codec you’ve been using. Where I see a lot of users having issue with H.265 is when it comes to handing it off to clients or for their own postproduction workflows. Let’s state it now, H.265 is unequivocally not designed for editing. Why? Without getting into the technological intricacies of exactly how each codec, Long GOP (groups of pictures) and Intraframe (individual video frame) encoding works, with H.265 being a compressed codec, your computer has to do a lot more heavy lifting to decompress that information to edit.

When a video has been shot with a long GOP codec, there’s a lot of interpolated information in many of the frames. Transcoding the footage into ProRes or another edit-ready codec converts those groups of pictures to an All-Intraframe codec so that your computer no longer has to devote CPU cycles to making sense of the interpolated data. It also allows you to upgrade the color from 4:2:0 to 4:2:2/4:4:4, which won’t make the colors better in your shot but gives you a bit less color/signal degradation when color grading.

Because You Can, Should You?

Sure, if you can invest in the top-of-the-line CPU with a lot of RAM and the fastest RAID for editing, it’s possible to edit H.265 in real time without dropping frames. But even if you have a system capable of real-time processing the H.265 footage, what happens when you need to edit a multicam project? Then you’re asking your computer to decade multiple streams of long GOP H.265 footage in real time.

What if you’re editing a huge project with a long running time and lots of H.265 footage? What generally happens is if your system is capable of editing an H.265 media stream with real time performance, it’s probably at or near the edge of your system’s capability. It’s always a good idea with client projects, deadlines and money on the line to have extra capability and additional overhead. For this reason, most experts agree that transcoding H.265 to a more edit-friendly codec is the safest way to assure that your system will have enough additional overhead to handle multiple streams and layers without choking by not being able to play back in real time.

Efficient H.265 Production/Post Workflows

The two most common solutions to the challenges presented by using a distribution codec like H.265 for editing are:

  1. Bypass H.265 by using an external recorder to record straight to an edit ready codec like Apple ProRes.
  2. Transcode H.265 to an edit-ready codec like Apple ProRes.
Dealing With HEVC/H.265 Footage
The Blackmagic Design Video Assist is an external 5” recorder (a 7” version is also available) designed to record incoming inputs in ProRes or DNxHD codecs.

External Recorders

Let’s break down each of these approaches. For number one, using an external recorder, many users report good results utilizing an external recorder like the Atomos Ninja V or the Blackmagic Design Video Assist. It’s nice to be able to shoot, return to your editing system with ready-to-edit ProRes or even compressed RAW files to edit with. All isn’t perfect, though, with this approach, there are several drawbacks and limitations in using an external recorder:

  • Additional expense for recorder, batteries to power it, recording media (SSDs or SDHC cards) and some kind of mounting hardware.
  • Additional size, bulk and weight added to your camera. In the case of a small camera like most mirrorless hybrids, you need to add a cage and mounting arm to accommodate an external recorder.
  • Additional clutter with cabling. You’ll need to run an HDMI or SDI cable from your camera to the recorder, adding a snagging, wear-and-tear hazard.
  • Adds fan noise to your recording. All of the external recorders seem to have internal fans that, while quiet in most shooting circumstances, can be too noisy in shooting quiet interviews in small rooms, etc.
  • Extra Logistics Management. With all external recorders, you add additional power and media management requirements, necessitating extra batteries or draining your larger V-Mount or Gold Mount batteries more rapidly than normal. You also have to keep an eye on and manage the media requirements for the recorder. 4K ProRes generally fills up large media rather quickly if you’re shooting long-form content like events and documentary shoots.

In general, external recorders solve one problem or challenge (getting around the H.265 computing requirements in post) by adding additional problems or challenges to your production workflow. It’s up to you if the tradeoffs are worth the advantages.

Dealing With HEVC/H.265 Footage
The FCP X import interface includes an option to ‘optimize’ (transcode) incoming media to ProRes.

Transcode H.265

For number two, transcoding H.265 to an edit-ready codec, you realize the same advantage of using an external recorder, you begin your post process with an editing computer-friendly codec like ProRes. Transcoding your footage overcomes almost all of the disadvantages of using an external recorder: no additional expense, weight, bulk, size and power requirements, and no additional HDMI or SDI cable going from your camera to a recorder. All isn’t perfect with shooting H.265 and transcoding after the shoot:

  • I edit using Apple’s FCP X. What’s great about using H.265 with FCP X is you can set FCP X to automatically transcode incoming video to ProRes upon ingest. Doing this really cuts out the middleman; you bring in H.265 and can immediately begin editing. FCP X will be quietly transcoding the H.265 in the background whenever you aren’t moving the mouse. You can begin going through your footage, tagging and adding metadata through building keyword collections. In my opinion, overall, FCP X is by far the fastest and most efficient editing tool on the market. Notice how I’m not saying it’s the “best” editing tool on the market? I understand that the choice of editing tool is a personal one and people become very attached to and defensive about their choice of editing software.
  • I don’t—I have edited with all of the “Big Four” editing program that most pros use (Adobe Premiere CC, Blackmagic Design Resolve, AVID Media Composer and FCP X) and I feel that anything can be easily edited with any of the four; it really comes down to your personal preferences. If you’re working with a lot of H.265 footage on an ongoing basis, you owe it to yourself to at least try FCP X. FCP X and Resolve are the fastest renderers on the market and FCP X, is hands down, the fastest editing workflow. Premiere and AVID have their advantages too, don’t get me wrong. I use and have used all four and each has their distinct advantages and disadvantages, but from an H.265 viewpoint, FCP X is the most elegant and straightforward solution.
  • If you don’t want to or can’t switch to FCP X to facilitate editing H.265 (and there are lots of legitimate reasons why one wouldn’t want to switch to FCP X—PC Users, people who are required to use Adobe CC, AVID users who are in an all-AVID multiuser environment, etc.). Your plan of action should be to externally transcode to ProRes or another edit-friendly codec.
Dealing With HEVC/H.265 Footage
Apple’s Compressor 4.0 is an excellent option to transcode your media to a more edit-friendly codec, regardless of your editing software as long as you’re on the Mac platform.

To that end, there are excellent utilities that aren’t very expensive that will efficiently transcode your footage. Even if you don’t want to use FCP X and you’re on the Mac platform, Apple Compressor is an excellent, easy to utilize, fully-featured transcoding and compression utility that’s optimized for the latest Apple OS and it only costs $49 from the App Store.

Dealing With HEVC/H.265 Footage
Brorsoft Video Converter is available on PC and Mac platforms if you’re looking for a flexible transcoding utility for your H.265 footage.

Brorsoft makes a utility called Video Converter for both PC and Mac platforms that can easily and efficiently transcode your H.265 to the best format for your editing system. There are numerous other utilities made by numerous other developers and most of them cost less than $60 and offer a free trial to make sure that the utility can do what you need it to. Most of these utilities also other functions that may be helpful to your post workflow for other file conversions.

Your Decision How To Deal with H.265 Footage

H.265 is a reality for professional media production right now. How you choose to deal with it in your workflow is a choice you’ll have to make using whatever factors make the most sense for your budget and workflow. For some, investing in the top-of-the-line hardware available is worth it to be able to edit H.265, at least at some level, in real time, is important to their business plan. For other workflows and uses, an external recorder makes the most sense. For those of us who prefer to deal with H.265 in postproduction, a software solution and transcoding makes the most sense.

Dealing With HEVC/H.265 Footage
The Moving Picture Experts Group (MPEG) is a working group of ISO/IEC in charge of the development of international standards for compression, decompression, processing and coded representation of moving pictures, audio and their combination.

An Addendum To Think About

Keep in mind that coming in 2020, we’ll see the introduction by the Motion Picture Experts Group (MPEG) of VVC, Versatile Video Coding that will also be known as H.266. From the intro documentation, it becomes clear that the design parameters of this new codec are definitely made with more advanced distribution properties. The MPEG requirements document defines this as a bit rate reduction of between 30 percent and 50 percent with the same perceptual quality as HEVC (H.265) Main Profile.

In terms of complexity, the same document states that “Encoding complexity of approximately 10 times or more than that of HEVC is acceptable for many applications.” Reading between the lines, it seems as if this new H.266 codec hasn’t been thought through any more effectively for shooting/acquisition than it was for H.265 in 2013. Meaning that for you, as a digital cinema/video user, you may need to continue these strategies I’ve outlined in this article much longer than just the useful lifespan of H.265 since it appears that the new version, H.266 is also mainly a distribution codec.

New codecs will always be coming into our workflows, it helps to have strategies in place for how to deal with them effectively.