One of the first things any post professional needs to get a handle on is the concept of a “codec.” Short for “code/decode,” a codec is the method that is used to write your video and usually audio into a file.
The analogy I find most helpful when talking about codecs is that of a language. You have an idea, and you can write that idea down in Hindu or Swahili or Cantonese. With video, you have an image, and you need to record it; you can also record it in any of a variety of codecs, depending on what you are hoping to do with that image and who you need to communicate with.
Codecs Vs. Wrappers
One of the first questions that come up is often, “oh, is a codec the difference between .mp4 and .mov and .mxf?” Actually, those aren’t codecs. Those are wrappers. If we want to continue our language analogy, you can think of those like the format of a written document. You can have a German language magazine or a Ukrainian magazine, but both are magazines. Or you can have a Ukrainian magazine and a Ukrainian book; both are Ukrainian but in different wrappers.
Codecs and Wrappers work the same way. You can have an H.264 (one of the common video release codecs) that you write into a .mp4 file, a .mov file or a .mxf file. It’s still written in the same language (in this case, H.264), but you put it in a different format depending on your use. In this case, MXF (media exchange format) is the most robust professional format, while MOV is an older QuickTime movie format, and MP4 is a popular consumer format. Something like YouTube accepts all three, but your phone is unlikely to be able to play an MXF file, while it’ll almost definitely play an MP4.
What Are You Using the Codec For?
We broadly divide up the uses of codecs into three categories: capture formats, intermediate (or editing) formats, and release formats. All of these tasks for a codec have different jobs, so we use different technology to achieve that job.
The job of a capture format is to capture as much of the on-set information as possible. You want the brightest brights and the darkest darks and the full range of colors in front of you captured as best you can. The job of an intermediate codec is to be easy to work with; you want your editing process to feel easy, with applications opening quickly, timelines whizzing by, and quick exports for client approval. For final release, the goal is different; the file will generally only be played linearly (forward from beginning to end), without skimming or image manipulation, so all you care about is making the best-looking image in the smallest file possible.
These are all different tasks, so we often use different codecs, or different flavors of codecs, to achieve them.
Download our free Guide to Codecs now.
At the capture stage, we frequently use either RAW formats, which aren’t even really codecs, or larger codecs with high bitrates to capture all the possible scene information. The bitrate of a codec refers to how many Mbs are allocated per second to create the image, with more Mbs offering higher quality images. There are capture formats that are close to 2000 Mb/s for 4k video capture, which can quickly fill up a hard drive and can be too onerous for even powerful computer systems to easily deal with.
This is a great time to remember the difference between Mb and MB: Mb is “megabits,” while MB is “megabytes.” Megabits are usually used for datarates of something happening in real-time (like uploading/downloading data or playing files), while Megabytes (and the larger gigabytes and terabytes) are used for data storage. Bits and Bytes aren’t equal: a byte is 8 bits. That means a 2000 Mb/s file capture format works out to around 250 MB/s.
To make matters worse, since we do multiple different things with files, you’ll often see their data rates written in different ways. For instance, intermediate codecs will often talk about quality in terms of MB/s, while a release codec like H.264 will often be discussed in terms of Mb/s. If you think about it, it makes sense since the intermediate codec will be taking up storage space on your hard drive, while that H.264 will be streaming over the internet, but it can take a second to get used to.
RAW
RAW isn’t a codec since it isn’t even video yet; it hasn’t gone through the “codec” step of “code/decode.” RAW formats take the RAW sensor data, unprocessed, and compress it. It still needs to go through all of the processing to even make it a usable video file.
Because of the unprocessed nature of RAW files and the massive file sizes of capture video codecs, as soon as we get into post, the first step is often doing a demosaic (sometimes called a debayer). This process takes a video file from one format (RAW) and translates it (there is our language analogy again) into a codec for use in editing. You don’t have to worry too much about this transcode process since we generally reconnect back to those original video files for our final color grade, so image quality loss in this transcode isn’t a major deal.
The Intermediate Codec
We then get to edit our smaller files wrapped up in a more manageable codec, usually with a data rate under 200 Mb/s (25MB/s), sometimes even down around 100 Mb/s or less (12MB/s). This makes life much, much easier while you are editing since the computer has to bring much smaller files “live” into memory while you are working, and your edit process can feel smooth and painless instead of slow and tortuous.
Another major feature of editing codecs is that they typically focus on intra-frame encoding instead of inter-frame encoding. This means that the compression they apply (and all the video we ever work with has some compression) is applied one frame at a time instead of looking at multiple frames and encoding them together. Intra means “inside,” and you can think of this as each frame is only compressed “inside” itself. With inter-frame encoding, the encoding is spread between frames.
Inter-frame encoding, sometimes called Long GOP encoding (for Group of Pictures), is wonderful for a final release format, like a file you put up on YouTube. But it’s very frustrating in post-production since you are going to be moving forwards and backward and scrubbing over clips quite a bit. To do that with a Long GOP codec, the computer has to constantly be looking at all the frames around an individual frame in order to “draw” that frame accurately.
With intra-frame encoding, the computer has to do a lot less work. Every time you land on a frame in your timeline, the computer only has to look at one individual set of data, one frame, to “draw” the frame on your screen. It’s much easier on the processor and makes for a faster edit process. Once you add a cut, with intra-frame encoding, you are A-OK. With inter-frame encoding, the computer still has to keep referencing those other frames outside the timeline to draw that frame.
The big codecs that are used in editing (Apple ProRes and Avid DNxHR) use intra-frame encoding and are available at a variety of bitrates, so you can choose a reliable codec for easy editing at a bitrate that fits the power of your editing workstations and the available space in your storage.
Online & Release
After you edit, you reconnect to your capture format, whether it was RAW or a larger capture codec, and do your online color grading session. Typically this requires a more powerful machine since you are working with the larger capture formats that were used on set. From there, you’ll transcode the movie into your release codecs.
This typically involves a larger file (something like a ProRes 4444 XQ file or similar) that will become your master file and will be delivered to your client and a smaller file for web delivery. That web delivery file is where a Long GOP codec like H.264 becomes a great fit. Since it’ll be watched linearly, the image benefits of Long GOP compression will outweigh the drawbacks.
Check out our handy guide on the most common codecs for a better understanding of how they work and how they fit into your post workflow.
For tips on post-production, check out MediaSilo’s guide to Post Production Workflows.
MediaSilo allows for easy management of your media files, seamless collaboration for critical feedback and out of the box synchronization with your timeline for efficient changes. See how MediaSilo is powering modern post production workflows with a 14-day free trial.
