Today’s Question: In a recent Ask Tim Grey eNewsletter you made reference to the “demosaicing” of a raw capture. What exactly is “demosaicing”?
Tim’s Quick Answer: The term “demosaicing” refers to the process of calculating the “missing” color values for each pixel in a raw capture, due to the fact that most raw captures only record a single color value for each pixel in an image.
More Detail: When you capture in raw mode on most digital cameras, the sensor is only recording a single color value for each pixel. For example, many digital cameras use a sensor that employ the “Bayer pattern” for the color values. In this configuration, for each four pixel values there will be two pixels that record only the green color value, along with one pixel recording the red value and one pixel recording the blue value.
The lack of full RGB color information for each pixel is a product of the image sensor in the camera, not the raw capture format. For photos captured with an actual image format, such as JPEG, the data gathered by the image sensor still only represents one out of the three required color values for each pixel. The difference is that with a JPEG capture, for example, the camera calculates the “missing” values for each pixel. With a raw capture, the “missing” color values are not calculated at the time of capture.
Therefore, with a raw capture, part of the process of rendering the raw capture into an actual image file is calculating the “missing” color values for each pixel. That process is referred to as “demosaicing”.
It may be hard to believe that software could “magically” calculate two-thirds of the “missing” data for a given pixel. However, when you consider the context for these calculations, it is I think much easier to understand. Let’s consider the green channel, for which half the pixels have information that was gathered by the image sensor and half the pixels have no information.
If you imagine a black and white photo where half of the pixels are missing, I think it isn’t all that difficult to imagine that software could figure out appropriate values for the “missing” pixels somewhat easily. Even for the red and blue channels where three-quarters of the pixels are missing, you can probably appreciate that software could calculate the values for the missing pixels based on the context of those pixels that do exist, again without too much difficulty.
This is the actual process behind the scenes with demosaicing. And considering we’ve all been happy with the photographic results we’re able to achieve with raw captures, obviously the process of demosaicing actually works in the real world.