The role of RGB#
Using RGB images for display#
So what’s the point of having RGB images, if they are so risky?
One of the biggest reasons to use RGB images in science is for presentation. While specialist image analysis software applications, such as ImageJ, are typically designed to handle a range of exotic image types and bit-depths, the same is not true for non-scientific software.
If you want an image to dispay exactly the same in ImageJ as in a PowerPoint® presentation or a figure in a publication, for example, we’ll probably want to convert it to RGB. If we don’t, the image might display very strangely on other software – or even not open at all.
Over the years, I have encountered a remarkable number of cases where a researcher has saved their fluorescence microscopy images only in an RGB format.
Their justification was that they tried saving the images in different way at the microscope, but ‘it didn’t work – the images were all black’.
The explanation is almost invariably that their images were really 16-bit or 32-bit, but they tried to open them in software that doesn’t handle 16-bit images very well (e.g. they just double-clicked the file to open it in the default image viewer). All they saw as a black, seemingly-empty image.
Whenever they tried exporting from the microscope’s acquisition software in different ways, they found an option that gave a viewable image – and stuck with that.
The problem with this is that it usually meant that they didn’t save their raw data at all! They only saved an RGB copy, with all the rescaling and LUT magic applied, which is wholly unsuitable for analysis.
The solution is to view images in ImageJ, or similar scientific software. This usually reveals that the image is not ‘all black’ after all. Rather, one only needs to adjust the brightness and contrast (using the LUT) to see the raw data in all its glory.
When RGB is all you’ve got#
All the previous comments about ‘don’t convert to RGB before analysis’ as based on the assumption that your raw data isn’t already RGB. This is usually the case with microscopy and medical imaging whenever accurate quantification is important.
Nevertheless, it’s not always the case.
A common example is brightfield imaging for histology or pathology. Here, the camera is often RGB and an RGB image is really as close to the raw data as we are likely to get.
Example RGB histology image, from https://openslide.org
Crucially, the analysis of brightfield images in histology usually aims to replicate (and sometimes improve upon) the visual assessment that a pathologist might make looking down a microscope. It’s often based on detecting, classifying and counting cells, measuring stained areas, or recognizing the presence of particular patterns – but not accurately quantifying staining intensity.
Multichannel images#
So far, we have focussed on 2D images with a single channel – that is, a single value for every pixel at every x,y coordinate in the image.
Such images can be converted to 8-bit RGB using a LUT. If we do this, then we get an image with three channels, where each channel is displayed using red, green and blue LUTs – with the colors blended together for display. But we shouldn’t do that conversion prior to analysis in case we lose our raw data.
Now, we turn to consider multichannel images that aren’t RGB images. Rather, the raw data itself has multiple channels.
In fluorescence microscopy, it’s common to acquire multichannel images in which pixel values for each channel are determined from light that has been filtered according to its wavelength. We might choose to visualize these channels as red, green and blue, but we don’t have to.
In principle, any LUT might be applied to each channel, but it makes sense to choose LUTs that somehow relate to the wavelength (i.e. color) of light detected for the corresponding channels. Channels can then be overlaid on top of one another, and their colors further merged for display (e.g. high values in both green and red channels are shown as yellow).
The important feature of these images is that the actual channel information is always retained, and so the original pixel values remain available. This means we can still extract channels or adjust their LUTs as needed.
Multichannel image from {numref}`fig-colors_composite_rgb` using different LUTs. Again, no information is lost: we can access the original pixel values, and update the LUTs if needed.
Just like with a single channel image, we can create an RGB image that allows us to visualize our multichannel image – using the LUTs to figure out which RGB values are needed to represent the color of each pixel.
Then, again if just like with the single channel image, this is problematic if we don’t keep the raw data – because we can never recover the original values from the RGB representation.
We can create an RGB image, but then we have three channels locked to red, green and blue – which have converted the original channels to 8-bit, and mixed up information due to the LUT colors used. We can no longer recover the original pixel values after converting to RGB.
Summary of color images#
The main message here can be distilled into two rules:
Always use the original image for analysis
If the raw data isn’t RGB, then don’t convert it before analysis!
Create an RGB copy of your image for display
Keep the RGB copy separate, so you always retain it and the raw image
RGB images aren’t bad – we almost always need them for display, and for some imaging applications (e.g. brightfield histology) they are the best raw data we can get. But we need to be cautious if our raw data isn’t RGB, and avoid converting to RGB too early.
In the end, it’s normal to keep at least two versions of each dataset: one in the original (possibly multichannel) format, and one as RGB for display. This RGB image is normally created as the final step, after applying any processing or LUT adjustments to the original data.