-
Digital images are composed of pixels
-
Each pixel has a numeric value, often related to detected light
-
The same pixel values can be displayed in different ways
-
In scientific image processing, image appearance can be changed independently of pixel values by modifying a lookup table
Images & pixels
Introduction
The smallest units from which an image is composed are its pixels.
The word 'pixel' is derived from picture element and, as far as the computer is concerned, each pixel is just a number.
When the image data is displayed, the values of pixels are usually converted into squares of particular colors – but this is only for our benefit to allow us to get a fast impression of the image contents, i.e. the approximate values of pixels and where they are in relation to one another.
When it comes to processing and analysis, we need to get past the display and delve into the real data: the numbers.
This distinction between data (the pixel values) and display (the colored squares) is particularly important in fluorescence microscopy. The pixels given to us by the microscope are measurements of the light being emitted by a sample. From these we can make deductions, e.g. more light may indicate the presence of a particular structure or substance, and knowing the exact values allows us to make comparisons and quantitative measurements. On the other hand, the colored squares do not matter for measurements: they are just nice to look at (Figure 1).
Still, two related facts can cause us trouble:
This makes it quite possible to analyze two different images that appear identical, but to get very different results. Therefore to be sure we are observing and measuring the right things, we need to know what is happening whenever we open, adjust and save our images. It is not enough to trust our eyes.
ImageJ & Fiji
So to work with our digital images we do not just need any software that can handle images: we need scientific software that allows us to explore our data and to control exactly what happens to it.
ImageJ, developed at the National Institutes of Health by Wayne Rasband, is designed for this purpose. The 'J' stands for Java: the programming language in which it is written. It can be downloaded for free from http://imagej.net, and its source code is in the public domain, making it particularly easy to modify and distribute. Moreover, it can be readily extended by adding extra features in the form of plugins, macros or scripts.[1]
This marvellous customizability has one potential drawback: it can be hard to know where to start, which optional features are really good, and where to find them all. Fiji, which stands for Fiji Is Just ImageJ, goes some way to addressing this. It is a distribution of ImageJ that comes bundled with a range of add-ons intended primarily for life scientists. It also includes its own additional features, such as an integrated system for automatically installing updates and bug-fixes, and extra open-source libraries that enable programmers to more easily implement sophisticated algorithms.
Therefore, everything ImageJ can do can also be accomplished in Fiji (because Fiji contains the full ImageJ inside it), but the converse is not true (because Fiji contains many extra bits). Therefore in this course we will use Fiji, which can be downloaded for free from http://fiji.sc/.
The user interface
It can take some time to get used to the ImageJ/Fiji[3] user interface, which may initially seem less friendly and welcoming than that found in some commercial software. But the good news is that, once you find your way around, it becomes possible to do a lot of things that would simply not be possible in many other software applications.
Some of the components we will be working with are shown in Figure 2. At first, only the main window containing the menu, tool and status bars is visible, and the others appear as needed when images are opened and processed. Should you ever lose this main window behind a morass of different images, you can bring it back to the front by pressing the Enter key.
Tips & tricks
Here are a few non-obvious tips that can making working with ImageJ or Fiji easier, in order of importance (to me):
-
Files can be opened quickly by dragging them (e.g. from the desktop, Windows Explorer or the Mac Finder) onto the
Status bar
. Most plugins you might download can also be installed this way. -
If you know the name of a command (including plugins) but forget where it is hidden amidst the menus, type Ctrl+L (or perhaps just L) to bring up the
Command Finder
– it may help to think of it as a List – and start typing the name. You can run the command directly, or selectShow full information
to find out its containing menu. Note that some commands have similar or identical names, in which case they might only be distinguishable by their menu. -
ImageJ’s Dimensions). While inconvenient if you are used to long undo-lists in software like Microsoft Word or Adobe Photoshop, there is a good rationale behind it: supporting undo could require storing multiple copies of previous versions of the image, which might rapidly use up all the available memory when using large data sets. The solution is to take care of this manually yourself by choosing to create a copy of the image before doing any processing you may not wish to keep.
has very limited abilities – it may be available if you modify a single 2D image, but will not be if you process data with more dimensions (see -
There is a wide range of shortcut keys available. Although the menus claim you need to type Ctrl, you do not really unless the option under
tells you otherwise. You can also add more shortcuts under -
To move around large images, you can use the
scrolling tool
, or simply click and drag on the image while holding down the spacebar. A small rectangular diagram (visible on the top left of the image window in Figure 2) indicates which part of the entire image is currently being shown. -
There are more tools and options than meet the eye. Double-clicking and right-clicking on icons in the
Tool bar
can each reveal more possibilities. -
Pressing Escape may abort the operation of any currently-running command… But it requires the command’s author to have implemented this functionality. So it might not do anything.
Finding more information
Links to more information for using ImageJ, including user guides including a detailed manual, are available at http://imagej.net/Introduction.
Data & its display
Comparing images
Now we return to the data/display dichotomy. In the top row of Figure 3, you can see four images as they might be shown in ImageJ. The first and second pairs both look identical to one another. However, it is only actually (A) and (C) that are identical in terms of content. Since these contain the original pixel values given by the microscope they could be analyzed, but analyzing either (B) or (D) instead may well lead to untrustworthy results.
Reliably assessing the similarities and differences between images in Figure 3 would therefore be impossible just based on their appearance in the top row, but it becomes much easier if we consider the corresponding image histograms below. These histograms (created with ) depict the total number of pixels with each different value within the image as a series of vertical bars, displayed above some extra statistics – such as the maximum, minimum and mean of all the pixels in that image. Looking at the histograms and the statistics below make it clear that only (A) and (C) could possibly contain the same values.
Mapping colors to pixels
The reason for the different appearances of images in Figure 3 is that the first three do not all use the same lookup tables (LUTs; sometimes alternatively called color maps), while in (D) the image has been flattened. Flattening will become relevant in Channels & colors, but for now we will concentrate on LUTs.
A LUT is essentially a table in which rows give possible pixel values alongside the colors that should be used to display them. For each pixel in the image, ImageJ finds out the color of square to draw on screen by 'looking up' its value in the LUT. This means that when we want to modify how an image appears, we can simply change its LUT – keeping all our pixel values safely unchanged.
Why use different LUTs?
The ability to change LUTs has several advantages. A simple one is that
we can use LUTs to make the colors in our image match with the
wavelengths of the light we have detected, such as by showing a DAPI
staining in blue or GFP in green. But often this is not really optimal,
and you may prefer to show an image using some multicolored LUT (e.g.
Fire
in ImageJ) that does not otherwise have any physical relevance.
This is because the eye is relatively poor at distinguishing different
shades of the same color, and presenting the identical information
using many different colors can make differences between pixel values
more apparent.
Modifying the LUT can help make information visible
But swapping one set of LUT colors for another is not the only way to change the appearance. We can also keep the same colors, but change which pixel values each color is used for.
For example, suppose we have chosen a gray LUT. Most monitors can (theoretically) show us 256 different shades of gray, so we can give a different shade to pixels with values from 0–255, where 0 corresponds to black and 255 corresponds to white. But suppose our image only contains interesting values in the range 5–50. Then we will only be using 46 rather similar shades of gray to display it, and not using either our monitor or our eyesight to their full capacities. It would easier to see what is happening if we made every pixel with a value ≤ 5 black and ≥ 50 white, and then distributed all our available shades of gray to the values in between. This would make full use of the colors we have in our LUT, and give us an image with improved contrast. Of course, we can also apply the same principle using any other LUT, replacing black and white with the first and last colors in the LUT respectively.
Adjusting the display range
This type of LUT adjustment is done in ImageJ using the
Shift+C; see Figure 4). The first two
scrollbars that appear are called Minimum
and Maximum
, and these
define the thresholds below and above which pixels are given the first
or last LUT color respectively. Modifying either of these sliders
automatically changes the Brightness
and Contrast
sliders.
Although the terms brightness and contrast are probably more familiar, it is
usually easier to work with Minimum
and Maximum
. If you know, for
example, you do not care to see anything in the darkest part of the
image, you can increase the value of Minimum
to clip it out of the
picture (only for display!), and devote more different colors for the
part of the image that is really interesting.
Properties & pixel size
Hopefully by now you are appropriately paranoid about accidentally changing pixel values and therefore compromising your image’s integrity, so that if in doubt you will always calculate histograms or other measurements before and after trying out something new to check whether the pixels have been changed.
This chapter ends with the other important characteristic of pixels for analysis: their size, and therefore how measuring or counting them might be related back to identifying the sizes and positions of things in real life. Sizes also need to be correct for much analysis to be meaningful.
Pixel sizes are found in ImageJ under Pixel width
and Pixel height
, defined in terms
of Unit of Length
. A useful way to think of these is as proportions of
the size of the total field of view contained within the image (see
Figure 6). For example, suppose we are imaging an area with
a width of 100µm, and we have 200 pixels in the horizontal direction of
our image. Then we can treat the 'width' of a pixel as
100/200 = 0.5µm. The pixel height can be defined and used
similarly, and is typically (but not necessarily) the same as the width.
Pixel sizes and measurements
Knowing the pixel size makes it possible to calibrate size measurements. For example, if we measure some structure horizontally in the image and find that it is 10 pixels in length, with a pixel size of 0.5µm, we can deduce that its actual length in reality is (roughly!) 10 × 0.5µm = 5µm.
This calibration is often done automatically when things are measured in
ImageJ (see Measurements & regions of interest), and so the sizes must be correct for the
results to be reasonable. All being well, appropriate pixel sizes will be written into an
image file during acquisition and subsequently read – but this does not
always work out (see Files & file formats), and so Properties…
should
always be checked. If ImageJ could not find sensible values in the image
file, by default it will say each pixel has a width and height of 1.0
pixel… not very informative, but at least not wrong. You can then
manually enter more appropriate values if you know them.
Pixel sizes and detail
In general, if the pixel size in a fluorescence image is large then we cannot see very fine detail (see Figure 6). However, the subject becomes complicated by the diffraction of light whenever we are considering scales of hundreds of nanometers, so that acquiring images with smaller pixel sizes does not necessarily bring us extra information – and might actually become a hindrance.
This will be explored in more detail in later chapters (Blur & the PSF and Noise).