CCD data reduction: Dithering

Assumptions

Master biases, darks and flats, superflats and fringing models are created from a finite amount of noisy images, thus they are noisy, too. This noise is called calibration noise, and it is copied into the images to which these calibration exposures are applied. If the telescope is not offset (dithered) between the target exposures, then the calibration noise does not average out. This problem increases the better (darker) the sky becomes.

Let's consider the pixel at position (x|y) in an exposure, and define the following variables and parameters:

		the noise in an individual DARK exposure (for cooled cameras comparable to the readout noise)
		the noise in an dark-subtracted and flat-fielded image, expressed in units of DARK noise
		the noise in the normalised flat field, expressed in units of DARK noise
		the number of DARK exposures
		the number of FLAT exposures
		the number of target exposures

S/N reduction due to calibration noise

The uncertainty of a function f depends on the uncertainties of its variables.
If the variables are uncorrelated, the law of error propagation reads

A flat-fielded and dark-subtracted image is written as

The uncertainty of a stack of n dithered images is then written as

where <F>, <D> and <S> denote the average flat, dark and sky levels.
Likewise, for n undithered (or stared) images we have

Note the additional two factors of n in the numerators of the last two additive terms. They arise because the master darks and flats get simply stacked n times on top of themselves, so that their noise does not average out (i.e., increases with sqrt(n)), but grows linearly with n.

Expressing both the sky noise and the flat noise in units of the dark noise, and comparing the S/N of the stack of dithered exposures to the stack of undithered exposures, we then find

If the flat field exposures have a sufficiently high illumination level of several 10000 ADU, then approximately k_F = 0.001, i.e. the noise in the normalised flat is a lot smaller than the noise in the dark. The average flat level, <F>, is usually also larger than the average sky level <S>. Since both terms go in quadratically in the ratio above, the last additive terms in the numerator and denominator become absolutely negligible. Only for very unusual settings this should not be valid.
Simplified, we have

The only difference is the additional factor of n in the numerator. Thus, if k_D is much larger than 1, then one is sky limited. If k_D is comparable to 1, then the dark noise becomes important. Its effect can be quite devastating as is shown in the examples below.

Examples

The following numbers are based on my 12.5" f4.5 Newtonian, running under f/5.1 with the Paracorr, and using a ST-10XME. The sky conditions are characteristic for a very good night at the Roque de los muchachos obseratory in La Palma, Spain, at 2400m altitude. This is about as dark as it can get on Earth. For light polluted areas, the sky levels have to be rescaled (probably by any factor between 3 and 50, the noise with the sqrt of these factors).

Exposure type		Exp. time [s]	Sky level	Full noise level	Sky noise only (no dark noise)
Red		1200	520	21	20
Green		1200	560	20	19
Blue		1200	280	14	12
Luminance		1200	1320	42	41
Hα (6nm)		1800	100	12	10
OIII (13nm)		1800	80	11	8
Bias		0	--	7	--
Dark (-20°C)		1200	--	7	--
Flat (30 kADU, normalised)		1	--	0.0048	--

The RGB filters are standard SBIG filters, the Luminance filter is from Astronomik and transmits between 390nm and 670nm. The two narrow-band filters are from Astronomik as well.

• Sky limited
  Observing from a much brighter sky, one would e.g. have 10000 ADUs in a 20min Luminance exposure, and a full noise level of about 110 ADU. Hence k_D = 110 / 7, which is about 16. Taking m_D = 4 dark exposures and a set of n = 5 (30) dithered and undithered exposures, the ratio in S/N between the dithered and the undithered set would be 1.002 for n = 5 and 1.014 for n = 30. In other words, no S/N is lost by not dithering.
• Dark sky, Luminance
  Under a very dark sky the values given in the table above apply. For the L filter and a 20 minute exposure k_D = 6, and the ratio between the two S/N is 1.014 (1.095) for n = 5 (30). In other words, by not dithering the 30 exposures one would lose (at least) 1.095² = 20% of the exposure time spent.
  If the exposure times were only 10 minutes, then k_D = 4.3, and for n = 30 (10) exposures the S/N dropped by 18% (6%) if the exposures were not dithered. This corresponds to a loss of exposure time of some 40% (12%). Using 10 instead of 4 darks would lower the loss to about 16% (2%).
  It is important to realise at this point that the fraction of wasted exposure time does not stay constant, but increases with the number of exposures made.
• Dark sky, RGB
  For the RGB filters non-dithering becomes very punishing. A blue exposure with 600s exposure time has k_D = 1.6. Having 10 such undithered exposures and 4 darks only, is equivalent to throwing away some stunning 80% of the exposure time spent. Using 10 darks means losing 34%.
• Dark sky, narrow band
  Not dithering with narrow-band exposures is absolutely prohibitive. Even with 1800s long exposures, k_D = 1.14 (1.43) for OIII (Hα), which rapidly translates to losses of factors 2 to 3.
• Worst case
  For short and filtered exposures under a dark sky, k_D approaches 1. In that case, and if many exposures and only one dark is taken, the stacked undithered image is only twice as deep as an individual exposure, irrespectively of how many images were taken.

Summary

Calculation of loss of exposure time
To evaluate how much S/N is lost by not dithering, the following formula applies:

Here, k_D is the ratio between the noise in a flat-fielded and dark-subtracted image (i.e., sky noise and calibration noise included) and the noise in an individual dark frame. m_D is the number of dark frames taken, and n is the number of exposures. To calculate how much exposure time is wasted, take the square of the above expression.

How to determine k_D
To measure k_D a programme that can determine the noise level (the rms value or standard deviation) of an image is needed. For a target exposure, this must be an area that is free of any stars or other objects, so that only sky is seen. The exposure should be dark-subtracted and flat-fielded.
To determine the noise level of an individual dark exposure, you must subtract two dark frames from each other, and determine the rms of the difference image. The rms needs to be divided by sqrt(2) to obtain the noise level of one individual dark frame. The reason for this is that (at least for the CCDs used in SBIG cameras) the dark current varies strongly from pixel to pixel, which gives the dark image a salty (or 'noisy') appearance. This is not the dark noise but only the dark current, which is very stable (hence dark subtraction works). The true noise level can only be determined from the difference image. Before subtracting the images they have to be converted to 32-bit float format, otherwise the negative values get clipped.
k_D is then the ratio of the rms measured for the sky and of the dark exposure.

Why is exposure time lost?
If exposures are not dithered, then the noise contained in the dark frames is copied into the exposures and does not grow with sqrt(n), but linearly with n. This is because it is stacked on top of itself, whereas dithered exposures would lead to averaging out.

When is exposure time lost?
The better the observing conditions, i.e. the darker the sky, the worse the effect of not dithering becomes. Using color or even narrow-band filters makes dithering even more important. It is very important to realise that the more exposures one takes, the larger the fraction of wasted observing time becomes. See the examples above.

How much exposure time is lost?
If the exposures are not limited by the sky noise anymore, the effect of calibration noise becomes very big. The effective exposure time spent can be reduced by factors of 1.5 to 2 for broad-band filters, and much more than this for narrow-band filters.

Which pixels lose exposure time?
The expressions given above are valid for one particular pixel, only. Is the pixel not sky limited, then it is affected by the negative effects outlined above. However, if a pixel of a non-sky limited exposure sits on top of a brighter object, then the photon noise of this object dominates over the calibration noise, and this pixel becomes sky limited (even though most other pixels are not).
This can lead to the odd effect that e.g. the brighter Hα-regions in a narrow-band image of a galaxy have good S/N even in undithered exposures, whereas the calibration noise dominates all the fainter regions which then acquire disproportionally little (or no) S/N. The resulting image depth hence depends strongly on the brightness distribution across the image, which a sharp decline once the background level gets so faint that the calibration noise takes over. Such images are very misleading, as from the well-defined brighter objects it is deferred that the fainter stuff is really faint as it does not make it very far over the noise level. In dithered exposures the fainter parts would show up with the correct and much higher S/N.

How many darks for a set of dithered exposures?
If a series of dithered exposures is not sky limited, then the number m_D of darks can have a somewhat significant impact on the S/N of the stacked image:

Polar alignment errors lead to a slow rotation of the field of view around the guide star used. Effectively, this is the same as dithering. However, the order of this effect is small: exposures have a relative rotation rate of about 0.01 degree per hour. In those corners of the detector which are furthest away from the guide star this corresponds to an offset of about 0.3 pixels per hour ( for a ST-10XME at 1600mm focal length). Hence one can not necessarily rely on polar alignment errors to provide this kind of inherent dithering.

Therefore:
Take a sufficiently large number of darks, and offset the telescope after every exposure. To this end, re-acquire the guide star from a different position on the guide chip (if your observing system does not allow for automatic dithering). If you don't do so, the autoguiding system will place the guide star on exactly the same pixel as it was before (that's its job).
But even if you are well limited by the sky, dithering is still a good idea as the detector is not free from blemishes and permanent defects. Dithering helps to average out those defects.