In this section the main pre-processing of the data takes place: debiasing and flatfielding. Superflats can be created as well.
Bias images are overscan corrected and median combined without rescaling. At least 3 exposures are needed in order to reject spurious events such as cosmics. We recommend at least 10 exposures in order to keep the noise low.
The master bias has the same name as the directory in which the individual bias frames reside, for example:
MYBIAS/MYBIAS_1.fits
Specify a lower and upper threshold in the min and max fields. Exposures with a statistical mode outside this interval are rejected from the processing and are moved to a BADMODE sub-directory.
Dark images are overscan corrected and median combined without rescaling. At least 3 exposures are needed in order to reject spurious events such as cosmics. We recommend at least 10 exposures in order to keep the noise low.
The master dark has the same name as the directory in which the individual bias frames reside, for example:
MYDARK/MYDARK_1.fits
Specify a lower and upper threshold in the min and max fields. Exposures with a statistical mode outside this interval are rejected from the processing and are moved to a BADMODE sub-directory.
Flat-field exposures are overscan corrected, have the master bias subtracted, are then rescaled to the same statistical mode and then median combined. Different gains in multi-chip cameras are preserved, thus after applying the master flat gain differences in the target exposures are corrected for. At least 3 flat field exposures are needed, but we recommend to go for 10. The illumination level of individual exposures should not exceed half the detector saturation level in order to stay clear from any non-linearities.
THELI automatically subtracts a master bias from the master flat. If you do not have a master bias or do not want this correction to take place, then set the do not apply bias switch at the top of the Calibration section.
The master flat has the same name as the directory in which the individual flat frames reside, for example:
MYFLAT/MYFLAT_1.fits
Specify a lower and upper threshold in the min and max fields. Exposures with a statistical mode outside this interval are rejected from the processing and are moved to a BADMODE sub-directory. This is useful to reject under- or overexposed flats.
Near-IR detectors have very different properties than the CCDs used in optical instruments. While they are not exposed, they are continuously resetted and rapidly reach a resetting equilibrium. On the other hand, when a series of exposures is made, the detector reaches an imaging equilibrium. Usually, the latter is established after about two to three exposures and recognised by a stable image background for the rest of the exposure series.
If you have dithered observations, and a series of e.g. 10 exposures was taken per dither point, then the detector will have reached the imaging equilibrium with e.g. the third exposure. All subsequent images will have the same background. While the telescope acquires the next dither position, the detector goes into the resetting equilibrium again and thus is in the same state as when you started the exposure series at the first dither position. Therefore, the image backgrounds are the same for the i-th exposures in the n-th sequence (apart from slower sky background variations). The consequence is that up to the defringing step in the Superflatting section they have to be processed separately from the other images. This script sorts the data in according directories where it is then processed automatically in the right manner.
Warning
It is essential that you do not mix sequences with different lengths in the SCIENCE directory. THELI assumes that only complete sequences of the same length are present.
You have 12 images per dither point. From subtracting one image from the next in the sequence, you found out that the detector settled into its imaging equilibrium starting with the third exposure. You would then enter 3 into the field # groups, and 12 into the one labelled Length.
The script will create three directories next to the SCIENCE directory, and redistribute the exposures onto them in the following manner:
* SCIENCE_S1: exposures 1, 13, 25, ...
* SCIENCE_S2: exposures 2, 14, 26, ...
* SCIENCE_S3: exposures 3-12, 15-24, 27-36, ...
The background modelling and background subtraction must be done separately for each of these directories. THELI will do this automatically for all SCIENCE_Si directories and, if applicable, OFFTARGET_Si directories. This holds for the Calibrate data and the Subtract SUPERFLAT tasks only. You do not have to make any changes in the directory tree defined in the Initialise section.
All other processing tasks do not (and do not have to) loop over the SCIENCE_Si directories. THELI will know that it has to do this loop if the Spread sequence (IR) task has either been executed, or is activated.
After the sky background has been subtracted, you merge the images again into the previous SCIENCE directory. This is done in the Superflatting section (see merge sequence). OFFTARGET fields are processed automatically after the SCIENCE data have been dealt with.
Images are overscan corrected, debiased (or dark-subtracted) and flat fielded. If suitable, a superflat is calculated from the data as well. You can choose between a static superflat, i.e. one that is calculated from all images in the directory, or a dynamic superflat, i.e. one calculated from a set of images taken immediately before and after an exposure.
Filename extension: After running through this step, images have the string OFC in their file name between the chip number and the suffix, e.g.:
NGC1234_1OFC.fits
Specify a lower and upper threshold in the min and max fields. Exposures with a mode outside this interval are rejected from the processing and are moved to a BADMODE sub-directory. If left empty, no thresholding will be done.
Do not apply BIAS / DARK: If you work with near-IR data, you probably do not want to subtract a master bias as the so-called pre-read has already been subtracted. If you do not have a master bias or master dark, you have to mark the same switch.
Do not apply FLAT: activate this switch if you do not want to calibrate your target exposures by a flat field. Mid-IR observers might want to do that.
Use DARK: If you want to subtract a master DARK instead of the master BIAS, then indicate this here.
Create SUPERFLAT: Opens the parameter configuration dialogue. Choose suitable settings for the creation of the superflat.
If you subtract a master dark from the data, then THELI looks up the dark’s exposure time. If the latter deviates by more than 1% from the exposure time of the SCIENCE images, then the master dark is rescaled accordingly. Thus you can subtract a high S/N masterdark with longer exposure time, introducing less noise in the calibrated target frames.
If your camera does not have an overscan, then the bias offset is still present in the master dark. The offset would be rescaled, hence adding a constant to the sky background of the target images. While this as such is not critical because the background gets subtracted anyway, it would modify the sky background statistics (which is not used by THELI, but you might be interested in it). To avoid this, the mode of the master dark is subtracted before a rescaling takes place. For cameras with overscan this correction has no effect as the mode is essentially zero.
Note
THELI does not perform a temperature rescaling of the master dark. If the dark current in your camera is temperature dependent, then make sure the darks are taken with identical settings. You can also attempt to manually rescale the dark frames in a trial and error manner.
If you are unfamiliar with this technique, then please read the section About superflatting for important background information.
For the calculation of a superflat we need to detect and mask objects. To this end THELI uses SExtractor. You should familiarise yourself how object detection works with this tool as THELI makes frequent use of it.
Choosing the right detection thresholds
A good starting point for optical data is DT = 1.0 and DMIN = 5, depending on the flatness of the image. If your data exhibit strong fringing, then you can no longer use very low detection thresholds as the fringes themselves will be detected as objects and removed from the superflat (or background model, if you prefer that terminology). In case of strong fringing use a higher detection threshold such as DT = 5. With near-IR detectors DT and DMIN often must be increased to 10 in order to not mask features in the very inhomogeneous sky background. If one or more of those fields is left empty, then the default values will be used without warning. The images where objects are masked are stored in a MASK_IMAGES sub-directory.
To check if your detection thresholds are too tight or too loose, go to the MASK_IMAGES directory. All objects above the specified thresholds appear masked. If
Repeat the creation of the superflat until you are satisfied.
Dynamic or static superflat (or background models)
The window size (w) regulates whether a static or a dynamic superflat is used. If the field is left empty (or set to zero), then a static superflat is calculated from all images in the SCIENCE directory. It will have the same name as the directory, e.g.:
SCIENCE/SCIENCE_1.fits
If w>0, then w defines the number of images taken before and after the current image from which to calculate the (dynamic) superflat. For example, if you set w=2, the superflat for the n-th exposure will be calculated from images n-2, n-1, n+1, and n+2. That is, 2 w exposures contribute to the superflat. If the current exposure is for example the first (n=1) in the list and w=2, then images n=2...5 will be used, i.e. the window size is kept constant if the window can not be centered symmetrically on the exposure that is to be corrected. Optionally, the current exposure can be included in the window.
Note
THELI does not look at FITS headers in order to determine which exposures were taken immediately before and after the image that is being corrected. THELI will use those images that are adjacent in an alphanumerically sorted list. As observatories usually use running indices or the observation time for the filename, this should be fine in most cases. But you should check that this is indeed the case.
Since the dynamic superflat superflat will be overwritten by the next one, it has to be applied immediately to the image for which it was calculated. Normally superflatting is done in the Superflatting section, but in this case it has to be done here and now. You have the same three options as in the Superflatting section:
To this end the Smoothing size parameter and the Use unsmoothed SUPERFLAT switch from the Superflatting section do apply. This is the only time where a parameter setting from one section in the GUI affects the processing steps in another section (for w>0). There is no need to run the according task in the Superflatting section afterwards. The chosen processing step will be highlighted there by the usual green background colour to indicate that is has been done already.
Note
If you have defined an OFFTARGET field, and set w>0, the SUPERFLAT is determined from the OFFTARGET data and applied to the SCIENCE data. This will only work well if SCIENCE and OFFTARGET exposures were alternated frequently!
Non-linearity usually arises when pixels approach their saturation limit and do not response in a linear manner anymore to the incoming flux. In some cases, e.g. near-IR detectors, non-linearities can also appear at very low exposure levels.
THELI can correct for the detector non-linearity provided that corresponding correction polynomials are known. Currently, this is only being implemented for WFC@INT. The polynomial is assumed to be of 3rd order and has the form
If you know the correction polynomial for an instrument called SOME_INSTRUMENT in THELI, then add the polynomial coefficients to a file SOME_INSTRUMENT.nldat and save it in the same place as SOME_INSTRUMENT.ini, i.e. in one of
/home/user/THELI/gui/scripts/instruments_professional/
/home/user/.theli/instruments_user/
Each chip of the camera is represented in SOME_INSTRUMENT.nldat by a single line containing the four polynomial coefficients
Unknown coefficients must be set explicitly to zero. For example, if
the correction polynomial is of 2nd order, then you must set 
.
In order to fully correct for it, one must :
This processing scheme is significantly different from the normal approach in THELI as it changes the data flow and requires the creation of intermediate data. THELI will do all of this automatically and invisible for you. You just have to activate the Non-linearity correction switch and proceed as normal. The only difference is that the processing takes significantly longer than without non-linearity correction.