The flagging scheme for BoA was mostly redesigned in a special branch of the BoA repository: The branch http://www.astro.uni-bonn.de/svn/boa/boa/branches/flagging was created for revision 2156 and merged with the trunk in revision 2182.
The main changes with respect to the old flagging scheme are:
- Independent binary flags
- Meaning of flags
- Flagging and unflagging routines
- Arguments of methods to access and plot data
Independent binary flags
The classes BolometerArray, ScanParameter, and DataEntity hold flags as Numeric.arrays (BolometerArray.Flags, ScanParameter.Flags, and DataEntity.DataFlags). Each element of these arrays represents as many independent binary flags as its datatype allows:
Flag |
Array |
Numeric typecode |
# Independent Flags |
ChannelFlags |
BolometerArray.Flags |
UInt32 |
32 |
IntegrationFlags |
ScanParameter.Flags |
UInt32 |
32 |
DataFlags |
DataEntity.DataFlags |
UInt8 |
8 |
The handling of the individual flags is done via the class FlagHandler, which encapsulates the necessary bitwise operations. Usually, there is no need for the BoA user to access this class directly. Manipulation and queriing of the flags is done in various flagging routines, described below.
Meaning of flags
The dirctories BolometerArray.cflags, ScanParameter.iflags, and DataEntity.dflags explain the meaning of (some of) values, the flags can assume:
Flag |
Directory |
Description |
Flag value |
ChannelFlags |
BolometerArray.cflags |
'NOT CONNECTED' |
1 |
|
|
'BAD SENSITIVITY' |
2 |
|
|
'LOW SENSITIVITY' |
3 |
|
|
'DARK BOLOMETER' |
4 |
|
|
'TEMPORARY' |
8 |
IntegrationFlags |
ScanParameter.iflags |
'TURNAROUND' |
1 |
|
|
'ACCELERATION THRESHOLD' |
2 |
|
|
'ELEVATION VELOCITY THRESHOLD' |
3 |
|
|
'SUBSCAN FLAGGED' |
7 |
|
|
'TEMPORARY' |
8 |
|
|
'BLANK DATA' |
9 |
DataFlags |
DataEntity.dflags |
'SPIKE' |
1 |
|
|
'GLITCH TYPE 1' |
2 |
|
|
'GLITCH TYPE 2' |
3 |
|
|
'SQUID FLUX JUMP' |
4 |
|
|
'DEMOD RAILED' |
5 |
|
|
'TEMPORARY' |
8 |
These values where chose in accordance with the flag definitions in the exchange data format (described in Data exchange with other reduction packages).
The meaning of the flags is not meant to be definitive. The various flagging routines flag by default according to this scheme: e.e. routine despike sets the DataFlag to 1('SPIKE'), routine flagTurnaround sets IntegrationFlag to 1 ('TURNAROUND'), but all flagging routines allow to chose other flags according to a different convention.
DataFlags has two flag values with special (or reserved) meaning: flag value 6 ('INTEGRATION FLAGGED') and 7 ('CHANNEL FLAGGED'). They indicate that the related ChannelFlag and/or IntegrationFlag is set and must not be manipulated by the user directly. (Instead, they are set by the related flagging methods.)
Flagging and unflagging routines
The following table gives an overview over all flagging/unflagging methods of class DataAna to be used by the user together with the default values for the flags:
method |
ChannelFlag |
IntegrationFlags |
DataFlag |
flagRms |
2 'BAD SENSITIVITY' |
|
7 'CHANNEL FLAGGED' (implied) |
flagFractionRms |
2 'BAD SENSITIVITY' |
|
7 'CHANNEL FLAGGED' (implied) |
flagAutoRms |
2 'BAD SENSITIVITY' |
|
7 'CHANNEL FLAGGED' (implied) |
flagChannels |
8 'TEMPORARY' |
|
7 'CHANNEL FLAGGED' (implied) |
unflagChannels |
[] All |
|
7 'CHANNEL FLAGGED' (implied) |
flagInTime |
|
8 'TEMPORARY' |
6 'INTEGRATION FLAGGED' (implied) |
flagMJD |
|
8 'TEMPORARY' |
6 'INTEGRATION FLAGGED' (implied) |
flagSubscan |
|
7 'SUBSCAN FLAGGED' |
6 'INTEGRATION FLAGGED' (implied) |
flagTurnaround |
|
1 'TURNAROUND' |
6 'INTEGRATION FLAGGED' (implied) |
flagSpeed |
|
3 'ELEVATION VELOCITY THRESHOLD' |
6 'INTEGRATION FLAGGED' (implied) |
flagAccel |
|
2 'ACCELERATION THRESHOLD' |
6 'INTEGRATION FLAGGED' (implied) |
unflagInTime |
|
[] All |
6 'INTEGRATION FLAGGED' (implied) |
unflagMJD |
|
[] All |
6 'INTEGRATION FLAGGED' (implied) |
unflagSubscan |
|
[] All |
6 'INTEGRATION FLAGGED' (implied) |
unflagTurnaround |
|
[] All |
6 'INTEGRATION FLAGGED' (implied) |
unflagSpeed |
|
[] All |
6 'INTEGRATION FLAGGED' (implied) |
unflagAccel |
|
[] All |
6 'INTEGRATION FLAGGED' (implied) |
despike |
|
1 'SPIKE' |
|
iterativeDespike |
|
1 'SPIKE' |
|
despike |
|
1 'SPIKE' |
|
flag |
|
8 'TEMPORARY' |
|
flagPosition |
|
8 'TEMPORARY' |
|
flagLon |
|
8 'TEMPORARY' |
|
unflag |
|
[] All |
|
unflagPosition |
|
[] All |
|
unflagLon |
|
[] All |
In principle, more than one flag value can be set in one call of one of these methods by specifiing a list of flag values. Passing an empty list means setting or unsetting all (non-reserved) flag values.
The methods BolometerArray.flag, BolometerArray.unflag, ScanParameter.flag, and ScanParameter.unflag must not be used by the user directly, since the related DataFlags cannot be manipulated properly by these methods.
Arguments of methods to access and plot data
To handle the new flagging scheme, the arguments of methods that access and plot the data have been unified.
As an example, the method ScanParameter.plotAzimuth has (among others) the arguments 'flag' and 'plotFlagged'. 'flag' is either 'None', a single integer, or a list of integers, while 'plotFlagged' is an integer. The meaning of the two arguments is the following:
flag |
plotFlagged |
Plot.. |
'None' |
0 |
all data |
[] |
0 |
unflagged data |
[] |
1 |
data with at least one flag set |
1 |
0 |
data with flag 1 not set |
1 |
1 |
data with flag 1 set |
[1,2] |
0 |
data with neither flag 1 nor flag 2 set |
[1,2] |
1 |
data with either flag 1 or flag 2 set |
For some routines, several Flags are relevant: E.g. DataEntity.signal() refers both to ChannelFlag and to DataFlag (e.g. one may want to plot unflagged data of flagged channels). For these routines, four arguments (channelFlag, plotFlaggedChannels, dataFlag, plotFlaggedData) are required. (E.g. signal(plotFlaggedChannels=1) plots unflagged data of flagged channels).
Please see the doc strings of the individual routines for more specific documentation.
In all cases, the default is to access or plot all unflagged data.