Olaf Wucknitz for the LBG version of 21 July 2011

The long-baseline group wants to piggyback on MSSS and use it as a survey for long-baseline calibrators and to provide rough images for the brightest sources. This will form the basis for further long-baseline commissioning and science observations.

We want to include international and RS stations in the observations. Because the long-baseline analysis will not be possible in real time, we have to transfer the data on long baselines to Juelich and then continue with our analysis there or in Bonn. The calibration of the Dutch baselines and source catalogues from the main MSSS project will make this analysis easier.

This memo is based on the document Proposal and setup for early MSSS test observations by G. Heald, G. de Bruyn, R. Nijboer, M. Wise \& R. Pizzo version 0.3, 23 May 2011.

A new version that partly incorporated our proposal can be found here (version 0.51 of 5th Sep 2011).

Tests on MSSS data are reported *here.

(1) We want to image small fields around pre-selected sources (based on a catalogue compiled by the LBG, and on the sources found by the general MSSS project). For the calibration (and partly for the imaging) we can “phase up” the core stations or the complete Dutch array by applying the general MSSS calibration and adding the visibilities. This provides us with high sensitivity to allow us to calibrate also the long baselines.

(2) We are planning to search for signals on long baselines in fringe space (delay/rate). This procedure has proven to be able to find sources in full station beam fields. Details of the procedure have to be worked out in the preparation phase. Simple variants can be implemented easily, more sophisticated approaches with higher sensitivity are also possible. By combining the spatial information from phasing up the Dutch array and from the fringe parameters we will be able to locate signals from compact structures and estimate their flux densities. This is sufficient to provide long-baseline calibrators for later observations, even though their structure will not be perfectly determined by MSSS. We will also try to image small fields around the brightest calibrators using similar strategies as for (1).

The requirements for the observations are the same for both projects. They also overlap with plans of the MKSP.

One major issue on long baselines is the limited uv coverage. More pointings per field are therefore generally preferred. The LBA plans with 9×10 minutes per field should be good enough for our purposes, but the HA span should be optimised also in view of long-baseline coverage. We can provide simulations for this purpose.

For the HBA we are asking for several pointings per field instead of 1×15 minutes. Because of the overhead for switching and additional calibrator scans, there is a clear trade-off between coverage and efficiency. Three or four pointings are a good compromise. The estimates below are calculated for 3×5 minutes. Simulations of the improved imaging quality will be provided later.

                            LBA       HBA(0)      HBA proposed(1)    (2)
pointings/field           9x10min    1x15min           3x 5min      3x 5min
pointing overhead         9x 1min    1x 1min           3x 1min      3x 1min
calibrator scans/field    9x 2min    1x 2min           3x 2min      1x 2min
pointing overhead (cal)   9x 1min    1x 1min           3x 1min      1x 1min
total per field            126min    1x19min           1x27min      1x21min
total per field/3           42min       6.3min            9min         7min
fields                      619       3522              3522        3522
total obs time             433h       372h               530h        411h

total time (fields+cal)    371h       333h               411h        333h
(for data volume)

The first columns, LBA and HBA(0), are the plans for the Dutch MSSS. HBA proposed(1) Assumes one calibrator scan per pointing. Because the network of calibrators is sparse anyway, one calibrator scan can be used for several fields which reduces the overhead. This is taken into account in (2) where one calibrator scan is shared between three field pointings at different hour angles. This should be worked out in detail, but a significant reduction of calibration overheads seems to be realistic.

The total observing time (LBA+HBA) is increased by 20% for scenario (1) and 5% for scenario (2). At the same time the additional overhead reduces the real-time computing load.

The frequency coverage (currently planned: 16 MHz in 8 blocks of 2 MHz, regularly spaced) may be optimised. Regular spacing is not the best option for fringe searches. On the other hand the use of 2 MHz blocks allows the use of simpler and more efficient algorithms. Small adjustments of the exact frequency spacings between these block may improve the performance.

Adding more stations produces more baselines and thus more correlated visibility data. Even though not all data have to be kept at full resolution, this increases the output data rate from the correlator and the data volume that has to be processed by NDPPP.

To estimate the total data volume I use the observing time from above (with calibrators, but without overheads). The calculations are based on 24 CS, 11 RS and 8 IS (international stations), data recorded with 64 channels per subbands and 1sec integration time.

The data volume is

(obs time/1sec) * (# baselines) * 244 [subbands] * 64 [channels] * 4 [polarisations] * 2 [complex] * 4 Bytes

After NDPPP (see below) the factor for complex components is 3 instead of 2 because of the fully filled data weight column.

Number of baselines (including autocorrelations, number of stations counts HBA “ears”):

                             stations  baselines    TB total
LBA  CS+RS (as planned)         35        630         420
LBA  CS+RS+IS (as proposed)     43        946         631

HBA  CS (as planned)            48       1176         704   obs schedule (0)
HBA  CS+RS                      59       1770        1060   obs schedule (2)
HBA  CS+RS+IS (as proposed)     67       2278        1365   obs schedule (2)

The increase in total data volume is significant, but as long as CEP can keep up, this does not pose problems.

We have to select the international baselines (including all autocorrelations) before they are transferred to Juelich. This requires a second NDPPP run on the full raw correlator output. This run will apply flagging but not average the data in order to keep the full field of view.

In the NDPPP run for the non-international part of MSSS, all long baselines can be deselected so that they do not affect the output data volume and the subsequent BBS runs.

We will need the calibration of the non-international part of MSSS to calibrate all Dutch stations (or at least all CS). The calibration tables have to be stored and transferred to Juelich as well.

The output of “our” NDPPP run has to be moved to Juelich almost in real time to free the CEP storage regularly.

For the estimate of the total data volume we need the number of baselines. International baselines, including all autocorrelations, for N_NL Dutch stations and N_INT international stations:

N_BL =  N_NL*(N_INT+1) + N_INT*(N_INT+1)/2

LBA   N_NL = 35  N_INT = 8         N_BL = 351    fraction of all:  37 %

HBA   N_NL = 59  N_INT = 8         N_BL = 567    fraction of all:  25 %

This means selecting only international baselines reduces the data volume by a factor of 3-4.

The corresponding data volumes that have to be transferred to Juelich are:

LBA     351 TB
HBA     509 TB

total:  860 TB

With the full bandwidth of the existing 10 Gb/s line to Juelich, this data can be transferred in about 10 days, which is much less than the total observing time. Even a fraction of this bandwidth is sufficient to transfer all data.

This 860 TB are less than the tape-based storage capacity of 1 PB that is available for LOFAR use in Juelich. It will thus be possible to keep the data there for the post processing of the long baselines.

In addition, we have to transfer the BBS calibration tables, source lists and possibly images to Juelich to use them as basis for our analysis. This will only be a small fraction of the total data volume.

It has not been decided yet, which parts of the analysis will be done in Juelich and which ones in Bonn. For the “blind fringe-search” we will probably use GPUs on a cluster at Bonn University. In the worst (and unlikely) case, all data have to be transferred through 1 Gb/s links, which would take a similar time as the survey itself. Proper estimates for the pure processing time can only be made once we have at least a prototype software ready. This will be developed after the summer break.

So far it is unclear how far we have to stay away from the Sun. On international baselines, interplanetary scintillation and scattering in the solar corona are problematic. We are currently analysing our observations of TauA to determine the limits.

On short baselines we have to stay away from the sun for different reasons, namely the leakage of solar radiation via sidelobes of the station beams. It is thus possible that international baselines do not add further constraints.

We would like to add a field around 3C147 for the test observations as an “easy case” for international baselines. This source has the advantage of absence of any beating from sub-components (as is the case in 3C196). It is strong on all baselines and one of the best calibrators we can think of.

The MKSP has very similar wishes concerning a modifies observing schedule (several pointings per field for HBA) and the transfer to Juelich. This justifies the additional efforts even more.