This list is anything but complete. Please add your own favourite data set if you cannot find it here!

You may also want to check MOM, in particular the LEA066 (long-baseline commissioning) project. The Solar project is LEA012.

Imaging observations of Jupiter and Saturn are LEA050.

Local transferlogs to Juelich.

"all" transferlogs on LOFAR wiki

• Raw voltages in fly's eye mode are correct from 28 Oct 2011 onwards (L2011_32955).
• DE601 clock offset before ca. 30 Oct 2011 is 1.1 microsec, 2.2 microsec in L2011_33159, and fixed since L34671 11/12 November 2011.
• Ordering of antennas in MS as written/read by LofarStMan: Ant1-Ant2 with
• Ant1⇐Ant2 until at least JD 55742.6, 29th June 2011, then swapped since
• at least MJD 55748.4, 5th July 2011. Swapped back when reading with new LofarStMan version from 2/3 Nov 2011 (2 on cep, 3 on widemap) onwards. See here.

120 subbands, 256 channels, 3 sec, 13 h, 6 antennas incl Ef, 30-80 MHz 307 GB

log: /globalhome/lofarsystem/oldlog/L2009_14319

“3C196 Long Baseline Re-observation” Anderson is really 3C196 120 subbands, 256 channels, 3 sec, 13 h, 6 antennas incl Ef, 30-80 MHz 307 GB

problems with CS302: X and Y have very different gain amplitudes (factor 2-3) and different phases in XX and YY (constant ~70 deg)

Bonn: moved to MPIfR /data9/lofar/L2009_14319

CygA LBA 10-30 MHz, 120 subbands, 256 channels

48h: 18 Sep 14:55 - 20 Sep 14:55 2009

3.4 TB

Bonn: moved to io09:/data9/lofar/L2009_14473

deleted

first semi-successful HBA observation, images in several talks

Bonn: moved to MPIfR io09 /data9/lofar/L2010_07608

3.8TB

removed

Bonn: moved to MPIfR io09 /data9/lofar/L2010_07804

391GB

Bonn: moved to MPIfR io09 /data9/lofar/L2010_07807

346GB

“Long baseline test with Tautenberg” Heald & Haas, 248 subbands, 6h, 1sec, 512 channels, originally 2.4 TB, ca. 30 subbands missing due to disk crash, now on io09:/data9/lofar/D2010_16370/

deleted

Bonn: moved to MPIfR io09 /data9/lofar/D2010_16704/

http://mom.astron.nl:8082/lofar-observations/observation274 2010-02-12.18:15:00 - 2010-02-13.00:15:00

used for first proper LBA map of 3C196 (??)

Long baseline test with Garching Polatidis 160 randomly scattered subbands, 6h, 1sec

1004 GB total

solar eclipse 4 Jan 2011

666 GB

→ io07:/data7/lofar/LOFAR_archive_io07

deleted

• 3C48
• 13 Jan 2011 16:18-17:18 (1h)
• 244 contiguous subbands, ca. 119-167 MHz
• 64 channels, 1 sec
• 100 GB
• 7+3 stations: CS005HBA0, CS007HBA0, DE603HBA, FR606HBA, RS106HBA, RS208HBA, RS306HBA, RS307HBA, RS503HBA
• no station calibration (international), melting snow on DE603HBA!
• where:
  • offline cluster: lse001-012 in /data4
  • AIfA Bonn: /widemap/raid0/LOFAR_archive/L2011_22744
• analysis and results:

• 3C147
• 13 Jan 2011 20:20-21:20 (1h)
• 244 contiguous subbands, ca. 119-167 MHz
• 64 channels, 1 sec
• 99 GB
• 7+3 stations: CS005HBA0, CS007HBA0, DE603HBA, FR606HBA, RS106HBA, RS208HBA, RS306HBA, RS307HBA, RS503HBA
• no station calibration (international), melting snow on DE603HBA!
• where:
  • offline cluster: lse001-012 in /data4
  • AIfA Bonn: /widemap/raid0/LOFAR_archive/L2011_22745
• analysis and results:

• 3C196
• 13/14 Jan 2011 21:30-03:30 (6h)
• 244 contiguous subbands, ca. 119-167 MHz
• 64 channels, 1 sec
• 590 GB
• 7+3 stations: CS005HBA0, CS007HBA0, DE603HBA, FR606HBA, RS106HBA, RS208HBA, RS306HBA, RS307HBA, RS503HBA
• no station calibration (international), melting snow on DE603HBA!
• where:
  • offline cluster: lse001-012 in /data4
  • AIfA Bonn: /widemap/raid0/LOFAR_archive/L2011_22746
  • copied to io07:/data7/lofar/LOFAR_archive_io07/L2011_22746 and MS removed on widemap
• analysis and results:
  • LSM 26 Jan 2011, Olaf Wucknitz

• 3C196
• 20/21 Jan 2011 17:30-05:30 (12h)
• 60 subbands (every 4th)
• 64 channels, 1 sec
• 239 GB
• 6+3 stations: CS302HBA0, DE603HBA, FR606HBA, RS106HBA, RS208HBA, RS306HBA, RS307HBA, RS503HBA, UK608HBA
• DE603HBA weak in the higher bands
• where:
  • offline cluster: lse001-012 in /data4
  • AIfA Bonn: /widemap/raid0/LOFAR_archive/L2011_22883
• analysis and results:
  • LSM 26 Jan 2011, Olaf Wucknitz
  • RFI from airplane (second one)

• MOM entries: 7162 and 7163
• 3C147
• 23/24 Jan 2011 15:30-03:30 (12h)
• 244 contiguous subbands, ca. 120-165 MHz
• 64 channels, 1 sec
• 969 GB
• 6+3=9 stations: CS302HBA0, DE603HBA, FR606HBA, RS106HBA, RS208HBA, RS306HBA, RS307HBA, RS503HBA, UK608HBA
• DE603HBA weak in the higher bands
• good signal on long baselines, simple source structure
• where:
  • offline cluster: lse001-012,022-024 in /data4
  • AIfA Bonn: /widemap/raid0/LOFAR_archive/L2011_22899, moved to io09:/data9/lofar/L2011_22899

• test data set for transfer to Jülich
• analysis and results:
• LSM 26 Jan 2011, Olaf Wucknitz

• MOM entries: 8084 and 8085
• Taurus A / crab / 3C144
• 13/14 Feb 2011 14:00-02:00 (12h)
• 244 contiguous subbands, 115-160 MHz
• 64 channels, 1 sec
• 1.3 TB
• 8+3=11 stations: CS032HBA0, DE601HBA, DE603HBA, FR606HBA, RS106HBA, RS205HBA, RS208HBA, RS306HBA, RS307HBA, RS406HBA, RS503HBA
• DE601HBA failed, DE603HBA weak in the higher subbands
• problem with bad subbands: good 0-29,61-90,122-151,183-212, bad: 31-60,92-121,152-181,213-242, all zero: 30,91,182,243
• /globalhome/lofarsystem/log/L23410/L23410.parset
• where:
  • offline cluster: lse001-012,022-024 in /data4
  • AIfA Bonn: /widemap/raid2/LOFAR_archive2/L2011_23410
• significant ionospheric delays in the afternoon, differential Faraday rotation
• strong signal on long baselines
• analysis and results
  • LSM 23 Feb 2011, Olaf Wucknitz
  • Taurus A section
  • RFI from airplane (first one)
  • working on it: Olaf Wucknitz, Ger de Bruyn, Sarod Yatawatta

monitoring

9 Apr 2011 16:00-17:00

Bonn → io07:/data7/lofar/LOFAR_archive_io07

deleted

location in Juelich

monitoring

9 Apr 2011 14:47-15:47

had to delete MS on wm9, but parset still there

location in Juelich

interferometry + beam-formed, 2+2 stations, 16 subbands, problems with beam-formed raw voltages

Antennas: 0=FR606, 1=RS106, 2=RS208, 3=UK608

Due to a bug in the software writing the raw voltage data, some of the files are corrupted. Incoherent Stokes and visibilities do not seem to be affected.

Bad raw data are:

• 0=FR606 (X: subband 0 bad, Y: subband 1 bad; all bad in first second)
• 3=UK608 (all bad)

The other antennas are fine.

interferometry + beam-formed, 2+2 stations, 16 subbands, problems with beam-formed raw voltages

monitoring, 1h, HBA_high

16 April 2011 15:00-16:00

missing subbands (SBnnn) and subbands with no data: 2,17,18,19,20,22,23,27,28,29,32,35,84,85,86,177,178,179

digital TV in almost all stations (not in FR606), also narrower peaks above 243 MHz

location in Juelich

flaring crab, 10h HBA_low 17 April 2011, failed (no data in MS)

17 April 2011 9:30-19:30

1h, 3 NL + 3 int. stations, interferometry plus beam-formed, 60 subbands some problems with the raw voltage data

antennas: DE603,FR606,RS106,RS208,RS307,UK608 all HBA 3/0 = RS208 X is missing in raw

location in Juelich

monitoring, 1h, HBA_high, only 204 subbands, 8.6GB each

25 April 2011 14:30-15:30

no data in FR606 and RS503, only a few minutes good in RS406

digital TV in all stations, particularly strong in DE603 and UK608, weaker in RS307

channel allocation (wikipedia)

also narrower peaks above 243 MHz

only selected baselines (INT-RS?) in Bonn at /widemap/raid2/LOFAR_archive2/L2011_25696 (127GB out of total 1.8TB)

location in Juelich

flaring crab, 10h HBA_low 24 April 2011, 9:30-19:30

244 subbands, 86GB each (20TB)

• DE603HBA: SB061-SB182 bad for first few minutes
• FR606HBA: no data after ca. 15:00 UTC
• RS503HBA: no data (all flagged)

selected baselines (int-int,int-rs,rs auto) on widemap in LOFAR_archive2/L2011_25697 (3.9GB per subband, total 930GB)

NDPPP version of all baselines in Jülich (8TB ?)

location in Juelich

Taurus A, 10h, LBA, 1 May 2011 09:30-19:30

only noise in international stations, see issue tracker, no 300

missing or bad subbands (SBnnn): 32,69,106,143,180,217

Bonn: moved from raid2 to io07:/data7/lofar/LOFAR_archive_io07/L2011_25807

deleted

location in Juelich

monitoring, 1h, LBA_OUTER, 30 April 2011 14:00-15:00

only noise in international stations, see issue tracker, no 300

missing or bad subbands (SBnnn): 126,163,200,237

/widemap/raid2/LOFAR_archive2/L2011_25808 moved to io07:/data7/lofar/LOFAR_archive_io07/L2011_25808

712 GB, also 149 GB in own selected_baselines

monitoring, 1h, 6 May 2011, 13:40-14:40

HBA low (?, maybe LBA)

all Dutch + DE602,DE603,UK608

only selected_baselines in /widemap/raid2/LOFAR_archive2/L2011_25982/selected_baselines/ (152GB)

Virgo A, 10h, HBA low, 7/8 May 2011, 16:00-2:00

Dutch remote + CS002 + DE602,DE603,UK608

international fringes very weak

1.4 TB in /widemap/raid2/LOFAR_archive2/L2011_25987 → io07:/data7/lofar/LOFAR_archive_io07/L2011_25987

deleted

244×5.8 GB

weight statistics for all SB??0: CS002HBA0 completely flagged, CS002HBA1 very strange (don't use!), RS503 completely flagged, DE603 problems in first ~30min in the middle 50% of all subbands, all other okay

Taurus A, 10h, LBA, 16 May 2011, 11:45-21:45

all Dutch + DE602,DE603,UK608

no proper signal in international stations, see issue #300

monitoring 3C147, 1h, LBA, 19 May 2011, 13:30-14:30 “all” Dutch plus DE602,DE603,FR606 (27 total) each SB has 2.8GB

only selected baselines (121GB) in /widemap/raid2/LOFAR_archive2/L2011_26679/selected_baselines/

10h LBA with all Dutch + DE602,DE603,FR606

21 May 2011 09:00-19:00

missing subbands: SB023,SB064,SB105,SB146,SB187,SB228

selected_baselines (all combinations(?) of RS,INT, 6.2GB/sb, 1.5TB total) in /widemap/raid2/LOFAR_archive2/L2011_26685/selected_baselines

NDPPP: cep2, all /data/scratch/pipeline/L26685/ (43GB/sb, 10TB total)

NDPPPed data transferred to Jülich. Missing subbands: SB023, SB039, SB064, SB080, SB105, SB121, SB146, SB162, SB187, SB203, SB228

raw data deleted on cep2

location in Juelich (dppp)

HBA_low monitoring, 3C147, 1h, 27 May 2011 13:30-14:30 Dutch RS + few CS + DE601, DE602, UK608

DE601 all flagged

missing subbands: SB023,064,105,146,187,228

237GB in /widemap/raid2/LOFAR_archive2/L2011_27760

HBA_low TauA 10h, 28 May 2011, 9:00-19:00 Dutch RS + few CS + DE601, DE602, UK608

something wrong with DE601, no fringes

missing subbands: SB023,064,105,146,187,228

/widemap/raid2/LOFAR_archive2/L2011_27759 238x10GB, total 2.4TB

NDPPP (only flagged), cep2, /data/scratch/pipeline/L27759, 14GB/sb

raw data deleted on cep2

1h 3C147 monitoring, HBA low

Dutch RS + few CS + DE602,DE603

30 May 2011 12:25-13:25

402GB in /widemap/raid2/LOFAR_archive2/L2011_27806/

12h 0957+561, HBA low, 05 June 2011 11:00-23:00

CS004,CS006,CS024,CS101,RS106,RS205,RS208,RS306,RS307,RS406,RS503,DE602,DE603

subbands SB024,066,108,150,192,234 missing

• where:
  • cep2
  • AIfA Bonn: /widemap/raid2/LOFAR_archive2/L2011_28128 moved to io07:/data7/lofar/LOFAR_archive_io07/L2011_28128 (deleted by mistake)

14GB/sb, 3.2TB total

12h Sun+TauA (each about half of the beams), LBA, 13 June 2011 05:00-17:00

all NL plus some international

original: 132GB/sb, 30TB total

pipelined version: averaged all channels, still 1sec (?), 1.7GB/sb, 399GB total (/widemap/wm2/raid1/LOFAR_archive/L2011_28318_pipeline/), moved to io09:/data9/lofar/L2011_28318_pipeline/

12h Sun+TauA (each about half of the beams), HBA low, 15 June 2011 05:00-17:00

original: 330GB/sb, 74TB total

SAP000: SB000-SB121 Sun

SAP001: SB122-SB243 TauA

alternating subbands to cover the full range

pipelined version: averaged all channels, still 1sec (?), 4.2GB/sb, 983GB total (/widemap/wm2/raid1/LOFAR_archive/L2011_28351_pipeline/), moved to io09:/data9/lofar/L2011_28351_pipeline

deleted

12h Sun+TauA (each about half of the beams), HBA high, 17 June 2011 05:00-17:00

failed, no data

LEA050

1:22h, 08 July 2011, 8:38-10:00, LBA

all RS, few CS, DE602,DE603,DE604

(CS003LBA,CS005LBA,CS007LBA,CS101LBA,DE602LBA,DE603LBA,DE604LBA,RS106LBA,RS205LBA,RS208LBA,RS306LBA,RS307LBA,RS406LBA',RS503LBA,RS509LBA)

DE603 dead

This was the second attempt. The first one just before (with a good chance for Io bursts) failed.

241 subbands, 1.2GB/sb, total 289GB

/widemap/raid2/LOFAR_archive2/L2011_29695

08 July 2011, 10:38-10:43

redo 25531 to test beam-formed data in fly's eye mode with incoherent Stokes and visibilities

raw: 12x27GB, uv: 60x14MB, incoherent: 60x58MB total 325GB

/widemap/raid0/LOFAR_archive/L2011_29699

Results of the comparisons of raw voltages, incoherent Stokes and visibilities are presented here.

LEA050

LBA 5 min, 08 July 2011 10:28-10:33

18 GB, 76 MB per subband

LEA050

LBA, 06 Aug 2011 4:00-9:30

10-58 MHz

CS007,CS017,CS101,CS103,CS302,RS106,RS205,RS208,RS306,RS307,RS406,RS503,RS508,RS509,DE602,DE603,DE604

CS302,DE602,RS306,RS508 dead

autocorrelations of RS205 show strange wide-band RFI

subbands missing: 055,118,181

6.2 GB per subband, total 1.5 TB

at AIfA: /widemap/raid2/LOFAR_archive2/L2011_30253 → /data7/lofar/LOFAR_archive_io07/L2011_30253

total 137GB

/widemap/raid0/LOFAR_archive/L2011_30254

LEA050

LBA, 13 Aug 2011 6:00-12:00

CS007,CS017,CS101,CS103,CS302,RS106,RS205,RS208,RS306,RS307,RS406,RS503,RS508,RS509,DE602,DE603,DE604

comment from Antonis: did not work, no data

total 206GB

/widemap/raid0/LOFAR_archive/L2011_30345

18 Aug 2011

208 GB in /widemap/raid0/LOFAR_archive/L2011_30440

20 Aug 2011 6:30-7:30

296 GB in /widemap/raid0/LOFAR_archive/L2011_30441

LEA050

LBA, 24 Aug 2011 0:00-6:00

CS007,CS017,CS101,CS103,CS302,RS106,RS205,RS208,RS306,RS307,RS406,RS503,RS508,RS509,DE602,DE603,DE604,DE605

aborted, no data

03 Sep 2011, 04:00-08:00

TaurusA HBA low

CS017,CS101,CS301,RS106,RS205,RS208,RS307,RS406,RS503,RS508,RS509,DE601,DE602,DE603,DE604,DE605

DE602 and DE605 failed (all data flagged)

DE601 clock offset: ca. 1 microsec

DE603 (e.g. to DE601 or RS106) is responding stronger to PKS0531+19 (2.6deg away towards south) than to TauA

DE604 weak

/widemap/raid2/LOFAR_archive2/L2011_30683

244 x 4.5GB, total 1.1 TB

10 Sep 2011, 1h with international stations

failed, no data

12 Sep 2011, 4:45-6:45, TauA HBA low

4.1 GB/sb, 935 GB total

some subbands missing: locus24 (SB019,101,183); locus37 (SB032,114,196);locus50 (SB045,127,209); locus76 (SB071,153,235)

CS002,CS017,CS024,CS101,CS301,RS106,RS205,RS208,RS307,RS406,RS503,RS508,RS509,DE602,DE603,DE604,DE605,FR606

Bonn: /widemap/raid0/LOFAR_archive/L2011_30820, moved to io09:/data9/lofar/L2011_30820

much weaker than others: DE603,DE604

relatively weak: FR606

weak or funny: both CS017

startTime='2011-09-15 04:45:00'

stopTime='2011-09-15 06:45:00'

TauA HBA

This was a test of SE607 (seems to be flagged).

stations [CS002HBA0,CS002HBA1,CS017HBA0,CS017HBA1,CS024HBA0,CS024HBA1,CS101HBA0,CS101HBA1,DE601HBA,DE602HBA,DE605HBA,FR606HBA,RS106HBA,RS205HBA,RS208HBA,RS307HBA,RS406HBA,RS503HBA,RS508HBA,RS509HBA,SE607HBA]

232 subbands, 3.4 GB per subband, total 763GB

Bonn: /widemap/raid0/LOFAR_archive/L2011_30852

moved unaveraged to io07:/data7/lofar/LOFAR_archive_io07/L2012_30852_unaveraged/

NDPPP with flagging and autoweight and averaging to 4 channels/SB and 4sec: /widemap/raid0/LOFAR_archive/L2011_30852/ndppp_4chan_4sec (226 x 107MB = 24GB total)

some subbands short: 050,068,132,150,214,232

HBA

Observation.startTime = '2011-09-16 04:00:00'
Observation.stopTime = '2011-09-16 06:00:00'

This was a test of SE607 which (I believe) did not deliver data.

[CS017HBA0 CS017HBA1 CS024HBA0 CS024HBA1 DE601HBA DE602HBA FR606HBA RS106HBA RS205HBA RS208HBA RS307HBA RS406HBA RS503HBA RS508HBA RS509HBA SE607HBA]

232 x 2 GB = 461 GB

Bonn: /widemap/raid0/LOFAR_archive/L2011_30864

moved unaveraged to io07:/data7/lofar/LOFAR_archive_io07/L2011_30864/UNAVERAGED

NDPPP to 4 channels, 4 sec → ndppp_4chan_4sec (232 x 64 MB, total 15 GB)

HBA

Observation.startTime = '2011-09-17 02:05:00'
Observation.stopTime = '2011-09-17 14:05:00'

ended prematurely, cannot be read in normal way

11310*136 rows can be read with pyrap (3.1h)

[CS002HBA0 CS002HBA1 CS101HBA0 CS101HBA1 DE601HBA DE602HBA DE603HBA DE604HBA DE605HBA RS106HBA RS205HBA RS208HBA RS307HBA RS406HBA RS508HBA RS509HBA]

MS deleted on widemap

TauA HBA 1.5h, 6 int stations plus Dutch, UK608 and SE607 flagged

1.2TB in /widemap/raid0/LOFAR_archive/L2011_30925/

moved to MPIfR io09:/data9/lofar/L2011_30925

CasA (SAP000) and CygA (SAP001) with LBA 30-77 MHz

MoM: 31561,31575

subbands 154-396 in steps of 2 (both parts use same frequencies)

raw data: 68 GB per subband, total 16 TB

NDPPPed to 1 channel, 4 sec → /widemap/raid0/LOFAR_archive/L2011_31848/NDPPP 891 MB/SB, total 200GB

strong amplitude ripple in CS030LBA!

CasA (SAP000) and CygA (SAP001) with HBA 115-162 MHz

MoM: 36116,36117

subbands 77-319 in steps of 2 (both parts use same frequencies)

raw data: 28 GB per subband, total 6.6 TB

NDPPPed to 1 channel, 4 sec → /widemap/raid0/LOFAR_archive/L2011_32455/NDPPP 373 MB/SB, total 88GB

3C123 with LBA outer

12h on 21/22 Oct 2011 19:00-07:00

25GB per subband, 5.7TB total (240 subbands)

superterp, all RS, 8 international (23 stations total)

OLD: /widemap/raid0/LOFAR_archive/L2011_32466/NDPPP: 323GB (1.4GB/SB) all subbands, flagged and averaged to 4 channels and 2sec (NDPPP), swapped antennas, deleted

new: all subbands, flagged, averaged to 4 channels and 4sec (NDPPP), mount type set to ALT-AZ, 240×735 MB, 173 GB total

3C123 with HBA low

12h on 23/24 Oct 2011 19:00-07:00

39GB per subband, 9.1TB total (244 subbands)

superterp, all RS, 8 international (29 total including both ears)

old (deleted): /widemap/raid0/LOFAR_archive/L2011_32468/NDPPP: 517GB (2.2GB/SB) all subbands, flagged and averaged to 4 channels and 2sec (NDPPP)

copy of this old NDPPP output on lse023:/data2/scratch/L32468_lbg for Adam Deller

DE604 has funny amplitudes, also on average a few percent missing in WEIGHT_SPECTRUM (do_antenna_statistics.py).

new: /widemap/raid0/LOFAR_archive/L2011_32468/NDPPP to 4 channels, 4 sec, mount type set to ALT-AZ, 244×1.1 GB, 276 GB total frequency layout not optimal, because 2 edge channels were skipped on both sides, instead of being flagged.

copy of NDPPP output (with mount type set to ALT-AZ!) at lse023:/data2/scratch/L32468_new, also uvfits at lse023:/data2/scratch/ndppp_12sec_21if_12chan_rl_sqrt.uvfits

weight statistics generally okay, most DE stations have a small flagged fraction

/widemap/raid0/LOFAR_archive/L2011_32468/NDPPP_clockcorr clock-corrected version of NDPPP, clock offset of 1microsec for DE601 corrected.

/widemap/raid0/LOFAR_archive/L2011_32468/NDPPP_clockcorr_combined : combined in blocks of 64 subbands, plus all (256, highest 12 flagged dummies)

/widemap/raid0/LOFAR_archive/L2011_32468/NDPPP_clockcorr_combined_phased : phased-up superterp

Crab pulsar with coherent superterp, raw voltages.

HBA low

50 subbands, 16 channels per subband

30min on 27 Oct 2011 3:05-3:35

2×132 GB = 263 GB

in Bonn: /widemap/raid0/LOFAR_archive/L2011_32772

L32773 - Complex Voltage Fly's Eye (15min) 1.1TB (8x2x66GB)

[CS002HBA0,CS002HBA1,CS003HBA0,CS003HBA1,DE602HBA,RS205HBA,SE607HBA,UK608HBA]

→ io09:/data9/lofar/L2011_32773

test raw-voltage data in fly's eye mode with simultaneous incoherent Stokes sum and correlations

900sec on 28 Oct 2011 5:45-6:00

8 stations: CS002(2x),CS003(2x),DE602,RS205,SE607,UK608

HBA low, 1sec integrations, 16 channels/SB, incoherent Stokes I (4 integration steps)

nominal 32 subbands, in MS: 1,18,27 missing, 3,6 with no data (moved to BAD/)

Bonn AIfA: /widemap/raid0/LOFAR_archive/L2011_32955

comparison between raw voltages and visibilities: generally okay, that means raw voltages in fly's eye mode are working now, but there seem to be problems in the visibilities whenever same samples within the integration are flagged (happens very often in antenna 4 = DE602HBA)

The analysis can be found here.

TauA with 8 IS and “all” Dutch stations, HBA dual (some out), HBA low

12h on 30/31 Oct 2011 20:45-08:45

108GB/SB, 244 subbands, 26TB total

Subbands 054,115,176,237 (all on locus087) are incomplete (395MB instead of 108GB). They cannot be processed by NDPPP after baseline selection.

all on locus087

49 stations

Selected baselines without CS on cep2, then NDPPP with aoflagger to 4 channels and 2 sec. The NDPPP output still has ANT1/ANT2 swapped (and data conjugated, UVW flipped).

selected/NDPPP → /widemap/raid0/LOFAR_archive/L2011_33159/selected_ndppp (240 x 764 MB, total 179 GB)

clock offset correction in DE601 has wrong sign, is now 2.2musec instead of 1.1musec before

International stations generally bad. Good fractions in one subband:

DE601HBA   0.987650
DE602HBA   0.000000
DE603HBA   0.983038
DE604HBA   0.000000
DE605HBA   0.210334
FR606HBA   0.000092
SE607HBA   0.000000
UK608HBA   0.000000

TauA LBA (30-80 MHz) with “full” international array, all 8 IS

06/07 November 2011 19:30-07:30

not downloaded to Bonn, yet

244 subbands (47 GB each), total 12 TB

statistics for SB000,061,122,183: NL all good, international all bad (only DE603 and DE605 have a small good fraction at lower subbands)

TauA LBA (30-80 MHz), almost all international (not DE602 and DE604), many CS and RS

all used stations delivered almost 100% good data

11/12 November 2011

244×36 GB = 8.5 TB total

NDPPP to 4 channels and 4sec on CEP2 [autoweight=true], Bonn AIfA: /widemap/raid0/LOFAR_archive/L2011_34671/NDPPP, mount type set to ALT-AZ

244×1.1 GB = 258 GB

NDPPP version: edge channels excluded, therefore non-uniform channel spacings between subbands; ndppp_4chan_4sec__to_1chan_8sec_in4if.uvfits (averaged within subbands) tries to correct this

clock offset in DE601 now close to zero

location in Juelich

Jupiter (possible Io-A bursts) raw voltages in fly's eye mode plus correlations

16 Nov 2011, 20:45-22:30

LBA outer, correlations 1sec, 61 subbands, 256 channels/SB, filter 10-90 MHz, subbands 72..132 (14.06-25.78 MHz)

stations: 0=DE603,1=FR606,2=RS106,3=RS307,4=SE607,5=UK608 (should have been DE602 instead of DE603)

MS copied to /widemap/raid2/LOFAR_archive2/L2011_34940/MS deleted (61×1.1GB = 66GB)

parset kept: /widemap/raid2/LOFAR_archive2/L2011_34940/L34940.parset

raw files: 24x288GB, 6.8 TB (split into 2×31 subbands, very last subband is junk)

not pointing at Jupiter, no signal seen

TauA HBA low 12h on 20/21 Nov 2011 19:00-07:00

244×108 GB = 26 TB total

all international + most Dutch stations, failed: DE602,DE604,FR606,SE607 (all flagged) other int. stations fine

NDPPP-averaged by 16 to 4 channels with flagging (aoflagger.pulsar=true) → ndppp (244×12 GB = 2.8 TB) [autoweight=true]

for imaging: averaged further down (no additional flagging etc.) to 1 channel and 4 sec → /widemap/raid2/LOFAR_archive2/L2011_35012/ndppp_1chan_4sec/ (244×1.4GB=339GB total)

for pulsar polarisation and giant pulses: from ndppp output (4 channels, 1 sec) selected RS and good international stations (DE601HBA DE603HBA DE605HBA RS106HBA RS205HBA RS208HBA RS306HBA RS307HBA RS406HBA RS503HBA RS508HBA RS509HBA UK608HBA) → /widemap/raid2/LOFAR_archive2/L2011_35012/ndppp_select (244×880 MB=210 GB total)

Jupiter 24/25 Nov 2011 23:30-1:45 UTC

visibilities (1sec) and raw voltages

antennas DE602LBA,FR606LBA,RS106LBA,RS307LBA,SE607LBA,UK608LBA

61 subbands, 256 channels per subband

subbands 93..153 (18.16-29.88 MHz)

MS files (SB53 incomplete): 1.4GB each, 83 GB total → /widemap/raid2/LOFAR_archive2/L2011_35624/MS/

raw voltage files → /widemap/raid2/LOFAR_archive2/L2011_35624/RAW (only antennas 0-2 plus antenna 5 pol y (B005_S1), because this seems to be the same as B002_S1, but the latter is incomplete)

raw 6x2x727 GB, total 8.6 TB (note the redundancy)

The first three antennas seen to be repeated in the last three(x2) datasets in the raw files!

location in Juelich The h5 files are missing in Juelich. Only station 0-2 in Juelich. (See above!)

Jupiter 11 Dec 2011 16:30-18:30 UTC, LBA

visibilities (1sec) and raw voltages

antennas DE602LBA, DE604LBA, DE605LBA, FR606LBA, SE607LBA, UK608LBA

61 subbands, 64 channels per subband

subbands 93..153 (18.16-29.88 MHz)

MS files (SB059 empty, 046 and 057 missing): 319 MB each, 19 GB total → /widemap/raid2/LOFAR_archive2/L2011_37058/MS/

raw voltage files not in Bonn yet

raw each 640 GB, should be 2×6, but only 9, missing: B003_S1 (FR606 Y), B005 both (UK608), total 5.7 T

location in Juelich (only stations 0-2)

TauA LBA 10-58 MHz 10h on 11/12 Dec 2011 18:50-4:50 UTC

256 channels/SB

244×61 GB = 15 TB total

all international + some (mostly RS) Dutch stations, 20 stations total, checked weights in SB000,082,164: DE603 all bad, DE604 0.8-1, DE601+605 low in SB164

CS002LBA, CS026LBA, CS501LBA, DE601LBA, DE602LBA, DE603LBA, DE604LBA, DE605LBA, FR606LBA, RS106LBA, RS205LBA, RS208LBA, RS306LBA, RS307LBA, RS406LBA, RS503LBA, RS508LBA, RS509LBA, SE607LBA, UK608LBA

first NDPPP test: average to 4 sec, to 8 channels (by 32) → ndppp_8chan_4sec (244x303MB=73GB) [autoweight=true]

→ /widemap/raid2/LOFAR_archive2/L2011_37065/ndppp_8chan_4sec/

weight statistics of this version for SB??0: DE603 failed, DE604 low weights, DE601+DE605 low weights in higher subbands

Jupiter 13 Jan 15:30-16:15 UTC, LBA

visibilities (1sec) and raw voltages

antennas DE601,602,603,604,FR606,SE607

All DE stations have problems, high (up to 100% flagged) fraction, DE603 has on average 5% good samples, other DE almost 0. See /widemap/raid2/LOFAR_archive2/L2012_42482/MS/weight_statistics/SB000/L42482_SB000_*. Only stations without problems (but see below): FR606 and SE607 (files B004 and B005).

61 subbands, 64 channels per subband

subbands 76…136 (14.84-25.56 MHz)

MS files (SB010 too short, probably broken): 120 MB each, 7.2 GB total → /widemap/raid2/LOFAR_archive2/L2012_42482/MS/

raw voltage files not in Bonn yet

raw each 240 GB, 2×6 files, but B005_S1 only 209 GB, total 2.8 T

problem of duplicated data apparently gone

raw data not analysed yet

ASTRON will delete all raw files B000 to B003 and only keep B004 and B005. B004 and B005 will be copied to Juelich together with the MS files.

HBA low, 1h

[CS001HBA0,CS002HBA0,CS003HBA0,CS004HBA0,CS005HBA0,CS006HBA0,CS007HBA0,CS013HBA0,CS021HBA0,CS024HBA0,CS026HBA0,CS028HBA0,CS030HBA0,CS031HBA0,CS032HBA0,CS101HBA0,CS103HBA0,CS201HBA0,CS301HBA0,CS302HBA0,CS401HBA0,CS501HBA0,DE602HBA,DE603HBA,DE604HBA,DE605HBA,FR606HBA,RS106HBA,RS205HBA,RS208HBA,RS307HBA,RS406HBA,RS503HBA,RS509HBA,SE607HBA,UK608HBA]

checked subbands 001,050,099,148,197: CS501 entirely flagged, DE602,DE603,DE605,DE606,SE607,UK608 flagged in lower subbands (001,050)

Observation.startTime = '2012-02-08 19:25:00'
Observation.stopTime = '2012-02-08 20:25:00'

244×4.9GB=1.2TB total

location in Juelich

NDPPPed version, flagged, averaged to 4 channels and 4 sec → L45591_ndppp_4chan_4sec (244x152MB=37GB total)

Bonn: /widemap/raid2/LOFAR_archive2/L2012_45591/L45591_ndppp_4chan_4sec

LBA outer, 1h

[CS001LBA,CS002LBA,CS003LBA,CS004LBA,CS005LBA,CS006LBA,CS007LBA,CS013LBA,CS021LBA,CS024LBA,CS026LBA,CS028LBA,CS030LBA,CS031LBA,CS032LBA,CS101LBA,CS103LBA,CS201LBA,CS301LBA,CS302LBA,CS401LBA,DE602LBA,DE603LBA,DE604LBA,DE605LBA,FR606LBA,RS106LBA,RS205LBA,RS208LBA,RS307LBA,RS406LBA,RS503LBA,RS509LBA,SE607LBA,UK608LBA]

checked subbands 001,031,061,091,121,151,181,211,241: all have weights near 1.0

244×4.6GB, total 1.1 TB

NDPPP with flagging to 4 channels and 4 sec → /cep2/locus*_data/scratch/wucknitz/L45744_ndppp_4chan_4sec/L*.MS [autoweight=false] (35 GB)

NDPPP with flagging to 4 sec (left at 64 channels) → /cep2/locus*_data/scratch/wucknitz/L45744_ndppp_64chan_4sec/L*.MS [autoweight=true] (244×1.7GB = 409 GB)

NDPPP (4/4) version → /widemap/raid0/LOFAR_archive/L2012_45744/L45744_ndppp_4chan_4sec

NDPPP (64/4) version → /widemap/raid0/LOfAR_archive/L2012_45744/L45744_ndppp_64chan_4sec

Observation.startTime = '2012-02-09 19:20:00'
Observation.stopTime = '2012-02-09 20:20:00'

location in Juelich

HBA low dual, 1.5h

[CS002HBA0,CS002HBA1,CS003HBA0,CS003HBA1,CS004HBA0,CS004HBA1,CS005HBA0,CS005HBA1,CS006HBA0,CS006HBA1,CS007HBA0,CS007HBA1,CS021HBA0,CS021HBA1,CS024HBA0,CS024HBA1,CS026HBA0,CS026HBA1,CS032HBA0,CS032HBA1,CS101HBA0,CS101HBA1,CS103HBA0,CS103HBA1,CS201HBA0,CS201HBA1,CS302HBA0,CS302HBA1,CS501HBA0,CS501HBA1,DE601HBA,DE602HBA,DE603HBA,DE604HBA,DE605HBA,FR606HBA,RS106HBA,RS205HBA,RS208HBA,RS306HBA,RS307HBA,RS406HBA,RS503HBA,RS508HBA,RS509HBA,SE607HBA,UK608HBA]

checked subbands SB000,001,043,070,083,084,126,153,166,167,209,236: all have weights near 1.0

Observation.startTime = '2012-02-10 19:15:00'
Observation.stopTime = '2012-02-10 20:45:00'

244x13GB, total 3.0 TB

NDPPP with flagging to 4 channels and 4 sec → /cep2/locus*_data/scratch/wucknitz/L45786_ndppp_4chan_4sec/L*.MS (91 GB)

NDPPP version → /widemap/raid0/LOFAR_archive/L2012_45786/L45786_ndppp_4chan_4sec [first was autoweight=false, but updated on 5th Apr 2012, now autoweight=true]

another copy on cep1: lse023:/data2/scratch/L45786_ndppp_4chan_4sec [autoweight=false]

some raw subbands on cep1: lse023:/data2/scratch/L45786_some_raw/

location in Juelich

diagnostic plots

Combined versions at /widemap/raid0/LOFAR_archive/L2012_45786/:

L45786_ndppp_4chan_4sec_combined.ms (all subbands, 79GB),

L45786_ndppp_4chan_4sec_combined_SB110-125.ms (16 subbands, 5.2GB)

versions with phased-up superterp (with phase+delay correction, now also with autocorrs and proper weight correction):

L45786_ndppp_4chan_4sec_combined_phased-up_fullcorr.ms (all subbands, 47GB),

L45786_ndppp_4chan_4sec_combined_SB110-125_phased-up_fullcorr.ms (16 subbands, 3.1GB)

For superterp delay/phase offsets, see diagnostic plots.

[ Circular versions and several tests of X-Y offsets in ~/astro_wm/lofar_obs_wm/L2012_45786/X-Y-offsets ]

3C196 with LBA, all stations

13h 6/7 March 2012

244x74GB total 18TB

checked SB000 SB061 SB122 SB183: no flagging, all 8 IS there

Observation.startTime = '2012-03-06 14:30:00'
Observation.stopTime = '2012-03-07 03:30:00'

widemap: /widemap/raid0/LOFAR_archive/L2012_50631/

L50631_ndppp_4chan_4sec: 244×2.3GB=539GB → /widemap/raid0/LOFAR_archive/L2012_50631/L50631_ndppp_4chan_4sec

Analysis of weights from autocorrelations

pre-processed data in Bonn under /widemap/raid0/LOFAR_archive/L2012_50631/ :

L50631_ndppp_4chan_4sec: individual subbands flagged and averaged to 4 channels and 4 sec (244×2.3GB=539GB)
L50631_ndppp_4chan_4sec_combined/L50631_all.ms: all subbands combined (469GB)
L50631_ndppp_4chan_4sec_combined/L50631_SB223-232.ms: 16 subbands combined (20GB)
L50631_ndppp_4chan_4sec_phased-up/L50631_SB223-232_phased-up.ms: those SBs, superterp phased up (with corrections, 15GB)
L50631_ndppp_4chan_4sec_phased-up_circular/L50631_SB223-232_phased-up_circular_nocorr.ms: converted to circular (no X-Y offset correction)
L50631_ndppp_4chan_4sec_phased-up_circular/L50631_SB223-232_phased-up_circular_xycorr_blsel.ms: converted to circular with (not perfect) X-Y offset correction

[ lots of info and diagnostic plots in ~wucknitz/astro_wm/lofar_obs_wm/L2012_50631 ]

Observation.startTime = '2012-04-15 13:03:00'
Observation.stopTime = '2012-04-15 14:33:00'

1.5h with “all” stations except for DE604 and SE607

244 x 15 GB, total 3.5TB

NDPPPed version (16 channels, 4 sec): 244 x 445 MB, total 106 GB

→ /widemap/raid0/LOFAR_archive/L2012_57168/NDPPPed

additional versions in /widemap/raid0/LOFAR_archive/L2012_57168:
L57168_ndppp_combined_SB110-125.MS (6.1 GB),
L57168_ndppp_combined_SB110-125_phased-up.MS (3.8 GB),
L57168_ndppp_combined_all.MS (92 GB),
L57168_ndppp_combined_all_phased-up.MS (57 GB)

not in LTA yet

Note: The phase centre for this observation was 03:33:59.00000 +54.34.43.0000, but the real position of the pulsar (J2000) is 03:32:59.368 54:34:43.57. This offset introduces up to 5-8 microsec delay differences on international baselines.

Corrected versions will be provided soonish!

→ NDPPP_posshift/ on widemap

This version was first shifted and then flagged. It turns out that it still has unflagged bad data in it. The reason for this is currently being investigated. At least for some of the bad points, the unshifted version had them flagged, but the new one not.

The reason for the flagging problem was a different setup of the aoflagger.overlapperc and aoflagger.timewindow parameters.

done: new NDPPP run with better flagging and with phase shift → NDPPP_posshift_new

deleted old versions, including combined versions

: redo the combining and phasing up and then continue with XY offsets and conversion to circular

TauA LBA 1h

Observation.startTime = '2012-03-07 16:40:00'
Observation.stopTime = '2012-03-07 17:40:00'

244 x 6.0 GB = 1.5 TB

NDPPPed version (16 channels, 4 sec)

→ /widemap/raid0/LOFAR_archive/L2012_50853/L50853_ndppp_4chan_4sec

244 x 186 MB = 45 GB

combined subbands:

/widemap/raid0/LOFAR_archive/L2012_50853/L50853_ndppp_4chan_4sec_SB000-255.MS (40 GB)
/widemap/raid0/LOFAR_archive/L2012_50853/L50853_ndppp_4chan_4sec_SB200-215.MS (2.6 GB)

superterp calibration (plots are still problematic for last flagged channels), X-Y offsets, conversion to circular

Observation.startTime = '2012-04-15 14:45:00'
Observation.stopTime = '2012-04-15 16:15:00'

1.5h with “all” stations except for DE604 and SE607

244 x 15 GB, total 3.5TB

NDPPPed version (16 channels, 4 sec): 244 x 445 MB, total 106 GB

→ /widemap/raid0/LOFAR_archive/L2012_57169/NDPPPed

additional versions in /widemap/raid0/LOFAR_archive/L2012_57169: L57169_ndppp_combined_SB110-125.MS (6.1 GB), L57169_ndppp_combined_SB110-125_phased-up.MS (3.8 GB), L57169_ndppp_combined_SB110-125_phased-up_iau_rl_xycorr.MS (3.8 GB), L57169_ndppp_combined_all.MS (92 GB), L57169_ndppp_combined_all_phased-up.MS (57 GB)

not in LTA yet

Observation.startTime = '2012-05-06 13:55:00'
Observation.stopTime = '2012-05-06 14:25:00'

244 x 4.2 GB = 990 GB total, some are incomplete and cannot be processed

NDPPPed version: to 4 channels, 4 sec, 241 x 127 MB = 30 GB total

→ /widemap/raid0/LOFAR_archive/L2012_57704/L57704

[CS002HBA0 CS002HBA1 CS003HBA0 CS003HBA1 CS004HBA0 CS004HBA1 CS005HBA0 CS005HBA1 CS006HBA0 CS006HBA1 CS007HBA0 CS007HBA1 CS017HBA0 CS017HBA1 CS021HBA0 CS021HBA1 CS024HBA0 CS024HBA1 CS026HBA0 CS026HBA1 CS032HBA0 CS032HBA1 CS101HBA0 CS101HBA1 CS103HBA0 CS103HBA1 CS201HBA0 CS201HBA1 CS302HBA0 CS302HBA1 CS501HBA0 CS501HBA1 DE601HBA DE602HBA DE603HBA DE604HBA DE605HBA FR606HBA RS106HBA RS205HBA RS208HBA RS307HBA RS406HBA RS503HBA RS508HBA RS509HBA UK608HBA]

: add more details.

Still working on this!

raw voltages in fly's eye mode, incoherent sum, correlations

5min test obs

voltages have about 5% fraction flagged, visibilities and incoherent sum not

Weights (at least of autocorrelations) in visibilities are funny. See ~/astro_wm/lofar_obs_wm/L2012_60895/

Short test obs (George Heald) to test phasing up the superterp and determining the offsets between superterp stations.

80×75 MB = 5.9 GB

→ /widemap/raid0/LOFAR_archive/L2012_62844/

diagnostic plots with superterp offsets etc

18h track in 2h chunks, superterp plus RS plus international (+ others?)

DE* failed for first bits

raw and NDPPPed version is in Juelich

Date: Tue, 30 Oct 2012 10:53:39 +0100
From: Wilfred Frieswijk frieswijk [at] astron [dot] nl

The observation was scheduled for 12 hours around transit using all available Dutch stations (16CS+9RS) and all international stations. Specification summary: HBA dual 110-190MHz 1s integration time 64ch/sb subbands 77..320

Observation ID: L71716

Start time 2012-10-27 20:39:39 UTC End time 2012-10-28 08:39:39 UTC

The correlator recorded 12 h of data in all 244 sub bands.

CS302HBA0 and RS208 showed significant flagging possibly caused by (internal) interference. CS021 and CS103 have very low sensitivity. CS024, CS101, CS302 and CS501 have low sensitivity.

For more details, the summary plots can also be found on: https://proxy.lofar.eu/inspect/HTML/71716/index.html

Data recording: No issues.

Data processing: The data has been pre-processed using settings provided by the PI.

Archiving: The raw and processed data will be transferred directly to Juelich by Richard Fallows.

MoM links:

1 2 3

LTA link in MoM

Bonn: NDPPPed at /widemap/raid2/LOFAR_archive2/L2012_71716/NDPPP

detailed report