WERA HF Radar Software - .USORT Data to Gridded Spectra

wera16_grid_ol

An optional parameter can be used to pass the name of a configuration file. This file must be located in ~/etc e.g. ~/etc/params_Hawaii.cfg . If no optional parameter is supplied, the configuration is read from ~/etc/params.cfg .

Behind the name of the configuration file, another single character, e.g. "1" or "A", can be passed to modify the name of internally generated files (gain normalization file and next-file message for postprocessing) in case of simultaneous use of wera16_grid_ol with different configuration files.

As of V1.1.0 (June 2009), this program checks the signal received from the direct path to compensate for Chirp Start variations.

wera16_grid_ol reads a series of coherent .USORT files (128 samples each) and combines them to form sub-spectra from NSER_AVG samples [yellow]. The parameters REPROC, UNAM_nnn, and CALF_nnn in params*.cfg define file names to control this process. Once the coherent series of .USORT files is interrupted, the process to combine files is restarted.

In case of reprocessing (REPROC > 0), a script may write the next .USORT file name to the file name given in UNAM_nnn once the file ~/etc/Next_File_CFAR.txt exists, remove ~/etc/Next_File_CFAR.txt, and wait for the next appearence of ~/etc/Next_File_CFAR[_X].txt . wera16_grid_ol makes this file when processing of the current .USORT file is completed.

It is essential to pass the correct calibration file by the file name defined in CALF_nnn to wera16_grid_ol. The range-dependent calibration (file name passed by FCAL_nnn) is applied before beam forming if the file exists.

Gain normalization between antenna channels is done by summing up the Bragg energy for all ranges at each antenna. The antenna gain is then calculated by leveling the power of all antennas to the same value. Until a total of 2048 samples is processed, a value of 1.0 is used to avoid gain errors due to insufficient statistics. If at least 2048 samples have been processed, the new gain normalization is calculated by last_gain * (1-alpha) + actual_gain * alpha, where the value of alpha is 0.0625 . After an interruption of 10 minutes, the gain-filter is re-initialized. Gain normalization values are stored in a file defined by GADJ_nnn when processing a series of .USORT files for the first time. The gain normalization values are stored in gain normalization files named yyyydddhhmmss_nnnnn_code[_X].gain (nnnnn = NSER_AVG) to enable re-processing of specific time segments without re-starting the gain-filtering process: If during re-processing these files are found, the gain normalization is red from these files. Gain normalization files are kept in the same directory as .USORT and .RFI files.

If CUR_SPEC is defined to .TRUE., these sub-spectra [yellow] are averaged until NCUR_AVG samples are covered to form the gridded spectra file yyyydddhhmmss_cod.CUR.SPEC [green]. These files are written at time increments of NCUR_STEP samples.

If WAV_SPEC is defined to .TRUE., these sub-spectra [yellow] are averaged until NWAV_AVG samples are covered to form the gridded spectra file yyyydddhhmmss_cod.WAV.SPEC [red]. These files are written at time increments of NWAV_STEP samples.

NSER_AVG, NCUR_AVG, NCUR_STEP, NWAV_AVG, NWAV_STEP must be multipes of 128 and are defined in params*.cfg.

This program calculates the power spectra at the grid cells defined in GRID_NAME using the range and azimuth from the WERA site to the grid points given in WGS84 coordinates.

If SPEC2 is defined to .TRUE., the compressed .SPEC data format is used to store the gridded spectra, if it is defined to .FALSE., the old uncompressed .SPEC data format is used.

If RFI_ENA is defined to .FALSE., the RFI processing is disabled, i.e. the program does not use the .RFI / .URFI files even if they exist.

Reads all information needed on number of channels, length of timeseries, etc. from the header (first 512 bytes of the .sort file) and the following values from PARAM_UV*.CFG (VMS) / params*.cfg (Linux):

Values in params.cfg: C------------------------------------------------------------------------------- C Enable / disable output file formats C------------------------------------------------------------------------------- SPEC2 = .TRUE. !.true. = write compressed .SPEC files c SPEC2 = .FALSE. !.true. = write compressed .SPEC files C------------------------------------------------------------------------------- C Enable / disable RFI processing C------------------------------------------------------------------------------- RFI_ENA = .TRUE. ! enable processing of .RFI files c RFI_ENA = .FALSE. ! disable processing of .RFI files C------------------------------------------------------------------------------- C Enable .SPEC files with different integration times C------------------------------------------------------------------------------- CUR_SPEC = .TRUE. !.true. = write .CUR.SPEC file c CUR_SPEC = .FALSE. !.true. = write .CUR.SPEC file WAV_SPEC = .TRUE. !.true. = write .WAV.SPEC file c WAV_SPEC = .FALSE. !.true. = write .WAV.SPEC file C------------------------------------------------------------------------------- C Define integration times C------------------------------------------------------------------------------- ! The following parameters must be multiples of 128 NSER_AVG = 512 ! Length of combined .USORT fractions NCUR_AVG = 1024 ! Integration time for current measurements, minimum 512 NWAV_AVG = 4096 ! Integration time for wave measurements, minimum 2048 C------------------------------------------------------------------------------- C Result file time steps within a coherent .USORT stream C------------------------------------------------------------------------------- ! The following parameters must be multiples of 128 NCUR_STEP = 512 ! Samples between .CUR.SPEC / .crad result files NWAV_STEP = 512 ! Samples between .WAV.SPEC / .wrad result files C------------------------------------------------------------------------------- C Number of WERA sites (radial files) C------------------------------------------------------------------------------- NST = 2 !Number of WERA sites (radial files) c REPROC = 0 !0 -> use set of file names given by CL7 realtime software c REPROC = 1 !Buesum: Site number for reprocessing .USORT data REPROC = 2 !Sylt: Site number for reprocessing .USORT data C------------------------------------------------------------------------------- C For each of the 1 to NST sites, provide SITE_nnn etc. C SITE_001 to SITE_012 are currently supported. C******************************************************************************* SITE_001 = 'Buesum' !Site name (max 11 chars) LEFT_001 = .FALSE. !Plot name to the left TLNG_001 = 17.0 !Length of name in map [mm] LATD_001 = 54.0 !LAT degrees LATM_001 = 7.18 !LAT minutes LOND_001 = 08.0 !LON degrees LONM_001 = 51.47 !LON minutes T_NO_001 = 349.0 !True_North of RX antenna RAN1_001 = 80.0 !Inner range 30 MHz RAN2_001 = 130.0 !Outer range 30 MHz !The following 8 parameters allow simultaneous use of a linear and !a square array within the same WERA system. They provide a mapping !of receiver channels to antenna numbers. A value of '0' indicates that !this antenna configuration does not exist in the actual WERA system. ! ARR1_001 = 1 !First antenna# in linear array ARR2_001 = 12 !Last antenna# in linear array ARRD_001 = 13.5 !Distance between elements [meters] STEER_001 = 60 !+/- max steering angle [degrees] SQA1_001 = 0 !Antenna 1 of square array SQA2_001 = 0 !Antenna 2 of square array SQA3_001 = 0 !Antenna 3 of square array SQA4_001 = 0 !Antenna 4 of square array !Name of file for FullCalibration C FCAL_001 = '$user1:[wera.etc]buesum.fullcal' !Name of file for linear array spacing c DIST_001 = '$user1:[wera.etc]antdist_Buesum.asc' APOS_001 = '$user1:[wera.etc]antpos_Buesum.asc' AWIN_001 = 'u' !Ultra Spherical c AWIN_001 = 'h' !Hamming !Direction finding geographical angle clockwise ANG1_001 = 0.0 !Direction finding from degrees ANG2_001 = 0.0 !Direction finding to degrees !*** DEFINE THESE PARAMETERS WHEN REPROCESSING ON-LINE DATA ! and select the correct set of filenames by defining "REPROC" ! ! calibration file CALF_001 = '$user1:[wera.etc]calibration_Buesum.wera' ! ! actual .USORT is in UNAM_001 = '$user1:[wera.etc]USORTnames_Buesum.asc' ! C****************************************************************************** SITE_002 = '.....' !Site name (max 11 chars) <----- snip ----> C------------------------------------------------------------------------------- C Name and size (NX, NY) of Grid File C Values used to generate AREA_grid_NX_NY.asc : C Spacing [km] 2.0 C Upper left corner: LAT 52 8.5 LON 3 40.0 C------------------------------------------------------------------------------- NX = 85 NY = 110 GRID_NAME = '$user1:[wera.etc]oms_grid.asc' C Name of WERA Gain File C GAIN_NAME = '/home/wera/etc/wera_gain.dat'

The .SPEC files are also used by Lucy Wyatt's wave algorithm.
These binary files can be transfered between platforms, e.g. Linux <--> VMS by FTP in binary mode without breaking the file structrure.

Program expects  'FMUNSO'  header code
and writes       'FMSPEC' or 'FMSPC2' header code.

wera16_grid_ol also adds information on the measurement grid, the name of the grid file, and the averaging process
to the header [uncompressed] (example from Hawaii):

NX=100 NY=140         --> Size of grid
NTL=  13              --> Number of sub-spectra averaged
NFTL= 512             --> Number of lines in the spectrum (= NSER_AVG)
Oahu_grid_100_140.asc --> Name of grid-file used
LAT=  21.866659       --> LAT(1,1), upper left corner of the grid
LON=-159.049942       --> LON(1,1), upper left corner of the grid
DGT=   1.000000       --> Grid spacing in [km]

NX= 85 NY=110         --> Size of grid
NTL=   5              --> Number of sub-spectra averaged
NFTL= 512             --> Number of lines in the spectrum (= NSER_AVG)
SS 0.000000           --> Seconds of start of acquisition
NCOV       4920       --> Number of covered grid points
LAT=  55.561667       --> LAT(1,1), upper left corner of the grid
LON=   6.216667       --> LON(1,1), upper left corner of the grid
DGT=   2.000000       --> Grid spacing in [km]

The following box shows a typical output generated by wera16_grid_ol :

wera16_grid_ol V1.0.0 OpenVMS: *** Reading configuration from $user1:[Wera.etc]param_uv_oms.cfg *** Number of sub-spectra for current processing: 5 *** Number of sub-spectra for wave processing: 29 *** Reading .USORT file names from $user1:[wera.etc]USORTnames_Sylt.asc *** Now processing $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008-03-077-13-30-00_syl.USORT nant_sort = 12 antennas Frequency = 10.8900003 MHz NSER = 128 Samples T_chirp = 0.260000 s F_bragg = 0.336677 Hz Last RC = 100 range cells RANGEresol = 1.5000 km RAN_OFF = 0.3000 RC RAN_OFF = 450 m TRUE_NORTH = 5 degrees DDS_Clock = 181.48430769 MHz MT = 1536 samples SSR = 0.000000000000 seconds *** Header: 0128 SAMPLES 17-MAR-08 13:30 UTC Bunker Hill FMUNSO FREQUENZ 10.890MHZ *** Reading antenna locations from $user1:[wera.etc]antpos_sylt.asc *** RFI flags: T *** Combined: NSER_in_COMB = 128 NANT_SORT = 12 NRAN = 100 RFI_OK = T *** Reading .USORT file names from $user1:[wera.etc]USORTnames_Sylt.asc *** Now processing $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008-03-077-13-30-33_syl.USORT nant_sort = 12 antennas ............ SSR = 33.279999000000 seconds *** Header: 0128 SAMPLES 17-MAR-08 13:30 UTC Bunker Hill FMUNSO FREQUENZ 10.890MHZ *** Reading antenna locations from $user1:[wera.etc]antpos_sylt.asc *** RFI flags: T T *** Combined: NSER_in_COMB = 256 NANT_SORT = 12 NRAN = 100 RFI_OK = T *** Reading .USORT file names from $user1:[wera.etc]USORTnames_Sylt.asc *** Now processing $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008-03-077-13-31-06_syl.USORT nant_sort = 12 antennas ............ SSR = 6.559998000000 seconds *** Header: 0128 SAMPLES 17-MAR-08 13:31 UTC Bunker Hill FMUNSO FREQUENZ 10.890MHZ *** Reading antenna locations from $user1:[wera.etc]antpos_sylt.asc *** RFI flags: T T T *** Combined: NSER_in_COMB = 384 NANT_SORT = 12 NRAN = 100 RFI_OK = T *** Reading .USORT file names from $user1:[wera.etc]USORTnames_Sylt.asc *** Now processing $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008-03-077-13-31-39_syl.USORT nant_sort = 12 antennas ............ SSR = 39.839996000000 seconds *** Header: 0128 SAMPLES 17-MAR-08 13:31 UTC Bunker Hill FMUNSO FREQUENZ 10.890MHZ *** Reading antenna locations from $user1:[wera.etc]antpos_sylt.asc *** RFI flags: T T T T *** Combined: NSER_in_COMB = 512 NANT_SORT = 12 NRAN = 100 RFI_OK = T *** Combine finished: 512 samples, SAMP_CNT = 512 *** INIT2 - Reading grid coordinates ....: $user1:[wera.etc]oms_grid.asc *** INIT2 - Calculating distance and angle from the radar site to grid points in WGS84 coordinates. *** INIT2 - Number of covered grid points = 4920 *** INIT2 - Calculating complex window function for beam forming. *** INIT2 - FullCalib does not exist ....: $user1:[wera.etc]Site.fullcal *** INIT2 - Reading WERA_Gain ...........: $user1:[wera.etc]wera_gain.dat *** Reading Gain Normalization: $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008077133000_00512_syl.gain *** Processing with RFI information. *** Detecting interference and ships, forming beams to grid points. *** Computing time = 31.832 seconds. *** Reading .USORT file names from $user1:[wera.etc]USORTnames_Sylt.asc *** Now processing $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008-03-077-13-32-13_syl.USORT nant_sort = 12 antennas ............ SSR = 13.119995000000 seconds *** Header: 0128 SAMPLES 17-MAR-08 13:32 UTC Bunker Hill FMUNSO FREQUENZ 10.890MHZ *** Reading antenna locations from $user1:[wera.etc]antpos_sylt.asc *** Shifting by 128 samples. *** RFI flags: T T T T *** Combined: NSER_in_COMB = 512 NANT_SORT = 12 NRAN = 100 RFI_OK = T *** Combine finished: 512 samples, SAMP_CNT = 640 *** Reading Gain Normalization: $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008077133033_00512_syl.gain *** Processing with RFI information. *** Detecting interference and ships, forming beams to grid points. *** Computing time = 29.930 seconds. *** Reading .USORT file names from $user1:[wera.etc]USORTnames_Sylt.asc *** Now processing $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008-03-077-13-32-46_syl.USORT nant_sort = 12 antennas ............ SSR = 46.399994000000 seconds *** Header: 0128 SAMPLES 17-MAR-08 13:32 UTC Bunker Hill FMUNSO FREQUENZ 10.890MHZ *** Reading antenna locations from $user1:[wera.etc]antpos_sylt.asc *** Shifting by 128 samples. *** RFI flags: T T T T *** Combined: NSER_in_COMB = 512 NANT_SORT = 12 NRAN = 100 RFI_OK = T *** Combine finished: 512 samples, SAMP_CNT = 768 *** Reading Gain Normalization: $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008077133106_00512_syl.gain *** Processing with RFI information. *** Detecting interference and ships, forming beams to grid points. *** Computing time = 29.801 seconds. *** Reading .USORT file names from $user1:[wera.etc]USORTnames_Sylt.asc *** Now processing $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008-03-077-13-33-19_syl.USORT nant_sort = 12 antennas ............ ............ SSR = 52.959991000000 seconds *** Header: 0128 SAMPLES 17-MAR-08 13:33 UTC Bunker Hill FMUNSO FREQUENZ 10.890MHZ *** Reading antenna locations from $user1:[wera.etc]antpos_sylt.asc *** Shifting by 128 samples. *** RFI flags: T T T T *** Combined: NSER_in_COMB = 512 NANT_SORT = 12 NRAN = 100 RFI_OK = T *** Combine finished: 512 samples, SAMP_CNT = 1024 *** Reading Gain Normalization: $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008077133213_00512_syl.gain *** Processing with RFI information. *** Detecting interference and ships, forming beams to grid points. *** Writing .SPEC file for currents: $DATA3:[Nordsee.Sylt.USORT.2008077.data]2008077133000_syl_CUR.SPEC *** Computing time = 43.160 seconds. ............

Here are the data formats of the input (.USORT) , uncompressed (.SPEC) , and compressed (.SPEC) files.

Compilation from source:

Linux: Source is in wera16_grid_ol.F Compilation and linking with './do_wera16_grid_ol.sh' Call with > wera16_grid_ol [params*.cfg] e.g. > wera16_grid params_Gijon.cfg

VMS: Source is in wera16_grid_ol.F Compilation and linking with @wera16_grid_ol Program needs to be defined as an external command to take parameters from the command line, e.g. $ WERA16_GRID_OL :== $$USER1:[WERA.Fortran]WERA16_GRID_OL.EXE Call with $ wera16_grid Param_uv*.cfg $ WERA16_GRID Param_uv_Gijon.cfg