Processing Notes

From OPS
Jump to: navigation, search

Contents

2002 Greenland P3

2002 Greenland P3: Accumulation Radar

The old format files have the following sampling rate. Note that the time-offset, t0, is unknown. The time offset, t0, should be something small on the order of tens of nanoseconds to account for the system time delay. The system time delay is the time for the signal to propagate through the RF transmitter, antenna feed cables, and RF receiver. For layer tracing, the time offset is usually not important.

The GPS is probably a low resolution GPS with 1-2 m accuracy at best. ATM lidar data is probably available for most datasets and can help provide absolute range if needed.

data and filtfin are the image matrices. The image processing steps are not documented, but they include coherent noise removal, coherent averaging, incoherent averaging, and are detrended log scale plots. There is virtually no radiometric information left and these images can probably only be used for layer tracking.


K>> sr % sampling rate
sr =
    50000000
K>> bw % bandwidth
bw =
     3.000022843480110e+08
K>> fm % chirp rate
fm =
     3.000022843480110e+12
K>> dr = sr * c/(2*nfft*fm)
  dt = sr/(nfft*fm)
  dt*c/2
dr =
   0.304962336476688
dt =
     2.034489716724874e-09
ans =
   0.304962336476688

Nt = size(data,2); or Nt = size(filtfin,2);
dt = 2.034489716724874e-09;
t0 = 0; % Unknown so set to zero
Time = t0 + dt*(0:Nt-1).';

2012 Greenland P3

2012 Greenland P3: Snow Radar

The radar included an IF filter bank which was switched on the fly during recording. The state of the IF filter was not recorded in notes. DESCRIBE PROCESS FOR ESTIMATING HERE.

  1. run_collate_est_nz.m (& collate_est_nz.m): Normalized cross correlation between the blocks of coherent noise in a day_segment is used to classify the day_segment into 'n' sets corresponding to 'n' different hardware nyquist zones. Additional check from detecting disconnects based on the max values from run_analysis for statistics. Inputs: Uses the coh_noise_*.mat files generated by run_collate_coh_noise.m and stats_*.mat files generated by run_analysis.m for max. Outputs: jpg, fig and mat files in ct_tmp\collate_est_nz\snow\2012_Greenland_P3\
  2. run_collate_est_nz_tables.m (& collate_est_nz_tables.m): Collates the results from run_collate_est_nz.m to create a ldap and then a table of unique coherent noise waveforms from entire season. Human input is required to generate the master list of hardware nyquist zones. This is copied to collate_est_nz sheet in params spreadsheet.
  3. run_collate_est_nz_records.m (& collate_est_nz_records.m): Uses the master list to update records by adding settings.nyquist_zone_hw and settings.records_mask (true for good records). Outputs: updated records and files in ct_tmp\collate_est_nz_table\snow\2012_Greenland_P3\ to cross check hardware nyquist zone with the siznal nyquist zone.
run_qlook settings inside for loop in automated section:
params(param_idx).radar.wfs(1).coh_noise_method = 'analysis';

2013 Greenland P3

2013 Greenland P3: Snow Radar

A DDC was introduced which allowed the data rate to be reduced by subbanding the data before storage. The downside is that the coherent noise is harder to track and remove because it is not sampled as well.

Timing/clock error in the digital system causes an along-track modulation of the signal.

2013 Antarctica Basler

EMI that was about 13+/-3 dB higher than thermal noise was present on every channel until fixed on Dec 28.

Some of the XML files have errors for the attenuator values for ADCs 2 through 8. The attenuator values are the same for every ADC and ADC 1 is always correct.

2013 Antarctica Basler: RDS

2013 Antarctica Basler RDS: Coherent Noise

Two pass coherent noise removal is required:

The filter coefficients are:

param_override.collate_coh_noise.method = 'firdec';
param_override.collate_coh_noise.firdec_fs = 1/7.5;
param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/30;

The first pass of collate_coh_noise threshold settings are:

% Threshold: This combines a -105 dB threshold for waveform 3, threshold
% estimate+6dB guard, and a coherent noise mask for the feed through,
% since the feed through is not stable and deep nulls appear at certain
% range bins, a max_filt1 is used to fill in the nulls.
for param_idx = 1:length(params)
  param = params(param_idx);
  if ~isfield(param.cmd,'generic') || iscell(param.cmd.generic) || ischar(param.cmd.generic) || ~param.cmd.generic
    continue;
  end
  for wf = 1:length(params(param_idx).radar.wfs)
    if wf == 3
      nt = -105;
    elseif wf == 1
      nt = -120;
    end
    params(param_idx).collate_coh_noise.threshold_eval{wf} = 'threshold = max(min(nt,threshold+6),max_filt1(10*log10(abs(noise.dft(:,1)).^2)+15-1e6*(time>(Tpd+1.2e-6)),5));';
  end
end

2014 Greenland P3

2014 Greenland P3: RDS

The radar was run with three different waveforms for most of the season:

Waveform Description
1 Short (1 us) pulse duration and low gain to capture the surface and shallow layers
2 Medium (3 us) pulse duration and high gain to capture the deeper shallow layers
3 Long (10 us) pulse duration and high gain to capture deep layers and the ice bottom

For the Canadian Arctic Archipelagos flights, 20140325_0[567], 20140401_03, 20140414_02, 20140505_01, and 20140506_01, we used transmit beam steering:

Waveform Beam Direction Description
1 Left Medium (3 us) pulse duration and medium gain (15 dB attenuation) to capture from the surface to the bottom
2 Nadir Medium (3 us) pulse duration and medium gain (20 dB attenuation) to capture from the surface to the bottom. This mode is likely to be saturated for the nadir surface return over bright targets such as smooth liquid water (e.g. sea ice leads and polynyas).
3 Right Medium (3 us) pulse duration and medium gain (15 dB attenuation) to capture from the surface to the bottom


2014 Greenland P3: RDS: Coherent Noise

Coherent noise requires two passes with analysis. The first pass is used to estimate the threshold. The second pass is used to estimate the coherent noise.

For 20140325_0[567], 20140401_03, 20140414_02, 20140505_01, and 20140506_01, the following parameters worked for coherent noise removal.

% ENABLE ON SECOND PASS
param_override.collate_coh_noise.in_path = 'analysis_threshold';
param_override.collate_coh_noise.out_path = 'analysis_threshold';

% ENABLE ON FIRST AND SECOND PASSES
for param_idx = 1:length(params)
  param = params(param_idx);
  if ~isfield(param.cmd,'generic') || iscell(param.cmd.generic) || ischar(param.cmd.generic) || ~param.cmd.generic
    continue;
  end
  for img = 1:length(params(param_idx).radar.wfs)
    Tpd = params(param_idx).radar.wfs(wf).Tpd;
    if 1
      params(param_idx).collate_coh_noise.method{img} = 'firdec';
      params(param_idx).collate_coh_noise.firdec_fs{img} = 1/30;
      params(param_idx).collate_coh_noise.firdec_fcutoff{img} = @(t) (t<Tpd+1.2e-6)*1/120 + (t>=Tpd+1.2e-6)*-1;
    else
      params(param_idx).collate_coh_noise.method{img} = 'dft';
      params(param_idx).collate_coh_noise.dft_corr_time{img} = inf;
    end
    
    nt = -130;
    params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold = max(min(nt,threshold+6),max_filt1(10*log10(abs(noise.dft(:,1)).^2)+15-1e6*(time>(Tpd+1.2e-6)),5));';
  end
  for wf = 1:length(params(param_idx).radar.wfs)
    rx_paths = params(param_idx).radar.wfs(wf).rx_paths;
    params(param_idx).radar.wfs(wf).Tadc_adjust = -0.00000164;
    params(param_idx).radar.wfs(wf).chan_equal_deg = [-96.4 142.3 0 -146.9 173.2 41.2 148.2 -168 -136.2 174.3 -169.6 -48.2 -131.4 -119.2 -119.2];
    params(param_idx).radar.wfs(wf).chan_equal_dB = [2.9 1.3 0 -2.5 0.5 0.1 1.4 2.5 1.5 -0.3 2 2.8 1.8 1.1 2.2];
    params(param_idx).radar.wfs(wf).Tsys = [3.73 3.09 0 5.59 3.05 0.92 2.28 -19.24 -22.03 -26.23 -29.8 -28.74 -25.71 -22.8 -19.42]/1e9;
  end
end

2014 Greenland P3: RDS: Deconvolution

20140506_01 may have good 3 us waveforms for deconvolution in frame 3.

2014 Greenland P3: RDS: Equalization

The final equalization coefficients were pulled from:

20140325_07 rec:14000-18500 ref_wf_adc_idx:3 git-hash:11b06dd18a876efea7785a5928e00a94c4918bea (30-Aug-2019 20:19:02)														
rx 1	rx 2	rx 3	rx 4	rx 5	rx 6	rx 7	rx 8	rx 9	rx 10	rx 11	rx 12	rx 13	rx 14	rx 15
wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc	wf-adc
 2-  2	 2-  3	 2-  4	 2-  5	 2-  6	 2-  7	 2-  8	 2-  9	 2- 10	 2- 11	 2- 12	 2- 13	 2- 14	 2- 15	 2- 16
Offsets from Old Coefficients (rows: equal_dB, equal_deg, Tsys_ns)														
0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
0	0	0	0	0	0	0	-0.03	0.01	-0.04	0.02	-0.01	0	-0.04	-0.01
New Coefficients														
2.9	1.3	0	-2.5	0.5	0.1	1.4	2.5	1.5	-0.3	2	2.8	1.8	1.1	2.2
-96.4	142.3	0	-146.9	173.2	41.2	148.2	-168	-136.2	174.3	-169.6	-48.2	-131.4	-119.2	-119.2
3.73	3.09	0	5.59	3.05	0.92	2.28	-19.27	-22.02	-26.27	-29.78	-28.75	-25.71	-22.84	-19.43
New Coefficients if using the old Tsys (in spreadsheet format)														
[2.9 1.3 0 -2.5 0.5 0.1 1.4 2.5 1.5 -0.3 2 2.8 1.8 1.1 2.2]														
[-96.4 142.3 0 -146.9 173.2 41.2 148.2 -168 -136.2 174.3 -169.6 -48.2 -131.4 -119.2 -119.2]														
[3.73 3.09 0 5.59 3.05 0.92 2.28 -19.24 -22.03 -26.23 -29.8 -28.74 -25.71 -22.8 -19.42]/1e9														
New Coefficients if updating with this Tsys (in spreadsheet format)														
[2.9 1.3 0 -2.5 0.5 0.1 1.4 2.5 1.5 -0.3 2 2.8 1.8 1.1 2.2]														
[-96.4 142.3 0 -146.9 173.2 41.2 148.2 -168 -136.2 174.3 -169.6 -48.2 -131.4 -119.2 -119.2]														
[3.73 3.09 0 5.59 3.05 0.92 2.28 -19.27 -22.02 -26.27 -29.78 -28.75 -25.71 -22.84 -19.43]/1e9														

2014 Greenland P3: RDS: Processing

for param_idx = 1:length(params)
  param = params(param_idx);
  if ~param.cmd.qlook
    continue;
  end
  for wf = 1:length(params(param_idx).radar.wfs)
    params(param_idx).radar.wfs(wf).coh_noise_method = 'analysis';
    params(param_idx).radar.wfs(wf).coh_noise_arg.fn = 'analysis_threshold';
  end
  for wf = 1:length(params(param_idx).radar.wfs)
    rx_paths = params(param_idx).radar.wfs(wf).rx_paths;
    params(param_idx).radar.wfs(wf).Tadc_adjust = -0.00000164;
%     params(param_idx).radar.wfs(wf).chan_equal_deg = zeros(1,max(rx_paths));
%     params(param_idx).radar.wfs(wf).chan_equal_dB = zeros(1,max(rx_paths));
%     params(param_idx).radar.wfs(wf).Tsys = zeros(1,max(rx_paths));
    params(param_idx).radar.wfs(wf).chan_equal_deg = [-96.4 142.3 0 -146.9 173.2 41.2 148.2 -168 -136.2 174.3 -169.6 -48.2 -131.4 -119.2 -119.2];
    params(param_idx).radar.wfs(wf).chan_equal_dB = [2.9 1.3 0 -2.5 0.5 0.1 1.4 2.5 1.5 -0.3 2 2.8 1.8 1.1 2.2];
    params(param_idx).radar.wfs(wf).Tsys = [3.73 3.09 0 5.59 3.05 0.92 2.28 -19.24 -22.03 -26.23 -29.8 -28.74 -25.71 -22.8 -19.42]/1e9;
  end
end

2014 Greenland P3: Snow Radar

A DDC was introduced which allowed the data rate to be reduced by subbanding the data before storage. The downside is that the coherent noise is harder to track and remove because it is not sampled as well.

Timing/clock error in the digital system causes an along-track modulation of the signal. The along-track modulation repeats with a harmonic of every 10 records. The recording rate is once every 480 us so this corresponds to a fundamental period of 4.8 ms. The worst harmonics are positive frequency 1/(4.8e-3)=625/3 Hz and negative frequency -1/(4.8e-3/4)=-2500/3 Hz. Notch filters at those frequencies should remove the majority of the modulated coherent noise. Averaging 10 records during coherent averaging or stacking places nulls right on top of these frequencies: this is 2.5 times as many averages as is normally done during processing.

Along-track FFT shows modulated coherent noise.

2017 Arctic Polar5

2017 Arctic Polar5: Snow

Four channel snow radar:

H or V defined based on side looking or cross-track scatter incidence angle plane. Therefore, H has the E-plane oriented parallel to the along-track. And, V has the E-plane oriented perpendicular to the along-track.

Waveform Description
1 H transmit (switch setting 1)
2 V transmit (switch setting 0)
ADC Description
1 V receive
2 H receive

The waveform-adc pairs are then:

wf-adc pair Description
[1 1] VH (V receiver, H transmit)
[1 2] HH
[2 1] VV
[2 2] HV (H receiver, V transmit)


2017 Arctic Polar5: Snow: Coherent Noise

Analysis settings are captured in the parameter spreadsheet.

  param_override.collate_coh_noise.method = 'firdec';
  param_override.collate_coh_noise.firdec_fs = 1/7.5;
  param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/30;
  • 20170330_0[1 3]: elevation compensation issues?
  • 20170410_01_00[4:9]: coherent noise removal could be better?


2017 Arctic Polar5: Snow: Deconvolution

Analysis settings are captured in the parameter spreadsheet.

params = ct_set_params(params,'collate_deconv.f0',4e9);
params = ct_set_params(params,'collate_deconv.f1',11.5e9);
params = ct_set_params(params,'collate_deconv.abs_metric',[58 9.8 -25 -35 inf inf]);
params = ct_set_params(params,'collate_deconv.SL_guard_bins',10);

for param_idx = 1:length(params)
  for img = 1:length(params(param_idx).analysis.imgs)
    params(param_idx).collate_deconv.rbins{img} = [-400 150];
  end
  if strcmpi(params(param_idx).day_seg,'20170402_01')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [1550896 200];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20170404_02')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [73456 100];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20170408_01')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [364224 100];
    end
  end
end


Waveforms with default metrics [58 9.8 -25 -35 inf inf]

Segment [2 1] [1 1] [1 2] [2 2] deconv based on frm # notes
20170330_01 did not pass did not pass passed passed using wf from 20170330_03
20170330_03 did not pass did not pass passed passed autom
20170402_01 passed did not pass passed passed 67
20170404_01 did not pass did not pass passed passed autom
20170404_02 did not pass did not pass passed passed 4
20170406_01 passed did not pass passed passed autom
20170408_01 passed did not pass passed passed 18
20170410_01 0 0 passed passed using wf from 20170404_02 landfast ice

2017 Greenland P3

2017 Greenland P3: Snow

2017 Greenland P3: Snow: BW_window setting

qlook: without deconvolution and matches the bandwidth of whatever was selected in hardware. It is 2.4 GHz-17 GHz for 2-18 GHz data segments and 2.4 GHz-8 GHz for 2-8 GHz data segments. For 2-6 GHz data segments, BW_window is 2.4-6 GHz. This data product is comparable to the snow “qlook” data product of the past. Note that the transmit waveform is very good (a high speed arbitrary waveform generator is used) and the qlook data product already has very low sidelobes. The reason for the selection:

  • 2-2.2 GHz has too many unwanted spurs for 2-18 GHz chirp and 2-2.075 GHz has too many unwanted spurs for 2-8 GHz chirp
  • beyond 17.5 GHz there may be no reference LO signal because the RF path can be delayed by up to 500 MHz relative to the reference LO
  • the raw data signal is clean at the end of the sweep which corresponds to the upper end of the RF frequency range
  • 14-18 GHz has too much roll off in signal power for 2017 Greenland P3
  • 17-18 GHz has too much roll off in signal power for all years

The windowing is done for 3 us propagation delay regardless of the actual propagation delay. This corresponds to 200 MHz for 2-18 GHz and 75 MHz for 2-8 GHz. We then took the maximum delay of 500 MHz and added the bad region (200 MHz or 75 MHz) to get 2000 + 500 + [200 75] (unwanted spurs) - [200 75] (3 us delay) = 2500 MHz minimum starting frequency for deconvolution. By truncating the frequency range to the propagation delay converted to IF frequency or lower on the upper end we ensure that we always have signal at the highest frequency used. For 2-18 GHz, this is 18000-200 = 17.8 GHz (which was further dropped to 17 GHz) and for 2-8 GHz, this is 8000-75=7.925 GHz (rounded to 7.9 GHz).

deconv: deconvolution, 2.5-7.9 GHz, this data product is most comparable with previous "deconv" products and the range resolution will be very similar. This product is not available for the 2-6 GHz segment. Resampling ratio format rat; (7.9-2.5)*2/(17-2.4) leads to [54 73] for the resampling. For the 2-8 GHz data product, the resampling ratio is format rat; (7.9-2.5)*2/(8-2.4) or [27 14].

uwb: deconvolution 2.5-17 GHz except for 2017 Greenland P3 it is 2.5-14 GHz. This is a new data product. This product is not available for the 2-6 and 2-8 GHz segments (only available for the 2-18 GHz segments). Resampling ratio format rat; (17-2.5)*2/(17-2.4) leads to [145 73] for the resampling. For 2.5-14 GHz, the resampling is format rat; (14-2.5)*2/(17-2.4) or [115 73].

kuband: deconvolution 12-17 GHz except for 2017 Greenland P3 it is 12-14 GHz. This data product is most comparable with previous “kuband deconv” products. This product is not available for the 2-6 and 2-8 GHz segments (only available for the 2-18 GHz segments). Resampling ratio format rat; (17-12)*2/(17-2.4) leads to [50 73] for the resampling. For 12-14 GHz, the resampling is format rat; (14-12)*2/(17-2.4) or [20 73].

NOTE: There is no “kuband qlook” data product (i.e. no before-deconvolution kuband quick look image). This was dropped as a time savings measure since the “kuband deconv” product would be better suited for most if not all applications.

Season 2017 2018 2019
Transmit, GHz param qlook deconv uwb kuband qlook deconv uwb kuband qlook deconv uwb kuband
2 - 18
BW_Window [ f0 - f1 ] [ 2.4 - 17 ] [ 2.5 - 7.9 ] [ 2.5 - 14 ] [ 12 - 14 ] [ 2.4 - 17 ] [ 2.5 - 7.9 ] [ 2.5 - 17 ] [ 12 - 17 ] [ 2.4 - 17 ] [ 2.5 - 7.9 ] [ 2.5 - 17 ] [ 12 - 17 ]
qlook.resample = 2* 5.4/14.6 = [ 54 73 ] = 2* 11.5/14.6 = [ 115 73 ] = 2* 2/14.6 = [ 20 73 ] = 2* 5.4/14.6 = [ 54 73 ] = 2* 14.5/14.6 = [ 145 73 ] = 2* 5/14.6 = [ 50 73 ] = 2* 5.4/14.6 = [ 54 73 ] = 2* 14.5/14.6 = [ 145 73 ] = 2* 5/14.6 = [ 50 73 ]
qlook.trim [12 8] [30 12] [12 8] [12 8] [30 12] [12 8] [12 8] [30 12] [12 8]
collate_deconv.rbins [-190 70] [-190 70] [-190 70] [-320 80] [-320 80] [-320 80]
collate_deconv.abs_metric [58 9.8 -25 -35 inf inf] [58 4.5 -25 -35 inf inf] [58 24 -25 -28 inf inf] [58 9.8 -25 -35 inf inf] [58 4.5 -25 -35 inf inf] [58 24 -25 -28 inf inf] [58 9.8 -25 -35 inf inf] [58 4.5 -25 -35 inf inf] [58 24 -25 -28 inf inf]
collate_deconv.SL_guard_bins 10 6 20 10 6 20 10 6 20
collate_deconv.out_path analysis analysis_uwb analysis_kuband analysis analysis_uwb analysis_kuband analysis analysis_uwb analysis_kuband
qlook.out_path qlook deconv qlook_uwb qlook_kuband qlook deconv qlook_uwb qlook_kuband qlook deconv qlook_uwb qlook_kuband
2 - 8
BW_Window [ f0 - f1 ] [ 2.4 - 8 ] [ 2.5 - 7.9 ] [ 2.4 - 8 ] [ 2.5 - 7.9 ]
qlook.resample = 2* 5.4/5.6 = [ 27 14 ] = 2* 5.4/5.6 = [ 27 14 ]
qlook.trim [4 4] [4 4]
collate_deconv.rbins [-80 35]
collate_deconv.abs_metric [65 4.5 -25 -35 inf inf] [65 4.5 -25 -35 inf inf]
collate_deconv.SL_guard_bins 6 6
collate_deconv.out_path analysis analysis
qlook.out_path qlook deconv qlook deconv
  • There is significant roll off above 12 GHz which is why I ended up not including frequencies above 14 GHz in the deconvolved data products since the noise amplification due to deconvolution above 14 GHz was too much.
  • The kuband product has very low SNR, but is probably still useable for some multi-frequency analyses. This situation is improved in the 2018 Greenland P3 dataset and further improved in the 2018 Antarctica DC8 dataset. Jilu judged the 2017 Antarctica P3 kuband product to be unusable.
  • All subbanding includes deconvolution so that images from different bands may be compared in a radiometric way. The subbanding is based off the 2.4-17 GHz product and then subbanded using a hanning window centered on the subband. The deconvolution filter is created and applied in the same way as before except for this frequency windowing operation.
  • Due to the deramp on receive operation and a large range of potential delays between the reference and transmitted signals, the edges of the frequency band (up to four Nyquist zones or ~600 MHz), the edge frequencies are removed so that I could ensure all frequencies used would be present for all potential target delays. For example, this is the reason for starting with 2.4-17 GHz for the 2-18 GHz mode.
  • Some data segments were collected with a 2-8 GHz radar waveform and some data segments were collected with a 2-18 GHz radar waveform. The deconv product signal powers should be the same for all data segments, but the noise floor is lower for 2-8 GHz data segments. The kuband and uwb products are not available for 2-8 GHz data segments. There is one very short data segment, 20170424_02, collected with 2-6 GHz which only has a qlook data product.
  • Regarding radiometric properties, I've made some effort to provide a radiometric product so that the intensity values are scaled to 1/R^2 assuming a specular reflection. For a 500 m range specular target, this translates to a value of about -54 dB. So, regardless of the bandwidth of the data product, a (perfectly) specular target should be -54 dB; note that many sea ice leads are dominated by their specular component, but are not perfectly specular and the small scale roughness may substantially lower the peak intensity. The system_dB value handles this offset. The radiometric_corr_dB field is always zero for this data product because there is only one channel and the system_dB corretion already takes care of each individual wf-adc pair.
  • Noise showing up as vertical stripes: 20170324_02_002 recs 45317-47316. Caused by narrow band noise sources (cause unknown). This is burst noise that could theoretically be removed before pulse compression.

2017 Greenland P3: Snow: Deconvolution

For run_collate_deconv:

% 2-18 GHz Deconvolution Settings (3 sets)
% params = ct_set_params(params,'collate_deconv.f0',2.5e9);
% params = ct_set_params(params,'collate_deconv.f1',14e9);
% params = ct_set_params(params,'collate_deconv.abs_metric',[58 4.5 -25 -35 inf inf]);
% params = ct_set_params(params,'collate_deconv.SL_guard_bins',6);
% param_override.collate_deconv.out_path = 'analysis_uwb';
% params = ct_set_params(params,'cmd.generic',0,'cmd.notes','2-8');
% param_override.collate_deconv.rbins{1} = [-190 70];

% params = ct_set_params(params,'collate_deconv.f0',2.5e9);
% params = ct_set_params(params,'collate_deconv.f1',7.9e9);
% params = ct_set_params(params,'collate_deconv.abs_metric',[58 9.8 -25 -35 inf inf]);
% params = ct_set_params(params,'collate_deconv.SL_guard_bins',10);
% param_override.collate_deconv.out_path = 'analysis';
% params = ct_set_params(params,'cmd.generic',0,'cmd.notes','2-8');
% param_override.collate_deconv.rbins{1} = [-190 70];

% params = ct_set_params(params,'collate_deconv.f0',12e9);
% params = ct_set_params(params,'collate_deconv.f1',14e9);
% params = ct_set_params(params,'collate_deconv.abs_metric',[58 24 -25 -28 inf inf]);
% params = ct_set_params(params,'collate_deconv.SL_guard_bins',20);
% param_override.collate_deconv.out_path = 'analysis_kuband';
% params = ct_set_params(params,'cmd.generic',0,'cmd.notes','2-8');
% param_override.collate_deconv.rbins{1} = [-190 70];

% 2-8 GHz Deconvolution Settings
params = ct_set_params(params,'collate_deconv.f0',2.5e9);
params = ct_set_params(params,'collate_deconv.f1',7.9e9);
params = ct_set_params(params,'collate_deconv.abs_metric',[65 4.5 -25 -35 inf inf]);
params = ct_set_params(params,'collate_deconv.SL_guard_bins',6);
param_override.collate_deconv.out_path = 'analysis';
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','2-18');
param_override.collate_deconv.rbins{1} = [-80 35];


Use these settings for run_collate_deconv_update:

% 2-18 GHz Deconvolution Settings (3 sets)
params = ct_set_params(params,'analysis.cmd{1}.abs_metric',[58 4.5 -25 -35 inf inf]);
param_override.update_collate_deconv.in_path = 'analysis_uwb';

params = ct_set_params(params,'analysis.cmd{1}.abs_metric',[58 9.8 -25 -35 inf inf]);
param_override.update_collate_deconv.in_path = 'analysis';

params = ct_set_params(params,'analysis.cmd{1}.abs_metric',[58 24 -25 -28 inf inf]);
param_override.update_collate_deconv.in_path = 'analysis_kuband';

% 2-8 GHz Deconvolution Settings
params = ct_set_params(params,'analysis.cmd{1}.abs_metric',[65 4.5 -25 -35 inf inf]);
param_override.update_collate_deconv.in_path = 'analysis';

Use the following for run_update_collate_deconv for segments with no good deconvolution waveforms or to remove bad deconvolution waveforms

  %%%% ct_set_params: to update this
  %%%% cmd:           Uses deconv H from this day_seg
  params = ct_set_params(params,'cmd.generic',0);
  if 0 % rerunning this because IMPOSTER!!! change the 0323_02 to 0407_02
    params = ct_set_params(params,'cmd.generic',1,'day_seg','20170323_03');
    param_override.collate_deconv_update.cmd = {};
    param_override.collate_deconv_update.cmd{end+1}.method = 'add';
    param_override.collate_deconv_update.cmd{end}.day_seg = {'20170407_02'}; % ex-20170323_02
    param_override.collate_deconv_update.cmd{end}.idxs = {[1]};
  elseif 0
    params = ct_set_params(params,'cmd.generic',1,'day_seg','20170311_02');
    param_override.collate_deconv_update.cmd = {};
    param_override.collate_deconv_update.cmd{end+1}.method = 'delete';
    param_override.collate_deconv_update.cmd{end}.idxs = [2];
  elseif 0
    params = ct_set_params(params,'cmd.generic',1,'day_seg','20170324_02');
    param_override.collate_deconv_update.cmd = {};
    param_override.collate_deconv_update.cmd{end+1}.method = 'replace';
    param_override.collate_deconv_update.cmd{end}.day_seg = {'20170324_01'};
    param_override.collate_deconv_update.cmd{end}.idxs = {[1]};
  elseif 0
    params = ct_set_params(params,'cmd.generic',1,'day_seg','20170411_02');
    param_override.collate_deconv_update.cmd = {};
    param_override.collate_deconv_update.cmd{end+1}.method = 'replace';
    param_override.collate_deconv_update.cmd{end}.day_seg = {'20170411_01'};
    param_override.collate_deconv_update.cmd{end}.idxs = {[1]};
  elseif 0
    params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-18');
    params = ct_set_params(params,'cmd.generic',0,'cmd.mission_names','^sea.*'); %sea
    param_override.collate_deconv_update.cmd = {};
    param_override.collate_deconv_update.cmd{end+1}.method = 'add';
    param_override.collate_deconv_update.cmd{end}.day_seg = {'20170407_02'};
    param_override.collate_deconv_update.cmd{end}.idxs = {[1]};
  elseif 0
    params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-8');
    params = ct_set_params(params,'cmd.generic',0,'cmd.mission_names','^sea.*'); %sea
    param_override.collate_deconv_update.cmd = {};
    param_override.collate_deconv_update.cmd{end+1}.method = 'add';
    param_override.collate_deconv_update.cmd{end}.day_seg = {'20170323_02'};
    param_override.collate_deconv_update.cmd{end}.idxs = {[1]};
  elseif 0 % %this was previously 2-18, then changed to 2-8, imposter!!!
    params = ct_set_params(params,'cmd.generic',1,'day_seg','20170312_02');
    param_override.collate_deconv_update.cmd = {};
    param_override.collate_deconv_update.cmd{end+1}.method = 'add';
    param_override.collate_deconv_update.cmd{end}.day_seg = {'20170323_02'};
    param_override.collate_deconv_update.cmd{end}.idxs = {[1]};
  end
  params = ct_set_params(params,'cmd.generic',0,'cmd.notes','do not process');

2017 Greenland P3: Snow: Tadc_adjust

233.1 us from start peak to end peak: observed from strong sea ice lead
235.2 us from rising edge to falling edge (when signal intersects noise level): also observed
Target delay: 3.1445 us --> Measured from echogram 
System delay: 64 ns --> Measured by comparing echogram to known elevation reference
Total delay: 3.209 us
240 us pulse duration
0.9 us rising and falling edge where no signal is present --> estimated by assuming symmetric rise and fall and computing (240-235.2-3.2)/2 = 0.8 us
Which then matches with the observed record: 240 - 2*0.8 - 3.2 = 235.2 us
Sweeps from 2-18 GHz
System turns on at 2.05333 GHz and turns off at 17.94666 GHz
REF 2 GHz (start transmit event) is bin 500 - (3.209e-6 + 0.8e-6)*250e6 = -502.25
REF 18 GHz (end transmit event) is bin -502.25+ 240e-6*250e6 = 59497
Expected onset of 2.05333 GHz signal: -502.25 + (0.8e-6+3.209e-6)*250e6 = 500
RF truncation: 3.209 + 0.8 us = 4.009 us which means:
	18e9-16e9/240e-6*4.009e-6 = 17.733 GHz is last good RF signal
	The bin is: 500 + (17.733e9-2.05333e9)/alpha*250e6 = 59300
t0 = -502.25/250e6 = -2.009e-6 --> We start capture 2.009 us after start of transmit event
hdr.t0_raw{img}(rec) was 0.08e-6 --> Delay is 80 ns (20 250 MHz clock counts) according to DAQ header
wfs(wf).Tadc_adjust = 2.009e-6 - 80e-9 = 1.929e-6

2017 Greenland P3: Snow: qlook

% Regular Settings

params = ct_set_params(params,'cmd.qlook',1);
params = ct_set_params(params,'cmd.qlook',0,'cmd.notes','Do not process');
params = ct_set_params(params,'radar.wfs(1).deconv.en',0);
params = ct_set_params(params,'qlook.out_path','qlook');
params = ct_set_params(params,'qlook.resample',[]);

% 2-8 GHz Deconvolution Settings

params = ct_set_params(params,'cmd.qlook',0);
params = ct_set_params(params,'cmd.qlook',1,'cmd.notes','2-8');
params = ct_set_params(params,'cmd.qlook',0,'cmd.notes','Do not process');
params = ct_set_params(params,'radar.wfs(1).deconv.en',1);
params = ct_set_params(params,'radar.wfs(1).deconv.fn','analysis');
params = ct_set_params(params,'qlook.out_path','deconv');
params = ct_set_params(params,'qlook.resample',[27 14; 1 1]);
params = ct_set_params(params,'qlook.trim',[4 4]);

% 2-18 GHz Deconvolution Settings

params = ct_set_params(params,'cmd.qlook',0);
params = ct_set_params(params,'cmd.qlook',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.qlook',0,'cmd.notes','Do not process');
params = ct_set_params(params,'radar.wfs(1).deconv.en',1);
params = ct_set_params(params,'radar.wfs(1).deconv.fn','analysis');
params = ct_set_params(params,'qlook.out_path','deconv');
params = ct_set_params(params,'qlook.resample',[54 73; 1 1]);
params = ct_set_params(params,'qlook.trim',[12 8]);

params = ct_set_params(params,'cmd.qlook',0);
params = ct_set_params(params,'cmd.qlook',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.qlook',0,'cmd.notes','Do not process');
params = ct_set_params(params,'radar.wfs(1).deconv.en',1);
params = ct_set_params(params,'radar.wfs(1).deconv.fn','analysis_uwb');
params = ct_set_params(params,'qlook.out_path','qlook_uwb');
params = ct_set_params(params,'qlook.resample',[115 73; 1 1]);
params = ct_set_params(params,'qlook.trim',[30 12]);

params = ct_set_params(params,'cmd.qlook',0);
params = ct_set_params(params,'cmd.qlook',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.qlook',0,'cmd.notes','Do not process');
params = ct_set_params(params,'radar.wfs(1).deconv.en',1);
params = ct_set_params(params,'radar.wfs(1).deconv.fn','analysis_kuband');
params = ct_set_params(params,'qlook.out_path','qlook_kuband');
params = ct_set_params(params,'qlook.resample',[20 73; 1 1]);
params = ct_set_params(params,'qlook.trim',[12 8]);

2017 Greenland P3: Snow: post.m

params = ct_set_params(params,'post.data_dirs',{'qlook'});
params = ct_set_params(params,'post.out_path','post');
params = ct_set_params(params,'cmd.generic',1);
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

params = ct_set_params(params,'post.data_dirs',{'deconv'});
params = ct_set_params(params,'post.out_path','post_deconv');
params = ct_set_params(params,'cmd.generic',0);
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-8');
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

params = ct_set_params(params,'post.data_dirs',{'qlook_kuband'});
params = ct_set_params(params,'post.out_path','post_kuband');
params = ct_set_params(params,'cmd.generic',0);
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

params = ct_set_params(params,'post.data_dirs',{'qlook_uwb'});
params = ct_set_params(params,'post.out_path','post_uwb');
params = ct_set_params(params,'cmd.generic',0);
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

2017 Greenland P3: Snow: compress_echogram.m

param_override.compress_echogram.echogram_dir = 'qlook';
param_override.compress_echogram.out_path = 'CSARP_post/qlook';
params = ct_set_params(params,'cmd.generic',1);
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

param_override.compress_echogram.echogram_dir = 'deconv';
param_override.compress_echogram.out_path = 'CSARP_post/deconv';
params = ct_set_params(params,'cmd.generic',0);
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-8');
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

param_override.compress_echogram.echogram_dir = 'qlook_kuband';
param_override.compress_echogram.out_path = 'CSARP_post/qlook_kuband';
params = ct_set_params(params,'cmd.generic',0);
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

param_override.compress_echogram.echogram_dir = 'qlook_uwb';
param_override.compress_echogram.out_path = 'CSARP_post/qlook_uwb';
params = ct_set_params(params,'cmd.generic',0);
params = ct_set_params(params,'cmd.generic',1,'cmd.notes','2-18');
params = ct_set_params(params,'cmd.generic',0,'cmd.notes','Do not process');

2018 Greenland P3

2018 Greenland P3: Accum

Four channel for four cross-track antennas and single channel (four channels combined in analog/RF) mode used: double check this.

Four transceiver boxes had an interference issue and boxes with power amp and transmit-receive switches had to be pulled. Test flights had all four channels, but after this only one channel was used except on April 18: double check this.

2018 Greenland P3: RDS

2018 Greenland P3 RDS: Coherent Noise Removal

There is coherent noise present on all channels. For some channels, the coherent noise is low enough that it may not affect the processed result. On some channels, the coherent noise is high enough that it does affect the result. The coherent noise can be separated into three categories:

  1. Direct path feed through that appears to be fairly stable (e.g. a time constant of at least 120 sec and centered on zero-wavenumber). This is present on all channels and it is only a problem before about 1.834e-6 sec in fast time since it rapidly dies out. The direct path/feed through also contains a random component that can change very rapidly and may indicate that there is a hardware instability that could or needs to be tuned.
  2. A DC voltage offset that is present through the entire range line (for all fast time). This seems to be very stable with a time constant of 8 hours or more and is also centered on zero-wavenumber.
  3. A slowly decaying ringing signal that may be due to the transmit/receive switch video feed through, but this has not been verified. This signal is present on adc 9, 10, and 11. It seems to only be a problem for flights on Apr 21 and later. Only the noise on adc 9 and adc 10 is above the noise floor after processing (it is nearly undetectable on adc 11). The time constant of this noise is 30 sec and is also centered on zero-wavenumber. This signal decays and is undetectable past about 30e-6 sec.

The following analysis settings should be used on the first pass of run_analysis coherent noise estimation. This pass is used to set the thresholds.

params = ct_set_params(params,'analysis.cmd{1}.threshold',[]);
params = ct_set_params(params,'analysis.out_path','analysis');

The second pass uses the thresholds from the first pass+collate coh noise. This pass is used to estimate the coherent noise that data_pulse_compress.m will use to subtract away the coherent noise. The second pass is only required for adc 9 and 10 for segments on or after Apr 21.

params = ct_set_params(params,'analysis.cmd{1}.threshold','');
params = ct_set_params(params,'analysis.out_path','analysis_threshold');

The following run_collate_coh_noise.m settings are used to deal with each regime of the coherent noise. Each section must be run separately:

for param_idx = 1:length(params)
  param = params(param_idx);
  if ~isfield(param.cmd,'generic') || iscell(param.cmd.generic) || ischar(param.cmd.generic) || ~param.cmd.generic
    continue;
  end
  for wf = 1:length(params(param_idx).radar.wfs)
    if any(wf == [1 2])
      params(param_idx).collate_coh_noise.threshold_eval{wf} = 'threshold(time>Tpd+0.85e-6 & threshold>-110) = -100; threshold(time<=Tpd+0.85e-6) = inf;'
    elseif any(wf == [3 4])
      params(param_idx).collate_coh_noise.threshold_eval{wf} = 'threshold(time>Tpd+2.3e-6 & threshold>-130) = -110; threshold(time<=Tpd+2.3e-6) = threshold(time<=Tpd+2.3e-6)+20;';
    elseif any(wf == [5 6])
      params(param_idx).collate_coh_noise.threshold_eval{wf} = 'threshold(time>Tpd+3e-6 + & threshold>-142) = -122; threshold(time<=Tpd+3e-6) = threshold(time<=Tpd+3e-6)+20;';
    else
      keyboard
    end
  end
end
  % First pass of run_analysis coherent noise estimation
  % WF 1-2
  param_override.collate_coh_noise.method = 'firdec';
  param_override.collate_coh_noise.firdec_fs = 1/30;
  param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/120*(t<1.834e-6) + -1*(t>=1.834e-6);
  param_override.collate_coh_noise.imgs = [1 2];
  param_override.collate_coh_noise.wf_adcs = [1:16];
  param_override.collate_coh_noise.in_path = 'analysis';
  param_override.collate_coh_noise.out_path = 'analysis';

  % WF 3-6
  param_override.collate_coh_noise.method = 'dft';
  param_override.collate_coh_noise.dft_corr_length = inf;
  param_override.collate_coh_noise.imgs = [3 4 5 6];
  param_override.collate_coh_noise.wf_adcs = [];
  param_override.collate_coh_noise.in_path = 'analysis';
  param_override.collate_coh_noise.out_path = 'analysis';

  % Second pass of run_analysis coherent noise estimation
  % Should be run after run_analysis and collate_coh_noise are run and run_analysis is run again using thresholds set in first pass
  % WF 1-2
  param_override.collate_coh_noise.method = 'firdec';
  param_override.collate_coh_noise.firdec_fs = 1/30;
  param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/120*(t<1.834e-6) + -1*(t>=1.834e-6);
  param_override.collate_coh_noise.imgs = [1 2];
  param_override.collate_coh_noise.wf_adcs = [1:16];
  param_override.collate_coh_noise.in_path = 'analysis'; % Turns out thresholding was not necessary for wf 1 and 2.
  param_override.collate_coh_noise.out_path = 'analysis_threshold';

  % WF 3-6 (except adc 9-10 for Apr 21 and later)
  param_override.collate_coh_noise.method = 'dft';
  param_override.collate_coh_noise.dft_corr_length = inf;
  param_override.collate_coh_noise.imgs = [3 4 5 6];
  param_override.collate_coh_noise.wf_adcs = [1:8,11:16]; % On and after Apr 21
  param_override.collate_coh_noise.in_path = 'analysis_threshold';
  param_override.collate_coh_noise.out_path = 'analysis_threshold';
  
  % WF 3-6 (adc 9-10 for Apr 21 and later)
  % For Apr 21 and later, adc 9 and 10 must be collated separately (otherwise can be added to the section above):
  param_override.collate_coh_noise.method = 'firdec';
  param_override.collate_coh_noise.firdec_fs = 1/7.5;
  param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/30*(t<30e-6);
  param_override.collate_coh_noise.imgs = [3 4 5 6];
  param_override.collate_coh_noise.wf_adcs = [9 10];
  param_override.collate_coh_noise.in_path = 'analysis_threshold';
  param_override.collate_coh_noise.out_path = 'analysis_threshold';

When running qlook or sar use:

params(param_idx).radar.wfs(wf).coh_noise_method = 'analysis';
params(param_idx).radar.wfs(wf).coh_noise_arg.fn = 'analysis_threshold';

2018 Greenland P3 RDS: Burst Noise

The wing elements attached to adcs 1-4 and 13-16 have occasional drop outs. Burst noise removal settings:


ADC 6 attached to the second from left fuselage antenna element has a noise floor that spikes upward in power occasionally. A comparison of the mean power as a function of range line is shown here with ADC6 (blue), ADC 7 (red), and ADC 5 (orange):

ADC 6 noise fluctuations.

2018 Greenland P3 RDS: Deconvolution

  • Three deconvolution segments: 20180322_03, 20180322_04, and 20180420_01.
  • Deconvolution data on 20180420_01 only has NMEA GPS data and is too low AGL for the 10 us waveform. This segment can be used to check the deconvolution results, but should not be used.
  • 20180322_04 has good quality waveforms for 1, 3, and 10 us.
  • 20180322_03 has low quality waveforms, possibly due to fast ascent or poor surface conditions. Only one small section towards the end of the file is good.
  • Deconvolution does not seem to be working on 20180404_02 (the only regular mission segment tested so far). The impulse response appears to be different on 20180404_02 as compared with the three deconvolution segments: the sidelobes for 20180404_02 appear to be lower than the sidelobes for the deconvolution segments. The delay, td, is different and the inboard antennas may have some fast-time gain changes (I don't think this is the problem because I tested inboard and outboard channels and both show the same problem; Also the time delay is fairly large, td > 5e-6, for the beginning of 20180404_02 so the fast time gain changes are probably small). The receiver attenuation was different (assuming this would not make much difference to the sidelobes of the transfer function). The beam pattern was steered for the 20180404_02 data: this seems to be a likely culprit... will have to search through the data segments with beam steering to see if there is any good deconvolution data. If this is the case, then the dominant transfer function sidelobes are transmit beam pattern dependent -- this makes some sense because the beam pattern is a function of frequency so that nadir, which is outside the main beam, would have a new frequency dependent weighting due to the antenna array gain.
  • Deconvolution on 20180420_01 works for main reflection (but not as well as on 20180322_03/4, probably because of temporal changes in the system response). It also works for the much weaker multiple. This suggests that compression is not the issue. Interestingly, the deconvolution does not work for the scattering from a rock outcrop (actually raises the sidelobe), but does work for the specular reflection just a few range bins later. The rock outcrop is likely coming from the side rather than from nadir which would strongly support the idea that the dominant transfer function sidelobes are DOA dependent.
params = ct_set_params(params,'collate_deconv.f0',181.5e6);
params = ct_set_params(params,'collate_deconv.f1',208.5e6);
params = ct_set_params(params,'collate_deconv.rbins',{[-38 33],[-105 75],[-350 25]});
params = ct_set_params(params,'collate_deconv.abs_metric',[58 4.5 -25 -35 inf inf]);
params = ct_set_params(params,'collate_deconv.SL_guard_bins',6);

% param_override.collate_deconv.debug_plots = {'visible','rbins','deconv'};
param_override.collate_deconv.debug_plots = {};
param_override.collate_deconv.debug_ylim = 120;
param_override.collate_deconv.stage_one_en = true;
param_override.collate_deconv.stage_two_en = true;

2018 Greenland P3 RDS: Processing

  for param_idx = 1:length(params)
    %param = params(param_idx);
    for wf = 1:length(params(param_idx).radar.wfs)
      params(param_idx).radar.wfs(wf).Tsys = [0.46 -4.66 0.14 -1.77 0 -2.63 -3.38 -69.66 -75.57 -75.45 -80.42 -80.49 -75.71 -77.69 -70.53]/1e9;
      params(param_idx).radar.wfs(wf).chan_equal_dB = [6.8 -0.6 3 0.1 0 3.5 3.9 7 3.3 4.8 6.1 6.2 4.6 3.1 6.2];
      params(param_idx).radar.wfs(wf).chan_equal_deg = [-166.2 -142.7 177 -95.9 0 -25.9 -86.5 -27.4 128.1 41.6 -46.8 43 90.7 121.3 31.6];
      params(param_idx).radar.wfs(wf).coh_noise_method = 'analysis';
      params(param_idx).radar.wfs(wf).coh_noise_arg.fn = 'analysis_threshold';
      params(param_idx).radar.wfs(wf).deconv.en = 0;
      params(param_idx).radar.wfs(wf).deconv.fn = 'analysis';
    end
  end

2018 Greenland P3: Snow

f0-f1 of 2-18 GHz uses a BW_window of: 2.1 to 17.5 GHz (15.4 GHz bandwidth or 57750 points)

f0-f1 of 2-8 GHz uses a BW_window of: 2.1 to 7.6 GHz (5.5 GHz bandwidth or 55000 points)

15.4*5/7 = 11 GHz (decimate for CSARP_deconv is 5/7 when 2-18 GHz and 2/1 when 2-8 GHz)

2018 Greenland P3: Snow: Deconvolution

For run_collate_deconv:

% 2-18 GHz Deconvolution Settings (3 sets)
% params = ct_set_params(params,'collate_deconv.f0',2.5e9);
% params = ct_set_params(params,'collate_deconv.f1',17e9);
% params = ct_set_params(params,'collate_deconv.abs_metric',[58 4.5 -25 -35 inf inf]);
% params = ct_set_params(params,'collate_deconv.SL_guard_bins',6);
% param_override.collate_deconv.out_path = 'analysis_uwb';

% params = ct_set_params(params,'collate_deconv.f0',2.5e9);
% params = ct_set_params(params,'collate_deconv.f1',7.9e9);
% params = ct_set_params(params,'collate_deconv.abs_metric',[58 9.8 -25 -35 inf inf]);
% params = ct_set_params(params,'collate_deconv.SL_guard_bins',10);
% param_override.collate_deconv.out_path = 'analysis';

% params = ct_set_params(params,'collate_deconv.f0',12e9);
% params = ct_set_params(params,'collate_deconv.f1',17e9);
% params = ct_set_params(params,'collate_deconv.abs_metric',[58 24 -25 -28 inf inf]);
% params = ct_set_params(params,'collate_deconv.SL_guard_bins',20);
% param_override.collate_deconv.out_path = 'analysis_kuband';

2018 Antarctica Ground

Ground based mission to Dome Fuji to help locate a deep drilling site.

2018 Antarctica Ground: RDS

2018 Antarctica Ground: RDS: Image Combine

2018 Antarctica TObas

This field campaign was conducted by the British Antarctic Survey for the Thwaites MELT project. The flightlines were flown both for MELT and a grounding line survey of Thwaites. Additional MELT flights were flown for 2019 Antarctica TObas.

Platform: Twin Otter

Tail Number: VP-FBL/MSN 839

2018 Antarctica TObas: Accum

2018 Antarctica TObas: Image Combine

For segment 20190205_01, we found clipping within the image combine. This was corrected by running run_update_img_combine using the code below.

params = ct_set_params(params,'cmd.generic',1,'day_seg','20190205_01');
mode = 'array'; % <== OFTEN CHANGED (qlook or array)

update_img_combine_param.out_path = 'standard';
update_img_combine_param.img_comb_mult = inf; % <== OFTEN CHANGED (inf default)
update_img_combine_param.img_comb_bins = 75; % <== OFTEN CHANGED (1 default)
update_img_combine_param.img_comb = [2e-6 -inf 1.2e-6]; % <== OFTEN CHANGED
update_img_combine_param.img_comb_layer_params = struct('name','surface','source','layerdata','layerdata_source','layerData');% <== OFTEN CHANGED

2018 Antarctica TObas: Coherent Noise Removal

Coherent noise is not 100% stationary (especially feed through, TR switch ringing in the record is fairly stable) and so a low pass filter with a 30 second time constant is used to remove the noise. Because of the large dynamic range of the signal, a two pass approach is required with thresholding on the second pass.

The second pass analysis arguments are:

params = ct_set_params(params,'analysis.cmd{1}.threshold','');
params = ct_set_params(params,'analysis.out_path','analysis_threshold');

The collate_coh_noise arguments for both passes are:

  param_override.collate_coh_noise.method = 'firdec';
  param_override.collate_coh_noise.firdec_fs = 1/7.5;
  param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/30;

for param_idx = 1:length(params)
  param = params(param_idx);
  if ~isfield(param.cmd,'generic') || iscell(param.cmd.generic) || ischar(param.cmd.generic) || ~param.cmd.generic
    continue;
  end
  for wf = 1:length(params(param_idx).radar.wfs)
    params(param_idx).collate_coh_noise.threshold_eval{wf} = 'threshold = max(min(-100,threshold + 20),10*log10(abs(noise.dft(:,1)).^2)+6);';
  end
end

The collate_coh_noise arguments for only the second pass are:

param_override.collate_coh_noise.in_path = 'analysis_threshold';
param_override.collate_coh_noise.out_path = 'analysis_threshold';

2018 Antarctica TObas: Deconvolution

The signal has a third harmonic show up near the end of the pulse close to 900 MHz (the harmonic is around 600 MHz so the assumption is that it is aliasing in). This artifact appears to be removable with deconvolution suggesting that it is a stable transmit distortion.

The third harmonic distortion is circled in the spectrogram below:

Spectrogram

Left image shows before deconvolution and right image shows after deconvolution:

Deconvolution example

params = ct_set_params(params,'cmd.generic',0);
params = ct_set_params(params,'cmd.generic',1,'day_seg','20190207_02'); % This is the segment with the best deconvolution waveform

params = ct_set_params(params,'collate_deconv.f0',615e6);
params = ct_set_params(params,'collate_deconv.f1',875e6);
params = ct_set_params(params,'collate_deconv.abs_metric',[58 5 -25 -35 inf inf]);
params = ct_set_params(params,'collate_deconv.SL_guard_bins',6);
params = ct_set_params(params,'collate_deconv.threshold',82);
params = ct_set_params(params,'collate_deconv.rbins',{[-200 650],[-200 650]});

The signal for wf=2 is slightly saturated on receive on 20190207_02 so that the deconvolution waveform for wf=1 should be used instead. This can be done by copying the wf_1 deconv file to wf_2 (cp CSARP_analysis/deconv_20190207_02_wf_1_adc_1.mat CSARP_analysis/deconv_20190207_02_wf_2_adc_1.mat). After this copy is made, then the deconvolution files for 20190207_02 should be copied to all other segments using run_update_collate_deconv.m. An example of the surface multiple after deconvolution of wf2 data using wf1 deconvolution waveform:

Deconvolution example

2018 Antarctica TObas: Radiometric

Deconvolution wf1 20190207_02 measured -56 dB peak at -8.1 us twtt from a polynya. This value should be scaled to 10*log10(1/(3e8/2*8.1e-6)^2)=-61.7 dB. Therefore all the adc_gains_dB should be adjusted upwards by 5.7 dB. So 27 (the original receiver gain setting for wf1) to 32.7 and 45 (the original gain setting for wf2) to 50.7 dB.

2018 Alaska SO

2018 Alaska SO: RDS

2018 Alaska SO: RDS: Coherent Noise Removal

Due to large signals potentially contaminating the coherent noise estimate, the coherent noise is removed in two steps:

Step one: The default threshold and out_path are used:

params = read_param_xls(ct_filename_param('rds_param_2018_Alaska_SO.xls'),'',{'analysis_noise','analysis'});
params = ct_set_params(params,'analysis.cmd{1}.threshold',[]);
params = ct_set_params(params,'analysis.out_path','analysis');

These results are collated with collate_coh_noise. This threshold_eval needs to be made more precise, but did work to show quick results.

param_override.collate_coh_noise.method = 'dft';
param_override.collate_coh_noise.dft_corr_length = inf;
params(param_idx).collate_coh_noise.threshold_eval{wf} = 'threshold = threshold+1;';
param_override.collate_coh_noise.debug_plots = {'visible','cn','debug','threshold','threshold_plot'};
param_override.analysis.cmd{1}.min_samples = 1500;


Step two: Note that the threshold value is set to an empty string. This causes the default threshold path to be used and the file generated from collate_coh_noise in step one is used to threshold the data during coherent noise estimation.

params = read_param_xls(ct_filename_param('rds_param_2018_Alaska_SO.xls'),'',{'analysis_noise','analysis'});
params = ct_set_params(params,'analysis.cmd{1}.threshold','');
params = ct_set_params(params,'analysis.out_path','analysis_threshold');

These results are collated with collate_coh_noise. Note that the input and output directories are different.

param_override.collate_coh_noise.method = 'dft';
param_override.collate_coh_noise.dft_corr_length = inf;
param_override.collate_coh_noise.debug_plots = {'visible','cn','debug'};
param_override.analysis.cmd{1}.min_samples = 1500;
params(param_idx).collate_coh_noise.in_path = 'analysis_threshold';
params(param_idx).collate_coh_noise.out_path = 'analysis_threshold';

To apply the coherent noise removal results in run_qlook, run_analysis (specular), and run_sar, add the following settings:

params = ct_set_params(params,'radar.wfs(1).coh_noise_method','analysis');
params = ct_set_params(params,'radar.wfs(1).coh_noise_arg.fn','analysis_threshold');

2018 Alaska SO: RDS: Deconvolution

Run the specular analysis in run_analysis.m:

params = read_param_xls(ct_filename_param('rds_param_2018_Alaska_SO.xls'),'',{'analysis_spec' 'analysis'});
params = ct_set_params(params,'radar.wfs(1).coh_noise_method','analysis');
params = ct_set_params(params,'radar.wfs(1).coh_noise_arg.fn','analysis_threshold');

Collate these results to extract the best deconvolution waveform(s) in run_collate_deconv.m:

params = read_param_xls(ct_filename_param('rds_param_2018_Alaska_SO.xls'),'',{'analysis_spec' 'analysis'});
params = ct_set_params(params,'analysis.cmd{1}.f0',1.2e6);
params = ct_set_params(params,'analysis.cmd{1}.f1',3.7e6);
params = ct_set_params(params,'analysis.cmd{1}.rbins',{[-50 40]});
params = ct_set_params(params,'analysis.cmd{1}.abs_metric',[58 4.5 -25 -35 inf inf]);
params = ct_set_params(params,'analysis.cmd{1}.SL_guard_bins',10);
param_override.collate_deconv.debug_plots = {'rbins','deconv','metric','final','visible'};

To apply these deconvolution settings to another segment (for example because that segment does not have a clean waveform for deconvolution in it) in run_update_collate_deconv.m:

params = read_param_xls(ct_filename_param('rds_param_2018_Alaska_SO.xls'),'',{'analysis_spec' 'analysis'});
  
params = ct_set_params(params,'analysis.cmd{1}.abs_metric',[58 4.5 -25 -35 inf inf]);
param_override.collate_deconv.out_path = 'analysis';
  
param_override.update_collate_deconv.cmd = {};
param_override.update_collate_deconv.cmd{end+1}.method = 'replace';
param_override.update_collate_deconv.cmd{end}.day_seg = {'20180819_01'};
param_override.update_collate_deconv.cmd{end}.idxs = {[1]};

To apply the deconvolution results in run_qlook.m and run_sar.m, add the following settings:

params = ct_set_params(params,'radar.wfs(1).deconv.en',1);
params = ct_set_params(params,'radar.wfs(1).deconv.fn','analysis');

2018 Greenland Polar6

2018 Greenland Polar6: Snow

Four channel snow radar:

H or V defined based on side looking or cross-track scatter incidence angle plane. Therefore, H has the E-plane oriented parallel to the along-track. And, V has the E-plane oriented perpendicular to the along-track.

Waveform Description
1 H transmit (switch setting 1)
2 V transmit (switch setting 0)
ADC Description
1 V receive
2 H receive

The waveform-adc pairs are then:

wf-adc pair Description
[1 1] VH (V receiver, H transmit)
[1 2] HH
[2 1] VV
[2 2] HV (H receiver, V transmit)


2018 Greenland Polar6: Snow: Coherent Noise

Analysis settings are captured in the parameter spreadsheet.

  param_override.collate_coh_noise.method = 'firdec';
  param_override.collate_coh_noise.firdec_fs = 1/7.5;
  param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/30;
  • Weird wavy lines: 20180410_01_0[49 50 55 56 61]


2018 Greenland Polar6: Snow: Deconvolution

Analysis settings are captured in the parameter spreadsheet.

params = ct_set_params(params,'collate_deconv.f0',4e9);
params = ct_set_params(params,'collate_deconv.f1',11.5e9);
params = ct_set_params(params,'collate_deconv.abs_metric',[58 9.8 -25 -35 inf inf]);
params = ct_set_params(params,'collate_deconv.SL_guard_bins',10);
for param_idx = 1:length(params)
  for img = 1:length(params(param_idx).analysis.imgs)
    params(param_idx).collate_deconv.rbins{img} = [-400 150];
  end
end

Waveforms with default metrics

Segment [2 1] [1 1] [1 2] [2 2] deconv based on frm # notes
20180410_01 passed passed passed passed autom

2019 Arctic Polar6

2019 Arctic Polar6: Snow

Four channel snow radar:

H or V defined based on side looking or cross-track scatter incidence angle plane. Therefore, H has the E-plane oriented parallel to the along-track. And, V has the E-plane oriented perpendicular to the along-track.

Waveform Description
1 H transmit (switch setting 1)
2 V transmit (switch setting 0)
ADC Description
1 V receive
2 H receive

The waveform-adc pairs are then:

wf-adc pair Description
[1 1] VH (V receiver, H transmit)
[1 2] HH
[2 1] VV
[2 2] HV (H receiver, V transmit)

2019 Arctic Polar6: Snow: Coherent Noise

Analysis settings are captured in the parameter spreadsheet.

  param_override.collate_coh_noise.method = 'firdec';
  param_override.collate_coh_noise.firdec_fs = 1/7.5;
  param_override.collate_coh_noise.firdec_fcutoff = @(t) 1/30;
  • ISSUE: Weird wavy lines: 20190407_02_0[32:37], 20190410_01_0[31:end], 20190410_02_0[65:69 74]
  • Bad coherent noise removal: 20190407_02_028
  • ISSUE: "on/off" coherent noise: 20190408_0[1 2]

2019 Arctic Polar6: Snow: Deconvolution

Analysis settings are captured in the parameter spreadsheet.

params = ct_set_params(params,'collate_deconv.f0',4e9);
params = ct_set_params(params,'collate_deconv.f1',11.5e9);
params = ct_set_params(params,'collate_deconv.abs_metric',[58 9.8 -25 -35 inf inf]);
params = ct_set_params(params,'collate_deconv.SL_guard_bins',10);

for param_idx = 1:length(params)
  for img = 1:length(params(param_idx).analysis.imgs)
    params(param_idx).collate_deconv.rbins{img} = [-400 150];
  end
  if strcmpi(params(param_idx).day_seg,'20190405_02')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [42688 100];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20190407_01')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [252432 100];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20190407_02')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [512432 100];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20190408_01')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [417432 100];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20190408_02')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [51408 100];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20190410_01')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [90896 100];
    end
  end
  if strcmpi(params(param_idx).day_seg,'20190410_02')
    for img = 1:length(params(param_idx).analysis.imgs)
      params(param_idx).collate_deconv.rec_adjustments{img} = [1128968 100];
    end
  end
end


Waveforms with default metrics

Segment [2 1] [1 1] [1 2] [2 2] deconv based on frm # notes
20190401_01 did not pass did not pass passed passed no real spec targets
20190402_01 did not pass did not pass passed passed using wf from 20190407_02 ALS: few leads
20190402_02 0 did not pass passed passed using wf from 20190410_02 ALS: few leads
20190405_02 passed did not pass passed did not pass 3 could work better
20190407_01 passed did not pass passed did not pass 12
20190407_02 passed did not pass passed did not pass 21
20190407_04 0 0 passed passed bad frames
20190408_01 passed did not pass passed did not pass 23
20190408_02 did not pass 0 did not pass did not pass 3 PSL FE fail
20190410_01 passed did not pass passed passed 5
20190410_02 passed did not pass passed passed 77

2019 Arctic GV

Gulfstream V: Johnson Space Center. Tail number N95NA.

2019 Arctic GV: Snow

Single channel snow radar.

Transmit antenna on left/port. Receive antenna on right/starboard. Connectors face each other (inboard). E-plane is oriented in cross-track. H-plane is oriented in along-track. Antenna: A-INFO LB-20180-NF. From datasheet: The 3dB beamwidth at 10 GHz for H-plane is 32.83 deg and E-plane is 32.61 deg.

2019 Arctic GV Snow: Coherent Noise

2019 Arctic GV Snow: Deconvolution

2019 Arctic GV Snow Deconvolution: 20190909_02

Around frames 16-25 a system problem happened (loose cable?) and the falling edge sidelobes get worse. The system impulse response seems to oscillate for a while and then settle into the bad state beyond frame 25. The radiometric accuracy while it is oscillating is lower and could be as much as 3 dB off for a specular target. The different impulse responses are solved by using two deconvolution waveforms. One from before the issue and one from after the issue. The noise floor changes noticeably when switching between deconvolution waveforms.

params = ct_set_params(params,'collate_deconv.gps_times',[1.568037672497153e+09 -inf 1.568037826300003e+09; 1.568038279510004e+09 1.568037826300003e+09 inf],'day_seg','20190909_02');

2019 Antarctica Ground

Ground based mission to Hercules Dome to help locate a deep drilling site.

2019 Antarctica Ground: RDS

Waveform overlap is insufficient due to a 2.5 us transmit/receive switch time. Recommendation for the future is to decrease TTL switching guard times if possible. Regardless, the time gates of the preceding waveforms needs to be increased to properly accommodate the T/R switching time. Currently only 0.5 us is given for a guard and at least 2.5 + 2 us = 4.5 us would have been more appropriate.

2019 Antarctica Ground: Processing Steps

  1. qlook_test: uses tukey fast-time window for internal layers, resample set to [2 1; 1 1], inc_B_filter/inc_dec set to 2x oversample
  2. Track equal_002 and equal_003 layers
  3. analysis equal for layers 2 and 3
  4. collate_equal
  5. analysis noise for layers 2 and 3
  6. Two pass collate_noise with tukey_trim(N,0.5) fast time window
  7. Two pass collate_noise with default hanning fast time window
  8. For all echograms below turn on coherent noise removal and use equalization coefficients
  9. qlook: final
  10. sar_tukey: uses tukey fast-time window, 1 m resolution
  11. sar: uses default hanning fast-time window, 1 m resolution
  12. standard: uses sar, ft_over_sample set to 2, line_rng/dline set to 2x oversample
  13. standard_tukey: uses sar_tukey for internal layers, ft_over_sample set to 2, line_rng/dline set to 2x oversample
  14. mvdr: uses sar, ft_over_sample set to 2, line_rng/dline set to 2x oversample
  15. music3D: uses sar, ft_over_sample set to 2, line_rng/dline set to 2x oversample

2019 Antarctica Ground: system_dB

  • Transmit power: 1000 W per element
  • Transmit power and transmit array factor gain increase with each tx_weight that is enabled: 10*LOG10(1000*SUM(1,0,1,0)^2) dB
  • Metal ground plane backed broadband dipole: each transmit element 6 dB
  • Metal ground plane backed broadband dipole: each receive element 6 dB
  • Standard terms for all radar systems with 50 ohm characteristic impedance: 20*LOG10(c/(8*pi*fc))+10*LOG10(50)
=10*LOG10(1000*SUM(1,0,1,0)^2)+6+6+20*LOG10(300000000/(8*PI()*(BJ39+BK39)/2))+10*LOG10(50)

2019 Antarctica Ground: Image Combine

Image combine adjusted to match where waveforms begin to resolve layers. img_comb settings are:

[3.6e-06 -Inf 5e-07 10.5e-06 -Inf 2.4e-06]

Some roll off is expected at the waveform combine boundaries because there was insufficient overlap in the fast-time between waveforms. Ideal settings would have accounted for the 2.5e-6 T/R switch timing and img_comb would have been:

[5.5e-06 -Inf 1e-06 12.5e-06 -Inf 3e-06]

2019 Antarctica Ground: Coherent Noise

20200107_01

wf 1, adc 1-4 have regular coherent noise POSSIBLY LOWER "+9 (shorted Tpd+1.2e-6) and +3"

wf 1, adc 5-8 have deep coherent noise

wf 2, adc 1-4 have coherent noise POSSIBLY LOWER "+9 (shorted Tpd+1.2e-6) and +3"


wf 3, adc 3-5 have coherent noise

      • SOLVED ***

wf 4, adc 1-2 have coherent noise

      • SOLVED ***

wf 1, adc 1-8 have feed through wf 2, adc 1,5-8 have feed through

      • SOLVED ***

wf 3-4, adc 5-8 has feed through

      • SOLVED ***

2019 Antarctica Ground: Equalization

Relative amplitude 20200701_01.
Relative phase 20200701_01.
20200701_01
wf 2
power in adc 2, 3, 5, 6 is good. ADC 4 is intermittent to good, ADC 1 is consistent but low by 9 dB. ADC 7 is bad. ADC 8 is intermittent bad and gets worse through the segment.
phase is consistent across all channels
time delay is zero across all channels

        params(param_idx).analysis.imgs = {[2*ones([8 1]),(1:8).']};
        params(param_idx).analysis.cmd{1}.start_time = struct('name','Herc_002','eval',struct('cmd','s=s-1.5e-6;'));

Waveform 2: (rows: equal_dB, equal_deg, Tsys_ns)
[10 6 10 9.6 -11.4 -0.8 0 9]
[-56 -61.8 116.5 134.1 87.1 65.9 0 -53.7]
[0 0 0 0 0 0 0 0]/1e9

Use the same for waveform 1.
Waveform 3: (rows: equal_dB, equal_deg, Tsys_ns)
[10.2 6.4 17.8 12.6 15.3 11.7 0 9.2]
[-52.7 -59 117.5 134.2 92.6 73.2 0 -57.6]
[0 0 0 0 0 0 0 0]/1e9

Waveform 4: (rows: equal_dB, equal_deg, Tsys_ns)
[10.6 6.8 20.2 16.3 20.2 6.1 1.4 10.6]
[-47.1 -53.7 127.2 147.1 111.9 122.6 11.2 -35.2]
[0 0 0 0 0 0 0 0]/1e9

2019 Greenland P3

Ground based mission to Hercules Dome to help locate a deep drilling site.

2019 Greenland P3: RDS

Only the center fuselage antenna elements were installed. The wing antenna arrays were not installed. The system was operated in a ping-pong beam-forming mode. This reduces signal to noise ratio but allows a wider swath to be imaged and with greater cross-track resolution.

Six waveforms were used in sequence to create a 3D image of the ice surface, englacial ice targets, and ice bottom: 1. Three left-most elements active and steering the beam 18 deg to the left, 1 us 2. Three right-most elements active and steering the beam 18 deg to the right, 1 us 3. Three left-most elements active and steering the beam 18 deg to the left, 3 us 4. Three right-most elements active and steering the beam 18 deg to the right, 3 us 5. Three left-most elements active and steering the beam 18 deg to the left, 10 us 6. Three right-most elements active and steering the beam 18 deg to the right, 10 us

2019 Greenland P3 RDS: Processing Steps

  1. Coherent noise removal
  2. Equalization
  3. qlook: final
  4. sar: uses default hanning fast-time window, 2.5 m resolution
  5. standard: uses sar, ft_over_sample set to 2, line_rng/dline set to 2x oversample
  6. mvdr: uses sar, ft_over_sample set to 2, line_rng/dline set to 2x oversample
  7. music3D: uses sar, ft_over_sample set to 2, line_rng/dline set to 2x oversample

2019 Greenland P3 RDS: system_dB

  • Transmit power: 337 W per element, only 3 elements excited at a time because of ping-pong operation
  • Transmit power and transmit array factor gain increase with each tx_weight that is enabled: 10*LOG10(337*SUM(1,1,1,0,0,0,0)^2) dB
  • Metal ground plane backed broadband dipole: each transmit element 6 dB
  • Metal ground plane backed broadband dipole: each receive element 6 dB
  • Standard terms for all radar systems with 50 ohm characteristic impedance: 20*LOG10(c/(8*pi*fc))+10*LOG10(50)
10*log10(337*sum([1,1,1,0,0,0,0])^2)+6+6+20*log10(300000000/(8*pi*(180e6+210e6)/2))+10*log10(50) = 39.5454

2019 Greenland P3 RDS: Image Combine

2019 Greenland P3 RDS: Coherent Noise

2019 Greenland P3 RDS: Equalization

2019 SouthDakota N1KU

Platform: Cessna 172

Tail Number: N1KU

2019 SouthDakota N1KU: Snow

Single channel snow radar.

2019 SouthDakota N1KU: system_dB

  • Transmit power: 0.1 W
  • A-INFO LB-20180 broadband TEM horn antenna for transmit gain: 10.5 dB at fc
  • A-INFO LB-20180 broadband TEM horn antenna for receive gain: 10.5 dB at fc
  • Standard terms for all radar systems with 50 ohm characteristic impedance: 20*LOG10(c/(8*pi*fc))+10*LOG10(50)
=10*LOG10(0.1)+10.5+10.5+20*LOG10(300000000/(8*PI()*(H3+I3)/2))+10*LOG10(50)

2019 SouthDakota N1KU: Coherent Noise

2019 SouthDakota N1KU: System Time Delay

Updated radar.wfs(1).t_ref based on run_check_surface results of following frames (near the basin region):

%% Frames near basin region for use in run_check_surface
params=ct_set_params(params,'cmd.generic',1,'day_seg','20200129_02'); params = ct_set_params(params,'cmd.frms',[11],'day_seg','20200129_02');
params=ct_set_params(params,'cmd.generic',1,'day_seg','20200209_02'); params = ct_set_params(params,'cmd.frms',[35, 36],'day_seg','20200209_02');
params=ct_set_params(params,'cmd.generic',1,'day_seg','20200208_02'); params = ct_set_params(params,'cmd.frms',[27, 28],'day_seg','20200208_02');
params=ct_set_params(params,'cmd.generic',1,'day_seg','20200202_06'); params = ct_set_params(params,'cmd.frms',[52, 53],'day_seg','20200202_06');
params=ct_set_params(params,'cmd.generic',1,'day_seg','20200131_03'); params = ct_set_params(params,'cmd.frms',[12],'day_seg','20200131_03');
params=ct_set_params(params,'cmd.generic',1,'day_seg','20200128_02'); params = ct_set_params(params,'cmd.frms',[49, 50],'day_seg','20200128_02');

%% Updated t_ref value in default_radar_params_2019_SouthDakota_N1KU_snow.m and in params spreadsheet
default.radar.wfs(1).t_ref = -103e-9;

2019 SouthDakota N1KU: Deconvolution

2019 SouthDakota N1KU: Ground Truth

From: Roundy, Joshua K <jkroundy@ku.edu> Sent: Friday, March 20, 2020 5:32 PM

Snow Course Sites measured by the National Resources Conservation Service (NRCS): Describes snow course measurements: https://www.wcc.nrcs.usda.gov/factpub/sect_4a.html

Below is a table of the snotel data for the segments flying near the blind park snotel site (https://wcc.sc.egov.usda.gov/nwcc/site?sitenum=354).

Date	Segment	Frames	Time of Day	SWE (in)	Depth (in)
20200128	3	[027]	10:32:58 AM	4.7	25
20200128	6	[045]	3:55:03 PM	4.8	25
20200131	2	[016]	8:58:37 AM	5.3	30
20200201	1	[034]	9:03:59 AM	5.4	29
20200202	1	[079]	9:02:02 AM	5.3	28
20200208	3	[006]	9:53:07 AM	5.9	30
20200209	1	[050]	4:05:09 PM	6	31

The other snotel site: (https://wcc.sc.egov.usda.gov/nwcc/site?sitenum=920).

The two files for the two different SNOTEL sites (Blind Park and North Rapid) are in the metadata directory. The SWE data are available hourly, but the depth is only available at the beginning of the day, so I just included the daily data. If you would like hourly data for the SWE or anything else, let me know.

2019 Antarctica TObas

This field campaign was conducted by the British Antarctic Survey for the Thwaites MELT project. The flightlines were flown two times to measure change (primarily for basal melt). The first set of flightlines were flown in Dec 2019 and were then repeated one month later in Jan 2020.

Platform: Twin Otter

Tail Number: VP-FBL/MSN 839

2019 Antarctica TObas: Accumulation

Single channel accumulation radar.

2019 Antarctica TObas: GPS

BAS GPS data is referenced to GRS80 ellipsoid. Requires conversion to WGS84 in make_gps_2019_antarctica_TObas.m.

  if ~isempty(regexpi(gps_source,'bas')) && isempty(regexpi(out_fn,'gps_20191225_gnss'))
    % gps_20191225_gnss is skipped because gps_20191225 is taken care of
    % when gps_20191225_imu is produced.
    %
    % BAS GPS data are referenced to:
    % # ITRF2014(2020.07117)
    % # Ellipsoid: GRS80 (a=6378137.0000, f=1/298.25722210)
    % Convert to WGS84... the effect seems to be very very small (0.1 mm difference between the ellipsoids)
    
    warning('Converting from GRS80 ellipsoid to WGS84 ellipsoid: %s', out_fn);
    
    physical_constants;
    gps = load(out_fn);
    old_gps = gps;
    [x,y,z] = geodetic2ecef(gps.lat/180*pi, gps.lon/180*pi, gps.elev, GRS80.ellipsoid);
    [gps.lat,gps.lon,gps.elev] = ecef2geodetic(x, y, z, WGS84.ellipsoid);
    gps.lat = gps.lat*180/pi;
    gps.lon = gps.lon*180/pi;
    
    if 0
      figure(1); clf;
      plot(gps.lon, gps.lat);
      hold on;
      plot(old_gps.lon, old_gps.lat,'--');
      grid on;
      
      figure(2); clf;
      plot(gps.elev);
      hold on;
      plot(old_gps.elev,'--');
      grid on;
    end
    
    save(out_fn,'-append','-struct','gps','lat','lon','elev');

  end

2019 Antarctica TObas: system_dB

  • Transmit power: 400 W
  • Four element custom built Vivaldi antenna array for transmit gain: 9 dB at fc
  • Four element custom built Vivaldi antenna array for receive gain: 9 dB at fc
  • Standard terms for all radar systems with 50 ohm characteristic impedance: 20*LOG10(c/(8*pi*fc))+10*LOG10(50)
=10*LOG10(400)+9+9+20*LOG10(300000000/(8*PI()*(H3+I3)/2))+10*LOG10(50)

The estimated system_dB from this equation is 25.046703 dB.

2019 Antarctica TObas: radiometric calibration

Using the 20200128_01 deconvolution file, we found the return to be -79 dB at a twtt of 8.20e-6 (8.20e-6*c/2 = 1229 m). The expected 1/R^2 value is -20*log10(8.20e-6*c/2) = -61.792091 dB. The system_dB value is -20*log10(8.20e-6*c/2) - -79 = 17.2 dB to large. The estimated system_dB value was 25.04670259 and the new system_dB is 7.84 dB. A measurement is only possibly for waveform 1 since waveform 2 is saturated. The system_dB value for waveform 2 is therefore updated in the same way as for waveform 1.

Using a specular lead return on 20200128, we found the return to be -69.6 dB in the qlook data product at a system time delay of 8.263e-6 sec using waveform 1. Waveform 2 did not provide a value because the signal is saturated.

We desire to find the radiometric_corr_dB estimate that when added to the -69.6 dB return, the value is 10*log10(1/R^2) = -61.859. For our return, R = 8.263e-6*c/2.

The radiometric_corr_dB is found as 10*log10(1/(8.263e-6*c/2)^2) - -69.6 = 7.74 dB. The same value is used for both waveform 1 and waveform 2.

Overlap between the two waveforms in segment 20200127_01 was used to determine that waveform 2 was about 1.5 dB higher than waveform 1. The adc_gains_dB for waveform 2 was increased from 50.7 to 52.2 dB to account for this.

2019 Antarctica TObas: Image Combine

NEEDS TO BE UPDATED

For segment 20190205_01, we found clipping within the image combine. This was corrected by running run_update_img_combine using the code below.

params = ct_set_params(params,'cmd.generic',1,'day_seg','20190205_01');
mode = 'array'; % <== OFTEN CHANGED (qlook or array)

update_img_combine_param.out_path = 'standard';
update_img_combine_param.img_comb_mult = inf; % <== OFTEN CHANGED (inf default)
update_img_combine_param.img_comb_bins = 75; % <== OFTEN CHANGED (1 default)
update_img_combine_param.img_comb = [2e-6 -inf 1.2e-6]; % <== OFTEN CHANGED
update_img_combine_param.img_comb_layer_params = struct('name','surface','source','layerdata','layerdata_source','layerData');% <== OFTEN CHANGED

2019 Antarctica TObas: Coherent Noise

            params(param_idx).collate_coh_noise.method{img} = 'firdec';
            params(param_idx).collate_coh_noise.firdec_fs{img} = 1/7.5;
            params(param_idx).collate_coh_noise.firdec_fcutoff{img} = @(t) 1/30;
            
            %params(param_idx).collate_coh_noise.threshold_eval{wf} = 'threshold = max(min(-100,threshold + 20),10*log10(abs(noise.dft(:,1)).^2)+6);';
            if any(strcmp(params(param_idx).day_seg,{'20200127_01'}))
              if wf == 1
                params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold(:) = -150;';
              else
                params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold = max(nanmin(orig_threshold(time>Tpd+1.2e-6 & time<time(end)-Tpd))+6,coh_noise_est+20)-10;';
              end
              
            elseif any(strcmp(params(param_idx).day_seg,{'20200126_01','20200128_01','20191230_02'}))
              params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold = max(nanmin(threshold(time>Tpd+1.2e-6 & time<time(end)-Tpd))+6*ones(size(threshold)),max_filt1(max(threshold+6,10*log10(abs(dft_noise(:,1)).^2)+15)-1e6*(time>(Tpd+3.6e-6)),5));';
            elseif strcmp(params(param_idx).day_seg(1:6),'202001')
              params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold = max(nanmin(threshold(time>Tpd+1.2e-6 & time<time(end)-Tpd))+6*ones(size(threshold)),max_filt1(min(threshold+6,10*log10(abs(dft_noise(:,1)).^2)+15)-1e6*(time>(Tpd+1.2e-6)),5));';
            elseif strcmp(params(param_idx).day_seg,'20191215_03')
              if wf == 1
                params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold(:) = -145;';
              else
                params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold(:) = -149;';
              end
            elseif strcmp(params(param_idx).day_seg(1:6),'201912')
              if wf == 1
                params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold(:) = -145;';
              else
                params(param_idx).collate_coh_noise.threshold_eval{img} = 'threshold(:) = -155;';
              end
            end

2019 Antarctica TObas: Deconvolution

2019 Antarctica TObas: LIDAR

2019 Antarctica TObas: Processing

Data products:

  • CSARP_qlook: This product has coherent noise removal and deconvolution applied. There are 20 additional coherent averages and it is decimated by the same amount. This is followed by 10 incoherent averages by the same amount. Data are oversampled by 2 in the frequency domain (600 MHz).
  • CSARP_standard: This product has coherent noise removal and deconvolution applied. SAR focusing (f-k migration) has been applied with resolution 0.25 m. A moving average window of length 21 is applied in along-track and the data are decimated by 11. The final resolution is 21*0.25 = 5.25 m with a posting or sampling rate of 2.75 m in along-track. Data are over sampled by 2 in the frequency domain (600 MHz).

2020 South Dakota N1KU

2020 South Dakota N1KU: Snow Radar

2021 Antarctica TObas

2021 Antarctica TObas: Snow Radar

Personal tools
Namespaces

Variants
Actions
Navigation
Tools