Acoustic Data Processing

After data acquisition, we identify acoustic echos of schooling CPS using a semi-automated data processing algorithm using Echoview software and in-house Posit code in the estimATM repository. With Echoview, we extract the backscatter of swim bladder fish and process using soundspeed and echosounder calibration values housed inside an Echoview Calibration Supplement (.ecs) file. The Echoview filters and thresholds were based on a sub-sample of echoes from randomly selected CPS schools. We complete the processing with the estimATM package in extract_CPS_NASC.R, where we further refine the backscatter selection to extract only CPS.

This page has two parts. Part 1: How to Process details the steps acousticians should follow to process raw acoustic data for CPS. Part 2: What Happens During Processing details the concepts and behind the scenes technical steps that happen during processing. Here we will cover the Echoview and estimATM semi-automated processing workflow.

Part 1: How to Process

Below are google drive links to procedures for processing acoustic data (internal NOAA only):

A Quick Overview: The first portion of acoustic processing will take place in Echoview. As acoustician’s we will do the following tasks in Echoview:

  • Add data (.raw acoustic data, add Echoivew calibration files created from UCTD/CTD casts)

  • Remove bad data regions

  • Create Integration Stop/Start lines

  • Produce csv and echogram image files containing swim bladdered fish backscatter

The second portion of acoustic processing will take place using the nasc.R script in RStudio. As acoustician’s we will do the following in RStudio :

  • Select backscatter specific to CPS and remove non-CPS backscatter

  • Produce csv containing CPS-only backscatter and generalized echogram plots displaying regions removed/retained.

Overview of CPS Acoustic Processing. Vessel position and attitude data is used from the POS MV (not shown).

Part 2: What Happens During Processing

In Echoview, we organize, clean, and extract acoustic backscatter of swim bladder fish. There are three key steps to this process:

  1. Data wrangling, cleaning, and noise removal (including surface noise and seabed removal)

  2. Echo classification for swim-bladdred fish

  3. Acoustic Backscatter Integration. The aim of the filter criteria is to retain at least 95% of the noise-free backscatter from CPS while rejecting at least 95% of the non-CPS backscatter.

Simplified dataflow of processing in Echoview. *Surface noise and seabed removal replies on two manual inputs: the Integration Start and Integration Stop lines (not pictured, see step 5).

Section 1) Data Wrangling, Cleaning, and Noise Removal:

  1. Load RAW EK80 acoustic data and Echoview Calibration Supplement (.ecs)file into Echoview. The .ecs file contains soundspeed information calculated from the nearest (temporally and spatially) Underway CTD cast and echosounder calibration parameters. This file is used at the very beginning to convert from power to Sv (volume-backscattering coefficient).

  2. Align sampling rate (time and geometry) from all transducers to the 38 kHz Sv. Acousticians call this step ‘matching geometry’ of all Sv variables. Making sure pings are aligned from all echosounders is important for calculating the frequency response of backscatter in steps later on.

  3. Remove passive-mode pings.

  4. Noise removal: estimate and subtract background noise using the background noise removal function described in De Robertis and Higginbottom (2007).

  5. Surface noise and seabed removal is completed by manually drawing an Integration Start and Integration Stop line in Echoview. The Integration Start line is drawn at the shallowest depth to include surface CPS schools but exclude transducer ring down and surface noise due to sea state (typically around 5 meters below the transducer face or ~10m depth). The Integration Stop line is drawn closest to the seabed to include bottom dwelling animals but exclude any non-living seabed features (typically 3 m above the estimated seabed (Demer et al. 2009) or to the maximum logging range (e.g., 350 m), whichever is shallowest). When drawing the lines, we set the color scale to a minimum Sv threshold of -60 dB which corresponds to a density of approximately three 20-cm-long Pacific Sardine per 100 m3). Doing this helps visually pick out schools from the seabed and from non-swim bladder animals that appear as diffuse scattering layers in the water column. The area of the water column between the two lines sets the depth range that will be integrated for swim bladder fish in steps later on.

  6. Average the noise-free Sv echograms using non-overlapping 11-sample by 3-ping bins.

  7. Expand the averaged, noise-reduced Sv echograms with a 7 pixel x 7 pixel dilation. This replaces each averaged datapoint from Step 6 with the maximum datapoint surrounding it in a 7x7 pixel region.

Section 2) Echo Classification for Swim Bladder Fish

Calculate Frequency Response:

  1. For each dilated pixel, compute:

    Sv 200kHz − Sv 38kHz

    Sv 120kHz − Sv 38kHz

    Sv 70kHz − Sv 38kHz

    The difference between Sv values provides the frequency response for those pixels. Swim bladder fish have a unique frequency response which we can use extract those acoustic returns in the next step. Acoustic returns that fall within the Sv ranges below are flagged as meeting the criteria for typical swim bladder fish, including CPS.

  2. Create a Boolean echogram for Sv differences in the CPS range:

    −13.85 < Sv 70kHz − Sv 38kHz < 9.89

    − 13.5 < Sv 120kHz −Sv 38kHz < 9.37

    − 13.51 < Sv 200kHz − Sv 38kHz < 12.53

Calculate Standard Deviation:

  1. For 120 and 200 kHz, compute the squared difference between the noise-filtered Sv (remove passive pings) and averaged Sv (11-sample x 3 ping bin averages).

  2. Average the results using an 11-sample by 3-ping window to derive variance.

  3. Compute the square root to derive the 120- and 200-kHz standard deviations (σ120kHz and σ200kHz, respectively).

  4. Expand the standard deviation echograms with a 7 pixel x 7 pixel dilation (same step as Section 1, Step 7).

  5. Create a Boolean echogram based on the standard deviations in the CPS range:

    σ120kHz > -65 dB

    σ200kHz > -65 dB

    Diffuse backscattering layers have low σ (Zwolinski et al. 2010) whereas fish schools have high σ. Intersect the two Boolean echograms to create an echogram with “TRUE” samples for candidate CPS schools and “FALSE” elsewhere. Mask the noise-reduced echograms using the CPS Boolean echogram .

Section 3) Acoustic Backscatter Integration

  1. Integrate the volume backscattering coefficients (sV , m2 m-3) attributed to CPS over 5-m depths and averaged over 100-m distances;
  2. Output the resulting nautical area scattering coefficients (sA; m2 nmi-2) and associated information from each transect and frequency to comma-delimited text (.csv) files.

RStudio Processing Workflow

The exported .csv file from Echoview contains all swim bladder backscatter which can include non-target species such as rockfish. In order to further refine the acoustic classification to retain only CPS backscatter, the processed .csv file proceeds to the final semi-automated processing step in Posit using extract_CPS_NASC.R, an R-based tool in the estimATM package.

Echoes from fishes with swimbladders (blue points, scaled by backscatter intensity) along an example acoustic transect (top) and the corresponding Echoview echogram image (bottom). In this example, the upper (blue) and lower lines (green) indicate boundaries within which echoes were retained. When the lower boundary is deeper than the seabed (black line), echoes above the seabed are retained. Echoes from deep, bottom-dwelling schools of non-CPS fishes with swimbladders, and from diffuse scatters near the surface are excluded.

The script will open a plot (top) where the acoustics technician can draw new integration start line (blue) and stop line (green). Blue points that fall below the stop line will be excluded while blue points below the start line will be included from the resulting .csv file. The goal is to visually review the Echoview exported echogram image (bottom) and remove backscatter that you believe are not CPS (e.g., rockfishes, hake), possibly contain accidental seabed, or any surface noise amd diffuse scattering layers. The process of picking out CPS from other swim-bladder fish can be tricky as CPS can have a range of characteristics. The Backscatter Identification section goes into detail on this process.

Once you are happy with the two lines, an image will appear showing the results of your editing. If the backscatter needs to be removed, or put back, you can re-run the script and the results will be replaced.

The result is a final .csv with only backscatter information for CPS targets.

References

De Robertis, Alex, and Ian Higginbottom. 2007. “A Post-Processing Technique to Estimate the Signal-to-Noise Ratio and Remove Echosounder Background Noise.” ICES Journal of Marine Science 64 (6): 1282–91. https://doi.org/10.1093/icesjms/fsm112.
Demer, David A., George R. Cutter, Josiah S. Renfree, and John L. Butler. 2009. “A Statistical-Spectral Method for Echo Classification.” ICES Journal of Marine Science 66 (6): 1081–90. https://doi.org/10.1093/icesjms/fsp054.
Zwolinski, Juan P., Paulo B. Oliveira, Victor Quintino, and Yorgos Stratoudakis. 2010. “Sardine Potential Habitat and Environmental Forcing Off Western Portugal.” ICES Journal of Marine Science 67 (8): 1553–64. https://doi.org/10.1093/icesjms/fsq068.