Backscatter mosaics of the seafloor are now routinely produced from multibeam sonar data, and used in a wide range of marine applications. However, significant differences (up to 5 dB) have been observed between the levels of mosaics produced by different software processing a same dataset. This is a major detriment to several possible uses of backscatter mosaics, including quantitative analysis, monitoring seafloor change over time, and combining mosaics. A recently concluded international Backscatter Working Group (BSWG) identified this issue and recommended that “to check the consistency of the processing results provided by various software suites, initiatives promoting comparative tests on common data sets should be encouraged […]”. However, backscatter data processing is a complex (and often proprietary) sequence of steps, so that simply comparing end-results between software does not provide much information as to the root cause of the differences between results. In order to pinpoint the source(s) of inconsistency between software, it is necessary to understand at which stage(s) of the data processing chain do the differences become substantial. We have invited willing software developers to discuss this framework and collectively adopt a list of intermediate processing steps. We provided a small dataset consisting of various seafloor types surveyed with the same multibeam sonar system, using constant acquisition settings and sea conditions, and have the software developers generate these intermediate processing results, to be eventually compared. If the experiment proves fruitful, we may extend it to more datasets, software and intermediate results. Eventually, software developers may consider making the results from intermediate stages a standard output as well as adhering to a consistent terminology, as advocated by Schimel et al. (2018). To date, the developers of four software (Sonarscope, QPS FMGT, CARIS SIPS, MB Process) have expressed their interest in collaborating on this project.

1. Backscatter Working Group Software Inter-comparison Project
Requesting and Comparing Intermediate Backscatter Processing Results
Project Facilitators:
Mashkoor Malik, NOAA, USA
Marc Roche, ECONOMIE, Belgium
Julian Le Deunf, SHOM, France
Giuseppe Masetti, CCOM UNH, USA
Alexandre Schimel, NIWA New Zealand
Margaret Dolan, NGU, Norway
Shallow Survey Conference
1 October 2018
Project Collaborators:
SonarScope, IFREMER
FMGT, QPS
HIPS & SIPS, Teledyne CARIS
MB Process, Curtin University, CMST

2. Why?
● Validation of backscatter processing steps
○ Using same data set, may result in different final products
○ Need input from developers to document the processing chain better
● Backscatter working group (BSWG, 2015) recommended:
“to check the consistency of the processing results provided by various software suites,
initiatives promoting comparative tests on common data sets should be encouraged [...]” (Lurton
and Lamarche, 2015: BSWG report ).
Proprietary
ProcessingRaw data Mosaic
Angular
response

3. Example of end user frustration !
(Mosaic to mosaic comparison)
End result of backscatter mosaic offers little insight as
What went wrong and where?
Lucieer et al. (2017); Roche et al. (2018)

4. What are we proposing?
A collaborative project between data users and software developers:
● Goal 1: Collectively come to an agreement over processing stages and
reporting of intermediate results;
○ Launched project at GeoHab 2018 (May 2018)
○ A mini-workshop with software developers conducted (June 2018)
○ Provided common data set (June 2018)
○ Software vendors provided processing results (September 2018)
● Goal 2: To identify discrepancies in the processing chain and explain
differences in results when processed with different processing tools;
○ Same dataset processed by software developers
○ Compare intermediate processing results
● Goal 3: To develop consensus among software developers about adopting a
standard processing chain, nomenclature and metadata.

5. Backscatter data processing sequence
(Schimel et al. 2018)

6. Backscatter data processing sequence
(Schimel et al. 2018)
Current outputs

7. Current outputs
Current outputs

8. BSWG recommended processing stages
Current outputs
Recommended outputs
Metadata
Metadata

9. BSWG recommended processing stages
Current outputs
Recommended outputs
Metadata
Metadata
Available Sept 2018
CARIS and Sonar
Scope (Augustin
talk in this session)
implemented
(some)
intermediate
results output

10. The inter-comparison dataset
Kongsberg EM302
Johnston Atoll area
NOAA - USA
Teledyne 7125
Wembury Bay UK
Shallow survey 2015
Kongsberg EM710 and EM3002
Carré Renard reference area
SHOM - FRANCE
Kongsberg EM2040
Kwinte reference area
FPS Economy - BELGIUM

11. Phase I: Request intermediate results from software
ASCII export in Ping Beam geometry with backscatter as provided by sonar and
processed results prior to mosaicking (BL0 and BL3)
Vessel Data
acquisition
Agency Location Weather Date Depth
range
EM 2040 RV Simon Stevin SIS FPS
Economy
Kwinte reference area Calm Dec
2016
23-26 m
EM 3002 HSL Guillemont SIS SHOM Carre Renard area,
Brest Bay, France
Calm Jan
2010
18-22 m
EM 710 BH2 Borda SIS SHOM Carre Renard area,
Brest Bay, France
Calm Feb
2013
18-22 m
EM 302 Okeanos
Explorer
SIS NOAA Johnston Atoll near
Hawaii, USA
Rough Aug
2017
~3000 m
Reson
7125
s7k Shallow
survey
2015
Plymouth, UK Calm 2015 < 10 m

12. Processing status (Sept 2018)
Software KB 710 KB 302 KB 3002 KB 2040 Reson 7125
Sonar Scope
FMGT
CARIS
Curtin
Hypack
Swath Ed
MB System
Available No results Awaiting confirmation

13. Backscatter results provided by different vendors
Software SonarScope FMGT CARIS Curtin
# columns 31 12 17 11
Time stamp Ping Time Timestamp Ping Time
Ping # Ping
First ping = 0
Ping Number
First ping = 1
Ping Ping Number
Beam # Beam
First beam = 1
Beam Number Beam Beam Number
Beam location (Lat / Long) Latitude/Longitude Latitude /Longitude Longitude /Latitude Longitude / Latitude
Beam location (E / N) GeoX / GeoY Easting / Northing Easting / Northing Easting / Northing
Beam depth BathyRT Depth Depth
Incidence angle IncidenceAngles True Angle IncidentAngle Incidence Angle
BS as read from data files ReflecKM Backscatter Value BLO Backscatter value
BS processed angular response ReflecSSc Corrected Bacskcatter Value BL3 Corr Backscatter
Value

14. RV Simon Stevin EM2040 single Rx L0005_20160412_104116.all (300kHz, 216µs)
Kwinte reference gravel and sandy gravel area
black dots = GIS beam
plot (lat long)
white line =
navigation
yellow =
calculation
area
Processing of data from Sonarscope, FMGT and
CARIS:
1. GIS integration using Lat Long associated with each
ping/beam
2. Extraction of the data inside the yellow calculation
area (homogeneous area, bs reference subarea!)
3. Per beam number, calculation of the average value
of the different backscatter provided:
• SonarScope:
• ReflecKM
• ReflecSSc
• FMGT:
• Bsvalue
• Bscorvalue
• CARIS:
• BL0
• BL3

15. RV Simon Stevin EM2040 single Rx L0005_20160412_104116.all (300kHz, 216µs)
Kwinte reference gravel and sandy gravel area BoxPlot : outliers 25% 50% 75%
outliers
1
%
99
%
(A selected area)
-17 dB
-15 dB
- 11.5 dB

16. (First 100,000 data points)

17. Why is CARIS incidence angles different ?

18. Within 5 cm
RV Simon Stevin EM2040 single Rx L0005_20160412_104116.all (300kHz, 216µs)
EM 2040 Depth (One Ping)

19. Within 15 dB
RV Simon Stevin EM2040 single Rx L0005_20160412_104116.all (300kHz, 216µs)
EM 2040 Backscatter (One Ping)

20. RV Simon Stevin EM2040 single Rx L0005_20160412_104116.all (300kHz, 216µs)
Kwinte reference gravel and sandy gravel area

21. EM302 L0213_20170717_112534_EX1706.all (main frequency =27.75kHz, pl = 40ms)
Johnston Atoll area
Processing of data from Sonarscope, FMGT and
CARIS:
1. GIS integration using Lat Long associated with each
ping/beam
2. Per beam number, calculation of the average value
of the different backscatter provided using all the
data:
• SonarScope:
• ReflecKM
• ReflecSSc
• FMGT:
• Bsvalue
• Bscorvalue
• CARIS:
• BL0
• BL3
Note: considering the heterogeneity of the seabed, a sub
calculation area should be select in order to keep a good
coherency of the statistics.
SSc Reflec KM

22. EM302 L0213_20170717_112534_EX1706.all (main frequency =27.75kHz, pl = 40ms)
Johnston Atoll area
(One survey line)
- 26.5 dB
- 26 dB
- 22 dB
- 21.5 dB
- 27.2 dB
- 24 dB
BL0 BL3

23. EM302 L0213_20170717_112534_EX1706.all (main frequency =27.75kHz, pl = 40ms)
Johnston Atoll area

24. EM302 L0213_20170717_112534_EX1706.all (main frequency =27.75kHz, pl = 40ms)
Johnston Atoll area
(One survey line)

25. (First 100,000 data points)
EM302 L0213_20170717_112534_EX1706.all (main frequency =27.75kHz, pl = 40ms)
Johnston Atoll area

26. EM710 0002_20130214_091514_borda.all (main frequency=81kHz, pl=0.3ms)
SHOM reference area
Processing of data from Sonarscope, FMGT and CARIS:
1. GIS integration using Lat Long associated with each
ping/beam
2. Per beam number, calculation of the average value of the
different backscatter provided using all the data:
• SonarScope:
• ReflecKM
• ReflecSSc
• FMGT:
• Bsvalue
• Bscorvalue
• CARIS:
• BL0
• BL3
Note: Due to the high homogeneity of the seabed in the Carré
Renard area, defining a sub area is not required.
FMGT Mosaic
(beam time series, default settings)

27. EM710 0002_20130214_091514_borda.all (main frequency=81kHz, pl=0.3ms)
SHOM reference area
-19.3 dB
-14.5 dB
- 10.4 dB

28. EM710 0002_20130214_091514_borda.all (main frequency=81kHz, pl=0.3ms)
SHOM reference area

29. EM710 0002_20130214_091514_borda.all (main frequency=81kHz, pl=0.3ms)
… area

30. EM710 0002_20130214_091514_borda.all (main frequency=81kHz, pl=0.3ms)
SHOM reference area

31. RESON 7125 20140729_082527_SMB Owen.s7k
Shallow Survey 2015
Processing of data from Sonarscope, FMGT and Curtin
1. GIS integration using Lat Long associated with each
ping/beam
2. Per beam number, calculation of the average value
of the different backscatter provided using all the
data:
• MB System :
• BL0
• BL3
• FMGT:
• Bsvalue
• Bscorvalue
• CARIS:
• BL0
• BL3
Reson 7125 area
FMGT Mosaic

32. 40 dB
- 55 dB
40 dB
RESON 7125 20140729_082527_SMB Owen.s7k
Shallow Survey 2015

33. RESON 7125 20140729_082527_SMB Owen.s7k
Shallow Survey 2015

34. Discussions needed
1. Large unexplained differences among results BL0 and BL3
2. An initial step to show that differences exist
3. More work needed to identify reasons of these differences:
Sonar dependent?
Depth dependent?
Differences in processing methodology?
4. Involvement of sonar manufacturers to clarify data acquisition assumptions?

35. Next Steps
● Revisit calculations of BL0
● Request other intermediate
processing results
● 2nd round of processing after
agreeing upon the processing
approaches
● An independent assessment of
processed results
● Other software ?

36. We need your support !!
● Users: To demand results processed by different tools should agree
reasonably with each other
○ Accuracy standards of backscatter still distant goal
● Software developers: To work together to implement best practices for
backscatter processing
● BSWG: Provide a platform to facilitate these discussions

37. Questions
Mashkoor Malik (mashkoor.malik@noaa.gov)
Alexandre C. G. Schimel (alexandre.schimel@niwa.co.nz)
Marc Roche (Marc.Roche@economie.fgov.be)
Giuseppe Masetti (gmasetti@ccom.unh.edu)
Margaret Dolan (Margaret.Dolan@ngu.no)
Julian Le Deunf (julian.le.deunf@shom.fr)
Thanks to software developers & BSWG

38. References cited
Lamarche G, Lurton X. 2017. Recommendations for improved and coherent acquisition and processing of backscatter data from seafloor-mapping
sonars. Mar. Geophys. Res. 39:5-22.
Lucieer V, Roche M, Degrendele K, Malik M, Dolan M, Lamarche G. 2017. User expectations for multibeam echo sounders backscatter strength
data-looking back into the future. Mar. Geophys. Res. 39:23-40.
Lurton X, Lamarche G. 2015. Backscatter measurements by seafloor-mapping sonars. Guidelines and recommendations. Retrieved 27th April 2018
from http://geohab.org/wp-content/uploads/2013/02/BWSG-REPORT-MAY2015.pdf.
Roche M, Degrendele K, Vrignaud C, Loyer S, Le Bas T, Augustin J-M, Lurton X. 2018. Control of the repeatability of high frequency multibeam
echosounder backscatter by using natural reference areas. Mar. Geophys. Res. 39: 89-104.
Schimel ACG, Beaudoin J, Parnum IM, Le Bas T, Schmidt V, Gordon K, Ierodiaconou D. 2018. Multibeam sonar backscatter data processing. Mar.
Geophys. Res. 39:121-137.