Seminar Series_final

The HabCam V4: An
Overview of Its Uses
d bili iand Capabilities as a
Survey Tool
Northeast
Fisheries
Science Center
Survey Tool
Tasha O’Hara
Nicole Charriere
January 27, 2016
Nicole Charriere

Why is Ecosystem Monitoring Important?y y g p
Ecosystem monitoring can assist with:
Identification of ne en ironmental concerns• Identification of new environmental concerns
• Prioritization of issues
• Evaluation of trends over time
Quality information on ecosystem status and trends is important
in maintaining a healthy, sustainable environment and economy.g y y
• Prior surveying has primarily utilized fishing gear
• Fishing gear impacts habitat and species mortality
• Push from industry and regulators to design more ecologically neutral
tools
U.S. Department of Commerce | National Oceanic and Atmospheric Administration | NOAA Fisheries | Page 2

Why is Ecosystem Monitoring Important?y y g p
• Ecosystem health and monitoring are main foci
of NEFSC as outlined in the Center’s strategicof NEFSC, as outlined in the Center s strategic
plan.
H bC i i i i ht i t th i• HabCam gives a unique insight into the marine
monitoring by
• Provides a snapshot of the ecosystem in a
specific area at fixed moment in time
• Utilizes various environmental and acoustic
sensors.
sensors.

History of the HabCam Seriesy
Previous Versions
HabCam
• Developed 2002 by Woods Hole Oceanographic Institution (WHOI) and Industry (Arnie’s
Fisheries)
• Monitor seafloor and scallop bedsp
• For use aboard commercial vessel F/V Kathy Marie
HabCam V2
• Single camera vehicle designed in 2005 as
update to the original version
• Changes include:Changes include:
Two additional strobes
16-bit color camera
Vehicle frame alterations

History of the HabCam Seriesy
HabCam V4
D l d i 2012 f NEFSC S S ll S•Developed in 2012 for NEFSC Summer Scallop Survey
•Designed by WHOI with input from scallop industry
•Major improvements from previous versions include:
Teledyne Benthos C3D side-scan sensor - captures highly detailed bathymetric features
Stereo-camera system - allows images to be overlapped and processed in 3D
New software database to store metadata
Web-based annotation Graphical User Interface (GUI)
Sensors for measuring chlorophyll, turbidity, CDOM (Color Dissolved Organic Matter),
wavelength-specific light attenuation, dissolved oxygen, pH, and plankton
Additional space, ports for future hardware
Automated sea scallop and substrate classification software

HabCam V4 Vehicle Componentsp
Fiber-optic Cable with
Zip-tie Fairing
Rear
Towing Bridle
Stabilization Fins
Steel Frame
Front
Lead Weights
Fiber-optic Telemetry Bottle

Cameras

Strobes

Cameras and Strobes
• Cameras (x2)
- 1 4 megapixels (1360x1024 resolution)1.4 megapixels (1360x1024 resolution)
- Can capture up to 20 frames per second
- Mounted 22cm apart (75% overlap of images)p ( p g )
- Provide approximately 1m2 field of view
- Housing rated to depth of 500m
• Strobes (x4)
- Short-duration, high-intensity pulses
- Synchronized with cameras
- Freeze motion, eliminate blur, and enhance image quality

Side-scan Sonar (C3D)

A-Spherep
ECO Triplet
CTDs
Altimeter
CPICS Bracket

Image Processing
Along with each image, related
environmental and acoustic sensor
d t i t d d t d
g g
data is captured and stored.
All images and associated
d t th h i t fdata go through a variety of
processes and corrections
for future assessment and
ti t dicomparative studies.
Image courtesy of Scott Gallager, WHOI

Image Processingg g
Each paired image gives a snapshot
of the habitat of a specific area at aof the habitat of a specific area at a
given point in time.
After annotations, associated
information includes:
i l ifi ti•organism classifications
•location
•counts
•lengths•lengths
•behaviors
•sediment type
•environmental sensor data
Above: processed and corrected image ready for annotation
environmental sensor data

Image Processingg g
The image capture frequency provides a 50% overlay of images,
which allows us to build mosaics of the seafloor. These mosaics,,
combined with side-scan data and environmental sensors, provides
a more precise method for extrapolating habitat diversity
and composition for up to a 100m swathand composition for up to a 100m swath.

Image Processing: Storageg g g
• NEFSC- houses 120TB storage drive
• ~3TB data stored per day on survey3TB data stored per day on survey
• Typically collect 40-60TB per survey
• Due to increasing annual data collection, likely require new,Due to increasing annual data collection, likely require new,
much larger drive <5 years
• Petabyte drive suggested by WHOI for future storage
For perspective:
A Terabyte could hold about 300 hours of good quality videoA Terabyte could hold about 300 hours of good quality video.
A Petabyte could fill >220,000 DVDs of data
3 PB is ~the entire digitized collection of the Library of
Congress, including all photos, audio, publications

HabCam V4 Survey Usagey g
UNOLS R/V H h R Sh
UNOLS R/V Hugh R. Sharp

HabCam V4 Surveysy
T d t V4 h b d i thTo date, V4 has been used in three surveys
1) NEFSC Scallop Survey (annual)
2) NEFSC Maryland Wind Energy Area and Black
Sea Bass Survey (July 2013)y ( y )
3) NEFSC Yellowtail Survey (October 2014)

NEFSC Scallop Surveyp y
HabCam V4 image of swimming sea scallops

NEFSC Scallop Survey Overviewp y
• Pre-cruise
- Camera calibration
- Fiber-optic termination
- Staging/Onload
• Cruise• Cruise
- General operations and protocols
- Image processing
- Annotations
• Post-cruise
- Destaging/OffloadDestaging/Offload
- Data backup
- Sensor calibrations
A t ti if d d- Annotations, if needed

NEFSC HabCam Collaborators
Pre-Cruise
DuringDuring
Post-CruisePost Cruise

Pre-Cruise: Camera Calibration
• Performed before every survey
• Intrinsic and extrinsic parameters to consider• Intrinsic and extrinsic parameters to consider
- Intrinsic: Distance between camera lens and an
internal light-sensitive circuit that converts theinternal, light sensitive circuit that converts the
image of the seafloor into an electrical charge
- Extrinsic: Orientation of the vehicle (pitch, roll) and(p , )
altitude
• Knowing the mathematical relationship between theg p
intrinsic and extrinsic parameters is essential in
correcting for in-water focal length, image center, and
pixel errors
pixel errors

HabCam V4 in WHOI dunk tank

Camera calibration with checkerboard target

• All images are captured and written to driveAll images are captured and written to drive
• Images analyzed at WHOI in MATLAB Calibration
Toolbox
• Values and error ranges obtained from Calibration
Toolbox are then utilized to correct images at seag
• If camera positions are changed in any way, such as in
an at-sea removal or a post-cruise repair, the
calibration must be repeated

Pre-Cruise: Fiber-Optic Terminationp

• V4’s camera and sensor information is converted by
a chip from Ethernet signals to light beams whicha chip from Ethernet signals to light beams, which
are sent along a fiber-optic winch cable on vessel
I d t d t th fib ti i h• In order to pass data, the raw, fiber-optic winch
cable must be stripped of metal armor jacket to
l fib d t i t d t th hi l dreveal fiber cores and terminated at the vehicle end

Metal armor jacket stripped to reveal fiber-optic cores
Image courtesy of ThorLabs
Metal armor jacket stripped to reveal fiber-optic cores

• Connectors are
applied to core endsapplied to core ends
and polished to allow
for clean signal
passage
• Connectors attached
d t d i il fill dand stored in oil-filled
junction box, which
also provides poweralso provides power
to the vehicle
Fib ti t
Fiber-optic connectors

Scientists attaching zip-tie fairing to fiber-optic cable

Pre-Cruise: Staging/Onloadg g
• Vehicle, associated equipment, and supplies loaded
onto R/V Hugh R Sharp for setuponto R/V Hugh R. Sharp for setup
- Two, 45TB servers and over 10 other monitors
d t f il t d il t t tiand computers for pilot and co-pilot, annotation,
side-scan, plankton imaging, and navigational
t tistations
- Extra vehicle hardware, spare camera and strobe
equipment, spare cables and wiring, toolboxes,
electrical tape, zip ties, etc.

Pre-Cruise: Staging/Onload
Lab setup in Sharp dry lab

Cruise Track
• HabCam V4 follows
random, computer-p
generated, and slightly
modified zig-zag patterns
• Areas of operationp
- Mid-Atlantic Bight
- Southern New England
Georges Bank- Georges Bank
• Track uploaded into
Nobeltec navigational
software for lab and vesselsoftware for lab and vessel
command to follow2013 HabCam Cruise Track

Cruise: Deployment
Crew of R/V Hugh R. Sharp preparing to deploy vehicle

Cruise: Deployment
Vehicle off deployment ramp and in water

Cruise: Pilot Training
• Monitoring
Winch control
Winch camera
monitor
Cruise track monitorSide-scan sonar monitor
- Winch control
- Cruise track
Side scan
monitor
Vehicle depth and altitude Image quality - Side-scan
- Vehicle depth
and altitude
Vehicle depth and altitude Image quality
and altitude
- Image quality
• Vertical profiles• Vertical profiles
• Position rotatesWinch control box
Pil t t ti
Pilot station

Cruise: Co-pilot Trainingp g
• Monitoring
- Cameras
- Disk storage
- Image quality
- Image notationsg
- Environmental
sensor readouts
• Position rotates
C il t t ti
Co-pilot station

Cruise: Annotator Trainingg
• Post-processing image
analysis
- Sediment type
- Scallop
measurementsmeasurements
- Presence of
fish/skates/dust
cloudsclouds
- Image highlights
- Image quality (goodg q y (g
vs. poor)
• Position rotatesAnnotation stations

Post-Cruise Operationsp
• Destage/offload all equipment and supplies on vessel
All d d i b k d• All raw and processed images are backed up on
land-based server
• Environmental sensors (CTDs, A-Sphere, and ECO
Triplet) removed from vehicle and sent out for
calibration to ensure accuracy of readings
• Further annotations (if necessary)( y)

2015 NEFSC Scallop Survey Resultsp y
• Travelled over 4500km on cruise track
• Collected 9.2 million stereo-image pairs and over
30TB of data, all of which were live-processed at sea30 o da a, a o c e e e p ocessed a sea
• Annotated a subset of 60,000 images at sea
- 85% of the number of images requested by NEFSC- 85% of the number of images requested by NEFSC
Population Dynamics Branch for 2015 scallop
assessmentassessment

Web-based Annotation GUI
• Designed by WHOI to improve annotations and classifications of
species and substrate
Abilit t t t i f t t d t th• Ability to annotate images from any computer connected to the
network, including those at sea
• Stable application, but able to update for efficiency- grows with
dsurvey needs
• Users are given training based on standard practices and tested
prior to annotating incoming data.
• Manually identify hundreds of taxonomic categories, classify
substrate, quantify habitat coverage, describe specific scallop
behaviors, and note possible fish avoidance eventsp
• Classify photos as unusable if necessary, and to alter lighting as
necessary to render the images usable

Annotation Rules: Some Scallop Sub-categoriesp g
Sea Scallop Category Breakdown
• “Live” – two valves together. Depression or tentacles may beg p y
present.
• “Clapper”- two valves together and gaping
• measure umbo to end of bottom shell (flat white side)• measure umbo to end of bottom shell (flat, white side)
• “Swimming” - when a scallop is above the seafloor, look for
shadows
• Important- measurements need to be adjusted
• “Probable”- unsure of live or dead classification
• “Inexact” unable to measure length or width due to object• Inexact - unable to measure length or width due to object
being cut off on border or partially obscured

Annotation Rules: Live Scallop Lengthsp g
• Live scallop lengths should
t t t th b d t d
Shadow of depression
in sediment
start at the umbo and extend
to the end of the shell.
• Width may be taken if
unable to determine shell
heightheight.
• Seed scallops may be
marked with point due to
size.
Slight gap and tentacles denote “live”

Annotation Rules: Scallop Countingp g
For counting scallops with 50% visible in
image, use “L” rule.
S ll l th “L” f th i (l ft id
All scallops with >50% visible in
image should be counted Scallops along the “L” of the image (left side
and bottom) should NOT be counted.
image should be counted
regardless of position.

Annotation Rules: Scallop Countingp g
• Those with 50% along the right and top sides should be counted.
Thi th d d th h f ti ti th l ti• This method reduces the chances of overestimating the population.

Annotation Rules: Dead Scallop Protocolsp
Lengths are not currently taken for dead sea scallops
Below: Scallop shell clearly
broken/damaged
Above: the hinge and muscle
indentation are visible inside a
dead shell.
Above: dark blueish/black
coloration typically
indicates dead scallop shell

Annotation Rules: Other Categoriesg
Fish categories may include: unidentified fish, flatfish, roundfish, or skate
• use box tool around the fish
• Fish annotations revisited by second annotator at later date- classified
at species level
Highlights (good examples rare species unusual phenomenon interestingHighlights (good examples, rare species, unusual phenomenon, interesting
image)
• highlight is in image note- may be 1-3 stars
Other organisms of interest that should be noted:
• Didemnum -measure with bounding box of some of it in the image
“Jonah or rock crab” measure with bounding box• “Jonah or rock crab”- measure with bounding box
• “Waved whelk”- measure with bounding box
“Dust cloud” - when there is a poof of sediment visible- use bounding boxDust cloud when there is a poof of sediment visible use bounding box

Four
shadows
>50%

NEFSC Maryland Wind Energy Area
and Black Sea Bass Surveyand Black Sea Bass Survey
HabCam V4 image of black sea bass and surrounding habitat

NEFSC Maryland Wind Energy Area
and Black Sea Bass Surveyand Black Sea Bass Survey
• Conducted July 2013 on board R/V Hugh R. Sharp – 5 days
• Area of operations: Continental shelf off Delmarva PeninsulaArea of operations: Continental shelf off Delmarva Peninsula
• Assessment of fish and habitat distribution within the
Maryland Wind Energy Area, and for assessment of
untrawlable habitats (reefs) occupied by black sea bass
• Towed along grid patterns
S r e collected 1 57 million stereo image pairs• Survey collected 1.57 million stereo-image pairs
• Both side-scan (V4) and multibeam (Sharp) data were
analyzed to compare and support image findingsanalyzed to compare and support image findings
• Black sea bass were located only in hard bottom

NEFSC Yellowtail Survey
HabCam V4 image of swimming yellowtail flounder

NEFSC Yellowtail Surveyy
• Conducted October 2014 on
board R/V Connecticut – 5
daysdays
• Area of operations: southern
flank of Closed Area II on
eastern Georges Bankeastern Georges Bank
• Three-vessel, gear-to-
technology comparison
surveysurvey
- R/V Connecticut
- F/V Hera
- NOAA Ship Henry B.
Bigelow

NEFSC Yellowtail Surveyy
• Visual distribution and relative abundance of
yellowtail flounder and other flatfishyellowtail flounder and other flatfish
• Connecticut towed V4 along parallel transects
0 9 t0.9nm apart
• Hera and Bigelow both equipped with trawl gear
and assigned paired, station locations in and among
V4 transects

Future Applications: Habitat Monitoringpp g
• Data viewed in real time and stored for further analysis
• Allows for long term monitoring of habitats and• Allows for long-term monitoring of habitats and
organisms
• Provides data for short-term, regulatory responses, g y p
• HabCam V4 can identify “hot spot” areas of dense
scallop seed
T i l t t t t b d til• Trigger regulatory responses to protect beds until
harvestable size
• Benefits stock survival and commercial interestsBenefits stock survival and commercial interests
• Identify new and emerging habitats for a variety of
species

Emerging Lobster Habitat
• Summer 2016-HabCam V4 will investigate lobster habitat in the Atlantis and
Al i CAlvin Canyons
• Depth range: ~125-250m
• Identify possible shifts in lobster habitats and scope of new habitat
• Aim to monitor and mitigate possible climate change effects on inshore
lobster habitat
Spread of Invasive SpeciesSpread of Invasive Species
• Map abundance and spread of organisms, including invasive species
• WHOI has used HabCam imagery to identify invasive tunicate species,
Didemnum vexillum
• Continuous monitoring to track its spread and effect on the density of
several benthic organismsg

Correlations between Side-scan and Multibeam Data
• Combining multibeam data with C3D data creates a more accurateCombining multibeam data with C3D data creates a more accurate
3D depiction of the seafloor
• Allows data users to track major and minor changes to these regions
• Track annual bathymetric shifts due to weather patterns or major• Track annual bathymetric shifts due to weather patterns or major
weather events
• Beneficial for tracking physical habitat shifts
H bC V4 C3D id h ll t d i t t i• HabCam V4 C3D side-scan sonar has collected a consistent series
of fine-scale bathymetric data in highly fished areas
• Multibeam information available from various research vessels and
recently available on R/V Hugh R. Sharp
• Assists in pilot navigation- ability to foresee bathymetric shifts
• Limited multibeam use on Sharp- cost prohibitiveLimited multibeam use on Sharp cost prohibitive

Future Applications: Assessmentspp
Fish Population Assessments
• Push to integrate HabCam data into fish stock assessments• Push to integrate HabCam data into fish stock assessments
• HabCam data not yet incorporated into fish stock assessments to
date
• NEFSC may attempt to incorporate HabCam data into skate
assessments as early as 2016-17
• HabCam ideal tool for untrawlable areas, including rocky or coral
dense habitats
• Vehicle assesses fish diversity and density and habitat interactions• Vehicle assesses fish diversity and density, and habitat interactions
• Monitor commercially important fish living in unsuitable trawl areas
such as redfish, black sea bass, and cusk

Future Applications: Assessmentspp
Fish Avoidance Studies
Fish avoidance a major concern withFish avoidance a major concern with
incorporating HabCam data into fish
assessments
• Dust cloud annotations concrete
evidence of fish evasion
f f• Establish frequency of evasion
phenomenon
• Relative abundance calculations• Relative abundance calculations
• Potential to incorporate front-facing
cameras will improve studiescameras will improve studies

Acknowledgementsg
NEFSC Center Staff who make our HabCam work possible:
Russell Brown, Wendy Gabriel, Bill Karp, Robert Johnston
NEFSC Ecosystems Surveys Branch Shellfish Team:
Michael Bergman, Larry Brady, Jonathan Duquette, Joe Godlewski, Nancy McHugh, Victor
Nordhal
WHOI HabCam Team:
Joe Futrelle, Scott Gallager, Peter Honig, Jon Howland, Steven Lerner, Jared Schwartz, Amber
YorkYork
NEFSC PopDy Scallop Assessment Group:
Jui-Han Chang, Dvora Hart, Burton Shank, Jiashen Tang
NEFSC Oceanography Branch:
Tamara Holzwarth-Davis
V l t ff th R/V H h R Sh d R/V C ti tVessel staff on the R/V Hugh R Sharp and R/V Connecticut

HabCam Seminar Schedule
Date Time Location Speaker Affiliation Seminar Title
The HabCamV4 Program: An Overview of
January
13th 2016
12:00‐
13:00
Clark
Conference
Room
Scott
Gallager WHOI
The HabCamV4 Program: An Overview of
Scallops, Groundfish, Habitat Characterization
and Other Uses of Data from the HabCamV4
System
January
27th 2016
12:00‐
13:00
Clark
Conference
Room
Tasha O'Hara
and Nicole
Charriere NEFSC
The HabCam V4: An Overview of Its Uses and
Capabilities as a Survey Tool
Clark Dvora Hart/ Design population estimation and use of
Feb 3rd
2016
12:00‐
13:00
Clark
Conference
Room
Dvora Hart/
Jui‐Han
Chang NEFSC
Design, population estimation and use of
automated image analysis for Habcam sea
scallop surveys
Clark Jon Duquette
Feb 10th
2016
12:00‐
13:00
Conference
Room
and Michael
Bergman NEFSC
A Brief Overview of the NESFC Annual Sea
Scallop Survey: the Past, Present, and Future
12 00
Clark
Conference Scott
TBD
12:00‐
13:00
Conference
Room
Scott
Gallager WHOI TBD

Seminar Series_final

Recommended

Recommended

More Related Content

What's hot

What's hot (11)

Viewers also liked

Viewers also liked (10)

Similar to Seminar Series_final

Similar to Seminar Series_final (20)

Seminar Series_final