Fish Detection with Transfer Learning

A YOLOv7-based fish detection model that investigates the impact of transfer learning on detecting small fish in underwater images with murky water conditions.

Overview

This project explores how transfer learning can improve fish detection models for challenging underwater conditions, specifically:

• Small-sized fish that are difficult to distinguish from backgrounds
• Murky water with low illumination
• Complex underwater backgrounds

Dataset

The project uses the OzFish dataset, which contains:

• 43,000 bounding box annotations across 1,800 frames
• Average of 25 fish per image
• Images of small fish in murky water conditions
• Collected by the Australian Research Data Commons Data Discoveries program

Model Architecture

Built on YOLOv7, the project evaluates three model variations:

1. Pretrained Model: YOLOv7 pretrained on a large, open-source fish dataset
2. Transfer Learning Model: Pretrained model fine-tuned on OzFish dataset
3. From Scratch Model: YOLOv7 trained solely on OzFish data

Key Findings

Impact of Transfer Learning

• Transfer learning models performed similarly to pretrained models (~0.5% difference in Mean Precision, ~0.8% in Mean Recall)
• Models trained from scratch underperformed compared to transfer learning approaches
• The model learns to better identify fish from backgrounds but still struggles with images containing many fish

Impact of Data Quantity

• More data = better performance in transfer learning
• Transfer learning with 25% and 50% of dataset performed worse than from-scratch models
• Transfer learning with 75%+ of dataset outperformed from-scratch models
• Best results achieved with 88-100% of augmented training data

Results

Overall Model Performance

Model	Mean Precision	Mean Recall
Pretrained (no transfer learning)	0.7628	0.9064
Transfer Learning (no augmentation)	0.7365	0.7323
Transfer Learning (with augmentation)	0.7581	0.9140
From Scratch (no augmentation)	0.7408	0.7312
From Scratch (with augmentation)	0.7366	0.7314

Impact of Training Data Amount

Model	Mean Precision	Mean Recall
From scratch w/ 100% training data	0.7366	0.7314
Pretrained, no Transfer Learning	0.7628	0.9064
Transfer Learning w/ 25% training data	0.6848	0.6902
Transfer Learning w/ 50% training data	0.7193	0.7203
Transfer Learning w/ 75% training data	0.7472	0.7332
Transfer Learning w/ 88% training data	0.7521	0.8379
Transfer Learning w/ 100% training data	0.7581	0.9140

Key Observations:

• Mean Precision: Models showed similar precision values across variations
• Mean Recall: Significant differences, indicating some models captured more fish present in images
• Best performance achieved with maximum data usage in transfer learning (88-100%)

Find full technical report here: Transfer Learning for Fish Detection in Underwater Images

Future Work

• Train on additional datasets containing small fish in murky water
• Collaborate with ecological researchers to create custom datasets
• Explore alternative object detection architectures:
  • SSD
  • Faster-RCNN
  • Advanced YOLO versions

Applications

Fish detection in underwater images is crucial for:

• Ecological research
• Understanding aquatic habitats
• Marine biodiversity monitoring

Acknowledgments

Based on public code from: YOLOv7 training

References

1. Muksit, A. A., et al. (2022). YOLO-Fish: A robust fish detection model to detect fish in realistic underwater environment. Ecological Informatics, 72, 101847.

2. Jalal, A., et al. (2020). Fish detection and species classification in underwater environments using deep learning with temporal information. Ecological Informatics, 57, 101088.

Contributors

• Ishvi Mathai - Stanford University
• Humishka Zope - Stanford University