Sim-to-Real Underwater Object Detection

Underwater object detectors are limited less by architecture than by the scarcity and cost of annotated real imagery. This dissertation evaluates target-aware, physics-informed DR for synthetic-to-real UOD under a deliberate constraint: detector weight updates use only synthetic images and SAM 3 pseudo-labels; no real annotations enter fine-tuning. Aggregate target statistics do inform two of the training regimes and earlier real-DUO scores informed the tuned recipe, so the reported figures are a target-informed evaluation, not an estimate from an untouched test set.

A five-stage pipeline generates 1,000 FLUX.2-dev benthic scenes, annotates them zero-shot with SAM 3, estimates relative depth with DAv2, and applies a forward Akkaynak-Treibitz-inspired transform (with a shared direct/backscatter coefficient) as the randomisation mechanism, in uncalibrated, DUO-calibrated and calibrated-plus-scatter regimes. Eight pretrained YOLO26x runs cross four data regimes with two training recipes and are evaluated on a three-class 778-image DUO derivative (Roboflow v2 export).

Under the baseline recipe, uncalibrated degradation was associated with real-domain mAP50 of 2.83% against Base 4.48%, while target-calibrated degradation reached 4.90%; under the tuned recipe the four regimes cluster between 10.31% and 11.82%, with the calibrated-plus-scatter run highest. Synthetic validation stayed at 98.1 - 98.8%, leaving a minimum 86.2% point transfer gap.

Because each cell is a single seed, clean and paired regimes differ in image exposure, and the evaluation partition influenced development, the regime differences are reported as exploratory associations; independent evaluation, matched exposure and repeated seeds are needed before attributing the observed differences to calibration.

Before the generative pipeline, a geometric route was built in Unreal Engine 5.8 on a licensed underwater asset pack: a benthic scene populated across four classes coral, kelp, rock, sponge a scripted camera path sampled at 500 poses, and per-actor 3D bounding boxes projected through the depth buffer for exact, occlusion-aware ground truth.

A detector trained on that output returned almost no detections on real underwater photographs. The labels were exact, but the imagery was optically clean: the route spent its effort on scene construction while leaving the actual transfer barrier of photometric realism untouched, and the classes it could depict were bounded by the asset pack rather than the benchmark. After seven weeks and two changes of direction over tooling failures, that result motivated the pivot to the generative pipeline, which defers optical appearance to its own controllable, calibratable stage. The simulator arm is retained as the UE Pipeline repository, the record of that pivot.

Scored on 778 real DUO images. Gap is synthetic-validation mAP50 minus real mAP50. Each cell is one seed-0 run.

RQ1. The nominally highest-scoring run reaches 11.82% mAP50 on the 778-image DUO derivative, while the transfer gap between synthetic and real never falls below 86.2 percentage points. Synthetic validation carried little descriptive information about real transfer in these eight runs and should not be used on its own as a selection criterion here.

RQ2. Under the baseline recipe the four regimes fell on both sides of the clean Base: uncalibrated 2.83%, calibrated 4.90%, Base 4.48%. The same separation is legible on the pixels - calibration moves ratio_rg from 0.944 to 0.705 against a target of 0.438, whereas the uncalibrated regime reaches only 0.790 and moves ratio_bg the wrong way. The reading is exploratory: volume, recipe bundling and single-seed variance all remain confounds.

RQ3. Sea urchin drives the aggregate rise across regimes; starfish is floored below 1.6% in every tuned run; box areas are around 4× oversized against real DUO: 3.6× for starfish, 5.3× for sea urchin, 1.1× for scallop. Photometric mismatch, morphology, scale and the SAM 3 pseudo-label ceiling all remain consistent with the evidence; none is separately measured.