Rig calibration - #19
Merged
Merged
Conversation
Convert pyproject.toml to a standard [project] table with the hatchling build backend, raising the supported Python floor to 3.10 (the version the postproc image runs) and dropping the per-dependency version markers that floor made necessary. ipdb moves to the dev dependency group. Remove the now-redundant poetry.lock, setup.py, and requirements.txt.
Replace the pip editable install in kamerapy.dockerfile with the standard uv Docker pattern: the uv binary is copied from the official distroless image, dependencies are synced from the lockfile in a cached layer before the source copy, and the project venv is put on PATH. Add a .dockerignore so a local .venv can't leak into the build context, and gitignore .venv.
The gui image installs kamera editable with --no-deps under ROS Noetic's Python 3.8, so pip's metadata check would reject the new requires-python >= 3.10. Bypass it with --ignore-requires-python; the modules the GUI imports still run on 3.8.
…ively
The conda env (environment.yml) supplies python + GDAL + uv; `make install`
then builds .venv on top of it with uv venv --system-site-packages and
uv sync --frozen. The kamerapy docker image now uses micromamba and the same
make install as the native setup scripts, replacing the Linux-only GDAL
wheel extra, and setup_postproc.{sh,ps1} are thin wrappers around the same
steps for Linux/macOS and Windows.
Pin python-preference = "only-system" in [tool.uv]: uv's default otherwise
substitutes a managed standalone interpreter for the conda one, leaving the
conda GDAL invisible through --system-site-packages. Commit uv.lock, which
--frozen requires, and fix bookworm-slim apt deps (libgl1 + libglib2.0-0;
libgl1-mesa-glx no longer exists in bookworm).
Verified py310 builds exist on conda-forge for linux-64, win-64, and osx-arm64. Provided via conda like GDAL, so the --system-site-packages venv sees it with no pyproject/lock change.
…teps The scripts were wrappers around three commands; the README now gives those directly for Linux/macOS and Windows (which runs the two uv commands from `make install`, since make usually isn't available there). Use explicit `conda env create` rather than update-as-create: micromamba's `env update` errors on a missing env. Also note GPU pycolmap selection: CUDA builds need driver 575+ (CUDA 12.9), older drivers silently fall back to CPU, and GPU only matters for full camera model calibration.
pip install -e . into the conda env is the simplest Windows path: one env, one activation, no uv.lock but pyproject floors keep it sane. Trim install docs and packaging comments to the essentials.
Rewrite the calibration pipeline as the kamera.calibration package. One COLMAP model holds all nine cameras: trigger-synchronized images form rig frames, INS positions are pose priors, and a prior-anchored rig bundle adjustment refines sensor_from_rig and intrinsics. Outputs per flight: camera model yamls in the INS frame, rig.yaml with the INS boresight and lever arm, DIVE v2 registration JSON and GIFs for every modality pair per channel, and a PDF report. - pass 1 maps every camera independently with position priors; pass 2 puts the rig from pass 1 onto the largest model, adds the IR images to their frames, triangulates and bundle adjusts twice - UV and IR frames are contrast-normalized; thermal-to-visible SIFT pairs are pruned; rig extrinsics are seeded from the densest cluster of per-frame estimates so a folded sub-model cannot bias them - environment moves to Python 3.13 and the conda-forge CUDA pycolmap 4.2; make install recreates .venv - remove the superseded per-camera calibration scripts
Cameras whose exposure midpoint does not coincide with the trigger sit an effective metre or so along track in the rig model (bundle adjustment cannot tell a delay from a lever arm on a translating rig), and a homography fit at infinity drops that baseline. Fit each pair for a nominal ground range instead (--registration_range_m, default the calibration flight's median scene range), read each camera's forward offset back into an exposure delay in rig.yaml and the report, and only write GIFs for frames the rig model registered.
Only the relative exposure midpoints are observable: the position priors absorb any delay common to the rig. Rename the field and fix the report text.
Configure ruff in pyproject.toml (the classic E4/E7/E9/F rule set, pinned so results do not depend on the ruff version), add it to the dev dependency group, and run ruff format over the calibration package and camera_models.py. No code changes beyond formatting.
- calibrate_rig builds the per-camera entries and counts frames in one pass, sorts the per-frame arrays once, and reads scene ranges through a named helper instead of a nested comprehension - RigCalibration gains center_in_ins_body and rotation_from_reference, which the yaml writer, the report and the delay estimate all used to spell out - InsTrajectory shares the segment lookup between pose and sample_gap - build_image_tree uses a plain if/else instead of a side-effect conditional expression and only starts a process pool when there is work - cli hoists the camera set and GIF frame list out of the loops they were recomputed in, and helpers come before main - derive_rig builds the translation array once; the triangulator's refine_intrinsics=False gets a comment explaining why it stays off - report drops semicolon-joined statements - docstrings, comments and long strings wrapped to 88 columns, which the formatter leaves alone
save_to_krtd had no callers. unproject_to_depth and save_depth_viz were defined on the base Camera class, where they shadowed the abstract unproject_to_depth and referenced a depth_map only DepthCamera has; they now live on DepthCamera. The bare except in Camera.__str__ catches Exception.
Creates or updates the conda env from environment.yml and builds .venv on top of it with uv from the lockfile, via conda run so nothing needs activating first. Windows previously used pip install -e . and skipped the lockfile. make install now calls the script; the Python version is read from environment.yml instead of being duplicated in the Makefile. README recommends Miniforge, with Miniconda/Anaconda as alternatives.
romleiaj
force-pushed
the
dev/rig-calibration
branch
from
September 21, 2026 15:27
dff749b to
e1083b4
Compare
Pass 2 refines each camera's rig offset only through tracks that survive COLMAP's 4 px triangulation filter, so a seed a degree off silently stalls near the seed while the reprojection RMS still looks fine. Print how many of the shared frames fell in the seed cluster, warn when the scatter is over half a degree or under half the frames made the cluster, and carry the observation count per camera into the terminal summary, the camera and rig yamls, and the report table, since that count collapses when the IR tracks were dropped.
Refining the distortion coefficients from the initial pair is degenerate on flat ground. On a 250-frame subset of the May 2025 flight it drove L_ir to a 30% focal error and k2 of -3, so every L_ir model died at three images and derive_rig failed with no frames shared with the reference; C_ir and R_ir only survived by luck. With distortion frozen, L_ir alone builds an 83-image model. Focal length stays free, and pass 2 refines the full intrinsics once the whole rig is posed.
- bootstrap.py falls back to micromamba, which is all the kamerapy image has; micromamba needs -y and rejects conda's --no-capture-output. - Stages build into <path>.partial and are moved into place when they finish, so an interrupted run redoes the stage instead of skipping it. Matching moves into the database stage for the same reason. - Registration GIFs read both images from the normalized tree; the raw UV frames are nearly black. - The report keeps a homography page when there are no GIF images (--gif_frames 0, or no frame with both cameras) instead of crashing.
The four save_to_file copies had drifted: DepthCamera wrote camera_quaternion with a leading space (invalid yaml) and its depth visualization into a directory that does not exist. model_type becomes a class attribute so it is right for every subclass, and the dead dist = "None" branch is dropped. Output is unchanged for the other three classes.
The script passed output_dir as save_shapefile_per_image, turning on per-image shapefiles and ignoring the directory. The function now takes output_dir (default <flight_dir>/processed_results) for all of its outputs, and the script checks flight_dir before using it.
Postflight finds camera models through <sys_cfg>/sys_config.json. The pipeline now writes a copy of the flight's file with the yaml paths pointed at the calibrated models, and --install_sys_config puts it in place, keeping the original as sys_config.json.orig.
Holding distortion at zero registers L_ir but puts the RGB corners about 25 px off, the mapper drops those observations, and the rig seed comes out several times looser (L_rgb scatter 0.05 -> 0.94 deg on the 250-frame subset). The config now carries k1, k2 per modality next to focal_px, rounded from the May 2025 calibration, and pass 1 keeps them fixed with only the focal length free. On the same subset that puts all three IR cameras in one 369-image model with every seed under 0.3 deg. Pass 2 refines the full intrinsics once the whole rig is posed, as before.
DIVE only uses ir->rgb and uv->rgb; the ir->uv files and GIFs follow from those two and only add noise to the outputs and the report.
Outputs now land in <flight>/calibration/camera_models/ so they stop colliding with the SfM models in pass1/ and rig/. The report is rebuilt page by page: - a flight summary page: dates and duration, triggers on disk versus complete, selected and registered frames, what a frame is, images per camera, the flight track zoomed to the registered frames, and the INS altitude profile - the intrinsics table ordered by modality so focal lengths compare at a glance, with distortion at three decimals and the per-pixel angle replaced by the full field of view and the ground sample distance at scene range - the rig geometry table grouped by swathe with rotation and lever arm split into x, y, z columns, and the optical-axes sketch drawn in aircraft body axes via the boresight, hanging from the mount plate, seen from behind - one full-page overlay per homography pair: the RGB frame in colour with the warped camera blended magenta over green inside its footprint, as DIVE shows a registration; the GIFs flip the same way - the boresight residual page and the error notes page are dropped
romleiaj
force-pushed
the
dev/rig-calibration
branch
from
September 22, 2026 15:02
44a023e to
c376ca0
Compare
Format geo_conversions, nav_conversions, postflight/utilities and create_flight_summary with ruff, drop unused imports and dead assignments, narrow a bare except, and rename single-letter variables. No behaviour change apart from one real fix: enu_to_llh built its ECEF x and y as one-element tuples (trailing commas), which numpy 2 refuses to convert to scalars, so every scalar call raised a TypeError. The round trip llh -> enu -> llh now closes to a micro-degree.
This was referenced Sep 23, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rig calibration
This PR creates a whole new process for camera calibration using colmap's rig calibration feature, released in 3.12 (this branch targets 4.2). This adds
kamera-calibrate, a tool that works out where every camera on the rig points and where it sits, from one calibration flight. It replaces the old per-camera calibration scripts.How it works
You point it at a KAMERA flight folder. It reads the images and the INS data, builds one 3D model (several intermediate ones) with COLMAP that includes all nine cameras, and from that model outputs:
The joint UV/EO model and separate IR model is created as before. But, the IR model is treated as a merely a intermediate product to get an estimate for the cameras' orientations. With this seed, there is a second pass over the original model and register IR images are registered into the same space, and the a global bundle adjustment is done over all imagery. The trick here is that all nine cameras fire on the same trigger, so the nine pictures of one trigger share one rig position and orientation. colmap can enforce that, and once it does, the IR cameras are pinned to the rig without any IR-to-visible matching. The INS positions pin the whole model to the real world, rather than arbitrary colmap coords. See
kamera/calibration/how_it_works.mdfor the step-by-step version.This also writes a sys_config.json file into the same folder, so you can test postflight processing immediately.
Notes
As a part of this process the fact the images aren't actually exposed at quite the same time surfaced (this appeared as a ~1m lever arm, which is wrong). This isn't a result of the hardware trigger, but rather poor accounting on my part. Phase One cameras' shutter takes about 20ms to actually expose, and the UV cameras' exposure is generally about 20ms, so a 10ms delay to mid-exposure. IR quickly exposes, generally within a couple ms. The long-term fix for this is that the Phase One cameras have a mid-exposure trigger pulse, so wire that to the INS and IR cameras. Short term is to fold this delay into the homography calculation, which is also in this branch. These timing estimates are output into the error PDF.
This initially started as just a cleanliness pass on dev/calibration, but then evolved into a full overhaul into rig calibration in its own package. The first AI-heavy branch being merged into this repo - as a standalone python package for pure postprocessing.
Evaluated on the 05/2025 calibration flight, it produces better models than before without any manual intervention (point clicking, model review, etc.). It also outputs the homographies now supported by DIVE/VIAME for model pipelines and annotation transfer.
As a part of this process, I pivoted away from poetry to a conda + uv environment used on other projects, conda to supply the heavy wheels that aren't very cross-platform (pycolmap, gdal), and uv for its speed for the rest. This should make Windows integration much easier. Ruff is now the de facto formatter.