Skip to content

Add support for UPAL: joint point-line detection and description - #163

Merged
B1ueber2y merged 7 commits into
mainfrom
features/upal
Sep 5, 2026
Merged

B1ueber2y merged 7 commits into
mainfrom
features/upal

Conversation

@B1ueber2y

@B1ueber2y B1ueber2y commented Sep 4, 2026 •

Copy link
Copy Markdown
Member

Integrates UPAL (ECCV 2026) as a line
detector, a line descriptor/matcher, and — via the previously unimplemented
use_joint_point_line_detection path — a frontend that produces keypoints,
keypoint descriptors, line segments and line descriptors from a single
network pass per image
.

Benchmarks

Detection and description only (no matching), 800x600 Hypersim images, RTX 3080:

stage ms/image output
deeplsd detect + wireframe describe 165.1 244 lines
aliked-n16 point detection 16.2 650 kpts
default frontend, detection total 181.3
upal detect + describe (lines only) 33.4 160 lines
upal joint, one pass (points + lines) 35.7 160 lines + 4096 kpts

4.9x on lines, 5.1x for the whole detection stage — the joint pass replaces
ALIKED as well. Adding the point half to UPAL costs only 2.3 ms, because it is
one extra descriptor sampling on a feature map that already exists.

Quickstart reconstruction (Hypersim ai_001_001, 98 images, groups off):

run point frontend line frontend points track reproj lines
A aliked (554 kpt) + LightGlue deeplsd + wireframe + gluestick 4849 7.98 1.108 584
B aliked (554 kpt) + LightGlue upal 4828 8.01 1.110 410
C upal (4096 kpt) + mutual NN upal 28210 7.42 1.287 452
D aliked (2289 kpt) + LightGlue deeplsd + wireframe + gluestick 17982 8.38 1.230 630

A vs B is the controlled comparison: identical point pipeline, so track
length (7.98 -> 8.01) and reprojection error (1.108 -> 1.110) are unchanged.
The line frontend swap costs 30% fewer 3D lines for 4.9x less time.

C and D are density-matched against each other (runs A and C differ in keypoint
count, so their point totals are not comparable — ALIKED is threshold-based and
returns ~554 by default, UPAL is top-k and returns exactly its budget). At
comparable density UPAL trails on track length (7.42 vs 8.38) and lines
(452 vs 630), most plausibly because mutual NN is matched against LightGlue and
GlueStick — UPAL has no trained matcher.

What's added

  • src/limap/image/joint_point_line/UPAL/ — detector and matcher. Both segments
    and endpoint descriptors come from one encoder pass; going through upstream's
    API would encode twice, since UPAL.describe_keypoints re-runs the backbone.
    A test asserts the single-pass descriptors match upstream's to 1e-5.
  • src/limap/image/joint_point_line/ — JointPointLineDetectionOptions and the
    stage function, which writes the three artifacts the separate point and line
    paths produce (hloc .h5, segments, line descinfo), leaving everything
    downstream unchanged.
  • BaseDetector.detect_and_extract_joint — the capability, declared alongside
    detect_and_extract. Line-only detectors are rejected up front.
  • Registered as upal in get_detector / get_extractor / get_matcher, with
    a _UNCERTAINTY2D_DEFAULTS entry.
  • cfgs/structure_triangulation/upal.yaml, README and docs entries.

Also fixes two pre-existing bugs that made the dual-function detector path
unreachable (detect_and_extract_all_images), committed separately.

Design notes

  • Association derives its line matcher from joint_point_line_detection when
    joint detection is enabled, rather than requiring line_detection to repeat
    the method. Otherwise a missing extractor_method would silently build a
    wireframe extractor against UPAL descinfo.
  • max_num_keypoints defaults to 4096. Note this is a hard count, not a cap:
    UPAL's detector is top-k with no score threshold, so every image returns
    exactly this many keypoints. It directly sets point density.
  • UPALOptions lives in line/register_detector.py rather than beside UPAL,
    because DetectorOptions carries the field and moving it would create a
    module-level import cycle.

Dependencies

Installed separately, like HAWP / TP-LSD / LBD / RoMa / Progressive-X — see
misc/install/upal.md. Deliberately kept out of
requirements.txt: UPAL is GPU-only in practice.

  • upal-local-features@git+... — the network. Not on PyPI.

Both are required: the network predicts keypoints, descriptors and a line
distance field but no segments; segments come from LSD seeded by those keypoints
and filtered by that field.

Weights are not shipped in the package; the checkpoint downloads on first use
from a pinned commit via the existing download_weights helper.
JointPointLineDetectionOptions.weight_path defaults to None, so the root
comes from weights_root() like every other detector (#161) rather than being
pinned here -- a test guards that, since pinning it would download the same
checkpoint into a second directory.

@B1ueber2y

Copy link
Copy Markdown
Member Author

cc: @francois141 @RaphaelKreft

@B1ueber2y
B1ueber2y marked this pull request as draft September 4, 2026 21:39
@B1ueber2y
B1ueber2y marked this pull request as ready for review September 5, 2026 10:37
@B1ueber2y
B1ueber2y merged commit f6a3d6b into main Sep 5, 2026
8 checks passed
@B1ueber2y
B1ueber2y deleted the features/upal branch September 5, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant