Repository navigation
Add support for UPAL: joint point-line detection and description - #163
Merged
Merged
Conversation
Member
Author
B1ueber2y
marked this pull request as draft
September 4, 2026 21:39
B1ueber2y
marked this pull request as ready for review
September 5, 2026 10:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrates UPAL (ECCV 2026) as a line
detector, a line descriptor/matcher, and — via the previously unimplemented
use_joint_point_line_detectionpath — a frontend that produces keypoints,keypoint descriptors, line segments and line descriptors from a single
network pass per image.
Benchmarks
Detection and description only (no matching), 800x600 Hypersim images, RTX 3080:
4.9x on lines, 5.1x for the whole detection stage — the joint pass replaces
ALIKED as well. Adding the point half to UPAL costs only 2.3 ms, because it is
one extra descriptor sampling on a feature map that already exists.
Quickstart reconstruction (Hypersim
ai_001_001, 98 images, groups off):A vs B is the controlled comparison: identical point pipeline, so track
length (7.98 -> 8.01) and reprojection error (1.108 -> 1.110) are unchanged.
The line frontend swap costs 30% fewer 3D lines for 4.9x less time.
C and D are density-matched against each other (runs A and C differ in keypoint
count, so their point totals are not comparable — ALIKED is threshold-based and
returns ~554 by default, UPAL is top-k and returns exactly its budget). At
comparable density UPAL trails on track length (7.42 vs 8.38) and lines
(452 vs 630), most plausibly because mutual NN is matched against LightGlue and
GlueStick — UPAL has no trained matcher.
What's added
src/limap/image/joint_point_line/UPAL/— detector and matcher. Both segmentsand endpoint descriptors come from one encoder pass; going through upstream's
API would encode twice, since
UPAL.describe_keypointsre-runs the backbone.A test asserts the single-pass descriptors match upstream's to 1e-5.
src/limap/image/joint_point_line/—JointPointLineDetectionOptionsand thestage function, which writes the three artifacts the separate point and line
paths produce (hloc
.h5, segments, line descinfo), leaving everythingdownstream unchanged.
BaseDetector.detect_and_extract_joint— the capability, declared alongsidedetect_and_extract. Line-only detectors are rejected up front.upalinget_detector/get_extractor/get_matcher, witha
_UNCERTAINTY2D_DEFAULTSentry.cfgs/structure_triangulation/upal.yaml, README and docs entries.Also fixes two pre-existing bugs that made the dual-function detector path
unreachable (
detect_and_extract_all_images), committed separately.Design notes
joint_point_line_detectionwhenjoint detection is enabled, rather than requiring
line_detectionto repeatthe method. Otherwise a missing
extractor_methodwould silently build awireframe extractor against UPAL descinfo.
max_num_keypointsdefaults to 4096. Note this is a hard count, not a cap:UPAL's detector is top-k with no score threshold, so every image returns
exactly this many keypoints. It directly sets point density.
UPALOptionslives inline/register_detector.pyrather than beside UPAL,because
DetectorOptionscarries the field and moving it would create amodule-level import cycle.
Dependencies
Installed separately, like HAWP / TP-LSD / LBD / RoMa / Progressive-X — see
misc/install/upal.md. Deliberately kept out ofrequirements.txt: UPAL is GPU-only in practice.upal-local-features@git+...— the network. Not on PyPI.Both are required: the network predicts keypoints, descriptors and a line
distance field but no segments; segments come from LSD seeded by those keypoints
and filtered by that field.
Weights are not shipped in the package; the checkpoint downloads on first use
from a pinned commit via the existing
download_weightshelper.JointPointLineDetectionOptions.weight_pathdefaults toNone, so the rootcomes from
weights_root()like every other detector (#161) rather than beingpinned here -- a test guards that, since pinning it would download the same
checkpoint into a second directory.