Skip to content

Avoid NaN gradients when undistortion skips singular Newton steps - #3798

Open
Afloat16 wants to merge 1 commit into
nerfstudio-project:mainfrom
Afloat16:fix/undistort-singular-backward
Open

Afloat16 wants to merge 1 commit into
nerfstudio-project:mainfrom
Afloat16:fix/undistort-singular-backward

Conversation

@Afloat16

Copy link
Copy Markdown

Problem

Radial/tangential undistortion skips a Newton update when the local Jacobian is singular. An unused singular solve can still contaminate backward gradients with nonfinite values.

Change

Guard the inactive singular Newton path so the skipped point retains its original value and has finite identity/zero derivatives, without changing the regular Newton update for other points in a batch.

Regression coverage

New CPU regressions cover one and three iterations, float32/float64, gradients with respect to points and distortion parameters, and mixed singular/regular batches checked against an explicit regular Newton solve. Historical focused baseline/fix validation was prepared on this branch but was not rerun today.

The repository's required Core/Docker gates have not been verified as passing for this pending PR. CUDA and the full rendering pipeline are untested.

The Newton undistortion loop selects a zero step when its Jacobian determinant is too small, but computes a division by the original determinant in the unused branch of `torch.where`. Forward output is finite while coordinate and calibration gradients become NaN at a singular Jacobian.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant