HiP3R: Long-Horizon 3D Reconstruction

the paper introduces the Hierarchical Pose Graphs (HiP3R) to solve long-window 3D reconstruction problems by distinguishing between aligning chunks for global layout and optimizing frames for shared placement and motion. After optimizing chunks in a common global world frame using $\operatorname{Sim}(3)$ transform, the method refines frame-level camera predictions with $\operatorname{SE}(3)$ On public benchmarks and indoor/outdoor sequences, HiP3R achieves large reductions in Absolute Trajectory Error (ATE) and Relative Pose Errors (RPE), making it applicable to various setups and achieving competitive results with current dense global pose SLAM methods. Problem setting and central contribution HiP3R addresses the failure of short-window feed-forward 3D reconstruction systems when applied to long video sequences. Models such as VGGT-based architectures produce accurate local camera poses and dense pointmaps, but their attention and memory costs limit inference to short windows. Long sequences must therefore be partitioned into overlapping chunks. Each chunk is reconstructed in an independent coordinate system, and the resulting local trajectories may disagree in scale, orientation, translation, and even the poses assigned to frames that occur in multiple windows. The paper’s central claim is that chunk-level alignment alone is insufficient for globally consistent reconstruction. A single Sim⁡(3)\operatorname{Sim}(3) transformation per chunk can correct the placement and scale of an entire local reconstruction, but it cannot alter the internal camera configuration predicted within that chunk. Conversely, optimizing every frame globally offers greater flexibility but is computationally expensive and can disturb the global metric layout. HiP3R resolves this tension through a hierarchical pose graph: a chunk-level Sim⁡(3)\operatorname{Sim}(3) optimization first establishes global layout and scale, followed by a frame-level SE⁡(3)\operatorname{SE}(3) optimization that assigns exactly one shared camera pose to every physical frame (2610.31635). This decomposition is important because it separates two distinct error sources. Long-range drift and loop inconsistency are treated as a global similarity-layout problem, whereas duplicate and mutually inconsistent estimates of overlapping frames are treated as a shared-camera trajectory problem. The method is backend-agnostic and operates on frozen predictions, requiring neither retraining nor test-time …