Relaxed Rigidity with Ray-based Grouping for Dynamic Gaussian Splatting


Junoh Lee1*     Junmyeong Lee2     Yeon-Ji Song3     Inhwan Bae4     Jisu Shin1     Hae-Gon Jeon2†     Jin-Hwa Kim3,5†
1GIST   2Yonsei Univ.   3SNU   4DGIST   5NAVER AI Lab
*Work done during an internship at NAVER AI Lab.  †Corresponding authors.
arXiv Code

Summary: We align the motions of neighboring Gaussians via ray-based grouping, without any modifications to the underlying model.

Abstract

The reconstruction of dynamic 3D scenes using 3D Gaussian Splatting has shown significant promise. A key challenge, however, remains in modeling realistic motion, as most methods fail to align the motion of Gaussians with real-world physical dynamics. This misalignment is particularly problematic for monocular video datasets, where failing to maintain coherent motion undermines local geometric structure, ultimately leading to degraded reconstruction quality. Consequently, many state-of-the-art approaches rely heavily on external priors, such as optical flow or 2D tracks, to enforce temporal coherence. In this work, we propose a novel method to explicitly preserve the local geometric structure of Gaussians across time in 4D scenes. Our core idea is to introduce a view-space ray grouping strategy that clusters Gaussians intersected by the same ray, considering only those whose α-blending weights exceed a threshold. We then apply constraints to these groups to maintain a consistent spatial distribution, effectively preserving their local geometry. This approach enforces a more physically plausible motion model by ensuring that local geometry remains stable over time, eliminating the reliance on external guidance. We demonstrate the efficacy of our method by integrating it into two distinct baseline models. Extensive experiments on challenging monocular datasets show that our approach significantly outperforms existing methods, achieving superior temporal consistency and reconstruction quality.

Model Pipiline

Pipeline Overview

The proposed method begins by initializing 3D Gaussians as static and models their motion linearly. During the optimization process, static and dynamic objects are automatically separated based on the their magnitude of motion. For dynamic Gaussians, we use a keyframe-based interpolation strategy. the positions and rotations of the 3D Gaussians are interpolated between keyframes to enable efficient modeling over time. The progressive training scheme ensures adaptability to varying durations, while the optimization process incorporates pruning and point backtracking to refine rendering results.

Results

Comparisons

We compare our method against baselines on multiple datasets. Select a dataset and model below.

BibTeX

@article{lee2026rrrg,
      title   = {Relaxed Rigidity with Ray-based Grouping for Dynamic Gaussian Splatting},
      author  = {Lee, Junoh and Lee, Junmyeong and Song, Yeon-Ji and Bae, Inhwan and Shin, Jisu and Jeon, Hae-Gon and Kim, Jin-Hwa},
      journal = {arXiv preprint arXiv:2603.24994},
      year    = {2026}
}

References