PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting

CVPR 2025

University of Maryland, College Park

* denotes equal contribution

Abstract

Recent advances in novel view synthesis have enabled real-time rendering speeds with high reconstruction accuracy. 3D Gaussian Splatting (3D-GS), a foundational point-based parametric 3D scene representation, models scenes as large sets of 3D Gaussians. However, complex scenes can consist of millions of Gaussians, resulting in high storage and memory requirements that limit the viability of 3D-GS on devices with limited resources. Current techniques for compressing these pretrained models by pruning Gaussians rely on combining heuristics to determine which Gaussians to remove. At high compression ratios, these pruned scenes suffer from heavy degradation of visual fidelity and loss of foreground details.

In this paper, we propose a principled spatial sensitivity pruning score that outperforms these approaches. It is computed as a second-order approximation of the reconstruction error on the training views with respect to the spatial parameters of each Gaussian. Additionally, we propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing the training pipeline.

After pruning 90% of Gaussians, a substantially higher percentage than previous methods, our PUP 3D-GS pipeline increases average rendering speed by 3.56× while retaining more salient foreground information and achieving higher image quality metrics than existing techniques on scenes from the Mip-NeRF 360, Tanks & Temples, and Deep Blending datasets.

Method

We find that the sensitivity of the L2 loss across the scene to a particular Gaussian Gi can be used as a surprisingly effective pruning score.

To compute this sensitivity, we first note that we can approximate the Hessian of the L2 loss across the scene with respect to a particular Gaussian Gi as:

Gi2L2=GiIGGiIGT,

where IG is a rendered training image and the sum is taken over all training images in the scene. This approximation is exact when the scene is converged. Please see our manuscript for additional details.

Using only the mean xi and scaling si parameters of this Gaussian G, we compute our sensitivity pruning score Ui as the log determinant of the Hessian:

Ui=log|xi,siIGxi,siIGT|.

Intuitively, Ui measures the sharpness of the L2 loss function around Gaussian Gi, where a higher Ui signifies that the L2 loss across the scene is more sensitive to Gi.

To prune a scene, we rank all of its Gaussians by this sensitivity pruning score and remove a given percentage of the least sensitive ones.

Results

LightGaussian [Fan 2023]
PUP 3D-GS (Ours)
LightGaussian [Fan 2023]
PUP 3D-GS (Ours)
LightGaussian [Fan 2023]
PUP 3D-GS (Ours)
LightGaussian [Fan 2023]
PUP 3D-GS (Ours)

BibTeX

@article{HansonTuPUP3DGS,
    author = {Hanson, Alex and Tu, Allen and Singla, Vasu and Jayawardhana, Mayuka and Zwicker, Matthias and Goldstein, Tom},
    title = {PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting},
    journal = {arXiv},
    year = {2024}
}