CaiYuanhao/X-Gaussian

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

ECCV 2024

5 CT scenes + 3 real scans · X-ray novel view synthesis · Direct 3D CT reconstruction

Paper · Code · Models · Data

Point Cloud Visualization

Training Process Visualization

GIF demonstrations are from the official X-Gaussian repository. The available CT and real-scan scenes are listed below.

Model Description

X-Gaussian represents X-ray attenuation using radiative 3D Gaussians. Introduced in Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis (ECCV 2024), it supports efficient X-ray novel view synthesis. These checkpoints use the current joint 2D/3D implementation, which also directly reconstructs a CT volume from the same Gaussian field while retaining the original Gaussian point attributes.

This repository provides eight scene-specific trained models: Chest, Abdomen, Foot, Head, Pancreas, and the real FIPS scans Pine, Seashell, and Walnut, paired with the X-Gaussian dataset. Each model represents its corresponding scene; reconstructing another scene requires training from that scene's projections and scanner geometry.

Available Checkpoints

Final checkpoints from the current scene recipes: 45,000 steps for synthetic scenes and 30,000 for real scenes. The previous 31,000-step release remains available at revision legacy-31000.

Scene	Gaussian checkpoint	Dataset	Gaussians
Chest	point_cloud.ply	synthetic/chest_50.pickle	250,000
Abdomen	point_cloud.ply	synthetic/abdomen_50.pickle	59,892
Foot	point_cloud.ply	synthetic/foot_50.pickle	77,881
Head	point_cloud.ply	synthetic/head_50.pickle	116,135
Pancreas	point_cloud.ply	synthetic/pancreas_50.pickle	246,359
Pine	point_cloud.ply	real/pine/	55,915
Seashell	point_cloud.ply	real/seashell/	66,001
Walnut	point_cloud.ply	real/walnut/	86,478

Each scene includes config.json, recipe.yaml, training-only initialization.npy, and the final point_cloud/iteration_<steps>/point_cloud.ply. Data paths are relative to the code repository. PLY files retain the original 62 per-point attributes and embed the decoder and voxel geometry. manifest.json records file sizes and SHA-256 checksums.

Download checkpoints

The root config.json lists this repository's checkpoints, supporting files, and implementation. Include it when downloading selected weights so those downloads can be counted by the Hugging Face Hub. Use the linked implementation and its matching runtime configuration to load the checkpoints.

pip install -U huggingface_hub
hf download CaiYuanhao/X-Gaussian --include "config.json" "synthetic/**" "real/**" "manifest.json" --local-dir pretrained

Training and Evaluation

The CT extension uses the original Gaussian point attributes and native data resolution, with 50 training projections per scene. Synthetic recipes use 30,000 base steps and 15,000 refinement steps; real recipes use 30,000 joint projection/3D-TV steps. Settings were selected on held-out training projections. Reference volumes and test projections are used only for evaluation.

3D PSNR uses a fixed range of 1 without clipping or intensity correction. SSIM averages nonempty reference slices along each axis, then across axes. Synthetic references are the original CT volumes; real references are full-view FDK pseudo-volumes, not measured CT ground truth.

Performance

X-ray Novel View Synthesis

Direct 3D CT Reconstruction

The CT extension uses the original Gaussian point attributes and native data resolution, with 50 training projections per scene. Synthetic recipes use 30,000 base steps and 15,000 refinement steps; real recipes use 30,000 joint projection/3D-TV steps. Settings were selected on held-out training projections. Reference volumes and test projections are used only for evaluation.

3D PSNR uses a fixed range of 1 without clipping or intensity correction. SSIM averages nonempty reference slices along each axis, then across axes. Synthetic references are the original CT volumes; real references are full-view FDK pseudo-volumes, not measured CT ground truth.

Scene	CT resolution	3D PSNR (dB)	3D SSIM
Chest	128 × 128 × 128	36.00	0.9391
Abdomen	512 × 512 × 463	34.27	0.8946
Foot	256 × 256 × 256	32.18	0.8815
Head	256 × 256 × 128	41.44	0.9768
Pancreas	512 × 512 × 240	22.38	0.8791
Average		33.25	0.9142

Real Data 3D CT Reconstruction

Real FIPS scans contain 50 training and 100 test projections at 560 × 560. The 3D reference is full-view FDK.

Scene	CT resolution	3D PSNR (dB)	3D SSIM
Pine	256 × 256 × 256	41.80	0.9465
Seashell	256 × 256 × 256	44.01	0.9638
Walnut	256 × 256 × 256	32.77	0.7169
Average		39.53	0.8757

See the real-data sources and attribution.

Usage

Please refer to the X-Gaussian repository for environment setup and model usage, including 2D projection rendering and direct 3D CT reconstruction.

Reconstruction Previews

Central orthogonal slices from the hosted models. In each image, the top row shows the reference CT, the middle row the reconstruction, and the bottom row the absolute error. Displayed CT intensities use a fixed range of 0–1 and errors use 0–0.1; metrics above use the unmodified arrays.

Chest

Abdomen

Foot

Head

Pancreas

Pine (real; full-view FDK reference)

Seashell (real; full-view FDK reference)

Walnut (real; full-view FDK reference)

Citation

If you use these models, please cite X-Gaussian and the SAX-NeRF dataset source:

@inproceedings{x_gaussian,
  title={Radiative gaussian splatting for efficient x-ray novel view synthesis},
  author={Yuanhao Cai and Yixun Liang and Jiahao Wang and Angtian Wang and Yulun Zhang and Xiaokang Yang and Zongwei Zhou and Alan Yuille},
  booktitle={ECCV},
  year={2024}
}

@inproceedings{sax_nerf,
  title={Structure-Aware Sparse-View X-ray 3D Reconstruction},
  author={Yuanhao Cai and Jiahao Wang and Alan Yuille and Zongwei Zhou and Angtian Wang},
  booktitle={CVPR},
  year={2024}
}