Add comprehensive model card for SV-DRR
Browse filesThis PR adds a comprehensive model card for the SV-DRR model.
It includes:
- Relevant metadata: `pipeline_tag` (image-to-image), `library_name` (diffusers), `license` (apache-2.0), and additional `tags`.
- A link to the paper: [SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model](https://huggingface.co/papers/2507.05148).
- A link to the GitHub repository: [https://github.com/xiechun298/SV-DRR](https://github.com/xiechun298/SV-DRR).
- The paper abstract and a brief overview.
- Key visual demonstrations of the model's capabilities from the GitHub README.
- A "Usage" section with environment setup, model download, and a Python code snippet for single image inference, taken directly from the GitHub README.
- The BibTeX citation for proper attribution.
This ensures the model is discoverable, provides users with essential information, and offers a quick way to get started.
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-to-image
|
| 3 |
+
library_name: diffusers
|
| 4 |
+
license: apache-2.0
|
| 5 |
+
tags:
|
| 6 |
+
- medical
|
| 7 |
+
- x-ray
|
| 8 |
+
- diffusion-models
|
| 9 |
+
- image-generation
|
| 10 |
+
- miccai
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model
|
| 14 |
+
|
| 15 |
+
**Paper:** [SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model](https://huggingface.co/papers/2507.05148) | **Code:** [https://github.com/xiechun298/SV-DRR](https://github.com/xiechun298/SV-DRR)
|
| 16 |
+
|
| 17 |
+
## Abstract
|
| 18 |
+
X-ray imaging is a rapid and cost-effective tool for visualizing internal human anatomy. While multi-view X-ray imaging provides complementary information that enhances diagnosis, intervention, and education, acquiring images from multiple angles increases radiation exposure and complicates clinical workflows. To address these challenges, we propose a novel view-conditioned diffusion model for synthesizing multi-view X-ray images from a single view. Unlike prior methods, which are limited in angular range, resolution, and image quality, our approach leverages the Diffusion Transformer to preserve fine details and employs a weak-to-strong training strategy for stable high-resolution image generation. Experimental results demonstrate that our method generates higher-resolution outputs with improved control over viewing angles. This capability has significant implications not only for clinical applications but also for medical education and data extension, enabling the creation of diverse, high-quality datasets for training and analysis.
|
| 19 |
+
|
| 20 |
+
## TL;DR
|
| 21 |
+
We propose a novel view-conditioned diffusion model for synthesizing multi-view X-ray images up to 1024x1024 resolution from a single view.
|
| 22 |
+
|
| 23 |
+
<p align="center">
|
| 24 |
+
<img src="https://github.com/xiechun298/SV-DRR/raw/main/assets/demo2.gif" alt="demo2.gif" width="500"/>
|
| 25 |
+
</p>
|
| 26 |
+
|
| 27 |
+
## Visual Comparison with SOTA Methods
|
| 28 |
+

|
| 29 |
+
|
| 30 |
+
## DRR vs. SV-DRR
|
| 31 |
+
The name SV-DRR, short for Single-View DRR, is inspired by Digitally Reconstructed Radiography (DRR).
|
| 32 |
+
|
| 33 |
+
Unlike DRR, which renders X-ray projections from a 3D CT volume, our method synthesizes novel views directly from a single 2D projection.
|
| 34 |
+
|
| 35 |
+

|
| 36 |
+
|
| 37 |
+
## Usage
|
| 38 |
+
|
| 39 |
+
### 🚀 Quick Start
|
| 40 |
+
|
| 41 |
+
#### 🛠️ Environment Setup
|
| 42 |
+
|
| 43 |
+
To ensure compatibility and reproducibility, follow these steps to set up the environment:
|
| 44 |
+
|
| 45 |
+
1. **Clone the Repository**:
|
| 46 |
+
```bash
|
| 47 |
+
git clone https://github.com/xiechun-tsukuba/svdrr.git
|
| 48 |
+
cd svdrr
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
2. **Create a Python Virtual Environment**:
|
| 52 |
+
```bash
|
| 53 |
+
conda create -f environment.yaml
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
#### ⏬ Download Pretrained Models
|
| 57 |
+
|
| 58 |
+
You can download the pretrained models by either:
|
| 59 |
+
|
| 60 |
+
**Option 1: Automated Download (Recommended)**
|
| 61 |
+
```bash
|
| 62 |
+
python scripts/download_models.py
|
| 63 |
+
```
|
| 64 |
+
This will download all models into the `models/` directory. Shared components will be stored in the `shared/` folder, and symbolic links will be created in each model folder accordingly.
|
| 65 |
+
|
| 66 |
+
**Option 2: Manual Download from Hugging Face**
|
| 67 |
+
- 256 resolution: https://huggingface.co/xiechun-tsukuba/svdrr-dit-fb-256
|
| 68 |
+
- 512 resolution: https://huggingface.co/xiechun-tsukuba/svdrr-dit-fb-512
|
| 69 |
+
- 1024 resolution: https://huggingface.co/xiechun-tsukuba/svdrr-dit-fb-1024
|
| 70 |
+
|
| 71 |
+
### 🔍 Inference
|
| 72 |
+
|
| 73 |
+
**Important Note:** The coordinate system of LIDC-IDRI-DRR is opposite to the intuitive one — the polar angle increases downward, and the azimuth angle increases when rotating to the left. To invert the pose coordinate system, use the `--flip_pose` option.
|
| 74 |
+
|
| 75 |
+
#### Single Image Inference
|
| 76 |
+
|
| 77 |
+
**Default views (azimuth angles from -90° to 90° in 5° increments):**
|
| 78 |
+
```bash
|
| 79 |
+
python test_svdrr_DiT.py --model_path models/DiT-fb-512 \
|
| 80 |
+
--image_path demo/real_xray.jpg \
|
| 81 |
+
--log_dir outputs/ \
|
| 82 |
+
--image_size 512 \
|
| 83 |
+
--simple_pose
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
## Citation
|
| 87 |
+
If you find this work useful, a citation will be appreciated via:
|
| 88 |
+
|
| 89 |
+
```bibtex
|
| 90 |
+
@InProceedings{XieChu_SVDRR_MICCAI2025,
|
| 91 |
+
author = { Xie, Chun AND Yoshii, Yuichi AND Kitahara, Itaru},
|
| 92 |
+
title = { { SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model } },
|
| 93 |
+
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2025},
|
| 94 |
+
year = {2025},
|
| 95 |
+
publisher = {Springer Nature Switzerland},
|
| 96 |
+
volume = {LNCS 15963},
|
| 97 |
+
month = {September},
|
| 98 |
+
page = {572 -- 582},
|
| 99 |
+
doi = {https://doi.org/10.1007/978-3-032-04965-0_54}
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
@misc{xie2025svdrr,
|
| 103 |
+
title = {SV-DRR: High-Fidelity Novel View X-Ray Synthesis Using Diffusion Model},
|
| 104 |
+
author = {Chun Xie and Yuichi Yoshii and Itaru Kitahara},
|
| 105 |
+
year = {2025},
|
| 106 |
+
eprint = {2507.05148},
|
| 107 |
+
archivePrefix = {arXiv},
|
| 108 |
+
doi = {https://doi.org/10.48550/arXiv.2507.05148},
|
| 109 |
+
}
|
| 110 |
+
```
|