---
library_name: pytorch
pipeline_tag: video-to-video
tags:
  - video-frame-interpolation
  - rife
  - optical-flow
  - frame-interpolation
license: apache-2.0
task_categories:
  - vision
  - video-processing
---

# WMS-RIFE v3 — Fine-tuned for WMS Satellite Sequences

## [Paper (ECCV2022)](https://arxiv.org/abs/2011.06294) | [Colab Demo](https://colab.research.google.com/github/hzwer/ECCV2022-RIFE/blob/main/Colab_demo.ipynb) | [Tutorial](https://www.youtube.com/watch?v=gf_on-dbwyU)

## Introduction

This repo provides a fine-tuned version of **RIFE v3 (HDv3)** adapted for **WMS satellite imagery (2k–5k)**.
Our model is optimized for smooth temporal interpolation of clouds, land/sea surfaces, and atmospheric patterns.

* Base architecture: [RIFE (ECCV 2022)](https://arxiv.org/abs/2011.06294)
* Training improvements: L1 + motion-weighted Charbonnier + (1-SSIM) + flow smoothness
* Domain: remote sensing imagery (multi-sensor, 2k–5k resolution)
* Performance: runs in real-time on RTX-class GPUs

---

## Results

| Model                | PSNR ↑    | SSIM ↑     |
| -------------------- | --------- | ---------- |
| Baseline (HDv3)      | 32.57     | 0.8867     |
| Fine-tuned (L1 only) | 34.10     | 0.8974     |
| Fine-tuned (Custom)  | **34.42** | **0.8991** |

*N = ~5.3k validation triplets from WMS sensors.*
Visual inspection confirms improved temporal stability and sharper cloud edges.

---

## Quick Start

### Installation

```bash
git clone https://huggingface.co/Anson-Saju-George/wms-rifev3
cd wms-rifev3
pip install -r requirements.txt
```

### Inference (pair of images)

```bash
python demo_infer_pair.py --img0 frame_000.png --img1 frame_001.png
```

This will output `interp_000_001.png`.

### Inference (video)

```bash
python finetune_infer_wms_sequence.py --video sample.mp4 --exp 1
```

Generates `sample_2X.mp4`.

---

## Training / Evaluation

* **Train:**

  ```bash
  python finetune_custom_loss_train.py
  ```
* **Eval:**

  ```bash
  python finetune_eval_wms.py
  ```

---

## Applications

* Satellite nowcasting & forecasting previews
* Meteorology (cloud motion, precipitation fronts)
* Ocean/atmosphere flow visualization
* Geophysics time-lapse smoothing
* Web maps & GIS visualization overlays

---

## Datasets

We trained on **curated WMS imagery** (2k–5k resolution).
Due to license restrictions, raw datasets are not redistributed. Please build from your own WMS feeds.

---

## Citation

If you use this model, please cite the base RIFE paper:

```bibtex
@inproceedings{huang2022rife,
  title={Real-Time Intermediate Flow Estimation for Video Frame Interpolation},
  author={Huang, Zhewei and Zhang, Tianyuan and Heng, Wen and Shi, Boxin and Zhou, Shuchang},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2022}
}
```

---

## License

Apache-2.0 (same as original RIFE). Please also respect the license of any datasets you use.

---