VideoSuperResolution - GitLite

KO | EN

gitlite — search

#hacktoberfest #react #machine-learning #rust #deep-learning #linux #docker #nodejs #awesome #cli #awesome-list #vue #security #mcp

VideoSuperResolution

(★ 1,687)

A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.

File Explorer

Download Latest Version (.zip)

- credentials.json
- crf.npz
- datasets.yaml
- README.md
- - 1. Inference using SRCNN.ipynb
  - 2. Building data pipeline with DataLoader.ipynb
  - 3. Calling model with executor.ipynb
- - Adding new model.md
  - Change backend.md
- Benchmark (paper).md
- Benchmark (reproduce).md
- Collections CVPR 2019.md
- Collections CVPR 2020.md
- Collections ECCV 2020.md
- Collections ICCV 2019.md
- Collections NeurIPS 2020.md
- - crf_s.npz
  - fake_datasets.yml
  - vespcn.yaml
- correlation_test.py
- dataset_test.py
- googledrive_test.py
- image_test.py
- imfilter_test.py
- initializer_test.py
- loader_test.py
- model_test.py
- motion_test.py
- space_to_depth_test.py
- training_test.py
- utility_test.py
- vgg_test.py
- virtualfile_test.py
- CelebA.py
- DND.py
- FastMetrics.py
- FFmpegHelper.py
- Image2Raw.py
- MakeHDF.py
- Misc.py
- NtireHelper.py
- Raw2Image.py
- SeqVisual.py
- Vimeo.py
- YoukuPackage.py
- check_dataset.py
- eval.py
- README.md
- train.py
- - __init__.py
- - __init__.py
  - Crop.py
  - Dataset.py
  - FloDecoder.py
  - Loader.py
  - NVDecoder.py
  - Transform.py
  - VirtualFile.py
  - YVDecoder.py
- - __init__.py
- - __init__.py
  - Config.py
  - Ensemble.py
  - GoogleDriveDownloader.py
  - Hook.py
  - ImageProcess.py
  - LearningRateScheduler.py
  - Math.py
  - PcaPrecompute.py
  - Utility.py
  - VisualizeOpticalFlow.py
- __init__.py
.gitignore
.gitmodules
.travis.yml
azure-pipelines.yml
CHANGELOG.md
LICENSE
prepare_data.py
README.md
setup.py

# Installation Guide

1. Get the code

git clone https://github.com/LoSealL/VideoSuperResolution

Downloads the entire project code from GitHub to your computer.

cd VideoSuperResolution

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install .

Installs the package published on PyPI directly — no need to clone the source.

jupyter notebook

Launches Jupyter in your browser so you can open and run the notebook (.ipynb) files.

✅ If it runs without errors and prints output in the terminal, it worked.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Video Super Resolution
A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.

**Project uploaded to PyPI now. Try install from PyPI:**
```shell script
pip install VSR
```

**Pretrained weights is uploading now.**

**Several referenced PyTorch implementations are also included now.**

**Quick Link:**
- [Installation](#install)
- [Getting Started](#Getting-Started)
- [Benchmark](https://github.com/LoSealL/VideoSuperResolution/blob/master/Docs/Benchmark%20(reproduce).md)

## Network list and reference (Updating)
The hyperlink directs to paper site, follows the official codes if the authors open sources.

All these models are implemented in **ONE** framework.

|Model |Published |Code* |VSR (TF)**|VSR (Torch)|Keywords|Pretrained|
|:-----|:---------|:-----|:---------|:----------|:-------|:---------|
|SRCNN|[ECCV14](https://arxiv.org/abs/1501.00092)|-, [Keras](https://github.com/qobilidop/srcnn)|Y|Y| Kaiming |[√](https://github.com/LoSealL/Model/releases)|
|RAISR|[arXiv](https://arxiv.org/abs/1606.01299)|-|-|-| Google, Pixel 3 ||
|ESPCN|[CVPR16](https://arxiv.org/abs/1609.05158)|-, [Keras](https://github.com/qobilidop/srcnn)|Y|Y| Real time |[√](https://github.com/LoSealL/Model/releases)|
|VDSR|[CVPR16](https://arxiv.org/abs/1511.04587)|-|Y|Y| Deep, Residual |[√](https://drive.google.com/open?id=1hW5YDxXpmjO2IfAy8f29O7yf1M3fPIg1)|
|DRCN|[CVPR16](https://arxiv.org/abs/1511.04491)|-|Y|Y| Recurrent ||
|DRRN|[CVPR17](http://cvlab.cse.msu.edu/pdfs/Tai_Yang_Liu_CVPR2017.pdf)|[Caffe](https://github.com/tyshiwo/DRRN_CVPR17), [PyTorch](https://github.com/jt827859032/DRRN-pytorch)|Y|Y| Recurrent ||
|LapSRN|[CVPR17](http://vllab.ucmerced.edu/wlai24/LapSRN/)|[Matlab](https://github.com/phoenix104104/LapSRN)|Y|-| Huber loss ||
|EDSR|[CVPR17](https://arxiv.org/abs/1707.02921)|-|Y|Y| NTIRE17 Champion |[√](https://github.com/LoSealL/Model/releases)|
|SRGAN|[CVPR17](https://arxiv.org/abs/1609.04802)|-|Y|-| 1st proposed GAN ||
|VESPCN|[CVPR17](https://arxiv.org/abs/1611.05250)|-|Y|Y| VideoSR |[√](https://drive.google.com/open?id=19u4YpsyThxW5dv4fhpMj7c5gZeEDKthm)|
|MemNet|[ICCV17](https://arxiv.org/abs/1708.02209)|[Caffe](https://github.com/tyshiwo/MemNet)|Y|-|||
|SRDenseNet|[ICCV17](http://openaccess.thecvf.com/content_ICCV_2017/papers/Tong_Image_Super-Resolution_Using_ICCV_2017_paper.pdf)|-, [PyTorch](https://github.com/wxywhu/SRDenseNet-pytorch)|Y|-| Dense |[√](https://drive.google.com/open?id=1aXAfRqZieY6mTfZUnErG84-9NfkQSeDw)|
|SPMC|[ICCV17](https://arxiv.org/abs/1704.02738)|[Tensorflow](https://github.com/jiangsutx/SPMC_VideoSR)|T|Y| VideoSR ||
|DnCNN|[TIP17](http://ieeexplore.ieee.org/document/7839189/)|[Matlab](https://github.com/cszn/DnCNN)|Y|Y| Denoise |[√](https://github.com/LoSealL/Model/releases)|
|DCSCN|[arXiv](https://arxiv.org/abs/1707.05425)|[Tensorflow](https://github.com/jiny2001/dcscn-super-resolution)|Y|-|||
|IDN|[CVPR18](https://arxiv.org/abs/1803.09454)|[Caffe](https://github.com/Zheng222/IDN-Caffe)|Y|-| Fast |[√](https://drive.google.com/open?id=1Fh3rtvrKKLAK27r518T1M_JET_LWZAFQ)|
|RDN|[CVPR18](https://arxiv.org/abs/1802.08797)|[Torch](https://github.com/yulunzhang/RDN)|Y|-| Deep, BI-BD-DN ||
|SRMD|[CVPR18](https://arxiv.org/abs/1712.06116)|[Matlab](https://github.com/cszn/SRMD)|-|Y| Denoise/Deblur/SR |[√](https://drive.google.com/open?id=1ORKH05-aLSbQaWB4qQulIm2INoRufuD_)|
|DBPN|[CVPR18](https://arxiv.org/abs/1803.02735)|[PyTorch](https://github.com/alterzero/DBPN-Pytorch)|Y|Y| NTIRE18 Champion |[√](https://drive.google.com/open?id=1ymtlOjhkGmad-od0zw7yTf17nWD4KMVi)|
|ZSSR|[CVPR18](http://www.wisdom.weizmann.ac.il/~vision/zssr/)|[Tensorflow](https://github.com/assafshocher/ZSSR)|-|-| Zero-shot ||
|FRVSR|[CVPR18](https://arxiv.org/abs/1801.04590)|[PDF](https://github.com/msmsajjadi/FRVSR)|T|Y| VideoSR |[√](https://github.com/LoSealL/Model/releases)|
|DUF|[CVPR18](http://openaccess.thecvf.com/content_cvpr_2018/papers/Jo_Deep_Video_Super-Resolution_CVPR_2018_paper.pdf)|[Tensorflow](https://github.com/yhjo09/VSR-DUF)|T|-| VideoSR ||
|CARN|[ECCV18](https://arxiv.org/abs/1803.08664)|[PyTorch](https://github.com/nmhkahn/CARN-pytorch)|Y|Y| Fast |[√](https://github.com/LoSealL/Model/releases/carn)|
|RCAN|[ECCV18](https://arxiv.org/abs/1807.02758)|[PyTorch](https://github.com/yulunzhang/RCAN)|Y|Y| Deep, BI-BD-DN ||
|MSRN|[ECCV18](http://openaccess.thecvf.com/content_ECCV_2018/papers/Juncheng_Li_Multi-scale_Residual_Network_ECCV_2018_paper.pdf)|[PyTorch](https://github.com/MIVRC/MSRN-PyTorch)|Y|Y| |[√](https://drive.google.com/open?id=1A0LoY3oB_VnArP3GzI1ILUNJbLAEjdtJ)|
|SRFeat|[ECCV18](http://openaccess.thecvf.com/content_ECCV_2018/papers/Seong-Jin_Park_SRFeat_Single_Image_ECCV_2018_paper.pdf)|[Tensorflow](https://github.com/HyeongseokSon1/SRFeat)|Y|Y| GAN ||
|NLRN|[NIPS18](https://papers.nips.cc/paper/7439-non-local-recurrent-network-for-image-restoration.pdf)|[Tensorflow](https://github.com/Ding-Liu/NLRN)|T|-| Non-local, Recurrent ||
|SRCliqueNet|[NIPS18](https://arxiv.org/abs/1809.04508)|-|-|-| Wavelet ||
|FFDNet|[TIP18](https://ieeexplore.ieee.org/document/8365806/)|[Matlab](https://github.com/cszn/FFDNet)|Y|Y| Conditional denoise||
|CBDNet|[CVPR19](https://arxiv.org/abs/1807.04686)|[Matlab](https://github.com/GuoShi28/CBDNet)|T|-| Blind-denoise ||
|SOFVSR|[ACCV18](http://arxiv.org/abs/1809.08573)|[PyTorch](https://github.com/LongguangWang/SOF-VSR)|-|Y| VideoSR |[√](https://github.com/LoSealL/Model/releases/download/sofvsr/SOFVSR_x4.zip)|
|ESRGAN|[ECCVW18](http://arxiv.org/abs/1809.00219)|[PyTorch](https://github.com/xinntao/ESRGAN)|-|Y|1st place PIRM 2018|[√](https://github.com/LoSealL/Model/releases/download/esrgan/esrgan.zip)|
|TecoGAN|[arXiv](http://arxiv.org/abs/1811.09393)|[Tensorflow](https://github.com/thunil/TecoGAN)|-|T| VideoSR GAN|[√](https://github.com/LoSealL/Model/releases/download/tecogan/tecogan.zip)|
|RBPN|[CVPR19](https://arxiv.org/abs/1903.10128)|[PyTorch](https://github.com/alterzero/RBPN-PyTorch)|-|Y| VideoSR |[√](https://drive.google.com/open?id=1Ozp5j-DBWJSpXY5GvxiEPKdfCaAbOXqu)|
|DPSR|[CVPR19](https://arxiv.org/abs/1903.12529)|[Pytorch](https://github.com/cszn/DPSR)|-|-|||
|SRFBN|[CVPR19](https://arxiv.org/abs/1903.09814)|[Pytorch](https://github.com/Paper99/SRFBN_CVPR19)|-|-||||
|SRNTT|[CVPR19](https://arxiv.org/abs/1903.00834)|[Tensorflow](https://github.com/ZZUTK/SRNTT)|-|-|Adobe||
|SAN|[CVPR19](http://openaccess.thecvf.com/content_CVPR_2019/papers/Dai_Second-Order_Attention_Network_for_Single_Image_Super-Resolution_CVPR_2019_paper.pdf)|[empty](https://github.com/daitao/SAN)|-|-| AliDAMO SOTA ||
|AdaFM|[CVPR19](https://arxiv.org/abs/1904.08118)|[Pytorch](https://github.com/hejingwenhejingwen/AdaFM)|-|-| SenseTime Oral ||

\*The 1st repo is by paper author.

\**__Y__: included; __-__: not included; __T__: under-testing.

You can download pre-trained weights through [`prepare_data`](./prepare_data.py), or visit the hyperlink at **√**.

## Link of datasets
*(please contact me if any of links offend you or any one disabled)*

|Name|Usage|#|Site|Comments|
|:---|:----|:----|:---|:-----|
|SET5|Test|5|[download](https://uofi.box.com/shared/static/kfahv87nfe8ax910l85dksyl2q212voc.zip)|[jbhuang0604](https://github.com/jbhuang0604/SelfExSR)|
|SET14|Test|14|[download](https://uofi.box.com/shared/static/igsnfieh4lz68l926l8xbklwsnnk8we9.zip)|[jbhuang0604](https://github.com/jbhuang0604/SelfExSR)|
|SunHay80|Test|80|[download](https://uofi.box.com/shared/static/rirohj4773jl7ef752r330rtqw23djt8.zip)|[jbhuang0604](https://github.com/jbhuang0604/SelfExSR)|
|Urban100|Test|100|[download](https://uofi.box.com/shared/static/65upg43jjd0a4cwsiqgl6o6ixube6klm.zip)|[jbhuang0604](https://github.com/jbhuang0604/SelfExSR)|
|VID4|Test|4|[download](https://people.csail.mit.edu/celiu/CVPR2011/videoSR.zip)|4 videos|
|BSD100|Train|300|[download](https://uofi.box.com/shared/static/qgctsplb8txrksm9to9x01zfa4m61ngq.zip)|[jbhuang0604](https://github.com/jbhuang0604/SelfExSR)|
|BSD300|Train/Val|300|[download](https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/BSDS300-images.tgz)|-|
|BSD500|Train/Val|500|[download](http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/BSR/BSR_bsds500.tgz)|-|
|91-Image|Train|91|[download](http://www.ifp.illinois.edu/~jyang29/codes/ScSR.rar)|Yang|
|DIV2K|Train/Val|900|[website](https://data.vision.ee.ethz.ch/cvl/DIV2K/)|NTIRE17|
|Waterloo|Train|4741|[website](https://ece.uwaterloo.ca/~k29ma/exploration/)|-|
|MCL-V|Train|12|[website](http://mcl.usc.edu/mcl-v-database/)|12 videos|
|GOPRO|Train/Val|33|[website](https://github.com/SeungjunNah/DeepDeblur_release)|33 videos, deblur|
|CelebA|Train|202599|[website](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)|Human faces|
|Sintel|Train/Val|35|[website](http://sintel.is.tue.mpg.de/downloads)|Optical flow|
|FlyingChairs|Train|22872|[website](https://lmb.informatik.uni-freiburg.de/resources/datasets/FlyingChairs.en.html#flyingchairs)|Optical flow|
|DND|Test|50|[website](https://noise.visinf.tu-darmstadt.de/)|Real noisy photos|
|RENOIR|Train|120|[website](http://ani.stat.fsu.edu/~abarbu/Renoir.html)|Real noisy photos|
|NC|Test|60|[website](http://demo.ipol.im/demo/125/)|Noisy photos|
|SIDD(M)|Train/Val|200|[website](https://www.eecs.yorku.ca/~kamel/sidd/)|NTIRE 2019 Real Denoise|
|RSR|Train/Val|80|[download]()|NTIRE 2019 Real SR|
|Vimeo-90k|Train/Test|89800|[website](http://toflow.csail.mit.edu/)|90k HQ videos|

Other open datasets:
[Kaggle](https://www.kaggle.com/datasets)
[ImageNet](http://www.image-net.org/)
[COCO](http://cocodataset.org/)

## VSR package
This package offers a training and data processing framework based on [TF](https://www.tensorflow.org).
What I made is a simple, easy-to-use framework without lots of encapulations and abstractions.
Moreover, VSR can handle raw NV12/YUV as well as a sequence of images as inputs.

### Install

1. Prepare proper tensorflow and pytorch(optional). For example, GPU and CUDA10.0 (recommend to use `conda`):

```shell
   conda install tensorflow-gpu==1.15.0
   # optional
   # conda install pytorch
   ```

2. Install VSR package

```bash
   # For someone see this doc online
   # git clone https://github.com/loseall/VideoSuperResolution && cd VideoSuperResolution
   pip install -e .
   ```
### Getting Started

1. Download pre-trained weights and (optinal) training datasets. For instance, let\'s begin with VESPCN and vid4 test data:
   ```shell
   python prepare_data.py --filter vespcn vid4
   ```
   
2. Customize backend
   cd ~/.vsr/
   touch config.yml
   ```yaml
   backend: tensorflow  # (tensorflow, pytorch)
   verbose: info        # (debug, info, warning, error)
   ```

3. Evaluate
   ```shell
   cd Train
   python eval.py srcnn -t vid4 --pretrain=/path/srcnn.pth
   ```

4. Train
   ```shell
   python prepare_data.py --filter mcl-v
   cd Train
   python train.py vespcn --dataset mcl-v --memory_limit 1GB --epochs 100
   ```

OK, that's all you need. For more details, use `--help` to get more information.

----
More documents can be found at [Docs](https://github.com/LoSealL/VideoSuperResolution/tree/master/Docs).