This is the official code of the IEEE Access paper Multiple Adverse Weather Removal Using Masked-Based Pre-Training and Dual-Pooling Adaptive Convolution.
We denote our proposed method as MPDAC (Masked-based Pre-training and Dual-pooling Adaptive Convolution).
MPDAC removes various types of degradations caused by adverse weather, including rain, fog, and snow.
| Raindrop Removal | Rain & Fog Removal | Snow Removal |
 |
 |
 |
Removing artifacts caused by multiple adverse weather, including rain, fog, and snow, is crucial for image processing in outdoor environments. Conventional high-performing methods face challenges, such as requiring pre-specification of weather types and slow processing times. In this study, we propose a novel convolutional neural network-based hierarchical encoder-decoder model that addresses these issues effectively. Our model utilizes knowledge of feature representations obtained from masked-based pre-training on a large-scale dataset. To remove diverse degradations efficiently, we employ a proposed dual-pooling adaptive convolution, which improves representational capability of weight generating network by using average pooling, max pooling, and filter-wise global response normalization. Experiments conducted on both synthetic and real image datasets show that our model achieves promising results. The performance on real images is also improved by a novel learning strategy, in which a model trained on the synthetic image dataset is fine-tuned to the real image dataset. The proposed method is notably cost-effective in terms of computational complexity and inference speed. Moreover, ablation studies show the effectiveness of various components in our method.
Google Colab Demo:
This code has been verified to work with torch==2.0 and cuda==11.7, and it is likely compatible with other versions. We are using timm==0.6.13, but since the necessary files are included in this repository, there is no need for a separate installation.
torch
torchvision
numpy
pillow
cd MPDAC
python demo.py \
--input_image_path ./imgs/degraded_imgs/sample_raindrop.png \
--save_dir ./imgs/restored_imgs \
--weights_path ./weights/Small_AllWeather.pth \
--model_size small
--weights_path: Path to model weights.--model_size: small or large.
Only small model weights are provided due to file size limitations. We provide './weights/Small_AllWeather.pth' in this repository.
If you use this code or models in your research and find it helpful, please cite the following paper:
@article{yamashita2024multiple,
title={Multiple Adverse Weather Removal using Masked-Based Pre-Training and Dual-Pooling Adaptive Convolution},
author={Yamashita, Shugo and Ikehara, Masaaki},
journal={IEEE Access},
year={2024},
publisher={IEEE}
}
This code-base uses certain code-blocks and helper functions from timm, ConvNeXt-V2, weightnet.pytorch, and Transweather.