單一影像除雨（Single Image Deraining）旨在從單張雨天影像中恢復乾淨場景。現有深度學習方法多依賴大量成對的雨天、乾淨影像進行監督式訓練，然而收集此類資料成本高昂。為此，近期研究提出自監督式學習框架，透過預測雨紋位置並生成偽乾淨影像以進行模型訓練。然而，既有方法通常直接將偽乾淨影像作為監督訊號，容易將偽標籤中的殘留誤差傳遞至除雨模型，並且模型僅能從單一雨型分佈中學習，限制其泛化能力。

為解決上述問題，本研究提出一種三階段自監督影像除雨框架。首先，在第一階段中，我們利用偽乾淨影像與雨紋遮罩訓練一個可控的加雨生成模型，以學習雨紋形成機制，並將偽乾淨影像的角色限定於雨建模階段，而非直接監督除雨模型。接著，在第二階段中，我們將訓練完成的加雨模型整合至除雨訓練流程中，透過生成多樣化雨紋配置，並結合雨感知重建損失與雨型一致性損失，引導模型學習對雨分佈變化具有穩定性的表示。最後，在第三階段中，我們引入自蒸餾機制，使最終除雨模型在強雨擾動條件下與原始輸入的除雨結果保持一致，進一步對齊主要的恢復目標。

實驗結果顯示，所提出的方法不僅在多個雨天影像資料集上穩定優於現有的自監督除雨方法，還能產生高品質的 pseudo-clean 資料集，並在用於訓練監督式除雨模型時，展現具競爭力的跨域泛化能力。

Single Image Deraining aims to restore clean scenes from a single rainy observation. Existing deep learning approaches largely rely on supervised training with paired rainy and clean images. However, collecting such datasets is costly and labor-intensive. To relax this dependency, recent studies have explored self-supervised frameworks that detect rain regions and generate pseudo-clean images for training. Nevertheless, directly using pseudo-clean images as supervision may lead to inaccurate restoration results, since the pseudo-clean images themselves are not fully accurate and may still contain residual rain streaks or reconstruction errors. Moreover, learning from only limited observed rain patterns may limit the diversity of rain variations available during training, thereby restricting generalization capability.

To address these limitations, we propose a three-stage self-supervised framework for single image deraining. In the first stage, we construct a spatially controllable rain generation model using pseudo-clean images and rain masks to explicitly learn rain formation mechanisms, while restricting the role of pseudo-clean images to rain modeling rather than direct restoration supervision. In the second stage, the trained rain generator is integrated into the deraining training pipeline to perform rain-informed augmentation by producing diverse rain configurations. By incorporating a rain-constrained reconstruction loss and a rain-variation consistency loss, the deraining network is encouraged to learn stable representations under diverse rain variations. In the third stage, we introduce a self-distillation mechanism that aligns the model's predictions under strong rain perturbations with those obtained from the original rainy input, thereby reinforcing consistency toward the primary restoration objective.

Extensive experiments demonstrate that the proposed method not only consistently outperforms existing self-supervised deraining approaches across multiple rainy datasets, but also produces pseudo-clean datasets that enable competitive cross-domain generalization when used to train supervised deraining models.

[1] Pablo Arbelaez, Micheal Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hierarchical image segmentation. In IEEE transactions on pattern analysis and machine intelligence, 2010.
[2] Yunhao Ba, Howard Zhang, Ethan Yang, Akira Suzuki, Arnold Pfahnl, Chethan Chinder Chandrappa, Celso M. De Melo, Suya You, Stefano Soatto, Alex Wong, and Achuta Kadambi. Not just streaks: Towards ground truth for single image deraining. In ECCV, 2022.
[3] Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. In ICML, 2019.
[4] Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. In ECCV, 2022.
[5] Yi-Lei Chen and Chiou-Ting Hsu. A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In CVPR, 2013.
[6] C. Fowlkes D. Martin, D. Tal, and J. Malik. Adatabase of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, 2001.
[7] Liang-Jian Deng, Ting-Zhu Huang, Xi-Le Zhao, and Tai-Xiang Jiang. A directional global sparse model for single image rain removal. In Applied Mathematical Modelling, 2018.
[8] David Eigen, Dilip Krishnan, and Rob Fergus. Restoring an image taken through a window covered with dirt or rain. In ICCV, 2013.
[9] Ning Gao, Xingyu Jiang, Xiuhui Zhang, and Yue Deng. Efficient frequency-domain image deraining with contrastive regularization. In ECCV, 2024.
[10] Kshitiz Garg and Shree K Nayar. Photorealistic rendering of rain streaks. In ACM Transactions on Graphics(TOG), 2006.
[11] Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, and Jianzhuang Liu. Neighbor2neighbor: Self-supervised denoising from single noisy images. In CVPR, 2021.
[12] Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu. Automatic single-image-based rain streaks removal via image decomposition. In TIP, 2011.
[13] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[14] Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-learning denoising from single noisy images. In CVPR, 2019.
[15] Wooseok Lee, Sanghyun Son, and Kyoung Mu Lee. Ap-bsn: Self-supervised denoising for real-world images via asymmetric pd and blind-spot network. In CVPR, 2022.
[16] Wei Li, Qiming Zhang, Jing Zhang, Zhen Huang, Xinmei Tian, and Dacheng Tao. Toward real-world single image deraining: A new benchmark and beyond, 2022. arXiv preprint arXiv:2206.05514.
[17] Yu Li, Robby T. Tan, Xiaojie Guo, Jiangbo Lu, and Michael S. Brown. Rain streak removal using layer priors. In CVPR, 2016.
[18] Yu Luo, Yong Xul, and Hui Ji. Removing rain from a single image via discriminative sparse coding. In ICCV, 2015.
[19] A. Mittal, Moorthy, A. K., and A. C. Bovik. No-Reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 2012.
[20] A. Mittal, Soundararajan, R., and A. C. Bovik. Making a “Completely blind” image quality analyzer. IEEE Signal Processing Letters, pages 209–212, 2012.
[21] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, and et al. Luca Antiga. Pytorch: An imperative style, high-performance deep learning library. In Advances in neural information processing systems, 2019.
[22] Yan-Tsung Peng and Wei-Hua Li. Rain2avoid: Self-supervised single image deraining. In ICASSP, 2023.
[23] Yan-Tsung Peng, Wei-Hua Li, and Zihao Chen. Rain2avoid: Learning deraining by self-supervision. In TMM, 2025.
[24] Ruijie Quan, Xin Yu, Yuanzhi Liang, and Yi Yang. Removing raindrops and rain streaks in one go. In CVPR, 2021.
[25] Shyam Nandan Rai, Rohit Saluja, Chetan Arora, Vineeth N Balasubramanian, Anbumani Subramanian, and CV Jawahar. Fluid: Few-shot self-supervised image deraining. In WACV, 2022.
[26] Gerald Schaefer and Michal Stich. Ucid: An uncompressed color image database. In SPIE, 2003.
[27] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. InCVPR, 2018.
[28] Yinglong Wang, Shuaicheng Liu, Chen Chen, and Bing Zeng. A hierarchical approach for rain or snow removing in a single color image. In TIP, 2017.
[29] Wei Wei, Deyu Meng, Qian Zhao, Zongben Xu, and Ying Wu. Semi-supervised transfer learning for image rain removal. In CVPR, 2019.
[30] Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo1, and Shuicheng Yan. Deep joint rain detection and removal from a single image. In CVPR, 2017.
[31] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, 2022.
[32] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Multi-Stage Progressive Image Restoration. In CVPR, 2021.
[33] Dan Zhang, Fangfang Zhou, Yuwen Jiang, and Zhengming Fu. Mm-bsn: Selfsupervised image denoising for real-world with multi-mask based on blind-spot network. In CVPRW, 2023.
[34] Fan Zhang, Shaodi You, Yu Li, and Ying Fu. Gtav-nightrain: Photometric realistic large-scale dataset for night-time rain streak removal, 2022. arXiv preprint arXiv:2210.04708.
[35] He Zhang, Vishwanath Sindagi, and Vishal M Patel. Image de-raining using a conditional generative adversarial network. In IEEE transactions on circuits and systems for video technology, 2019.
[36] Yunhao Zou, Chenggang Yan, and Ying Fu. Iterative denoiser and noise estimator for self-supervised image denoising. In ICCV, 2023.