影像除雨是一種低階還原任務，在過去十幾年中變得非常熱門。影像除雨目的為恢復有雨的影像中的架構與細節紋理，並同時能處理各種影像大小與不同場景。儘管已存在許多強大的影像處雨模型，但大部分的研究方法都著重在建構更深更複雜的架構來訓練網路。因此，我們提出一個根據Transformer架構的除雨架構，並將網路切分成兩個部分，其中包含一個全域雨局部的雨水感知注意力模組;一個有效收集不同解析度下紋理資訊的MLP空洞卷機模組。透過廣泛的實驗與驗證，實驗結果顯示所提出方法之優越性。

Image deraining is a low-level restoration task that has become quite popular during the past decades. Its claims not only recover the spatial detail and high-level contextual structure of input rainy image but also need to deal with various resolutions or scenes. Although there exist a lot of robust deraining networks, the dominant researchers devote to constructing more deeper and complicated architecture to reconstruct the image texture. This work presents a transformer emulator which includes its own design Global and Local Rain-aware Attention (GLRA) and Atrous Convolution with MLP (ACMLP) for efficient exploring both detailed structure and semantic contexts. We validate performance via wide-ranging experiments, including synthetic and real world datasets showing the proposed method's superiority.

[1] Yu Li, Robby T Tan, Xiaojie Guo, Jiangbo Lu, and Michael S Brown, “Rain streak removal using layer priors,” in Proc. Conf. Computer Vision and Pattern Recognition, 2016.
[2] Yi Chang, Luxin Yan, and Sheng Zhong, “Transformed low-rank model for line pattern noise removal,” in Proc. Int’l Conf. Computer Vision, 2017.
[3] Shyam Nandan Rai, Rohit Saluja, Chetan Arora, Vineeth N Balasubramanian, Anbumani Subramanian, and CV Jawahar, “Fluid: Few-shot self-supervised image deraining,” in Proc. of the IEEE/CVF Winter Conf. on Applications of Computer Vision, 2022.
[4] Yang Liu, Ziyu Yue, Jinshan Pan, and Zhixun Su, “Unpaired learning for deep image deraining with rain direction regularizer,” in Proc. Int’l Conf. Computer Vision, 2021.
[5] Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan, “Deep joint rain detection and removal from a single image,” in Proc. Conf. Computer Vision and Pattern Recognition, 2017.
[6] Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Baojin Huang, Yimin Luo, Jiayi Ma, and Junjun Jiang, “Multi-scale progressive fusion network for single image deraining,” in Proc. Conf. Computer Vision and Pattern Recognition, 2020.
[7] Hong Wang, Qi Xie, Qian Zhao, and Deyu Meng, “A model-driven deep neural network for single image rain removal,” in Proc. Conf. Computer Vision and Pattern Recognition, 2020.
[8] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao, “Multi-stage progressive image restoration,”in Proc. Conf. Computer Vision and Pattern Recognition, 2021.
[9] Yuanchu Liang, Saeed Anwar, and Yang Liu, “Drt: A lightweight single image deraining recursive transformer,” in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[10] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang, “Restormer: Efficient transformer for high-resolution image restoration,” in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[11] Hunsang Lee, Hyesong Choi, Kwanghoon Sohn, and Dongbo Min, “Knn local attention for image restoration,” in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[12] Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh, and I-Hau Yeh, “Cspnet: A new backbone that can enhance learning capability of cnn,” in Proc. Conf. Computer Vision and Pattern Recognition, 2020.
[13] Wenhan Yang, Robby T Tan, Jiashi Feng, Zongming Guo, Shuicheng Yan, and Jiaying Liu, “Joint rain detection and removal from a single image with contextualized deep networks,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 2019.
[14] He Zhang, Vishwanath Sindagi, and Vishal M Patel, “Image de-raining using a conditional generative adversarial network,” IEEE transactions on circuits and systems for video technology, 2019.
[15] Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley,“Removing rain from single images via a deep detail network,” in Proc. Conf. Computer Vision and Pattern Recognition, 2017.
[16] He Zhang and Vishal M Patel, “Density-aware single image de-raining using a multistream dense network,” in Proc. Conf. Computer Vision and Pattern Recognition, 2018.
[17] Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson W.H. Lau, “Spatial attentive single-image deraining with a high quality real rain dataset,”in Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[18] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. Int’l Conf. Learning Representations, 2021.
[19] Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao, “Pre-trained image processing transformer,” in Proc. Conf. Computer Vision and Pattern Recognition, 2021.
[20] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie, “Feature pyramid networks for object detection,” in Proc. Conf. Computer Vision and Pattern Recognition, 2017.
[21] Khan Muhammad, Jamil Ahmad, Zhihan Lv, Paolo Bellavista, Po Yang, and Sung Wook Baik, “Efficient deep cnn-based fire detection and localization in video surveillance applications,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018.
[22] Chenyi Chen, Ari Seff, Alain Kornhauser, and Jianxiong Xiao, “Deepdriving: Learning affordance for direct perception in autonomous driving,” in Proc. Int’l Conf. Computer Vision, 2015.
[23] Shuhang Gu, Deyu Meng, Wangmeng Zuo, and Lei Zhang, “Joint convolutional analysis and synthesis sparse representation for single image layer separation,” in Proc. Int’l Conf. Computer Vision, 2017.
[24] Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu, “Automatic single-image-based rain streaks removal via image decomposition,” IEEE Trans. on Image Processing, 2011.
[25] Yi-Lei Chen and Chiou-Ting Hsu, “A generalized low-rank appearance model for spatio-temporally correlated rain streaks,” in Proc. Int’l Conf. Computer Vision, 2013.
[26] Sen Deng, Mingqiang Wei, Jun Wang, Yidan Feng, Luming Liang, Haoran Xie, Fu Lee Wang, and Meng Wang, “Detail-recovery image deraining via context aggregation networks,” in Proc. Conf. Computer Vision and Pattern Recognition, 2020.
[27] Siyuan Li, Wenqi Ren, Jiawan Zhang, Jinke Yu, and Xiaojie Guo, “Single image rain removal via a deep decomposition–composition network,” Computer Vision and Image Understanding, 2019.
[28] Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang, “Learning deep cnn denoiser prior for image restoration,” in Proc. Conf. Computer Vision and Pattern Recognition, 2017.
[29] Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha, “Recurrent squeeze-and-excitation context aggregation net for single image deraining,” in Proceedings of the European conference on computer vision (ECCV), 2018.
[30] Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng, “Progressive image deraining networks: A better and simpler baseline,” in Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[31] Bo Pang, Deming Zhai, Junjun Jiang, and Xianming Liu, “Single image deraining via scale-space invariant attention neural network,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020.
[32] Rajeev Yasarla and Vishal M Patel, “Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining,” in Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[33] Hongyuan Zhu, Xi Peng, Joey Tianyi Zhou, Songfan Yang, Vijay Chanderasekh, Liyuan Li, and Joo-Hwee Lim, “Singe image rain removal with unpaired information: A differentiable programming perspective,” in Proc. Nat’l Conf. Artificial Intelligence, 2019.
[34] Changfeng Yu, Yi Chang, Yi Li, Xile Zhao, and Luxin Yan, “Unsupervised image deraining: Optimization model driven deep cnn,” in Proceedings of the 29th ACM International Conference on Multimedia, 2021.
[35] Xin Jin, Zhibo Chen, Jianxin Lin, Zhikai Chen, and Wei Zhou, “Unsupervised single image deraining with self-supervised constraints,” in Proc. Int’l Conf. Image Processing. IEEE, 2019.
[36] Yanyan Wei, Zhao Zhang, Yang Wang, Mingliang Xu, Yi Yang, Shuicheng Yan, and Meng Wang, “Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking,” IEEE Trans. on Image Processing, 2021.
[37] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proc. Int’l Conf. Computer Vision, 2017.
[38] Wei Wei, Deyu Meng, Qian Zhao, Zongben Xu, and Ying Wu, “Semi-supervised transfer learning for image rain removal,” in Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[39] Rajeev Yasarla, Vishwanath A Sindagi, and Vishal M Patel, “Syn2real transfer learning for image deraining using gaussian processes,” in Proc. Conf. Computer Vision and Pattern Recognition, 2020.
[40] Yuntong Ye, Yi Chang, Hanyu Zhou, and Luxin Yan, “Closing the loop: Joint rain generation and removal via disentangled image translation,” in Proc. Conf. Computer Vision and Pattern Recognition, 2021.
[41] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” Proc. Neural Information Processing Systems, 2017.
[42] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
[43] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko, “End-to-end object detection with transformers,”in Proc. Euro. Conf. Computer Vision. Springer, 2020.
[44] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proc. Int’l Conf. Computer Vision, 2021.
[45] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li, “Uformer: A general u-shaped transformer for image restoration,” in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[46] Lin Liu, Lingxi Xie, Xiaopeng Zhang, Shanxin Yuan, Xiangyu Chen, Wengang Zhou, Houqiang Li, and Qi Tian, “Tape: Task-agnostic prior embedding for image restoration,” arXiv preprint arXiv:2203.06074, 2022.
[47] Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng, “All-inone image restoration for unknown corruption,” in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[48] Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M Patel, “Transweather: Transformer-based restoration of images degraded by adverse weather conditions,”in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[49] Yancheng Wang, Ning Xu, Chong Chen, and Yingzhen Yang, “Adaptive cross-layer attention for image restoration,” arXiv preprint arXiv:2203.03619, 2022.
[50] Akshay Dudhane, Syed Waqas Zamir, Salman Khan, Fahad Shahbaz Khan, and Ming-Hsuan Yang, “Burst image restoration and enhancement,” in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[51] Wenhan Yang, Robby T Tan, Shiqi Wang, Yuming Fang, and Jiaying Liu, “Single image deraining: From model-based to data-driven and beyond,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 2020.
[52] Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu, “Automatic single-image-based rain streaks removal via image decomposition,” IEEE Trans. on Image Processing, 2011.
[53] Jin-Hwan Kim, Jae-Young Sim, and Chang-Su Kim, “Video deraining and desnowing using temporal correlation and low-rank matrix completion,” IEEE Trans. on Image Processing, 2015.
[54] Yu Luo, Yong Xu, and Hui Ji, “Removing rain from a single image via discriminative sparse coding,” in Proc. Int’l Conf. Computer Vision, 2015.
[55] Kewen Han and Xinguang Xiang, “Decomposed cyclegan for single image deraining with unpaired data,” in Proc. Int’l Conf. Acoustics, Speech, and Signal Processing. IEEE, 2020.
[56] Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley, “Clearing the skies: A deep network architecture for single-image rain removal,” IEEE Trans. on Image Processing, 2017.
[57] Jie Hu, Li Shen, and Gang Sun, “Squeeze-and-excitation networks, 7132–7141,”in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, UT, 2018.
[58] Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson WH Lau, “Spatial attentive single-image deraining with a high quality real rain dataset,” in Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[59] Dongwei Ren, Wei Shang, Pengfei Zhu, Qinghua Hu, Deyu Meng, and Wangmeng Zuo, “Single image deraining using bilateral recurrent network,” IEEE Trans. on Image Processing, 2020.
[60] Chenghao Chen and Hao Li, “Robust representation learning with feedback for single image deraining,” in Proc. Conf. Computer Vision and Pattern Recognition, 2021.
[61] Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, and Chengpeng Chen, “Hinet: Half instance normalization network for image restoration,” in Proc. Conf. Computer Vision and Pattern Recognition, 2021.
[62] Kuldeep Purohit, Maitreya Suin, AN Rajagopalan, and Vishnu Naresh Boddeti,“Spatially-adaptive image restoration using distortion-guided networks,” in Proc. Int’l Conf. Computer Vision, 2021.
[63] Yizhou Li, Yusuke Monno, and Masatoshi Okutomi, “Single image deraining network with rain embedding consistency and layered lstm,” in Proc. of the IEEE/CVF Winter Conf. on Applications of Computer Vision, 2022.
[64] Pengpeng Li, Jiyu Jin, Guiyue Jin, Lei Fan, Xiao Gao, Tianyu Song, and Xiang Chen, “Deep scale-space mining network for single image deraining,” in Proc.
Conf. Computer Vision and Pattern Recognition, 2022.
[65] Yuuto Nanba, Hikaru Miyata, and Xian-Hua Han, “Dual heterogeneous complementary networks for single image deraining,” in Proc. Conf. Computer Vision and Pattern Recognition, 2022.
[66] Xiang Li, Wenhai Wang, Xiaolin Hu, and Jian Yang, “Selective kernel networks,”in Proc. Conf. Computer Vision and Pattern Recognition, 2019.
[67] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al., “Pytorch: An imperative style, high-performance deep learning library,” Proc. Neural Information Processing Systems, 2019.
[68] Diederik P Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
[69] Quan Huynh-Thu and Mohammed Ghanbari, “Scope of validity of psnr in image/video quality assessment,” Electronics letters, 2008.
[70] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Processing, 2004.