Omni-Kernel Modulation for Universal Image Restoration

Yuning Cui, Wenqi Ren, Alois Knoll

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Image restoration is the process of recovering a clean image from a degraded observation. In order to achieve this, it is essential to refine features at multiple scales. This paper develops an effective omni-kernel modulation module to enhance multi-scale representation learning for image restoration. The module consists of three branches, namely global, large, and local branches, which are designed to learn global-to-local feature representations efficiently. Specifically, the global branch achieves a global perceptive field via the dual-domain channel attention and frequency-gated mechanism. Furthermore, to provide multi-grained receptive fields, the large branch is formulated using different shapes of depth-wise convolutions with unusually large kernel sizes. Moreover, we complement local information with a point-wise depth-wise convolution. Finally, we demonstrate the effectiveness of our omni-kernel modulation module in two cases: general image restoration and all-in-one image restoration tasks. Incorporating our method into a convolutional backbone results in a model that achieves state-of-the-art performance on the 15 datasets for three representative image restoration tasks, including image dehazing, desnowing, and defocus deblurring. Moreover, by integrating our module into a pure Transformer-based backbone, the model demonstrates competitive performance against state-of-the-art algorithms in two all-in-one image restoration settings: the three-task and five-task settings.

Original languageEnglish
Pages (from-to)12496-12509
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume34
Issue number12
DOIs
StatePublished - 2024

Keywords

  • Omni-kernel modulation
  • all-in-one image restoration
  • image restoration

Fingerprint

Dive into the research topics of 'Omni-Kernel Modulation for Universal Image Restoration'. Together they form a unique fingerprint.

Cite this