TY - JOUR
T1 - Color Image Restoration Exploiting Inter-Channel Correlation with a 3-Stage CNN
AU - Cui, Kai
AU - Boev, Atanas
AU - Alshina, Elena
AU - Steinbach, Eckehard
N1 - Publisher Copyright:
© 2007-2012 IEEE.
PY - 2021/2
Y1 - 2021/2
N2 - Image restoration is a critical component of image processing pipelines and for low-level computer vision tasks. Conventional image restoration approaches are mostly based on hand-crafted image priors. The inter-channel correlation of color images is not fully exploited. Motivated by the special characteristics of the inter-channel correlation (higher correlation for red/green and green/blue channels than for red/blue) in color images and general characteristics (green channel always shows the best image quality among the three color components) of distorted color images, in this paper, a three-stage convolutional neural network (CNN) structure is proposed for color image restoration tasks. Since the green channel is found to have the best quality among all three channels, in the first stage, the network is designed to reconstruct the green component. Then, with the guidance of the reconstructed green channel from the first stage, the red and blue channels are reconstructed in the second stage with two parallel networks. Finally, the intermediate reconstructions from the previous stages are concatenated and further refined jointly. We demonstrate the capabilities of the proposed three-stage structure with three typical color image restoration tasks: color image demosaicking, color compression artifacts reduction, and real-world color image denoising. In addition, we integrate pixel-shuffle convolution into our scheme to improve the efficiency, and also introduce a quality-blind training strategy to simplify the training process for the compression artifacts reduction task. Extensive experimental results and analyses show that the proposed structure successfully exploits the spatial and inter-channel correlation of color images and outperforms the state-of-the-art image reconstruction approaches.
AB - Image restoration is a critical component of image processing pipelines and for low-level computer vision tasks. Conventional image restoration approaches are mostly based on hand-crafted image priors. The inter-channel correlation of color images is not fully exploited. Motivated by the special characteristics of the inter-channel correlation (higher correlation for red/green and green/blue channels than for red/blue) in color images and general characteristics (green channel always shows the best image quality among the three color components) of distorted color images, in this paper, a three-stage convolutional neural network (CNN) structure is proposed for color image restoration tasks. Since the green channel is found to have the best quality among all three channels, in the first stage, the network is designed to reconstruct the green component. Then, with the guidance of the reconstructed green channel from the first stage, the red and blue channels are reconstructed in the second stage with two parallel networks. Finally, the intermediate reconstructions from the previous stages are concatenated and further refined jointly. We demonstrate the capabilities of the proposed three-stage structure with three typical color image restoration tasks: color image demosaicking, color compression artifacts reduction, and real-world color image denoising. In addition, we integrate pixel-shuffle convolution into our scheme to improve the efficiency, and also introduce a quality-blind training strategy to simplify the training process for the compression artifacts reduction task. Extensive experimental results and analyses show that the proposed structure successfully exploits the spatial and inter-channel correlation of color images and outperforms the state-of-the-art image reconstruction approaches.
KW - Color image restoration
KW - Compression artifacts reduction
KW - Convolutional Neural Network
KW - Demosaicking
KW - Inter-channel Correlation
KW - Realistic image denoising
UR - http://www.scopus.com/inward/record.url?scp=85097925726&partnerID=8YFLogxK
U2 - 10.1109/JSTSP.2020.3043148
DO - 10.1109/JSTSP.2020.3043148
M3 - Article
AN - SCOPUS:85097925726
SN - 1932-4553
VL - 15
SP - 174
EP - 189
JO - IEEE Journal on Selected Topics in Signal Processing
JF - IEEE Journal on Selected Topics in Signal Processing
IS - 2
M1 - 9286520
ER -