TY - GEN
T1 - A Perspective on Deep Vision Performance with Standard Image and Video Codecs
AU - Reich, Christoph
AU - Hahn, Oliver
AU - Cremers, Daniel
AU - Roth, Stefan
AU - Debnath, Biplob
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Resource-constrained hardware, such as edge devices or cell phones, often rely on cloud servers to provide the required computational resources for inference in deep vision models. However, transferring image and video data from an edge or mobile device to a cloud server requires coding to deal with network constraints. The use of standardized codecs, such as JPEG or H.264, is prevalent and required to ensure interoperability. This paper aims to examine the implications of employing standardized codecs within deep vision pipelines. We find that using JPEG and H.264 coding significantly deteriorates the accuracy across a broad range of vision tasks and models. For instance, strong compression rates reduce semantic segmentation accuracy by more than 80% in mIoU. In contrast to previous findings, our analysis extends beyond image and action classification to localization and dense prediction tasks, thus providing a more comprehensive perspective.
AB - Resource-constrained hardware, such as edge devices or cell phones, often rely on cloud servers to provide the required computational resources for inference in deep vision models. However, transferring image and video data from an edge or mobile device to a cloud server requires coding to deal with network constraints. The use of standardized codecs, such as JPEG or H.264, is prevalent and required to ensure interoperability. This paper aims to examine the implications of employing standardized codecs within deep vision pipelines. We find that using JPEG and H.264 coding significantly deteriorates the accuracy across a broad range of vision tasks and models. For instance, strong compression rates reduce semantic segmentation accuracy by more than 80% in mIoU. In contrast to previous findings, our analysis extends beyond image and action classification to localization and dense prediction tasks, thus providing a more comprehensive perspective.
KW - Image Classification
KW - Image Compression
KW - Object Detectio
KW - Optical Flow Estimation
KW - Semantic Segmentation
KW - Video Compression
UR - http://www.scopus.com/inward/record.url?scp=85206472629&partnerID=8YFLogxK
U2 - 10.1109/CVPRW63382.2024.00580
DO - 10.1109/CVPRW63382.2024.00580
M3 - Conference contribution
AN - SCOPUS:85206472629
T3 - IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
SP - 5712
EP - 5721
BT - Proceedings - 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024
PB - IEEE Computer Society
T2 - 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024
Y2 - 16 June 2024 through 22 June 2024
ER -