JPEG XS (standardized as ISO/IEC 21122) is an image and video codec that offers both visually and mathematically lossless quality. It is a special-purpose codec that is specifically designed to allow for low-complexity and low-latency implementations.T. Richter, J. Keinert, S. Foessel, A. Descampe, G. Rouvroy and J. -B. Lorent, "JPEG-XS -- A High-Quality Mezzanine Image Codec for Video Over IP," in SMPTE Motion Imaging Journal, vol. 127, no. 9, pp. 39-49, Oct. 2018, doi: 10.5594/JMI.2018.2862098 link. Target applications of the standard include streaming high-quality content for professional video over IP (SMPTE ST 2022 and ST 2110) in broadcast and other applications: virtual reality, drones, autonomous vehicles using cameras, gaming. Although there is not an official acronym definition, XS was chosen to highlight the extra small and extra speed characteristics of the codec.
Features
Three main features are key to JPEG XS:
Visually transparent compression: XS compressed content is indistinguishable from the original uncompressed content (passing ISO/IEC 29170-2 tests, also called AIC-2) for compression ratios between 2:1 and 10:1. Low latency: The total end-to-end latency, introduced by the JPEG XS compression-decompression cycle, is minimal. Depending on the configuration, XS typically imposes only between 1 and 32 lines of additional end-to-end latency, when compared to the same system using uncompressed video. Lightweight: JPEG XS is designed to have low computational and memory complexity, allowing for efficient low-power and low-resource implementations on various platforms such as CPU, GPU, FPGA and ASIC.
Relying on these key features, JPEG XS is suitable for any application where uncompressed content is the norm, yet still allowing for significant savings in the required bandwidth usage, preserving quality and low latency. Among the targeted use cases are video transport over professional video links (like SDI and professional video over IP), real-time video storage, memory buffers, omnidirectional video capture and rendering, and image sensor compression (for example in cameras and in the automotive industry). JPEG XS favors visually lossless quality in combination with low latency and low complexity, over data reduction through compression. It is not a direct competitor to alternative image codecs like JPEG 2000 and JPEG XL or video codecs like AV1, AVC/H.264 and HEVC/H.265 which tend to focus on compression efficiency.
Other important features are:
Exact bitrate allocation: JPEG XS allows an accurately targeted bitrate to perfectly match the available bandwidth (also referred to as constant bitrate). Multi-generation robustness: JPEG XS allows for at least 10 encoding-decoding cycles without significant quality degradation. This feature allows, for example, transparent chaining of multiple devices that recompress the signal, without any significant quality degradation taking place. Multi-platform interoperability: The algorithms used in JPEG XS allow for efficient implementations on different platforms, like CPU, GPU, FPGA and ASIC. Each of these platform architectures is best exploited when a specific degree of parallelism is available in the implementation. For instance, a multi-core CPU implementation will leverage coarse-grained parallelism, while a GPU or FPGA implementation will work better with fine-grained parallelism. Moreover, the choice of parallelism used in the implementation at the encoder will not affect that of the decoder. This means that real-time encoding and decoding between platforms is possible, without sacrificing the low-complexity, low-latency or high-quality properties. Support for mathematical lossless coding (MLS): JPEG XS is also capable of coding images in a mathematically lossless way, to achieve perfect reconstruction at the decoder side (new profile supported by 2nd edition). Support for High Dynamic Range (HDR) content: The current version of JPEG XS supports bit-depths of up to 16 bits per component, and it provides several parameterizable non-linear transforms to efficiently compress HDR content. Support for RAW Bayer/CFA compression: JPEG XS has also the capability to compress color filter array (CFA) content, such as RAW Bayer content produced by digital cameras. A special color transform, called Star-Tetrix, allows for efficient and direct compression of the original RAW sample values, without the need for converting the Bayer samples to RGB samples first.T. Richter, S. Fößel, A. Descampe and G. Rouvroy, "Bayer CFA Pattern Compression With JPEG XS," in IEEE Transactions on Image Processing, vol. 30, pp. 6557-6569, 2021, doi: 10.1109/TIP.2021.3095421 link. Accurate flow control: A JPEG XS encoder continuously monitors the amount of bits sent out, and adjusts its rate allocation process to neither overflow nor underflow a normatively defined decoder input buffer.
Application domains
JPEG XS is actively used in several application domains.
Transport over video links and IP networks
Video bandwidth requirements are growing as video resolutions, frame rates, bit depths, and the number of video streams are constantly increasing. Although the capacities of video links and communication channels are also growing, there are applications where the reduced data rate afforded by JPEG-XS is advantageous.
Both the broadcast and pro-AV markets are shifting towards AV-over-IP-based infrastructure, with a preference going to Gigabit Ethernet links for remote production and faster fiber-based Ethernet for in-house facilities. Given the available bandwidth and infrastructure cost, relying on uncompressed video may be cost-prohibitive if 4K, 8K, increased bit depths for HDR, and higher frame rates need to be supported.
With XS, it is, for example, possible to repurpose existing SDI cables to transport 4K60 over a single 3G-SDI (at 4:1 compression ratio), and even over a single HD-SDI (at 8:1 compression ratio). Similar scenarios can be used to transport 8K60 content over various SDI cable types (e.g., 6G-SDI and 12G-SDI). Alternatively, XS enables transporting 4K60 content over Gigabit Ethernet and 8K60 over 5G or 10 Gigabit Ethernet, which would be impossible without compression.
The following table shows expected compression ranges for some typical use cases.
Video stream Video throughput Link type Available throughput Compression ratio 2k 60 fps 4:2:2 10 bpc HD-SDI ~2:1 4k 60 fps 4:2:2 10 bpc 3G-SDI ~4:1 2k 60 fps 4:2:2 10 bpc 1G Ethernet ~3:1 2k 60 fps 4:4:4 12 bpc 1G Ethernet ~6:1 4k 60 fps 4:4:4 12 bpc 10G Ethernet ~2.2:1 8k 60 fps 4:2:2 10 bpc 10G Ethernet ~6:1 8k 120 fps 4:2:2 10 bpc 25G Ethernet ~4:1
Real-time video storage and playout
Related to the transport of video streams is the storage and retrieval of high-resolution streams where bandwidth limitations similarly apply. For instance, video cameras use internal storage like SSD drives or SD cards to record high-resolution streams of images, yet the maximum data rates of such storage devices may be below that required to store uncompressed video in real time.
Sensor compression
JPEG XS has built-in support for the direct compression of RAW Bayer/CFA images using the Star-Tetrix Color Transform. This transform takes a RAW Bayer pattern image and decorrelates the samples into a 4-component image with each component having only a quarter of the resolution. This means that the total amount of samples to further process and compress remains the same, yet the values are decorrelated similarly to a classical Multiple Component Transform.
Avoiding such conversion prevents information loss and allows this processing step to be done outside of the camera. This is advantageous because it allows deferring demosaicing the Bayer content from the moment of capturing to the production phase, where choices regarding artistic intent and various settings can be better made; The demosaicing process is irreversible and requires certain choices, like the choice of interpolation algorithm or the level of noise reduction, to be made upfront. The demosaicing process can be power-hungry and will also introduce extra latency and complexity.
Standards
JPEG XS (ISO/IEC 21122)
The JPEG XS coding system is an ISO/IEC suite of standards that consists of the following parts:
Part 1st edition 2nd edition (in force) 3rd edition Title Description 1 2019 2022 2024Amd1:2026 Core coding system Describes the core coding system of JPEG XS. Defines the syntax and, similarly to other JPEG and MPEG image codecs, the decompression process to reconstruct a continuous-tone digital image from its encoded codestream. Provides some guidelines for the inverse process that compresses a digital image into a compressed codestream, the encoding process, but leaves implementation-specific optimizations and choices to the implementers. 2 2019 2022 2024Amd1:2026DAmd2 Profiles and buffer models Builds on top of Part 1 to segregate different applications and uses of JPEG XS into reduced coding tool subsets with tighter constraints. The definition of profiles, levels, and sublevels allows for reducing the complexity of implementations in particular application use cases, while also safeguarding interoperability. Profiles represent interoperability subsets of the codestream syntax specified in Part 1. In addition, levels and sublevels provide limits to the maximum throughput in the encoded (codestream) and the decoded (spatial and pixels) image domains. Also specifies a buffer model, consisting of a decoder model and a transmission channel model, to enable guaranteeing low latency requirements to a fraction of the frame size. 3 2019 2022 2024 Transport and container formats Specifies transport and container formats for JPEG XS codestreams. It defines the carriage of important metadata, like color spaces, mastering display metadata (MDM), and Exif, to facilitate transport, editing, and presentation. Defines the XS-specific ISOBMFF boxes, an Internet media type registration, and additional syntax to allow embedding XS in formats like MP4, MPEG-2 TS, or the HEIF image file format. 4 2020 2022 2025 Conformance testing Provides conformance testing and buffer model verification. 5 2020 2022 2025 Reference software A reference software implementation (written in ISO C11) of the JPEG XS Part 1 decoder, conforming to the Part 2 profiles, levels and sublevels, as well as an exemplary encoder implementation.
RFC9134 - RTP Payload Format for ISO/IEC 21122 (JPEG XS)
RFC 9134 describes a payload format for the Real-Time Transport Protocol (RTP) to carry JPEG XS encoded video. In addition, the recommendation also registers the official media type registration for JPEG XS video as , along with its mapping of all parameters into the Session Description Protocol (SDP).
The RTP payload format for JPEG XS, in turn, enables the use of JPEG XS in SMPTE 2110 environments using SMPTE ST 2110-22 for CBR compressed video transport.
See also TR-08,TR-08: Transport of JPEG XS Video in ST 2110-22 published by the Video Services Forum.
MPEG-TS for JPEG XS
ISO/IEC 13818-1:2023, known as MPEG-TS 9th edition, specifies carriage support for JPEG XS in MPEG transport streams. See also VSF TR-07TR-07: Transport of JPEG XS Video in MPEG-2 TS over IP published by the Video Services Forum.
NMOS with JPEG XS
A Networked Media Open Specifications (NMOS) standard that enables registration, discovery, and connection management of JPEG XS endpoints uses the AMWA IS-04 and IS-05 NMOS specifications. See AMWA BCP-006-01,AMWA BCP-006-01: NMOS With JPEG XS published by Advanced Media Workflow Association.
JPEG XS in IPMX