Signal Processing: Image Communication 99 (2021) 116503 Contents lists available at ScienceDirect Signal Processing: Image Communication journal homepage: www.elsevier.com/locate/image Real-time 16K video coding on a GPU with complexity scalable BPC-PaCo Carlos de Cea-Dominguez a ,∗, Juan C. Moure b , Joan Bartrina-Rapesta a , Francesc Aulí-Llinàs a a Department of Information and Communications Engineering, Universitat Autònoma de Barcelona, 08193 Bellaterra, Spain b Department of Computer Architecture and Operating Systems, Universitat Autònoma de Barcelona, 08193 Bellaterra, Spain ARTICLE INFO ABSTRACT Keywords: The advent of new technologies such as high dynamic range or 8K screens has enhanced the quality of digital High throughput image and video coding images but it has also increased the codecs’ computational demands to process such data. This paper presents a GPU video codec that, while providing the same coding features and performance as those of JPEG2000, can process CUDA 16K video in real time using a consumer-grade GPU. This high throughput is achieved with a technique that JPEG2000 introduces complexity scalability to a bitplane coding engine, which is the most computationally complex HTJ2K stage of the coding pipeline. The resulting codec can trade throughput for coding performance depending on the user’s needs. Experimental results suggest that our method can double the throughput achieved by CPU implementations of the recently approved High-Throughput JPEG2000 and by hardwired implementations of HEVC in a GPU. 1. Introduction When the algorithms exhibit fine-grained parallelism, implemen- tations in GPUs can achieve high throughput thanks to the inherent Image and video coding are primary needs of industries such as Single Instruction Multiple Data (SIMD) architecture of these devices digital cinema, content streaming or video production, among others. in combination with a Multiple Instruction Multiple Data (MIMD) Two main standards satisfy the requirements of many such industries, architecture. Together, both characteristics allow processing thousands namely, JPEG2000 [1] and HEVC [2]. JPEG2000 is commonly em- of threads executing the same instruction on different data. Some ployed in digital cinema and medical imaging, whereas HEVC is often algorithms can be accelerated up to 20× as compared to implemen- used for media streaming and video production. Both standards have tations on traditional Central Processing Units (CPUs) [17]. Unfortu- advanced features like high compression efficiency, quality scalability, nately, such speedups are not achieved when implementing conven- interactive transmission, or error resilience. Both standards also de- tional image/video codecs because their core algorithms exhibit poor mand ample computational resources, posing a challenge when high fine-grained parallelism. In general, these algorithms are devised to quality video (of 4K or more resolution and/or with high dynamic exploit only the MIMD-based architecture of CPUs (or GPUs), which can range) need to be coded in real time. In computational-constrained process tenths of threads executing different instructions on different devices, the image quality may need to be reduced to achieve real-time data. processing. Other scenarios such as digital cinema or medical imaging The coding pipeline of a traditional JPEG2000 codec has three main require the highest quality possible, so expensive hardware solutions stages: discrete wavelet transform (DWT), bitplane and entropy coding are often in use. The literature employs different approaches to increase the codecs’ (BPC), and codestream re-organization (CR). The DWT and CR stages throughput. Some works focus on the coding algorithms to reduce com- can be easily mapped to a SIMD-based architecture since their opera- putational complexity [3–5]. Others implement the codec in hardware tions can be parallelized and do not hold critical data dependencies. devices such as Field-Programmable Gate Arrays (FPGAs) [6–9]. FPGAs Contrarily, the BPC stage has data dependencies that force the samples are attractive despite their high price due to their high performance, to be coded sequentially in each tile of data. This stage accounts for so they are used in scenarios such as digital cinema [10] or medical 85% of the total execution time, so it is key in the codec’s overall imaging [11–13]. The highly parallel architecture of Graphics Process- throughput. ing Units (GPUs) has also been employed to parallelize the codec’s Aimed to provide more parallelism to the BPC engine, the Joint tasks [14–16]. The lower cost and the capacity for general-purpose Photographic Experts Group approved in 2019 Part 15 of the standard, computing of GPUs have made these accelerators very popular in recent named High-Throughput JPEG2000 (HTJ2K). This new part adopts the years. algorithm proposed in [18], which exploits vector (SIMD) instructions ∗ Correspondence to: Escola Enginyeria, UAB - 08193 Bellaterra, Spain. E-mail addresses: carlos.decea@uab.cat (C. de Cea-Dominguez), juancarlos.moure@uab.cat (J.C. Moure), joan.bartrina@uab.cat (J. Bartrina-Rapesta), francesc.auli@uab.cat (F. Aulí-Llinàs). https://doi.org/10.1016/j.image.2021.116503 Received 1 March 2021; Received in revised form 2 September 2021; Accepted 11 September 2021 Available online 21 September 2021 0923-5965/© 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 Fig. 1. Evaluation of rate–distortion performance for JPEG2000, HTJ2K, BPC-PaCo and CS BPC-PaCo (with 𝐾 = 0.5) when transmitting the color image ‘‘Portrait’’ at 100 Fig. 2. Memory hierarchy of a Nvidia GPU. different rates. different test conditions. Finally, experimental results evaluate the cod- included in modern CPUs and GPUs. HTJ2K can increase the through- ing performance, throughput, and power consumption of the proposed put of a conventional JPEG2000 codec by about 10× at the expense method compared to other state-of-the-art codecs. of sacrificing: codestream compliance, compression efficiency (about The rest of the paper is organized as follows. Section 2 reviews the 10%) and, quality scalability. Codestream compliance and compression architecture of the GPU, JPEG2000 and HTJ2K. Section 3 overviews efficiency are inevitably affected when modifying the coding engine, the architecture of our codec, examines the aspects of BPC-PaCo that but these features are not essential in most scenarios. Quality scal- limit its throughput, and describes the implementation of CS BPC-PaCo. ability, on the other hand, is a valuable feature that allows partial Experimental results are presented in Section 4 comparing the proposed decoding of the codestream at different rates while minimizing the method with JPEG2000, HTJ2K, and HEVC. Section 5 concludes with distortion of the recovered image. See, for instance, in Fig. 1, the a brief summary. performance achieved by JPEG2000 and HTJ2K when the ‘‘Portrait’’ image (of corpus ISO/IEC 12640-1) is compressed and then transmitted 2. Background at 100 different rates distributed between 0.01 and 3 bits per sample (bps). The vertical axis of the figure reports the quality of the recovered 2.1. GPU architecture image in Peak Signal to Noise Ratio (PSNR), whereas the horizontal axis is the transmission rate. The quality achieved by HTJ2K is much lower Arguably, the most popular accelerators are currently those man- than that achieved by the original JPEG2000 due to the lack of quality ufactured by Nvidia due to their low price, high performance, and scalability. capacity for general-purpose computing through the CUDA program- This paper continues our line of research focused on providing ming language, so they are employed in this work. Nvidia GPUs are fine-grained parallelism to all coding stages of an image/video codec. constituted by many individual computing units called Streaming Mul- Our work originates in coding techniques that break the causality tiprocessors (SM). Each SM is responsible of managing the execution of multiple 32-wide vector instructions in parallel. A GPU can have of classical coding strategies [19–21]. These techniques led to the from one to a hundred of SMs. CUDA virtualizes each lane of a 32- development of a lightweight arithmetic coder that allows fine-grained wide vector instruction into a software thread. The group of 32 threads parallelism [22,23]. After that, the research focused on the stages of is referred to as a warp. Warps are organized in thread blocks, which a JPEG2000-like codec. First, we proposed a GPU implementation of are assigned to a SM for execution. Each thread block within a SM the DWT [14,24] employing a highly-efficient register-based strategy. can execute tasks independently from the others, so different kernels Second, the BPC engine was reformulated, resulting in a BPC with (i.e., CUDA functions) from the same or different applications can run parallel coefficient processing (BPC-PaCo) [25,26] that can efficiently concurrently on the same SM. To organize the execution of kernels, exploit the resources of a GPU [15]. Third, we presented the GPU CUDA provides the so-called streams. Each stream executes one or architecture for the end-to-end codec [16]. This codec can code up to various kernels of an application in a pre-determined order. Since the 12K video in real time, achieves a compression efficiency comparable to GPU has abundant computational resources, concurrent streams of the that of the original JPEG2000 standard, and does not sacrifice quality same application can be executed in parallel to process different data. scalability. See in Fig. 1 that the coding performance achieved by this Until CUDA v6.2, every thread in a warp executed instructions codec is approximately only 2% inferior to that of JPEG2000. in a synchronous, lock-step fashion with the other threads of the Our last step proposes a complexity scalable technique for the warp. Implicit synchronization was featured at the end of every di- coding engine. Complexity scalability allows trading computational vergence in the execution flow. Since the release of CUDA v7.0, every complexity by compression efficiency so that the user can tune the thread in a warp can be executed asynchronously, so synchronization codec to run more or less rapidly while marginally increasing the size among threads must be explicitly programmed when needed. Our of the compressed file. As it was studied in [27] and seen in Fig. 1, the codec considers this aspect, producing the same result regardless of the proposed Complexity Scalable BPC-PaCo (CS BPC-PaCo) decreases only architecture employed. For simplicity, the following sections assume slightly the coding performance with respect to BPC-PaCo. This paper that implicit synchronization is employed. extends that work by first analyzing the computational bottleneck of As Fig. 2 depicts, the memory architecture of a GPU has three levels: the original BPC-PaCo in the GPU, which guides the development of global memory, shared memory, and registers. The global memory the complexity scalable technique. Second, the proposed technique is is located in the device RAM or DRAM, has a size in the order of introduced in our end-to-end codec evaluating its memory footprint, oc- GBs, and is accessible by all SMs. This memory has high latency but cupancy and performance, as well as the overall throughput achieved in relatively high bandwidth, so data transfers are to be carried out in a 2 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 coalesced way (i.e., using consecutive memory positions) to maximize codeblock that can be truncated and re-organized in different layers of performance. The shared memory has a size in the order of MBs, has quality in the final codestream by the CR stage, the last of the coding low latency and higher bandwidth, and can be accessed by all threads pipeline. of a block. Each SM has an individual memory bank for this memory. The coding of codeblocks by independent threads provides the The registers have very fast access, very high bandwidth, and a size of coarse-grained parallelism that suits CPUs, but GPUs need a finer par- typically 256 KB. When the registers cannot hold all the data required allelism. The BPC stage has many data dependencies. The most crucial by the application, some data are temporarily moved to a reserved is imposed by the arithmetic coder, which requires the result of the last space in the device memory, the so-called local memory. This is called processed bit to start coding the next. The contextual information and register spilling. It significantly affects the application’s performance the group of coefficients selected in each coding pass also depend on the because transfers from/to the device memory render threads in an idle previously coded data, though these dependencies might be avoided at state due to the memory latency. the expense of more computations. These aspects prevent parallelism The memory architecture of the GPU is devised so that each execu- at a coefficient level, which is the kind of fine-grained parallelism that tion kernel transfers the data required for computation from the global GPUs may exploit more efficiently. memory to the registers and then transfers back the results to the global Part 15 of JPEG2000 (ISO/IEC 15444-15) provides more opportu- memory. Communication among threads is commonly carried out via nities for fine-grained parallelism [18]. The logical partition in code- the shared memory or register shuffling. Each GPU has a Level 1 (L1) blocks is maintained but, instead of using a bitplane coding strategy, and Level 2 (L2) cache to minimize the latency when moving data the coefficients are coded with a single coding pass in sets of 4 × 4 co- from/to the device memory to/from the shared memory and registers. efficients called quads. Most of the operations to code each quad do not The L1 cache is located in the memory bank within the SM that also hold critical dependencies with other quads. Entropy coding is carried holds the shared memory, whereas the L2 cache is in a separate memory out via variable-to-variable length codes, allowing parallel processing bank between the SMs and the device memory. of quads. This coding strategy allows the use of vector instructions in modern CPUs and GPUs. Nonetheless, the use of a single coding pass 2.2. JPEG2000 architecture disables quality scalability because the bitstream of each codeblock cannot be truncated and re-organized as in the original JPEG2000. As As previously stated, the JPEG2000 coding pipeline has three main seen in Fig. 1, this significantly reduces the quality of a compressed stages. The first reduces the spatial redundancy of the image through image transmitted at progressive rates. Also, the compression efficiency the DWT. The input to the DWT is either a gray image or a color image is penalized due to the use of a less efficient entropy coder than that of that has been converted to a color space that holds the luminance in the the original JPEG2000. first component and the blue and red chrominance in the second and third component, respectively. The color transform (CT) is a pixel-wise 3. Proposed method operation without dependencies, so it is easily mapped to SIMD-based instructions. The DWT applies a series of arithmetic operations to 3.1. Codec architecture all rows and columns of the image employing the so-called lifting scheme [28]. There are no dependencies between rows/columns, so The method proposed in this work extends our previous GPU-based these operations can be performed in parallel, suiting well SIMD pro- architecture for the end-to-end codec [16] by introducing complexity gramming too. The resulting wavelet coefficients are then reordered in scalability. The goal is to accelerate the coding process at the expense four different subbands of one quarter the size of the original image. of decreasing compression efficiency in a way that can be controlled by One of these subbands holds the low-pass details of the image, whereas the user. the other three hold the high-pass details. In general, the DWT is First, let us briefly describe the architecture of our codec. Fig. 3 applied 5 times on the low-pass subband to further compact the image depicts the employed kernels. The architecture is devised so that each energy. JPEG2000 provides reversible and irreversible operations for kernel performs all operations to a chunk of data before it needs to be both the CT and DWT operations. The reversible path employs integer re-organized for the following operations. This minimizes the transfers operations, so the original image can be recovered losslessly. The from/to the global memory since the data are fetched and returned to irreversible path employs floating-point operations that provide higher this memory only once in each kernel. More precisely, the CT kernel compression efficiency but produce losses in the reconstructed image. processes data tiles containing three color components from an image These losses can be controlled via the dead-zone quantization that is region, the DWT kernel processes data tiles containing samples of a applied just after the DWT. single component, the BPC-PaCo kernel codes codeblocks, and the CR The second stage of the coding pipeline is the BPC. It is applied kernel re-organizes the bitstreams produced for each codeblock in the independently on data tiles that typically contain a set of 64 × 64 final codestream. This organization allows each kernel to compute wavelet coefficients, called codeblocks. The coefficients within each many small data tiles in parallel, maximizing the overall throughput. In codeblock are processed in a bitplane-by-bitplane fashion. A bitplane addition, the codec leverages the computational resources of the GPU is the collection of bits 𝑏𝑗 from all coefficients at binary position 𝑗, through asynchronous I/O and multi-stream processing, and favors with [𝑏𝑀−1 , 𝑏𝑀−2 , … , 𝑏1 , 𝑏0 ], 𝑏𝑖 ∈ {0, 1} representing the binary repre- the use of register-based operations to communicate among threads in sentation of integer 𝜐 produced by the DWT (and quantization when detriment of shared memory to avoid the latency of this memory. using the irreversible path). The first non-zero bit of each coefficient, Fig. 3 depicts below each kernel its computational load. BPC-PaCo denoted by 𝑏𝑠 , is referred to as significant bit. The sign of 𝜐 is denoted approximately spends 85% of the total execution time, so its optimiza- by 𝑑 ∈ {+, −} and is coded immediately after 𝑏𝑠 so that the decoder can tion may significantly increase the overall throughput. The remaining start approximating 𝜐 as soon as possible. JPEG2000 codes all bits of kernels represent less than 15% of the total load and their operations each bitplane in three coding passes. Each pass scans all the coefficients are indispensable and already highly optimized. As it is formulated within the codeblock but only codes the bits of a group of selected in [16], the BPC-PaCo kernel uses two coding passes per bitplane. coefficients. This three-pass strategy codes first the information that The significance pass codes the bits of those coefficients that were not mostly reduces the distortion of the image. Each bit, with contextual significant in previous bitplanes, more precisely, those with 𝑠 ≤ 𝑗, information about the coefficient’s neighbors, is fed to an arithmetic with 𝑗 representing the current bitplane. The refinement pass codes coder. The arithmetic coder employs this contextual information to the bits of the remaining coefficients. The scanning order is devised adaptively adjust the probabilities of the processed bits, which is key so that each thread of a warp visits two columns of coefficients from to achieve compression. The output of the BPC stage is a bitstream per the top to the bottom row. For significance coding, the context of 3 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 Fig. 3. Illustration of the codec architecture when employing a single stream of execution in the GPU. Table 1 Evaluation of memory and cache transfers when the kernels BPC-PaCo and CS BPC-PaCo code a 4K image in a RTX 2080 Ti GPU. Data reading (MB) Data writing (MB) Cache hit rates 𝐷 →  𝐿2 → 𝐿1 𝐿1 → 𝐿2  → 𝐷 L1 L2 BPC-PaCo 251 283 129 108 69% 39% K = 0.5 234 265 119 97 70% 39% K = 1 182 212 106 78 72% 43% CS BPC-PaCo K = 2 118 145 96 58 76% 51% K = 6 91 108 94 57 79% 55% the current coefficient 𝜐 is determined via the significance state of its transfers between caches, which are similar, and the cache hit rates, ∑ eight adjacent neighbors as 𝜙𝑠𝑖𝑔 (𝜐, 𝑗) = 𝑘 𝛷(𝜐𝑘 , 𝑗), with 𝜐𝑘 , 1 < 𝑘 ≤ 8 which are moderately high because the data employed by the threads denoting the neighbors and 𝛷(𝜐 , 𝑗) = 1 or 0 when 𝜐𝑘 is significant or 𝑘 are frequently accessed and easily foreseeable. Despite the acceleration not, respectively. The context employed to code sign 𝑑 is denoted by that the caches may provide, register spilling severely handicaps this 𝜙𝑠𝑖𝑔𝑛 (𝜐, 𝑗) and employs a similar strategy, whereas the refinement pass kernel. uses a single context since little gain is achieved with more complex models [21], so 𝜙𝑟𝑒𝑓 (𝜐, 𝑗) = 0. The probability estimate that the encoded 3.2. Complexity scalable BPC-PaCo bit 𝑏𝑗 is 0 is extracted from a lookup table (LUT) known by encoder and decoder that is accessed as 𝑢 [𝑗][𝜙𝑠𝑖𝑔 (⋅)], with 𝑢 denoting the wavelet The register spilling that occurs in BPC-PaCo is mainly caused by subband. This LUT is pre-computed offline with a training set of images the multiple scanning of the coefficients during the coding process. according to The average number of coded bitplanes is 8, resulting in 16 accesses 𝑗 −1 2∑ per coefficient. This generates multiple data transfers back and forth 𝐹𝑢 (𝜐 | 𝜙𝑠𝑖𝑔 (𝜐, 𝑗)) from the local memory since registers cannot hold all the coefficients 𝜐=0 𝑃𝑠𝑖𝑔 (𝑏𝑗 = 0 | 𝜙𝑠𝑖𝑔 (𝜐, 𝑗)) = , (1) simultaneously. 2𝑗+1 ∑−1 The only way to reduce register spilling is minimizing the number of 𝐹𝑢 (𝜐 | 𝜙𝑠𝑖𝑔 (𝜐, 𝑗)) times that the coefficients are visited. However, the two-pass strategy is 𝜐=0 necessary to provide accurate estimates that achieve compression, and where 𝐹𝑢 (𝜐 | 𝜙𝑠𝑖𝑔 (𝜐, 𝑗)) is the probability mass function of the quantized multiple truncation points that achieve quality scalability. The adopted coefficients at bitplane 𝑗 given their context. Its support is [0, … , 2𝑗+1 − strategy must alleviate the impact on these coding features, regulating 1] since it contains coefficients that were not significant in bitplanes the coding passes performed in each codeblock but without affecting greater than 𝑗. Probabilities for sign and refinement coding are de- the most relevant information in terms of distortion. rived similarly. Their respective LUTs are denoted by 𝑢′ and 𝑢′′ . The technique employed herein was presented in [27] from a the- Entropy coding of the emitted bits and their probabilities are carried oretical perspective that evaluates the impact on coding performance out by each thread with an arithmetic coder that produces fixed-length and quality scalability, but without implementing it in our end-to- codewords [29]. Threads cooperate among them to dispatch these end GPU codec. Its main insight is to code bitplanes [𝑀 − 1, 𝑁] with codewords to the bitstream in a quality embedded order. the same two-pass strategy of BPC-PaCo, and then use a fast mode In BPC-PaCo, coefficients 𝜐 and ancillary data to process them are that codes bitplanes [𝑁 − 1, 0] in a single pass. This codes the most stored in the registers. Typically, each thread processes 128 coefficients relevant information in terms of distortion (contained in the highest (belonging to 2 columns of 64 coefficients), ideally requiring 128 bitplanes [𝑀 − 1, 𝑁]) more progressively than the lesser relevant infor- registers of 32 bits plus some more for ancillary data. In current GPUs, this is too much information to hold in the register space. Each SM in mation, minimizing the impact on compression efficiency and quality current GPUs (Turing architecture) has 256 KB for registers and can scalability. run a maximum of 1024 threads. If all threads run in parallel, they can Choosing a suitable 𝑁 is key to balance throughput and compression only access a maximum of 64 registers without causing register spilling efficiency. A high 𝑁 causes the coding of many bitplanes in fast mode, and rendering some threads in an idle state. increasing the throughput but penalizing coding performance. A low As illustrated in the first row of Table 1, register spilling is the 𝑁 does not affect coding performance significantly though it does main bottleneck of the BPC-PaCo kernel. The table reports the transfers not provide significant throughput gains either. Instead of using the between memory device 𝐷 and registers  that occur when the same 𝑁 for all codeblocks, the strategy proposed in [30] uses different kernel processes a 4K image (gray scale, 8 bps). The component’s data 𝑁s depending on the codeblock’s wavelet subband 𝑢 and magnitude approximately requires 32 MB but, as seen in the table, 251 MB are read bitplanes 𝑀 according to from 𝐷 due to the extensive use of local memory, as much as 8×. This ( ⌊ ⌋) 𝐾 increase is approximately 4× for writing. Table 1 also reports the data 𝑁 = min 𝑀, 𝑀 ⋅ . (2) 𝑢 4 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 Algorithm 1 CS BPC-PaCo coefficient (including the possible data expansion that the lossy DWT Parameters: 𝑢 subband, 𝑡 stripe, 𝑀 total magnitude bitplanes, 𝑁 bitplanes may produce), so ancillary data are stored in the remaining bits. Fig. 5 coded in fast mode illustrates the binary representation of a GPU register. The lowest 24 1: for 𝑗 ∈ [𝑀 − 1, 𝑁] do bits store the magnitude and sign of 𝜐, with the sign stored at the lowest 2: SignificancePass() bit. The highest 8 bits are employed for auxiliary information. The 3 3: RefinementPass() upper bits are used in the SignificancePass() and RefinementPass() of 4: end for Algorithm 1 to signal information regarding the significance state of the 5: for 𝑦 ∈ [0, numRows − 1] do coefficient. The remaining 5 bits are employed to store the significance 6: for 𝑥 ∈ [𝑡 ⋅ 2, 𝑡 ⋅ 2 + 1] do 7: 𝑐 ← 𝜙𝑠𝑖𝑔 (𝜐𝑦,𝑥 , 𝑁 − 1) bitplane of the coefficient (i.e., 𝑠), which is employed in the fast mode 8: for 𝑗 ∈ [𝑁 − 1, 0] do to accelerate the operations that compute the context (i.e., in 𝜙𝑠𝑖𝑔 (⋅)). 9: if 𝛷(𝜐𝑦,𝑥 , 𝑗 + 1) = 0 then In the example of Fig. 5, 𝑀 = 8 and 𝑁 = 4. The bitplanes depicted in 10: ACencode(𝑏𝑗 , 𝑢 [𝑗][𝑐], 𝑡) blue represent those that are coded with two coding passes, whereas 11: if 𝑏𝑗 = 1 then the bitplanes depicted in red are coded with the fast mode. 12: ACencode(𝑑, 𝑢′ [𝑗][𝜙𝑠𝑖𝑔𝑛 (𝜐𝑦,𝑥 , 𝑁 − 1)], 𝑡) This bit-allocation strategy in the registers minimizes the amount of 13: end if local memory employed during execution time. The previous analysis 14: else of memory transfers for BPC-PaCo depicted in Table 1 also reports 15: ACencode(𝑏𝑗 , 𝑢′′ [𝑗][0], 𝑡) the results obtained for CS BPC-PaCo when different 𝐾s are used. 16: end if Data reading from 𝐷 to  is proportionally reduced to the value of 17: end for 18: end for 𝐾. High 𝐾s employ more extensively the fast mode of CS BPC-PaCo, 19: end for reducing memory transfers. When 𝐾 = 6, the proposed method only requires 36% of the memory transfers employed by the original BPC- PaCo. Memory transfers from 𝐿2 to 𝐿1 are reduced similarly. Data writing is reduced slightly less, though for 𝐾 = 6 the proposed method 𝑢 is the L2 norm of the synthesis basis vectors of the subband filter- approximately halves the transfers of BPC-PaCo. Since fewer data are bank (which is computed offline assuming equal energy gain factor in employed by the algorithm, the cache hit rates for both 𝐿1 and 𝐿2 are all subbands). Large 𝐾s result in large 𝑁s, so more bitplanes are coded increased, which provides even faster access to the data. with the fast mode, increasing the codec’s throughput. Note that 𝐾 is Table 2 illustrates the impact in the throughput achieved by the the user parameter that controls the speedup or, in other words, the CS BPC-PaCo kernel when using different 𝐾s as a result of reducing mechanism through which complexity scalability is managed. memory transfers. This evaluation employs the same 4K image of the The coding technique embodied in Eq. (2) sets lower 𝑁s to code- test reported in Table 1. The second column reports the average number blocks within subbands in smaller resolution levels. Although these of clock cycles that each executed instruction is blocked due to the subbands have fewer codeblocks than in larger levels, these codeblocks latency of the local memory, and the average cycles needed to execute have higher entropy than the rest, so coding them with more coding each instruction (CPI). These metrics clearly illustrate the beneficial passes significantly enhances the quality scalability of the system. This effect of using less local memory. BPC-PaCo blocks almost 10 cycles is illustrated in Fig. 4. It depicts the images recovered when coding per instruction, requiring 15 cycles to execute each instruction. CS BPC- the ‘‘Portrait’’ image with (a) the same 𝑁 in all codeblocks, and (b) the PaCo reduces the number of cycles in which instructions are blocked proposed strategy. Both codecs are set to achieve the same throughput.1 proportionally to the use of the fast mode. For 𝐾 = 6, instructions are The strategy that uses a fixed 𝑁 (Fig. 4(a)) significantly degrades the blocked only 1.56 cycles, whereas instructions only require 8 cycles image quality because the bitstream of some codeblocks within the to complete, on average. This improvement is also noted in the in- lowest resolution levels are not included in the final codestream. The structions executed per cycle (IPC) reported in the third column, which proposed strategy (Fig. 4(b)) provides an image with much higher is almost doubled as compared to BPC-PaCo. The memory bandwidth quality. This holds for other coding parameters and images. (fourth column) employed by the kernel indicates that less bandwidth is needed as fewer coding passes are performed. The warp efficiency 3.3. Implementation and GPU occupancy (fifth and sixth columns) is almost the same for all kernels since the algorithms have similar divergence (i.e., conditional Algorithm 1 details the proposed kernel from a thread perspective. paths in the execution flow). The total number of executed instructions The bitplanes in the range [𝑀 − 1, 𝑁] are coded with the original (seventh column) is slightly higher in CS BPC-PaCo due to more instruc- BPC-PaCo (lines 2 and 3) as described in [15,16]. A significance and tions are needed when switching to the fast mode. Despite executing refinement pass are employed in each bitplane. The fast mode is used more instructions, the execution time of CS BPC-PaCo is reduced for from bitplane 𝑁 − 1 to 0. Instead of visiting the coefficients twice per all 𝐾s because of the fewer memory transfers, with a reduction of 31% bitplane, the fast mode visits them only once and codes all their bits. when 𝐾 = 6. Lines 5 and 6 in Algorithm 1 scan the coefficients. Since the context Nvidia allows developers to manually set the number of registers for significance coding is the same from bitplane 𝑁 − 1 onward, it is assigned to the threads of a kernel. Evidently, to use too few registers only computed once in line 7. Our implementation avoids this operation per thread requires more local memory, while too many may cause an when all the coefficients are already significant. The loop in line 8 underuse of the GPU. Since the throughput achieved by our method codes all bits of the coefficient considering its significance state. The highly depends on the local memory employed, the register assignment arithmetic coder employs the procedure described in [15], which is not is carefully studied to yield maximum performance. Table 3 provides detailed herein for simplicity. an evaluation of the throughput achieved when a different number of As previously stated, key to achieve high throughput is to reduce registers per thread is assigned to the proposed kernel. Different 𝐾s are the number of registers that each thread employs. To this end, the employed to consider different running conditions. The test is carried 32-bit registers of the GPU hold all the information needed by the out for the same conditions as in previous evaluations, though results algorithm. In general, 24 bits are enough to hold the value of the hold for other images and parameters. The third and fourth columns of the table depict the theoretical maximum and real GPU occupancy 1 Coding parameters for this test are: lossy compression, 2 DWT levels, achieved, respectively. The maximum occupancy is calculated as the 64 × 64 codeblocks, target rate 0.25 bps, 𝑁 = 15, and 𝐾 = 6 for the fixed and number of threads that can be run in parallel using the assigned number variable strategy, respectively. of registers. It is only from a theoretical point of view since, in practice, 5 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 Fig. 4. Visual evaluation of a (a) fixed and (b) variable strategy to set the bitplanes coded in fast mode with CS BPC-PaCo. Fig. 5. Illustration of a codeblock and the bit-allocation strategy in the 32-bit registers of the GPU employed by CS BPC-PaCo. Table 2 Evaluation of throughput metrics when the kernels BPC-PaCo and CS BPC-PaCo code a 4K image in a RTX 2080 Ti GPU. #cycles per inst. IPC Bandwidth Warp Occupancy #inst. Time (ms) blocked (total CPI) (GB/s) efficiency (×106 ) BPC-PaCo 9.58 (15.29) 65 200 53% 46% 153 2.17 K = 0.5 6.49 (12.95) 77 156 56% 46% 159 1.93 K = 1 3.73 (10.33) 94 107 55% 46% 157 1.81 CS BPC-PaCo K = 2 1.86 (8.32) 114 74 53% 45% 153 1.61 K = 6 1.56 (8.00) 122 73 53% 46% 152 1.39 threads are commonly blocked due to register spilling and other as- running in parallel do not have enough resources. In all tests reported pects. As seen in Table 3, even though 64 registers per thread achieves a in this work, the CS BPC-PaCo kernel uses 72 registers per thread. theoretical maximum occupancy of 100%, the real occupancy achieved The speedup that the proposed kernel achieves with respect to the is about 63%. To assign 72 registers per thread decreases the maximum original BPC-PaCo is evaluated in Fig. 6. The figure reports in the vertical axis the speedup achieved for the 𝐾s depicted in the horizontal occupancy to 87,5%, though in practice is about 62% too. Also, to use axis, for both lossy and lossless compression when using a video se- 72 registers instead of 64 improves the throughput achieved since less quence (see below). The results indicate that our method yields higher register spilling occurs, decreasing the execution time of the kernel speedups for lossless compression, reaching a speedup of 70% for and of the end-to-end codec (5th and 6th columns in the table). To the highest 𝐾 evaluated. Lossy compression achieves more moderate use more registers per thread slightly improves the performance of the speedups, approximately up to 30%. This is because the floating-point CS BPC-PaCo kernel because less register spilling occurs, though for the DWT produces wavelet coefficients with higher magnitudes, and so overall end-to-end codec this is not beneficial because the other kernels more bitplanes are coded. We remark that the increase in throughput is 6 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 Table 3 Evaluation of occupancy and execution time achieved by CS BPC-PaCo with different 𝐾s when assigning a different number of registers to the threads, for a 4K image in a RTX 2080 Ti GPU. Registers per thread Occupancy Time (in ms) Maximum Real Kernel Total Total av. K = 0.5 48% 1.86 11.77 K = 1 49% 1.76 10.6 96 62.5% 10.17 K = 2 48% 1.6 9.65 K = 6 48% 1.4 8.65 K = 0.5 56% 1.88 11.43 K = 1 54% 1.81 10.53 80 75% 10.06 K = 2 55% 1.6 9.65 K = 6 56% 1.38 8.65 K = 0.5 62% 1.93 11.5 K = 1 61% 1.81 10.21 72 87.5% 9.94 K = 2 62% 1.61 9.47 K = 6 64% 1.39 8.59 K = 0.5 63% 1.96 11.62 K = 1 61% 1.86 10.73 64 100% 10.4 K = 2 64% 1.67 9.78 K = 6 66% 1.44 9.47 Fig. 6. Evaluation of the throughput gain achieved by CS BPC-PaCo with respect to BPC-PaCo for different 𝐾s. Table 4 microarchitecture called Turing (CUDA capability v7.5) and runs in a Evaluation of lossless compression performance for JPEG2000, HTJ2K, HEVC, workstation with an Intel i9-9900K CPU with 16 GB of RAM. The latter BPC-PaCo, and CS BPC-PaCo (with different 𝐾s). Results are reported in bps. uses the previous microarchitecture Pascal (CUDA capability v6.0) and CS BPC-PaCo BPC-PaCo JPEG2000 HTJ2K HEVC runs in a workstation with an Intel i7-3770 CPU with 8 GB of RAM. Re- 𝐾 = 0.25 3.82 sults for JPEG2000 and HTJ2K are obtained with Kakadu (v8.0.3) [31], 𝐾 = 0.5 3.83 which is among the fastest CPU implementations for JPEG2000 opti- 𝐾 = 0.75 3.89 𝐾 =1 3.91 3.82 3.79 4.06 4.03 mized with assembly and vector instructions. It runs in the i9-9900K 𝐾 = 1.5 4.00 workstation with 16 execution threads, yielding higher throughput than 𝐾 =2 4.01 implementations of JPEG2000 for GPUs such as CuJ2K [32] and GPU- 𝐾 =6 4.05 J2K [33]. Although HTJ2K can also be optimized for GPUs [34], to the best of our knowledge, there is no implementation that allows testing in the environment employed herein. Results for HEVC are obtained with the Nvidia implementation of the standard [35] running higher than the increase in rate in all cases. When 𝐾 = 6, for instance, in both GPUs, which use a hardwired and specialized chip in the CS BPC-PaCo achieves a speedup of approximately 65% (23%) while device. Coding parameters for our method and JPEG2000 are: lossy the rate increase is 14% (8%) for lossless (lossy) compression. or lossless compression as indicated, 5 DWT levels, and codeblocks of 64 × 64. For HTJ2K, parameter ‘‘Cplex={6,EST,0.25,0}’’ is also 4. Experimental results employed to allow the codec to attain the specified target rate. HEVC uses a rate control method with constant quantization (1-51) for lossy The proposed CS BPC-PaCo is compared with BPC-PaCo and state- compression, GOP=32, and high performance mode, which achieves of-the-art codecs widely employed in the field, more precisely, the maximum throughput in our tests. Throughput and power consumption standard JPEG2000 and its new part HTJ2K, and the standard HEVC. results use a 2-minute segment of the ‘‘Star Wars: The Last Jedi’’ movie All results below report the coding performance or throughput achieved at 4K that has 2,880 color frames, resulting in 67.8 GB of uncompressed by the end-to-end codec instead of only focusing on the CS BPC-PaCo data. Coding performance results use the color image ‘‘Portrait’’ (with kernel as in the previous section. Our method is run in two commodity a size of 2560 × 2048) and a segment of the previous video sequence GPUs from Nvidia, namely, the RTX 2080 Ti (68 SMs with 4352 cores at containing 948 gray-scale frames at 2K. 1.6 GHz with 11 GB of RAM) and the GTX 1080 Ti (28 SMs with 3585 The first test evaluates lossy coding performance. Fig. 7 extends the cores at 1.9 GHz with 11 GB of RAM). The former GPU uses the Nvidia results of Fig. 1 by including different 𝐾s for the proposed method 7 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 Fig. 7. Evaluation of rate–distortion performance for JPEG2000, HTJ2K, BPC-PaCo and CS BPC-PaCo (with different 𝐾s) when transmitting an image at 100 different rates (left) and a video sequence at 30 different rates (right). and the results for video. We recall that this test depicts rate vs. The previous test evaluates throughput for very high video quality. quality when the codestream is compressed and then transmitted at Some scenarios may allow lower video quality due to transmission or different rates. For both tests, the performance achieved by CS BPC- visualization aspects. The next test evaluates the throughput achieved PaCo decreases as more coding passes are coded in fast mode (i.e., with when different quality levels are employed. Fig. 9 depicts in the hor- higher values of 𝐾). The results indicate that the quality scalability of izontal axis the quality of the recovered video, which is set from 50 the proposed method is significantly better than that of HTJ2K since to 38 dB in all codecs. Lower quality yields similar results to those even when 𝐾 = 6 and most passes are coded in a single pass, the drop in obtained for 38 dB. Again, results are reported in MS/s for the encoder quality is approximately 5 dB with respect to JPEG2000 and BPC-PaCo, and decoder. Blue and purple plots respectively correspond to BPC- as compared to the losses of about 15 dB of HTJ2K. PaCo and CS BPC-PaCo with the same 𝐾s as those employed before. The second test evaluates lossless compression. Table 4 reports As expected, the lower the quality, the higher the throughput since the rate achieved when coding the video with all methods evaluated. fewer data are coded. The highest gains are achieved by the encoder BPC-PaCo yields almost same performance to that of JPEG2000, while of the proposed method. At 38 dB, all codecs except the proposed CS BPC-PaCo penalizes it slightly more with increments in rate of about achieve similar throughput when encoding, which is about 3 to 4× 7% when 𝐾 = 6. This increment is lower than that of HTJ2K, which lower than that of CS BPC-PaCo. The decoder presents more variations, almost obtains the same performance to that of CS BPC-PaCo when with HEVC gaining much throughput for low qualities. It is worth 𝐾 = 6. noting that HTJ2K yields similar results regardless of the quality, The third test evaluates throughput for both lossy and lossless video obtaining the same throughput to that of JPEG2000 when encoding or compression. In this test, the quality of the recovered video for lossy decoding at 38 dB. These results suggest that the proposed CS BPC-PaCo compression yields 50 dB in all codecs. Fig. 8 shows the results for achieves the highest throughput gains when using high quality, while all codecs and GPUs (or CPU for JPEG2000 and HTJ2K), reported low qualities render the throughput of the codec to almost the same as in mega samples coded per second (MS/s). Two bars are depicted that of BPC-PaCo. for each codec. The left bar corresponds to the encoder whereas the The last experimental test is aimed at energy consumption. The right to the decoder. BPC-PaCo is depicted with wide blue bars. The power demand of codecs running in the GPUs (CPUs) is obtained with proposed CS BPC-PaCo is depicted with three thinner purple bars the nvidia-smi (PowerTOP) tool, which provides the real consumption within that of BPC-PaCo, corresponding to the throughput achieved of the microprocessor depending on the workload. Fig. 10 reports when 𝐾 = {0.75, 2, 6}, with the thinnest bar for the highest 𝐾. The the results in MS coded per Joule consumed when coding video at figure also shows with horizontal lines the throughput needed to code 50 dB. The figure illustrates the results in the same form as that of 4K, 8K, 12K, and 16K video at 24 frames per second in real time. Fig. 8. The proposed method reduces energy consumption with respect As seen in the figure, the proposed method significantly increases the to BPC-PaCo, with less consumption for higher 𝐾s. Even so, these throughput with respect to BPC-PaCo, mostly in the encoder. In the improvements are not as high as those obtained with the throughput. decoder the gains are not as significant because the decoding process This is seen as the larger bars corresponding to CS BPC-PaCo in Fig. 8 in our implementation needs more ancillary data, which hinders the vs. those depicted in Fig. 10. These results indicate that more energy overall throughput achieved. HTJ2K yields high throughput too, being has to be spent per coded sample to increment the codec’s throughput. slightly superior to that of our method for the GTX 1080 Ti in the case Even so, the results achieved with the RTX 2080 Ti suggest that our of lossless compression, though being 50% inferior (or more, depending method consumes less energy than the other codecs evaluated. HTJ2K on the 𝐾) when CS BPC-PaCo runs in the RTX 2080 Ti. The throughput also consumes little energy compared to JPEG2000, which is the most achieved by JPEG2000 is much lower than that of HTJ2K due to the demanding. The hardwired chip of HEVC in the GPU yields good results lack of opportunities for fine-grained parallelism in the algorithm. The as well, except for decoding with the GTX 1080 Ti, which consumes throughput achieved by HEVC is modest as compared to the other energy similarly to the decoder of JPEG2000. codecs despite using a hardwired chip in the GPU. This is due to the techniques employed in this coding system, which achieve high 5. Conclusions coding performance at the expense of higher computational complexity. Finally, we remark that the scalability by complexity introduced in High-throughput and low-power consumption image and video BPC-PaCo allows our codec to encode 16K (12K) lossy video in real codecs are a current necessity for new applications, cameras, and time with the RTX 2080 Ti (GTX 1080 Ti) when 𝐾 = 2, obtaining a displays to allow real-time processing of very high resolution video and good tradeoff between coding performance and throughput. to extend the battery life of power-constrained devices. International 8 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 Fig. 8. Throughput evaluation of lossy and lossless video compression for all codecs and GPUs/CPUs. Fig. 9. Throughput evaluation for lossy compression of video at different quality levels. Results are for BPC-PaCo and CS BPC-PaCo except when indicated. is that it needs to transfer too much data from the local memory of the GPU to the registers (and vice versa) due to the coding of the image samples in many successive passes. The complexity scalable technique employed herein is tailored to increase the codec throughput in GPUs by reducing the coding passes performed. The proposed technique allows a user-handled control of the speedup achieved while minimiz- ing losses in coding performance and quality scalability. Experimental results suggest that our codec attains higher throughput than other state-of-the-art codecs without sacrificing any feature of the coding system. Under some coding conditions, our method achieves real-time 16K coding of color video in a consumer-grade GPU (considering also memory transfers from host-to-device and vice versa), which is well above the current needs of most practical scenarios. Declaration of competing interest Fig. 10. Evaluation of energy consumption for lossy video compression for all codecs and GPUs/CPUs. The authors declare that they have no known competing finan- cial interests or personal relationships that could have appeared to organizations and researchers are proposing novel techniques, systems, influence the work reported in this paper. and standards to fulfill these requirements. Some works pursue this goal by exploiting the high-performance computing of massively parallel Acknowledgments architectures such as those found in Graphics Processing Units (GPUs). This is the line of research followed in this paper, which began by adapting and implementing all stages of a JPEG2000-based coding This work has been partially supported by the Spanish Ministry of pipeline to the fine-grained parallelism that suits GPUs. The bottleneck Economy and Competitiveness and the European Regional Develop- of the resulting codec is the bitplane and arithmetic coding stage, which ment Fund under Grants TIN2017-84553-C2-1-R and RTI2018-095287- spends most of the execution time. This work has analyzed this bottle- B-I00 (MINECO/FEDER, UE), and by the Catalan Government, Spain neck by carefully profiling its execution on a GPU. Its main drawback under Grants 2017SGR-463 and 2017SGR-313. 9 C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta et al. Signal Processing: Image Communication 99 (2021) 116503 References [17] M.S. Nobile, P. Cazzaniga, A. Tangherloni, D. Besozzi, Graphics processing units in bioinformatics, computational biology and systems biology, Brief. Bioinform. [1] ISO/IEC, Information Technology - JPEG 2000 Image Coding System - Part 1: 18 (5) (2017) 870–885. Core Coding System, Dec. 2000. [18] D. Taubman, A. Naman, R. Mathew, High throughput block coding in the [2] International Telecommunication Union, High Efficiency Video Coding Standard, HTJ2K compression standard, in: Proc. IEEE International Conference on Image 2013. Processing, 2019, pp. 1079–1083. [3] I. Marzuki, J. Ma, Y.-J. Ahn, D. Sim, A context-adaptive fast intra coding [19] F. Auli-Llinas, Local average-based model of probabilities for JPEG2000 bitplane algorithm of high-efficiency video coding (HEVC), J. Real-Time Image Process. coder, in: Proc. IEEE Data Compression Conference, 2010, pp. 59–68. 16 (2019) 883–899. [20] F. Auli-Llinas, I. Blanes, J. Bartrina-Rapesta, J. Serra-Sagrista, Stationary model [4] G. Correa, P. Assuncao, L. Agostini, L.A. da Silva Cruz, Complexity scalability of probabilities for symbols emitted by bitplane image coders, in: Proc. IEEE for real-time HEVC encoders, J. Real-Time Image Process. 12 (2016) 107–122. International Conference on Image Processing, 2010, pp. 497–500. [5] Y. Wu, P. Liu, Y. Gao, K. Jia, Medical ultrasound video coding with H.265/HEVC [21] F. Auli-Llinas, Stationary probability model for bitplane image coding through based on ROI extraction, PLoS One (Nov.) (2016). local average of wavelet coefficients, IEEE Trans. Image Process. 20 (8) (2011) [6] K.H. Yanzhe Li, Luc. Claesen, M. Zhao, A real-time high-quality complete system 2153–2165. for depth image-based rendering on FPGA, IEEE Trans. Circuits Syst. Video [22] F. Auli-Llinas, Highly efficient, low complexity arithmetic coder for JPEG2000, Technol. 29 (4) (2019) 1179–1193. in: Proc. IEEE International Conference on Image Processing, 2014, pp. [7] J.W. P, et al., A low-cost and high-throughput FPGA implementation of the 5601–5605. retinex algorithm for real-time video enhancement, IEEE Trans. Very Large Scale [23] F. Auli-Llinas, Entropy-based evaluation of context models for wavelet- Int. Syst. 28 (1) (2020) 101–114. transformed images, IEEE Trans. Image Process. 24 (1) (2015) 57–67. [8] X.W. Xin Guo, Y. Liu, An FPGA implementation of multi-channel video processing [24] P. Enfedaque, F. Auli-Llinas, J.C. Moure, Strategies of SIMD computing for image and 4k real-time display system, in: International Congress on Image and Signal coding in GPU, in: Proc. IEEE International Conference on High Performance Processing, BioMed. Eng. Inform. (2018) 1–6. Computing, 2015, pp. 345–354. [9] Z. He, H. Huang, M. Jiang, Y. Bai, G. Luo, FPGA-Based real-time super-resolution [25] F. Auli-Llinas, P. Enfedaque, J.C. Moure, I. Blanes, V. Sanchez, Strategy of mi- system for ultra high definition videos, in: Annual International Symposium on croscopic parallelism for bitplane image coding, in: Proc. IEEE Data Compression Field-Programmable Custom Computing Machines (FCCM), 2018, pp. 181–188. Conference, 2015, pp. 163–172. [10] A. Descampe, F.-O. Devaux, G. Rouvroy, J.-D. Legat, J.-J. Quisquater, B. Macq, [26] F. Auli-Llinas, P. Enfedaque, J.C. Moure, V. Sanchez, Bitplane image coding A flexible hardware JPEG 2000 decoder for digital cinema, IEEE Trans. Circuits with parallel coefficient processing, IEEE Trans. Image Process. 25 (1) (2016) Syst. Video Technol. 16 (11) (2006) 1397–1410. 209–219. [11] I. Chiuchisan, A new FPGA-based real-time configurable system for medical [27] C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta, F. Auli-Llinas, Complexity image processing, in: E-Health and Bioengineering Conference (EHB), 2013, pp. scalable bitplane image coding with parallel coefficient processing, IEEE Signal 1–4. Process. Lett. 27 (2020) 840–844. [12] V. Kasik, Z. Chvostkova, FPGA in technical resources of medical imaging, in: IEEE [28] W. Sweldens, The lifting scheme: A construction of second generation wavelets, 11th International Symposium on Applied Machine Intelligence and Informatics SIAM J. Math. Anal. 29 (2) (1998) 511–546. (SAMI), 2013, pp. 193–196. [29] F. Auli-Llinas, Context-adaptive binary arithmetic coding with fixed-length [13] Junying. Chen, Shunfeng. Zhou, Huaqing. Min, Implementation of parallel codewords, IEEE Trans. Multimedia 17 (8) (2015) 1385–1390. medical ultrasound imaging algorithm on CAPI-enabled FPGA, in: International [30] F. Auli-Llinas, J. Serra-Sagrista, JPEG2000 quality scalability without quality Conference on Field-Programmable Technology (FPT), 2016, pp. 311–314. layers, IEEE Trans. Circuits Syst. Video Technol. 18 (7) (2008) 923–936. [14] P. Enfedaque, F. Auli-Llinas, J.C. Moure, Implementation of the DWT in a GPU [31] D. Taubman, Kakadu software, http://www.kakadusoftware.com, Jul. 2020. through a register-based strategy, IEEE Trans. Parallel Distrib. Syst. 26 (12) [32] University of Stuttgart, CuJ2K, http://cuj2k.sourceforge.net/, Jul. 2020. (2015) 3394–3406. [33] Poznan Supercomputing, Networking Center, GPUJ2K, http://apps.man.poznan. [15] P. Enfedaque, F. Auli-Llinas, J.C. Moure, GPU implementation of bitplane coding pl/trac/jpeg2k/wiki, Feb. 2020. with parallel coefficient processing for high performance image compression, [34] A. Naman, D. Taubman, Decoding high-throughput JPEG2000 (HTJ2K) on a IEEE Trans. Parallel Distrib. Syst. 28 (8) (2017) 2272–2284. GPU, in: Proc. IEEE International Conference on Image Processing, 2019, pp. [16] C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta, F. Auli-Llinas, GPU- 1084–1088. Oriented architecture for an end-to-end image/video codec based on JPEG2000, [35] Nvidia, HEVC SDK, https://developer.nvidia.com/nvidia-video-codec-sdk, Dec. IEEE Access 8 (1) (2020) 68474–68487. 2018. 10