J. Vis. Commun. Image R. 89 (2022) 103672 Contents lists available at ScienceDirect J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci Full Length Article Accelerating BPC-PaCo through Visually Lossless Techniques✩ Francesc Aulí-Llinàs ∗, Carlos de Cea-Dominguez, Miguel Hernández-Cabronero Department of Information and Communications Engineering, Universitat Autònoma de Barcelona, Bellaterra, 08193, Spain ARTICLE INFO ABSTRACT Keywords: Fast image codecs are a current need in applications that deal with large amounts of images. Graphics High-throughput image coding Processing Units (GPUs) are suitable processors to speed up most kinds of algorithms, especially when Visually lossless coding they allow fine-grain parallelism. Bitplane Coding with Parallel Coefficient processing (BPC-PaCo) is a JPEG2000 recently proposed algorithm for the core stage of wavelet-based image codecs tailored for the highly parallel architectures of GPUs. This algorithm provides complexity scalability to allow faster execution at the expense of coding efficiency. Its main drawback is that the speedup and loss in image quality is controlled only roughly, resulting in visible distortion at low and medium rates. This paper addresses this issue by integrating techniques of visually lossless coding into BPC-PaCo. The resulting method minimizes the visual distortion introduced in the compressed file, obtaining higher-quality images to a human observer. Experimental results also indicate 12% speedups with respect to BPC-PaCo. 1. Introduction without modifying their inner algorithms. This maintains compliance with the original systems, but limits the parallel strategies that can be Imagery employed in myriad scenarios is nowadays experiencing employed. The main handicap of conventional coding engines is that a qualitative and quantitative step forward. Images with very high the core of the coding system commonly uses a single-thread procedure spatial and temporal resolution and with high dynamic range are to code a whole chunk of data, so only coarse-grain parallelism can currently commonplace. This quality increment is accompanied with a be applied over a multi-tile division of the image. GPUs are mostly proliferation of image-based applications such as teleconferencing [1], based on the single-instruction multiple data architecture. To fully geographic information systems [2], and online TV [3], among oth- use their resources, the core algorithms must be tailored for fine- ers [4–6]. The fast increase in both quality and quantity requires faster grain parallelism [30], so that vector instructions can be efficiently and more efficient codecs to accelerate the coding and transmission, employed. and to reduce the storage costs of these images. The Joint Photographic Experts Group realized this facet and in The development of faster codecs has followed different paths in 2019 a new part to the JPEG2000 [31] standard called High Through- the field of image coding. A common strategy is to reduce the com- put JPEG2000 (HTJ2K) was introduced [32]. The new part uses a core putational complexity of the algorithms. Popular coding systems such algorithm that supports vector instructions sets included in modern as SPIHT [7] or EBCOT [8], which are employed in a wide vari- CPUs and GPUs [33–35], so it can be employed for fast TV production, ety of environments such as the medical, the remote sensing, or the digital cinema, and other scenarios that require real-time processing. video edition, have been re-formulated many times with the aim to The main drawback of HTJ2K is that it sacrifices quality scalabil- reduce their complexity and/or memory requirements [9–14]. Another ity, which is a useful feature to transmit the image progressively by strategy is to implement the codec in hardware architectures such as quality [36]. field-programmable gate arrays [15–17] or application-specific inte- Well before the standardization of HTJ2K, we started a research grated circuits [18–21], so that these codecs can be employed in mobile line whose goal is to introduce fine-grain parallelism to all stages of a devices such as (video)cameras. More recently, the advent of multi- core Central Processing Units (CPUs) and highly parallel architectures JPEG2000-like codec while keeping all features of the original system, such as Graphics Processing Units (GPUs) has stimulated the implemen- including quality scalability. The first proposed techniques extricated tation of codecs with multi-thread capabilities for CPUs [22–25] and the causality of single-threaded coding strategies [37], introduced a GPUs [26–29]. These works focus on the acceleration of the codecs lightweight arithmetic coder [38], and a fast GPU implementation of ✩ This paper has been recommended for acceptance by Zicheng Liu. ∗ Corresponding author. E-mail address: francesc.auli@uab.cat (F. Aulí-Llinàs). https://doi.org/10.1016/j.jvcir.2022.103672 Received 20 June 2022; Received in revised form 22 September 2022; Accepted 15 October 2022 Available online 25 October 2022 1047-3203/© 2022 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by- nc-nd/4.0/). F. Aulí-Llinàs et al. Journal of Visual Communication and Image Representation 89 (2022) 103672 the discrete wavelet transform [26]. The bitplane coder is the second strategies code all bits from 𝑀 −1, the highest bitplane of the codeblock, and most complex stage of the system. It was re-formulated and tailored to bitplane 0. The collection of bits in each bitplane is coded in three for fine grain-parallelism, resulting in a Bitplane Coding engine with coding passes. The first is called Significance Propagation Pass (SPP) Parallel Coefficient processing (BPC-PaCo) [27,39]. The end-to-end and codes bits from coefficients that were not significant in previous GPU implementation of our codec was presented in [29] achieving 12K bitplanes but that have significant adjacent neighbors. A coefficient resolution video compression in real time with only 2% penalization becomes significant at bitplane 𝑠 when 𝑏𝑠 = 1 with 𝑏𝑠′ = 0, 𝑠′ > 𝑠. in coding efficiency. Our last step introduced a modification to BPC- The second coding pass is referred to as Magnitude Refinement Pass PaCo to trade coding efficiency for computational complexity in an (MRP) and codes the bits of coefficients that where significant in adjustable fashion, resulting in a Complexity Scalable BPC-PaCo engine previous bitplanes, more precisely, those bits 𝑏𝑟 , 𝑟 < 𝑠. The last is the (CS BPC-PaCo) [36,40]. Cleanup Pass (CP) and codes the bits of the remaining coefficients. If a The trading between throughput and image quality in CS BPC- coefficient becomes significant in the SPP or CP, its sign 𝑑 ∈ {+, −} is PaCo is regulated via a user parameter. Rate–distortion optimization coded immediately after so that it can be partially reconstructed at the techniques minimize the impact in coding performance when more decoder. The bitstream can be truncated at the end of each coding pass, throughput is required, but the parameter neither regulates the amount if necessary, by the rate–distortion optimization method to minimize of sacrificed quality nor the increase in throughput. This may not suit the distortion of the image when it is progressively transmitted. scenarios in which visual quality is essential. Visually lossless tech- As seen in Fig. 1, the main difference between JPEG2000 and niques are interesting in such scenarios since they preserve the level of CS BPC-PaCo is that JPEG2000 uses a single thread (per codeblock) that quality at which the image distortion becomes indistinguishable for the scans the coefficients one by one, sending 𝑏𝑖 to an arithmetic coder that human visual system. These techniques set a visually lossless threshold produces an embedded bitstream. This bitstream can only be decoded that estimates the maximum absolute error between the original and bit by bit through the inverse procedure, without opportunities for par- the compressed image, resulting in just imperceptible distortion for a allelization. Contrarily, CS BPC-PaCo employs a parallel scanning order human observer [41]. In the framework of JPEG2000, visually lossless in which half the coefficients in a row (typically, 32 for codeblocks of coding has been proposed in [42–45] employing different distortion 64 × 64) are coded in parallel. The bits of these coefficients are coded models. with multiple arithmetic coders that are synchronized at some points to The work proposed herein joins the complexity scalability tech- produce the final bitstream. This is the main insight to obtain fine-grain niques employed in CS BPC-PaCo with a visually lossless model tailored parallelism. for our engine. Briefly described, the coding process uses the original Regardless of using one or multiple threads of execution, the coeffi- algorithm of BPC-PaCo until the visually lossless threshold is attained. cients are scanned many times. This slows the coding process because, This maximizes the quality when the image is progressively transmit- in most architectures, not all coefficients can be simultaneously stored ted. Then, the codec is maximally accelerated at the expense of coding in the register space of the processor and they are transferred back efficiency via a fast mode. This penalizes the efficiency in compression and forth from the memory, causing the so-called register spilling [36]. only in data segments that are not visually relevant. Since the employed To accelerate the coding engine, register spilling must be minimized. model determines the visually lossless threshold autonomously, the The complexity scalable technique employed in CS BPC-PaCo selects codec does not require user supervision and maximizes the throughput a bitplane 𝑁 from which the bits of all coefficients are coded in a and quality of the image that is visually relevant for a human observer. single pass. So, bitplanes [𝑀 −1, 𝑁] are coded with three coding passes, The rest of the paper is organized as follows. Section 2 reviews the providing truncation points that can be used by the rate–distortion original CS BPC-PaCo. Section 3 describes the distortion model and optimization method to optimize the image quality, and bitplanes [𝑁 − details its integration in the coding engine. The proposed method is 1, 0] are coded in a single pass to avoid register spilling, speeding up implemented in a Java framework. Coding performance and compu- the coding process. The selection of 𝑁 is key to balance throughput and tational complexity are compared with BPC-PaCo and CS BPC-PaCo compression efficiency. CS BPC-PaCo determines it as in Section 4. Experimental results show that the proposed method ( ⌊ ⌋) 𝐾 significantly reduces the impact on image quality when the image is 𝑁 = min 𝑀, 𝑀 ⋅ , (1) 𝑢 transmitted progressively while increasing the coding speed by about 8%, on average. The last section provides conclusions. with 𝑢 being the L2 norm of the synthesis basis vectors of the filter- bank of subband 𝑢 (equal energy gain factor is assumed in all sub- 2. Review of complexity scalable BPC-PaCo bands). 𝐾 is the user-controlled parameter. Experimental evidence suggests that this parameter should be in the range 𝐾 ∈ [0.5, 1.5] to The main stages of the JPEG2000 coding pipeline are [46]: wavelet obtain a compromise between throughput and quality. transform, bitplane coding, and codestream re-organization. The dis- crete wavelet transform decorrelates the image information producing 3. Proposed method a multi-resolution decomposition of the image organized in wavelet subbands that contain the vertical, horizontal, and diagonal details 3.1. Distortion model of the image. Each subband is partitioned in small sets of typically 64 × 64 wavelet coefficients called codeblocks. Each codeblock is For the user, parameter 𝐾 is only indicative. A given value of 𝐾 may coded independently by the bitplane coding stage, which produces a sacrifice quality and throughput differently for different images. A more quality embedded bitstream that can be truncated at different rates. suitable strategy is to reduce the compression efficiency only when The last stage (possibly truncates and) re-organizes the bitstreams of all the visual quality of the image is indistinguishable from the original, codeblocks in a single or multiple layers of quality satisfying a target i.e., when the visually lossless threshold is attained. The most accurate rate, commonly employing rate–distortion optimization techniques. distortion model for the deadzone quantizer employed by JPEG2000 is Bitplane coding is the most sophisticated stage of the image codec, described in [43]. Such a quantizer reconstructs the wavelet coefficient entailing approximately 80% of the total coding time [27]. Let 𝜔 denote as { a wavelet coefficient and 𝜐 an integer with the magnitude of the quan- 0 if 𝑗 > 𝑠 tization index obtained for such coefficient. The binary representation 𝑤̂ = , (2) 𝑑([𝑏𝑀−1 , 𝑏𝑀−2 , … , 𝑏𝑗 ] + 𝛿) ⋅ 𝛥𝑢 2𝑗 otherwise of 𝜐 is expressed as [𝑏𝑀−1 , 𝑏𝑀−2 , … , 𝑏1 , 𝑏0 ], 𝑏𝑖 ∈ {0, 1}, with an 𝑀 sufficiently large to hold all coefficients within the codeblock. The same where 𝛿 ∈ [0, 1) is commonly set to 0.5 to reconstruct the coeffi- bit position 𝑏𝑗 of all coefficients is defined as bitplane 𝑗. Bitplane coding cient in the middle of the quantization interval. 𝛥𝑢 is the step size 2 F. Aulí-Llinàs et al. Journal of Visual Communication and Image Representation 89 (2022) 103672 Fig. 1. Scanning order employed in: (a) JPEG2000 standard, and (b) CS BPC-PaCo. employed for subband 𝑢. The distortion model of [43] considers that Algorithm 1 VL BPC-PaCo coefficients within the deadzone (i.e., those reconstructed as 0 above), Parameters: 𝑀 total magnitude bitplanes, 𝜎 2 codeblock’s variance, 𝑡 thread have the same distribution as the original coefficients, and the remain- 1: 𝑗 ← 𝑀 − 1 ing coefficients have a uniform quantization distortion. This model is 2: repeat employed to estimate visually lossless thresholds for an assumed coef- 3: SignificancePropagationPass(𝑗) ficient variance 𝜎 2 and quantization step size 𝛥𝑢 , denoted as VT𝑢 (𝜎 2 ). 4: MagnitudeRefinementPass(𝑗) 5: CleanupPass(𝑗) These thresholds are based on the well-known work of Watson and 6: 𝐷′ = 𝛥𝑖 2𝑗 Daly [47,48]. 7: 𝑗 ←𝑗−1 In the encoder of [43], the above perceptual model is introduced 8: until 𝐷′ ≤ VT𝑢 (𝜎 2 ) as follows. At the end of each coding pass the maximum absolute 9: for 𝑦 ∈ [0, numRows − 1] do error between the original coefficients and those reconstructed by the 10: for 𝑥 ∈ [𝑡 ⋅ 2, 𝑡 ⋅ 2 + 1] do decoder is computed as 𝐷 = max(|𝑤 − 𝑤|). ̂ When 𝐷 ≤ VT𝑢 (𝜎 2 ), the 11: for 𝑗 ′ ∈ [𝑗, 0] do remaining passes are discarded since they do not contain visually 12: if 𝑗 ′ ≥ 𝑠 then relevant information. VT𝑢 (𝜎 2 ) is determined at the beginning of the 13: ACEncodeSignificance(𝑏𝑗 ′ , 𝑡) 14: if 𝑗 ′ = 𝑠 then coding process computing the coefficients’ variance. Other techniques 15: ACEncodeSign(𝑑, 𝑡) (such as visual masking) are also employed in [43] to better refine 16: end if this threshold, though they are not employed in our codec due to their 17: else computational costs. 18: ACEncodeRefinement(𝑏𝑗 ′ , 𝑡) 19: end if 3.2. Visually lossless BPC-PaCo 20: end for 21: end for 22: end for The goal of [43] is to reduce the codestream size by discarding the lesser visually relevant information. Contrarily, the method presented herein employs the distortion model to accelerate the codec only when coding such information. To this end, three coding passes per bitplane bitplanes 𝑀. The second is to employ the distortion upper bound 𝐷′ are generated before reaching the threshold, and then one more for instead of 𝐷. 𝐷′ is determined in bitplane 𝑗 as the visually non-relevant information. The aim of this strategy is to } provide many coding passes at visually relevant rates, optimizing the 𝛥𝑖 2𝑗 if pass = CP if ∃ 𝑤̂ = 0 𝛥𝑖 2𝑗+1 otherwise quality scalability of the codestream when the visual information is 𝐷′ = 𝑗 } . (3) most relevant. To this end, instead of computing the bitplane at which 𝛥𝑖 2 if pass = SPP 𝑗−1 otherwise the codec is accelerated via Eq. (1) and parameter 𝐾, it is determined 𝛥𝑖 2 otherwise autonomously during the coding process when VT𝑢 (𝜎 2 ) is attained. In our engine, 𝜎 2 does need to be estimated since it can be com- Unfortunately, the method proposed in [43] is not suitable for our puted when the data are produced just after the wavelet transform, engine. The main difficulty is that the encoding process requires a negligibly increasing computational costs. Bounding the distortion via version of the reconstructed coefficients (i.e., 𝑤) ̂ to compute 𝐷. This 𝐷′ is helpful and thus applied to our codec because avoids the need may significantly increase the memory needed per coefficient, resulting in more register spilling. An alternative is to use the model employed for computing the reconstructed coefficients, maintaining the same in [44], which is devised for the decoder. It modifies [43] in two memory requirements of the original CS BPC-PaCo. aspects. The first is to estimate the variance 𝜎 2 for the codeblock Algorithm 1 details the main operations of the proposed Visually depending on the wavelet subband 𝑢 and the number of magnitude Lossless BPC-PaCo (VL BPC-PaCo). The algorithm is expressed as the 3 F. Aulí-Llinàs et al. Journal of Visual Communication and Image Representation 89 (2022) 103672 Fig. 2. Evaluation of coding performance for the ‘‘Cafeteria’’ image when the codestream is transmitted progressively with a (a) lossy and (b) lossless regime. execution of one of the threads employed to code the data of a code- Table 1 Evaluation of lossless coding performance. Results are reported in bps. The lowest block. 𝑡 denotes the thread and serves to compute the columns that are results among CS BPC-PaCo and VL BPC-PaCo are reported in bold. coded. The first loop (lines 2 to 8) codes the data in the conventional JP2 BPC-PaCo mode of BPC-PaCo, i.e., using three coding passes per bitplane.1 At the Original VL CS CS CS end of each bitplane (i.e., just after the CP) 𝐷′ is computed to check 𝐾 = 0.5 𝐾 = 0.75 𝐾 = 1 whether the visually lossless threshold has been attained or not. If so, Portrait 3.80 4.00 4.02 4.00 4.04 4.08 all remaining bitplanes of each coefficient are coded at once (lines Cafeteria 4.68 4.80 4.81 4.82 4.92 4.99 9 to 22). Functions ACEncode {Significance| Sign| Refinement} code Fruit 3.96 4.15 4.19 4.16 4.20 4.22 the corresponding bit using the arithmetic coder for that thread. More Wine 3.94 4.12 4.13 4.13 4.16 4.19 details on the context formation and processing of the bitstream can be Bicycle 3.90 4.09 4.10 4.10 4.16 4.20 found in [39,40]. Orchid 3.44 3.68 3.73 3.69 3.72 3.75 Musicians 5.34 5.52 5.54 5.58 5.77 5.86 It is worth noting that Eq. (3) is not strictly applied in Algorithm 1. Candle 4.74 4.87 4.89 4.91 5.03 5.10 To do so, 𝐷′ should be computed after the SPP only when there is none average 4.22 4.42 4.44 4.44 4.50 4.55 reconstructed coefficient (i.e., 𝑤) ̂ equal to 0 at bitplane 𝑗. Experimental tests indicate that this is an extremely rare case, so it is deliberately omitted in our algorithm to simplify and accelerate the implementation. CS BPC-PaCo penalizes more the coding performance for larger values 4. Experimental results of 𝐾. This penalization is approximately uniform from low to high rates. Contrarily, at low rates, VL BPC-PaCo achieves almost the same The 8 color images of the ISO 12640-1 corpus are employed in the performance as that of the original BPC-PaCo. At medium and high following experiments. They are 2560 × 2048 and have a bit-depth of rates in lossy regimes, the quality of the transmitted image starts 8 bits per sample (bps). Images of different sizes and from different decreasing since most codeblocks attain the visually lossless threshold. corpora produce similar results as those reported in the following tests. In lossless regimes, the proposed method penalizes almost nothing the The results report the performance achieved by JPEG2000, the original image quality as compared to the original BPC-PaCo. Regardless of the BPC-PaCo, CS BPC-PaCo (with 𝐾 ∈ {0.5, 0.75, 1}), and the proposed compression regime, the penalization in quality produced by VL BPC- VL BPC-PaCo. The values of 𝐾 employed for CS BPC-PaCo are chosen PaCo occurs at a higher rate than when using CS BPC-PaCo. This to achieve high throughput (when 𝐾 = 1), minimize the impact on indicates that VL BPC-PaCo preserves better the quality at low rates, coding performance (when 𝐾 = 0.5), and a compromise in between. when it is more visually relevant. Similar results hold for the remaining All methods are implemented in the same Java framework [49] so images of the corpus. differences in throughput and coding efficiency are only related to the Table 1 reports the coding performance achieved with a lossless methods evaluated. To assess the throughput, the codec is run with a regime for all images. The proposed method penalizes very little the single thread, with the execution time providing an approximation of coding performance, with an increase of only 0.02 bps on average computational complexity. The reversible CDF 5/3 and irreversible CDF with respect to the original BPC-PaCo. This increase is the same as 9/7 filter banks are respectively used for lossless and lossy compression that achieved with CS BPC-PaCo when 𝐾 = 0.5. Nonetheless, the applying 5 levels of decomposition. performance achieved by VL BPC-PaCo when the image is progressively Figs. 2 and 3 report the results achieved for the ‘‘Cafeteria’’ and transmitted is significantly higher than that achieved by CS BPC-PaCo ‘‘Musicians’’ images when they are transmitted progressively from 0.01 with the same 𝐾, as seen in Figs. 2 and 3. bps to approximately 4 bps, at 50 equally spaced rates. The vertical axis The next test evaluates the throughput achieved by the proposed reports the Peak Signal to Noise Ratio (PSNR), whereas the horizontal method. Results are reported as the percentage of throughput increase axis reports the bit rate of the compressed image. As seen in the figures, in the bitplane coding engine achieved by the indicated method with JPEG2000 and the original BPC-PaCo achieve similar performance. respect to the original BPC-PaCo. Fig. 4 depicts average results for all images when coding in lossy and lossless regimes. As seen in the figure, VL BPC-PaCo obtains a throughput increase between that achieved for 1 Our previous work also describes a variation of this coding scheme that 𝐾 = 0.5 and 𝐾 = 0.75. For lossy compression, the proposed codec employs two coding passes per bitplane (significance and refinement). The accelerates both the encoding and decoding process by approximately proposed method can be directly applied when using two coding passes too, 8.5%, whereas for lossless compression the speedup is approximately so it is omitted herein for simplicity. 6%. Table 2 details the results for all images of the corpus. For some 4 F. Aulí-Llinàs et al. Journal of Visual Communication and Image Representation 89 (2022) 103672 Fig. 3. Evaluation of coding performance for the ‘‘Musicians’’ image when the codestream is transmitted progressively with a (a) lossy and (b) lossless regime. Fig. 4. Evaluation of the throughput increase achieved by VL BPC-PaCo (VL) and CS BPC-PaCo (CS) with respect to the original BPC-PaCo, on average for all images of the corpus when using (a) lossy and (b) lossless compression. The left column in each method reports the results for the encoder, whereas the right column for the decoder. Table 2 Evaluation of the throughput increase (in percentage) achieved by VL BPC-PaCo and CS BPC-PaCo with respect to the original BPC-PaCo. Results are reported for the encoder (left) and decoder (right) in each cell. lossy lossless VL CS BPC-PaCo VL CS BPC-PaCo BPC-PaCo 𝐾 = 0.5 𝐾 = 0.75 𝐾 = 1 BPC-PaCo 𝐾 = 0.5 𝐾 = 0.75 𝐾 = 1 Portrait 10.8 11.2 0.6 3.4 10.6 11.1 15.5 18.0 10.1 9.5 0.7 0.2 16.2 15.3 23.3 23.2 Cafeteria 9.3 8.2 6.7 5.1 17.2 17.7 26.0 26.1 3.8 5.9 4.2 4.2 21.1 20.5 29.4 28.5 Fruit 12.9 10.8 2.4 2.5 13.6 12.4 18.6 17.8 5.9 4.2 0.1 2.1 12.5 13.0 21.7 21.3 Wine 8.0 9.5 2.5 2.1 11.2 13.6 20.2 19.2 7.8 7.1 0.2 0.4 16.1 17.6 23.7 23.4 Bicycle 8.5 10.0 2.1 3.8 13.0 15.1 21.4 20.4 6.1 10.4 1.6 3.8 19.6 18.9 26.6 27.4 Orchid 12.4 13.3 1.3 3.9 9.8 11.1 15.4 18.9 8.2 11.7 −1.7 2.4 12.6 16.5 18.1 21.1 Musicians 4.5 3.7 5.2 4.2 17.7 17.0 23.7 25.3 −1.0 0.3 2.0 1.4 18.8 19.8 27.3 30.3 Candle 4.2 5.3 3.7 3.8 14.9 16.9 23.4 24.9 5.2 7.1 6.1 5.5 21.1 22.6 30.3 32.1 Average 8.4 8.6 3.3 3.7 14.0 14.8 21.1 21.8 5.4 6.8 1.9 2.1 17.6 18.3 25.5 26.4 images, the proposed method accelerates the codec by more than CPUs and GPUs. Despite achieving high throughput, the bottleneck of 12%. In general, images with fewer details reach the visually lossless BPC-PaCo originates in the use of three coding passes per bitplane, threshold after coding fewer bitplanes, so the achieved speedup is which causes register spilling since the coefficients are transferred back higher. and forth from the memory to the register space of the processor. This shortcoming is alleviated with the introduction of a complexity 5. Conclusions scalable technique that safeguards quality scalability while accelerating the coding process. The drawback of CS BPC-PaCo is that the quality BPC-PaCo is a bitplane coding engine for wavelet-based coding sys- and speedup can only be roughly controlled via a user parameter. This tems tailored for fine-grain parallelism in highly parallel architectures. paper introduces a visually lossless model for BPC-PaCo that, without Its main insight is to code multiple coefficients within a codeblock needing the user supervision, maximizes both the throughput and the in parallel through vector instructions such as those found in modern quality of the image that is visually relevant for a human observer. 5 F. Aulí-Llinàs et al. Journal of Visual Communication and Image Representation 89 (2022) 103672 VL BPC-PaCo is implemented for this paper in a Java framework [16] Y. Li, L. Claesen, K. Huang, M. Zhao, A real-time high-quality complete system for to compare its coding performance and computational complexity with depth image-based rendering on FPGA, IEEE Trans. Circuits Syst. Video Technol. BPC-PaCo and CS BPC-PaCo. Experimental tests suggest that the pro- 29 (2019) 1179–1193. posed method significantly smooths the impact on quality when the [17] J.W. Park, H. Lee, B. Kim, D.-G. Kang, S.O. Jin, H. Kim, H.-J. Lee, A low-cost and high-throughput FPGA implementation of the retinex algorithm for real-time image is progressively transmitted while accelerating the codec by video enhancement, IEEE Trans. VLSI Syst. 28 (2020) 101–114. about 8% on average. Future work will evaluate this algorithm on [18] A.K. Gupta, S. Nooshabadi, D. Taubman, M. Dyer, Realizing low-cost high- modern GPUs. throughput general-purpose block encoder for JPEG2000, IEEE Trans. Circuits Syst. Video Technol. 16 (2006) 843–858. Declaration of competing interest [19] K. Mei, N. Zheng, C. Huang, Y. Liu, Q. Zeng, VLSI design of a high-speed and area-efficient JPEG2000 encoder, IEEE Trans. Circuits Syst. Video Technol. 17 The authors declare that they have no known competing finan- (2007) 1065–1078. cial interests or personal relationships that could have appeared to [20] M. Dyer, S. Nooshabadi, D. Taubman, Design and analysis of system on a chip influence the work reported in this paper. encoder for JPEG2000, IEEE Trans. Circuits Syst. Video Technol. 19 (2009) 215–225. Data availability [21] S. Kim, D. Lee, J.-S. Kim, H.-J. Lee, A high-throughput hardware design of a one-dimensional SPIHT algorithm, IEEE Trans. Multimedia 18 (2016) 392–404. [22] H.-C. Fang, Y.-W. Chang, T.-C. Wang, C.-J. Lian, L.-G. Chen, Parallel embedded Data are from public corpus ISO 12640-1. block coding architecture for JPEG 2000, IEEE Trans. Circuits Syst. Video Technol. 15 (2005) 1086–1097. Acknowledgments [23] Y. Li, M. Bayoumi, A three-level parallel high-speed low-power architecture for EBCOT of JPEG 2000, IEEE Trans. Circuits Syst. Video Technol. 16 (2006) This work has been partially supported by the Spanish Ministry 1153–1163. of Science, Innovation and Universities (MICIU) and by the European [24] K. Sarawadekar, S. Banerjee, An efficient pass-parallel architecture for embedded Regional Development Fund (FEDER) under Grants RTI2018-095287- block coder in JPEG 2000, IEEE Trans. Circuits Syst. Video Technol. 21 (2011) B-I00 and PID2021-125258OB-I00, by the Catalan Government under 825–836. Grants 2018-BP-00008 and 2017SGR-463, and by the Horizon 2020 [25] Y. Jin, H.-J. Lee, A block-based pass-parallel SPIHT algorithm, IEEE Trans. under the Marie Skłodowska-Curie grant agreement #801370. Circuits Syst. Video Technol. 22 (2012) 1064–1075. [26] P. Enfedaque, F. Auli-Llinas, J.C. Moure, Implementation of the DWT in a GPU through a register-based strategy, IEEE Trans. Parallel Distrib. Syst. 26 (2015) References 3394–3406. [27] P. Enfedaque, F. Auli-Llinas, J.C. Moure, GPU implementation of bitplane coding [1] H. Chang, M. Varvello, F. Hao, S. Mukherjee, A tale of three videoconferencing with parallel coefficient processing for high performance image compression, applications: Zoom, Webex, and Meet, IEEE/ACM Trans. Netw. (2022) in press. IEEE Trans. Parallel Distrib. Syst. 28 (2017) 2272–2284. [2] Y. Al-Mulla, A. Al-Ruheili, A. Al-Lawati, K. Parimi, A. Ali, N. Al-Sadi, F. Al- [28] M. Diaz, R. Guerra, P. Horstrand, E. Martel, S. Lopez, J. Lopez, R. Sarmiento, Harrasi, Assessment of urban expansion’s impact on changes in vegetation Real-time hyperspectral image compression onto embedded GPUs, IEEE J. Sel. patterns in Dhofar, Oman, using remote sensing and GIS techniques, IEEE Access Topics Appl. Earth Observ. Remote Sens. 12 (2019) 2792–2809. 10 (2022) 86782–86792. [29] C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta, F. Auli-Llinas, GPU- [3] S.-H. Lee, S.-H. Yoon, H.-W. Kim, Prediction of online video advertising inventory oriented architecture for an end-to-end image/video codec based on JPEG2000, based on TV programs: A deep learning approach, IEEE Access 9 (2021) IEEE Access 8 (2020) 68474–68487. 22516–22527. [4] C. Yan, B. Gong, Y. Wei, Y. Gao, Deep multi-view enhancement hashing for [30] M.S. Nobile, P. Cazzaniga, A. Tangherloni, D. Besozzi, Graphics processing units image retrieval, IEEE Trans. Pattern Anal. Mach. Intell. 43 (2021) 1445–1451. in bioinformatics, computational biology and systems biology, Brief. Bioinform. 18 (2017) 870–885. [5] C. Yan, Y. Hao, L. Li, J. Yin, A. Liu, Z. Mao, Z. Chen, X. Gao, Task-adaptive attention for image captioning, IEEE Trans. Circuits Syst. Video Technol. 32 [31] Information technology - JPEG 2000 image coding system - part 1: Core coding (2022a) 43–51. system, 2000. [6] C. Yan, L. Meng, L. Li, J. Zhang, Z. Wang, J. Yin, J. Zhang, Y. Sun, B. Zheng, [32] Information technology - JPEG 2000 image coding system - part 15: High Age-invariant face recognition by multi-feature fusion and decomposition with throughput JPEG 2000, 2019. self-attention, ACM Trans. Multimedia Comput. Commun. Appl. 18 (2022b) 1–18. [33] A. Naman, D. Taubman, Decoding high-throughput JPEG2000 (HTJ2K) on a [7] A. Said, W.A. Pearlman, A new, fast, and efficient image codec based on set GPU, in: Proc. IEEE International Conference on Image Processing, 2019, pp. partitioning in hierarchical trees, IEEE Trans. Circuits Syst. Video Technol. 6 1084–1088. (1996) 243–250. [34] D. Taubman, A. Naman, R. Mathew, High throughput block coding in the [8] D. Taubman, High performance scalable image compression with EBCOT, IEEE HTJ2K compression standard, in: Proc. IEEE International Conference on Image Trans. Image Process. 9 (2000) 1158–1170. Processing, 2019, pp. 1079–1083. [9] W.A. Pearlman, A. Islam, N. Nagaraj, A. Said, Efficient, low-complexity image [35] A. Naman, D. Taubman, Encoding high-throughput JPEG2000 (HTJ2K) images coding with a set-partitioning embedded block coder, IEEE Trans. Circuits Syst. on a GPU, in: Proc. IEEE International Conference on Image Processing, 2020, Video Technol. 14 (2004) 1219–1235. pp. 1171–1175. [10] G. Xie, H. Shen, Highly scalable, low-complexity image coding using zeroblocks [36] C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta, F. Auli-Llinas, Real-time of wavelet coefficients, IEEE Trans. Image Process. 15 (2005) 762–770. 16K video coding on a GPU with complexity scalable BPC-PaCo, ELSEVIER Signal [11] M. Dyer, D. Taubman, S. Nooshabadi, A.K. Gupta, Concurrency techniques Process.: Image Commun. 99 (2021) 1–10. for arithmetic coding in JPEG2000, IEEE Trans. Circuits Syst. I 53 (2006) [37] F. Auli-Llinas, Stationary probability model for bitplane image coding through 1203–1212. local average of wavelet coefficients, IEEE Trans. Image Process. 20 (2011) [12] M. Rhu, I.-C. Park, Optimization of arithmetic coding for JPEG2000, IEEE Trans. 2153–2165. Circuits Syst. Video Technol. 20 (2010) 446–451. [38] F. Auli-Llinas, Entropy-based evaluation of context models for wavelet- [13] F. Auli-Llinas, M.W. Marcellin, Scanning order strategies for bitplane image transformed images, IEEE Trans. Image Process. 24 (2015) 57–67. coding, IEEE Trans. Image Process. 21 (2012) 1920–1933. [39] F. Auli-Llinas, P. Enfedaque, J.C. Moure, V. Sanchez, Bitplane image coding with [14] X. Song, Q. Huang, S. Chang, J. He, H. Wang, Three-dimensional sepa- parallel coefficient processing, IEEE Trans. Image Process. 25 (2016) 209–219. rate descendant-based SPIHT algorithm for fast compression of high-resolution [40] C. de Cea-Dominguez, J.C. Moure, J. Bartrina-Rapesta, F. Auli-Llinas, Complexity medical image sequences, IET Image Process. 11 (2017) 80–87. scalable bitplane image coding with parallel coefficient processing, IEEE Signal [15] A. Descampe, F.-O. Devaux, G. Rouvroy, J.-D. Legat, J.-J. Quisquater, B. Macq, Process. Lett. 27 (2020) 840–844. A flexible hardware JPEG 2000 decoder for digital cinema, IEEE Trans. Circuits [41] A. Watson, G. Yang, J. Solomon, J. Villasenor, Visibility of wavelet quantization Syst. Video Technol. 16 (2006) 1397–1410. noise, IEEE Trans. Image Process. 6 (1997) 1164–1175. 6 F. Aulí-Llinàs et al. Journal of Visual Communication and Image Representation 89 (2022) 103672 [42] Z. Liu, L.J. Karam, A.B. Watson, JPEG2000 encoding with perceptual distortion [46] D.S. Taubman, M.W. Marcellin, JPEG2000 Image Compression Fundamentals, control, IEEE Trans. Image Process. 15 (2006) 1763–1778. Standards and Practice, Kluwer Academic Publishers, Norwell, Massachusetts [43] H. Oh, A. Bilgin, M. Marcellin, Visually lossless encoding for JPEG 2000, IEEE 02061 USA, 2002. Trans. Image Process. 22 (2013) 189–201. [47] A. Watson, Efficiency of a model human image code, J. Opt. Soc. Amer. 4 (1987) [44] L. Jimenez-Rodriguez, F. Auli-Llinas, M. Marcellin, Visually lossless strategies to 2401–2417. decode and transmit JPEG2000 imagery, IEEE Signal Process. Lett. 21 (2014) [48] S. Daly, The visible differences predictor: An algorithm for the assessment of 35–38. image fidelity, in: Proc. SPIE Human Vision, Visual Processing and DigitalDisplay, [45] H. Oh, A. Bilgin, M. Marcellin, Visually lossless JPEG 2000 for remote image 1992, pp. 2–15. browsing, Information 7 (2016) 1–20. [49] F. Auli-Llinas, BOI codec, 2022, URL: https://deic.uab.cat/~francesc/software/ boi. 7