Journal of Vision (2021) 21(6):10, 1–22 1 Matching visual induction effects on screens of different size Department of Information and Communication Technologies, Universitat Pompeu Fabra, Trevor Canham Barcelona, Spain Computer Vision Center and the Computer Sciences Department at Universitat Autònoma de Barcelona, Javier Vazquez-Corral Cerdanyola del Vallès, Spain Elise Mathieu Grenoble INP – Phelma, France Instituto de óptica, Spanish National Research Council Marcelo Bertalmío (CSIC), Spain This work is dedicated to the memory of Jihyun when the appearance of an image region moves away Yeonan-Kim. from that of its local neighborhood. See Figure 1 for some examples. In the film industry, the same movie is expected to be The groundbreaking experiments of Helson in watched on displays of vastly different sizes, from 1963 (Helson, 1963) aimed to quantify the perceptual cinema screens to mobile phones. But visual induction, phenomena first formally described by von Bezold the perceptual phenomenon by which the appearance (1874) and Gelb (1930), using matching experiments of a scene region is affected by its surroundings, will be with printed induction bar patterns and isolated different for the same image shown on two displays of Munsell patches. Specifically, observers had to judge different dimensions. This phenomenon presents a the appearance of grey bars over white or black practical challenge for the preservation of the artistic intentions of filmmakers, because it can lead to shifts in backgrounds. When the bars were very thin, the image appearance between viewing destinations. In this observers reported assimilation; as the bars increased in work, we show that a neural field model based on the width, the assimilation effect became less pronounced, efficient representation principle is able to predict and after some point the observers started to report induction effects and how, by regularizing its associated contrast, whose effect became increasingly more energy functional, the model is still able to represent pronounced as the width of the bars increased induction but is now invertible. From this finding, we (Figure 2). A similar result for the chromatic case was propose a method to preprocess an image in a reported by Fach and Sharpe (1986), who modulated screen–size dependent way so that its perception, in the spatial frequency of patterns as opposed to the terms of visual induction, may remain constant across target background proportionality variation of Helson. displays of different size. The potential of the method is Their conclusion was that, for higher spatial frequencies demonstrated through psychophysical experiments on visual induction takes the form of assimilation, synthetic images and qualitative examples on natural whereas for lower spatial frequencies it takes the form images. of contrast. Although not all confirm these early observations, there exists a large body of later work (e.g., Brenner, Ruiz, Herraiz, Cornelissen, & Smeets, 2003; Brown & MacLeod, 1997; Harrar & Vienot, Introduction 2005; Monnier & Shevell, 2003; Shevell & Wei, 1998; Shevell & Monnier, 2005; Wesner & Shevell, 1992) In visual perception, induction designates the effect corroborating the importance of the spatial distribution by which the lightness and chroma of a stimulus are and variability of inducing surrounds. Regarding affected by its surroundings. Visual induction can take visual induction models, we single out the work of two forms: assimilation, when the perception of an Otazu, Parraga, and Vanrell (2010), which is based on object shifts toward that of its surround, or contrast, wavelet decompositions, and the very recent work of Citation: Canham, T., Vazquez-Corral, J., Mathieu, E., & Bertalmío, M. (2021). Matching visual induction effects on screens of different size. Journal of Vision, 21(6):10, 1–22, https://doi.org/10.1167/jov.21.6.10. https://doi.org/10.1167/jov.21.6.10 Received July 27, 2020; published June 18, 2021 ISSN 1534-7362 Copyright 2021, The Authors This work is licensed under a Creative Commons Attribution 4.0 International License. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 2 Figure 1. Induction examples. Left: lightness contrast; the center gray squares have the same luminance value but the one surrounded by white is perceived darker and the one surrounded by black is perceived lighter. Middle: lightness assimilation; all gray bars have the same luminance value but the gray bars surrounded by black are perceived as being darker than the ones surrounded by white, which are seen as being lighter. Right: chromatic induction; the central and inducing rings on both sides have the same RGB tristimulus values, but all rings are perceived differently due to their rearrangement. when distributing to different display scenarios. In this process, a skilled artist works to ensure that the visual storytelling intentions of the piece in its original format are preserved. For instance, a piece that had an original theatrical release may be remastered for separate releases to home video, broadcast television, streaming, and so on. However, new standards and developments in the display industry of the past decade (high dynamic range, wide color gamut, 4K and 8K displays, mobile devices) have increased the variability in potential content destinations to the point where manual processing is no longer a feasible solution for handling distribution masters. For this reason, it is an increasingly relevant effort for the motion picture industry to develop solutions to adjust content automatically considering the specific viewing scenario parameters of the viewer. With these notions in mind, in this work we make three main contributions. First, we show that a neural field model based on the efficient Figure 2. Induction type depends on spatial frequency. Low representation principle is able to predict induction spatial frequencies induce contrast, while high spatial effects, and this model is validated using existing frequences induce assimilation. Reprinted with permission psychophysical data. Second, we prove that, by from Helson (1963). © The Optical Society. regularizing its associated energy functional, the model becomes invertible. Finally, based on this invertible formulation we propose a method to preprocess an image in a screen–size dependent way so that Song, Faugeras, and Veltz (2019), that uses a neural its perception, in terms of visual induction, may field model. remain constant across displays of different size. The In the film industry, the same movie is expected potential of the method is demonstrated through to be watched on displays of vastly different sizes, novel psychophysical experiments on synthetic images from cinema screens to mobile phones. But the typical and a validation experiment with natural images; viewing angle depends on screen size, being larger all these data are made available as Supplementary for larger displays; therefore, the same image content Material. will have a higher spatial frequency when seen on a small screen than when seen on a larger one. As a consequence, the visual induction effects on both screens may not be of the same magnitude or even Neural field model for induction type: in the smaller display, induction effects of the contrast kind will have less magnitude and tend toward effects assimilation. It is common practice in motion picture distribution The efficient representation principle, introduced to manually modify the original mastered picture by Attneave (1954) and Barlow (1961), is a general Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 3 strategy observed across mammalian, amphibian and x and y increases, I0 is the original image channel and insect species, where visual processing considers the α, β and γ are positive weights. statistics of the visual stimulus and adapts to its changes The first term in the functional of Equation 1 (Smirnakis, Berry, Warland, Bialek, & Meister, 1997). In measures the dispersion around the mid-range response fact, efficient representation requires that the statistics of 12 , as in the gray world hypothesis for color constancy of the image input are matched by the coding strategy, which states that in a sufficiently varied scene the and although a global part of this coding strategy average color will be perceived as gray (an observation must have evolved on long timescales (development, made by Judd (1940, 1979) and formalized by evolution), to be truly efficient the coding must also Buchsbaum (1980) and therefore the illuminant color adapt to the local spatiotemporal changes of natural can be estimated from the color average of the scene; images occurring at timescales of hours (e.g., from this implies that the minimization of E (I ) will make the daybreak to dawn), seconds (e.g., when we move from image mean tend to 12 , so that the first term is small, and one environment into another), or fractions of a second corresponding to the case where the illuminant is white. (e.g., when our eyes move around). By constantly The second term in the functional measures the adapting to the statistical distribution of the stimulus, contrast as the sum of the absolute value of the pixel the visual system can encode signals that are less differences (weighted, through w, by the distance redundant and this in turn produces metabolic savings between said pixels); because of the negative sign in by having weaker responsiveness after adaptation, since front of this term, minimizing E (I ) will increase the action potentials are metabolically expensive (Kohn, contrast. 2007). Finally, because the third term measures the Atick and Redlich (1992) make the point that there difference with the original image I0 , the minimization are two different types of redundancy or inefficiency in of E (I ) will yield a result that can’t be too far away an information system like the visual system: from I0 . The gradient descent equation for this functional is (1) If some neural response levels are used more    frequently than others. For this type of redundancy, 1 It (x)= −α I (x) − +γ w(x, y)sgn(I (x) the optimal code is the one that performs histogram 2  equalization. There is evidence that the retina is carrying out this type of operation at photo-receptor −I (y))dy − β(I (x) − I0 (x)) (2) level (Olshausen & Field, 2000), because their Starting from I = I0 , Equation 2 is iterated until a response curves match the cumulative histogram of steady state is reached (corresponding with a minimum the luminance distribution of the environment. of E), that will be the result of this algorithm. (2) If neural responses at different locations are not The energy in Equation 1 introduces the influence independent from one another. For this type of of spatial neighbors through the distance function redundancy, the optimal code is the one that w. Without it (and with β = 0) the energy becomes performs decorrelation. There is evidence in the the one proposed by Sapiro and Caselles (1997), retina, the lateral geniculate nucleus and the visual whose minimization produces a (global) histogram cortex that receptive fields act as optimal “whitening equalization of the original image. Therefore, we can filters,” locally decorrelating the signal. argue that the evolution Equation 2 performs local histogram equalization. From the above, a local histogram equalization This method was applied channel-wise on color (LHE) process would simultaneously reduce both types images in RGB (Bertalmío et al., 2007) and in a of redundancy. Bertalmío, Caselles, Provenzi, and color opponent color space like CIELAB (Zamir, Rizzi (2007) propose a variational method to improve Vazquez-Corral & Bertalmío, 2017), and the results the color appearance of images, that performs LHE. showed that the LHE method has several good They introduce the following energy functional, whose properties: minimization yields the method’s result:     (1) It has a very good local contrast enhancement α 1 2 E (I )= I (x) − dx − γ w(x, y)|I (x) performance, producing results without visual 2  2 2 artifacts of any kind (only when the width of the  β locality kernel w is very small do haloes start to −I (y)|dxdy + (I (x) − I0 (x))2 dx, (1) appear). 2  (2) It “flattens” the histogram, approaching histogram where I is an image channel in the range [0,1],  is the equalization, as expected due to the relationship image domain, x, y are pixels, w is a distance function of Equation 1 with the one in the histogram such that its value decreases as the distance between equalization model of Sapiro and Caselles (1997). Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 4 (3) It reproduces visual perception phenomena such as is very wide, and therefore we can expect that the local simultaneous contrast and the Mach Band effect; contrast enhancement procedure of Bertalmío et al. this is consistent with the functional of Equation 1 (2007) will always produce contrast, not assimilation, modeling perceived contrast in a localized manner, because, as we mentioned elsewhere in this article, with close neighbors exerting a higher influence than assimilation is linked to high spatial frequencies far-away points. (Shevell, 2003). To overcome the intrinsic limitations (4) It yields very good color constancy results, being of Bertalmío et al. (2007) with respect to induction, able to remove strong color casts and to deal with we should introduce spatial frequency in the energy non-uniform illumination (a challenging scenario functional. In Bertalmío (2014a) this is done by for most color constancy algorithms, as discussed in making the parameter γ in Equation 2 change both Bertalmío (2014b)). spatially and with each iteration, according to the local standard deviation: if the neighborhood over which it is computed is sufficiently small, the standard deviation Additionally, the LHE model of Bertalmío et al. can provide a simple estimate of spatial frequency. (2007) is closely related to the neural field model of But also, the standard deviation is commonly used in Wilson and Cowan, as pointed out in Bertalmío et the vision literature as an estimate of local contrast. al. (2007) and further discussed in Bertalmío and The model in Bertalmío (2014a) can predict lightness Cowan (2009). In particular, the evolution Equation assimilation and further improves efficiency by reducing 2 is very similar to the Wilson-Cowan equations (see redundancy: flattening the histogram and whitening the Bressloff, Cowan, Golubitsky, Thomas, & Wiener 2002; power spectrum. Other attempts to modify the LHE Wilson & Cowan, 1972, 1973), which have a long formulation so that it better deals with induction are and thriving history of modelling cortical low-level discussed in Bertalmío (2019). dynamics Cowan, Neuman, and Drongelen (2016). Unfortunately, the modifications introduced to the It has been proven recently (Bertalmío et al., 2020) LHE model in Bertalmío (2014a) do not fit well with the that the Wilson-Cowan equations are not variational, basic postulates of Wilson and Cowan’s theory. This is in the sense that they can’t be minimizing an energy why in this section we propose to adapt the LHE model functional, and that the simplest modification that in a different manner to predict induction, with changes makes them variational yields the LHE method of that are motivated by neurophysiology data and that Bertalmío et al. (2007); furthermore, the LHE model now keep the model consistent with the Wilson-Cowan provides a better reproduction of visual illusions formulation. Specifically, we want to take into account than the Wilson-Cowan model. The study of visual the following biological phenomena. illusions has always been key in the vision science community, as the mismatches between reality and perception provide insights that can be very useful to Photoreceptor response develop new models of visual perception (Kingdom, Photoreceptor response curves can be approximated 2011) or of neural activity (Murray, Vanrell, Otazu, & very well with the Naka-Rushton equation: Parraga 2013), and also to validate existing ones. It is commonly accepted that visual illusions arise owing to In neurobiological constraints (Purves, Wojtach, & Howe, R(I ) = Rmax , (3) 2008) that limit the ability of the visual system, and are I n + Isn therefore related to efficient representation. In short, the LHE method (in its original formulation of Bertalmío where R is the response, Rmax is the maximum or et al. (2007) and also when it considers orientation saturation response, I is the intensity, n is an exponent (Bertalmio, Calatroni, Franceschi, Franceschiello, & of around 0.75, and Is is the so-called semi-saturation Prandi, 2020) is the generalization of the Wilson-Cowan value, the intensity at which the response is one-half of equations that makes them compliant with the its maximum value and that roughly corresponds to the efficient representation principle, and at the same time average intensity level. Notice that the Naka-Rushton this allows for an improved reproduction of visual equation is a monotonically increasing function and is perception phenomena. therefore invertible; this point will become important elsewhere in this discussion. If we increase Is and plot R in linear-log coordinates, as in Figure 3, then the curve Modifying the LHE model so that it predicts moves to the right, the same curve-shifting phenomena induction observed when the background level increases. Therefore, light adaptation can be seen as changing the Looking at Equation 2, we can see that the spatial semi-saturation constant in the Naka-Rushton equation arrangement of the image data is only taken into (Shapley & Enroth-Cugell, 1984). Furthermore, from account by the weighting function w. But in practice w Equation 3 and if n = 1, we can obtain Weber’s law. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 5 Whittle, 1992) that demonstrates that our sensitivity to brightness is enhanced at values near the average or background level. Retinal lateral inhibition can explain assimilation Lateral inhibition creates the typical center-surround structure of the receptive field of retinal ganglion cells (RGCs), with the excitatory center owing to the feed-forward cells (photoreceptors and bipolar cells) and the inhibitory surround owing to the Figure 3. Photoreceptor response curves for primate cones, for inhibitory feedback from interneurons (horizontal and different background levels. Reprinted from Valeton and van amacrine cells). This center-surround organization is Norren (1983), Copyright 1983, with permission from Elsevier. a very important instance of efficient representation, performing signal decorrelation and allowing to For this and other factors, it appears that the perceptual represent with less resources large uniform regions effects of light adaptation can be mostly accounted for because they generate little or no activity. It should by retinal processing (Meister & Berry, 1999). also be pointed out that a more recent work (Rucci & Victor, 2015) contends that decorrelation is already performed by the rapid eye movements that happen Neural response nonlinearities and signal equalization during fixations, and therefore that the signal arrives Neural adaptation performs a (constrained) signal already decorrelated at the retina: the subsequent spatial equalization by matching the system response to the filtering performed at the retina and downstream must stimulus mean and variance (Dunn & Rieke, 2006), have other purposes, like enhancing contrast. thus ensuring visual fidelity under a very wide range of Classical studies assumed that assimilation had to lighting conditions. Figure 4 (left) shows that when the take place at a later stage than the retina, most probably mean light level is high, the nonlinear curve that models at the cortex, because it needs a much longer range of retinal response to light intensity is a sigmoid function interaction between image regions than what lateral with less steep slope than when the mean light level is inhibition could provide with the classical receptive low. Figure 4 (right) shows that at a given ambient level, field size. But in Yeonan-Kim and Bertalmío (2016a) the slope of the sigmoid is lower when the contrast is Yeonan-Kim and Bertalmío showed that, in fact, higher. In both cases, the data are consistent with the assimilation can start already in the retina. They took nonlinearity of the neural response to light performing classic retinal models, those of Wilson (1997) and van histogram equalization, since the nonlinearity behaves Hateren (2005), and adapted them so that parasol as the cumulative histogram (which is the classical tool RGCs have a surround that is now dual, with a narrow used in image processing to equalize a histogram) does: component of large amplitude and a wide component darker images and images with lower contrast typically of smaller amplitude. This different form for the have less variance and therefore their cumulative surround is based on more recent neurophysiological histograms are steeper. The psychophysical experiments data showing that retinal interneurons have retinal in Kane and Bertalmío (2016) corroborate that the fields that are much more extended than previously visual system performs histogram equalization by assumed, and RGC responses show a component that showing how observers prefer display nonlinearities goes beyond the classical receptive field. that allow the displayed image to be perceived as having Based on this discussion, we propose the following a brightness distribution as close to uniform (i.e., with two-stage model: an equalized histogram) as possible. (1) The image stimulus I, which is a scalar-valued linear Asymmetry of neural response nonlinearity image (i.e., an image channel proportional to light intensity) is passed through the photoreceptor Recent works from neurophysiology prove that OFF nonlinearity, modeled as a Naka-Rushton equation, cells (those that respond to stimuli with values below yielding J0 : the average stimulus level) change their gain more than ON cells during adaptation (Ozuysal & Baccus, 2012), In and that the nonlinear responses of retinal ON and J0 = NR(I ) = , (4) OFF cells are different (Kremkow et al., 2014; Turner I n + Is n & Rieke, 2016; Turner, Schwartz, & Rieke, 2018, see Figure 5). These data on neural activity is consistent where the exponent of the NR equation is chosen so with psychophysical data (Kane & Bertalmío, 2019; as to maximize the equalization of the histogram of Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 6 Figure 4. Neural adaptation to mean and variance. Left: neural response to higher (in green) and lower (in blue) mean luminance. Right: neural response to higher (in red) and lower (in blue) luminance variance. Adapted from Dunn and Rieke (2006), Copyright 2006, with permission from Elsevier. Figure 5. ON and OFF cells have different nonlinear response functions. Reprinted from Turner and Rieke (2016), Copyright 2016, with permission from Elsevier. J0 , and the semi-saturation constant Is is the median term has this form: average of the image.  (2) The following evolution equation is run until a Kc (x, y)φ(J(x) − J(y))dxdy (6) steady state is reached: 2 where φ(·) is a function whose derivative is the Jt (x) = − α(J(x) − Km ∗ J(x)) sigmoid σ (·).  +γ Kc (x, y)σ (J(x) − J(y))dy Let’s call this model LHEI (I for “induction”) for the  sake of brevity. − β(J(x) − J0 (x)) (5) Here Km , Kc denote kernels each expressed as a sum Methods: LHEI model validation of two Gaussian functions and ∗ is the convolution operation, so now instead of a global mean 1/2 as in To validate LHEI, we will use the chromatic Equation 2 we have a local mean Km ∗ J(x) and local induction data of Monnier (2008). In that work, neighbors exert more influence but very far apart observers were shown a test ring of some given points can affect the response as well. Furthermore, chromaticity, surrounded by 16 concentric rings σ is a sigmoid function such that σ (0) = 0, but not (one-half on each side of the test) that constitute the necessarily anti-symmetric, hence allowing positive inducing pattern. This is the test image. The surrounding and negative responses to be of different magnitude. rings alternated between two chromaticities, which in Let us note that Equation 5 is the gradient descent isolation seem to be lime and purple, selected because equation for an energy functional where the contrast they differently stimulate the S cones only. Next to Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 7 Figure 6. Chromatic experiment stimuli. Left: test ring surrounded by concentric inducing rings of two alternating chromaticities. Right: comparison ring over uniform background. Note that the comparison and test rings are presented at the same chromaticity, and in the actual experiment, these patterns are placed over a black surround. this image, the observer was shown a comparison ring, with the same dimensions as the test ring, but Figure 7. Results from applying the LHEI model to observer data in this case simply presented over a uniform grey from Monnier (2008). Triangles represent mean observer background (i.e., without inducing patterns). This responses and lines represent LHEI model predictions, both in is the comparison image. Observers adjusted the terms of S-chromaticity difference between test and hue, saturation and brightness of the comparison comparison rings. The 95% confidence intervals are included ring to match the appearance of the test ring. See for each of the observer data points. Values above zero: results Figure 6 for an illustration of this experimental when inducing rings next to the test ring have a lime hue. set-up. Values below zero: results when inducing rings next to the test The resulting chromaticity of the comparison ring is ring have a purple hue. not the same as the chromaticity of the test, owing to the induction effects produced by the lime and purple rings that surround the test ring: the difference in the with purple error bars for the purple/lime patterns and S-chromaticity (associated to the S cones) between test green triangles with blue error bars for the lime/purple and comparison rings is a color shift that quantifies the patterns. The error bars represent 95% confidence error induction and can be plotted against the S-chromaticity intervals about the mean, averaged across observers and of the test ring. Monnier performed this experiment trial repetitions. The fits of the model are shown in solid with four observers, seven test-ring chromaticities, and lines, in orange for the purple/lime pattern case and in the two possible alternating orders for the inducing blue for the lime/purple case. As we can see, the fit is rings (lime followed by purple, or the other way quite good and qualitatively similar to the one obtained round). by Song et al. (2019) using a neural field model based We have optimized the parameters of the LHEI on the Wilson-Cowan formulation. For the purple/lime model so that when we apply it to the test and pattern case, our model makes predictions that are comparison images, the resulting S-chromaticity within the range of experimental error for all test ring difference between test and comparison ring is as close S channel values; however, it does not properly fit the as possible to the one reported in the psychophysical steeper slope of the lime/purple case. experiments. For each of the initial conditions, we run our method using both the original rings, and the comparison ring adjusted by observers as input. Then, Invertible model for induction our minimization looks at the difference between the test ring in these two images. The error between the two effects images is computed as the L2 difference between the value of the central rings. Finally, the error for each of As stated in the Introduction, we want to derive a the initial conditions is summed up to obtain the total method that matches induction effects among screens error to minimize. of different size, not a method that estimates induction effects and their appearance. The difference is very relevant, and it is similar to the fact that colorimetry Results and color spaces allow us to determine quite accurately The resulting psychophysical data, averaged over when two colors are perceived as different or the same, the observers, is shown in Figure 7 as orange triangles but they cannot tell us the perceived appearance of said Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 8 colors, as there are many external factors that play a invertible. In what follows, we show how to modify the role in this; we must remark though that this approach LHEI model so as to make it invertible. contrasts with that of works like (Bertalmío et al., In Kim, Batard, and Bertalmío (2016), the authors 2020), where the output of the algorithm was explicitly went back to the retinal models that were updated simulating the appearance. and analyzed in Yeonan-Kim and Bertalmío (2016b), Let’s say that we have a color appearance model M studied what were their most essential elements, and that is invertible and capable of reproducing induction produced the simplest possible form of equations to effects. We consider two viewing scenarios A and B model the retinal feedback system that are nonetheless in which the same image stimulus I is presented on capable of predicting a number of significant contrast a display, and both scenarios have identical viewing perception phenomena like brightness induction conditions except that the screen in A has a different (assimilation and contrast) and the band-pass form size than the screen in B. In this study, we isolate for of the contrast sensitivity function. These equations viewing angle and its effect on color perception; it is form a system of partial differential equations that well-known that other viewing parameters, like ambient minimize an energy functional, closely related to the illumination, screen luminance and dynamic range, one of the LHE method of Bertalmío et al. (2007), but display color gamut, and so on, may have a significant where the absolute value function in the second term of impact on perception, but the usual practice in the Equation 1 is raised to the power of two. This has the literature, given the challenges in modeling vision, is to effect of regularizing the functional, making it convex, vary one of these elements while the others are kept and therefore its minimum can be computed with a fixed. The model M predicts for image I an appearance single convolution, whereas the functional in Bertalmío MA (I ) in scenario A and an appearance MB (I ) in et al. (2007) is non-convex and as a consequence its scenario B. These appearances will be different because minimum has to be found by the iteration of the MA and MB are two instances of model M that will gradient descent equation. If we follow this approach generally have different parameter values. The reason to modify the contrast term of the energy functional for this is that, as we mentioned in the section on neural associated to the LHEI method (Equation 6), we processes adapt to the scene statistics, and by scene obtain: we mean the whole field of view, a part of which is the  screen where the image stimulus is displayed. Therefore, Kc (x, y)(J(x) − J(y))2 dxdy, (7) different viewing angles will result in different scenes, 2 consequently yielding different adaptation processes. In fact, in linear-nonlinear (L+NL) models of vision (and where as usual  is the rectangular domain of the image the model M we will be proposing shortly will be of that is displayed (i.e., not the whole field of view). this kind), adaptation is actually defined as the change With this modification, the gradient descent equation of the model parameters when the input changes, and previously shown in Equation 5 now becomes: the full-view scenes in A and B provide different inputs to the visual system because the viewing angle of the Jt (x)= − α(J(x) − Km ∗ J(x))  screen is different. Then, our induction matching goal can be +γ Kc (x, y)(J(x) − J(y))dy  expressed as determining the parameters for the compensation method C = MB−1 · MA , because when − β(J(x) − J0 (x)) (8) the preprocessed image C(I ) is shown on screen B its appearance, including induction effects, will be Now the minimum can be computed directly by convolving the input image J0 with a kernel S: MB (C(I )) = MB · MB−1 · MA (I ) = MA (I ), that is, the same as if the image was seen on screen A. In short,   having an invertible appearance model M for induction − β allows us to have an explicit analytical expression for S=F 1 , (9) α + β − γ − αF (Km ) + γ F (Kc ) C, and the parameter values for C might be found so that they match psychophysical data. Furthermore, and very importantly, we don’t need to optimize M so where F represents the Fourier transform. The kernel S that it accurately predicts induction effects in image clearly has an inverse kernel S−1 such that S ∗ S−1 = δ: appearance, which is a very challenging open problem:   we just need to optimize C so that the induction effects −1 α + β − γ − αF (Km ) + γ F (Kc ) − match in the two conditions. S =F 1 (10) β This implies, however, that neither the LHEI model nor any of the color induction models in the literature (e.g., Otazu et al., 2010; Song et al., 2019) can be used We propose the following modified version of the for our induction compensation goal, as they are not LHEI model, also consisting of two stages: Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 9 (1) The first stage is identical to the first stage of the is [0,1]. To prevent these issues, in practice we clip LHEI model: all out-of-range values of SB−1 ∗ SA ∗ NRA (I ) so that negative values are set to 0 and values greater than 1 are In set to 1; nonetheless, it is not expected that this clipping J = NR(I ) = , (11) I n + Is n procedure produces visible artifacts, as attested by the natural image examples in Figure 12. where we recall that we consider I to be a Following Equation 14, our goal is to fit the two scalar-valued image, I :  → [0, +∞), so exponents of the Naka-Rushton equations and J :  → [0, 1). the two convolutions SB−1 and (SA ). Following the (2) The second stage produces the output O as the approach used in the section on the Invertible model of convolution of J with the kernel S of Equation 9: conduction effects to represent kernel S, we define what we call the compensation kernel SC as SC = SB −1 ∗ SA : O=S∗J (12)   −1 −1 D2 + D1 F (KF ) SC = SB ∗ SA = F (15) C2 + C1 F (KF ) Let’s call this model M. The output O = M(I ) can be expressed as M(I ) = S ∗ NR(I ). The inverse of the Naka-Rushton where KF is the weighted sum of four Gaussians, and equation is C1 , C2 , D1 , and D2 are real numbers. The formulation was presented for single-channel   1n images. For color images, we will apply the induction −1 J compensation C channel-wise. To this end, given an NR (J ) = Is · , (13) 1−J input image in the display-referred RGB space, we will first transform it to a cone-space representation and the inverse kernel S−1 was defined in Equation 10. (CAT02 LMS space) (Rich, 2006) by applying the Therefore, the inverse of M can be expressed as electro-optical transfer function of the input space, M −1 (O) = NR−1 (S−1 ∗ O). converting to CIEXYZ 2-degree tristimulus values given the chromaticity coordinates of the primaries and white point, and finally applying the 3 × 3 linear transformation matrix from XYZ to CAT02. In this Induction compensation method space, we will apply the first Naka-Rushton equation NRA to the individual L, M, and S channels. In this step Based on model M, defined in Equations 11 and the Naka-Rushton exponents will be equal for all the 12, we propose the following method for induction channels. After this is done, we will further convert the compensation for screens of different size. color representation to an opponent one, with channels If an image I is to be shown on screen B producing that we call Y , op1 and op2 , computed as: the same induction effects as if it were shown on screen Y =L+M +S (16) A, in both cases under the same viewing conditions, then a compensation method C must be applied to the op1 = L − M (17) image I, yielding an image C(I ). When C(I ) is displayed on screen B the induction effects are the same as when I op2 = 2S − (L + M ). (18) is displayed on screen A. The compensation method C is: Then, our method will convolve each of the channels with the compensation kernel SC . Let us note that C(I ) = MB−1 (MA (I )) there will be two different compensation kernels: = NR−1 −1 B (SB ∗ SA ∗ NRA (I )) (14) one for the chromatic channels and another for the achromatic one. Once this is done, our method will The linear filter S of model M has a center-surround apply the inverted opponent channel transformation, form that, as mentioned in the Section on the Invertible clipping to the range [0,1] and the inverse of the second model of conduction effects, can perform decorrelation Naka-Rushton equation NRB (see Equation 14). Again and contrast enhancement. For images with very here, the Naka-Rushton exponent is kept equal for the high contrast, convolution with SA could produce three channels. Finally, to convert the processed image over enhancement, resulting in some undershoot or back to a state that is ready for display, the inverse 3 overshoot values falling outside the range [0,1], and in × 3 linear transformation (CAT02 to XYZ) from the some cases these values might still remain out of range forward process is applied followed by the primary after convolution with SB−1 , making it impossible to matrix (XYZ to RGB) and the inverse electro-optical apply the function NR−1 B to them because its domain transfer function of the destination space. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 10 To validate our method, we consider a scenario where A corresponds with a cinema screen and B with a mobile display. We perform psychophysical experiments for both achromatic and chromatic induction patterns where observers look at a display with two scales of the same image, and they have to adjust the values of a given region of the small-scale image (corresponding with the mobile viewing scenario) so that it matches the appearance of that region on the large-scale image (corresponding with the cinema viewing condition). For cinema, three picture heights viewing distance (a common figure for mastering) is assumed resulting in a vertical viewing angle of 18.92◦ ; for the mobile condition, the same viewing angle as in Canham, Murdoch, and Long (2018) is used resulting in a scaling factor of 0.39 between the two viewing scenarios. Using the data from these experiments the parameters of two separate kernels—one for the achromatic channel, another one for the chromatic channels—and the two Naka-Rushton exponents nA , nB will be found by minimizing the error between the observer data and the method results. Figure 8. Achromatic experiment stimulus. Note that all gray bars are presented at the same value, and in the actual experiment, these patterns are placed over a black surround. Methods: Achromatic induction Following the preceding work Bertalmío, Batard, and Kim (2016), the achromatic experiment was intended Sony PVM-A250 reference monitor, representing an to be a direct expansion of the experiments of Helson easily controllable cinema-like viewing environment. (1963) for the case of emissive stimuli. To this effect, we The monitor was calibrated to Rec. 709 primaries used the same type of induction pattern with a fixed with a D65 white point and a 2.4 gamma decoding inducer bar width and varied the comparison bar width. nonlinearity. These settings were verified routinely In this case however, observers reported the necessary before experimental sessions using a Klein K10-A correction factor directly by adjusting the luminance colorimeter. The experimental cadence was controlled of the comparison bars in the mobile scaling to match by a MATLAB test bed using the Psychophysics those in the cinema scaling (test). The additional toolbox (Brainard, 1997; Pelli, 1997) to display stimuli. variable of starting comparison bar luminance was also Observers adjusted the comparison bar luminance via varied between experimental presentations such that keyboard input to the experimental test bed. observers could approach their response from different Figure 8 shows the presented stimuli for the directions. The complete matrix of experimental factors achromatic experiment. As can be seen in the figure, is shown in Table 1. two patterns were presented on screen (except over a Laboratory setup: Experiments were conducted under black background, as opposed to the white surround dark surround viewing conditions on a calibrated they are presented over in the figure.) Preliminary experiments showed that the background color was a relevant factor, so we elected to display patterns over a black background to match the dark surround Factor Type Levels viewing condition. Each pattern consists of two sides, Comparison Variable 0.19◦ , 0.38◦ , 0.54◦ , 0.76◦ , 0.96◦ representing positive (white/gray) and negative contrast width (black/gray) respectively. Observers adjusted these two Initial Variable 4.0,8.1,22 cd/m2 sides separately, but both are included simultaneously comparison such that lightness references remain constant. The luminance white fields were presented just below the maximum Inducing bar Constant 0.19◦ monitor white at a value of 90 cd/m2 , while the black width fields were presented at a value of 0.6 cd/m2 . Procedure: Observers were ushered into the laboratory Table 1. Achromatic experimental factors. Visual angles and the experimental instructions were read aloud. The correspond with the cinema size patterns. instructions covered the purpose of the experiment, the Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 11 observer task and the control scheme. Before starting the trials, observers were given a test trial with the experimenter in the room to familiarize themselves with the controls. Then, observers completed the experiment task for the 15 patterns with different target bar widths and presentation values as they were presented on the screen in a random order. Observers: Ten observers (two female and eight male) aged between 23 and 39 years participated in the experiment. All observers had normal or corrected acuity (20/20). Three observers are authors, and the remaining seven were naive to the purpose of the study. Optimization: The compensation model C was optimized to fit the achromatic experiment data as generally as possible, meaning that the mean observer response from each target bar width experiment were considered simultaneously in the error function. The error was calculated as the sum squared difference in luminance values between the observer responses and Figure 9. Achromatic experiment results. Yellow: original gray a value sampled from the center of the comparison value stimulus. Orange: average observer-selected value for bars after the method was applied. By optimizing in gray bars over black background, showing 95% confidence this way, we show that the method can be made to intervals. Blue: average observer-selected value for gray bars work generally for different spatial configurations. over white, showing 95% confidence intervals. Green: The values obtained for this experiment following prediction of induction compensation model C for bars over the above procedure are: nA = 0.7861, nB = 0.7063, black. Purple: model predictions for bars over white. KF = −1.14G156 + 1.86G29 + 0.13G3 − 1.76G40 , C1 = 3.94, C2 = 2.54, D1 = 2.46, D2 = 2.72. These values have been obtained in relation to images of size 800 × 800. three-channel adjustment caused a great increase in the difficulty of the experimental task. Thus, we simplified the procedure by decreasing the number of variables. Results In this case, we test four color sets in the mobile and Figure 9 shows the average observer responses in cinema sizes, and we test for three different comparison the experiment and the prediction provided by the ring starting colors. To avoid observer fatigue, compensation model C. We can see how the observer experiments were conducted two color sets at a time. results are consistent with those of Helson (1963) in two The complete matrix of experimental factors is shown in key points: first, when the visual angle (equivalently the Table 2. comparison bar line width) decreases, the appearance For the chromatic experiments we took inspiration tends to assimilation, and hence the compensation from Monnier and Shevell (2004) and used concentric requires enhancing the contrast; second, as the visual circular induction patterns as shown in Figure 10. angle increases, the amount of necessary compensation Observers must adjust the CIELAB values of the should decrease. Our model responses are consistently central ring of the comparison pattern (the achromatic inside the range of experimental error for the observer circular ring on the right side of each set) so that it data. matches the appearance of the central ring of the test pattern (the concentric circular pattern on the left side of each set). This procedure was repeated for patterns Methods: Chromatic induction at mobile and cinema scaling settings, and the observer reported correction was found by taking the difference Based on evidence that the phenomenon of between responses (cinema − mobile). Otherwise, an induction occurs after visual signals are separated equivalent procedure to the achromatic experiment was into different visual pathways, we expand on the conducted in this case. achromatic experiments by making color matches in Laboratory setup: The laboratory conditions regarding an opponent channel space, with the intention of the display, surround, experimental test bed were all applying our compensation model C to the channels the same as the achromatic experiment. One change, separately. In this case, we chose CIELAB space however, was that observers input their responses because it has some degree of perceptual uniformity. via a tangent element color correction panel, which In initial experiments we found that this expansion to allowed for multichannel adjustments to stimuli with Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 12 Factor Type Levels Pattern scaling Variable cinema and mobile (39%) scaling Initial comparison level Variable original, +10 a*b*, -10 a*b* Test pattern colors Variable sets 1-4 Test pattern diameter Constant 11.0◦ Test to comparison distance (center to center) Constant 15.6◦ Pattern center diameter Constant 4.39◦ Table 2. Chromatic experimental factors. Test pattern element sizes are relative to the cinema condition, but their size in proportion to each other is preserved for the mobile case. Figure 10. Chromatic experiment stimuli corresponding with sets one through four. On the right side of each set is the comparison ring surrounded by an achromatic field, which observers were asked to match to the test ring on the left, surrounded by the induction pattern. To illustrate the strength of the visual illusion, the comparison and test rings are presented with the same RGB value here. Note that in the actual experiment, these patterns are placed over a black background. separate knobs allowing for a more natural and reactive L*a*b* values within the Rec. 709 color gamut were experimental interface in comparison with keyboard generated (a color gamut is the range of colors that a input. display can reproduce, and Rec. 709 is the default gamut Stimuli: The stimuli for the chromatic experiments specification most commonly observed by display included four concentric circular induction patterns of and television manufacturers). These patterns were different color arrangements similar to those used in then shown side by side at the different scaling factors Monnier and Shevell (2004), as shown in Figure 10. tested in the experiment. Then, patterns for which The two relevant features of these patterns are that a hue shift could be identified between the different their circular shape results in less after-images when scaling factors were singled out. Finally, the experiment compared to the bars, and their use of dual inducing procedure was conducted for a single observer using colors leads to a stronger induction effect, allowing all selected patterns from the previous step. From these for more significant results to be gleaned from the results, the final patterns were selected based on the experiment. The L* value of all rings in these patterns criteria that the sensation of the target ring could be is kept consistent such that the focus of the observers’ reproduced successfully in isolation, given the gamut task could be on correction for chromatic induction. of the monitor, and that a statistically significant This said, observers were still permitted to adjust the correction (given 95% confidence intervals) was called L* channel value as equiluminance between pattern for by the observer between the two test pattern scaling regions was not confirmed. factors. After several iterations of the experiment, six To find patterns that exhibited a strong inductive total color sets were found. Administering the test to effect, an experiment was performed in which 100 multiple observers revealed that two of the sets should patterns containing regions with randomly selected be removed, because the target colors were too close Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 13 Figure 11. Chromatic experiment results for the four tested color sets, with the a*b* value of the test ring centered at the origin. The vectors represent the magnitude and direction of the color difference between the test ring and the inducing rings. The blue vectors represent the first inducers for each test case, or the color of the ring immediately adjacent to the test ring, and the red vectors represent the second inducer. The purple and green crosses represent the observer suggested compensation with 95% confidence error bars, and the red stars represent the response predicted by our correction method. The bottom right plot (set four) was used to test the model fit while the remaining three were used to train the model parameters. to the gamut boundary for observers to make reliable CIELAB, taking the monitor white point of D65 at 100 observations. cd/m2 for the reference illuminant. The optimization Observers: For the second color set, four observers is performed in order to minimize the E error on participated in the experiment (one female, three male) the test ring, and as we are using three different sets and for the remaining three color sets, three observers for training the minimization considers the maximum participated (three male). In both cases observer age value of the E error on the three test rings. Since ranged between 23 and 36, and two observers are our method is working in a color opponent space authors while the remainder were naive to the purpose different from CIELAB, when convolving kernel of the study. All observers had normal or corrected SC with our chromatic channels, shifts in the L* acuity (20/20). channel value may occur. To better comply with the Optimization: We performed these experiments for four observer responses, which reported no L* correction different concentric ring patterns, then optimized our to be necessary, we decided to replace the L* channel model C so that it fits the data for three of these images of our result by the L* channel of the original and finally validated our results on the remaining image. image. To accomplish this, we apply the kernel SC only The values obtained for the case where set 4 is used to the two chromatic components of our opponent for testing (corresponding to results in Figure 11 and channel space. The forward model is otherwise applied column 5 in Table 3) are: nA = 0.5187, nB = 0.4439, as described earlier in the section, however during KF = −1.53G103 − 0.67G43 + 0.67G4 + 0.34G26 , the inverse process we stop after applying the 3 × 3 C1 = 2.81, C2 = 1.30, D1 = 2.27, D2 = 1.60. Let us transformation from CAT02 to XYZ, and convert note that these values have been obtained in relation to this representation of the corrected image directly to images of size 800 × 800. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 14 Train on: 2-4 Train on: 1,3,4 Train on: 1,2,4 Train on: 1-3 Original Improvement Test on: 1 Test on: 2 Test on: 3 Test on: 4 error (on test set) Set 1 8.92 1.58 2.84 1.87 17.39 48.70% Set 2 1.50 5.35 2.84 1.95 13.20 59.47% Set 3 1.61 1.58 1.67 1.65 4.29 61.05% Set 4 1.17 1.46 2.79 8.18 15.00 45.48% Table 3. Error between the average observer response and our model’s prediction. We have performed the training for all combinations of three sets, testing on the remaining set (columns two to five). Column six represents the original error, and column 7 represents our improvement, w.r.t. the original error in the test set. Results we perform our experiments by training in three of In Figure 11, the results of the chromatic experiment the sets and testing in the remaining one. This gives are plotted in the two dimensional a*b* plane. The us four different cases. In the table, columns two results are limited to the chroma channels, as the to five represent each of the cases, with the model corrections reported by observers in the L* dimension error for the testing set shown in blue; in particular, were not statistically significant (95% confidence error column five corresponds with the case illustrated in ranges overlapped the origin for all tested cases.) For Figure 11. Column six presents in red what we call each of the patterns, the origin of the coordinate system the “original” error, the E difference between the is placed at the starting a*b* value of the test ring. In original data and the result of the observer correction. order to illustrate their directional influence, the plots Finally, column seven shows the improvement that depict the value of the inducing rings with a blue vector our method presents over the original error, which for the value of the inducing ring that is closer to the is in the range (45% − 60%), therefore highlighting test ring, which we call the first inducer, and a red vector the advantage of applying our compensation method for the value of the other inducing rings, that we call instead of doing nothing and just rescaling the original the second inducer. The average observer response is image. depicted with purple and green 95% confidence error bars. Looking at the observers’ responses, the induction compensation results selected by observers tend to show Methods: Induction in natural images contrast mainly in the direction opposite to the first inducer, which implies that the appearance of the mobile Because the compensation method C was designed viewing condition shows assimilation in the direction of with a direct imaging application in mind, it is the first inducer (because assimilation is compensated important to analyze its effect in the context of natural by contrast). In this way, the results for sets two through images. In comparison with the synthetic stimuli used four were consistent with the classic assumptions on to optimize and validate the model, the context of induction, as well as the results of Helson (1963); Fach real images introduces a significant increase in the and Sharpe (1986); Monnier (2008). However, our spatial complexity of stimuli. For example, the image first set shows that this cannot be taken as a general content could provide references for cognitive grouping rule, as observers reported the necessary correction to feedback and other higher order processes which could be roughly in the assimilation direction of the second be impactful to the induction effect (Murgia, 2016). inducer. Thus, before the method can be proposed for practical Regarding the ability of our compensation model C use, it is vitally important to first probe its behavior for for fitting this data, in Figure 11 we added our results a variety of test content and viewing contexts to ensure for the optimized values presented. Our resulting that it is tuned such that it improves the preservation of corrections predicted by C for each color set are creative intent with changes in presentation size as a depicted with a red star. We can see for the train whole. cases that the predictions are within the range of In addition to the image content, our experiments experimental error, and for all cases compensate input revealed a number of additional viewing scenario in the proper direction. factors which were influential to the induction effect. To further study our model, Table 3 shows the error First, in initial iterations of the achromatic experiments, (measured as E difference) between the average we found that the background and surround conditions observer response and our model’s prediction. As can completely change the nature of the induction effect explained above, given that we have four different sets, (changing the direction of required compensation). We also observed that induction effects are not only Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 15 dependent on the relative scale adjustment between complete the experiment with ample observation time stimuli, but also on the absolute scaling. Due to this, in 20 minutes. a different correction would be required for stimuli Observers: A total of 16 observers participated in the with absolute scaling of 2 and 1, than for stimuli with experiment (10 male, six female) aged between 24 and absolute scaling of 1 and 0.5. Finally, our observers 58 years. None of the observers are authors and all were reported during the experiments that there were visible naive to the purpose of the experiment. One-half of the shifts in the appearance of the synthetic induction observers work in an imaging related field and can thus patterns with the amount of time spent viewing be considered expert observers, and the other one-half them. were non-experts. Procedure: With all of these factors in mind, an online validation experiment was designed and conducted to evaluate the performance of the method Results on natural images. The experiment was designed Figure 12 illustrates the qualitative results of our and distributed using the PychoJS library and the induction compensation method C on some natural psychophysics-centered hosting platform Pavlovia images. We can see how in our results the colors are (Pierce et al., 2019). In the experiment, observers were subtly but noticeably more vivid, for example, the first given instructions to extinguish any direct light orange cone in the first row, the green teapot in the sources as to make their viewing environment as dark third, the kid’s blue jacket and boots and the grass as possible. Then, observers conducted the virtual chin in the fourth row, the yellow fish in the bottom row. rest test of Li, Joo, Yeatman, and Reinecke (2020) to This increased vividness corresponds with a contrast determine their pixel-per-degree viewing angle so that enhancement in the chroma, which should be cancelled stimuli could be adjusted to the correct presentation out by the visual assimilation (and resulting chroma size. They were asked to maintain their seating position contrast reduction) produced when observing the from this point in the experiment onward. Then, image under a smaller field of view; the relationship observers were given instructions that explained between contrast enhancement and more vivid colors that they would be presented with original full-sized is discussed in detail in Zamir et al. (2017); Zamir, reference images and corresponding down-scaled pairs Vazquez-Corral, and Bertalmío (2021) and Bertalmío (some of which are altered with respect to their color (2019). and contrast and others which are unaltered) at timed Although these results do not show visual artifacts intervals (5 seconds on, 2 seconds off), and would be of any kind, these problems cannot be ruled out as they asked to evaluate the match using one of the following might appear if the method’s parameters are optimized options: differently and/or the method is tested on other images. In analyzing the quantitative experimental results, it is important to acknowledge that unlike the model (1) The colors of this image have been altered optimization experiments, this experiment did not (2) The colors of this image may have been altered take place in a controlled laboratory setting and there (3) The colors of this image have not been altered could be significant variation in final image appearance between observers owing to display performance and After this, the observers conducted the body of the calibration, viewing environment limitations, and experiment, iterating through the test images in random adherence to the experimental cadence. Although order. we took steps to limit this variation in the online Stimuli: The down-scaled versions include the original, setting, this allowed for a lesser degree of control in unaltered image and the image corrected with our comparison to the previous experiments. The results of compensation method C. The method correction was Figure 13 show that, within this presentation context, applied following the process detailed at the beginning the induction effect is subtle in natural images as in the of the section, applying the kernel optimized in the majority of trials observers did not see a difference in achromatic experiment to the Y channel and the kernel color appearance between scaling settings. In contrast, optimized in the chromatic experiment to the opponent the results of our method C were seen as having had channels op1 and op2 . A series of images were selected their color appearance shifted in the majority of trials. which reflect a cinema or television shooting and Although this experiment was not a direct 2AFC grading style. The images cover a range of scene types comparison between the control and the results of and include important memory colors such as skin our proposed method, these were the only two types tones, product labels, natural colors, and so on. To of images presented to observers and thus the results avoid observer fatigue, considering the two repetitions can be interpreted comparatively, showing that the of each image and the minimum presentation time of correction provided by our method was of greater approximately 10 seconds, the number of test images magnitude than the shift caused by the induction effect was limited to 33 to allow for observers to be able to in this scenario. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 16 Figure 12. Left: original Images. Right: results of our method with parameters optimized for training sets one through three. Notice in our results how colors are slightly more vivid, and the absence of visual artifacts. Images sourced from Deng et al. (2009). Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 17 Figure 13. Validation experiment results: percentages of each response for the control (left) and for our proposed method (right). experiment was more or less informal in nature, its Discussion results demonstrate the rarity of strong induction effects given random color combinations, even if they The primary goal of these experiments was to are arranged in synthetic patterns which emphasize determine the correction required to match the induction. Another interesting anomaly that can be appearance of induction pattern targets at two different observed from this experiment is that despite the field of view scales. Based on the results of Helson random nature in which they were generated and (1963) and Fach and Sharpe (1986), we took the simple selected, the final four patterns seem to be quite similar hypothesis that a greater degree of contrast would to each other, all containing an inducer of violet hue. always be observed in the larger field of view pattern. The results of the chromatic experiments presented Thus, the correction from small pattern to large for here clearly show that the original hypothesis (that a given channel should always be in the direction of contrast effects will shift toward assimilation with an contrast, and the results of the achromatic experiment increase in test pattern spatial frequency) can be broken. confirmed our hypothesis. Although three of the color sets showed this behavior, For the chromatic experiments, making the we can see that the first color set breaks the trend, and assumption that the phenomenon of induction occurs is closer to requiring correction in the assimilation after visual signals are separated into different visual direction with respect to the second inducer. One clue pathways, we chose to make color matches in an to this differing behavior is related to the violet inducers opponent space with the intention of applying our which appear in each pattern. In all patterns for which corrective method C to the channels separately, induction effects behaved as expected, the violet inducer with different optimization values compared with was directly adjacent to the test and was the primary the achromatic case. We found that the use of the induction influence. However, for the first color set, the simple bar patterns of Fach and Sharpe (1986) violet field serves as the second inducer, which is not caused a multitude of problems in the chromatic case. directly adjacent to the test field, but still acts as the Observers reported weak induction effects as well as primary induction influence. strong afterimages when shifting their gaze between Outside of these data, we also found patterns that test patterns. In addition, the direct comparison of broke our simple hypothesis in preliminary experiment the cinema-sized pattern to the mobile pattern was iterations. From these iterations we observed that the confusing to observers, as the inducers in the smaller luminance level of the background/surround and the pattern appeared to be significantly less saturated. As a hues of inducing and target patches to be relevant solution, we took inspiration from Monnier and Shevell factors. This type of conflicting and paradoxical finding (2004) and used concentric circular induction patterns, seems to be common in the study of induction, with as shown in Figure 10, whose main features are that many works being based on the discovery of scenarios their circular shape results in less afterimages when which contradict previous findings (Monnier, 2008; compared to the bars, and their use of dual inducing Murgia, 2016). Although this phenomenon can be colors leads to a stronger effect, allowing for more found in all research topics and is a sign of progress, its significant results to be gleaned from the experiment. frequency in this area is an indication that induction A procedure for the selection of color sets that as a whole is still very much an open problem, despite exhibited a strong induction effect is explained its earliest formal works dating back nearly a century in Methods: Chromatic induction. Although this and a half. This can be justified by the fact that the Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 18 phenomenon is the result of complex interactions latter. The results showed that the method C makes involving both physiological and cognitive processes corrections of similar accuracy when it is required to (Singer & D’Zmura, 1994) on multiple visual pathways. reach out into larger color volumes. Although the model C was designed with the intention that it would always correct the test targets in the contrast direction, we have had some success in optimizing a kernel which works more generally Conclusion in fitting to these four test cases. These results were garnered by model fitting using three chromatic sets In this work, we have shown that a neural field model and testing on a fourth one. Looking at Table 3, one performing local histogram equalization is able to can see that the model C performs slightly better in predict chromatic induction effects. This is a variational predicting examples which are within its test set in model, an embodiment of the efficient representation comparison to when they are excluded, implying that principle, and by regularizing its associated energy its behavior is somewhat biased toward its training functional the model is still able to represent induction set. and now becomes invertible. This fact allows us to use Although the achromatic training set included a the new invertible model as the basis for an induction number of cases for which the necessary induction compensation method, which we call C, to preprocess correction was negligible, our chromatic kernel is an image in a screen—size dependent way so that its trained exclusively on examples which required a large perception, in terms of visual induction, may remain correction between presentation sizes. Our process constant across displays of different size. The potential of selecting these examples showed that this effect is of the method is demonstrated through psychophysical quite rare, even among synthetic patterns which are experiments on synthetic images, both achromatic specifically designed to produce strong induction effects. and chromatic. Our results show that the established The results of the validation experiment showed that the assumption in the literature that induction tends effect is even more subtle in the case of natural images toward assimilation as the spatial frequency increases is when presented in a “wild” context with variation in sometimes contradicted by the experimental data, and display and viewing environment conditions, with a therefore can’t be taken as a general principle. significant majority of observers reporting no color We believe there are three main avenues to explore in shift in the control examples. In comparison, the order to improve our proposed approach: correction provided by our method was detectable by observers in the majority of cases. Thus, it is likely that (1) Our induction compensation technique C is based our method in its current optimized state is producing on a color appearance model that follows the an exaggerated correction for what the practical classic formulation of a cascade of linear-nonlinear application requires. For this reason, the compensation (L+NL) modules (Martinez-Garcia, Cyriac, Batard, method presented here is to be interpreted as a proof of Bertalmio, & Malo, 2018) and has a biological concept and a contribution to the research in the field correlate, consisting of a nonlinear stage (the as opposed to a procedure which is ready to be used in Naka-Rushton equation that models photoreceptor practice. responses) followed by a linear stage (convolution A further interesting challenge is that the with a kernel that models lateral inhibition in the compensation value observers reported to adjust retina). A L+NL model is valid for stimuli of a given between mobile to cinema appearance could be outside distribution seen under given viewing conditions, in of any given monitor gamut space, or outside of the which case it may provide a good match to the firing gamut of physically realizable colors, depending on rate. But visual adaptation, an essential feature of the position of the test target and the magnitude and the neural systems of all species by which changes direction of the induction shift between screen sizes. In in the stimuli produce a change in the input–output these scenarios, induction effects will only be partially relation of the system (Wark, Fairhall, & Rieke, compensated for by the method C. We encountered 2009) alters the visual system response. Visual this issue with three of our four test sets, and opted adaptation is clearly a key element of the efficient to clip all observer and kernel reported corrections representation principle and it affects, among other to the Rec. 709 gamut. By doing this, our model’s things, the spatial receptive field and temporal results can be readily reproduced by the most common integration properties of neurons, requiring changes displays, including those which we used for visual in the linear and/or the nonlinear stages of a L+NL proofing during its development. We later performed model in order to explain neural responses (Meister a preliminary analysis with input encoded under the & Berry, 1999). So, for example, depending on the larger standard color gamuts Rec. 2020 and CIE 1931 input the receptive field of a single neuron can have XYZ, where the clipping of observer corrections is different sizes or preferred orientations (Coen-Cagli, smaller for the former and almost negligible for the Dayan, & Schwartz, 2012), or even change polarity Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 19 (ON/OFF) (Jansen et al., 2018). For our purposes I00 (MCIU/AEI/FEDER, UE). JVC received of induction compensation, we should study how to funding from the project PID2019-109628RJ- make the convolution kernel S depend on the input, I00/AEI/10.13039/501100011033 by the Ministerio de or instead to use a filter bank as is the traditional Ciencia, Innovación y Universidades (MCIU) and the approach with L+NL models for visual perception Agencia Estatal de Investigación (AEI) of the Spanish (Wandell, 1995; Graham, 2011). government. (2) Another option is to study how to make the LHEI model invertible while keeping it as a nonlinear Commercial relationships: none. neural field model (i.e., without regularizing its Corresponding author: Trevor Canham. associated functional), looking into a gradient ascent Email: trevor.canham@upf.edu. equation or alternatively considering changing the Address: Department of Information and sign of the parameter γ in the model, as it has Communication Technologies, Universitat Pompeu been shown that with one sign for γ the model Fabra, Barcelona, Spain. increases the contrast while with the opposite sign the model reduces the contrast (Bertalmío, Caselles, & Provenzi, 2009; Zamir et al., 2021; Zamir, Vazquez-Corral, & Bertalmío, 2014). We believe this option has more potential because the resulting References compensation model would not be of L+NL form. (3) Finally, a third avenue to explore, compatible with Atick, J. J., & Redlich, A. N. (1992). What does the previous two, would be to design and carry the retina know about natural scenes? Neural out psychophysical experiments for induction Computation, 4(2), 196–210. compensation where observers are asked to adjust Attneave, F. (1954). Some informational aspects of values over the whole image and not just on a visual perception. Psychological Review, 61(3), 183. particular region like the gray bars or the test ring. Barlow, H. B. (1961). Possible principles underlying Using this data, the correction method C could the transformation of sensory messages. Sensory be optimized such that it produces a balanced Communication, 1, 217–234. correction for all of the spatially adjacent regions Bertalmío, M. (2014a). From image processing to in the patterns simultaneously, accounting for their computational neuroscience: a neural model interdependent effects. Additionally, it would be an based on histogram equalization. Frontiers in interesting expansion of the work to include more Computational Neuroscience, 8, 71. test sets in the chromatic case, which do not produce strong illusions, as was done in the achromatic case, Bertalmío, M. (2014b). Image processing for cinema. as this would likely better represent the behavior of Boca Raton, FL: CRC Press, Taylor & Francis. the visual phenomenon in response to natural images. Bertalmío, M. (2019). Vision models for high dynamic range and wide colour gamut imaging: Techniques Keywords: color perception, visual induction, and applications. New York: Academic Press. efficient representation principle, neural field models, Bertalmío, M., Calatroni, L., Franceschi, V., local histogram equalization, variational models, Franceschiello, B., Gomez-Villa, A., & Prandi, D. Wilson-Cowan equations (2020). Visual illusions via neural dynamics: Wilson- cowan-type models and the efficient representation principle. Journal of Neurophysiology. Acknowledgments Bertalmio, M., Calatroni, L., Franceschi, V., Franceschiello, B., & Prandi, D. (2020). Cortical-inspired wilson–cowan-type equations The authors thank those who served as experimental for orientation-dependent contrast perception observers, without whom this work would not have modelling. Journal of Mathematical Imaging and been possible. In addition, we would like to thank Vision, 1–19. Patrick Monnier and Anna Song for their help in obtaining the data from Monnier (2008). Bertalmío, M., Caselles, V., & Provenzi, E. (2009). Issues about retinex theory and contrast enhancement. Funding from the European Union’s Horizon International Journal of Computer Vision, 83(1), 2020 research and innovation programme under 101–119. grant agreement number 761544 (project HDR4EU) Bertalmío, M., Caselles, V., Provenzi, E., & Rizzi, and under grant agreement number 780470 (project A. (2007). Perceptual color correction through SAUCE), and by the Spanish government and variational techniques. IEEE Transactions on Image FEDER Fund, grant ref. PGC2018-099651-B- Processing, 16, 1058–1072. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 20 Bertalmío, M., & Cowan, J. (2009). Implementing the Hateren, H. van. (2005). A cellular and molecular Retinex algorithm with Wilson-Cowan equations. model of response kinetics and adaptation in Journal of Physiology, Paris. primate cones and horizontal cells. Journal of Bertalmío, M., Batard, T., & Kim, J. (2016). Correcting Vision, 5(4), 5–5. for induction phenomena on displays of different Helson, H. (1963). Studies of anomalous contrast size. In Vision Sciences Society Annual Meeting. and assimilation. Journal of the Optical Society of von Bezold, W.. (1874). Die farbenlehre. Braunschweig: America, 53(1). Westerman. Jansen, M., Jin, J., Li, X., Lashgari, R., Kremkow, J., Brainard, D. H. (1997). The Psychophysics Toolbox. & Bereshpolova, Y. et al. (2018). Cortical balance Spatial Vision, 10, 433–436. between on and off visual responses is modulated by the spatial properties of the visual stimulus. Brenner, E., Ruiz, J., Herraiz, E. M., Cornelissen, F., Cerebral Cortex, 29(1), 336–355. & Smeets, J. (2003). Chromatic induction and the layout of colours within a complex scene. Vision Judd, D. (1940). Hue, saturation, and lightness Research, 43, 1413–1421. of surface colors with chromatic illumination. Journal of the Optical Society of America, 30(1), Bressloff, P., Cowan, J., Golubitsky, M., Thomas, 2–32. P., & Wiener, M. (2002). What geometric visual hallucinations tell us about the visual cortex. Neural Judd, D. (1979). Contributions to color science. Vol. 545, Computation, 14(3), 473–491. p. 516–522. National Bureau of Standards (USA). Brown, R., & MacLeod, D. (1997). Color appearance Kane, D., & Bertalmío, M. (2016). System gamma as depends on the variance of surround colors. a function of image-and monitor-dynamic range. Current Biology, 7, 844–849. Journal of Vision, 16(6), 4–4. Buchsbaum, G. (1980). A spatial processor model for Kane, D., & Bertalmío, M. (2019). A reevaluation object colour perception. Journal of the Franklin of whittle (1986, 1992) reveals the link between Institute, 310, 337–350. detection thresholds, discrimination thresholds, and brightness perception. Journal of Vision, 19(1), Canham, T., Murdoch, M., & Long, D. (2018). 16–16. Influence of ambient chromaticity on portable display color appearance. In Smpte 2018 Annual Kim, J., Batard, T., & Bertalmío, M. (2016). Retinal Technical Conference and Exhibition. processing optimizes contrast coding. Journal of Vision, 16(12), 1151–1151. Coen-Cagli, R., Dayan, P., & Schwartz, O. (2012). Cortical surround interactions and perceptual Kingdom, F. A. (2011). Lightness, brightness and salience via natural scene statistics. PLoS transparency: A quarter century of new ideas, Computational Biology, 8(3), e1002405. captivating demonstrations and unrelenting controversy. Vision Research, 51(7), 652–673. Cowan, J. D., Neuman, J., & Drongelen, W. van. (2016). Wilson–cowan equations for neocortical dynamics. Kohn, A. (2007). Visual adaptation: physiology, Journal of Mathematical Neuroscience, 6(1), 1. mechanisms, and functional benefits. Journal of Neurophysiology, 97(5), 3155–3164. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale Kremkow, J., Jin, J., Komban, S. J., Wang, Y., Lashgari, hierarchical image database. In Cvpr09. R., & Li, X. et al. (2014). Neuronal nonlinearity explains greater visual spatial resolution for darks Dunn, F. A., & Rieke, F. (2006). The impact of than lights. Proceedings of the National Academy of photoreceptor noise on retinal gain controls. Sciences, 201310442. Current Opinion in Neurobiology, 16(4), 363–370. Li, Q., Joo, S., Yeatman, J., & Reinecke, K. (2020). Fach, C., & Sharpe, L. T. (1986). Assimilative hue shifts Controlling for participants’ viewing distance in in color gratings depend on bar width. Perception & large-scale, psychophysical online experiments using Psychophysics, 40(6), 412–418. a virtual chinrest. Nature Scientific Reports. Gelb, A. (1930). Handbuch Norm. Pathol. Physiol., 12, Martinez-Garcia, M., Cyriac, P., Batard, T., Bertalmio, 594. M., & Malo, J. (2018). Derivatives and inverse of Graham, N. V. (2011). Beyond multiple pattern cascaded linear+ nonlinear neural models. PLoS analyzers modeled as linear filters (as classical v1 One, 13(10), e0201326. simple cells): Useful additions of the last 25 years. Meister, M., & Berry, M. J. (1999). The neural code of Vision Research, 51(13), 1397–1430. the retina. Neuron, 22(3), 435–450. Harrar, M., & Vienot, F. (2005). Regulation of Monnier, P. (2008). Standard definitions of chromatic chromatic induction by neighboring images. Journal induction fail to describe induction with s-cone of the Optical Society of America, 22, 2197–2206. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 21 patterned backgrounds. Vision Research, 48, Shevell, S. (2003). The science of color (pp. 149–190). 2708–2714. Elsevier Science Ltd. Monnier, P., & Shevell, S. (2003). Large shifts in color Shevell, S., & Wei, J. (1998). Chromatic induction: appearance from patterned chromatic backgrounds. Border contrast or adaptation to surrounding light? Nature Neuroscience, 6, 801–802. Vision Research, 38, 1561–1566. Monnier, P., & Shevell, S. (2004). Chromatic induction Singer, B., & D’Zmura, M. (1994). Color contrast from s-cone patterns. Vision Research, 44, 849–856. induction. Vision Research, 34(23), 3111–3126. Murgia, M. (2016, May). Perceptual belongingness Smirnakis, S. M., Berry, M. J., Warland, D. K., Bialek, determines the direction of lightness induction W., & Meister, M. (1997). Adaptation of retinal depending on grouping stability and intentionality. processing to image contrast and spatial scale. Vision Research, 126, 69–79. Nature, 386(6620), 69. Murray, N., Vanrell, M., Otazu, X., & Parraga, C. Song, A., Faugeras, O., & Veltz, R. (2019). A neural field A. (2013). Low-level spatiochromatic grouping model for color perception unifying assimilation for saliency estimation. IEEE Transactions on and contrast. PLoS Computational Biology, 15(6), Pattern Analysis and Machine Intelligence, 35(11), e1007050. 2810–2816. Turner, M. H., & Rieke, F. (2016). Synaptic rectification Olshausen, B., & Field, D. (2000). Vision and the controls nonlinear spatial integration of natural coding of natural images. American Scientist, 88(3), visual inputs. Neuron, 90(6), 1257–1271. 238–245. Turner, M. H., Schwartz, G. W., & Rieke, F. (2018). Otazu, X., Parraga, C., & Vanrell, M. (2010). Toward Receptive field center-surround interactions mediate a unified chromatic induction model. Journal of context-dependent spatial contrast encoding in the Vision, 10(5), 1–24. retina. bioRxiv, 252148. Ozuysal, Y., & Baccus, S. A. (2012). Linking the Valeton, J., & van Norren, D.. (1983). Light adaptation computational structure of variance adaptation to of primate cones: an analysis based on extracellular biophysical mechanisms. Neuron, 73(5), 1002–1015. data. Vision Research, 23(12), 1539–1547. Pelli, D. G. (1997). The VideoToolbox software for Wandell, B. A. (1995). Foundations of vision (Vol. 8). visual psychophysics: transforming numbers into Sinauer Associates, Sunderland, MA. movies. Spatial Vision, 10, 437–442. Wark, B., Fairhall, A., & Rieke, F. (2009). Timescales Pierce, J., Gray, J., Simpson, S., MacAskill, M., of inference in visual adaptation. Neuron, 61(5), Höchenberger, R., & Sogo, H., et al. (2019). 750–761. Psychopy2: Experiments in behavior made easy. Wesner, M., & Shevell, S. (1992). Color perception Behavior Research Methods, 51, 195–203. within a chromatic context: Changes in red/green Purves, D., Wojtach, W. T., & Howe, C. (2008). Visual equilibria caused by noncontiguous light. Vision illusions: an empirical explanation. Scholarpedia, Research, 32, 1623–1634. 3(6), 3706. Whittle, P. (1992). Brightness, discriminability and Rich, D. C. (2006). Publication cie 159: A colour the “crispening effect”. Vision Research, 32(8), appearance model for colour management systems: 1493–1507. Ciecam02. Color Research & Application, 31(2), Wilson, H., & Cowan, J. (1972). Excitatory and 158–158. inhibitory interactions in localized populations Rucci, M., & Victor, J. D. (2015). The unsteady eye: an of model neurons. Biophysical Journal, 12, information-processing stage, not a bug. Trends in 1–24. neurosciences, 38(4), 195–206. Wilson, H., & Cowan, J. (1973). A mathematical theory Sapiro, G., & Caselles, V. (1997). Histogram of the functional dynamics of cortical and thalamic modification via differential equations. Journal of nervous tissue. Biological Cybernetics, 13(2), Differential Equations, 135, 238–266. 55–80. Shapley, R., & Enroth-Cugell, C. (1984). Visual Wilson, H. R. (1997). A neural model of foveal light adaptation and retinal gain controls. Progress in adaptation and afterimage formation. Visual Retinal Research, 3, 263–346. Neuroscience, 14(3), 403–423. Shevell, P., & Monnier, S.K.and. (2005). Color shifts Yeonan-Kim, J., & Bertalmío, M. (2016a). Retinal from s-cone patterned backgrounds: Contrast lateral inhibition provides the biological basis of sensitivity and spatial frequency selectivity. Vision long-range spatial induction. PLoS One, 11(12), Research, 45, 1147–1154. e0168963. Journal of Vision (2021) 21(6):10, 1–22 Canham, Vazquez-Corral, Mathieu, & Bertalmío 22 Yeonan-Kim, J., & Bertalmío, M. (2016b). Retinal Zamir, S. W., Vazquez-Corral, J., & Bertalmío, M. lateral inhibition provides the biological basis of (2014). Gamut mapping in cinematography through long-range spatial induction. PLoS One, 11(12), perceptually-based contrast modification. IEEE e0168963. Journal of Selected Topics in Signal Processing, 8(3), Zamir, S. W., Vazquez-Corral, J., & Bertalmío, M. 490–503. (2021). Vision models for wide colour gamut Zamir, S. W., Vazquez-Corral, J., & Bertalmío, M. imaging in cinema. IEEE Transactions on Pattern (2017). Gamut extension for cinema. IEEE Analysis and Machine Intelligence, 43(5), 1777–1790, Transactions on Image Processing, 26(4), 1595–1606. doi:10.1109/TPAMI.2019.2938499.