EDITORIAL published: 22 January 2020 doi: 10.3389/fbioe.2019.00483 Editorial: Integrating Visual System Mechanisms, Computational Models and Algorithms/Technologies Hedva Spitzer 1*, Xavier Otazu 2 and Hagit Hel-Or 3 1 School of Electrical Engineering, Tel Aviv University, Tel Aviv, Israel, 2 Computer Science Department, Computer Vision Center, Autonomous University of Barcelona, Barcelona, Spain, 3 Department of Computer Science, University of Haifa, Haifa, Israel Keywords: computational models, algorithms, technologies, visual system, mechanisms Editorial on the Research Topic Integrating Visual System Mechanisms, Computational Models and Algorithms/Technologies The Research Topic on “Integrating Visual System Mechanisms, Computational Models and Algorithms/Technologies” collects novel studies that display a strong synergy between three entities: (1) the visual system from its various angles including physiological, psychophysical, and perceptual, (2) computational models whether descriptive or predictive, and (3) vision inspired algorithms and applications. The interaction between modeling and the various aspects of the visual system is expressed in the reciprocal contributions between the two. On one hand, visual mechanisms and neuronal units provide inspiration and basis for modeling approaches and their computational units within, and on the other hand, modeling provides novel insights and new understandings of the visual system mechanisms and its associated behaviors. Furthermore, computational models, and the underlying visual mechanisms, provide a basis for developing practical algorithms to perform image processing and image understanding. Edited and reviewed by: The articles in this Research Topic present computational models of the visual system Richard D. Emes, ranging from neuronal mechanisms, through visual mechanisms, to visual perceptual behavior University of Nottingham, United Kingdom and visual illusions. Modeling efforts take different computational approaches from building blocks that are inspired by mechanisms of the visual system, to a more global Gestalt approach *Correspondence: that attempts to explain a phenomenon regardless of the underlying elements using functional, Hedva Spitzer hedva@eng.tau.ac.il statistical, or learning approaches. Other articles develop applications ranging from visual system inspired measures such as image quality and image esthetics to applications such as classification Specialty section: and segmentation. This article was submitted to Several studies in this issue, present computational models of the visual system at the neuronal Bioinformatics and Computational level, and some include feasible physiological components in the model. In Gonzalez and Tsotsos, Biology, the authors suggest a computational model of attention based on the adaptation mechanisms and a section of the journal selective tuning of the V4 neurons which is expressed in the neurons’ firing rate during attentional Frontiers in Bioengineering and tasks. Different computational models are tested, coinciding with different interpretations of the Biotechnology attention mechanism: (a) enhancing responses due to attention or (b) suppressing irrelevant Received: 21 November 2019 signals. The authors follow a model of the second type and are able to predict the temporal profiles Accepted: 27 December 2019 of neurons’ firing rate, similar to those found electrophysiologically. Through their modeling, the Published: 22 January 2020 authors show that high level vision processes can also be explained by low-level processes, namely, Citation: that selectively tuning a model of attention, can reprsoduce properties of neuron firing rates related Spitzer H, Otazu X and Hel-Or H to attention. In another article Banerjee et al., the authors propose a computational model, based (2020) Editorial: Integrating Visual System Mechanisms, Computational on the extreme value theory, for the integration of two sensory modalities, namely, the olfactory Models and Algorithms/Technologies. input and visual sensitivity of zebrafish. The authors show that the neural signals (pattern and rate Front. Bioeng. Biotechnol. 7:483. of neuronal firing) differ in their statistical fit when the signals are uni-modal (visual) or multi- doi: 10.3389/fbioe.2019.00483 modal (visual + olfaction). They further showed this by developing a Machine Learning based Frontiers in Bioengineering and Biotechnology | www.frontiersin.org 1 January 2020 | Volume 7 | Article 483 Spitzer et al. Visual System, Computational Models, Algorithms classifier that was able to successfully distinguish between these Finally, a selection of articles uses computational models to neural signals. This study forms a contribution to the intriguing predict and explain high level visual tasks, perceptual behavior, area of interactions between different sensory modalities. and visual phenomena. Some of these studies experiment with Two additional articles deal with the chromatic properties ambiguous stimuli and suggest explanations of visual system of the visual system as expressed in the retinal layer and mechanisms that contribute to the stabilization of the visually cortical layers. In Barkan and Spitzer, a computational model perceived display content. The article Cohen-Duwek and is presented which suggests an explanation of the underlying Spitzer, models the Filling-In phenomenon and, specifically, visual mechanisms for compensating chromatic aberrations. The the alternating effects in which the background of a stimulus computational model takes into account the spatio-chromatic may lead to two different types of perceived color: original or properties of the color-coded cells in the retina while taking complementary color. The model successfully predicts both into account the significance of the anatomical separation of effects through a heat diffusion function that is triggered by the Konio and Parvo chromatic pathways in the visual system. both the chromatic edges of the stimulus and the achromatic Furthermore, the model predicts the enigmatic phenomenon of remaining contours, in contrast to previous studies that use S-cone pattern reported by Shevell and Monnier. In a review the edges as blockers for diffusion and not as triggers. In article, by Patterson et al., the authors discuss the role of retinal another article Cohen-Duwek and Spitzer, a computational midget RGC cells and cortical double opponent cells in the model is presented that predicts spatial Filling-In effects such context of hue perception on one hand and spatial perception as the Watercolor illusion and the Cornsweet effects, that on the other. The authors present hypotheses that in some form have several chromatic edges. The model is based on the heat are not in accord with those supported by some other models diffusion equation where the scene gradients serve as heat including that of Barkan and Spitzer mentioned above. As usual sources. The model successfully predicts both the assimilative in Science, especially in neuroscience, conflicting results are and non-assimilative watercolor effects, as well as additional always an interesting source for promotion of discussion and Filling-In visual effects. The study thus supports the theory that comparison of opposite/different ideas. a shared visual mechanism is responsible (or partly responsible) Another group of studies develop computational models in for the vast variety of the “conflicting” filling-in phenomena. order to assist in understanding specific vision mechanisms. Two articles studied motion integration using bi-stable moving In Piu et al., the authors acquired experimental data and visual stimuli that can induce two different percepts (e.g., then performed statistical analysis on the data to obtain a coherent and transparent). In Li et al., a bi-stable moving visual representation of pupil size changes. They analyzed oscillatory stimuli of line segments was presented to participants and their dynamics of the pupil at rest by extracting features from the individual biases were modeled using a Bayesian modeling cross-recurrences of these oscillators as expressed in the power approach indicating a preference for one of the two possible spectrum. The authors state that their novel analysis approach interpretations of the scene. The authors found that increasing can form an adaptable diagnostic tool for identifying alertness density shows increasing bias in observers and that this effect and/or pathological status and thus might assist in clinical is greater in regular patterns than in irregular patterns. The assessments of pathologies associated with the autonomous authors tested a number of Bayesian models and show that nervous system. In Reynaud and Hess, the authors analyze a motion segregation prior best explains the interaction of their previously measured dataset and assess the visual disparity density and regularity observed in the collected experimental sensitivity of subjects across different spatial frequencies. The data. The authors suggest that bias is used by observers to computational factor in their study is the data analysis methods stabilize visual perception of the world. In the article Liu et al., in which they applied inter-correlations and factor analysis motion integration in normal observers was compared to on the data and found two spatial frequency channels for integration by observers with Anisometropic Amblyopia, a disparity sensitivity: one tuned to high spatial frequencies and neurodevelopmental disorder of the visual system. They showed one tuned to low spatial frequencies. The authors suggest that that when the stimuli contrast is reduced, the control observers this tuning of disparity channels could be important in computer exhibit a change in percept patterns, but amblyopic eyes do not. vision to design multi-scale stereo matching algorithms. In Using Baysian modeling, the authors show that indeed contrast Marić and Domijan, binary attention maps are modeled using a affects motion integration. Considering this together with the recurrent competitive network with excitatory-inhibitory nodes. modeling outcomes, the authors suggest that there is a different The model reproduces top-down mechanisms of attentions that motion coding mechanism in the amblyopic visual system. enhance perceived saliency of low-level features. The model is Finally, in Yankelovich and Spitzer, Boundary Completion was based on an extension of previously suggested Winner Take modeled, using a functional optimization approach in which All (WTA) choice models, and is inspired by neurological there is no need to extract different image features. The model components such as dendritic non-linearity that act on the evaluates several possible interpretations of the input and excitatory units and modulate synaptic transmission. The assigns a cost to each. The interpretation with minimal cost is model integrates a large set of data in visual attention and the model’s output. The model successfully predicts real and successfully predicts several attentional effects including the illusory contours. Additionally, for ambiguous stimulus, the ability to integrate information across space and time to form model is able to find multiple possible image interpretations, the intersection or union of two maps that are defined by which are ranked according to the probability they different features. are perceived. Frontiers in Bioengineering and Biotechnology | www.frontiersin.org 2 January 2020 | Volume 7 | Article 483 Spitzer et al. Visual System, Computational Models, Algorithms A different group of papers in this special issue, propose models can explain the vision system from the neuronal level practical algorithms and applications that were inspired by to the behavioral level providing understanding, and novel elements of the Human Visual System, or include components insights. On the other hand, the visual system provides ideas that do so. In Tsitiridis et al., the authors attempt to develop a and inspiration for the computational units and driving rules system to detect “Presentation Attacks” where a person’s image of the models. The interaction cycle continues with the design is illegally reproduced and used to abuse a biometric system. of practical algorithms and applications in the field of computer The authors develop a biologically-inspired presentation attack vision, that arise from the computational models and the ensuing detection model, based on features that mimic neurobiological understanding of the visual system. Some of the papers in processes in the human visual system. Machine learning tools this collection, even succeeded in achieving algorithms that are exploited to successfully predict whether incoming data perform on par with state-of-the-art capabilities, due to the is a spoofing-attack or is a legitimate image. In the article adoption of ideas from the visual systems. Other papers provide Paulun et al., a new system for dynamic visual recognition is inspiration for future possible algorithms to accomplish different introduced that combines bio-inspired sensor and hardware with visual tasks. a brain-like spiking neural network that mimics the layered Within this cycle of mutual contributions, we can learn some structure and the retinotopic organization of the retina and visual intriguing ideas and raise interesting questions. cortex. Following training, the network showed a very high object A recurring notion is the idea of the visual system providing classification accuracy. Finally, two papers in this group deal educated guesses on the visual scene, based on the visual with image quality and esthetics. In Martinez-Garcia et al., the input as well as on priors, and internal representations and authors address the important question of biased or imbalanced computations. Multi-stable inputs in the 3D world, occluded and datasets and their effect on quantitative modeling of the visual ambiguous scenes, allow several interpretations. However, these system. The authors show this in a specific case of layered retina- are processed by the visual system that considers the possible cortex models that learn to predict subjective quality ratings of interpretations and produces an “educated guess” as the best images. They show that the database under-represents certain explanation of the visual scene. Such a mechanism tends to lend stimuli (such as cross-masking between different frequencies) stability and consistency to our visual world. and thus the model trained on this database does not generalize An interesting insight that has been previously established, is well. The authors show that by augmenting the database with the importance of visual illusions as a basis for research on the synthetic examples, the model shows significant improvement visual system. As several of the articles in this issue have shown, in performance and generalization. The authors impress that illusions serve to mirror “errors” and “biases” of the visual system naturalistic databases should be combined with artificial stimuli as well as provide a window into the visual system’s mechanics via to improve model performance. visual perception. In the comprehensive review Brachmann and Redies, the Finally, we note that several of the articles introduce the authors describe the advances achieved by the Vision Science notion of aesthetics of the visual scene and raise the point and the Computer Vision communities in the parallel fields that beyond a comprehensive review, a small step has been of experimental visual aesthetics and computational visual taken toward the famous philosophical-psychophysical problem aesthetics. The paper highlights the similarities between the also regarding to visual aesthetics through the discussion of types of features exploited for these tasks by both communities originality and creativity. and the similarities between the quantitative tools used to analyze and define these features. The review covers models and AUTHOR CONTRIBUTIONS algorithms that supply prediction of ratings, style, and artist identification as well as computational methods in art history of All authors listed have made a substantial, direct and intellectual painting and photograph images. The review covers methods at contribution to the work, and approved it for publication. both sensorial (low-level bottom-up) and cognitive levels (high- levels), including modern methods of deep learning. In addition, Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a the review summarizes results from the field of experimental potential conflict of interest. aesthetics and deal with several specific image properties. The authors show that a close interaction between computational Copyright © 2020 Spitzer, Otazu and Hel-Or. This is an open-access article and experimental approaches are fundamental to answering distributed under the terms of the Creative Commons Attribution License (CC BY). difficult questions. The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original In this special issue, we have collected a variety of articles publication in this journal is cited, in accordance with accepted academic practice. that look at the intriguing cycle of: visual system, computational No use, distribution or reproduction is permitted which does not comply with these models, and applications. The studies show how computational terms. Frontiers in Bioengineering and Biotechnology | www.frontiersin.org 3 January 2020 | Volume 7 | Article 483