Image processing

Publications related to image processing.

  1. T. Lourens and E. I. Barakova. Orientation Contrast Sensitive Cells in Primate V1 -a computational model [607 KB pdf]. Natural Computing, 6(3):241-252, September 2007.

    Abstract

    Many cells in the primary visual cortex respond differently when a stimulus is placed outside their classical receptive field (CRF) compared to the stimulus within the CRF alone, permitting integration of information at early levels in the visual processing stream that may play a key role in intermediate-level visual tasks, such a perceptual pop-out (Knierim and van Essen, 1992; Nothdurft et al., 1999), contextual modulation (Levitt and Lund, 1997; Das and Gilbert, 1999; Dragoi and Sur, 2000), and junction detection (Sillito et al., 1995; Das and Gilbert, 1999; Jones et al., 2002). In this paper we construct a computational model in programming environment TiViPE (Lourens, 2004) of orientation contrast type of cells and demonstrate that the model closely resembles the functional behavior of the neuronal responses of non orientation (within the CRF) sensitive 4CB cells (Jones et al., 2002), and give an explanation of the indirect information flow in V1 that explains the behavior of orientation contrast sensitivity. The computational model of orientation contrast cells demonstrates excitatory responses at edges near junctions that might facilitate junction detection, but the model does not reveal perceptual pop-out.

  2. T. Lourens and E. I. Barakova. Simulation of Orientation Contrast Sensitive Cell Behavior in TiViPE [455 KB pdf]. In J. Mira and J. R. Alvarez, editors, IWINAC 2005, number 3561 in Lecture Notes in Computer Science, pages 57-66, Las Palmas de Gran Canaria, Spain, June 2005. Springer-verlag.

    Abstract

    Many cells in the primary visual cortex respond differently when a stimulus is placed outside their classical receptive field (CRF) compared to the stimulus within the CRF alone, permitting integration of information at early levels in the visual processing stream that may play a key role in intermediate-level visual tasks, such a perceptual popout [11], contextual modulation [7, 3, 4], and junction detection [13, 3, 5]. In this paper we construct a computational model in programming environment TiViPE [9] of orientation contrast type of cells and demonstrate that the model closely resembles the functional behavior of the neuronal responses of non orientation (within the CRF) sensitive 4CB cells [5], and give an explanation of the indirect information flow in V1 that explains the behavior of orientation contrast sensitivity.

  3. T. Lourens and E. I. Barakova. TiViPE Simulation of a Cortical Crossing Cell Model. [1786 KB pdf]. In J. Cabastany, A. Prieto, and D. F. Sandoval, editors, IWANN 2005, number 3512 in Lecture Notes in Computer Science, pages 122-129, Barcelona, Spain, June 2005. Springer-verlag.

    Abstract

    Many cells in cat and monkey visual cortex (area V1 and area 17) respond to gratings and bar patterns of different orientation between center and surround [18]. It has been shown that these cells respond on average 3.3 times stronger to a crossing pattern than to a single bar [16]. In this paper a computational model for a group of neurons that respond solely to crossing patterns is proposed, and has been implemented in visual programming environment TiViPE [10]. Simulations show that the operator responds very accurately to crossing patterns that have an angular difference between 2 bars of 40 degrees or more, the operator responds appropriately to bar widths that are bound by 50 to 200 percent of the preferred bar width and is insensitive to non-uniform illumination conditions, which appear to be consistent with the experimental results.

  4. T. Lourens, E. I. Barakova, H. G. Okuno, and H. Tsujino. A computational model of monkey cortical grating cells [478 KB pdf]. Biological Cybernetics, 92(1):61-70, January 2005. DOI: 10.1007/s00422-004-0522-2.

    Abstract

    Grating cells were discovered in the V1 and V2 areas of the monkey visual cortex by von der Heydt et al. (1992). These cells responded vigorously to grating patterns of appropriate orientation and periodicity. Computational models inspired by these findings were used as texture operator (Kruizinga and Petkov 1995, 1999; Petkov and Kruizinga 1997) and for the emergence and self-organization of grating cells (Brunner et al. 1998; Bauer et al. 1999). The aim of this paper is to create a grating cell operator G that demonstrates similar responses to monkey grating cells by applying operator G to the same stimuli as in the experiments carried out by von der Heydt et al. (1992). Operator G will be tested on images that contain periodic patterns as suggested by De Valois and De Valois (1988). In order to learn more about the role of grating cells in natural vision, operator G is applied to 338 real-world images of textures obtained from three different databases. The results suggest that grating cells respond strongly to regular alternating periodic patterns of a certain orientation. Such patterns are common in images of human-made structures, like buildings, fabrics, and tiles, and to regular natural periodic patterns, which are relatively rare in nature.

  5. T. Lourens and R. P. Wurtz. Extraction and matching of symbolic contour graphs [3770 KB pdf]. International Journal of Pattern Recognition and Artificial Intelligence (IJPRAI), 17(7):1279-1302, November 2003.

    Abstract

    We describe an object recognition system based on symbolic contour graphs. The image to be analyzed is transformed into a graph with object corners as vertices and connecting contours as edges. Image corners are determined using a robust multiscale corner detector. Edges are constructed by line-following between corners based on evidence from the multiscale Gabor wavelet transform. Model matching is done by finding subgraph isomorphisms in the image graph. The complexity of the algorithm is reduced by labeling vertices and edges, whereby the choice of labels also makes the recognition system invariant under translation, rotation, and scaling. We provide experimental evidence and theoretical arguments that the matching complexity is below O(#V3), and show that the system is competitive with other graph-based matching systems.

  6. T. Lourens and E. I. Barakova. Real Time Object Recognition in a Dynamic Environment -An application for soccer playing robots [1203 KB pdf]. In R. P. Wurtz and M. Lappe, editors, 4th Workshop on Dynamic Perception, pages 189-194, Bochum, Germany, November 2002. IOS press.

    Abstract

    In dynamic environments, such as RoboCup [4], vision systems play a crucial role. In general, systems requiring real-time vision are either implemented in hardware, or as software systems that take advantage of the domain specific knowledge to attain the necessary efficiency. The goal of this paper is to describe a vision system that is able to reliably detect objects in real time and that is robust under different lighting conditions, this in contrast to most models used in robot soccer. The resulting objects serve as input for intelligent prediction of robot behavior [1].

  7. T. Lourens, H. G. Okuno, and H. Kitano. Automatic Graph Extraction from Color Images [1694 KB pdf]. In E. Ardizzone and V. Di Gisu, editors, Proceedings of the 11th International Conference on Image Analysis and Processing, ICIAP 2001, pages 302-308, Palermo, Italy, September 2001.

    Abstract

    An approach to symbolic contour extraction will be described that consists of three stages: enhancement, detection, and extraction of contours and corners. Contours and corners are enhanced by models of monkey cortical complex and endstopped cells. Detection of corners and local contour maxima is performed by selection of local maxima in both contour and corner enhanced images. These maxima form the anchor points of a greedy contour following algorithm that extracts the contours. This algorithm is based on an idea of spatially linking neurons along a contour that will fire in synchrony to indicate an extracted contour. The extracted contours and detected corners represent the symbolic representation of the image. The advantage of the proposed model over other models is that the same low constant thresholds for corner and local contour maxima detection are used for different images. Closed contours are guaranteed by the contour following algorithm to yield a fully symbolic representation which is more suitable for reasoning and recognition. In this respect our methodology is unique, and clearly different from the standard (edge) contour detection methods. The results of the extracted contours (when displayed as being detected) show similar or better results compared to the SUSAN and Canny-CSS detectors.

  8. T. Lourens, H. G. Okuno, and H. Kitano. Detection of Oriented Repetitive Alternating Patterns in Color Images -A Computational Model of Monkey Grating Cells [5531 KB pdf]. In J. Mira, editor, Proceedings of the International Workshop on Artificial Neural Networks, IWANN 2001, volume 1676, Part I of Lecture Notes in Computer Science, pages 95-107, Granada, Spain, June 2001. Springer-Verlag.

    Abstract

    In 1992 neurophysiologists [20] found a new type of cells in areas V1 and V2 of the monkey primary visual cortex, which they called grating cells. These cells respond vigorously to a grating pattern of appropriate orientation and periodicity. Three years later a computational model inspired by these findings was published [9]. The study of this paper is to create a grating cell operator that has similar response profiles as monkey grating cells have. Three different databases containing a total of 338 real world images of textures are applied to the new operator to get better a insight to which natural patterns grating cells respond. Based on these images, our findings are that grating cells respond best to repetitive alternating patterns of a specific orientation. These patterns are in common human made structures, like buildings, fabrics, and tiles.

  9. T. Lourens, K. Nakadai, H. G. Okuno, and H. Kitano. Graph extraction from color images [149 KB pdf]. In Proceedings of the 9th European Symposium on Artificial Neural Networks, ESANN 2001, pages 329-334, Brugge, Belgium, April 2001.

    Abstract

    An approach to symbolic contour extraction will be described that consists of three stages: enhancement, detection, and extraction of edges and corners. Edges and corners are enhanced by models of monkey cortical complex and end-stopped cells. Detection of corners and local edge maxima is performed by selection of local maxima in both edge and corner enhanced images. These maxima form the anchor points of a greedy contour following algorithm that extracts the edges. This algorithm is based on an idea of spatially linking neurons along the edge that will fire in synchrony to indicate an extracted edge. The extracted edges and detected corners represent the symbolic representation of the image. The advantage of the proposed model over other models is that the same low constant thresholds for corner and local edge maxima detection are used for different images. Closed contours are guaranteed by the contour following algorithm to yield a fully symbolic representation which is more suitable for reasoning and recognition. In this respect our methodology is unique, and clearly different from the standard edge detection methods.

  10. T. Lourens, K. Nakadai, H. G. Okuno, and H. Kitano. A Computational Model of Monkey Grating Cells for Oriented Repetitive Alternating Patterns [192 KB pdf]. In Proceedings of the 9th European Symposium on Artificial Neural Networks, ESANN 2001, pages 315-322, Brugge, Belgium, April 2001.

    Abstract

    In 1992 neurophysiologists [5] found an new type of cells in areas V1 and V2 of the monkey primary visual cortex, which they called grating cells. These cells respond vigorously to a grating pattern of appropriate orientation and periodicity. A few years later a computational model inspired by these findings was published [3]. The study of this paper is to model a grating cell operator that responds in a very similar way as these grating cells do. Three different databases containing a total of 338 real world images of textures were applied to the operator. Based on these images, our findings were that grating cells respond best to repetitive alternating patterns of a specific orientation. These patterns are mostly human made structures, like buildings, fabrics, and tiles.

  11. T. Lourens, K. Nakadai, H. G. Okuno, and H. Kitano. Selective Attention by Integration of Vision and Audition [3897 KB pdf]. In Proceedings of the First IEEE-RAS International Conference on Humanoid Robots, Humanoids2000, pages 20, file: 44.pdf, The Massachusetts Institute of Technology, Boston, U.S.A., September 2000.

    Abstract

    Selective attention is one of the tasks humans solve with great ease, still in computer simulations of human cognition this is a very complicated problem. In humanoid research it even becomes more complicated due to physical restrictions of hardware. Compared to a human, camera’s, e.g., have small visual fields and low resolution while motion causes a lot of noise, which makes audition a more complicated task. Combining vision and audition in humanoids is beneficial for both cues: vision because it does not suffer from noise, while audition is not restricted to an approximately 40o x 40o receptive field area, neither to partly or fully occluded objects. Low localization accuracy of both human (±8o) and artificial (±10o) audition systems can be compensated for by using vision. In this paper we propose a model that simulates selective attention by integrating vision and audition. A learning mechanism is incorporated as well to make the model adaptive to any arbitrary scene. The input of the model is formed by specific and robust features that are extracted from a huge amount of sensor data, hence part of the paper will focus on feature extraction. Audition is employed to improve selective attention because objects can be occluded or outside the visual field of a camera or human vision. Visual fields can be made wider by lenses, but never reach the full 360 degrees, hence a map is needed. This map contains information about all recognized objects over time, where objects are represented by features in a symbolic description. This map, in fact, represents a kind of artificial (temporal) memory. The location information of the objects (given by real world coordinates) is stored in the map as well. Also features from both vision and audition cues are integrated in this map. Storing information over time in such a map facilitates and speeds up the selective attention model. The map can be easily extended to incorporate extracted features from other types of sensors. In a simple natural environment the functionality of the model as well as the symbiosis between vision and audition are illustrated. The scenario will show that interaction between vision and audition is beneficial which is found rarely in literature. Promising results of the scenario show that audition was needed to localize an initially invisible object, while vision after that was used to accurately localize the object.

  12. R. P. Wurtz and T. Lourens. Corner detection in color images through a multiscale combination of end-stopped cortical cells. [1127 KB pdf]. Image and Vision Computing, 18(6-7):531-541, April 2000.

    Abstract

    We assess the corner-detection capabilities of a model for end-stopped cells in the visual cortex (F. Heitger, L. Rosenthaler, R. von der Heydt, E. Peterhans, O. Kubler, Simulation of neural contour mechanisms: from simple to end-stopped cells, Vision Research 32(5) (1992) 963–981). The responses of one of these cells alone cannot account for the percept of a corner. This shortcoming can be greatly alleviated by a combination over several scales. The resulting corner detection method can deal with high frequency texture, low contrast, and rounded corners and is competitive in comparison with other corner detectors. Starting from known cortical cell types we hypothesize a colorsensitive equivalent of simple cells. This allows to extend corner detection to color-sensitive channels. The combination of grey-scale and color corner-detection yields a biologically plausible model of corner perception and may also be of interest for computer vision applications.

  13. T. Lourens. A Biologically Plausible Model for Corner-based Object Recognition from Color Images [5296 KB pdf]. Shaker Publishing B.V., Maastricht, The Netherlands, March 1998.

    1.1 Outline of the thesis

    The flow of visual information via the M or magno-cellular (from magnus = large) pathway is basically used to order the chapters in the thesis, with one exception: Chapter 7 is about color which belongs to the P or parvo-cellular (from parvus = small) pathway.
    We know that our visual system detects and locates corners and edges accurately. In Chapter 2 we take a closer view of the main flow of visual information from eye to primary visual cortex. This information flow is known as early vision.
    Chapter 3 gives a general overview of the most widely used models for different cells in early vision. The modeled properties of most of these cells form the basis for our approach of artificial vision. We model the so-called center-surround, simple, and complex cell types. In the visual system these cell types can differ in their spatial (and temporal) resolution. Some cells only respond to very small parts of the visual field and are involved with highly detailed vision, while other cells respond to large parts of the visual field. These cells interact at different levels of accuracy (scales). Interaction at different scales raises the questions which scales are useful and how these scales can be ordered. In natural vision systems the spatial accuracy drops with eccentricity, but in most static artificial systems accuracy is uniform.We do pay attention to this phenomenon, however, for the sake of completeness. The decrease in spatial accuracy with eccentricity highly reduces the amount of information. A reduction of information is important since it makes the system relatively fast when motion is involved. Motion is not included in this thesis but it will be added to the model in future research.
    In Chapter 4 a corner operator based on responses of cortical end-stopped cells proposed by Heitger et al. [71] will be modeled (with some minor but important changes) and compared to six standard corner enhancing operators. We are aiming at a robust operator with respect to the position under different conditions (e.g. rotation of the image), since it forms the basis for the graph. Rotation and position at different scales are analyzed to determine the robustness of the operator.
    Chapter 5 improves the end-stopped operator by using multiple scales. This is necessary since the operator is noise sensitive at small scales and does not respond to all different corners at a single scale. The response of the operator will be examined for different scales at convex corners, rounded corners, and several junctions.We discuss the choice for a proper operator to combine multiple scales and motivate the choice for the number of scales. Finally the multiple scale corner operator will be compared with standard operators at both single and multiple scales.
    Chapter 6 describes a line-segment extraction algorithm where the corners obtained with the end-stopped operator at multiple scales are used together with the “edge enhanced image”. Edges between a pair of corner points will be extracted only, hence edges without two corner points are not detected. The content of this chapter is meant as an intermediate step towards a graph representation and should be regarded as preprocessing for graph matching (see Chapter 8).
    Chapter 7 extends the model with color. We use two “color-opponent” channels which are found in natural color vision. In previous chapters we gave a model which is based on achromatic vision. In this chapter we use this model but apply it to two different opponent channels. Hence models for biologically plausible color-opponent cells are proposed: one opponent cell type which responds to edges of a preferred orientation and a type which responds to corners. In natural vision two opponent color channels are found; in combination with the achromatic channel, every color can be reconstructed. The three channels are combined to yield the final edge and corner detection model.
    Chapter 8 gives a graph matching algorithm which searches copies of different known objects in the input graph. The representation of these objects accomplishes scale, rotation, and translation invariance. The matching algorithm is based on a standard back-tracking algorithm which is a time consuming (NP-hard) problem. Hence angle and length ratio attributes which are found in every two-dimensional graph are added to speed up the search. Appendix A describes some functions used in linear filtering, such as the Gaussian, Laplacian of a Gaussian, and the difference of Gaussians. This appendix aims at the reader who is interested in the differences and properties of these functions.
    Parts of Chapter 3 and Appendix A have been published in [120, 121], parts of Chapters 4, 5, and 7 in [198], and parts of Chapters 6 and 8 in [122].

    1.2 Contributions

    In this section we give the contributions of this thesis.

    1. An overview of early vision from a computational point of view is given. With this overview an artificial vision system based on line and corner enhancement can be constructed (Chapter 3).
    2. The corner detecting qualities of the model of end-stopped cells, proposed by Heitger et al. [71], are assessed (Chapter 4).
    3. We propose a new corner detector by a multi-scale combination of the modeled end-stopped cells (Chapter 5), which yields:
      • a physiological model for the percept of a corner and
      • a useful corner operator for computer vision.
    4. Edge, and corner enhancement algorithms are generalized to color channels. We use the properties of the complex and end-stopped cells and assume that these cells respond excitatory to one color and inhibitory to another color (Chapter 6).
    5. We develop a line detection algorithm, based on the assumption that corners are more stable than lines, and use it to extract line-segments by following enhanced edges from one corner to another (Chapter 7).
    6. We develop an attributed graph format for views of objects, which is suited for objects in which all edges are spanned by corners.
    7. A graph matching algorithm will be used for object recognition (Chapter 8), where the choice of attributes leads to:
      • invariance under translation, rotation and scaling,
      • robustness under small perspective changes and undetected lines, and
      • reduction of evaluations from N! to less than N3.

    Points 4-6 apply also to physiologically motivated color channels and complete color images. This is still done rarely in computer vision.

  14. R. P. Wurtz and T. Lourens. Corner detection in color images by multiscale combination of end-stopped cortical cells [634 KB pdf]. In W. Gerstner, A. Germond, M. Hasler, and J. D. Nicoud, editors, Proceedings of the International Conference on Artificial Neural Networks, ICANN’97, volume 1327 of Lecture Notes in Computer Science, pages 901-906. Springer Verlag, October 1997.

    Abstract

    We present a corner-detection algorithm based on a model for end-stopping cells in the visual cortex. Shortcomings of this model are overcome by a combination over several scales. The notion of an end-stopped cell and the resulting corner detector is generalized to color channels in a biologically plausible way. The resulting corner detection method yields good results in the presence of high frequency texture, noise, varying contrast, and rounded corners. This compares favorably with known corner detectors.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.