Download Correlation Between Luminance and Colour Edges

2012
institute:
author:
mentor:
ETH Zürich; Institute for
Neuro-Informatics
Matej Znidaric
dr. Daniel C. Kiper
Correlation Between
Luminance and Colour
Edges in Natural Images
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
ABSTRACT
Organization of the visual system is still under debate, with two opposing groups. First supporting the
idea of segregated achromatic/luminance and chromatic visual pathways and the opposing idea
where both pathways maintain aggregated and processed dependently of each other. To get further
evidence, luminance and chromatic edges were exposed in natural images using two popular
software programs; Adobe Photoshop and MathWorks MatLab. High correlation between chromatic
and luminance edges was observed in both ‘Natural’ and ‘Manmade’ image set, This suggests that
the best strategy for the visual system would be to use cell populations that are simultaneously
selective to the orientation of both luminance and chromatic edges, thereby arguing against the
segregation hypothesis. In addition, comparing two image software tools resulted in low correlation
of edges in the same natural image due to unavailability of scientific background of Photoshop
algorithms.
1
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
INTRODUCTION
Light, Vision & Edges
Vision is one of the most crucial sensory systems in the lives of the vast majority of vertebrates. It
was shaped through evolution by the task we perform in order to survive by analysing natural scenes
from environment that surrounds us. Nearly a half of the entire human cerebral cortex is devoted to
process visual information, which shows how demanding and important the task is. Visual system is
able to detect and process only a specific portion of the electromagnetic radiation, specifically
between 390nm (violet colour) and 750nm (far red), called light.
Two distinct kinds of photoreceptors; rods and cones transform electromagnetic energy into
neuronal activity, which is than transmitted via different types of bipolar cells to ganglion cells
(Masland 2001). Quickly saturated rods are characterized with low noise, higher effectiveness and
lower resolution (Gary L. Savage 1991; D.M. Schneeweis 2000) for photon reception and are
consequently primarily used during the night, this kind of vision is also called scotopic vision (Gary L.
Savage 1991; Manning and Brainard 2009). Cones on the other hand can operate during the daylight
(D.M. Schneeweis 1999) but are noisier than rods (Fu, Kefalov et al. 2008) and have more complex
network of postsynaptic cells (Masland 2001). No matter of the relative number of each receptor
which varies from species to species (Walls 1942; Collin, Davies et al. 2009). Processing of the
acquired information starts already in the retina (Masland 2001) which could be looked at as a 5
layered structure, a developmental derivative of the brain (Nakamura, Igawa et al. 1986). Most
vertebrates also have different types of cones with different spectral sensitivities, which are basis for
perceiving colours – photopic vision (Dacey and Packer 2003). Humans and several other primates
have 3 different cone types referred as short-, middle- and long-wavelength sensitive (S, M, L)
(Nathans, Thomas et al. 1986; Collin, Davies et al. 2009), according to their relative position of a peak
sensitivity and are basis for human trichromatic vision (Stockman and Sharpe 2000; Ebrey and
Koutalos 2001; Masland 2001). Information gathered from three types of cones is then transformed
into achromatic and two opponent chromatic channels (Buchsbaum and Gottschalk 1983; Mullen
and Beaudot 2002; Clifford, Spehar et al. 2003). Achromatic channel underlies the basis for
perception of brightness which is a subjective measure of luminous intensity per unit area of light
travelling in specific direction, known as luminance. On the other hand two opponent chromatic
channels are responsible for colour perception. It is believed that cortical pathways maintain the
early segregation of luminance and chromatic channels (Livingstone 1990; Mante, Frazor et al. 2005).
Some oppose the idea of segregation of luminance and chromatic information and state that they
are coupled and linked throughout the visual processing pathway. Information from the retina is
transduced via lateral geniculate nucleus (LGN) to the primary visual cortex (also known as V1, striate
cortex or Broadmann area 17) (D.H.Hubel 1977). Visual transduction remains mainly unchanged to
the V1 area in the occipital lobe (Kuffler 1953; D.H.Hubel 1977), where most of the visual
computation takes place, including processing of edges (D.H.Hubel 1977; Livingstone 1990). Edge
detection is one of the independent mechanisms that concerns with detection of localization of
significant variations in luminance / chromatic changes (Ziou and Tabbone 1998). Such a cells, called
simple cells are characterized with their receptive fields which are most prominent to be stimulated
2
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
with rectangular – bar or edge shaped stimulus of various orientation and sizes (D. H. Hubel 1968;
Wiesel 1976; Vision 2002). Two distinct types of edges could be identified in the complex image;
luminance and chromatic edges. Difference in luminance between the neighbouring parts of the
image is the main focus in detecting luminance edges, while difference in chromatic information not
bound to luminance levels. Information is later subdivided to 2 streams with numerous areas in the
occipital and temporal lobe; dorsal or ‘where’ pathway and ventral or ‘what’ pathway (Goodale and
Milner 1992; Ganel and Goodale 2003). Both streams process more complicated information and are
not part of the discussion.
The problem
The visual system, its anatomical structure and processing characteristics are relatively well known,
however there are numerous opposing information about the organization of the luminance and
chromatic pathways. Even though the importance of colour in perception of natural images has been
described before (Wichmann, Sharpe et al. 2002; Clifford, Spehar et al. 2003), it is not known
whether the pathways for colour and luminance information remain segregated (Livingstone 1990;
Mante, Frazor et al. 2005; Johnson, Hawken et al. 2008) till the information is then finally integrated
in V1 (Thorell, De Valois et al. 1984; Johnson, Hawken et al. 2001; Sumner, Anderson et al. 2008) or
V2 region (Gegenfurtner, Kiper et al. 1996) of visual cortex. On the other hand claims that their
linkage remains constant throughout information processing steps (Switkes, Bradley et al. 1988;
Reisbeck and Gegenfurtner 1998; Mullen, Beaudot et al. 2000), is being supported by increasingly
more studies as well (Hamburger, Hansen et al. 2007). Though scientific data seem to stand behind
both groups, recent work of Hansen and Gegenfurtner on the independence of colour and luminance
edges in natural scenes (Hansen and Gegenfurtner 2009), where luminance and chromatic edges
were detected and correlated in the natural images, drew attention. In the publication, the authors
show that luminance and chromatic edges are independent in the natural scenes and provide
chromatic edges thus provide an independent source of information. (Hansen and Gegenfurtner
2009). Since the natural input images were highly processed to meet known processing steps in the
visual pathway, I developed the idea on a level of natural images themselves. Individual groups of
natural scenes – complex images (Shevell and Kingdom 2008) were correlated and luminance chromatic edges were compared; following the notion that correlation of edges differs in different
image sets. In case of high correlation between luminance and chromatic edges one could argue that
it is very likely that the same cells should be able to detect both types of edges and code for their
orientation. High correlation of luminance and chromatic edges would thus argue against the
segregation hypothesis. In contrast, if correlation between the luminance and chromatic edges is
low, it would make sense that the visual system has developed independent neuronal populations to
detect these two types of edges. In addition, extraction of edges from natural images was tested with
two popular software programs; image software program Adobe Photoshop and programming tool
MathWorks MatLab. Since the algorithms used in digital computing are very different from edge
detection in the brains, I checked the effectiveness of the programs in edge extraction and the
correlation of the edges extracted from the same image with mentioned programs was measured. As
the same basic operator for edge extraction was used in both programs (please refer to Method
section for further details), high correlation between the edges extracted in two programs was
expected.
3
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Sep. 2012
METHODS
Pictures were acquired from McGill calibrated pictures database which is available online at
http://pirsquared.org/research/mcgilldb/browsedownload.html. Only several out of 9 available
categories have been chosen, namely: Flowers, Foliage, LandWater, Textures, Man-made; from
which 2 distinct groups were created; Man-made (including: Man-made, textures) and Natural
(including: Flowers, Foliage and LandWater). Correlation between luminescence and colour edges
were than compared between the groups.
Acquiring and preparing the images
Images (natural scenes) were acquired from 2 online image databases. Majority of images were
downloaded from McGill database (Olmos and Kingdom 2004), which were taken with two Nikon
Coolpix 5700 digital cameras and cropped by the authors from original 1920 X 2560 pixel size to 786
X 576. Reference in the database points to calibration of images, following work of PhD thesis of
Parraga (Parraga C. 2003), but since only the edges on the natural images were compared, calibration
(calibration of the cameras used; estimation of the spectral sensitivity of the cameras sensors,
gamma-correction and calibration of the cameras colour space and conversion into L, M, S colour
space) was skipped. Downloaded images were in TIF file format with standard 72 dpi resolution and
joint into following image sets; ‘Animals’, ‘Flowers’, ‘Foliage’, ‘Fruit’, ‘LandWater’, ‘Man-Made’ and
‘Textures’. Additionally Outex Texture Database (Vision 2002) was used to gain another batch of
texture images; ‘Textures 2’. Images were originally in bmp file format in 100 dpi resolution and were
before any processing, transformed to 72 dpi tiff images. Combined, 1368 images were analysed and
luminance / chromatic edges were exposed and correlated. Two distinct image sets were prepared
from them; ‘Nature’ and ‘Man-made’. In first group images with natural subject of the image were
grouped. Animals, Flowers, Foliage, Fruit and LandWater image groups belonged into ‘Nature’ set
(648 images), while ManMade, Textures1 and Textures2 were composing the ‘Man-made’ image set
(720 images). Representative images from each category and number of pictures in each category is
presented on the figure on the next page.
4
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
Figure 1: Original input images acquired from McGill
database. Images from 7 categories; Animals, Flowers,
Foliage, Fruit, LandWater, ManMade and Textures1
and 2, were chosen for filtering and grouped into two
image sets. ‘Nature’ image sets shown on the left half
of the figure and ‘Man-made’ image set combined
from two categories shown on the right side. Number
of images in each category is written next to the title.
Image processing
Image processing and transformations took place in programming tool MathWorks MatLab R2012a
(v.7.14.0.739) and popular desktop image software program Adobe Photoshop CS6 for the
comparison.
In both cases the basis for edge detection was a discrete differentiation operator, Sobel operator. It
uses two 3x3 kernels, represented below; valued filter with which the original image (A) is convolved
in horizontal (Gx) and vertical direction (Gy). Result of the operator is the calculation of the gradient
5
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Sep. 2012
of the image intensity at specific point on the image which reliably predicts edges and their
orientation.
−1 0 1
𝐺𝑥 = �−2 0 2� ∗ 𝐴
−1 0 1
−1 −2 −1
𝐺𝑦 = � 0
0
0 �∗𝐴
1
2
1
Finding luminance edges
Luminance edges are edges due to variation of brightness between two neighbouring objects on the
image.
Adobe Photoshop
Luminance edges on the picture were exposed using Adobe Photoshop in 2 steps. Firstly colour
image was transformed to black-white one using “Black & White filter” (Image – Adjustments – Black
& White), discarding the chromatic information. Secondly, luminance edges were exposed using
predefined “Find Edges” filter from Photoshop filter gallery (Filter – Stylize – Find Edges). Process was
automatized using “actions” (automatic scripting) and run from Adobe Bridge.
Furthermore in order to standardize the output of the two programs images from MatLab had to be
inverted in colour space (edges in the output image were represented white on the black background
instead of black on the white background as in Adobe Photoshop). Inversion was done in Adobe
Photoshop with the help of “Invert” function (Image – Adjustments – Invert). This action was as well
automatized in Photoshop and run from Adobe Bridge.
MathWorks MatLab
In MatLab, luminance edges were found on the same way as in the Adobe Photoshop, using homewritten script (code is shown in the appendix section). Image was transformed into black and white
image and then Sobel operator, already incorporated in to the MatLab, was used to detect the
luminance edges on the picture. Because Sobel operator in MatLab returns the binary image of the
same size (where edges are found pixel got value of 1 and 0 is elsewhere). Output values were
normalized to 255 (maximal value), resulting in white edges on black background. Threshold was
chosen automatically by the function. Afterwards image was saved and process repeated for all the
images in the specified folder.
Finding chromatic edges
Different / unequal amounts of red, green and blue in the specific areas on the image creates
chromatic edges.
Adobe Photoshop
Exposure of chromatic edges was achieved using same filter as in luminance edges; “Find Edges”
(Filter – Stylize – Find Edges) which is based on Sobel operator working per channel (Adobe 2012),
consequently resulting in exposing the chromatic edges - edges for each channel; red, green and blue
separately. Process was automatized using actions and run from Adobe Bridge.
MathWorks MatLab
Process of defining chromatic edges using MatLab was a bit more complicated but it followed the
same protocol as in the Adobe Photoshop (script used is posted in the appendix). Image was loaded
and then divided to 3 basic RGB channels. Each ‘colour stack’ from original image was than
6
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Sep. 2012
separately analysed using same Sobel operator as before. After normalization the stacks were recomposed to the single output image where chromatic edges were shown.
Measurement of correlation
Correlation between images was set up in order to answer to 2 distinct questions; firstly, what is the
processing output of Adobe Photoshop compared to MathWorks MatLab in the same image set and
secondly, what is the correlation between chromatic and luminance edges on the pictures. Is the
correlation affected by the nature of the images (nature vs. manmade images)?
In both cases two images were compared using 2-D correlation coefficient in home-written MatLab
script. 2-D Correlation coefficient function in MatLab computes the coefficient between two matrices
or vectors of same size, using function represented below (MathWorks ; Ziou and Tabbone 1998):
𝑟=
∑𝑚 ∑𝑛(𝐴𝑚𝑛 − 𝐴̅)(𝐵𝑚𝑛 − 𝐵�)
�(∑𝑚 ∑𝑛(𝐴𝑚𝑛 − 𝐴̅)2 )(∑𝑚 ∑𝑛(𝐵𝑚𝑛 − 𝐵�)2 )
The return value is a scalar double (MathWorks), which was transformed to percentages if multiplied
with a 100.
Comparison between Photoshop and MatLab processed images
To compare image processed in 2 different programs (Adobe Photoshop and MathWorks MatLab)
correlation between same images processed in a different programs was created. Images processed
in MatLab were inverted in colour space as the output of the processing were white edges on dark
background as oppose to dark edges and white background in images processed in Photoshop.
Inversion was done in Adobe Photoshop with “Invert” function. Images with chromatic edges were
transformed to black and white images using “Black & White” filter in Adobe Photoshop with default
settings. Both actions were automatized and run from Adobe Bridge. Additionally, ‘Threshold’ filter
was applied to images processed with Photoshop, or Photoshop and MatLab, to compensate for
dynamic threshold which is pre-default setting for sobel operator used in MatLab. When threshold
effect was applied to images processed in Photoshop and MatLab, values of threshold were equal for
both programs (30/255 or 0.117). Threshold filter transforms all pixels lighter than threshold to
white and all pixels darker than threshold set to black and the result is a bivalent image.
MatLab script (whole script is in the appendix section) was written to compute the 2-D correlation
coefficient between them. Output in percentages was calculated from the correlation coefficient and
it directly reflects the proportion of the overlap between the images. For each set of images such a
correlation was established and output percentages were written in the separate file (appended).
Comparison between different sets of images processed in MatLab or Photoshop
Correlation between chromatic and luminance edges on the picture was computed and different
image sets namely; Flowers, Foliage, LandWater, Man-made and Textures were compared. Same
image with exposed luminance edges, processed in a specific program, was correlated with image
processed in the same program, showing chromatic edges. As the matrices of the two images had to
be the same size, image with chromatic edges had to be transformed to the black and white image,
which was done using “Black and White” filter in Adobe Photoshop. Percentages were written in
separate file and image sets were compared.
7
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
RESULTS
Image processing
Adobe Photoshop
Output images after the filtering process in Adobe Photoshop is represented with a few examples on
Figure 2 (luminance edges: Figure 2b; chromatic edges: Figure 2c). Algorithm for finding edges in the
picture used in Adobe Photoshop is very precise and it most probably it uses larger kernels / area to
compute the gradient of intensity at each pixel. Output of the ‘Find Edges’ filter in Adobe Photoshop
is not bivalent value but rather the spectrum of values and the objects on the pictures consequently
remain relatively well recognizable, also dimmer and less noticeable edges are preserved. This can be
especially well seen on the pictures with lot of small objects (example Figure 2: IIb, IIc, IVb, IVc). Also
the entire picture is brighter and the number of the edges discovered is remarkably high. Space
without noticeable edges becomes white. When chromatic edges were exposed, the edges remain
coloured (Figure 2c) but the details remain prominent as in luminance edges.
8
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
Figure 2: Original input picture (a) is
presented along image with exposed
luminance edges (b) and image with
exposed chromatic edges (c). Rows
marked with roman numbers represent
from which category image was taken
from. Random picture from each group
used in the analysis is represented on
the figure; I: Animals, II: Flowers, III:
Foliage, IV: Fruit, V: LandWater, VI:
ManMade, VII: Textures 1 and VIII:
Textures 2. Number of detected edges
was high as well as the details
preserved on the image. In case of
exposed luminance as well as chromatic
edges, Photoshop output was in whole
spectrum of values.
To transform processed images into the images with bivalent values (black or white) the ‘Threshold’
filter was used. With filter bivalent output of the Sobel operator used in MatLab was compensated
for. Images with exposed luminance edges and with threshold filter applied are shown on Figure 3b
and images with exposed chromatic edges on Figure 3c.
9
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
Figure 3: Downloaded picture – original
file (a). Image with exposed luminance
edges, inverted in colour space and with
applied ‘Threshold’ filter (b). (c) column
represents same images as in (b) with
exposed chromatic edges. Nature of the
‘Threshold’ filter returns black and white
image so chromatic information is lost.
Rows marked with roman numbers
represent from which category image was
taken from. Random picture from each
group used in the analysis is represented
on the figure; I: Animals, II: Flowers, III:
Foliage, IV: Fruit, V: LandWater, VI:
ManMade, VII: Textures 1 and VIII:
Textures 2..
MathWorks MatLab
Images were processed with home-written script that was based on Sobel operator. Output
of the operator is a bivalent value which results in images composed out of pixels with
bivalent values; black & white in case, where luminance edges were exposed, or
representative colours in case of chromatic edges; they were not represented with spectrum
of the colours but rather with defined ‘value’ of the colour. Consequently, objects were less
10
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
recognizable and especially on the parts of the less contrast parts of the image, edges could
have been missed. Background without edges became black.
Figure 4: Downloaded picture – original
file (a). Picture in the middle (b) shows
luminance edges where natural image
was first transformed to black-white
color space and then edges were
exposed using 'Sobel operator' filter.
Chromatic edges on the picture were
exposed using ‘Sobel operator’ function
individually for each channel (c). Rows
marked with roman numbers represent
from which category image was taken
from. Random picture from each group
used in the analysis is represented on
the figure; I: Animals, II: Flowers, III:
Foliage, IV: Fruit, V: LandWater, VI: ManMade, VII: Textures 1 and VIII: Textures
2..
11
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Sep. 2012
CORRELATION RESULTS
Comparison between Photoshop and MatLab processed and correlated images
with exposed luminance edges
Correlation between luminance edges in images processed in two programs was low, as shown in
Table 1. Table represents pooled data, from comparisons of all images, no matter from which image
set they originate. With the mean value of just above 50% (Table 1a) the correlation is low, which is
due to much more sophisticated algorithm used in Adobe Photoshop, compared with home-written
MatLab script which used simple Sobel operator to expose edges on the picture. Images compared
vary a lot in their contents, as a result standard deviation is relatively high; 13,93 (table 1a). After
‘Threshold’ filter was applied to images processed in Photoshop, mean value of the image correlation
remained nearly the same (Table 1b), while standard deviation rose to 19,59 (Table 1b). When
specific threshold was applied to images processed in either Photoshop or MatLab, mean correlation
jumped for almost 20%; from 50,90% to 68,06% (Table 1c). Standard deviation on other hand
dropped to 6,89 (Table 1c), indicating higher correlation between images.
Mean (%)
Standard deviation
a) No threshold
50,90 %
13,93
b) PS Threshold
49,60 %
19,59
c) Both Thresholds
68,06 %
6,89
Table 1: Correlation of images processed in Photoshop or MatLab with exposed luminance edges. Mean correlation value
and standard deviation is presented for images processed without specified threshold (a), where only Photoshop
processed images had defined threshold value (b) and when images processed with either Photoshop or MatLab had
defined same value for threshold (c).
Correlation data for each image set in presented in the Table 3. Correlation in different image sets is
quite variable, most correlated images were in Manmade set (59,77% mean correlation) while the
least correlated are LandWater and Textures, all below 50% (47,24 for LandWater and 49,79%
Textures 1 and 43,60% for Textures 2 respectively). Standard deviations vary extensively as well and
they reach a peak in LandWater set of images with 21,43. Explanation for large standard deviation in
LandWater and Texture sets is that pictures in these areas are most diverse. Surprisingly low
standard deviation is in Manmade images, nevertheless the images in this set are extremely diverse,
standard deviation is only 11,9. Very low standard deviation was observed In Animals and Fruit image
sets (below 6%), both with relatively small number of images. Threshold filtered image sets remained
with negligible changes both to mean values and standard deviation (Table 4). An exception was
Texture 2 image set, where Threshold effect dramatically lowered mean correlation (form 43,6 to
24,29%).
LandWate Manmade Textures 1 Textures 2
r
49,54
50,87
51,23
51,89
47,24
59,77
49,79
43,60
Mean (%)
5,45
10,99
13,84
5,58
21,43
11,97
19,95
7,76
S.D.
Table 2: Correlation between Photoshop and MatLab processed images, with exposed luminance edges. Correlation
between images in either image set is between 40 and 60% and it varies extensively. No threshold value was specified
for images processed in either Photoshop or MatLab.
Mean (%)
S.D.
Animals
Flowers
Foliage
Fruit
Animals
Flowers
Foliage
Fruit
54,57
15,47
50,60
11,66
52,79
13,63
47,43
14,33
12
LandWate
r
45,49
22,62
Manmade
59,86
12,78
Textures
48,46
19,76
Textures 2
24,29
22,39
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Sep. 2012
Table 3: Images with exposed luminance edges, processed with MatLab or Photoshop were correlated. Photoshop
images were further processed with ‘Threshold’ filter to gain bivalent image, as it is the output image processed in
MatLab. Mean correlation and standard deviation were not changed significantly.
Animals
Flowers
Foliage
Fruit
LandWate Manmade Textures
Textures 2
r
72,41
61,96
66,30
67,77
68,42
68,40
67,33
72,62
Mean (%)
4,10
3,37
4,82
4,26
8,24
4,02
5,98
11,71
S.D.
Table 4: Images with exposed luminance edges were correlated. Mean value and standard deviation is presented for
images that were processed in Photoshop or MatLab. In both cases same value of threshold was applied. Mean value is
between 60% and 75% and standard deviation 3 and 12.
Comparison between Photoshop and MatLab processed and correlated images
with exposed chromatic edges
Correlation of images with exposed chromatic edges is as in case of luminance edges, low.
Correlation between images from all image sets is 53,3% (Table 5a), which is similar compared with
correlation of images with exposed luminance edges. Standard deviation is smaller than in luminance
edges (9,96 compared to 13,93 in luminance edges). Furthermore, when ‘Threshold’ filter was
applied to images processed in Photoshop the mean correlation value was not changed significantly
(Table 5b), but standard deviation rose from 9,96 to 21,08, indicating that the threshold filter
resulted higher variability between the images. In the case when same threshold value was applied
to images processed in either of the programs, mean correlation rose to 68,8%. In comparison with
mean correlation of images with exposed luminance edges the results are very similar, 68,8% and
68,0% respectively. Standard deviation was not significantly changed.
Mean (%)
Standard deviation
a) No Threshold
53,34 %
9,96
b) PS Threshold
51,80 %
21,08
c) Both Thresholds
68,83 %
12,93
Table 5: Correlation of Photoshop and MatLab processed images with exposed chromatic edges. Mean correlation value
and standard deviation is presented for images processed without specified threshold (a), where only Photoshop
processed images had defined threshold value (b) and when images processed with either Photoshop or MatLab had
defined same value for threshold (c). Latest processing resulted in highest correlation results.
If we leave image sets segregated (Table 6), we see that overall the mean values are higher
compared with correlation of luminance edges. LandWater which was one of the least correlated
image sets (47,2%) in the luminance edges is now the most correlated image set (60,8%). Also
standard deviation which was the highest in the luminance edges (21,4) is now lowest (8,4). Changes
between luminance and chromatic edge correlation in other image sets are less noticeable. On the
other hand, mean value of the image set ‘Textures’ is very similar in luminance edges (49,79%) and
chromatic edges (49,4%). After ‘Threshold’ filter was applied to images processed in Photoshop
(Table 7), mean correlation stayed in the same range as before (between 40 and 60%) but standard
deviation rose, indicating higher variability between the images. An exception is again Texture 2
image set, where mean correlation was only 29,1%. Furthermore, correlation was higher (between
61% and 74%) in case images were processed in with same threshold value (Table 8).
Animals
Flowers
Foliage
Fruit
LandWate Manmade
Textures
Texture 2
r
56,14
54,82
49,36
58,10
60,81
55,42
49,44
51,95
Mean (%)
5,66
7,41
10,06
6,39
8,39
10,98
10,91
8,32
S.D.
Table 6: Correlation of images processed in MatLab or Photoshop, with exposed chromatic edges. Mean values and
standard deviations are presented for each image set individually. No definite threshold value was applied. Mean
correlation value and standard deviation varies between 49% and 60% and between 5 and 11 respectively.
13
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Animals
Flowers
Foliage
Fruit
Animals
Flowers
Foliage
Fruit
Sep. 2012
LandWate Manmade
Textures
Texture 2
r
58,56
50,85
56,89
46,76
46,88
60,06
51,48
29,11
Mean (%)
16,22
14,29
13,67
14,99
24,60
17,45
20,28
25,57
S.D.
Table 7: Mean correlation and standard deviation of images with exposed chromatic edges, processed in either
Photoshop or MatLab. Threshold was applied to images processed in Photoshop but no significant changes in mean
correlation values were observed. Standard deviation is higher and varies between 13 and 25.
LandWate Manmade
Textures
Texture 2
r
74,10
63,03
70,45
61,65
69,60
69,71
72,70
64,59
Mean (%)
5,12
9,65
7,11
12,58
16,03
14,62
9,17
18,03
S.D.
Table 8: Mean correlation and standard deviation of images with exposed chromatic edges, processed in either
Photoshop or MatLab. Same threshold value was applied to images processed in both programs, which resulted in higher
mean correlation values (from 63% to 74%). Standard deviation varies extensively (between 5 and 18) depending on the
image set.
Correlation between images with exposed luminance and chromatic edges for
images processed in MathWorks MatLab
Images with exposed luminance edges correlate highly (above 85% for pooled data) with images with
exposed chromatic edges (Table 9). Processing was in both cases done in MatLab and in order to
compare images, chromatic edges had to be transformed to black-white image. In the transformation
chromatic information about the edges was lost, but edges were still based on chromatic information
from the picture. High correlation is also accompanied with low standard deviation (8,08) which
further points to high correlativity of the images.
Mean (%)
Standard deviation
85,42 %
8,08
Table 9: correlation between luminance and chromatic edges processed with MatLab. Mean value of the correlation is
high (85,42%) and standard deviation is low (8,08).
Analysis of the individual image sets (Table 10) reveal higher correlation of the ‘Manmade’ (88,69%)
and ‘Textures’ (87,33%) image sets, but the differences between the sets are minimal. Standard
deviation of the ‘LandWater’ image set, with 13,05 pops out from the otherwise more monotone
values between 4,7 – 10 for the rest of the image sets.
Animals
Flowers
Foliage
Fruit
LandWater
Manmade
Textures
Textures 2
Mean (%)
S.D.
87,34
83,54
86,71
83,97
82,51
88,69
87,33
81,73
5,51
5,09
4,80
4,78
13,05
4,76
7,03
10,66
Table 10: Correlation between luminance and chromatic edges processed with MatLab segregated by individual image
sets.
Furthermore, when we combine images into 2 distinguishable sets; ‘Nature’; containing ‘Animals’,
‘Flowers’, ‘Foliage’, ‘Fruit’ and ‘ LandWater’ and ‘Manmade’; containing ‘Manmade’, ‘Textures1’ and
‘Textures2’ the difference becomes even less prominent (85,42% for ‘Nature’ and 85,43%
‘Manmade’) as shown in Table 11 and Graph 1.
Nature
Mean (%)
Standard Deviation
85,42
7,19
Manmade
85,43
8,80
Table 11: correlation between luminance and chromatic edges in images processed with MatLab. Under 'Nature' there
are 5 image sets; ‘Animals’, 'Flowers', 'Foliage', ‘Fruit’ and 'LandWater' and under 'Manmade' there are 3; 'Manmade',
'Textures1' and ‘Textures 2’.
14
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Correlation between images with exposed
luminance and chromatic edges processed in MatLab
correlation (%)
100
Nature
Manmade
90
80
70
60
50
Sep. 2012
Graph 1: Correlation between luminance
and chromatic edges on images processed
in MatLab, grouped into 2 sets; 'Nature' and
'Manmade'. Correlation in both image sets
is high (above 85%); Considering standard
deviation (7,19 for ‘Nature’ and 8,80 for
‘Manmade’) the difference is negligible.
image set
Correlation between images with exposed luminance and chromatic edges for
images processed in Adobe Photoshop
Correlation between luminance and chromatic edges on pictures processed in Adobe Photoshop are
high (Table 12). Mean value is even higher compared to correlation of pictures processed in MatLab,
above 90% (specifically: 93,61%) and standard deviation still remains low (8,07), which indicates to
higher correlation of luminance and chromatic edges on the pictures processed in Photoshop than
pictures processed in MatLab.
Mean (%)
Standard deviation
93,61 %
8,07
Table 12: Correlation of chromatic and luminance edges in the pictures processed in Adobe Photoshop. Mean value and
standard deviation.
Correlation of each image set, as shown in Table 13, remains similar compared with MatLab
processing. Correlation of each image set is still above 90%, the highest value in Textures1
(95,86%) and Animals (95,63%) image sets. Standard deviation is extremely low, ranging
from 2,82 (Animals) and 3,46 (Foliage) to 11,05 (LandWater), which could point to lower
variability in images processed in Photoshop.
Animals
Flowers
Foliage
Fruit
LandWater Manmade
Textures
Textures 2
95,63
92,34
94,63
91,93
92,02
94,63
95,86
91,64
Mean (%)
2,82
3,83
3,46
4,07
11,05
9,55
5,71
10,47
S.D.
Table 13: Correlation between images with exposed luminance and chromatic edges processed with Photoshop for each
image set individually.
When image sets are combined into ‘Nature’ and ‘Manmade’ as described in previous section,
differences between 2 sets is negligible (93,78% for ‘Nature’ and 93,45% for ‘Manmade’). Results are
displayed in Table 14 and Graph 2.
Nature
Mean (%)
Standard Deviation
93,78
5,66
Manmade
93,45
9,74
Table 2: correlation between luminance and chromatic edges in images processed with Photoshop. Under 'Nature' there
are 3 image sets; 'Flowers', 'Foliage', 'LandWater' and under 'Manmade' there are 2; 'Manmade' and 'Textures'.
Correlation between images with exposed
luminance and chromatic edges processed in Photoshop
Nature
Manmade
correlation (%)
100
90
80
70
60
50
image set
Graph 2: Correlation between luminance and
chromatic edges on images processed in
Photoshop, grouped into 2 sets; 'Nature' and
'Manmade'. Both correlations are high (above
90%) and standard deviation below 10.
‘Manmade’ image set is slightly higher but the
there are no significant differences.
15
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
DISCUSION & CONCLUSION
Two main questions, highlighted in the introduction, were answered in the project; Firstly I
compared two different software tools; image processing software Adobe Photoshop and
programming tool MathWorks MatLab. Secondly look into luminance and chromatic edges in natural
images and define what the correlation between them is.
Photoshop and MatLab
Comparison in edge extraction between two programs, namely Adobe Photoshop and MathWorks
MatLab was made. Main advantages of Adobe Photoshop are its simplicity of usage which does not
require knowledge of any programming language and relatively good processing capabilities for large
number of pictures (actions). In addition, it uses powerful algorithms for image filtering and
processing, which have proven to be more precise and sensitive compared to the simple Sobel
operators run from MatLab (Table 12 compared with Table 9). On the other hand the commercial
nature of image processing software like Adobe Photoshop leads to the main problem and practical
non-usability for scientific work, like image filtering and processing. Not only the user has no control
of any kind over the algorithms used in the functions, there are practically no references about
scientific / mathematical background or its implementation in to the algorithms. In contrast, MatLab
is a programming tool where references for every function are provided and modifiable. Output of
edge detection function was also different compared with Photoshop, where output of the pixel is a
range of values which results in detail preservation and easier object recognizing compared to image
filtered in MatLab (Figure 2b, compared to Figure 4b). Correlation between images filtered in MatLab
and Photoshop is consequently surprisingly very low, just above 50% (Table 1 and Table 5). The major
reason for such low correlation is difference in algorithms used, nevertheless they base on same
edge extraction operator. When ‘Threshold’ effect was applied on the images filtered in Photoshop
to compensate for bivalent output of the Sobel operator used in MatLab, the results are still
inconvenient and just slightly higher mean correlation was observed (Table 1b and Table 5b, Table 2
and Table 6). On the other hand wider range of mean correlation values was observed, resulting in
higher standard deviation. In some cases threshold filter reduce the difference between the bivalent
output of the MatLab processing and Photoshop processing in other cases the difference was
enhanced. Explanation for the effects lies in Sobel operator used in MatLab where threshold is
calculated and applied automatically to individual image. Furthermore, when images were processed
with same static threshold value, mean correlation rose for nearly 20% (Table 1c and Table 5c, Table
4 and Table 8), clearly indicating the importance of threshold in edge detection algorithms. In
addition, standard deviation was lower, pointing to less variability between images processed in
MatLab and Photoshop than before. Nevertheless, highest mean correlation between images is still
around 70%, which is lower than I expected.
In conclusion, Adobe Photoshop and MathWorks MatLab software correlate poorly in edge
detection, no matter that the same basic operator was used. MatLab remains better option for edge
extraction just because user has control over the algorithms and functions used, despite Photoshop
produced better results with a few clicks.
16
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
Luminance and chromatic edges
Correlation between luminance and chromatic edges was high in both cases; when images
where processed in MatLab (above 85%; Table 5) or in Photoshop (above 93%; Table 8),
indicating that information about luminosity is the most important component of the edges
in the natural image. Furthermore, standard deviation of correlation in case of comparison
luminance edges was smaller in both cases, compared to images with extracted chromatic
edges. Correlation of images belonging to ‘Man-made’ group was identical or extremely
close compared to images from ‘Natural’ image set (table 7 and 10), the difference is
negligible and it falls well into the area of standard deviation, concluding that there is no
noticeable difference between luminance and chromatic edge correlation in ‘Man-made’
and ‘Natural’ image sets. In terms of physiology, high correlation between the edges shows
supports the notion that luminance and color should be treated by the same cells. This
argues indirectly against the notion that two separate systems, one dedicated to luminance
and one to color would be an efficient way to process natural images.
17
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
RESOURCES
Adobe (2012). "Find Edges filter." Retrieved 7 June 2012, from http://forums.adobe.com/message/4473809.
Buchsbaum, G. and A. Gottschalk (1983). "Trichromacy, opponent colours coding and optimum colour
information transmission in the retina." Proc R Soc Lond B Biol Sci 220(1218): 89-113.
Clifford, C. W., B. Spehar, et al. (2003). "Interactions between color and luminance in the perception of
orientation." J Vis 3(2): 106-115.
Collin, S. P., W. L. Davies, et al. (2009). "The evolution of early vertebrate photoreceptors." Philosophical
Transactions of the Royal Society B: Biological Sciences 364(1531): 2925-2940.
D. H. Hubel, T. N. W. (1968). "Receptive fields and functional architecture of monkey striate cortex." The
Journal of Physiology 195: 215-243.
D.H.Hubel, T. N. W. (1977). "Ferrier lecture: Functional Architecture of Macaque Monkey Visual Cortex."
Biological Sciences 198.
D.M. Schneeweis, J. L. S. (1999). "The Photovoltage of macaque Cone Photoreceptors: Adaptation, Noise and
Kinetics." The Journal of Neuroscience: 1203-1216.
D.M. Schneeweis, J. L. S. (2000). "Noise and light adaptation of the macaque monkey." Visual Neuroscience
17(5): 659-666.
Dacey, D. M. and O. S. Packer (2003). "Colour coding in the primate retina: diverse cell types and cone-specific
circuitry." Curr Opin Neurobiol 13(4): 421-427.
Ebrey, T. and Y. Koutalos (2001). "Vertebrate photoreceptors." Prog Retin Eye Res 20(1): 49-94.
Fu, Y., V. Kefalov, et al. (2008). "Quantal noise from human red cone pigment." Nat Neurosci 11(5): 565-571.
Ganel, T. and M. A. Goodale (2003). "Visual control of action but not perception requires analytical processing
of object shape." Nature 426(6967): 664-667.
Gary L. Savage, M. S. B. (1991). "Scotopic Visual Efficiency: Constraints by optics, receptor properties and rod
pooling." Visual Research 32: 645-656.
Gegenfurtner, K. R., D. C. Kiper, et al. (1996). "Processing of color, form, and motion in macaque area V2." Vis
Neurosci 13(1): 161-172.
Goodale, M. A. and A. D. Milner (1992). "Separate visual pathways for perception and action." Trends in
Neurosciences 15(1): 20-25.
Hamburger, K., T. Hansen, et al. (2007). "Geometric-optical illusions at isoluminance." Vision Res 47(26): 32763285.
Hansen, T. and K. R. Gegenfurtner (2009). "Independence of color and luminance edges in natural scenes." Vis
Neurosci 26(1): 35-49.
Johnson, E. N., M. J. Hawken, et al. (2001). "The spatial transformation of color in the primary visual cortex of
the macaque monkey." Nat Neurosci 4(4): 409-416.
18
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
Johnson, E. N., M. J. Hawken, et al. (2008). "The orientation selectivity of color-responsive neurons in macaque
V1." J Neurosci 28(32): 8096-8106.
Kuffler, S. (1953). "Discharge patterns and functional organization of mammalian retina." Journal of
neurophysiology 16(1).
Livingstone, M. (1990). "Segregation of form, color, movement, and depth processing in the visual system:
anatomy, physiology, art, and illusion." Res Publ Assoc Res Nerv Ment Dis 67: 119-138.
Manning, J. R. and D. H. Brainard (2009). "Optimal design of photoreceptor mosaics: why we do not see color at
night." Vis Neurosci 26(1): 5-19.
Mante, V., R. A. Frazor, et al. (2005). "Independence of luminance and contrast in natural scenes and in the
early visual system." Nat Neurosci 8(12): 1690-1697.
Masland, R. H. (2001). "The Fundamental plan of the retina." Nature Neuroscience 4(9).
MathWorks. "2-D Correlation coefficient." Retrieved 25. August, 2012, from
http://www.mathworks.com/help/toolbox/images/ref/corr2.html.
Mullen, K. T. and W. H. Beaudot (2002). "Comparison of color and luminance vision on a global shape
discrimination task." Vision Res 42(5): 565-575.
Mullen, K. T., W. H. Beaudot, et al. (2000). "Contour integration in color vision: a common process for the blueyellow, red-green and luminance mechanisms?" Vision Res 40(6): 639-655.
Nakamura, H., H. H. Igawa, et al. (1986). "Analysis of brain development by quail-chick chimera. I. Capacity of
prosencephalon or rhombencephalon to differentiate into optic tectum." Neuroscience Research Supplements
3(0): S25.
Nathans, J., D. Thomas, et al. (1986). "Molecular genetics of human color vision: the genes encoding blue,
green, and red pigments." Science 232(4747): 193-202.
Olmos, A. and F. A. A. Kingdom (2004). "A biologically inspired algorithm for the recovery of shading and
reflectance images." Perception 33(12): 1463-1473.
Parraga C., A. (2003). Is the Human Visual System Optimised for Encoding the Statistical Information of Natural
Scenes?
Reisbeck, T. E. and K. R. Gegenfurtner (1998). "Effects of contrast and temporal frequency on orientation
discrimination for luminance and isoluminant stimuli." Vision Res 38(8): 1105-1117.
Shevell, S. K. and F. A. A. Kingdom (2008). "Color in Complex Scenes." Annual Review of Psychology 59(1): 143166.
Stockman, A. and L. T. Sharpe (2000). "The spectral sensitivities of the middle- and long-wavelength-sensitive
cones derived from measurements in observers of known genotype." Vision Res 40(13): 1711-1737.
Sumner, P., E. J. Anderson, et al. (2008). "Combined orientation and colour information in human V1 for both LM and S-cone chromatic axes." Neuroimage 39(2): 814-824.
Switkes, E., A. Bradley, et al. (1988). "Contrast dependence and mechanisms of masking interactions among
chromatic and luminance gratings." J Opt Soc Am A 5(7): 1149-1162.
Thorell, L. G., R. L. De Valois, et al. (1984). "Spatial mapping of monkey V1 cells with pure color and luminance
stimuli." Vision Res 24(7): 751-769.
19
Matej Žnidarič
ETH Zürich
Correlation Between Luminance and
Colour Edges in Natural Images
Sep. 2012
Vision, U. o. O. C. f. M. (2002). "Outex Texture Database." from
http://www.outex.oulu.fi/index.php?page=image_database.
Walls, G. L. (1942). The vertebrate eye and its adaptive radiation, Bloomfield Hills, Mich., Cranbrook Institute of
Science.
Wichmann, F. A., L. T. Sharpe, et al. (2002). "The contributions of color to recognition memory for natural
scenes." J Exp Psychol Learn Mem Cogn 28(3): 509-520.
Wiesel, D. H. H. A. T. N. (1976). "Functional architecture of macaque monkey visual cortex." Biological Sciences
198: 1-59.
Ziou, D. and S. Tabbone (1998). "Edge Detection Techniques-An Overview." Pattern Recognition & Image
Analysis 8(4): 537-559.
20
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Sep. 2012
APPENDIX
MathWorks MatLab functions
Luminance Edges
function luminance_edges()
cd 'images98_ML_LU';
number_of_files_in_data = size(dir,1)-2;
for i=1:number_of_files_in_data
name = strcat('1 (',int2str(i));
name = strcat(name,').tif');
A = imread(name);
Get to the folder of interest
Count all the pictures in it (-2 files;
name of the folder and position)
For each pictures name, transform
the image into black-white
Convolve the black-white image
using sobel operator and normalize
to 255. (sobel operator returns 0 / 1
value).
CB = rgb2gray(A);
cb = edge(CB,'sobel')*255;
imwrite(cb, name, 'tif');
Save the new picture.
end
cd '..';
end
Chromatic Edges
function chromatic_edges()
cd 'images98_ML_CH';
number_of_files_in_data = size(dir,1)-2;
for i=1:number_of_files_in_data
name = strcat('1 (',int2str(i));
name = strcat(name,').tif');
A = imread(name);
Ar = A(:,:,1);
Ag = A(:,:,2);
Ab = A(:,:,3);
Er = edge(Ar,'sobel')*255;
Eg = edge(Ag,'sobel')*255;
Eb = edge(Ab,'sobel')*255;
Get to the folder of interest
Count all the pictures in it (-2 files;
name of the folder and position)
For each pictures name, transform
the image 3 parts according to RGB
spectrum; blue, red, green part.
Convolve each part with sobel
operator and normalize to 255
(sobel operator returns 0 / 1 value).
Merge all 3 parts
A(:,:,1) = Er;
A(:,:,2) = Eg;
A(:,:,3) = Eb;
Save the new picture.
imwrite(A, name, 'tif')
end
cd '..';
end
21
Correlation Between Luminance and
Colour Edges in Natural Images
Matej Žnidarič
ETH Zürich
Sep. 2012
Correlation
Comparison between 2 images with the same names, saved in pre-defined folders.
function Correlation()
For number of pictures in the
folder
N = number_of_pictures_in_the_folder;
for i=1 : N
folder_slike_matlab
= './folder_path1/';
folder_slike_photoshop = './folder_path2/';
name1 = strcat(folder_slike_matlab, '1 (' );
temp1 = strcat(int2str(i), ')' );
name1 = strcat(name1, temp1);
slika1 = imread( name1, 'tif' );
name2
=
strcat(folder_slike_photoshop,'1
);
temp2 = strcat(int2str(i), ')' );
name2 = strcat(name2, temp2);
slika2 = imread( name2, 'tif' );
temp = corr2(slika1,slika2);
korelacijski_koeficijent(i) = temp;
('
Open a picture and the
pictures
from
separate
folder with the same name.
Than callculate the 2-D
correlation coefficient and
save it in to the file.
In the separate file save the
coefficients multiplied with
100.
end
save('save_file1.txt',
'korelacijski_koeficijent', '-ascii');
procenti = korelacijski_koeficijent*100;
save('save_file2.txt', 'procenti', '-ascii');
end
Correlation results
Correlation between Photoshop and MatLab processed images with exposed luminance edges: “photoshop matlab - luminance edges.txt”
Correlation between Photoshop and MatLab processed images with exposed luminance edges and threshold
effect applied on Photoshop images: “photoshop - matlab - luminance edges - threshold.txt”
Correlation between Photoshop and MatLab processed images with exposed luminance edges and threshold
effect applied: “photoshop - matlab - luminance edges – threshold - threshold.txt”
Correlation between Photoshop and MatLab processed images with exposed chromatic edges: “photoshop matlab - chromatic edges.txt”
Correlation between Photoshop and MatLab processed images with exposed chromatic edges and threshold
effect applied on Photoshop images: “photoshop - matlab - chromatic edges - threshold.txt”
Correlation between Photoshop and MatLab processed images with exposed chromatic edges and threshold
effect applied: “photoshop - matlab - chromatic edges – threshold - threshold.txt”
Correlation between luminance and chromatic images processed in MatLab: “MatLab.txt”
Correlation between luminance and chromatic images processed in Photoshop: “Photoshop.txt”
22