Reading 15: Color

The human eye is rather like a camera. So first, let’s look at a modern digital camera. They we can look at the human eye and see how it differs.

A Camera

A digital camera consists of a light-tight box with a flat photosensor, a lens, an iris and (for a still camera) a shutter. The photosensor has a regular array of **photosites* which convert light into electricity; the analog charge stored is amplified, measured with an analog to digital converter and sent to the image storage as a digital value for each photosite.

Sensor response

Source:

Nikon D700 study, MaxMax.com

Colored filters in front of each photosite mean that some react mainly to red wavelengths, some to green, and some to blue. The filters are designed so that all visible wavelengths are covered, and the overlap between filters is carefully contolled so that all colors can be well distinguished.

An iris diaphragm allows the amount of light to be adjusted, and a shutter opens for a brief period to allow the image to be captured at the right exposure. (For a video camera, the photosensor is constantly scanned for each frame).

A compound lens

Source:

Wikipedia, Photographic lens design

At the front, a compound lens gathers light from the scene and focusses it on the sensor. Multiple lens elements (some convex, some concave) made from glass with different refractive indices ensure that all wavelengths are focussed to a similar place. The position of some lens elements can be changed (by a motor, or by hand) to focus on nearby or far-away subjects as desired.

The interior of the camera is lined with matt black, light-absorbing material to cut down on internal reflections.

The Eye

Source: Lilley, Lin, Hewitt, & Howard, Colour in Computer Graphics, University of Manchester.

Here are key parts of the anatomy of the eye:

• The cornea is the transparent, curved membrane on the front of the eye.
• The aqueous humor fills the cavity between the cornea and the lens, and provides most of the optical power of the eye because of the large difference between its refractive index and the refractive index of the air outside the cornea.
• The iris is the colored part of the eye, which covers the lens. It is an opaque muscle, with a hole in the center called the pupil that lets light through to fall on the lens. The iris opens and closes the pupil depending on the intensity of light. It opens in dim light, and closes in bright light.
• The lens focuses light. Under muscle control, it can move forward and backward, and also get thinner or fatter to change its focal length.
• The retina is the surface of the inside of the eye, which is covered with light-sensitive receptor cells.
• The fovea is the spot where the optical axis (center of the lens) impinges on the retina. The highest density of photoreceptors can be found in the fovea. The fovea is the center of your visual field.
• The blind spot is where all the nerves gather to go off to the brain. There are no photoreceptors here. Normally, you don’t notice this.

• The retina is curved, not flat (this actually makes things easier, all parts are at a similar distance fromthe center of the lens. Curved photosensors are too hard to make, currently)
• The retina is not uniform; the central portion and the edges have different properties
• There is no compound lens. Just one simple convex lens (the aqueous humor) to do most of the focussing, and another simple convex lens to do fine focussing for close and far subjects. So not all wavelengths are focussed to the same point (the lens has chromatic aberration), as we will see later.

Cone responses

Source: Lilley, Lin, Hewitt, & Howard, Colour in Computer Graphics, University of Manchester.

There are two kinds of photoreceptor cells in the retina. Rods operate under low-light conditions - night vision. There is only one kind of rod, with one frequency response curve centered in green wavelengths, so rods don’t provide color vision. Rods saturate at moderate intensities of light, so they contribute little to daytime vision. Cones respond only in brighter light. There are three kinds of cones, called S, M, and L after the centers of their wavelength peaks. Short (S) cones have very weak frequency response centered in blue. Medium (M) and Long (L) cones are two orders of magnitude stronger, and their frequency response curves are almost the same.

Evidence from after-images, and linguistic analysis (you can have a yellowish-green but not a greenish-red, for example) indicated that we do not percieve the cone outputs directly. In 1905, Ewald Hering and Johannes von Kries described color vision as three pairs of opponent colors: black-white, red-green, and yellow-blue.

Ganglion cells

Source: Lilley, Lin, Hewitt, & Howard, Colour in Computer Graphics, University of Manchester.

This hypothesis was later confirmed experimentally. The rods and cones do not send their signals directly to the visual cortex. Instead, the signals are recombined into three channels by the retinal ganglion cells.

Brightness is produced by the M and L cones (and the rod)s. This is the only channel really active at night. The other two channels convey color differences, red-green and blue-yellow. For the red-green channel, for example, high responses mean red, and low responses indicate green.

We can use opponent colors in UI design: red and green are good contrasting colors because they drive the red-green channel to opposite extremes (for people with normal color vision; see below for an important issue though). Similarly, black/white and blue/yellow are good contrasting pairs.

Also, because the simple lenses in the eye are not well-corrected, it is better to have yellow for the foreground (graphics, text) and blue for the background. The blue seems ‘further away’. Blue text on a yellow background is harder to see, and produces more eye strain.

Color deficiency affects a significant fraction of human beings. Because it is a recessive trait and carried on the “X” chromosome, an overwhelming number of them are male (XY). Most of those affected have only minor problems, and it is often un-noticed until later in life. Atypical color response is a matter of degree (this is why the term “color blindness” is a poor one).

There are three kinds of color deficiency, which we can understand better now that we understand a little about the eye’s anatomy:

• Protanopia is missing or defective L cones. The consequence is reduced sensitivity to red-green differences (the L-M channel is weaker), and also reds are perceived as darker than normal.
• Deuteranopia is caused by missing or defective M cones. Red-green difference sensitivity is reduced, but reds do not appear darker.
• Tritanopia is caused by missing or defective S cones, and results in blue-yellow insensitivity.

Red/green deficiency affects about 8% of males and 0.4% of females. Blue/yellow deficiency is much much rarer.

Since color blindness affects so many people, it is essential to take it into account when you are deciding how to use color in a user interface. Don’t depend solely on color distinctions, particularly red-green distinctions, for conveying information. Microsoft Office applications fail in this respect: red wavy underlines indicate spelling errors, while identical green wavy underlines indicate grammar errors.

Traffic lights are another source of problems. How do red-green colorblind people know whether the light is green or red? Fortunately, there’s a spatial cue: red is always above (or to the right of) green. Protanopia sufferers (as opposed to deuteranopians) have an additional advantage: the red light looks darker than the green light.

Chromatic Aberration

• Different wavelengths focus differently

  • Highly separated wavelengths (red & blue) can’t be focused simultaneously

• Guideline: don’t use red-on-blue text

  • It looks fuzzy and hurts to read

The refractive index of the lens varies with the wavelength of the light passing through it. Just like a prism, different wavelengths are bent at different angles. So your eye needs to focus differently on red features than it does on blue features.

As a result, an edge between widely-separated wavelengths - like blue and red - simply can’t be focused. It always looks a little fuzzy. So red-on-blue or (worse!) blue-on-red text is painful to read, and should be avoided at all costs.

Apple’s ForceQuit tool in Mac OS X, which allows users to shut down misbehaving applications, unfortunately fell into this trap. In the dialog, unresponding applications are helpfully displayed in red. But the selection is a blue highlight. The result is incredibly hard to read.

A number of anatomical details conspire to make blue a bad color choice when small details matter.

First, as we have seen, the fovea has very few S cones, so you can’t easily see blue features in the center of your vision (unless they have high contrast with the background, activating the M and L cones).

Second, older eyes are far less sensitive to blue, because the lens and aqueous humor slowly grow yellower, filtering out the blue wavelengths.

Finally, the lens gets weaker with age. Blue is at one extreme of its focusing range, so older eyes can’t focus blue features as well (and can’t focus on different distances as well, either).

As a result, avoid blue text, particularly small blue text, and blue markers on line charts.

Source:

Sploid

These four squares are all actually moving at the same speed. Due to the low density of S cones in the retina, your eye has more trouble distinguishing fine detail in the blue from the black, so the blue squares appear to be moving in a jerky fashion.

Chromatic adaptation

No white balance

In the outside world, the color of the light changes constantly. Full sunshine is yellower than the light on a cloudy day; objects in shadow are illuminated with a bluer light; in a room lit with incandescent bulbs or candles the light is orange-yellow, and around sunset the light is strongly orange-red. The measured spectrum of light reflected from a given object is constantly changing. Despite this, we perceive that “the light changed” not “all the objects changed color”. Rather like a camera with auto white balance, our eye constantly recalibrates to what it sees as “white light”, and all other colors are judged relative to that white. This is called chromatic adaptation.

You can see this yourself with a sheet of white paper. Look at it indoors, in the evening with lights on; outside, shaded from direct sunlight. Tt still looks white. But if you take a picture (and have disabled auto white balance on the camera) you will see an orange sheet under tungsten light, and a cyan sheet in shadow, respectively.

Later in the reading we will examine the Lab color space, which takes chromatic adaptation into account.

Color Constancy

Color constancy

Color constancy is another phenomenon that can modify our color perceptions. We unconsciously apply our knowledge of the world to what we see.

You’ve probably seen this image before. Measured in isolation, the color of A and the color of B are identical, but our brain sees them as different because of the shadow. We perceive colors as constant, if we have reason to, even if they are under different light.

Strawberries

Source:

Akiyoshi Kitaoka,

Department of Psychology, Ritsumeikan University, Kyoto, Japan.

Despite the strange bluish lighting, we still see these as red strawberries. But there are no red pixels in the whole image. The strawberries are actually a neutral grey (#BCBCBC). The eye tries to adapt to the strongly cyan cast, and fails, but the state of partial chromatic adaptation, plus our memory of what strawberries look like, results in the sensation of redness.

Red-Green-Blue (RGB)

Now let’s look at how colors are represented in GUI software. At the lowest level, the RGB model rules. The RGB model is a unit cube, with (0,0,0) corresponding to black, (1, 1, 1) corresponding to white, and the three dimensions measuring levels of the red, green, and blue primaries. The RGB model is used directly by CRT and LCD screens for display, since each pixel in a screen has separate red, green, and blue components. Often, the values are given on a 0 to 255 (or 0x00 to 0xFF) scale, eight bits representing the full range for each color component, because the frame buffer in a graphics card is usually 8 bits deep.

By itself, RGB does not fully specify a color because there is no information on exactly what color the red, green and blue primaries are. sRGB aims to fix that by standardizing a common RGB color space. Colors on the Web are defined to be in sRGB. There are other RGB color spaces, such as AdobeRGB and ProPhoto (used in photography), and DCI P3 (used in digital cinema and 4k television).

Gamma Correction

Source:

Charles Poynton, Gamma FAQ

Also, because our eyes see intensity on a logarithmic scale, the RGB values are stored in gamma corrected form:

R gamma = R linear to the power γ, where γ is typically 2.4. (and the same for G and B)

Gamma correction is the process of standardizing the intensity so that a linear response is obtained.

This was originally done to match the characteristics of Cathode Ray Tube (CRT) displays; when the red component of an RGB value (0-1) is mapped directly to a voltage applied to an electron gun, the intensity of light produced does not vary linearly from 0 to 1, but typically follows a power curve. In addition, gamma correction makes the values more evenly spaced and thus makes the best use of the limited 8-bit depth. Although CRT screens are rarely used nowadays, for backwards compatibility, modern flat screens are designed to produce the same overall transfer function.

An unfortunate consequence of this is that, when mixing or interpolating colors in RGB, the values should really be un-gamma-corrected to linear light intensities, interpolated, and the results re-corrected. Most software does not bother to do this, producing a range of visual artifacts as a result.

Cyan-Magenta-Yellow-Black (CMYK)

CIE XTZ

The way that a color like pure red (RGB=1,0,0) actually looks depends strongly on the display’s characteristics, so your application can’t be sure it will get exactly the right color.

The science of colorimetry is concerned with accurate measurement and reproduction of color. Most of it is outside the scope of this course, but here are a few things you should know. Colorimetry stems from color matching experiments in the 1930s where a wide variety of colors had to be matched by varying the intensities of three colored lights. In any physical system, some colors will be out of gamut (not displayable); for example, many green and blue colors required a negative amount of the red light.

To get around that, the experimental data was transformed into a set of theoretical primary colors by the CIE (the International Committee on Illumination). The Y primary was defined to be the same as luminance (how light or dark something is) and the X and Z primaries defined to carry only color information, no luminance. X, Y, and Z are thus imaginary colors in the sense that they cannot be produced by a physical light source or perceived by the human eye. But they’re useful bases for the space, much like imaginary numbers are useful in algebra.

As a result, all human-visible colors are bounded within the positive octant – so that any visible color can be made as a mix of positive amounts of X, Y, and Z. The solid area on the left shows the visible colors perceivable to the human eye. Black is of course at the origin. To consider hue and saturation in isolation, we can also look at a plane of constant luminance, shown on the right. This ** chromaticity diagram** has a wedge-shaped slice showing the space of human-perceivable colors, with fully-saturated, pure-wavelength colors around its perimeter.

CIE XYZ is an additive, linear light colorspace useful for calculating color mixtures, but is not perceptually uniform. The distance apart of two colors in XYZ does not directly relate to how similar or different they appear. Luminance is accurately represented, but our eye sees lightness differences on a log scale. Chromatic adaptation is also not well shown in XYZ.

CIE Lab

Source: Chris Lilley

CIE Lab has a central Lightness axis, going from black at 0 to the current white at 100. All colors are chromatically adapted to a color temperature of 5000K (D50) so that measurements made under different conditions can be compared with each other. The two color axes, a and b represent opponent colors. Positive a is a reddish magenta, negative a is a teal green. Positive b represents yellow, while negative b is a sky blue.

The distance between two colors in Lab colorspace accurately predicts how similar or different they look.

The gamut of a display (or indeed, a printer or a camera) is most accurately represented as a volume in Lab space.

CIE LCH

Source: Chris Lilley

Instead of rectangular coordinates a and b, it is convenient instead to use a hue angle H (measured from the positive a axis) and a chroma, C (distance from the central Lightness axis, which is the same in Lab and LCH) which represents the colorfullness.

Chromaticity

A chromaticity diagram is a 2D slice of a 3D colorspace (CIE XYZ or CIE Luv), along a plane of constant luminance. So it shows only hue and saturation. The curved edge corresponds to the pure, single-wavelength colors of a spectrum. The straight line joining the two ends of the curve indicates purple (a mixture of red and violet light).

This is a comparison of the CIE xy chromaticity diagram on the top, and the CIE uv chromaticity diagram on the bottom

One drawback of xy is that it’s not perceptually uniform - green occupies a huge chunk at the top of the wedge, while yellow is a narrow little line. It would be preferable if the distance in color space produced the same difference in perception in both areas.

One of the goals of the uv chromaticity diagram was to have the sizes of each area better conform to our perceptual mappings.

Any given display device can produce a range of colors (called the device’s gamut), defined by three colors used by the device as its primary colors - e.g., the exact colors of red, green, and blue in a screen’s pixels - and the white point. The gamut can be represented in 2D on a chromaticity diagram, where it forms a triangle or (more accurately, and using computer graphics) as a 3D shape in Lab.

Devices with different gamuts will produce different colors from the same RGB value. This problem can be addressed by calibrating the display device, producing an ICC profile that specifies how the device’s RGB space maps into a standardized space like XYZ or Lab. (ICC is the International Color Consortium).

If a given color is out of gamut, calibration will not make it displayable. Calibration just gives color consistency for colors which in gamut.