color imaging - fundamentals and applications - e. reinhard, et al., (a k peters, 2008)

✐
✐
✐
✐
✐
✐
✐
✐
Color Imaging
✐
✐
✐
✐
✐
✐
✐
✐
Color Imaging
Fundamentals and Applications
Erik Reinhard
Erum Arif Khan
Ahmet O
˘
guz Aky
¨
uz
Garrett Johnson
A K Peters, Ltd.
Wellesley, Massachusetts
✐

✐
✐
✐
✐
✐
✐
✐
Editorial, Sales, and Customer Service Office
A K Peters, Ltd.
888 Worcester Street, Suite 230
Wellesley, MA 02482
www.akpeters.com
Copyright 2008 by A K Peters, Ltd.
All rights reserved. No part of the material protected by this copyright notice may
be reproduced or utilized in any form, electronic or mechanical, including photo-
copying, recording, or by any information storage and retrieval system, without
written permission from the copyright owner.
Library of Congress Cataloging-in-Publication Data
Reinhard, Erik, 1968–
Color imaging : fundamentals and applications / Erik Reinhard [etal.].
p. cm.
Includes bibliographical references and index.
ISBN: 978-1-56881-344-8 (alk. paper)
1. Computer vision. 2. Image processing. 3. Color display systems. 4. Color
separation. I. Title.
TA1634.R45 2007
621.36’7 dc22
2007015704
Printed in India
1211100908 10987654321

✐
✐
✐
✐
✐
✐
✐
✐
Contents
Preface xiii
I Principles 1
1 Introduction 3
1.1 Color in Nature . . . 4
1.2 Color in Society . . . 10
1.3 In this Book 12
1.4 Further Reading . . . 14
2 Physics of Light 17
2.1 Electromagnetic Theory 18
2.2 Waves 28
2.3 Polarization 38
2.4 Spectral Irradiance . 45
2.5 Reflection and Refraction . . . 47
2.6 Birefringence 63
2.7 Interference and Diffraction . . 66
2.8 Scattering . 78
2.9 Geometrical Optics . 84
2.10 Application: Image Synthesis . 96
2.11 Application: Modeling the Atmosphere 104
2.12 Summary . 119

2.13 Further Reading . . . 120
3 Chemistry of Matter 121
3.1 Classical Physics . . 122
3.2 Quantum Mechanics 124
3.3 Atoms and Ions . . . 138
v
✐
✐
✐
✐
✐
✐
✐
✐
vi Contents
3.4 Molecules 144
3.5 Sources of Radiation 159
3.6 Polarization in Dielectric Materials . . 182
3.7 Dichroism 190
3.8 Application: Modeling of Fire and Flames . . 191
3.9 Further Reading . . 197
4 Human Vision 199
4.1 Osteology of the Skull . . . 200
4.2 Anatomy of the Eye 201
4.3 The Retina 212
4.4 The Lateral Geniculate Nucleus 228
4.5 The Visual Cortex . 230
4.6 A Multi-Stage Color Model . 237
4.7 Alternative Theory of Color Vision . . 245
4.8 Application: Modeling a Human Retina . . . 247

4.9 Further Reading . . 250
5 Perception 251
5.1 Lightness, Brightness, and Related Definitions 252
5.2 Reflectance and Illumination 254
5.3 Models of Color Processing . 256
5.4 Visual Illusions . . 259
5.5 Adaptation and Sensitivity . 270
5.6 Visual Acuity . . . 279
5.7 Simultaneous Contrast . . . 282
5.8 Lightness Constancy 286
5.9 Color Constancy . . 295
5.10 Category-Based Processing . 298
5.11 Color Anomalies . 302
5.12 Application: Shadow Removal from Images . 309
5.13 Application: Graphical Design 312
5.14 Application: Telling Humans and Computers Apart . . . . . 314
5.15 Further Reading . . 314
II Color Models 317
6 Radiometry and Photometry 319
6.1 The Sensitivity of the Human Eye . . 320
6.2 Radiometric and Photometric Quantities . . . 322
6.3 The Efficacy of Optical Radiation . . 337
6.4 Luminance, Brightness, and Contrast . 340
✐
✐
✐
✐
✐
✐
✐

✐
Contents vii
6.5 Optical Detectors . . 342
6.6 Light Standards . . . 345
6.7 Detector Standards . 346
6.8 Measurement of Optical Radiation . . . 347
6.9 Visual Photometry . . 356
6.10 Application: Measuring Materials . . . 359
6.11 Further Reading . . . 362
7 Colorimetry 363
7.1 Grassmann’s Laws . 364
7.2 Visual Color Matching 366
7.3 Color-Matching Functions . . 373
7.4 CIE 1931 and 1964 Standard Observers 375
7.5 Calculating Tristimulus Values and Chromaticities 378
7.6 Practical Applications of Colorimetry . 387
7.7 Application: Iso-Luminant Color Maps 397
7.8 Further Reading . . . 403
8 Color Spaces 405
8.1 RGB Color Spaces . 411
8.2 Printers . . 418
8.3 Luminance-Chrominance Color Spaces 427
8.4 Television and Video 430
8.5 Hue-Saturation-Lightness Spaces 439
8.6 HVS Derived Color Spaces . . 444
8.7 Color Opponent Spaces 448
8.8 Color Difference Metrics . . . 459
8.9 Color Order Systems 465
8.10 Application: Color Transfer between Images . . . 467
8.11 Application: Color-to-Gray Conversion 474

8.12 Application: Rendering 478
8.13 Application: Rendering and Color-Matching Paints 480
8.14 Application: Classification of Edges . . 484
8.15 Further Reading . . . 490
9 Illuminants 491
9.1 CIE Standard Illuminants and Sources . 491
9.2 Color Temperature . 503
9.3 Color-Rendering Index 508
9.4 CIE Metamerism Index 512
9.5 Dominant Wavelength 514
9.6 Excitation Purity . . 517
9.7 Colorimetric Purity . 517
✐
✐
✐
✐
✐
✐
✐
✐
viii Contents
9.8 Application: Modeling Light-Emitting Diodes 518
9.9 Application: Estimating the Illuminant in an Image . . . . . 520
9.10 Further Reading . . 524
10 Chromatic Adaptation 525
10.1 Changes in Illumination . . . 526
10.2 Measuring Chromatic Adaptation . . . 530
10.3 Mechanisms of Chromatic Adaptation 532
10.4 Models of Chromatic Adaptation . . . 538
10.5 Application: Transforming sRGB Colors to D50 for an ICC

Workflow 553
10.6 Application: White Balancing a Digital Camera 555
10.7 Application: Color-Accurate Rendering . . . 562
10.8 Further Reading . . 564
11 Color and Image Appearance Models 565
11.1 Vocabulary 566
11.2 Color Appearance Phenomena 582
11.3 Color Appearance Modeling 591
11.4 Image Appearance Modeling 605
11.5 Applications of Color and Image Appearance Models . . . . 620
11.6 Further Reading . . 629
III Digital Color Imaging 631
12 Image Capture 633
12.1 Optical Image Formation . . 635
12.2 Lenses . . 649
12.3 Aberrations 654
12.4 The Diaphragm . . 667
12.5 The Shutter 668
12.6 Filters and Coatings 669
12.7 Solid-State Sensors 672
12.8 In-Camera Signal Processing 678
12.9 A Camera Model . 682
12.10 Sensor Noise Characteristics 683
12.11 Measuring Camera Noise . . 688
12.12 Radiometric Camera Calibration . . . 694
12.13 Light Field Data . . 697
12.14 Holography 701
12.15 Further Reading . . 706
✐
✐

✐
✐
✐
✐
✐
✐
Contents ix
13 High Dynamic Range Image Capture 709
13.1 Multi-Exposure Techniques . . 710
13.2 Response Curve Recovery . . 715
13.3 Noise Removal . . . 722
13.4 Ghost Removal . . . 726
13.5 Image Alignment . . 733
13.6 Single Capture High Dynamic Range Images . . 734
13.7 Direct High Dynamic Range Capture . . 737
13.8 Application: Drawing Programs 739
13.9 Application: Image-Based Material Editing . . . 740
13.10 Further Reading . . . 741
14 Display Technologies 743
14.1 Cathode-Ray Tubes (CRTs) . . 743
14.2 Liquid Crystal Displays (LCDs) 746
14.3 Transflective Liquid Crystal Displays . . 767
14.4 Plasma Display Panels (PDPs) 768
14.5 Light-Emitting Diode (LED) Displays . 770
14.6 Organic Light-Emitting Diode Displays 772
14.7 Field Emission Displays . . . 775
14.8 Surface-Conduction Electron-Emitter Displays . 776
14.9 Microcavity Plasma Devices . 777
14.10 Interferometric Modulator (IMOD) Displays . . . 777
14.11 Projection Displays . 779

14.12 Liquid Crystal Display (LCD) Projectors 781
14.13 Digital Light Processing (DLP
R

) Projectors . . . 782
14.14 Liquid Crystal on Silicon (LCoS) Projectors . . . 785
14.15 Multi-Primary Display Devices 787
14.16 High Dynamic Range Display Devices . 791
14.17 Electronic Ink 794
14.18 Display Characterization . . . 794
14.19 Further Reading . . . 803
15 Image Properties and Image Display 805
15.1 Natural Image Statistics 806
15.2 Dynamic Range . . . 816
15.3 Cross-Media Display 827
15.4 Gamut Mapping . . . 833
15.5 Gamma Correction . 841
15.6 Ambient Light 843
✐
✐
✐
✐
✐
✐
✐
✐
x Contents
16 Color Management 849
16.1 A Generic Color Management System 849
16.2 ICC Color Management . . . 851

16.3 Practical Applications 874
17 Dynamic Range Reduction 881
17.1 Spatial Operators . 885
17.2 Sigmoidal Compression . . . 888
17.3 Local Neighborhoods 892
17.4 Sub-Band Systems 895
17.5 Edge-Preserving Smoothing Operators 897
17.6 Gradient-Domain Operators . 899
17.7 Histogram Adjustment . . . 900
17.8 Lightness Perception 901
17.9 Counter Shading . 905
17.10 Post-Processing . . 906
17.11 Validation and Comparison . 910
17.12 Further Reading . . 926
IV Appendices 929
A Vectors and Matrices 931
A.1 Cross and Dot Product . . . 931
A.2 Vector Differentiation 933
A.3 Gradient of a Scalar Function 933
A.4 Divergence 934
A.5 Gauss’ Theorem . . 934
A.6 Curl . . . 935
A.7 Stokes’ Theorem . 936
A.8 Laplacian 937
A.9 Vector Identities . . 937
A.10 Homogeneous Coordinates . 937
B Trigonometry 939
B.1 Sum and Difference Formulae 939
B.2 Product Identities . 940
B.3 Double-Angle Formulae . . 941

B.4 Half-Angle Formulae 941
B.5 Sum Identities . . . 941
B.6 Solid Angle 942
✐
✐
✐
✐
✐
✐
✐
✐
Contents xi
C Complex Numbers 945
C.1 Definition . 945
C.2 Euler’s Formula . . . 946
C.3 Theorems . 947
C.4 Time-Harmonic Quantities . . 948
D Units and Constants 949
E The CIE Luminous Efficiency Functions 951
F CIE Illuminants 955
G Chromaticity Coordinates of Paints 959
Bibliography 961
Index 1027

✐
✐
✐
✐
✐
✐

✐
✐
Preface
Color is one of the most fascinating areas to study. Color forms an integral part
of nature, and we humans are exposed to it every day. We all have an intuitive
understanding of what color is, but by studying the underlying physics, chemistry,
optics, and human visual perception, the true beauty and complexity of color can
be appreciated—at least to some extent. Such understanding is not just important
in these areas of research, but also for fields such as color reproduction, vision
science, atmospheric modeling, image archiving, art, photography, and the like.
Many of these application areas are served very well by several specifically
targeted books. These books do an excellent job of explaining in detail some as-
pect of color that happens to be most important for the target audience. This is
understandable as our knowledge of color spans many disciplines and can there-
fore be difficult to fathom.
It is our opinion that in application areas of computer science and computer
engineering, including such exciting fields as computer graphics, computer vi-
sion, high dynamic range imaging, image processing and game development, the
role of color is not yet fully appreciated. We have come across several applications
as well as research papers where color is added as an afterthought, and frequently
wrongly too. The dreaded RGB color space, which is really a collection of loosely
similar color spaces, is one of the culprits.
With this book, we hope to give a deep understanding of what color is, and
where color comes from. We also aim to show how color can be used correctly
in many different applications. Where appropriate, we include at the end of each
chapter sections on applications that exploit the material covered. While the book
is primarily aimed at computer-science and computer-engineering related areas,
as mentioned above, it is suitable for any technically minded reader with an in-
terest in color. In addition, the book can also be used as a text book serving a
graduate-level course on color theory. In any case, we believe that to be useful in

any engineering-related discipline, the theories should be presented in an intuitive
manner, while also presenting all of the mathematics in a form that allows both a
deeper understanding, as well as its implementation.
xiii
✐
✐
✐
✐
✐
✐
✐
✐
xiv Preface
Most of the behavior of light and color can be demonstrated with simple ex-
periments that can be replicated at home. To add to the appeal of this book, where
possible, we show how to set-up such experiments that frequently require no more
than ordinary household objects. For instance, the wave-like behavior of light is
easily demonstrated with a laser pointer and a knife. Also, several visual illusions
can be replicated at home. We have shied away from such simple experiments
only when unavoidable.
The life cycle of images starts with either photography or rendering, and in-
volves image processing, storage, and display. After the introduction of digital
imaging, the imaging pipeline has remained essentially the same for more than
two decades. The phosphors of conventional CRT devices are such that in the
operating range of the human visual system only a small number of discernible
intensity levels can be reproduced. As a result, there was never a need to capture
and store images with a fidelity greater than can be displayed. Hence the immense
legacy of eight-bit images.
High dynamic range display devices have effectively lifted this restriction, and
this has caused a rethinking of the imaging pipeline. Image capturing techniques

can and should record the full dynamic range of the scene, rather than just the
restricted range that can be reproduced on older display devices. In this book,
the vast majority of the photography was done in high dynamic range (HDR),
with each photograph tone-mapped for reproduction on paper. In addition, high
dynamic range imaging (HDRI) is integral to the writing of the text, with excep-
tions only made in specific places to highlight the differences between conven-
tional imaging and HDRI. Thus, the book is as future-proof as we could possibly
make it.
Acknowledgments
Numerous people have contributed to this book with their expertise and help.
In particular, we would like to thank Eric van Stryland, Dean and Director of
CREOL, who has given access to many optics labs, introduced us to his col-
leagues, and allowed us to photograph some of the exciting research undertaken
at the School of Optics, University of Central Florida. Karen Louden, Curator
and Director of Education of the Albin Polasek Museum, Winter Park, Florida,
has given us free access to photograph in the Albin Polasek collection.
We have sourced many images from various researchers. In particular, we
are grateful for the spectacular renderings given to us by Diego Gutierrez and
his colleagues from the University of Zaragoza. The professional photographs
donated by Kirt Witte (Savannah College of Art and Design) grace several pages,
and we gratefully acknowledge his help. Several interesting weather phenomena
were photographed by Timo Kunkel, and he has kindly allowed us to reproduce
✐
✐
✐
✐
✐
✐
✐
✐

Preface xv
some of them. We also thank him for carefully proofreading an early draft of the
manuscript.
We have had stimulating discussions with Karol Myszkowski, Grzegorz
Krawczyk, Rafał Mantiuk, Kaleigh Smith, Edward Adelson and Yuanzhen Li,
the results of which have become part of the chapter on tone reproduction. This
chapter also benefitted from the source code of Yuanzhen Li’s tone-reproduction
operator, made available by Li and her colleagues, Edward Adelson and Lavanya
Sharan. We are extremely grateful for the feedback received from Charles Poyn-
ton, which helped improve the manuscript throughout in both form and substance.
We have received a lot of help in various ways, both direct and indirect, from
many people. In no particular order, we gratefully acknowledge the help from
Janet Milliez, Vasile Rotar, Eric G Johnson, Claudiu Cirloganu, Kadi Bouatouch,
Dani Lischinski, Ranaan Fattal, Alice Peters, Franz and Ineke Reinhard, Gordon
Kindlmann, Sarah Creem-Regehr, Charles Hughes, Mark Colbert, Jared Johnson,
Jaakko Konttinen, Veronica Sundstedt, Greg Ward, Mashhuda Glencross, Helge
Seetzen, Mahdi Nezamabadi, Paul Debevec, Tim Cox, Jessie Evans, Michelle
Ward, Denise Penrose, Tiffany Gasbarrini, Aaron Hertzmann, Kevin Suffern,
Guoping Qiu, Graham Finlayson, Peter Shirley, Michael Ashikhmin, Wolfgang
Heidrich, Karol Myszkowski, Grzegorz Krawczyk, Rafal Mantiuk, Kaleigh
Smith, Majid Mirmehdi, Louis Silverstein, Mark Fairchild, Nan Schaller, Walt
Bankes, Tom Troscianko, Heinrich B
¨
ulthoff, Roland Fleming, Bernard Riecke,
Kate Devlin, David Ebert, Francisco Seron, Drew Hess, Gary McTaggart, Habib
Zargarpour, Peter Hall, Maureen Stone, Holly Rushmeier, Narantuja Bujantog-
toch, Margarita Bratkova, Tania Pouli, Ben Long, Native Visions Art Gallery
(Winter Park, Florida), the faculty, staff, and students of the Munsell Color Sci-
ence Laboratory, Lawrence Taplin, Ethan Montag, Roy Berns, Val Helmink,
Colleen Desimone, Sheila Brady, Angus Taggart, Ron Brinkmann, Melissa An-

sweeney, Bryant Johnson, and Paul and Linda Johnson.
✐
✐
✐
✐
✐
✐
✐
✐
Part I
Principles

✐
✐
✐
✐
✐
✐
✐
✐
Chapter 1
Introduction
Color is a phenomenon that relates to the physics of light, chemistry of matter,
geometric properties of objects, as well as to human visual perception and cog-
nition. We may call sunlight yellow, and in this case we refer to a property of
light. A car may be painted red, in which case the color red is an attribute of the
car. When light enters the eye, a complex chain of events leads to the sensation of
color, a perceived quantity. Finally, color may be remembered, associated with
events, and reasoned about. These are cognitive aspects of color. Color means
different things under different circumstances [814].

At the same time, it is clear that an understanding of color will involve each
of these aspects. Thus, the study of color theory and its applications necessar-
ily spans several different fields, including physics, chemistry, optics, radiometry,
photometry, colorimetry, physiology, vision science, color appearance modeling,
and image processing. As a result, what on the surface appears to be a rela-
tively simple subject, turns out to have many hidden depths. Perhaps this is the
reason that in practical applications found in computer science—computer vision,
graphics and image processing—the use of color is often under-explored and mis-
understood.
In our view, to understand color with sufficient depth, and to be able to apply
this knowledge to your own area of interest, it is not enough to read the literature
in any one specific discipline, be it computer graphics, computer vision, photog-
raphy, art, etc. Instead, it is necessary to step outside one’s own field in order to
appreciate the subtleties and complexities of color.
The purpose of this book is therefore to explain color theory, its development
and current state-of-the-art, as well as its practical use in engineering-oriented
disciplines such as computer graphics, computer vision, photography, and film.
Along the way, we delve into the physics of light, and its interaction with matter
3
✐
✐
✐
✐
✐
✐
✐
✐
4 1. Introduction
at the atomic level, such that the origins of color can be appreciated. We find that
the intimate relationship between energy levels, orbital states, and electromag-

netic waves helps to understand why diamonds shimmer, rubies are red, and the
feathers of the blue jay are blue. Even before light enters the eye, a lot has already
happened.
The complexities of color multiply when perception is taken into account. The
human eye is not a simple light detector by any stretch of the imagination. Human
vision is able to solve an inherently under-constrained problem: it tries to make
sense out of a 3D world using optical projections that are two-dimensional. To
reconstruct a three-dimensional world, the human visual system needs to make
a great many assumptions about the structure of the world. It is quite remark-
able how well this system works, given how difficult it is to find a computational
solution that only partially replicates these achievements.
When these assumptions are violated, the human visual system can be fooled
into perceiving the wrong thing. For instance, if a human face is lit from above, it
is instantly recognizable. If the same face is lit from below, it is almost impossi-
ble to determine whose face it is. It can be argued that whenever an assumption is
broken, a visual illusion emerges. Visual illusions are therefore important to learn
about how the human visual system operates. At the same time, they are impor-
tant, for instance in computer graphics, to understand which image features need
to be rendered correctly and which ones can be approximated while maintaining
realism.
Color theory is at the heart of this book. All other topics serve to underpin
the importance of using color correctly in engineering applications. We find that
too often color is taken for granted, and engineering solutions, particularly in
computer graphics and computer vision, therefore appear suboptimal. To redress
the balance, we provide chapters detailing all important issues governing color
and its perception, along with many examples of applications.
We begin this book with a brief assessment of the roles color plays in different
contexts, including nature and society.
1.1 Color in Nature
Living organisms are embedded in an environment with which they interact. To

maximize survival, they must be in tune with this environment. Color plays an
important role in three ways:
• Organisms may be colored by default without this giving them a specific
advantage for survival. An example is the green color of most plants (Fig-
✐
✐
✐
✐
✐
✐
✐
✐
1.1. Color in Nature 5
Figure 1.1. The chlorophyll in leaves causes most plants to be colored green.
ure 1.1), which is due to chlorophyll, a pigment that plays a role in photo-
synthesis (see Section 3.4.1).
• Color has evolved in many species in conjunction with the color vision
of the same or other species, for instance for camouflage (Figure 1.2), for
attracting partners (Figure 1.3), for attracting pollinators (Figure 1.4), or for
appearing unappetizing to potential predators (Figure 1.5).
• Biochemical substances may be colored as a result of being optimized to
serve a specific function unrelated to color. An example is hemoglobin
which colors blood red due to its iron content. Such functional colors are
normally found inside the body, rather than at the surface.
Plants reflect green and absorb all other colors of light. In particular, plants
absorb red and blue light and use the energy gained to drive the production of
carbohydrates. It has been postulated that these two colors are being absorbed as
a result of two individual mechanisms that allow plants to photosynthesize with
maximum efficiency under different lighting conditions [518, 769].
✐

✐
✐
✐
✐
✐
✐
✐
6 1. Introduction
Figure 1.2. Many animals are colored similar to their environment to evade predators.
Figure 1.3. This peacock uses bright colors to attract a mate; Paignton Zoo, Devon, UK.
(Photo by Brett Burridge (www.brettb.com).)
✐
✐
✐
✐
✐
✐
✐
✐
1.1. Color in Nature 7
Figure 1.4. Many plant species grow brightly colored flowers to attract pollinators such
as insects and bees; Rennes, France, June 2005.
Figure 1.5. These beetles have a metallic color, presumably to discourage predators.
✐
✐
✐
✐
✐
✐
✐

✐
8 1. Introduction
Figure 1.6. This desert rose is colored to reflect light and thereby better control its
temperature.
Color in plants also aids other functions such as the regulation of temperature.
In arid climates plants frequently reflect light of all colors, thus appearing lightly
colored such as the desert rose shown in Figure 1.6.
In humans, color vision is said to have co-evolved with the color of fruit [787,
944]. In industrialized Caucasians, color deficiencies occur relatively often, while
color vision is better developed in people who work the land. Thus, on average,
color vision is diminished in people who do not depend on it for survival [216].
Human skin color is largely due to pigments such as eumelanin and phaeome-
lanin. The former is brown to black, whereas the latter is yellow to reddish-brown.
Deeper layers contain yellow carotenoids. Some color in human skin is derived
from scattering, as well as the occurrence of blood vessels [520].
Light scattering in combination with melanin pigmentation is also the mech-
anism that determines eye color in humans and mammals. There appears to be
a correlation between eye color and reactive skills. In the animal world, hunters
who stalk their prey tend to have light eye colors, whereas hunters who obtain
their prey in a reactive manner, such as birds that catch insects in flight, have dark
eyes. This difference, as yet to be fully explained, extends to humankind where
dark-eyed people tend to have faster reaction times to both visual and auditory
stimuli than light-eyed people [968].
✐
✐
✐
✐
✐
✐
✐

✐
1.1. Color in Nature 9
Figure 1.7. Color plays an important role in art. An example is this photograph of the
Tybee Light House, taken by Kirt Witte, which won the 2006 International Color Award’s
Masters of Color Photography award in the abstract category for professional photogra-
phers (see also www.theothersavannah.com).