The goal of our visual system is to infer the physical properties of objects. But last week's famous dress photo revealed that we can reach dramatically different conclusions about this seemingly straightforward task.

The heated debate over whether the famous dress is black/blue or gold/white is simmering down, but the deeper questions remain: Why do we see different colors? Why is it so difficult to switch from one percept to another? Does our brain have fundamentally different mechanisms to process lights and objects? If so, do all of the previous theoretical and empirical results regarding human color perception still hold? Why does this particular image allow such radical interoperation?

Scientists all over the world have been offering hypothesis. Over a course of a few days, several illustrative demos have emerged.

Color Constancy

So far, the consensus is that this phenomenon is related to color constancy, a feature of the human color perception system that allows us to perceive stable object color under varying illumination conditions. For example, you roughly perceive your car as having the same color when it is lit by the sunlight, incandescent light, or at different times of the day. In order to achieve constancy, one has to discount the effect of the color of the light and recover the color of the surface. In most scenes, there are enough visual cues to figure out the illumination, such as a nearby white object, relative reflectance of other objects, specular highlights on the surface, cast shadow, etc. But when the scene context is significantly reduced, color constancy can become quite poor.

In the photo of the dress, the illumination is very ambiguous. The photo lacks a reference white object (such as skin) and clear indication of the lighting direction. When the visual system faces ambiguity, observer has to "guess" the illumination color and such guesses strongly depend on their prior beliefs. For example, some people tend to see the dress in the shadow and shadows usually have a bluish hue. Hence, the dress under the shadow might look white and gold after discounting the illumination color.

Assumption About Illumination Color

It is possible that people make different guesses about the color of the ambient light in the photo. Neuroscientist Rosa Lafer-Sousa created a famous demo inspired by fellow neuroscientist Beau Lotto, and showed that if people assume a yellowish color of the overall light, they tend to overcompensate to see the dress being blue and black, whereas if people assume a bluish color of the overall light, they tend to see the dress as white and gold. However, some people still report seeing the dresses in the two images as having the same color.

Assumption About Illumination Geometry  

Others suggest people assume different geometry of the lighting condition. The demo by Yukiyaso Kamitani, neuroscientist from ART lab in Japan, shows that if you assume the dress is in shadow (as the woman on the right), you tend to compensate the illumination and think it is white and gold, whereas if you assume the dress is lit from the front (as the woman on the left), you tend to compensate and view it as blue and black.
Photo by Yukiyasu Kamitani  

Assumption About Reflectance Properties  

Different perceptions of the material properties of the dress can also contribute to illumination estimation. Even though the shoulder part of the dress appears to be glossy, it is difficult to estimate the material composition of the dress from the photo. Some people might see the whitish spot on the dress being the reflection of the illumination, hence they conclude the dress is blue and black. Others might perceive the bluish part of the scene as shadow and conclude that the dress is white and gold.  

Why Do People Have Different Assumptions?  

Why do people make different assumptions of the illumination? The image that enters our retina is the same (assuming the same device and viewing angle), but the transformation from the retina image to a color representation can be different for everyone. A New York Times article initially suggests that it could be due to the fact that people have different distributions of long, medium, and short wavelength sensitive cone photoreceptors in their retinas. But previous results from David Williams, professor of medical optics at the University of Rochester, have shown that this has little effect on color perception.  

In my opinion, the illusion happened primarily cortically. The individual difference can originate from different scene parsing strategy (whether the dress was lit with the same light source as the backdrop) and different visual cue recruitment strategy (e.g. does the specular highlights on the shoulder, inter-reflections on the concave part of the dress provide consistent cues about the illumination). 

In addition, it is known that people are better at discounting the color of the illumination in the scene when they don't happen along the blue-yellow axis, which is the axis of the daylight spectrum (work by Anya Hulbert from the University of Newcastle in UK and Karl Gegenfurtner from Giessen University in Germany). This uncertainty might contribute to the ambiguity in estimating the illumination in the dress photo. However, it seems that flipping the color 180 degrees (see the image below) resulted in an image that has much less individual difference in the color of the dress. So the reason why this particular image causes so much individual difference might also depends on the particular luminance values of the dress in the original photo.

In forming high-level opinions, we are not surprised that prior belief will influence our opinions of a current event (e.g. Is global warming a thing?). However, most previous results in perception have been obtained from averaged data across observers. The individual difference diminished into various sized error bars. Perceptual priors, assumption of the illumination for example, can differ significantly among individuals. This dress illusion provides proof that we are not designed to see the physical world in the same way.  

What experiments can help us to resolve this deep mystery? I believe we need to reconsider many basic experiments to explore individual difference in hue discrimination, scene segmentation, color constancy of different material properties, and color memory. There is no doubt the discussion of the dress will prompt immediate scientific endeavor to dig deep into individual differences in perception, its computational mechanisms, and its neural origin.
Photo by Karl Gegenfurner

Want to Learn More? 

The Vision Sciences Society Annual Meeting takes place in Florida May 15-20. My colleagues and I will present a demo at the conference—called the Blue Dress Pavilion—to show how different lighting conditions affect color perception.