In 2015, the picture of a white-and- gold dress (or was it black-and-blue?) divided humankind in two irreconcilable factions while revolutionizing scientists’ understanding of color perception. It was a brand-new category of illusion, in which different people perceived the same image in diametrically opposing ways. The two sections were locked in their respective perceptions. Try as they might, neither blue/black nor white/gold adherents could make themselves see the garment as the other side did.

Similarly baffling Internet sensations followed: a dresser that people saw as either white/pink or blue/gray, a sneaker that looked pink/white or green/gray to different observers, and an Adidas jacket that was either blue/white or brown/black depending on whom you asked.

Despite their differences, a common feature of these described images is that they were flukes, revealed by happenstance. The serendipity of their discoveries raised the question of whether scientists had a true understanding of how the newfound illusions might come about.

Pascal Wallisch, a neuroscientist at New York University, believes that the key to the puzzle is observers’ previous knowledge of lighting sources and materials such as fabrics—what psychology researchers call “priors.” To prove it, Wallisch and his New York University collaborator Michael Karlovich devised a method of creating color illusions that are just as confounding as those previously found by chance. The Crocs and socks photograph at the beginning of this article is one example. To create the image, Wallisch and Karlovich started with an object that looks pink under white light (a pair of “Ballerina Pink Classic Crocs”) and instead illuminated it with green light, equalizing its appearance to gray. Then, they made the background pitch-black, removing any contextual color cues that the visual system might utilize. As a result, the Crocs might be any color or at least any of the 28 different hues that you might find at your favorite Crocs retailer.

Depending on your past familiarity with white tube socks (your prior), your visual system may correctly conclude that the socks are truly white but illuminated by green lighting. If so, you may be able to retrieve the Crocs’ original pink color in your perception. Observers who lack the white sock prior may instead perceive the Crocs as grayish.

People believe that they see things “how they really are,” Wallisch says. “But does this mean the colors of the pixels in isolation or of the whole shoe in context? Those two [interpretations] can be different for different people.”