New research shows that reorganizing a model’s visual representations can make it more helpful, robust and reliable
“Visual” artificial intelligence (AI) is everywhere. We use it to sort our photos, identify unknown flowers and steer our cars. But these powerful systems do not always “see” the world as we do, and they sometimes behave in surprising ways. For example, an AI system that can identify hundreds of car manufacturers and models might still fail to capture the commonalities between a car and an airplane, i.e. both are large vehicles made primarily of metal.
To better understand these differences, today we’re publishing a new paper in Nature analyzing the important ways AI systems organize the visual world differently from humans. We present a method for better aligning these systems with human knowledge, and show that addressing these discrepancies improves their robustness and ability to generalize.
This work is a step towards building more intuitive and trustworthy AI systems.
Why AI struggles with the “odd one out”
When you see a cat, your brain creates a mental representation that captures everything about the cat, from basic concepts like its color and furriness to high-level concepts like its “cat-ness.” AI vision models also produce representations, by mapping images to points in a high-dimensional space where similar items (like two sheep) are placed close together, and different ones (a sheep and a cake) are far apart.
To understand the differences in how human and model representations are organized, we used the classic “odd-one-out” task from cognitive science, asking both humans and models to pick which of three given images does not fit in with the others. This test reveals which two items they “see” as most similar.
Sometimes, everyone agrees. Given a tapir, a sheep, and a birthday cake, both humans and models reliably pick the cake as the odd one out. Other times, the right answer is unclear, and people and models disagree.
Interestingly, we also found many cases where humans strongly agree on an answer, but the AI models get it wrong. For the third example below, most people agree the starfish is the odd one out. But most vision models focus more on superficial features like background color and texture, and choose the cat instead.
