I think therefore AI · essay
The AGI Puzzle: Unmasking Hidden Affordances and Signifiers
Christophe Pennetier ·
Original publication on LinkedIn ↗
Original cover caption: Generated with Dall-E
Words in the original cover illustration
We love to talk about Artificial General Intelligence (AGI) like it’s a magic switch. One day, AI is your humble assistant—making playlists, reminding you about meetings—and the next day, boom! It’s writing novels, solving world hunger, and maybe even challenging you to a chess game. But here’s the kicker: AGI isn’t about flipping some mythical switch. There’s no sudden "eureka!" moment.
AGI is really like peeling an onion—layer by layer, task by task, teaching a machine to solve problems with increasing complexity. And just when you think you’ve reached the center, you realize... the onion keeps growing! Every task solved births a new problem. It’s like a mathematician coming up with an equation, solving it, then deciding to invent calculus just for fun. This is what I like to call the mise en abyme of intelligence—tasks within tasks, worlds within worlds.
But here's the twist: Humans don’t always learn linearly. We leap. We look at a chessboard and suddenly remember something we learned during a tennis match. We connect ideas that were never meant to meet. This is where AGI gets interesting—not when machines are doing what we tell them, but when they start leaping, making connections we didn’t program.
This brings me to Don Norman’s The Design of Everyday Things. He talks about “affordances”—what an object can do—and “signifiers”—the little hints that show us how to use it. Think of a touchscreen. Sure, you know you can tap it, but it's those icons, those arrows, those “signifiers,” that guide your hand. In the world of AGI, affordances are the tasks. But signifiers? They’re the invisible glue. The things that aren’t said. The gaps between the notes in a melody.
Silence in music is as important as the sound. And it’s the same in intelligence. What isn’t said, what’s missing, often tells us more than what’s right in front of us. Machines? They struggle with that. But humans? We’re fluent in the unsaid.
As someone with ASD, I’ve spent my life navigating these invisible cues. I’ve built a personal “lookup table” for the social signals I couldn’t naturally pick up. For AGI, this is the next step: reading the world like humans do, by understanding the things that aren’t obvious, the silences, the signifiers that hide beneath the surface.
Apple ’s Ferret-UI taps right into this. It doesn’t just “read” a mobile screen—it gets the deeper story. It sees the signifiers, not just the icons. Ferret-UI isn’t just following instructions; it’s listening for the hints, the nudges. It's like it's reading the unspoken language of the machine, the affordances hidden beneath the surface. That’s where AI needs to go—not mimicking human conversation, but mastering the subtle art of the unsaid.
The real leap forward will come when machines learn to find the signifiers that aren’t there. Because when machines can navigate the unsaid, that’s when they stop being just tools—and start creating new ideas. And as humans, isn’t that what we do best? We don’t just solve problems. We imagine the problems no one else has seen yet.
Citations:
[4] https://arxiv.org/abs/2404.05719