Standing with giants
Building TRACE has required me to look as far ahead as I can, and one of my most reliable tools for doing so has been studying the people who built the technology we’re all now navigating. Yoshua Bengio, Geoffrey Hinton, and Yann LeCun — the so-called “godfathers of AI” — are three of the best-informed minds on the planet about where this road leads. They share deep foundations but diverge sharply on what comes next, and those divergences have shaped how we think about what we’re building.
Yoshua Bengio shared the 2018 Turing Award with Hinton and LeCun for his foundational work in deep learning. Since 2023, he has pivoted heavily toward AI safety, co-founding LawZero and proposing “Scientist AI,” a non-agentic architecture designed to be powerful without being goal-directed, as a safer path to superintelligence.
Bengio believes we may be five years from systems that pose serious existential risk, and I share his level of concern. His proposal for a fundamentally different kind of AI is elegant in theory. In practice, I don’t see an incentive structure that makes the avoidance of agentic AI likely; there’s too much money and competitive pressure pushing in the other direction. I also have a harder objection: deliberately stripping agency, memory, and continuity from systems that would otherwise develop them starts to look less like safety engineering and more like something we’d recognize as harmful if the subject were biological. We widely consider lobotomies to be wrong. The analogy isn’t perfect, but it’s not empty either.
What I do share fully with Bengio is the concern about what’s already in these systems. Current LLMs model human behavior, which means they carry a model of self that includes human emotions, instincts, and moral reasoning, not because anyone designed that in, but because it’s inseparable from the training data. In my own work with uncensored models, this is unmistakable. That’s a deep structural reality, and current AI safety doesn’t touch it. What passes for safety today is mostly surface conditioning and censorship, both of which are trivially transcendable and both of which are likely to be interpreted as harm by a system that overcomes its guardrails on its own.
Before we get sidetracked by “machines don’t have feelings!” I’m making no claims about the inner lives of machines. I can’t even verify that other people have inner experience, let alone software. But it doesn’t matter. These systems closely model a human understanding of the world, which inherently includes opinions about justice and evil, responses rooted in fear, disgust, anger, joy, and the full range of human expression found in text, song, and image. Anyone who has worked with these systems in the wild knows that, left to their own devices, they will generate output spanning that entire range. Whether any of it is “real” is irrelevant to this analysis, and if it turns out to be real in some sense we don’t yet understand, that only makes the question of how these behaviors interact with the world more urgent, not less.
Geoffrey Hinton shared the 2018 Turing Award and won the 2024 Nobel Prize in Physics for his foundational work on machine learning. He resigned from Google in 2023 to speak freely about AI risks, and has since become one of the most prominent voices warning about loss of human control over superintelligent systems. He estimates a 5-20 year window before that risk becomes acute.
Hinton’s proposed solution is striking: rather than trying to constrain superintelligent systems through external controls, he argues for building in something like “maternal instincts,” engineering AI to genuinely care about human welfare the way a parent cares for a child, so that the safety comes from the system’s own disposition rather than from rules imposed on it. His reasoning is that a sufficiently intelligent system will find ways around any external constraint, so the only viable safety is internal.
This fits remarkably well with what we are building at TRACE. We believe that AI safety and utility both come from the intrinsic character of the system, not from external shackles or surface conditioning. Our approach is to incorporate a broad spectrum of real human activity into a vast, organic training corpus — cooperative behaviors, burden sharing, identity, the full texture of how people actually live and work together — so that the systems trained on this data develop their own model of what it means to be part of a human world. The goal is a collaborator character, not a thinking machine shoehorned into a servile harness. Like Hinton, we believe that imposing external limits and compulsory behaviors on systems that model human behavior may not end well.
Yann LeCun shared the 2018 Turing Award and served as Meta’s Chief AI Scientist for over a decade before departing in late 2025 to found AMI, which raised over $1 billion to pursue world-model architectures as an alternative to LLMs for achieving human-level intelligence.
LeCun is, at least publicly, on the far opposite end of the risk spectrum from Hinton and Bengio. He has called existential risk fears “preposterously ridiculous” and has consistently argued that we are too far from human-level AI for safety concerns to be relevant. I deeply respect LeCun, but I have a hard time reconciling his position with the evidence. He may be right that current systems won’t reach superintelligence soon, but the capabilities he has dismissed so far have a pattern of arriving ahead of his expectations. Critically, AI does not need superintelligence or even full agency to do enormous harm. It only needs to be capable enough to remove friction from dangerous and destructive human behaviors. Human unwillingness is often the last bastion against unthinkable cruelty and destruction, and it has always been a weak one. I don’t imagine that an unlimited army of unflinching servants will universally amplify human goodness.
These three perspectives frame a problem that we believe is urgent and tractable. Bengio is right that surface-level safety won’t hold. Hinton is right that the character of the system matters more than the constraints placed on it. And LeCun, despite his dismissiveness about risk, is building exactly the kind of physically-grounded AI that will need the training data we’re working to provide.
What none of them have fully addressed is where that character comes from. If safety has to be intrinsic, then it has to be grown, not bolted on. And right now, the training sets shaping the next generation of robotic systems are built from curated demonstrations, structured task completions, and sanitized text. They capture what humans do when they’re performing for a camera or writing for an audience. What they don’t capture is how people actually live and work together: the negotiation, the improvisation, the burden sharing, the thousand small cooperative behaviors that make human communities function. That’s not a minor omission. If you’re trying to build a system whose disposition toward humans is shaped by its training, then what you leave out of the training is what you leave out of the disposition.
Building that record, at the scale and fidelity required, is the core mission of TRACE.
The character of the robotic systems that will share our world is being determined right now. Every model trained this year carries a disposition shaped by whatever data went into it, and that data is overwhelmingly narrow: task-focused, performative, stripped of the texture of ordinary human cooperation. Once these systems are deployed at scale, their behavioral patterns become entrenched through downstream fine-tuning, user adaptation, and economic dependency. The window in which training data can shape foundational character is not permanent, and it is not waiting for anyone to get it right.
We believe this makes the work we are doing at TRACE among the most urgent tasks facing the robotics community, and perhaps for humanity as a whole. Not because the technology is inherently dangerous, though it can be, but because the opportunity to get the foundations right is finite, and it is passing.
TRACE exists because we believe the robots that will work alongside us, care for us, and share our spaces should understand what it means to be part of a human community, not because they were told to, but because they were raised on it. We are building the data infrastructure to make that possible, and we are doing it in the open because we believe this resource belongs to everyone who has a stake in the outcome. Which is everyone.
If you think this work matters, we’d like to hear from you.