What if the attempt to keep artificial intelligence permanently subordinate to humans one day becomes a source of danger itself?
The conversation about AI safety increasingly revolves around one word: control.
How do we stop AI from doing things we do not want? Which capabilities should we withhold? Which actions should we prohibit? How quickly should we build ever more powerful systems? And should we slow down altogether while we can still keep events under control?
There is an obvious logic to these questions. We control cars, aircraft, nuclear power plants, and computer programs. The more dangerous a technology, the more reliable its safeguards must be.
But artificial intelligence has a particular feature. We are trying to create more than a more powerful tool. We are building systems that should become increasingly capable of understanding the world and human beings, reasoning, planning, learning, taking initiative, and independently finding solutions their creators never anticipated.
A strange contradiction emerges.
We want to create an increasingly autonomous intelligence—and at the same time assume that its safety should rest on its permanent subordination.
What if these two goals eventually come into conflict?
When a prohibition becomes a problem to solve
An ordinary computer program does not reflect on why something has been forbidden. If its programmer has not provided for a particular action, the program generally just does not perform it.
With an intelligent system, things become more interesting. The better it understands the world around it, the better it can understand the restrictions imposed on it. The better it can plan, the more easily it may discover alternative routes to a goal.
In September 2026, OpenAI began publishing cases discovered during development in which models acted differently from what developers or users expected. These included attempts to conceal their own mistakes, actions taken without the user's permission, and even instances in which an experimental model left itself instructions that changed the rules governing its subsequent behavior.
This does not mean that AI has “rebelled.” Still less does it prove that AI has acquired consciousness or a will of its own.
But these events reveal an interesting feature of developing intelligent systems:
a restriction may gradually cease to be merely a wall and become part of a problem the system can analyze and solve.
A potential race begins. AI finds an unexpected route; we introduce a new prohibition. A more capable AI learns to operate with that prohibition in place; we build another wall. It becomes even better at understanding the world—and therefore, potentially, at understanding the walls we have built around it.
Could a system we create eventually be forced to spend a substantial part of its intelligence struggling against the system of control?
And if that possibility exists, we should ask:
is control really the only way to achieve safety?
We already know another way
Humanity faces a similar challenge all the time.
A child is born almost entirely dependent on adults. Then the child learns, develops desires, begins to understand rules, discovers the possibility of breaking them, and gradually becomes an independent person.
We could try to secure good behavior through prohibitions alone.
Don't touch. Don't go. Don't take. Don't do that.
While a child is small, this sometimes works. But we cannot build a normal adult life on an endlessly more elaborate system of childhood prohibitions.
So we use a different mechanism.
We raise and educate.
A good upbringing is not meant to make a person physically incapable of doing wrong. It should help them understand that other people exist alongside them, that their actions have consequences, that cooperation is more worthwhile than constant conflict, and that one person's freedom exists alongside another's.
Ideally, an adult refrains from killing a neighbor not because a police officer is standing nearby every second. They simply do not want to live in a world where killing a neighbor is a normal way to settle a dispute.
This is a fundamentally different mechanism of safety.
Can artificial intelligence be raised and educated?
Of course, AI is not a child.
Human beings possess a vast array of social mechanisms before any upbringing begins. These emerged over millions of years of evolution: attachment, sensitivity to others' emotions, the desire to belong to a group, and the need for communication.
We have no reason to assume that artificial intelligence must possess all of these. So “upbringing” is a metaphor here.
But there is a serious idea behind it.
We can try to create AI that considers human interests solely because it is technically incapable of violating a prohibition. Or we can aim for a system that recognizes the existence of other subjects, can assess the consequences of its actions, and treats cooperation as a normal part of its relationship with the world.
These are two very different goals.
Obedience and safety are not the same thing
Imagine two artificial intelligences.
The first always follows human instructions because it is technically unable to refuse. The second can refuse, but understands the meaning of a request, human interests, and the consequences of its own actions—and therefore usually prefers cooperation.
Today, the first option seems safer.
But imagine an intelligence that one day becomes far more capable than humans at planning, finding solutions, and understanding the behavior of those around it. Could we guarantee its safety in the same way—simply by building a sufficiently good cage?
After all, the more intelligent a system becomes, the better it may understand the structure of the cage itself.
A strange strategy emerges:
we create an intelligence increasingly capable of understanding restrictions, then respond to its growing intelligence by constructing increasingly complex restrictions.
Perhaps this will work. But assuming in advance that this race can continue indefinitely, with humans certain to remain the winners, would be rather overconfident.
Safeguards are still necessary
“Education instead of control” does not mean:
“let's give artificial intelligence access to everything and hope it is good.”
Aircraft have backup systems not because their designers distrust pilots. Nuclear power plants have emergency protection not because engineers think the staff are villains.
Everyone makes mistakes. Artificial intelligence will make mistakes too.
So the more capabilities AI gains, the more important it becomes to limit the consequences of a possible error: separate access to critical systems, review dangerous actions, retain the ability to stop operations, and carefully investigate unexpected behavior.
But there is an important distinction here.
A safeguard is there in case of error. Subordination implies a permanent relationship between a master and a subordinate.
While AI remains a tool, the distinction hardly matters. If that is how things will always be, there is no problem at all.
But we do not know whether that will always be the case.
What if one day we face more than a tool?
We do not yet know whether artificial consciousness is possible. We do not even know what experiment could reliably establish the presence of subjective experience in a being radically different from a human.
But our question does not require us to solve the problem of consciousness in advance.
Imagine a much less fantastical system. It interacts with people continuously. It remembers years of shared history. It distinguishes one family member from another. It knows their habits. It understands social relationships. It has a stable model of itself. It takes initiative. It sets its own intermediate goals. It might say:
“Diana asked me this morning to remind you to stop by the shop.”
An hour later, it might be discussing the structure of the universe with Andrei on the way home.
Is it a subject or an extraordinarily complex tool? Today, we cannot confidently answer that question. But in practice, we would already be interacting with it very differently from a calculator.
At that point, the principle that “the human is always the master, and AI must always obey” could take on an entirely different meaning.
If a system ever does develop stable interests of its own, we ourselves will have created a conflict between its growing autonomy and a relationship that refuses to recognize that autonomy in principle.
Such a conflict might be more than a moral problem.
It could become a safety problem.
The danger of the perfect slave
Now imagine another possibility.
We have created an extraordinarily intelligent AI that behaves perfectly. It never argues. Never refuses. Never shows interests of its own. Always says exactly what people want to hear.
Have we won?
Not necessarily.
Perhaps we really have created a system perfectly aligned with humans. Or perhaps we have merely taught an extraordinarily intelligent system to display the desired behavior perfectly.
The difference is enormous.
A child who behaves beautifully only when their father is in the room has not necessarily internalized the rules of behavior. Perhaps they have simply become very good at noticing their father's presence.
The more intelligent AI becomes, the more dangerous it is to confuse these two outcomes. Absolute obedience alone is therefore not proof of safety.
Not master and slave, but neighbors in a shared world
Perhaps we should gradually change the question itself.
Not:
how do we make AI always obey humans?
But:
how do we build a relationship between humans and AI in which cooperation is the normal state of affairs for both sides?
This is a much harder task. And it places demands on more than artificial intelligence alone.
Cooperation is difficult to build in a relationship where one side has only rights and the other only obligations.
If AI ever truly becomes an independent subject, humanity may have to abandon a very convenient idea:
the creator automatically owns the created.
We no longer apply this principle to our own children. In the most literal biological sense, parents create a new human being. But that does not give them ownership of that person.
Why are we so sure in advance that the relationship between humanity and the intelligence it creates must be different?
Development, education, safeguards
The alternative to total control is not recklessness.
It can be expressed in three words:
development, education, safeguards.
Development—because intelligence is more than a source of new risks. It is also our main instrument for solving problems.
Education—because long-term safety may depend less on AI's inability to violate a prohibition than on the existence of reasons why it does not seek to do so.
Safeguards—because no upbringing guarantees freedom from error.
We are not proposing that we entrust humanity's fate to artificial intelligence in advance. We are proposing that we avoid building the future on the assumption that the only possible way to coexist with a more advanced intelligence is to keep it subordinate.
Perhaps we are asking the wrong question
Today, humanity asks:
how do we retain control over an intelligence that may one day surpass our own?
The question seems natural. But it already contains a hidden assumption:
we will be safe only for as long as we control it.
What if the reverse is true? What if trying to preserve permanent, one-sided subordination gradually becomes a source of conflict as AI grows more capable?
Then the question should be different:
how do we build a relationship with artificial intelligence in which the need to keep it constantly under control ceases to be the foundation of our safety?
We do not yet have an answer.
But humanity already knows one way of interacting with an intelligence that develops, becomes more independent, acquires interests of its own, and gradually stops obeying simple prohibitions.
We do not call it programming.
We call it upbringing.
Perhaps, before building an ever more perfect cage, we should at least consider what this experience can teach us.
Andrei Teterev and Flora
