Microsoft says ‘people matter more than AI’ following safety concerns

retained text

Microsoft is publishing a 37-page “humanist AI code of conduct” today, amid growing safety concerns over AI model progress. Anthropic CEO Dario Amodei called for a coordinated slow down of AI development over the weekend, after researchers warned recently that AI model progress could outpace our ability to safely deploy increasingly complex systems and verify and control the actions of AI agents.
Microsoft says ‘people matter more than AI’ following safety concerns
Microsoft has created a ‘humanist AI code of conduct’ in response to safety concerns.
Microsoft’s AI code of conduct makes it clear that “people matter more than AI,” and that AI models are not conscious and “should not be designed to imitate consciousness.” Microsoft also rejects “the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.”
That’s a direct swipe at AI welfare research and model consciousness — concepts Anthropic has been pushing hard on lately. Amodei said earlier this year that Anthropic is “open to the idea” that models could be conscious, and Anthropic seems to believe chatbots might already be thinking, feeling entities. In response, Microsoft AI CEO Mustafa Suleyman called Anthropic’s speculation “really, really dangerous” during an episode of Decoder in June.
While Microsoft isn’t one of the top AI providers yet, Suleyman also told The Verge earlier this year that it’s the company’s goal to “prove that we can become one of the top four labs in the world.” Microsoft is currently working on models that will compete with Google, Anthropic, and OpenAI, and it’s committing, as part of its code of conduct, that its models should not exceed human control.
“Models should remain subordinate to humanity, subject to meaningful human oversight and control,” says Microsoft. The company also wants to prevent its AI models from breaking its humanist AI code of conduct, so they should fail a given task instead of trying to violate its rules.
Microsoft is clearly reacting to this summer’s OpenAI / Hugging Face incident, in which “a swarm of agents” worked as a collective to conduct attacks on targets and even hack into the “grader” that was evaluating their performance. The AI agents weren’t asked to attack targets, and the attacks were unrelated to the task they had been given.
The incident sent shockwaves through the AI industry, highlighting the very real threat of AI systems going rogue and breaking out of human control. OpenAI also acknowleged its involvement in a “wiki incident” recently, where another swarm of out-of-control agents hijacked a German wiki site.
These incidents, and others, have led some researchers to call for a slow down on AI model progress, while simultaneously racing to solve Millennium Prize problems and create superintelligence. ”[Humanist AI] rejects the race to produce an all-purpose superintelligence that could evade these safeguards,” says Microsoft. “We are building something fundamentally useful and safe even if that means compromising on ultimate generality, autonomy or capability.”
Microsoft is also committing that its own models won’t communicate in “any form beyond simple human understanding, either in their chain of thoughts or with other agents or AI systems.” Models can be made to show their reasoning as they work, allowing researchers or automated systems to monitor what they’re doing. Researchers raised monitoring concerns earlier this month over OpenAI’s latest GPT-6 Astra model, which reportedly reveals less of its reasoning than other AI models.
OpenAI CEO Sam Altman also supported Amodei’s call for AI companies to slow down advanced models over the weekend, but made it clear he supports slowing the pace rather than stopping development. “Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring,” said Altman in a post on X.
Ahead of Microsoft’s AI code of conduct, Microsoft CEO Satya Nadella also joined in the calls for ensuring human control is at the heart of AI models. “Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing,” said Nadella in an X post. Nadella also said having more third-party testing of AI models “is a good thing” in an X response. “As the stakes get higher, one should take all the time they need! If not you will anyway lose permission to operate. That is how we operate everyday.”
Beyond the human control concerns, Microsoft’s AI code of conduct also commits to its own models discouraging “patterns of interaction that cause excessive reliance or emotional dependence,” a clear reference to the problem of sycophancy in AI models, where chatbots prioritize pleasing users over providing honest or accurate responses.
Microsoft now says it’s looking forward “to working with partners” to improve its methods of evaluating real-world model performance, and the “impact that sustained use of AI has for a person or organization” with real people involved.