OpenAI just wants to win

extract

OpenAI has spent the last few years planting flags across the increasingly difficult terrain in mathematics. This week, it claimed one of its biggest prizes yet: a solution to a legendary Millennium Prize problem. In normal circumstances, this would have been celebrated as a historic achievement. OpenAI just wants to win Two mathematicians at the heart of recent controversies tell The Verge how AI giants’ ‘childish’ rivalries are upending the field. OpenAI just wants to win Two mathematicians at the heart of recent controversies tell The Verge how AI giants’ ‘childish’ rivalries are upending the field. Instead, many mathematicians have watched OpenAI’s relentless advance with growing unease. To them, the company appears less like an enthusiastic newcomer than an impossibly well-resourced interloper, charging into problems they have dedicated their lives to studying with little apparent regard for long-standing norms or the consequences for those left in its wake. At the heart of that unease is a sense that OpenAI is doing mathematics for different reasons. Mathematicians want to advance the field. OpenAI wants to win. Mathematics is not normally this dramatic, so how did things get this bad? This week, The Verge spoke with more than a dozen mathematicians, including Tristan Buckmaster and Andreas Thom, who are at the center of recent controversies surrounding OpenAI’s work in the field. Even those skeptical of the most serious allegations described a field shaken by the tech giant’s conduct and fearful of what it might do next in its determination to trounce its rivals. Buckmaster has accused OpenAI of failing to adequately explain whether work he did through its tool Codex could have contributed to its recent successes. In a statement to The Verge, OpenAI spokesperson Laurance Fauconnet strenuously denied that material from his prompts had played a role: “We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.” Buckmaster remains unconvinced. “Given their behavior up until this point, one should take such statements with great skepticism,” he said. Mathematics is not normally this dramatic, so how did things get this bad? A rumor was all it took for tensions to boil over. OpenAI says it heard some researchers were making progress on Millennium Prize problems and decided to see whether one of its advanced, unreleased models could make headway too. It turns out it could. OpenAI says it took roughly 10,000 agents, tens of millions of dollars of compute, and just 88 hours to find a solution to the Navier-Stokes problem, which concerns the flow of fluids. The company had also discovered who it was racing against: Buckmaster, an NYU professor, and Levent Alpöge, a researcher at one of its fiercest rivals, Anthropic. Among several lines of research, the pair were pursuing Navier-Stokes, though had not yet completed a proof. Some details of what happened next are fiercely contested, but the two sides broadly agree on the basic sequence of events. One thing is particularly clear: Alpöge’s involvement was a problem for OpenAI, despite his saying it was a “personal collaboration” independent of his work with Anthropic. Even those skeptical of the most serious allegations described a field shaken by the tech giant’s conduct and fearful of what it might do next in its determination to trounce its rivals. Buckmaster said he contacted OpenAI after learning the company had become aware of their progress and was racing toward a solution of its own. He said discussions with OpenAI researcher Sébastien Bubeck grew contentious and, in his view, threatening, but the company offered a path forward for him — one that excluded Alpöge. Buckmaster said he was offered practically “unlimited compute” to finish his own work, and the opportunity to be the sole author of OpenAI’s paper announcing the breakthrough, which would of course credit its tools. “All I had to do was throw Levent under the bus,” Buckmaster told The Verge in a phone interview. He said he flatly rejected Bubeck’s offer, which he viewed as a “bribe,” and also began questioning whether OpenAI may have benefited from his use of Codex, one of the company’s AI tools he had been using to tackle the problem. OpenAI has denied that anyone — or any agent — accessed his specific user data, and until its more recent comments acknowledged it could not rule out the possibility data derived from his use of the products was used to improve the model. Buckmaster ultimately decided to go public with both his work and his account of OpenAI’s conduct. His office, he said, had been transformed into something of a “war room,” with colleagues helping scrutinize his mathematics, coordinate outreach, and even get in touch with lawyers. Bubeck has rejected Buckmaster’s characterization of the conversations on social media and in an interview with The New York Times. He acknowledged offering OpenAI’s resources to help Buckmaster complete his own proof or to have him take over the writing of the company’s. Strikingly, Bubeck said OpenAI had made similar arrangements with other mathematicians, though did not identify them. But his account nevertheless makes clear that Alpöge’s affiliation with Anthropic was a sticking point. “From our perspective, how can we have an internal OpenAI project with an Anthropic employee?” he told the Times. If the goal is to compete in mathematics, there are few bigger trophies than solving a Millennium Prize problem. The seven problems, set out by the Clay Mathematics Institute in 2000, are widely considered among the most formidable challenges in the field. Each carries a $1 million bounty for whoever solves it. Many had already endured decades of intense scrutiny by the time the prizes were established. In the quarter-century since, only one — the Poincaré conjecture, a topological problem concerning three-dimensional spheres — has fallen. For an AI company looking to prove that its models are the best at mathematics, then, they are irresistible targets. To Buckmaster and many other mathematicians The Verge spoke to, that helps explain why OpenAI moved so ferociously when it heard others were closing in — particularly once a rival AI company appeared to be involved. For Buckmaster, the episode reinforced something he already believed strongly from a previous spell collaborating with Google DeepMind: “All these tech people are obsessed” with solving big famous problems and are “obsessed with scooping,” he said. “Its all about competition.” In that world, Tristan Buckmaster said there is an intense fixation on prestige, fame, being first, and being seen to be first. “That’s the only currency,” he said. He said what often gets “lost” when companies race to solve famous problems are the mathematicians themselves — not just the people whose accumulated work makes these breakthroughs possible, but the reasons they do mathematics to begin with. Yes, some may pursue prestige, but most are simply not trophy hunters. Andras Juhasz, a professor of mathematics at the University of Oxford, described mathematics as an elegant discipline that is part science, part art, with many different motivations driving those working there. “Often there is no immediate practical application,” he said. “They do it because it’s beautiful. They enjoy it. It’s the sense of discovery. It’s natural.” Unlike classroom-level mathematical exercises, frontier mathematics rarely has a prescribed route to an answer. Researchers can attack problems from any number of angles, some radically different, which makes the ideas that lead to a solution — and who developed them — especially important, perhaps more so than solving a problem itself. Mathematicians care deeply about this lineage because it is how the field expands, with new techniques and methods often proving more consequential than the problem they were designed to solve. To Buckmaster, his exchanges with Bubeck typify the chasm that separates the worlds of research mathematics and Big Tech, and highlight the differences between what is considered valuable in research. Reading from notes he took while chatting with Bubeck, he said the OpenAI researcher was visibly taken aback when he rejected the company’s offer to take credit. “I could see Sébastien’s face. He was shocked when I said I don’t care about the Millennium Prize,” he recalled. Buckmaster said he had encountered a similar mentality among tech researchers before. In that world, he said there is an intense fixation on prestige, fame, being first, and being seen to be first. “That’s the only currency,” he said. Buckmaster isn’t the only mathematician to come away from an encounter with OpenAI concerned about the company’s motivations. Andreas Thom, a professor at the Technical University of Dresden in Germany, found himself at the center of a controversy last month after OpenAI announced an impressive mathematical result that built heavily on work by him and fellow researcher Gábor Kun. The company quietly amended its announcement to acknowledge the pair’s contribution without announcing or publicly disclosing the change. Thom described the ordeal as “not a very pleasant experience,” but told The Verge he had largely put it behind him until Buckmaster went public. His allegations prompted Thom to revisit an unresolved question about OpenAI’s breakthrough: whether conversations he and his colleagues had with ChatGPT about the research could have been used to help improve the models that ultimately cracked the problem he’d spent years working on. Only OpenAI has the information needed to answer that question, Thom said. “To be honest, I suspect that they don’t even know.” The people training the models and using them to produce mathematical results are “a different kind of people,” he said. To him, that’s hardly an excuse for the uncertainty. “Because it effectively means that they don’t really care, right?” “The prospect of competing with powerful AI companies, whose resources far exceed those available to academic research groups, could make them even more reluctant to pursue ambitious questions.” The tension echoes fights already playing out elsewhere. Writers, musicians, artists, and media companies have all challenged AI companies over systems built from vast stores of human-created work, often without permission, recognition, or compensation. While mathematics may seem a world apart, the underlying question is the same: What do companies owe to the people whose accumulated work they ingested to build their systems? In Thom’s case, the question remains unresolved. OpenAI did not respond to The Verge’s question on whether data from conversations Thom and his colleagues had with ChatGPT could have contributed to the company’s solution that built on his work. More broadly, Thom said he resents what he sees as a failure to recognize the “the communal effort that this entire community has put into all the research results” underpinning AI’s recent mathematical advances. Companies, he said, “are just now using [it] as if it was kind of nothing.” “I think there is a certain attitude that I don’t like in that,” he said. It’s not that mathematicians are strangers to competition — researchers care deeply about priority and bitter disputes over who came first litter mathematical history — but while competition does not preclude cooperation, these are no ordinary competitors. Scooping in mathematics has historically been relatively difficult for obvious reasons: Very few people have the specialized expertise to swoop in on a discovery at speed. AI companies operate on a different scale. Researchers worry they could turn scooping into something of an industrial process mathematicians would have little chance of fighting back against, rapidly spinning up thousands upon thousands of agents and enormous amounts of compute whenever word spreads that a breakthrough is close. To Buckmaster, OpenAI could have easily collaborated with researchers rather than race them to results. Indeed, the company seemed perfectly willing to work with him. The problem was Alpöge, or, more specifically, his ties to Anthropic. “They were in such a rush to publish, to beat Anthropic,” he said. They barely took note of the researchers caught in the middle. For all the rush, OpenAI won’t know whether it has won the Millennium Prize for solving Navier-Stokes for years. The Clay Mathematics Institute requires a period of two years to have passed since a result was published, during which it must have “received general acceptance in the global mathematics community.” For now, Navier-Stokes occupies a peculiar limbo: The Institute has removed it from its list of unsolved problems, though hasn’t yet declared it solved. “The process is deliberately unhurried,” the Institute said in a statement. OpenAI, meanwhile, has already moved on. In a statement to The Verge, OpenAI’s Fauconnet said that “since the completion of Navier-Stokes we have made substantial progress on another Millennium Prize problem,” adding that the company is “working through how to share these results thoughtfully.” Which problem remains unclear. Unconfirmed speculation on social media suggests this could be the Hodge conjecture, which concerns, very roughly speaking, how complex geometric shapes can be understood in terms of simpler building blocks. Rumors are also circulating that Anthropic is closing in on a Millennium Prize problem of its own. As the two giants of AI race to collect yet more mathematical trophies, they are discovering that astonishing results alone are not enough to earn the trust of the community they are transforming. Mathematicians are beginning to push back. Many The Verge spoke to, even the most enthusiastic proponents of AI in the field, worried the companies were having a chilling effect on research, pushing mathematicians to be more secretive about unfinished work for fear someone may swoop in and beat them to it. Several said colleagues who had previously compiled lists of important unsolved problems were reconsidering the practice, concerned that what was intended as a useful resource for the field could instead become a list of targets for AI companies. Resistance is becoming increasingly public. In June, mathematicians published the Leiden Declaration, a set of principles for the responsible use of AI in mathematics that has been endorsed by the International Mathematical Union and signed by nearly 3,900 people, an increase of nearly 500 people since I last covered it in mid-August. It urges policymakers, governments, the media, and other groups to not buy into “the hype” created by companies who “overstate the capabilities of their products.” As the Millennium Prize controversy raged, OpenAI withdrew its sponsorship of an undergraduate mathematics hackathon at Caltech following fierce opposition decrying the intrusion of corporate interests and worries the event would create a deluge of low-quality “slop mathematics.” “They don’t care anything about us as a community. It’s all about this petty drama between two trillion-dollar companies that are acting like children.” Shing-Tung Yau, a professor of mathematics at China’s Tsinghua University, an emeritus professor at Harvard, and a recipient of the prestigious Fields Medal, told The Verge he worries about the potential effect on young researchers. “Working on hard problems already carries considerable risk for Ph.D. students and junior faculty,” he said. “The prospect of competing with powerful AI companies, whose resources far exceed those available to academic research groups, could make them even more reluctant to pursue ambitious questions.” Yau declined to weigh in on allegations that researchers’ work may have been used by OpenAI, but said he is in favor of an independent review to establish what happened. More broadly, he worries that a lack of transparency and rush to announce first could obscure the intellectual lineage behind a breakthrough. Mathematical credit, he said, should reflect intellectual contributions, not who had budget for the most compute or made the loudest announcement. Yau pointed to another problem, too. OpenAI and other AI companies occupy a peculiar position as both the providers of important research tools and, in a way, researchers. It “raises a serious conflict-of-interest concern,” he said, particularly as they could benefit from privileged access to customers’ unfinished and unpublished work. “That concern deserves a substantive response,” he said. “It should not simply be dismissed as ordinary competition.” A lot of this growing sense of unease comes down to trust. Mathematicians do not have to accept the most explosive allegations against OpenAI to worry about a company that both provides their research tools and, simultaneously, competes with them. “Yeah, quite honestly, I don’t think that some data security announcement or whatever will really solve it,” Thom said. “I don’t really trust them.” Buckmaster felt similarly: “Why should we trust anything they said?” Buckmaster said the reaction from colleagues to his going public had been overwhelmingly positive. But there was an undercurrent of something else too: fear. He told The Verge he initially intended to thank those who supported him when he went public with his experiences. He elected not to after many expressed discomfort at the idea of having their names publicly attached. “The reality is that mathematicians are actually scared of them,” Buckmaster said, referring to the AI companies. And fear is hardly a solid foundation on which to build a productive and healthy research community. Buckmaster isn’t convinced it matters much to those companies involved. “They don’t care anything about us as a community,” he said. “It’s all about this petty drama between two trillion-dollar companies that are acting like children.”