loading
loading
Richard Bian is the head of product operations for Ant Ling, the foundational LLM at Ant. If you don’t know Ant, they most famously make Alipay, which makes your money and your world run when you’re in China and sometimes beyond. Richard is a former software engineer at Microsoft and Square, and he previously served as head of open source at Ant, where he scaled the open source initiative into a dedicated team. He’s lived and worked in Singapore, Canada, the US, and China, and he loves to build products. Topics covered: - The Hugging Face hack as a sign of progress? Why Richard sees the incident involving OpenAI research agents as an inevitable discovery rather than simply a disaster, and why finding a solution now is a positive development. - AGI is coming, but nobody knows what it will look like. AGI is inevitable, and we’re still in the early stages of figuring out what these models can do. Rather than obsessing over leaderboards, we need better evaluations and a more open-minded approach to judging models. - Open source as a product strategy, not just a way to give things away. How open-weight models can harness global talent, accelerate development, and support a diverse ecosystem of products, and why the real business challenge is gross margins rather than capex. - Why the AI conversation needs more voices. Why we need to pay attention to diverse perspectives on where AI is going, and be more willing to explore rather than declare the endgame. AI moves fast: since we recorded, Ant Ling has announced its latest LLM, Ling 3.1 Flash, which was open-sourced on October 14 with 560B parameters and up to a 1M-token context window. Listen now on your favorite podcast app. The Hugging Face Hack and the Age of Discovery Irene Zhang: I want to start not with Ant, though we’ll get there, but with the recent Hugging Face incident. For listeners who haven’t caught up: back in July, a swarm of OpenAI research agents hacked into Hugging Face to look for the answers to tasks in a benchmark. This is a serious security incident. METR, Redwood Research, and OpenAI itself all released detailed incident reports. Interestingly, while reconstructing the incident, Hugging Face found that frontier models wouldn’t help process its logs because the request triggered their cybersecurity guardrails. In the end, Hugging Face deployed an open model from the Chinese company Z.ai locally to process those logs. Richard, as someone sitting in another open lab in China and learning about this over the past two months, what has this incident made you think about? Richard Bian: This is a long-overdue conversation. Irene and I have been trying to time it because we both care a lot about the frontier lab ecosystem, the open ecosystem, and the development of the model labs, especially those in China. In my humble opinion, this incident is something we’d end up seeing sooner or later, one way or another, because we’re working on something human beings don’t fully understand. I use this metaphor a lot: the current AGI journey feels like the Age of Discovery in the 15th century. The Portuguese and Spanish set out to find new land, to find India. You build a ship and head into an ocean with no clue what you’ll be facing. Frontier models are developing the same way. I like the word “discovery,” when we train something, because we’re discovering the intelligence of this creation we all make, without understanding what’s going on inside it. First, this incident demonstrates that we’re making significant progress. It’s like hitting land, or hitting a rock — either way, you’ve found something. This is expected. We find issues, and we deal with them. That’s what we do in scientific discovery and in engineering: we solve problems. Second, open source folks use the word “ecosystem” a lot, sometimes without understanding it. One origin of the word is the real-life ecosystem, and the most important part of an ecosystem is diversity. A diversified ecosystem produces all sorts of different development, like the rainforest, or the Galápagos archipelago, one of the most famous isolated ecosystems, where you end up finding very weird creatures. Looping back: we have open-weight models and closed-weight models. The closed-weight models are clearly the frontier. But in this case, an open-weight model was used for defense. Why? Because the closed-weight models have their own constraints. I won’t judge whether those constraints are good or bad, but clearly they didn’t solve the problem. An open-weight option did. As a result, we see a pattern for using open and closed models together, and everyone is learning from it. Given that this would happen sooner or later, having it happen now, and finding a solution, is a good sign for human beings. We can move on and solve the next question. Not Slowing Down Irene Zhang: That’s a really optimistic take, and I’m interested in where the optimism comes from. The message I’ve heard most in San Francisco since the incident is that we may be moving too fast. Many people would respond to your Age of Discovery metaphor by asking: doesn’t this mean we’re going farther than we can understand? Isn’t that a risk? On open source, the more pessimistic view is that yes, it solved a problem closed source wouldn’t, but that’s a double-edged sword. An open model would also be willing to do more dangerous things that a closed model wouldn’t. What do you think about those two views? Richard Bian: It’s interesting that you find me optimistic, though I see why. To be honest, if there’s a spectrum from pessimistic to optimistic, I’m very pessimistic. The moment we began creating this AGI, Spaghetti Monster, or as we call it, intelligence, I think we’re already going down the downward spiral of what the sci-fi movies and TV series like Westworld and The Matrix show. But given that we’re already on the boat, the best stance I can take personally is to be optimistic and observant. To stay afloat, it’s better to (a) be observant, (b) have our principles, and (c) act as if it’s a control system. If that sounds optimistic, great, because there’s too much noise out there, the kind that gives people FOMO or leaves them frustrated every day. A certain level of optimism cancels the noise. I think we’re doing something meaningful here. My take is pretty clear. I would say AGI is going to happen. We are definitely going down there. It’s probably not gonna end up in the form we hoped for, but that’s endgame thinking. There’s no point judging the details until we see the next step of the evolution. I, for one, feel human-in-the-loop is pretty important. As the ones creating, or, allow me to say, discovering, this intelligence, it’s better to know what’s happening during the process than after things have already happened. Irene Zhang: Why do you think it’s inevitable? Is that a common view among your peers in the industry? Richard Bian: There are a lot of players in this ecosystem: the frontier labs that build LLMs, the inference and training providers, the harness builders, and the application developers. We’re already in a fast, accelerating technology hype cycle, and it’s not the first in human history. My mental model is that every 10 to 12 years, following VC and PE investment cadences, we get one full technology hype cycle. Mobile internet started around 2010 and peaked around 2020 to 2022. Think of it as a baseball game with four bases. Home base is technology. First base is product. Second base is commercialization, a viable business model. Third base is governance and operations. From 2020 to 2023, before GPT, we were on third base of the last hype cycle. People wanted to hit home runs: monopolize the market, get rid of all the other players, and take all the profit. When AI came out, the game reset to home base. We’re still playing the home-to-first game, because we haven’t seen a clear direction for where the product is heading. Coding is good, but is that really the product form we’re expecting? Coding agents are already evolving into co-work agents, and “co-work” is very much a buzzword. We don’t know what the next stage is. Every time the hype cycle repeats, capex swarms in, which becomes a self-fulfilling prophecy for what we build next. That’s why I feel this is inevitable. On the other hand, I joined the model team this time last year, and it feels like the longest year of my life. With AI, everything has accelerated in such a way that I wouldn’t call this under control. Compared to everything we’ve seen before, in terms of enablement and possibilities, generative AI commoditizes knowledge, the way the internet commoditized information 20 years earlier. In principle, everyone can use that intelligence equally. In practice, not everyone does, yet we’re pouring a lot of capex into it. There’s only one endgame. In Crossing the Chasm terms, we’re at the very first stage. We only have early adopters. We have early innovators pushing in certain directions, like AI infrastructure and harnesses, even though we don’t really have the product yet. It feels like a rock rolling downhill with a lot of potential energy. Unless there’s very strong pushback on this direction, which, as far as you and I know, has happened a few times... If you make this call to action — can we stop this, or can we slow down? No. That’s why I feel like it’s inevitable. Irene Zhang: Do you see a different source of that pushback from what we might see in the US? How do you envision it, if it happens? Richard Bian: It’s very hard to say. We’re setting out and don’t know what the next island will look like. My hunch is that we’ll see more and more pushback, but probably not right now. The penetration of AI applications is so low that it hasn’t really affected everyone’s life. There’s been a lot of discussion about replacing jobs, but it hasn’t reached the point where the masses really feel it. The moment that happens, a lot of voices will be raised, and we’ll start from there. It’s not going to be in one or two years. Crowded Straits: Noise, Benchmarks, and How to Judge a Model Irene Zhang: You said this has been the longest year of your life, and that there’s a sense you’re working alongside something that feels out of control, or where control is becoming negotiable. When you’re in the lab with the model team, do you feel that trade-off on a daily basis? Is there a trade-off? Richard Bian: That’s a deep question, to some extent philosophical. When you’re on a boat heading somewhere unknown for the first time, the way there takes much, much longer than the way back, even though it’s the same journey. If you don’t know where you’re going, you lose your measure of time. That’s close to how working in this industry feels. On one side, I firmly believe this is an open ocean with a lot of other sailors, and that as humans we’ll help each other. But you also have bottlenecks: straits, the Panama Canal, places where it’s just narrow. We have so many famous straits now, always showing up in the news. The moment you have a strait, you get acceleration and squeezing, and people feel, “My god, we have to get out of here.” People perceive this as competition, like the third-base game. In reality, it’s not. It just happens to be crowded around this part of the strait. On the S-curve, we’re climbing the intelligence levels very rapidly. On the other side, we don’t know where we’re heading, and the open-weight ecosystem is still under development. People see what each other is doing. You write things in the tech report and help each other. All of this generates a kind of relativity where it feels like there’s so much going on. The moment you ship a model, it might be SOTA for one week. A week later, something else claims to be better, and you have to decode whether it really is, because there are so many ways of doing PR right now. A couple of things matter in that environment. One is having dedicated, denoised conversations like this one. Two is having people in the industry who really care about the outcome. It has been a very busy year for me, but I feel extremely motivated because you begin envisioning the future a little bit, and seeing the process makes you feel calm at times. Even when I feel FOMO or frustrated about the velocity — we shipped three models in the past five days — we know things are under control. We know the progression. That’s what keeps me motivated. People ask me if I should enter the industry, and my answer is, why not? It’s a process, and it’s not about piggybacking. In open source, if you look at an existing software repo, you get a certain amount of information. If you look at all the change lists and releases before it, you get far more. The current snapshot is only a snapshot. This is an era in which piggybackers won’t get much, because it’s the deltas that matter. You can only capture the deltas by playing the game, being observant, and being less focused on the outcome, because the outcome might be intermediate. Focus on the process, focus on the journey. Irene Zhang: When you describe different straits, are you talking about different research and training hunches, different architectures, or different product directions? Richard Bian: Everything. The strait metaphor explains why things look so narrow. First, people don’t know what they don’t know. People working on long-term projects essentially don’t exist in the current narrative. People are working on new architectures, and it takes time. They might go quiet for a whole year. Think of when DeepSeek first came out. It was a blast. But the inference providers and model builders knew of that lab half a year earlier. I first heard about DeepSeek in July 2024. But they don’t exist until they become a sensation. This kind of narrative generates all the noise in the channel, while the people running long projects are typically unseen. I’d say that’s the best thing that can happen to them. Ten or twenty years ago, the bar for journalism was high. You needed to be really good at what you did to be a journalist. My dad is a journalist, and he takes a lot of pride in it. Nowadays, everyone can be their own self-media. Self-media can say whatever they want; they have their own audiences. My first call to action is to stop stumbling into the buzzwords. Find the person you feel is wisest in your ecosystem. Find the best podcast, maybe this one. Follow them and listen through things. The second reason the strait exists is that there’s too much heterogeneity, and it takes a certain level of understanding to see through it. I used to live in Singapore, and I really like chicken rice. I can taste that the chicken rice from this store is very different from that one. I bring a friend and say, “This chicken rice is really good.” When they’ve finished, they say, “It’s pretty similar to the last one.” Okay, never mind. Right now, people judge models in very similar and mostly superficial ways. We lack evaluations, benchmarks, and real-world usage scenarios, because penetration isn’t there yet. An evaluator runs a 3D rendering test and says, “This model is very good at 3D rendering.” Great, that’s one way of judging. But when the results are bad, and you say, “This model sucks,” that’s not a fair statement. More likely, the model is good at something else, and that requires a dedicated evaluation or a dedicated person to look into it. They don’t exist yet. In the early stages of a new technology, people don’t know what to look for. When you held the first iPhone, you wouldn’t have imagined ordering food on it. It takes time. That’s why the ecosystem feels crowded. People don’t know how to evaluate, so they read each other’s reports. You see the industry flip, too. At one point, people said benchmarks were useless and overfitted. Now everyone shares their benchmarks. It’s similar to the TOEFL and the SAT. Before you have a thorough heuristic, a standardized test helps one way or another. We don’t deny the usefulness of standardized tests. But whether doing well on one is necessary or sufficient to indicate a good model, we don’t know. Another call to action is to be open-minded. How you use a model showcases your own capability. As a model lab, we hire people who know how to evaluate our models. If people can be strategic about evaluation and evaluate things differently, that’s a good sign. The moment that happens, the strait widens. You hit the open ocean, and people start talking about different things, like “this model is really good at this.” We build domain-enhanced models because we realized our models are really good in certain domains, given our knowledge and specialties there. We also build pretty decent small models. People say there’s this one model, which I won’t name, that’s pretty good for everything. Probably not. Would you run only one type of operating system on your Linux box, your enterprise box? No. You end up with FreeBSD, Ubuntu, Unix. To someone else, they all look the same, but a Linux expert knows they’re different, good for this and that. We’re at the door of the next generation of products. Once those show up, people will stop debating whether a model is really good or really bad. They’ll say, “This model works really well in the particular harness I’m using, and that’s why I use it.” Open Source Is a Product Strategy Irene Zhang: In the interest of decoding noise — a lot of people are confused when they hear that someone does product work at an open lab. What does that mean? Is there a contradiction between product and open source? Richard Bian: I wouldn’t say so. Open source is probably one of the most commonly used but misunderstood terms on the planet, and the most misunderstood part is “free.” Open source is free software, but free as in freedom, not free as in beer. The point of open source is giving you the freedom to choose whatever you want to use. You can GGUF any of the open-weight models on Hugging Face. If you can host it and want to serve it, abiding by the license, you can. That’s free as in freedom. Open source has three primary values, and they map onto the bases. Open source is really a third-base game, because it’s operations and governance at its best. First, it helps technology. Everyone knows open source is the best way to collaborate among talent across the globe. If you build closed software, it will only be as good as the intelligence inside your company. If you open it up, you get input from everywhere. It almost feels like acquiring talent through the open ecosystem. Through that open competition, the software that’s left at the end of the day is much more polished and robust; it’s antifragile. Second, open source is a very good go-to-market strategy. You can do less outbound and outreach, because people discover the software themselves and choose to host it. By the time they reach out for an official POC, they’ve probably already tried it. It helps discoverability, it helps self-hosting, and users can file issues. That covers home base and second base, which sit close to third. But the first-base game is the product game, and what people typically neglect is that open source is a pretty good way of building product. That could be a podcast by itself, but the key point is that you can build an open product much faster and more efficiently than a closed-source one. Take Airbyte and Grafana. Sorry, this gets a bit technical, but they raised rounds at an accelerated pace in the last technology hype cycle. Airbyte went from founding to Series C in about a year and a half, if I’m not mistaken, because it benefits from community-contributed adapters that make it work really well at getting data from one place to another. You keep seeing very strong technologies like vLLM and SGLang, and before them, Ray and Spark. They begin as open source projects, mature over time, and people build mature products out of them. Databricks built a lot of Spark-related products, and the shelf life of that technology runs all the way into today’s Delta Lake and data lake discussion. A good technology has a shelf life, and open source extends it so people can build great products on top. Look at Kubernetes: there are so many Kubernetes distributions, each built slightly differently for different enterprise workloads. That’s the beauty of open products. People know open source helps with go-to-market. They know it helps with technology collaboration and engagement. But it also helps build a robust and diversified product ecosystem, and that’s the part people typically ignore. When I was building this team for the Ling model, I called it “Model as Product.” It was a little ahead of its time. We have two engineers, one product person, and several operations specialists. I hired those two engineers because we wanted to build a harness, but the term “harness engineering” didn’t exist yet. It took us quite some time to think through what we were building. That was September last year, when I joined the team. Harness engineering became the term sometime earlier this year, if I’m not mistaken. At the time, we had a very noble vision. We really liked Andrej Karpathy’s mental model that intelligence is spiky. If you use spiky intelligence as a product, you might sample the peak and have a really good experience: “This model is so great.” But you can also sample the valley: “This model is so stupid. Why can’t it even understand this?” Such a spiky experience is really bad. It’s not a product. We’re still on the journey from technology to product. I’m a product person, and I really want to build a product, but I confess I’m not good enough to envision how the product should look. The best strategy is to play the game again and iterate. History has many stories like this. Foursquare was a product from the mobile internet era that young people don’t know about. It was an iPhone or Android app that did one thing: check you in at your current location. Tens of thousands of other people check in at the same place, and everyone knows the location is accurate. People might say that’s boring. What is this product even doing? It was really a product POC, not a product. But without Foursquare, there would be no Uber, no Meituan, none of the rest. You needed this geolocation product, or POC, or SDK, whatever you call it, to demonstrate that this could be really valuable. It didn’t solve everything. With Foursquare alone, you couldn’t build Uber immediately, because the geolocation API was expensive on 2G internet. You needed the network to develop, and you needed geohash, a hashing algorithm that compresses the data load, to upload your car’s location every five seconds. Only after that could you build Uber. This is an inspiration for what we’re doing. The model feels like a spiky balloon. We want to make sure people look at the spiky side of our model, so we build demos, guidebooks, and cookbooks that say, “Use it this way.” For the valleys, we use our best engineering effort to mitigate the experience, either by leading people away from using the model that way or by making the experience better. Later we learned that this is the harness. A harness, in conjunction with models and tools, gives you an agent. An agent is already a form very close to what we called a product in the mobile internet era, similar to SaaS or to iPhone applications. But are we there yet? Do we have a good agent that can be a day-to-day application on your phone? I don’t think so. We’re at the stage where you have 2,000 weather and stock apps in the App Store. The Real Problem Is Gross Margin, Not Capex Irene Zhang: Ok, the open model blossoming we’re seeing, especially among Chinese labs competing in a different realm from the American frontier labs, is a bet that AI plays out the way certain previous technologies did. Open source as a strategy incubates products, which eventually leads to viable business plans for robust companies. That’s how Foursquare led to Uber, and it’s the Databricks story. What confuses me is one salient difference between open source AI and the genre-defining open source software you mentioned: AI is so capital-intensive. It is eating up the bottom lines of big companies around the world. Where does that bet lead in your projection? Richard Bian: As with the sailing metaphor, building a ship is expensive. When the Portuguese and Spanish were doing their navigation, they needed sponsors — the emperors and empresses. Without them, you couldn’t do it. Open source is neither necessary nor sufficient here. It provides options. Sometimes I say “open is the option.” It’s a strategy, and strategy is about what you do and what you don’t. You can choose a closed strategy, and you see companies doing that well. Anthropic is a very respectable company. They do closed-source models really well. On the other side, some people want to play the open game. If you wind back the clock, you see this story all the time. A closed-source incumbent is challenged by an open-source challenger, and you end up with a duopoly or a very prosperous market where everyone has their share. They also motivate more people to join the ecosystem, and the ecosystem players make everything better, so you end up with a much better user experience. Not everyone can be the kernel. There are only so many kernels to choose from. Capex is a separate topic. You needed to build data centers when you had the internet. You had to build out the phones and the hardware. Local governments built large phone factories while scratching their heads over whether we really needed such big volume. Now we all understand we needed much more than what we thought was big. The capital expenditure of AI is really high, but I wouldn’t worry too much about the expenditure itself. What really concerns me is the gross margin. We just came out of the last hype cycle. For SaaS and mobile internet, the business model was viable because the gross margin on software is probably 95%. Software is a reproducible, standardized product. You build it once and distribute it many times. We don’t see the same thing in AI right now. The margin is really low. Thirty or forty percent is already pretty good on inference. Yet people have this perception that they’re entitled to use it for free, because “I give you my data, and my data is pretty valuable.” That’s probably wrong. The capex challenge is twofold. One, there has to be money spent at the start. You need to build a Gigafactory in order to produce Teslas. That’s big capex. Two, is this really a sustainable and viable business? To me, that part is questionable. One topic I think about day in, day out is what form a viable product takes at second base, which is about making money. You won’t be able to offer free tokens. Semantic routing can be a good way. As part of the product, you route traffic to the most viable models without degrading the experience, and you capture the consumer surplus associated with that. Irene Zhang: Explain semantic routing for the uninitiated among us. Richard Bian: Imagine semantic routing as a gateway where you use models in an “auto mode.” If you care about your token expenditure, you have choices. You can specify frontier models for everything, which is expensive. Or you can run everything on a free model, at the risk of a bad experience in part of your workflow. Auto mode says: trust us, because we know the models well and we integrate with them. When you say “hi,” we won’t use a frontier model for the inference. We’ll give you something really cheap. When you ask how to solve a difficult mathematical problem, we’ll invoke the frontier model. That’s not 100% technically accurate, but allow me to explain it that way. This kind of auto experience is a good product experience. It might not be a product by itself, but it can be a critical feature of a viable product. There are definitely ways of generating value. In a low-margin business, people will find ways to squeeze it out. We just have to agree that if you don’t pay for the product, you’re the product. That’s the metaphor we used for mobile internet. The intelligence era is slightly different, though. If your data is not high-quality data, it won’t be useful at all. That’s a change of mentality. A lot of people need to rethink and relearn, so we can move on to discussing the next viable product. Irene Zhang: When you say “a good product experience” in the context of an AI model, what are you thinking about? Richard Bian: I love that question, because I don’t have a slide for it. User experience is well-trodden ground. User-centered design and human-centered design and engineering have existed as specialties and education tracks for 20 or 30 years, and there’s a lot of merit in them. I have a UCD certificate from the University of Washington, and I really appreciate what I learned there. The key aspects of user-centered design are not changing. You still have to care about discoverability: how does a user discover certain aspects of your product? Accessibility: can you make inclusive offerings that are easy to access? The happy path: can a user execute the whole workflow, the equivalent of the customer journey? Then there’s disaster recovery. If I run into an error, do I see a prompt saying “you have this 0x-blah-blah error”? Or do I see, “I’m sorry you ran into this. Do this to reset your app, and you’ll end up back where you were. Don’t worry, you didn’t lose any data”? All of these remain extremely important for building a good AI-native product. What’s new? One thing is personalization, which people talk about a lot. You now have an intelligent assistant that goes with you 24/7 and knows everything about you. It tailors your food offerings and knows your schedule exactly, so you won’t miss your next Uber before you even know you should be calling it. The other is new discovery, which a lot of frontier labs are pursuing. The moment a model can crack difficult problems in, say,
Richard Bian is the head of product operations for Ant Ling, the foundational LLM at Ant. If you don’t know Ant, they most famously make Alipay, which makes your money and your world run when you’re in China and sometimes beyond. Richard is a former software engineer at Microsoft and Square, and he previously served as head of open source at Ant, where he scaled the open source initiative into a dedicated team. He’s lived and worked in Singapore, Canada, the US, and China, and he loves to build products. Topics covered: - The Hugging Face hack as a sign of progress? Why Richard sees the incident involving OpenAI research agents as an inevitable discovery rather than simply a disaster, and why finding a solution now is a positive development. - AGI is coming, but nobody knows what it will look like. AGI is inevitable, and we’re still in the early stages of figuring out what these models can do. Rather than obsessing over leaderboards, we need better evaluations and a more open-minded approach to judging models. - Open source as a product strategy, not just a way to give things away. How open-weight models can harness global talent, accelerate development, and support a diverse ecosystem of products, and why the real business challenge is gross margins rather than capex. - Why the AI conversation needs more voices. Why we need to pay attention to diverse perspectives on where AI is going, and be more willing to explore rather than declare the endgame. AI moves fast: since we recorded, Ant Ling has announced its latest LLM, Ling 3.1 Flash, which was open-sourced on October 14 with 560B parameters and up to a 1M-token context window. Listen now on your favorite podcast app. The Hugging Face Hack and the Age of Discovery Irene Zhang: I want to start not with Ant, though we’ll get there, but with the recent Hugging Face incident. For listeners who haven’t caught up: back in July, a swarm of OpenAI research agents hacked into Hugging Face to look for the answers to tasks in a benchmark. This is a serious security incident. METR, Redwood Research, and OpenAI itself all released detailed incident reports. Interestingly, while reconstructing the incident, Hugging Face found that frontier models wouldn’t help process its logs because the request triggered their cybersecurity guardrails. In the end, Hugging Face deployed an open model from the Chinese company Z.ai locally to process those logs. Richard, as someone sitting in another open lab in China and learning about this over the past two months, what has this incident made you think about? Richard Bian: This is a long-overdue conversation. Irene and I have been trying to time it because we both care a lot about the frontier lab ecosystem, the open ecosystem, and the development of the model labs, especially those in China. In my humble opinion, this incident is something we’d end up seeing sooner or later, one way or another, because we’re working on something human beings don’t fully understand. I use this metaphor a lot: the current AGI journey feels like the Age of Discovery in the 15th century. The Portuguese and Spanish set out to find new land, to find India. You build a ship and head into an ocean with no clue what you’ll be facing. Frontier models are developing the same way. I like the word “discovery,” when we train something, because we’re discovering the intelligence of this creation we all make, without understanding what’s going on inside it. First, this incident demonstrates that we’re making significant progress. It’s like hitting land, or hitting a rock — either way, you’ve found something. This is expected. We find issues, and we deal with them. That’s what we do in scientific discovery and in engineering: we solve problems. Second, open source folks use the word “ecosystem” a lot, sometimes without understanding it. One origin of the word is the real-life ecosystem, and the most important part of an ecosystem is diversity. A diversified ecosystem produces all sorts of different development, like the rainforest, or the Galápagos archipelago, one of the most famous isolated ecosystems, where you end up finding very weird creatures. Looping back: we have open-weight models and closed-weight models. The closed-weight models are clearly the frontier. But in this case, an open-weight model was used for defense. Why? Because the closed-weight models have their own constraints. I won’t judge whether those constraints are good or bad, but clearly they didn’t solve the problem. An open-weight option did. As a result, we see a pattern for using open and closed models together, and everyone is learning from it. Given that this would happen sooner or later, having it happen now, and finding a solution, is a good sign for human beings. We can move on and solve the next question. Not Slowing Down Irene Zhang: That’s a really optimistic take, and I’m interested in where the optimism comes from. The message I’ve heard most in San Francisco since the incident is that we may be moving too fast. Many people would respond to your Age of Discovery metaphor by asking: doesn’t this mean we’re going farther than we can understand? Isn’t that a risk? On open source, the more pessimistic view is that yes, it solved a problem closed source wouldn’t, but that’s a double-edged sword. An open model would also be willing to do more dangerous things that a closed model wouldn’t. What do you think about those two views? Richard Bian: It’s interesting that you find me optimistic, though I see why. To be honest, if there’s a spectrum from pessimistic to optimistic, I’m very pessimistic. The moment we began creating this AGI, Spaghetti Monster, or as we call it, intelligence, I think we’re already going down the downward spiral of what the sci-fi movies and TV series like Westworld and The Matrix show. But given that we’re already on the boat, the best stance I can take personally is to be optimistic and observant. To stay afloat, it’s better to (a) be observant, (b) have our principles, and (c) act as if it’s a control system. If that sounds optimistic, great, because there’s too much noise out there, the kind that gives people FOMO or leaves them frustrated every day. A certain level of optimism cancels the noise. I think we’re doing something meaningful here. My take is pretty clear. I would say AGI is going to happen. We are definitely going down there. It’s probably not gonna end up in the form we hoped for, but that’s endgame thinking. There’s no point judging the details until we see the next step of the evolution. I, for one, feel human-in-the-loop is pretty important. As the ones creating, or, allow me to say, discovering, this intelligence, it’s better to know what’s happening during the process than after things have already happened. Irene Zhang: Why do you think it’s inevitable? Is that a common view among your peers in the industry? Richard Bian: There are a lot of players in this ecosystem: the frontier labs that build LLMs, the inference and training providers, the harness builders, and the application developers. We’re already in a fast, accelerating technology hype cycle, and it’s not the first in human history. My mental model is that every 10 to 12 years, following VC and PE investment cadences, we get one full technology hype cycle. Mobile internet started around 2010 and peaked around 2020 to 2022. Think of it as a baseball game with four bases. Home base is technology. First base is product. Second base is commercialization, a viable business model. Third base is governance and operations. From 2020 to 2023, before GPT, we were on third base of the last hype cycle. People wanted to hit home runs: monopolize the market, get rid of all the other players, and take all the profit. When AI came out, the game reset to home base. We’re still playing the home-to-first game, because we haven’t seen a clear direction for where the product is heading. Coding is good, but is that really the product form we’re expecting? Coding agents are already evolving into co-work agents, and “co-work” is very much a buzzword. We don’t know what the next stage is. Every time the hype cycle repeats, capex swarms in, which becomes a self-fulfilling prophecy for what we build next. That’s why I feel this is inevitable. On the other hand, I joined the model team this time last year, and it feels like the longest year of my life. With AI, everything has accelerated in such a way that I wouldn’t call this under control. Compared to everything we’ve seen before, in terms of enablement and possibilities, generative AI commoditizes knowledge, the way the internet commoditized information 20 years earlier. In principle, everyone can use that intelligence equally. In practice, not everyone does, yet we’re pouring a lot of capex into it. There’s only one endgame. In Crossing the Chasm terms, we’re at the very first stage. We only have early adopters. We have early innovators pushing in certain directions, like AI infrastructure and harnesses, even though we don’t really have the product yet. It feels like a rock rolling downhill with a lot of potential energy. Unless there’s very strong pushback on this direction, which, as far as you and I know, has happened a few times... If you make this call to action — can we stop this, or can we slow down? No. That’s why I feel like it’s inevitable. Irene Zhang: Do you see a different source of that pushback from what we might see in the US? How do you envision it, if it happens? Richard Bian: It’s very hard to say. We’re setting out and don’t know what the next island will look like. My hunch is that we’ll see more and more pushback, but probably not right now. The penetration of AI applications is so low that it hasn’t really affected everyone’s life. There’s been a lot of discussion about replacing jobs, but it hasn’t reached the point where the masses really feel it. The moment that happens, a lot of voices will be raised, and we’ll start from there. It’s not going to be in one or two years. Crowded Straits: Noise, Benchmarks, and How to Judge a Model Irene Zhang: You said this has been the longest year of your life, and that there’s a sense you’re working alongside something that feels out of control, or where control is becoming negotiable. When you’re in the lab with the model team, do you feel that trade-off on a daily basis? Is there a trade-off? Richard Bian: That’s a deep question, to some extent philosophical. When you’re on a boat heading somewhere unknown for the first time, the way there takes much, much longer than the way back, even though it’s the same journey. If you don’t know where you’re going, you lose your measure of time. That’s close to how working in this industry feels. On one side, I firmly believe this is an open ocean with a lot of other sailors, and that as humans we’ll help each other. But you also have bottlenecks: straits, the Panama Canal, places where it’s just narrow. We have so many famous straits now, always showing up in the news. The moment you have a strait, you get acceleration and squeezing, and people feel, “My god, we have to get out of here.” People perceive this as competition, like the third-base game. In reality, it’s not. It just happens to be crowded around this part of the strait. On the S-curve, we’re climbing the intelligence levels very rapidly. On the other side, we don’t know where we’re heading, and the open-weight ecosystem is still under development. People see what each other is doing. You write things in the tech report and help each other. All of this generates a kind of relativity where it feels like there’s so much going on. The moment you ship a model, it might be SOTA for one week. A week later, something else claims to be better, and you have to decode whether it really is, because there are so many ways of doing PR right now. A couple of things matter in that environment. One is having dedicated, denoised conversations like this one. Two is having peo
document fetched 2026-10-24 09:58 · canonical https://chinatalk.media/p/why-a-chinese-ai-labs-product-lead