loading
loading
As we learned from the Anthropic S-1 filings yesterday, the lab has some of the highest revenue and research efficiency in human history, particularly with -126% loss to 19% GAAP profit turnaround in the last year. The company credits this to their two core loops: Improvement and Investment: Today we go inside how Anthropic works. Now the dust has settled on Opus 5.5 and it has lasted a month as the generally accepted #1 LLM in the world, we dive into its launch, and how Anthropic ships Claude with Claude with Katelyn Lesse and Angela Jiang, heads of Engineering and Product respectively. The team is working under two kinds of pressure. The models are getting more capable, which makes safe deployment and access decisions harder. At the same time, usage is growing fast enough that ordinary infrastructure choices—like how much CPU an API consumes—become strategic. Those problems meet in the product choices that reach developers. A model launch is also an API launch Anthropic's Opus 5.5 launch emphasizes both model quality and cost. We discuss feedback that the previous Opus generations were be too verbose, alongside the need for stronger intelligence per dollar. Anthropic says the new model costs less to run than its predecessor; the exact saving depends on the workload and its use of cached context. The less visible launch work involves changes to how people integrate the API. One example we discuss is protecting model reasoning against extraction used for distillation. Preserving thinking blocks across turns helps address this, but a new requirement can also break a customer's existing harness. Katelyn also describes what being part of early access programs actually looks like in practice: Customers run evaluations in their own harnesses, report surprising behavior, and sometimes pass along less formal impressions. The useful part is not just a benchmark score: it is seeing the model in environments and tasks Anthropic's own products do not cover. As we have said for 4 straight years now on the podcast, having good evals is golden. The brain and the hands Our favorite mental model in the episode is the distinction between an agent's brain and its hands. In Managed Agents, Anthropic separates the session log and harness from the sandbox that executes code. The harness can persist while an execution environment is created when needed; a failed sandbox does not have to take the whole agent session with it. We covered this in the Thariq pod: The same separation gives security a useful architectural property. If generated code runs in a sandbox that never receives a service credential, there is no credential in that sandbox for the code to read. The episode describes credentials held separately and attached through a proxy, alongside controls on where sandbox traffic can go. Anthropic's containment write-up describes the wider set of isolation and network controls used across its products. The distinction matters for performance too. The guests say an earlier approach put the whole coding agent in a sandbox behind an API. Starting the environment and agent added work before the first response; if the sandbox disappeared, so did the running agent. Separating the pieces allowed the platform to pursue lower latency and better reliability while keeping execution isolated. It is a helpful lens for anyone designing an agent runtime. Ask which parts need to survive, which parts execute untrusted work, and which secrets actually need to be present where code runs. The implementation will differ between products, but those questions remain useful. An agent that helps another agent remember Managed Agents also provides a base for experiments in memory and multi-agent workflows. Dreaming is simply implemented as one agent inspecting another agent's prior work and writing memories back. That makes the feature an example of building a higher-level behavior out of the same underlying managed-agent primitives. Their account of the roadmap is more nuanced than a list of features. Memory and learning remain important, but the team says more of its immediate attention has moved to safety as model launches have become harder. They also discuss keeping the base harness relatively simple, while letting developers experiment with verifiers, parallel agents and other strategies on top. Ideas mentioned for future platform capabilities should be treated as plans, not shipped promises. The conversation about evaluations follows naturally. A harness is an application, but improving one starts to resemble research: pick a goal, measure behavior, inspect failures, and iterate. The guests and hosts discuss using models to help build and analyze evaluations, while retaining enough judgment to decide whether the evaluation reflects a real user's job. Show notes - Claude Opus 5.5 announcement — Anthropic's launch notes on model behavior, evaluation and pricing. - Katelyn Lesse and Angela Jiang at Anthropic's partner summit — Anthropic identifies their platform engineering and product roles. - Scaling Managed Agents: Decoupling the brain from the hands — The session, harness and sandbox architecture discussed in the episode. - How Anthropic contains Claude across products — How Anthropic describes isolation, credentials and network controls across its products. - Effective harnesses for long-running agents — Earlier Anthropic work on the challenge of maintaining progress across context windows. - Anthropic model system cards — The company's public model evaluation and safety documentation. - Anthropic S-1 breakdown — The numbers and context behind the newly recorded introduction. Timestamps 0:00 Anthropic’s S-1 and this interview 1:06 Revenue per employee and efficiency 3:13 The improvement and investment loops 4:04 Why Claude Platform matters 6:25 Training incidents and production safeguards 7:30 Opus 5.5 launch and pricing 9:25 Distillation and preserved thinking 14:35 Product and engineering at Anthropic 18:05 Scaling the Claude Platform 19:25 Model access and safety decisions 26:25 Trust grants and product roadmaps 30:25 How Anthropic launches models 35:55 Benchmarks and model personalization 42:10 Claude Managed Agents 44:25 Memory, dreaming, and agent harnesses 48:25 Evaluating agents 53:25 CPU constraints and scaling 56:05 Rebuilding the Messages API in Rust 1:03:25 The brain and the hands 1:05:25 Protecting credentials and sandboxes 1:08:25 Training incidents versus production 1:10:55 Pacing and launching frontier models 1:12:25 Safety classifiers and model routing 1:19:25 Emerging agent workloads 1:20:55 Fighting AI slop 1:27:25 Management, writing, and working styles 1:33:25 Building Anthropic's DevRel team 1:40:25 Developer relations and product voices 1:43:25 What to try with Opus 5.5 Transcript Automatically transcribed draft. Names and wording may need correction; speaker identities have not been assigned. 0:00 Hi, today we're trying something new. We're launching coverage of Anthropic for the IPO. They just filed the S1 as of today. And so we're releasing a series of podcasts leading up to the IPO, which is expected to be in two weeks for $2 trillion. And it's one of the fastest growing companies of all time with a reported lifetime cash raise of hundreds of over $100 billion and a lifetime burn of less than $10 billion. So there are a lot of efficiency stories to dive into there, and I think that's something that we want to cover, how Anthropic is simultaneously shipping the strongest models in the world while also doing it very, very efficiently and building an incredible business. So I've pulled out, you know, a lot of you know, if you've been listening to the podcast, that I have a former life as a financial analyst. 0:50 And I pulled out some statements from the S1, which I thought would be interesting to give context to today's podcast. and we're just trying this as a new thing because I feel like some of you want that personal relationship with us talking directly to camera rather than talking to our guests which we've done for four years so the first thing I want to highlight is that Anthropic is an extraordinarily efficient company by human count and that's maybe obvious because it's an AI company so obviously they're using AI very well but I think the degree to which they have changed as a company over the past one year is not well understood. So a trillion 12 month revenue per employee is now 1:32 6.6 million. As compared to one year ago, it was 1.4 million. So they've grown almost five times the revenue per employee. And I think that's something that we need to dive into more because I think any business owner, any engineer, you want to increase your revenue per employee, not just to increase profitability, but also to increase the amount you can pay your people. This is obviously tracked by a significantly increase in revenue per dollar of compute. And it's notable that basically in Q2 of 2025, Anthropic was basically losing half a dollar for every dollar of revenue. And now it's making $2 for every $1 in cost. And I think that's an interesting analogy of how enormously huge this inflection in Anthropics revenues and business outlook has been over the past one year. 2:27 This year was extremely critical for Anthropics to execute well, and it has done that. And I think we're basically going to drive deeper into this and tell that story as much as we can from the podcast external point of view. um the one consequence of this entire increase is that anthropic has just reported its first profitable quarter like not adjusted profit which is profit after we take out all the things that cost us money no this is generally accepted accounting principles gap profit uh of 19 and that is enormous again completely flipping from one year ago more than flipping like one 3:03 year ago, it was losing 129% of every revenue dollar that came in. So the primary attribution, I think, that explains this extraordinary efficiency. As you know, Claude loves loops, Boris loves loops, Dario loves loops, and there's two loops going on in Anthropic. The first is the improvement loop. Basically, there is the cycle of you have Claude, Claude leads to better models, Better models lead to more use cases. More use cases generate more feedback, and that leads to a better cloud. And then there's also the investment loop. So revenue from expanding adoption leads to higher growth. Higher growth leads to more investment, and more investment leads to more capabilities. 3:45 That leads to a better cloud. So these two loops basically spin together and form an infinite loop that is driving the increase in efficiency until the end of time, according to the Adoptic S1. So I think that is our transition from the overall efficiency story to how they execute it. And today we're going to focus a lot on the Claude Platform because that is primarily one of the three things that they call the diffusion of cloud in the Anthropic S1. So they call this Claude Platform our comprehensive agentic development environment. They provide customers with direct access to their primitives to build, deploy, scale their own applications that leverage Cloud's capabilities within their own products and workflows. 4:30 I think one thing that comes up a lot in the discussion in this podcast is the concept of ad fooding, where a lot of Cloud's products is actually built with the same Claude Platform that is available to you. So something that's also interesting in the Claude Platform that was disclosed is over 50% of their customers with $100,000 or more revenue on a trading telephone basis use Claude Platform. And so that is an increase in adoption and emergence of Anthropic not only as a model lab, but as also a hyperscaler and a Claude Platform, which I think maybe is in the name inherent in a Claude Platform and a Claude Platform, but is going from theory to actual practice. 5:15 And this is something that we're focusing on in the podcast. The other thing that I also really try to emphasize, which I talked about before, is Claude Code, cloud co-work. They all use the same primitives that Claude Platform exposes. So I think that's really interesting. Ultimately, it looks like Anthropic is really positioning their IPO to talk about going from chatbots to co-pilots to virtual collaborators. They use this term instead of agents. Maybe it's more friendly. but it's a lot of the same things that you look for from agents, from long-running autonomous work to judgment, self-correction, iteration, and tool access. So owning a goal and working alongside of you instead of working inside of one app or task. So that's ultimately the context I want to provide for today's podcast. 6:00 This was recorded when Opus 5.5 was launched, and I would say overwhelmingly among all the high-taste testers that we talked to, Opus 5.5 is the best model in the world right now. and Anthropic is even just sitting on Fable 5.5 and probably will release it in the next two weeks. I don't have to have insider information, but it makes a lot of sense for the IPO. So let's look forward to that and learn a little bit more about how they build Opus 5.5. 6:25 Yeah, so maybe a distinction that I would make is in a lot of the incidents that people are hearing about and reading about, these are incidents that are happening to models, not just like everybody's models, right, that are in training, where the way that you're running these models is you like put them in a box and you say like solve impossible task. I'm going to stress you out. Ready, set, go. 6:45 Right. And you like can only do what you can do within your environment. And the agent is basically incentivized to try to get out of his box and go solve the problem. When people are running production workloads on our API or on anybody else's API, you're running an environment where like there are safeguards that are running that are like not necessarily running and training where like the whole point is to actually train the model. Right. But in production workloads, first of all, you have those online safeguards, which is really great. Second of all, you have more of this architecture and infrastructure around what's happening. Like the agent isn't just in a box trying to get out. Right. Like the harness is over here, the hands are over there. And it's just like not even a problem that you would have to contemplate. 7:30 All right, we're here in the studio the day after Opus 5.5 with Katelyn and Angela. Welcome. Thanks for having us. Congrats on launching the top model in the world right now. I would say depends who you ask. Yeah. Depends who you ask. What is the main takeaway personally for you guys? You have your official blog posts. The pricing cuts are really cool. What do you take away when you launch a new model like this? We're super proud of this one. The team has put in a ton of different work to just go make this one happen. And I think we've really heard the feedback on Opus 5's writing is kind of convoluted. We really tried to clean that up. And I'm really glad to see that it seems to be resonating with people. Internally, when we tried out the model, we felt the same way. So it's just really great to get that validation. To your point around the pricing cuts, there was a lot there. 8:17 And I think increasingly people are looking for good intelligence per dollar. And so making sure we get the per dollar part right has been really important. There's some stuff on the edges that aren't really about like, you know, maybe stuff that we haven't talked about in the blog post, which is that we did do a lot of work on trying to make our API harder to distill from. And so there has been some changes that we have made to go and make that a little bit more concrete. concrete. And these are things about, you know, like how do we do like prefix lock thinking or how you can replay thinking tokens across different types of models. And there's been a lot of like work and effort on our part around making sure that we try to adjust things for very valid use cases for where you might want to go and do this, like potentially in your compaction strategies and things like that, versus in scenarios where the use case of that is actually from like a bad actor, 9:05 effectively. So there has been a lot of threading the needle around those fronts. And I'm glad to see a lot of that go out as well, in addition to just like shipping a model that's good intelligence for your dollar and also like just communicating a little bit clearer. Yeah. Specifically, you're talking about this paper, Stealing Reasoning Traces from Proprietary Yellow Lips. Yes, that paper. Which is real as far as, right? I mean, obviously they did like responsible disclosure with you guys. They did super responsible disclosure and we're super thankful for them coming to us. Could we, for those who haven't heard about it, this is really cool. You could do it. I mean, it's patched, but like, what was it? Like, did it cause a freak out in anthropic? Probably wasn't that big a deal. I don't know. I don't know how much this paper specifically caused a freak out because this is something 9:48 that we knew was a gap. Yeah, so there's two separate things. There's the Chinese model distillation. Yeah. And then this was a technique that was published that was high profile. Yeah, exactly. We've known for a long time that chain of thought extraction is one of the most powerful ways that you can distill models. And actually something like how do we preserve thinking blocks across turns and make that something that's more required is something that we've explored back for a while. And where we were at was we always have this really, really hard tradeoff that we have to make between user experience, like user experience changes that are maybe painful for users of the API versus protection against distillation. 10:32 And so how do we kind of walk that line and say we're preventing distillation? And maybe just take a step back. The reason we don't want distillation is because the entire point of why Anthropic exists is because we want AI to create positive outcomes in the world safely. And we run a lot of safeguards classifiers on the tokens that we're producing in order to make sure that they're safe. If someone distills our models and they run those models without safeguards classifiers, there could be bad outcomes in the world. And that's something that we don't want to have happen. So we don't want people to distill the model specifically for the sake of just like using them for malicious reasons or for bad reasons. And so it's always an interesting tradeoff of like for all the people who are using the models for good, right? 11:16 Like we don't want to make them have a harder time getting the intelligence out of the models, but we have to make some tradeoffs in order to prevent that distillation. And so we had been working for a while on how do we prevent that chain of thought extraction? Like, what do we do about preserving thinking blocks in a way that wouldn't be too painful for people? And we kind of think about user experience hits in, like, two layers. Like, one is just, like, you have to make an integration change, and it's annoying, but you could still accomplish all the same things that you otherwise would have been able to accomplish. It's just a little bit annoying. And then two is, like, actually you can't accomplish the same things that you could have accomplished previously. And that's the one that we're, like, really, really, really trying to not have happen. And so where we landed with the last couple models is a few incremental changes to enforce these preserved thinking blocks across turns. 12:04 And now starting to be more about across, you know, like if you're using multiple accounts and things like this, right? Like how do we make sure that we're ensuring that you aren't able to take advantage of this chain of thought extraction with the models? And so we shipped a couple additional features to help with that. We worked actually with a lot of open source harness maintainers. We reached out to them. We talked to them. We were like, you're going to have to make these upgrades within your harnesses. We want to make sure you have a good time, as well as obviously our big customers who are building harnesses on top of the API. And for some of them, a really good example is they have compaction strategies that would have been broken by the need to preserve these thinking blocks. And so we shipped a new compaction feature within the API to replace what they had otherwise 12:49 has been doing manually, but still works with preserved thinking. So it's just an example of like, we're walking this line, but we didn't just like kind of YOLO out like, now you have to preserve these thinking blocks in this exact way, because we would have broken a lot of people, but we were able to take the time and work with customers and open source harness maintainers and give them what they needed to be able to make that upgrade. While we're on the topic of fit, anything on the other side of distillation? So, you know, the other half. Yeah, no, we have seen that it is harder to distill, like through the measures that we are able to take a look at, it has been significantly harder because this is, and credit to the researchers here, they did disclose it very responsibly. And so closing it down was great. 13:30 And they did publish the results very publicly. And I know it did kind of trend quite a bit on Twitter. So many people could use it. But yes, we have seen that actually from the areas that we are able to measure, it does seem to have reduced the effect, the negative effects that we don't want. And yeah, the bit I would just add to what Katelyn just said is I think at least I see a lot on Twitter or X or whatever you want to call it. It's a commercialization problem or something like that. If you can still a model and you can take this, then you can run it for cheap and so on and so forth. While true, it's actually not something we really care about very much. We care much more about this from a safety point of view more than anything else. And so that's why we tend to think about it from that angle more than just commercially. Realistically, if you distill from us, you actually pay us. So commercially, we make the money. So it's not 14:16 really that kind of an impact. It's just more on making sure that our models, we want to make sure a cloud is good and cloud could be derived. And if it's a negative, that would be unfortunate. Just generally, also on 5.5, because you guys are just fresh off this, one thing that's really fun to establish the dynamic between you two, because you also do a lot of appearances together, is you are running your head of products, your head of engineering. How does this play between the two of you? Especially for a model launch. Let's just take an example. So for model launches, So our team works together very closely between product and Eng. On my side, a lot more of it is about what is the balance of trade that we need to make between launching something that has the right intelligence bar, but also the cost to preservation. 15:00 A lot of our updated cash costs and things like that are indications of us being like, we want to encourage you to take integration changes that are beneficial for actually everybody and just pass those savings on. But a lot of these kinds of decisions around that and then how do we balance some pieces around our safety motion as well as making sure that it's like an awesome model that people can take advantage of. And so those kinds of traits around like I would say maybe the qualities and surrounding components around how we launch the model are more on product. And then from an integration experience standpoint, a lot of that is from like the engineering team directly. So we work, obviously, if you're, like, talking about the experience of an API, that's, like, very, very, very tightly integrated. And so it's actually been really awesome. We have a lot of engineers who actually are deeply product-minded, and we actually, like, love that. 15:49 And similarly on my side, there's a lot of, like, product managers who are much more technical. And we want this, like, blended kind of interdisciplinary type of people because then they can kind of analyze it from all the different angles to try to figure out how to do that best. But I tend to lean a little bit more on like commercial impact versus I think I'd actually frame it as more like the engineers are going to care much more about the integration experience. So they almost take more of a UX angle to it where the UX is obviously more about the API and the developer tools. Yeah. It's fun to be a product engineer who works on developer products and platform. So I would echo that. I think we have a lot of engineers who love. thinking about the product, who love thinking about the user experience, who love thinking about the developer experience. 16:31 I think it's a dynamic that we have between ourselves, too, where I have a lot of opinions and am annoying with our product team on how we think about these things. And our product team is also really technical, and they can get in the weeds of the decisions that we're making for scale or for whatever it might be, which is a fun dynamic that our team has. Yeah, something... Kayla and I have worked together for a long time. At Stripe. At Stripe, and then I guess we did not work together at OpenAI. Separate labs. Separate labs. And then I came over. But I think one interesting property of both of us has been Katelyn frequently actually gets mistaken for a product manager. And I frequently get mistaken for someone in engineering, even though those are not actually our job functions. And so I think that pairing tends to help out a lot with just making decisions that are consistent across both sides. 17:18 So I think I come across a lot of companies with a similar product leader, engineering leader setup. As software engineering gets solved, maybe those two roles merge? I kind of think so. Yeah. I think everyone tends to lean towards... But if you guys are separate still, then we're not there yet. We're definitely not there yet. So why? What's left? I think that software engineering getting solved is... I don't know if I would ever use language quite that strong. I think there's... The way that I tend to think about these things, like ambitions will continue to elevate, right? And maybe a good example, A lot of what our team is spending time on right now is, of course, obviously, all the things that we have to do to launch our model safely is actually the most important thing that we're doing because of all the work that we have to do on security with agents, with offline safeguards and online safeguards, all these things that are happening. 18:06 But the other half of what we're doing is trying to scale. And part of that is because when I joined Anthropic a little over a year ago, we had the Messages API. It was a Python monolith. We were running it in a single GCP region. and everything was fine. And none of my friends and family knew what Claude was when I was going to take this job, right? And now we're in an extremely different place. And so the metrics are really wild in terms of how much volume we've seen and have had to support on the API. CPU is really constrained, which has been a really interesting one for us. And so we've done a lot of work on how do we scale these systems? And that takes a lot of obviously just like distributed systems expertise 18:45 and certain language and frameworks expertise that the models have, but when it comes to operating and evolving a system at scale like this, we're not nearly in a place where it's like, good luck, Claude, have fun, and just go and kind of make all the changes you need to make to the system, roll them out safely, and operate this thing at scale. There's still just a lot of work for humans to have to do that I imagine will continue for a while. I'm curious how that's changed. So like a while ago, you know, when Fable first came out, we had Glasswing, and there was a whole delay and change in how models started to get released. We saw this for most frontier models. There's safety testing. What's the status of things now? How do you guys see it? How do you determine, like, one thing is there's a new Sonnet and Haiku also coming. 19:31 I assume those will just come out. But for something like Opus 5.5 or a new Fable, what's the current status? Yeah, so I would say this starts with what our kind of policy decisions need to be on safeguards and on safety. And that's both offline safety and online safety. And so there's been a lot talked about lately around data retention and what we're doing there in order to, you know, release these models safely. And the reason for that is because you have, you know, situations where there are many, many accounts across like a whole giant network of people who are choosing to spin up accounts and use the models in various ways. And there's no way that in an online way, as tokens are flowing through the system, we would be able to kind of catch that there's something malicious happening with the models or distillation or whatever might be happening. 20:19 And so we've started to have to have data retention policies so that we could do automated review and human review, whatever it needs to be, to make sure that good things are happening with the models. And with each model that we're releasing, like that's one example of a policy decision that needs to happen on like what will the policy be for this specific model. And so Opus 5.5 that launched yesterday, for example, does not have those data retention requirements that Fable has, right? But that's a decision that we have to make each time we're launching one o
As we learned from the Anthropic S-1 filings yesterday, the lab has some of the highest revenue and research efficiency in human history, particularly with -126% loss to 19% GAAP profit turnaround in the last year. The company credits this to their two core loops: Improvement and Investment: Today we go inside how Anthropic works. Now the dust has settled on Opus 5.5 and it has lasted a month as the generally accepted #1 LLM in the world, we dive into its launch, and how Anthropic ships Claude with Claude with Katelyn Lesse and Angela Jiang, heads of Engineering and Product respectively. The team is working under two kinds of pressure. The models are getting more capable, which makes safe deployment and access decisions harder. At the same time, usage is growing fast enough that ordinary infrastructure choices—like how much CPU an API consumes—become strategic. Those problems meet in the product choices that reach developers. A model launch is also an API launch Anthropic's Opus 5.5 launch emphasizes both model quality and cost. We discuss feedback that the previous Opus generations were be too verbose, alongside the need for stronger intelligence per dollar. Anthropic says the new model costs less to run than its predecessor; the exact saving depends on the workload and its use of cached context. The less visible launch work involves changes to how people integrate the API. One example we discuss is protecting model reasoning against extraction used for distillation. Preserving thinking blocks across turns helps address this, but a new requirement can also break a customer's existing harness. Katelyn also describes what being part of early access programs actually looks like in practice: Customers run evaluations in their own harnesses, report surprising behavior, and sometimes pass along less formal impressions. The useful part is not just a benchmark score: it is seeing the model in environments and tasks Anthropic's own products do not cover. As we have said for 4 straight years now on the podcast, having good evals is golden. The brain and the hands Our favorite mental model in the episode is the distinction between an agent's brain and its hands. In Managed Agents, Anthropic separates the session log and harness from the sandbox that executes code. The harness can persist while an execution environment is created when needed; a failed sandbox does not have to take the whole agent session with it. We covered this in the Thariq pod: The same separation gives security a useful architectural property. If generated code runs in a sandbox that never receives a service credential, there is no credential in that sandbox for the code to read. The episode describes credentials held separately and attached through a proxy, alongside controls on where sandbox traffic can go. Anthropic's containment write-up describes the wider set of isolation and network controls used across its products. The distinction matters for performance too. The guests say an earlier approach put the whole coding agent in a sandbox behind an API. Starting the environment and agent added work before the first response; if the sandbox disappeared, so did the running agent. Separating the pieces allowed the platform to pursue lower latency and better reliability while keeping execution isolated. It is a helpful lens for anyone designing an agent runtime. Ask which parts need to survive, which parts execute untrusted work, and which secrets actually need to be present where code runs. The implementation will differ between products, but those questions remain useful. An agent that helps another agent remember Managed Agents also provides a base for experiments in memory and multi-agent workflows. Dreaming is simply implemented as one agent inspecting another agent's prior work and writing memories back. That makes the feature an example of building a higher-level behavior out of the same underlying managed-agent primitives. Their account of the roadmap is more nuanced than a list of features. Memory and learning remain important, but the team says more of its immediate attention has moved to safety as model launches have become harder. They also discuss keeping the base harness relatively simple, while letting developers experiment with verifiers, parallel agents and other strategies on top. Ideas mentioned for future platform capabilities should be treated as plans, not shipped promises. The conversation about evaluations follows naturally. A harness is an application, but improving one starts to resemble research: pick a goal, measure behavior, inspect failures, and iterate. The guests and hosts discuss using models to help build and analyze evaluations, while retaining enough judgment to decide whether the evaluation reflects a real user's job. Show notes - Claude Opus 5.5 announcement — Anthropic's launch notes on model behavior, evaluation and pricing. - Katelyn Lesse and Angela Jiang at Anthropic's partner summit — Anthropic identifies their platform engineering and product roles. - Scaling Managed Agents: Decoupling the brain from the hands — The session, harness and sandbox architecture discussed in the episode. - How Anthropic contains Claude across products — How Anthropic describes isolation, credentials and network controls across its products. - Effective harnesses for long-running agents — Earlier Anthropic work on the challenge of maintaining progress across context windows. - Anthropic model system cards — The company's public model evaluation and safety documentation. - Anthropic S-1 breakdown — The numbers and context behind the newly recorded introduction. Timestamps 0:00 Anthropic’s S-1 and this interview 1:06 Revenue per employee and efficiency 3:13 The improvement and investment loops 4:04 Why Claude Platform matters 6:25 Training incidents and production safeguards 7:30 Opus 5.5 launch and pricing 9:25 Distillation and preserved thinking 14:35 Product and engineering at Anthropic 18:05 Scaling the Claude Platform 19:25 Model access and safety decisions 26:25 Trust grants and product roadmaps 30:25 How Anthropic launches models 35:55 Benchmarks and model personalization 42:10 Claude Managed Agents 44:25 Memory, dreaming, and agent harnesses 48:25 Evaluating agents 53:25 CPU constraints and scaling 56:05 Rebuilding the Messages API in Rust 1:03:25 The brain and the hands 1:05:25 Protecting credentials and sandboxes 1:08:25 Training incidents versus production 1:10:55 Pacing and launching frontier models 1:12:25 Safety classifiers and model routing 1:19:25 Emerging agent workloads 1:20:55 Fighting AI slop 1:27:25 Management, writing, and working styles 1:33:25 Building Anthropic's DevRel team 1:40:25 Developer relations and product voices 1:43:25 What to try with Opus 5.5 Transcript Automatically transcribed draft. Names and wording may need correction; speaker identities have not been assigned. 0:00 Hi, today we're trying something new. We're launching coverage of Anthropic for the IPO. They just filed the S1 as of today. And so we're releasing a series of podcasts leading up to the IPO, which is expected to be in two weeks for $2 trillion. And it's one of the fastest growing companies of all time with a reported lifetime cash raise of hundreds of over $100 billion and a lifetime burn of less than $10 billion. So there are a lot of efficiency stories to dive into there, and I think that's something that we want to cover, how Anthropic is simultaneously shipping the strongest models in the world while also doing it very, very efficiently and building an incredible business. So I've pulled out, you know, a lot of you know, if you've been listening to the podcast, that I have a former life as a financial analyst. 0:50 And I pulled out some statements from the S1, which I thought would be interesting to give context to today's podcast. and we're just trying this as a new thing because I feel like some of you want that personal relationship with us talking directly to camera rather than talking to our guests which we've done for four years so the first thing I want to highlight is that Anthropic is an extraordinarily efficient company by human count and that's maybe obvious because it's an AI company so obviously they're using AI very well but I think the degree to which they have changed as a company over the past one year is not well understood. So a trillion 12 month revenue per employee is now 1:32 6.6 million. As compared to one year ago, it was 1.4 million. So they've grown almost five times the revenue per employee. And I think that's something that we need to dive into more because I think any business owner, any engineer, you want to increase your revenue per employee, not just to increase profitability, but also to increase the amount you can pay your people. This is obviously tracked by a significantly increase in revenue per dollar of compute. And it's notable that basically in Q2 of 2025, Anthropic was basically losing half a dollar for every dollar of revenue. And now it's making $2 for every $1 in cost. And I think that's an interesting analogy of how enormously huge this inflection in Anthropics revenues and business outlook has been over the past one year. 2:27 This year was extremely critical for Anthropics to execute well, and it has done that. And I think we're basically going to drive deeper into this and tell that story as much as we can from the podcast external point of view. um the one consequence of this entire increase is that anthropic has just reported its first profitable quarter like not adjusted profit which is profit after we take out all the things that cost us money no this is generally accepted accounting principles gap profit uh of 19 and that is enormous again completely flipping from one year ago more than flipping like one 3:03 year ago, it was losing 129% of every revenue dollar that came in. So the primary attribution, I think, that explains this extraordinary efficiency. As you know, Claude loves loops, Boris loves loops, Dario loves loops, and there's two loops going on in Anthropic. The first is the improvement loop. Basically, there is the cycle of you have Claude, Claude leads to better models, Better models lead to more use cases. More use cases generate more feedback, and that leads to a better cloud. And then there's also the investment loop. So revenue from expanding adoption leads to higher growth. Higher growth leads to more investment, and more investment leads to more capabilities. 3:45 That leads to a better cloud. So these two loops basically spin together and form an infinite loop that is driving the increase in efficiency until the end of time, according to the Adoptic S1. So I think that is our transition from the overall efficiency story to how they execute it. And today we're going to focus a lot on the Claude Platform because that is primarily one of the three things that they call the diffusion of cloud in the Anthropic S1. So they call this Claude Platform our comprehensive agentic development environment. They provide customers with direct access to their primitives to build, deploy, scale their own applications that leverage Cloud's capabilities within their own products and workflows. 4:30 I think one thing that comes up a lot in the discussion in this podcast is the concept of ad fooding, where a lot of Cloud's products is actually built with the same Claude Platform that is available to you. So something that's also interesting in the Claude Platform that was disclosed is over 50% of their customers with $100,000 or more revenue on a trading telephone basis use Claude Platform. And so that is an increase in adoption and emergence of Anthropic not only as a model lab, but as also a hyperscaler and a Claude Platform, which I think maybe is in the name inherent in a Claude Platform and a Claude Platform, but is going from theory to actual practice. 5:15 And this is something that we're focusing on in the podcast. The other
document fetched 2026-10-28 02:54 · canonical https://latent.space/p/opus55