loading
loading
[Ed. note: We’re big fans of data that supports some of the stories we’re hearing around the industry. We’ve been hearing plenty of conflicting information about the value of a harness vs. a model. The folks at Stacklok did some research on agents offered to share their survey results with our community and give a little context on what it all means.] As a co-creator of Kubernetes, I spent a good chunk of my career working to protect enterprises from being trapped by infrastructure. Kubernetes has thrived over the past decade in part because enterprises want to pick their vendors, not belong to them. Does that sentiment persist as we move into the AI-native era? Earlier this month, my team at Stacklok surveyed 520 technical leaders to understand whether they were concerned about being locked into frontier labs. The data says they’re already swapping models in and out, but the trap they’re concerned about is the layer on top of the model. A note on our sample. Every respondent runs LLMs or agents in pilot or production at a North American company with more than 500 employees and holds a manager-or-above title. Nobody is single-model anymore Only 8% of respondent organizations use one model provider, whereas 25% use five or more. It’s already a multi-model world in the enterprise. Lock-in at the model layer appears to be a solved problem. So what's hard to move? The integrations. Fifty-three percent of respondents named "integrations to internal systems or MCP servers" as one of their two hardest elements to migrate. Multi-step agent workflows are right behind at 52%. Guardrails and access-control config come next. And if you move to another model provider, these integrations don’t move with you; they have to be re-built. Lock-in is a concern, but not a buying criteria Eighty-one percent of these leaders agree that frontier labs and hyperscalers make it hard to move between vendors. Eighty-four percent say avoiding lock-in is important to their AI strategy. More than 80% of respondent organizations have already delayed, scaled back, or restructured a migration because switching costs looked too high; 29% say it happens frequently. And when asked how long a migration off their current provider would take, 57% said four months or more. But as someone who thinks a lot about preserving optionality and avoiding lock-in, here’s the part that stings. When asked what actually factors into AI, "Ability to switch model providers without rebuilding" ranked sixth out of eight possible criteria. "Avoiding dependence on a single vendor" ranked dead last. The ranking is rational, and that’s what stings. Security and privacy are the priorities, followed by cost control and compliance. The person choosing the stack today gets judged on this quarter’s bill and this year’s audit, not on a rebuild two years from now; migrations between models and providers is a problem for later (and possibly for a future team). The people running five models have figured something out Here’s where the organizations using five or more model providers look different from everyone else…they are building optionality into their AI architecture now. Let’s compare organizations with five or more model providers to those with just one or two model providers. Seventy-six percent of the former have built integrations to internal systems or MCP servers, versus 42% of the latter. Forty-five percent use open source agent frameworks, versus 22%. Sixty-seven percent say portability across providers is critically or very important, compared to 47%. And 35% say they're "highly likely" to evaluate an open source alternative to the vertically integrated stacks from frontier labs, versus 13%. If your organization is using five or more model providers or will be in the future, it’s time to think about flexibility. Once you run enough heterogeneous things, you stop tolerating a harness that only works with one of them, and you start treating the model the way we eventually learned to treat the node: as a replaceable component behind a stable interface. The teams furthest along didn't get there through principle. They got there because running five providers through five bespoke integration layers is miserable. What respondents want from an open architecture The appetite for a more open alternative is broad. Eighty percent say they'd be likely to evaluate an open alternative to closed frontier architectures, and 88% expect to take more control of their AI architecture as they gain experience rather than continuing to lean on current vendors. They're also specific about the concerns keeping them up at night when building on a closed platform. Thirty-nine percent of respondents mentioned product or API deprecation, and 37% worry specifically about being unable to export agent state, memory, or evals. People are starting to see their agent layer as an asset that can be held hostage. When we asked what would get them to move more workloads to an open architecture, the answers were practical rather than ideological. Security and compliance guarantees, performance parity with frontier models, and an enterprise support model. They want to keep the frontier models they already use and get an open harness with comparable features. What to do next If you're writing MCP servers, agent workflows, or policy config today, you are building the layer this survey says is hardest to move. A few things are worth doing while it's still cheap. Keep tool definitions on the open protocol rather than a vendor's extension of it. Put orchestration in code you own. Make sure your evals can run against any model endpoint. Build (don’t rent) a harness that keeps you in control of identity, memory, tools, and more. And finally, find experts that you trust to help you navigate this fast-changing landscape. The full report is free (no forms) and includes additional data and detail.
document fetched 2026-10-24 11:26 · canonical https://stackoverflow.blog/2026/10/21/survey-says-you-ll-swap-the-model-but-everything-you-build-around-it-is-staying-put