IBM CEO says open weight models are now irresistibly good and cheap on-prem

IBM CEO Arvind Krishna has told investors that open weight models now make brilliant sense for on-prem enterprise use. Speaking on the company’s Q3 earnings call, Krishna said that in 2024 closed frontier models were plainly superior to open alternatives. “If you ran some assessment, some benchmark, and it was broad-based, not narrow, the top model might score a 70, and the open model might score a 20 or a 30.” “In late 2025, we began to see that gap shrink, where maybe the gap was not 70 to 20 or 30, it was 70 to 50,” he continued. The CEO said open weight models currently deliver “anywhere from 25 percent to 10 percent of the token cost of the frontier models. That is very attractive given that they’re now 90 percent as effective.” “It’s a simple question for any enterprise. Would you like something that’s 10 percent of the cost if it’s 90 percent as good as the best, or do you want to pay that 10 times higher cost?” Krishna thinks businesses will most often choose not to incur the higher costs. “Then the question is: how many tasks require that? I would tell you some do, but the vast majority, summarizing an email, examining a contract, answering a customer call, composing an email, all of those can be done as well with the much cheaper answer.” The CEO said those numbers mean IBM is seeing booming interest in on-prem AI infrastructure. While he thinks most businesses can build useful AI infrastructure in just two or three racks, he feels many will struggle to get it running. “The question becomes: how do you do that?” he said. “The question that the enterprise struggles with is: ‘Where are my kernel engineers? Where are the people who worry about the underlying network? Who can make sure that the model is clustered on GPUs that are close to each other in network proximity? Who can make sure that you’re doing the right caching of results so you get an incredible response?’” He didn’t have answers, other than to say IBM will produce software that eases AI deployments. Krishna also predicted demand for on-prem AI will spur sales of Big Blue’s Spyre mainframe accelerator cards. The CEO said current-gen z17 mainframes have delivered $1bn more revenue than any previous model, and said Big Blue’s new rackable models delivered “the strongest midrange launch in IBM Z in nearly two decades.” Mainframe revenue, however, fell 20 percent – as IBM expects given it’s now 18 months since it launched the z17. IBM’s infrastructure revenue grew five percent …