IBM, Together AI and NVIDIA aim to scale open-source AI inference on IBM Cloud under a multi-year deal

IBM and Together AI have announced a multi-year collaboration aimed at scaling open-source artificial intelligence inference, with NVIDIA AI infrastructure deployed on IBM Cloud, according to a report published Tuesday.

Under the agreement, IBM is set up to deploy a large compute cluster to support inference workloads, the report said. Inference is the step where trained AI models are used to generate responses or predictions for users and applications, which often becomes a capacity and cost bottleneck at scale.

The same report described the deal as a multi-year $240 million arrangement between IBM and Together AI. While the article framed the initiative around open-source model serving, it did not provide further breakdowns on compute volume, pricing, or performance targets.

NVIDIA’s role in the collaboration centers on providing AI infrastructure. NVIDIA generally supplies the accelerated computing and software building blocks used to train and run modern AI systems, and the report specifically tied those capabilities to deployments on IBM Cloud.

The announcement adds another data point to a broader pattern in enterprise AI infrastructure, where cloud providers and specialized model-serving vendors seek to make inference more reliable and easier to operate for customers running open-source AI. Compared with training, inference typically requires sustained access to high-throughput compute resources and well-integrated orchestration, which is why partnerships across hardware, cloud, and application layers have become common.

For IBM, the positioning is tied to being a provider of enterprise-grade AI infrastructure. For Together AI, the reported focus on scaling open-source inference implies a business emphasis on helping customers use models in production without needing to manage the full operational stack alone, though the report did not specify which customer segments or deployment models are targeted first.

The report did not detail when the cluster or expansion would begin, what specific open-source model families are in scope, or whether there are service-level commitments such as uptime, latency, or throughput metrics. It also did not disclose revenue attribution, margins, or how the deal may interact with IBM’s existing AI offerings on its cloud platform.

Investors and customers will likely watch for additional specificity as IBM and Together AI move from announcement to deployment, including timelines for the compute expansion, any published benchmarks for inference performance on NVIDIA hardware, and whether the partners clarify how workloads are onboarded, metered, and supported in production.