NVIDIA says its new Vera Rubin NVL72 AI rack debuted in MLPerf Inference with up to 3.7x the throughput of GB300
On September 16, 2026, NVIDIA said its Vera Rubin NVL72 data-center AI rack made its first MLPerf Inference v6.1 appearance, with up to 3.7x the throughput of GB300 NVL72 on one test. This article covers the figures, test conditions and caveats NVIDIA published, and what they mean for ordinary users of AI services. All figures are NVIDIA's own claims.
News date: About 6 min read
Contents
What NVIDIA published
According to an NVIDIA blog post dated September 16, 2026 (author Zhihan Jiang), MLPerf Inference v6.1 results were published that day. MLPerf Inference is a benchmark suite measuring AI inference performance; inference is the computation actually performed when an AI service answers questions or generates images or video. NVIDIA submitted results for its next-generation Vera Rubin NVL72 system for the first time, in preview form, and said the submissions covered tests on two AI models, DeepSeek-R1 and Qwen3-VL.
NVIDIA said that in Qwen3-VL's offline, server and interactive scenarios, Vera Rubin NVL72 running vLLM with the open-source NVIDIA Dynamo inference framework delivered up to 3.7x the throughput of GB300 NVL72; on DeepSeek-R1, using the TensorRT-LLM library, up to 2.5x. NVIDIA noted these figures are in MLPerf's Closed Division and were retrieved from mlcommons.org on September 16. It also said partner Nebius submitted preview results for Vera Rubin NVL72.
Key figures NVIDIA published
| Item | System | NVIDIA's figure | Notes |
|---|---|---|---|
| Qwen3-VL throughput | Vera Rubin NVL72 vs GB300 NVL72 | Up to 3.7x | Preview submission; uses vLLM and NVIDIA Dynamo |
| DeepSeek-R1 throughput | Vera Rubin NVL72 vs GB300 NVL72 | Up to 2.5x | Preview submission; uses TensorRT-LLM |
| SemiAnalysis AgentX | Vera Rubin NVL72 vs GB300 NVL72 | 30x | NVIDIA calls it a preview test, not an MLPerf item |
| Multi-rack scaling efficiency | GB300 NVL72, 72 to 288 GPUs | 99% | DeepSeek-R1, offline scenario |
| WAN 2.2 text-to-video | GB300 NVL72 vs a single node | 0.65 720p videos per second; 9x throughput, 7.5x lower latency | 5.7 seconds per video |
| Software optimization | GB300 NVL72, v6.1 vs v6.0 | Up to 1.6x | Qwen3-VL |
Where the performance comes from: NVIDIA's technical explanation
NVIDIA attributes the results to hardware-software co-design. It says Vera Rubin's enhanced Tensor Cores and Transformer Engine accelerate both inference phases, prefill (reading the prompt) and decode (generating the reply token by token), while the low-precision NVFP4 format cuts the memory footprint of model weights, attention and the KV cache (data held temporarily during inference). NVIDIA says this raises throughput with minimal loss in output quality.
- Splitting the two phases: NVIDIA says its submissions made heavy use of disaggregated serving, handling prefill and decode separately, together with large-scale expert parallelism for mixture-of-experts (MoE) models such as DeepSeek-R1 and Qwen3-VL.
- Fast links inside the rack: NVIDIA says sixth-generation NVLink and NVLink Switch offer 10x the packet rate and 3x lower latency than off-the-shelf Ethernet.
- Continuing software optimization: NVIDIA says GB300 NVL72's gains on Qwen3-VL came from lower KV cache precision, more kernel (low-level compute routine) fusion, better kernels and disaggregated serving.
Other submissions and partners
NVIDIA said it also used TensorRT Edge-LLM to submit results for the Jetson AGX Thor edge device on the newly introduced Edge-Agentic benchmark (using Qwen3.6-27B). NVIDIA added that 19 partners took part, 8 of them on multi-node Blackwell NVL72 systems; the participating partners include ASUS, Azure, Cisco, CoreWeave, Dell Technologies, HPE, Oracle Cloud Infrastructure and Supermicro, among others. It also mentioned the upcoming MLPerf Endpoints benchmark, which it said will provide standardized measurement for AI agent workloads (AI that reasons, plans and acts across multiple steps).
For background, in a separate post dated July 21, 2026, NVIDIA said Vera Rubin NVL72 production is ramping and that the systems are already running at partners including CoreWeave, Google Cloud, Microsoft Azure, Oracle Cloud Infrastructure and Nebius.
What it means for ordinary readers
Vera Rubin NVL72 is a rack-scale data-center system, not a PC or phone component, so ordinary users will not encounter it directly. Its significance is on the back end: the inference computation behind AI answers, images and video runs on this kind of equipment. If NVIDIA's throughput and scaling claims hold in real deployments, cloud providers could in theory serve more users with the same hardware; whether that becomes faster responses or lower prices depends on each provider's decisions, which NVIDIA's post does not address. Remember also that benchmarks reflect specific models and scenarios, and real-world performance may differ.
Frequently asked questions
What is MLPerf Inference v6.1?
It is a benchmark suite that measures AI inference performance. According to NVIDIA's note, MLPerf is a name and trademark of the MLCommons Association, and results are published at mlcommons.org. NVIDIA says the v6.1 results were published on September 16, 2026, and cited Closed Division results.
Does 3.7x mean every AI workload is 3.7x faster?
No. NVIDIA says 3.7x is the "up to" value on the Qwen3-VL benchmark; on DeepSeek-R1, the highest multiple NVIDIA published is 2.5x. Results differ by model and scenario.
Have these results been independently verified?
All figures in this article come from NVIDIA's official website and are the vendor's own claims. NVIDIA says the cited data was retrieved from mlcommons.org, but it also acknowledges that GPT-OSS-120B and DLRMv3 results obtained after the submission deadline are not yet verified by MLCommons, and the Vera Rubin results are preview submissions.
Where does the 30x figure come from?
NVIDIA says that in a preview test on the SemiAnalysis AgentX benchmark, Vera Rubin NVL72 performed 30x as well as GB300 NVL72. This is not an MLPerf result, and NVIDIA's post does not detail the test conditions.
Does the average person need to do anything now?
No. This is data-center-grade hardware, and this article offers no purchasing advice. Ordinary users can treat it as a signal of how AI cloud backend hardware is evolving, and judge by how services actually perform.
Browse the latest news in this topic
Found this guide helpful? Share it with someone who needs it
NVIDIA's AI Infra Summit Power Efficiency Talk: What's New With Vera Rubin and DSX
On September 15, 2026, at the AI Infra Summit in Santa Clara, California, NVIDIA presented the Vera Rubin platform and DSX data-center power-management software, saying the metric for AI infrastructure is shifting from peak performance to tokens produced per megawatt. Based on NVIDIA's blog posts from that day and its January and August press releases, this article checks what was actually new, which numbers are measured, which NVIDIA labels as projections, and what remains unstated.
NVIDIA and Microsoft at Ignite: Vera Rubin NVL72 in Preview on Azure, Nemotron 3.5 in Foundry, Windows 365 Direct Stream in Limited Preview
Google Quantum AI Makes the Cover of Science: Quantum Contextuality Holds at Up to 105 Qubits, With a 98.3% Win Rate in a Coordination Game
Google: ARMORY CASTLE Exploited VMware vCenter Zero-Day CVE-2026-123890, Affecting vCenter 7.x and 8.x; Broadcom Has Released a Patch
Sources
- NVIDIA Vera Rubin NVL72 Delivers Leading Performance in MLPerf Inference v6.1 Debut | NVIDIA BlogChecked:
- Zhihan Jiang Author Page | NVIDIA BlogChecked:
- NVIDIA Vera Rubin Driving Performance Per Watt, Lower Token Costs for Partners Worldwide | NVIDIA BlogChecked:
- NVIDIA Vera Rubin NVL72 Delivers Leading Performance in MLPerf Inference v6.1 Debut | NVIDIA BlogChecked:
- - Archives Page 1 | NVIDIA BlogChecked:
Compiled from the references and verification dates listed above; fares, schedules, product features, and terms can change, so please confirm the latest details on the relevant official website before travelling, booking, or purchasing.