Nvidia's Blackwell GB300 hits 1,648 TFLOPs per GPU as software optimizations compound

Nvidia's GB200 NVL72 rack systems running DeepSeek R1 0528 achieved a 4x improvement in tokens per second per megawatt over three months, driven entirely by software optimizations that the company says apply to more than 90 percent of AI models.

"These results come from a platform whose hardware, interconnect, and software are designed together and continuously optimized," Nvidia said in a technical blog post detailing the benchmarks. "The record-setting performance of today is only the foundation for even higher performance tomorrow."

The gains stem from 38 major optimizations tested across 250,000 simulated configurations, consuming 1.4 million GPU hours. On the training side, the GB300 NVL72 — Nvidia's Blackwell Ultra rack — set a world record on DeepSeek-V3 671B pre-training at 1,648 TFLOPs per GPU across 256 GPUs, nearly three times the 606 TFLOPs delivered by the prior-generation GB200. The same hardware has improved 1.5x in six months through software updates alone.

The dual-generation strategy — extracting more value from deployed Blackwell systems while rolling out Vera Rubin, which offers roughly 10x the token throughput at equivalent power — strengthens Nvidia's competitive position against Advanced Micro Devices and custom ASIC rivals such as Google's TPU and Amazon's Trainium. Nvidia shares trade at roughly 35x forward earnings, and the continued software-driven performance gains could push analyst estimates higher as customers see extended return on invested capital.

Software optimizations compound across frameworks

The performance improvements are not limited to Nvidia's proprietary Megatron Core framework. On TorchTitan, PyTorch's native training stack, the GB300 NVL72 delivered 1,197 TFLOPs per GPU on DeepSeek-V3 671B — a 6x improvement versus the unoptimized baseline of 199 TFLOPs. JAX, the Google-backed framework popular in research, showed an even steeper trajectory: throughput rose to 4,082 tokens per second per GPU in July 2026 from 418 in January, a 10x gain that translates to 1,025 TFLOPs per GPU.

Nvidia engineers contributed directly to both open-source frameworks, landing optimizations that compound over time. The company said it has run more than 250,000 simulated configurations to identify the most effective changes, with over 90 percent of the 38 optimizations applicable across different AI models — meaning customers running Llama, GPT, or other architectures benefit without additional engineering work.

Scaling efficiency approaches theoretical limits

As frontier models shift to mixture-of-experts architectures — where each token activates only a subset of parameters — communication between GPUs has become the primary bottleneck. Nvidia's GB300 NVL72 addresses this with fifth-generation NVLink, giving each GPU 1.8 TB/s of bandwidth and 130 TB/s of non-blocking all-to-all bandwidth across the 72-GPU rack.

The result is near-linear scaling from 256 to 1,024 GPUs: Megatron Core maintains 98.5 percent efficiency, while TorchTitan and JAX each hold 97 percent. The 800 Gb/s Scale-Out networking chip within each NVL72 rack ensures gradient traffic stays hidden behind compute, so adding GPUs strictly increases total throughput rather than saturating the network.

Vera Rubin, Nvidia's successor architecture, is already entering global deployment, delivering approximately 800,000 tokens per second at 150 megawatts — roughly 10x the 80,000 tokens per second of the GB200 NVL72 at equivalent power. But the company is not sunsetting Blackwell. The strategy mirrors the Hopper generation: continue optimizing software for deployed hardware even as newer architectures ship, extending customer return on investment and deepening the platform's software moat.

For hyperscale cloud providers and enterprise customers who have invested billions in Blackwell infrastructure, the 4x energy efficiency improvement translates directly to lower cost per token and higher utilization rates. That economics matters as AI training workloads continue to scale — DeepSeek-V3 671B, with 671 billion total parameters and 37 billion activated per token, represents the kind of model that benefits most from Nvidia's tightly coupled scale-up architecture.

This article is for informational purposes only and does not constitute investment advice.