OpenAI's Jalapeño Chip Challenges Nvidia's AI Dominance

The long-standing dominance of Nvidia in the advanced AI chip market is facing new challenges. A growing number of tech giants, including OpenAI, Google, AWS, and Meta, are now announcing their own custom-designed semiconductors.

OpenAI recently unveiled its inaugural AI chip, the Jalapeño, which it claims offers "industry-leading speed and efficiency." This development comes as Nvidia has experienced a significant surge in its stock price, driven by the immense demand for its chips in data centers for both AI model training and day-to-day AI operations (inference).

However, major cloud providers and AI companies are making substantial progress in creating their own silicon to power AI systems. Adrien Sanchez, a technology analyst at Yole Group, stated that the Jalapeño chip, specifically designed for inference, demonstrates that "a hyperscaler-designed chip can now match or beat Nvidia's Blackwell-class GPUs on inference efficiency."

While Nvidia still holds a substantial majority of the AI compute market and benefits from its robust CUDA software ecosystem, Sanchez pointed out that OpenAI's new chip represents a "threat to Nvidia's inference margins, which is the field growing the most at the moment."

OpenAI has released initial benchmarking results for Jalapeño, suggesting it will enable users to experience "faster responses, more responsive agents, and more reliable access" as demand for AI services escalates. This new chip is being developed in collaboration with Broadcom and is slated for deployment within OpenAI's infrastructure by the end of the year. Furthermore, OpenAI has indicated that it is already working on the second and third generations of this semiconductor.

Industry-Leading Efficiency

OpenAI first announced Jalapeño in June, describing it as a chip that would be "built from the ground up for current and future LLMs across the industry." Alexander Harrowell, a senior principal analyst at Omdia, lauded Jalapeño as an "impressive achievement, most of all in terms of efficiency." He elaborated that in large-scale deployments, this efficiency would translate into significant savings on power, cooling, and infrastructure, thereby improving unit economics.

Fion Chiu, an analyst at TrendForce, suggested that OpenAI's custom chip could eventually reduce its dependence on Nvidia for inference workloads. However, for more demanding tasks such as large-scale model training and frontier AI development, Chiu believes "Nvidia GPUs will remain important given their broad programmability, performance, software ecosystem, and ability to handle a wide range of workloads."

OpenAI's Jalapeño Chip vs. Nvidia

The research firm SemiAnalysis reported on its visit to OpenAI's labs to benchmark Jalapeño. Their findings indicated that the chip surpassed Blackwell in performance per watt across most tested scenarios. However, they cautioned that the comparison was "somewhat incomplete and unfair" as Jalapeño utilizes newer HBM4 memory. A more direct comparison would be against Nvidia's Rubin platform, which also uses HBM4. SemiAnalysis analysts noted in a blog post that "Jalapeño is really competing against chips like Rubin that also use HBM4." They also pointed out that while "Vera Rubin systems are starting to ship to customers right now, while it will still be some time before OpenAI has anything beyond engineering samples of Jalapeño."

Competitors Gain Ground on Nvidia

OpenAI is not alone in its pursuit of custom AI silicon. In April, a wave of deals for custom AI chips, also known as application-specific integrated circuits (ASICs), were announced. Google revealed new chips for AI training and inference, its tensor processing units (TPUs). Meta announced a significant deal to deploy 1 gigawatt of custom AI chips using Broadcom technology. Anthropic committed to spending over $100 billion on AWS technology over the next decade, including Amazon's custom AI chips like Trainium.

Harrowell of Omdia anticipates that custom ASIC chips like Jalapeño will "exceed GPUs in volume by 2028, although revenue will take much longer as GPUs are considerably more expensive." He further emphasized that this trend represents "the biggest competitive threat to NVIDIA, as about half the capital expenditure on AI infrastructure comes from hyperscale cloud providers who either have a custom chip program or could reasonably have one."

Numerous startups, including Cerebras, SambaNova, D-Matrix, Etched, and Fractile, are also actively developing chips for AI applications. OpenAI has historically been a major purchaser of Nvidia's GPUs for training and running its large AI models. The development of its own chip could significantly alter this relationship. Sanchez highlighted that OpenAI has been "one of the largest single consumers of Nvidia GPUs," and Jalapeño "raises the stakes for Nvidia's largest customer relationship specifically."