OpenAI unveils Jalapeño, its first custom AI chip, built in collaboration with Broadcom

OpenAI has officially made its way from the software world into semiconductors with the unveiling of Jalapeño, its first ever custom AI processor. Developed in collaboration with Broadcom, the new chip shows a huge transition on OpenAI’s side as the company is aiming for more control over the hardware underlying its AI products including ChatGPT and Codex.

This release marks OpenAI’s entry into an arena that includes other tech titans who have started working on developing custom AI chips. Companies such as Google, Amazon, Microsoft, and Meta have already developed their own proprietary hardware solutions. Hence, OpenAI decided to follow their lead.

Why Did OpenAI Design a New Chip?

Regular AI accelerators target different applications, Jalapeño was developed for inference workloads only. Inference is the process when the model processes the user’s input and generates output based on it. All conversations in ChatGPT, all API requests, and AI code generations rely on inference infrastructure.

With the ever growing use of AI technology and increased number of deployed models, the cost of inference has now become one of the biggest operational expenses of the leading AI providers. This is why OpenAI has decided to design a new inference chip which could improve the efficiency of the process and reduce costs and reliance on outside hardware providers.

Indeed, this initiative is quite unusual for OpenAI as, so far, it has always used Nvidia GPUs both for inference and training purposes. Even though Nvidia is still a dominant player in AI infrastructure and computing, it seems like many large companies want to diversify their technology stack.

Designed in Just Nine Months

One of the most impressive facts about this project is how quickly it has been implemented. According to OpenAI and Broadcom, it took less than nine months to design, prototype, and manufacture the new chip. Such a fast development cycle is quite unusual for any high performance AI chip.

According to the companies statements, some parts of the design process were automated using OpenAI’s own AI systems. While exact details are scarce, this shows how AI is now being used even in such industries as engineering and semiconductor design.

This chip was designed by OpenAI while Broadcom helped with networking and silicon implementation as well as infrastructure integration. Manufacturing is expected to be performed by TSMC. System integration partners will assist in bringing the hardware to production scale deployment.

Better Performance per Watt and Lower Costs

OpenAI has not yet provided any benchmarks for the new chip. Still, preliminary tests have shown that Jalapeño provides much better performance per watt compared to existing AI accelerators. As for Broadcom executives, they claim that the chip is going to compete with some of the most efficient AI accelerators including Blackwell architecture from Nvidia and Tensor Processing Units from Google. Moreover, there are some claims that OpenAI will see drastic drops in inference costs after the chip enters production. However, independent verification has not taken place yet.

Industry experts note that custom chips are often much more efficient due to their specialization and optimization for particular workload.

A Long Term Strategy for Infrastructure

It should be noted that Jalapeño is not described as a temporary experiment. OpenAI sees it as the first step in a long term strategy of developing hardware that would allow supporting future growth of AI infrastructure.

In fact, the company has already signed a strategic cooperation agreement with Broadcom which aims at deploying up to 10 gigawatts worth of OpenAI designed AI infrastructure by 2029. To give you some context, 10 gigawatts of computing resources is a truly monstrous number which would make OpenAI one of the largest consumers of AI hardware resources.

Jalapeño is expected to be deployed initially by the end of 2026 while broader deployment will take place once the production is ramped up. First engineering samples have already been successfully running AI workloads within OpenAI test infrastructure.

Implications for Nvidia

This news might seem very exciting, OpenAI is not planning on leaving Nvidia. Training of frontier AI models is still one of the most computationally complex tasks and Nvidia is still the market leader in that segment.

The fact that OpenAI designed its first chip clearly shows how this trend of diversifying hardware infrastructure is emerging across the industry.

Major AI companies want to have more control over the whole technology stack behind their AI products. Having proprietary chips gives companies more efficiency and bargaining power and helps with avoiding supply issues.

TheTweaks Take

OpenAI’s new chip is yet another evidence that we have entered a new stage of AI competition. Now, success is not guaranteed only by better AI models or larger data sets. Hardware efficiency and infrastructure ownership are becoming the same important competitive advantages.

With AI systems serving millions of users and consuming more and more computing power, companies that own more of their tech stack will get a serious advantage.

Indeed, OpenAI’s Jalapeño is a clear example of it. Visit TheTweaks for more latest tech updates!