Putting an end to years of speculation, Microsoft has officially revealed its entrance into the custom silicon arena with the introduction of two innovative Arm-based processors.
The announcement confirmed longstanding rumors surrounding “Project Athena,” an initiative aimed at reducing Microsoft’s reliance on off-the-shelf hardware from vendors like Nvidia.
The unveiling of the two “homegrown” chips, namely the Microsoft Azure Maia 100 AI Accelerator and the Cobalt 100 CPU, cements the tech giant’s commitment to revolutionizing AI and general computing workloads within the Azure cloud.
The Maia 100 AI Accelerator, a direct outcome of Project Athena, stands out with its impressive specifications. Built on TSMC’s 5nm process and featuring a staggering 105 billion transistors, this chip is specifically designed for running large language models such as GPT-3.5 Turbo and GPT-4.
Noteworthy is its support for various MX data types, including sub-8-bit formats, promising faster model training and inference times.
In comparison to industry counterparts, the Maia 100 AI Accelerator boasts a significant transistor count, surpassing Nvidia’s H100 AI Superchip and falling slightly short of AMD’s Instinct MI300X. However, direct performance comparisons remain pending, leaving room for speculation on how it will fare against existing chips in the AI services domain.
One remarkable feature of the Maia 100 is its aggregate bandwidth of 4.8 Terabits per compute unit, achieved through a custom Ethernet-based network protocol. This innovation enables superior scaling and end-to-end performance, positioning Microsoft for a potential leap in the competitive AI landscape.
Adding to the lineup, Microsoft introduced the Cobalt 100 CPU, a 64-bit, 128-core Arm-based processor focusing on general Azure computing workloads.
Leveraging Arm Neoverse Compute Subsystems, this chip promises a substantial 40 percent performance improvement over current-generation hardware in commercial Arm-based servers. Cobalt 100-based servers are slated to power services like Microsoft Teams and Windows 365.
Rani Borkar, head of Azure infrastructure systems at Microsoft, emphasised the company’s two-decade-long experience in co-engineering silicon for Xbox and Surface, underscoring the strategic significance of these homegrown chips.
A notable collaboration was also revealed between Microsoft and OpenAI during the development of the Maia 100. The partnership aimed to refine the architecture and test GPT models, benefiting both entities in optimizing Azure’s end-to-end AI architecture and advancing the capabilities of AI models.
While Microsoft is making strides toward utilising custom silicon, the company assures that it will continue using off-the-shelf hardware in the near future, promoting supply chain diversification and providing customers with a broader range of infrastructure choices.
As the tech giant forays into this new era of custom silicon, industry watchers are keenly observing how these advancements will impact the competitive landscape, especially in a field where major players like Amazon, Meta, and Google also pursue homegrown silicon efforts to stay ahead in the race for AI supremacy.
