Chip designer Arm has entered the artificial intelligence (AI) hardware space with its first in-house processor designed to power AI agents. Unlike traditional chatbots, these are much smarter systems that can take proactive actions to achieve goals with little human input or oversight.
By focusing specifically on enhancing AI agents, Arm’s chips have the potential to accelerate the adoption and proliferation of agent AI in both business and personal life, bringing AI closer to what people expect from virtual assistants.
you may like
In this case, think of the CPU as the conductor of an orchestra of GPUs and other AI accelerators (hardware specifically designed to run LLM).
To that end, Arm representatives announced in a statement that the company’s new AGI CPU is a custom design that includes a 3-nanometer process node, up to 136 Neoverse V3 cores capable of reaching clock speeds of 3.7 GHz, and 6 gigabytes per second of memory bandwidth per core for use in data centers to power active AI agents.
All of these features are aimed at achieving the goal of providing better performance and efficiency than traditional CPUs using the x86 architecture, which was developed by Intel in 1978 and is still the primary computing architecture used in processors today.
The future of custom chips
The relentless growth of AI and the introduction of smart agents will require more data center-based hardware to power these systems. However, the generality of CPUs means that they are not inherently designed to perform the specific orchestration required for agent AI.
Arm’s AGI CPU uses the Armv9.2-A architecture for its core. This architecture is designed for the specialized needs of actually doing AI, called inference. This speciality eliminates the need for AGI CPUs to maintain legacy support for other processes and applications, as is the case with x86 chips (traditional processors used in regular computers).
This should result in faster and more efficient performance for AI. Arm representatives said the company’s AGI CPUs deliver more than twice the performance per server rack compared to x86 CPUs.
AGI CPUs are designed to pack two chips with dedicated memory and input/output (I/O) capabilities into a single server blade, for a total of 272 cores per blade. The blades can then be stacked into 30 server racks, with thousands of cores working in parallel, for a total of 8,160 cores to deliver sustained performance for “large” agent AI workloads.
What to read next
Arm’s chip design expertise focuses on delivering strong performance at relatively low power consumption. This is one reason why all smartphone chips use Arm-based processors or instruction sets. For example, Qualcomm uses Arm technology in its Snapdragon chips, and Apple uses Arm technology in its iPhone and MacBook chips.
As AI continues to migrate from training LLMs to actively deployed agent AI, the need for CPU-based processing power in data centers will increase. This is expected to significantly increase the energy demand for AI.
AGI CPUs are designed to pack two chips with dedicated memory and input/output (I/O) capabilities into a single server blade, for a total of 272 cores per blade. The blades can then be stacked into 30 server racks, with thousands of cores working in parallel, for a total of 8,160 cores to deliver sustained performance for “large” agent AI workloads.
Arm’s chip design expertise focuses on delivering strong performance at relatively low power consumption. This is one reason why all smartphone chips use Arm-based processors or instruction sets. For example, Qualcomm uses Arm technology in its Snapdragon chips, and Apple uses Arm technology in its iPhone and MacBook chips.
As AI continues to migrate from training LLMs to actively deployed agent AI, the need for CPU-based processing power in data centers will increase. This is expected to significantly increase the energy demand for AI.
Cumers Afifi Sabe
Channel Editor, Technology
Arm has the potential to disrupt what has become something of an arms race for computer chips. Delivering a CPU that is more efficient than x86-based CPUs while delivering strong AI inference performance could potentially disrupt Intel, AMD, and hardware giant Nvidia with its own Arm-based Vera CPUs, while also curbing growing energy demands.
This architecture is already being used in chips for AI data centers, so chip designers are well-positioned to make their own forays into offering “off-the-shelf” CPUs.
Arm has traditionally licensed its designs to other chipmakers, but the AGI CPU will be its first attempt to produce hardware that other companies can buy and deploy in their own data centers. This points to a future in which more hardware will be custom-designed to power AI, whether it’s to run LLM more efficiently, as seen in Google’s TPUs or the application-specific integrated circuit (ASIC) architectures found in Amazon’s Trainium chips, or for inference in the case of Microsoft’s Maia 200 chips.
Custom chips that can overcome some of the hardware constraints of operating AI at scale have the potential to disrupt the traditional configuration of common computing hardware in data centers. This could accelerate the path to artificial general intelligence (AGI). AGI is a hypothetical AI system that can learn, understand, and apply knowledge across multiple domains at the human level or beyond.
Source link
