AMD to acquire chip start-up Taalas in challenge to NVIDIA
What's the story
In a strategic move to challenge NVIDIA's stronghold in the AI hardware market, Advanced Micro Devices (AMD) said on Thursday it would buy Taalas, an innovative AI chip start-up. The Toronto-based company has developed a unique technology that directly etches model weights into silicon, significantly improving inference performance. The deal was announced after market close on Thursday but the financial details of the transaction have not been disclosed yet.
Innovative technology
Taalas's unique approach to AI inference
Founded in 2023, Taalas's approach to inference is unlike traditional GPUs or dataflow architectures.
The company's chips, dubbed model-specific integrated circuits (MSICs), don't use high-bandwidth memory (HBM) for storing model weights. Instead, they etch these weights directly into the silicon.
This revolutionary method was showcased in February when Taalas unveiled its first test chip on TSMC's 6nm process tech, the HC1.
Performance metrics
HC1 chip's impressive performance
The HC1 chip from Taalas was able to serve Meta's Llama 3.1 8B at an astonishing speed of 16,960 tokens per second.
This was a whopping 48 times faster than NVIDIA's GPUs and 8.5 times faster than Cerebras's accelerators when it was unveiled last February.
The performance of this chip shows the potential of Taalas's technology in revolutionizing AI inference.
Future prospects
Potential integration with AMD's existing systems
For its second-gen HC2 chip, Taalas plans to increase the parameter count to 20 billion. This could be done using pipeline parallelism across multiple accelerators.
AMD has a rack-scale compute platform and an in-house system design team that can easily accommodate this.
The company is likely to integrate its Instinct-based Helios racks with chips based on Taalas's tech, creating an architecture where compute-heavy prompt processing is done on GPUs while token generation is offloaded to Taalas-based accelerators.
Market implications
Impact on model development and reasoning times
The acquisition of Taalas by AMD could have a major impact on model development.
Developers have been reducing hallucinations by trading time for accuracy, a technique called test-time scaling.
This method consumes more tokens, making it costly and delaying responses from chatbots or code assistants.
However, if AMD's Taalas buy can reduce token costs and increase output speeds significantly, developers may be tempted to extend reasoning times even further.