Microsoft Funded d-Matrix AI Startup Thrashes NVIDIA H100 in AI Generative Workloads With “Corsair C8” DIMC

AMD and NVIDIA are no longer alone in the lucrative AI market, with a startup company, funded by Microsoft making some very serious claims.

d-Matrix, you’ve never heard of them and neither have I, the Californian based AI outfit has secured $110 million USD from investors such as Microsoft and SK Hynix through means of demonstrating its AI compute powerhouse, the Corsair C8.

While AMD struggles in the AI industry from lack of adoption and software, NVIDIA has its own problems, those problems being insufficient 4N wafer allocation from TSMC in regards to their top of the line (and most profitable) product, the H100. With TSMC proclaiming at least one and half years of supply constraints for NVIDIA’s flagship AI accelerator.

d-Matrix however are proclaiming several times more throughput and performance in AI generative workloads compared to the lucrative NVIDIA H100 with its Corsair C8 accelerator.

The Corsair C8 is not a graphics card as such as opposed to AMD’s MI300 or NVIDIA’s H100, rather it’s a modular DIMC (Digital in Memory Computing) accelerator that combines processing and memory into singular logic chips.

It allows for computations to be performed directly within the memory, eliminating the need to transfer data back and forth between the processor and memory, this results in faster processing times and improved energy efficiency over simple monolithic core designs lined to the brim with large quantities of memory buffer be it HBM (High Bandwidth Memory) or traditional GDDR memory.

For instance, NVIDIA’s H100 comes packing 80 GB of HBM3 memory across a tremendous 5120-bit memory interface, allowing for 2 TB/sec peak memory bandwidth. While d-Matrix’s Corsair C8 contains 2,048 DIMC cores across eight individual chiplets totaling 130 billion transistors, while NVIDIA’s H100 tops out at only 80 billion on a far more advanced TSMC processing node.

The Corsair C8 also comes with a total of 256 GB of shared LPDDR5 RAM which is extremely bizarre on first thought, but then again the memory itself is apart of the core logic, all eight chiplets are interconnected via PCIe Gen 5 pathways allowing for a whopping 8 TB/s die-to-die bandwidth in total.

The actual LPDDR5 RAM itself however is only good for around 400 GB/s effective bandwidth, but the purpose of the LPDDR5 is to handle overflow because the real cherry on top comes in the form of the Corsair C8’s 2GB of SRAM on board providing 150 TB/s of bandwidth, unbelievable.

d-Matrix themselves have stated that early adopters in the Corsair C8 have as many as 16 individual cards pooling together a total of 32GB of SRAM, which provides an effective bandwidth of 150 TB/s. Fucking hell.

This is specialized hardware tailored for AI generative workloads, unlike AMD and NVIDIA who are customizing their products for the task at hand, these smaller companies are able to thrash these giants in specific tasks and workloads.

Because the AI generative performance claims from d-Matrix are outrageous, with the Corsair C8 claimed to provide as much as 9600 TOPs of INT4 or around 2,400 TFLOPs of FP8 performance, by comparison the NVIDIA H100 in FP16 is slightly above 200 TFLOPS, a little over 50 TFLOPS in FP32, and just above 25 TFLOPS in FP64. All of these figures are theoretical, of course.

d-Matrix aren’t only proclaiming a performance advantage over the world’s largest startup company, but rather in terms of costs, they claim the Corsair C8 to be between 13 to 27 times cheaper than the H100. It is also 20 times more energy-efficient, has 20 times less latency, and boasts a 40 times higher memory bandwidth. when utilized in a cluster of multiple cards of course.

I am genuinely all for these innovative startup companies biting at the heels of NVIDIA and AMD, though given how the latter has virtually zero presence in the AI sector despite having superior products to NVIDIA, it’s only natural that d-Matrix would be more of a threat towards AMD than they would be towards NVIDIA, so I can imagine that Lisa Su is probably frothing at the teeth over d-Matrix right now.

The landscape of computing is about to change rapidly, a lot more of these startups are going to barge into the room and take names, majority of them will more than likely eventually be bought out by the likes of AMD, Intel and NVIDIA.

Though there’s also the chance that this Corsair C8 is pure vaporware and that financial donors get fleeced, I’d be alright with that outcome too because Microsoft deserve to rot.