The Groq node is a turnkey solution featuring eight Groq cards in a single chassis. Multiple Groq nodes can be connected in a wide variety of network topologies, enabling computation at a new scale. Each Groq node delivers up to 6 PetaOPs of performance, while consuming just 3.3KW of power in a 5U form factor. The high performance and low power of the Groq node offers TCO (Total Cost of Ownership) benefits.
Groq continues to ship its PCIe card featuring its TSP chip, adding industry standard server management capabilities. Groq’s TSP is capable of 18,900 IPS (inferences per second) on ResNet-50 v2 at batch size one, making it among the fastest commercially available AI/ML accelerators, with a responsiveness measured in hundredths of a millisecond. This eliminates the typical trade-off seen between performance and latency.
The announcement of Groq’s hardware also includes the latest release of the Groq ware SDK, allowing developers to build high-performance models on Groq cards and nodes.
Andrew Fursman, Co-Founder and CEO at 1QBit said, “1QBit is pushing further into AI, and the bottlenecks we are running into are almost all of a computational nature. That's where Groq's solutions and architecture have helped us tremendously. We look forward to working further with Groq and are excited to see them expand their customer base in the healthcare, life sciences, and financial markets."
Groq is working with a number of global customers in the autonomous vehicle and financial spaces as well as a number of national labs, who are using the technology to address a wide variety of large compute challenges.
