Demand for Real-time AI Inference from Groq® Accelerates Week Over Week
MOUNTAIN VIEW, Calif., April 2, 2024 /PRNewswire/ -- Groq®, a generative AI solutions company, announced today that more than 70,000 new developers are using GroqCloud™and more than 19,000 new applications are running on the LPU™ Inference Engine via the Groq API. The rapid migration to GroqCloud since its launch on March 1st indicates a clear demand for real-time inference as developers and companies seek lower latency and greater throughput for their generative and conversational AI applications.
- It's clear that developers are hungry for low-latency AI inference capabilities, and we're thrilled to see how it's being used to bring innovative ideas to life.
- Today, ~40% of AI chips are leveraged for inference, and that alone would put the TAM for chips used for inference at ~$48B by 2027.
- Once applications reach maturity they often allocate 90-95 percent of resources to inference, indicating a much larger market over time.
- With nearly every industry and government worldwide looking to leverage generative and/or conversational AI, the TAM for AI chips, and systems dedicated to inference in particular, appears to be limitless.