Groq: our review

Groq doesn't build models: the company builds chips, LPUs (Language Processing Units), and rents out the compute that runs on them to serve open models — Llama, GPT OSS, Qwen — faster than a standard GPU allows. It's a precise card to play: generation speed, not model choice or a polished interface.

Speed as the pitch, with numbers behind it

The pricing page shows, model by model, a rate in tokens per second next to the price: 1,000 TPS for GPT OSS 20B, 840 for Llama 3.1 8B Instant. That figure has concrete meaning for a conversational agent or a live transcription tool, where the user notices every second of waiting. A developer building a responsive AI agent finds a measurable advantage here, not a marketing promise.

What it costs, model by model

Billing follows the token, on both input and output, with no subscription. Llama 3.1 8B Instant charges $0.05 per million input tokens and $0.08 on output — one of the lowest rates on the market for an open model usable in production. GPT OSS 120B climbs to $0.15 and $0.60, and Llama 3.3 70B Versatile to $0.59 and $0.79. A genuine Free Plan, with its own rate limits, lets you test before switching to the paid Developer Plan, which unlocks wider limits as well as batch processing.

What's missing: the processing region

This is the point Groq doesn't handle the way Scaleway or OVHcloud AI Endpoints do. The Data Processing Addendum, a public document readable without technical detours, simply states that data may be processed "in the United States and other countries where Groq or its subprocessors maintain operations" — without naming a single specific country. So there's no way to answer a client asking where their data runs. In exchange, the Services Agreement commits Groq to not training its models on client inputs and outputs, and offers zero retention that can be switched on in the console: the promise about data use is clearer than the one about its location.

Where it falls short

Three reservations before committing. The first, already noted: no precise processing region is published. The second concerns certifications: Groq communicates about SOC 2 and HIPAA, but the Trust Center that should prove it loads entirely in JavaScript (a Vanta portal), and no directly readable page confirms these certifications in writing. The third is structural: Groq sells an API, not a product. Without a developer to integrate it, there's nothing to get out of it.

Our recommendation

If latency is your main constraint — a voice agent, a chat that must answer with no perceptible delay — Groq is a defensible choice, worth testing first on the Free Plan. If proving where data processing happens is a contractual requirement, turn instead to a host that documents it, such as Scaleway. And without a developer on the team, our automation services meet the need better than a raw API.