Understanding Latency in Generative AI Applications
In production conversational AI, voice assistants, and agentic workflows, perceived latency is divided into two distinct phases: Time to First Token (TTFT) and Token Generation Throughput (tokens/second).
The Anatomy of an LLM API Request
- Network Transit & Handshake (Ping): The physical TCP/TLS roundtrip from your client or edge worker to the AI provider's inference cluster (e.g.
us-east-1). - Prefill Phase (Prompt Ingestion): The transformer parallelizes processing of all input prompt tokens. Large prompts (e.g. 50k tokens of RAG documents) introduce noticeable prefill compute delay before the first token is emitted.
- Decoding Phase (Streaming Output): Autoregressive generation where tokens are generated sequentially one by one. The throughput (e.g. 100 tok/sec on GPT-4o vs 280 tok/sec on Groq LPUs) dictates how fast text streams across the user's screen.
Mitigating Latency in Agent Loops
For multi-step agent loops that perform planning, searching, and tool calling across 3 to 5 sequential rounds, total latency compounds quickly. Using faster sub-models (like Claude 3.5 Haiku or Gemini 1.5 Flash) for intermediate classification and tool routing reduces end-to-end user wait times by up to 70%.