Time to first token (TTFT)
Time to first token is the elapsed time from a request's arrival at the serving system to the emission of its first output token.
- Formal definition
- For a request r, TTFT(r) = t_first_token(r) − t_arrival(r). It covers queueing, scheduling, and the prefill (prompt-processing) phase; it is the user's wait before anything appears.
- Units
- Milliseconds (
ms). Report as percentiles (p50/p95/p99) over a stated request set — never as a mean alone. - Includes
- Queueing delay, admission and batching decisions, prompt prefill, any prefix-cache lookup.
- Excludes
- Client-side network time unless explicitly measured end-to-end and declared as such.
- What determines it
- Prompt length; batching and scheduling policy (chunked-prefill chunk size directly trades TTFT against decode throughput); parallelism strategy; engine and hardware. TTFT and TPOT are in tension: choices that raise decode throughput typically inflate TTFT — which is why goodput, not either metric alone, is the optimization target.
- Common misuse
- Reporting means without percentiles; comparing TTFT across different prompt-length distributions; omitting queueing delay under load.
- Related terms
- time per output token · inference goodput · SLO attainment
- Preferred citation
gdput Research. "Time to first token (TTFT)." gdput definitions, v0.1, 2026. https://gdput.com/definitions/ttft