Research
gdput's core research program treats the service-level objective as a function of the inference configuration — and works on learning that function.
An LLM deployment's SLO metrics — TTFT, TPOT, and ultimately goodput — are determined by its configuration: the serving engine (vLLM, SGLang, TensorRT-LLM), its runtime flags (batching capacity, KV-cache fraction, CUDA graphs, chunk sizes), parallelism degrees, hardware, and workload mix. Today that surface is mostly explored by trial and error, per engine, per cluster. The program sits in the seam between two literatures: LLM serving systems, which build the configuration knobs, and configuration-performance learning, which has a decade of machinery — transfer learning, meta-learning, structured Bayesian optimization — for modeling exactly this kind of surface but has only begun pointing it at LLM inference.
Open problems the program targets
- Engine choice as a modeled dimension. Configurations are not portable across engines; nearly all existing performance models fix the engine and tune flags. Treat the engine as another environment axis.
- Few-shot transfer across the (model, hardware, workload) triple. A config→SLO model trained in one environment should adapt to a new one from few samples, not be rebuilt from scratch.
- Feasibility-aware modeling. The configuration surface is non-smooth — many configurations crash outright or blow the latency target. Crashes are observations, not noise.
- Fidelity versus cost in label generation. Simulators produce cheap training labels; a useful model knows when the simulator can be trusted and when real profiling is required.
- A standard control plane. Exposing learned SLO models to orchestrators and agents through the Model Context Protocol (MCP), so configuration selection becomes a queryable, self-improving service rather than a bespoke scheduler component.
Outputs
The agenda anchors a doctoral research program (2026–2029). It is executed in public where possible: canonical definitions, versioned measurement methodology, configuration-sweep datasets, technical papers as reproducible packages, and open-source modeling tools in the gdput GitHub organization.
answer-sized research notes in preparation