Roofline Labs
Fleet economics for self-hosted LLM inference. We measure what your hardware actually delivers, score it against the memory-bandwidth roofline, and tell you which config change closes the gap.
What we do
A decode-bound LLM cannot run faster than memory bandwidth ÷ bytes read per token. That ceiling is the roofline. Most self-hosted deployments sit well below it, and nobody can tell whether the cause is the model, the quantization, the serving engine, or a flag. Roofline Labs builds the tooling that answers that question continuously: utilization and cost per token against the roofline for every model on every box in a fleet, with the concrete recommendation that moves the number.
Proof so far
Everything starts from measurement. Our public benchmark set records model × hardware × engine × quantization → tokens per second, each row scored as a percentage of that hardware's roofline, under stated configs, from serving runs we ran ourselves.
Benchmark table: 45 measured decode results on NVIDIA DGX Spark (GB10)
A few things the data already shows on one GB10 (187 GB/s effective decode bandwidth):
| Same model, same weights | decode tok/s | % roofline |
|---|---|---|
| Qwen3.6-35B-A3B NVFP4, llama.cpp, no speculation | 55.1 | 81% |
| Qwen3.6-35B-A3B NVFP4, vLLM, no speculation | 75.4 | 111% |
| Qwen3.6-35B-A3B NVFP4, vLLM, MTP speculation on | 124.9 | 184% |
| Qwen3-Coder-Next 80B-A3B, third-party NVFP4 requant, SGLang | 35.8 | 32% |
A 2.3× spread from engine and speculation choices alone, on identical hardware and weights. That gap is the product.
Who it is for
Teams running one to a few dozen inference boxes on their own hardware: labs, on-prem AI teams, and small providers who need to know whether they are buying more GPUs or fixing a flag.
Company
- Entity
- Roofline Labs LLC, Texas, founded 2026
- Stage
- Self-funded, pre-seed
- Contact
- hello@rooflinelabs.com