Lab L12 · Platform
Fit the Model
- Launch lab
- MVP set
- ≈ 45 min
- Interactive version coming
A light 3D server view. GPU memory fills with weights, KV cache and overhead as you change the model, precision, context length and batch.
What you’ll practise
- Computing KV-cache bytes per token, including the grouped-KV trap
- Choosing tensor parallelism and precision so a 70B-class model fits with headroom
- Checking latency targets with a simulated benchmark and writing the capacity memo