Skip to content
modelread

Loading devices…

Remembered on this device. Multi-GPU: search “2x”, “8x”, “node”.

How the numbers work

modelread answers from the repository itself, not from a parameter count. It lists the actual files, reads safetensors and GGUF headers over HTTP range requests (nothing is downloaded), and builds the architecture from the config: attention type per layer, KV heads, sliding windows, MoE routing, recurrent state, per-layer embeddings, vision towers.

Memory

Weights come from real file sizes. KV cache is computed per layer kind (full, sliding-window, linear/recurrent, MLA, shared-KV), so hybrid and SWA models are not overcounted. A device's usable budget subtracts the runtime context, the compute buffer and OS headroom, with an uncertainty band. When the fast tier is not enough, the placement moves MoE experts, then layers, to system RAM, and says so.

Speed

Decode is a roofline over the bytes each token actually streams (active experts only for MoE, gathered embeddings excluded) at the device's bandwidth and a calibrated efficiency, plus a fixed per-token overhead. Prefill is compute-bound, with attention cost growing with depth. Multi-user throughput uses a batching model, and multi-GPU nodes get a tensor-parallel estimate fitted to published serving tables. Every number is shown as a range, with a badge for how it is known.

Calibration

Each prediction is scored against a measurement before that measurement is used for fitting.

DeviceSourceWhatError
RTX 3090measureddecode: dense, hybrid and MoE±1–15% (learned per-architecture factors)
RTX 3090measuredprefill at 8K/16K depth≤2%; gemma-4 (head_dim 512) −6..+14%
RTX 4060 Laptopmeasureddecode / max context / MoE offload placement+1% / −0.4% / correct --n-cpu-moe
Jetson Orin Nano, Thormeasuredmax context / decode−3% / −3..−19% (conservative)
Ryzen 5900X, CPU onlymeasureddecode+1..+10%
RTX 3090 concurrency sweepmeasuredaggregate tok/s, 1–32 sequences−1..+11%
RTX 3090 quant sweepmeasuredIQ2/IQ3 kernel cost vs K-quantslearned ×1.24
Phonespublisheddecode per SoC (arXiv 2607.05475, apple-silicon-llm-bench, LiteRT cards)Gemma-4-E2B on iPhone 17 Pro −14%
8× H100 / H200 / A100publishedtensor-parallel per-user decode, TP2–TP8 (NVIDIA NIM / TensorRT-LLM)−12..+15%

Verdicts

Fits
Runs entirely in fast memory with headroom.
Tight fit
Fits, but within the uncertainty of the memory budget.
Offload
Runs with part of the model in system RAM: works, slower.
Vendor-reported
Only the vendor's own claim is available.
Won't fit
Does not fit as configured, even with offload.
Unknown
Not enough evidence to decide.

Speed badges

Quality

Quant quality is an expected loss from published KL-divergence tables, scaled by model size (bigger models lose less) and MoE active size. Benchmark percentiles compare a model to every indexed model with that benchmark; vendor-reported and independent scores are kept apart. Derivatives that change the weights (abliterated, merged, pruned, distilled, renamed fine-tunes) rank below faithful quants, with the reason shown.

Known limits

Make it better

Run modelfit bench --submit (llama-bench) or modelfit harvest (llama-server logs). Measurements change predictions for that device and teach the cross-device architecture and quant-kernel factors.

Askwhat should I run?
Browsebest models per device
Devicesall hardware
Connect your agentMCP · API
How the numbers workcalibration