How GPUDojo Estimates LLM Speed and AI Value

The methodology behind the comparison table and GPU detail pages | Updated July 2026

The Reference Workload

The current estimates use a representative 14B-class quantized model with approximately 10GB of weights in memory. This keeps every GPU on the same basis rather than mixing results from different models and quantizations.

• Reference parameters: 14 billion
• Reference model size: 10GB
• Generation efficiency factor: 75%
• Prefill efficiency factor: 50%
• Assumed workload: single-user inference with the model fully resident in GPU memory

Generation Speed

Autoregressive token generation repeatedly reads model weights from memory, so it is often limited by memory bandwidth. The table's Gen t/s estimate is:

The 75% factor represents a simplified allowance for memory-controller efficiency, KV-cache traffic, dequantization and framework overhead. GPUDojo currently applies the same factor to every card so the comparison remains consistent. Architecture and software maturity are handled separately in the AI-value ranking.

Examples shown by the current table

Prefill Speed

Prefill is the initial processing of a prompt. It is more compute-sensitive than token generation and is estimated from advertised FP32 TFLOPS:

Prefill is shown on GPU detail pages and included in CSV exports, but it is intentionally absent from the compact comparison table. Advertised TFLOPS are an especially rough proxy for quantized inference, so this estimate should be treated as directional.

Usable VRAM and Model Fit

Model-fit tiers use usable VRAM, not always the memory printed on the board. Most GPUs expose their full capacity to one model. The Tesla K80 is an important exception: its advertised 24GB is split across two independent 12GB GPUs, so GPUDojo treats it as 12GB usable unless software explicitly shards a model.

• 8GB: 7-8B quantized models
• 12GB: 7-8B comfortably; some 14B quantizations
• 16GB: 14B comfortably; larger models with compromises
• 24GB: many 30-32B Q4 models
• 48GB+: many 70B Q4 models

Actual fit also depends on quantization, context length, KV-cache size, framework overhead and whether other applications are using VRAM.

How the Default AI-Value Ranking Works

The homepage does not rank by speed alone. Its default score rewards capacity and generation speed, then adjusts for real ownership friction and price:

Squaring usable VRAM intentionally gives substantial weight to fitting larger models. The other factors prevent an extremely cheap but obsolete or difficult card from automatically winning.

Setup adjustment

• Easy (1.00): conventional desktop card with low setup friction
• Moderate (0.85): some software or platform verification required
• Advanced (0.72): typical datacenter-card concerns such as passive cooling and no display output
• Avoid (0.25): severe practical drawbacks for current local-AI use

Compatibility adjustment

Current, broadly supported NVIDIA architectures receive the full multiplier. Older NVIDIA generations and non-CUDA ecosystems receive lower multipliers to reflect missing kernels, obsolete toolchains or additional framework verification. The K80 receives the strongest penalty because it combines split VRAM with Kepler-era software support.

These adjustments are editorial heuristics, not hardware benchmarks. Their purpose is to make the default shopping order more honest. Users can always override it by sorting directly by price, usable VRAM, generation speed, setup difficulty or $/usable GB.

Important Limitations

How to Use the Estimates

Start with usable VRAM to determine what can fit. Use Gen t/s to compare likely interactive speed within that capacity class. Then consider setup and software compatibility before comparing the final delivered price. For an exact GPU and model combination, look for benchmarks using the same model file, quantization, context length, framework and batch size you plan to run.