NVIDIA GeForce RTX 2080 Ti

NVIDIA · 11GBGDDR6 · Can run 59 models

Manufacturer	NVIDIA
VRAM	11 GB
Memory Type	GDDR6
Architecture	Turing
CUDA Cores	4,352
Tensor Cores	544
Bandwidth	616 GB/s
TDP	250W
MSRP	$1,199
Released	Sep 20, 2018

AI Notes

The RTX 2080 Ti remains a popular choice for budget local AI on the used market. Its 11GB VRAM handles 7B models well and can run 13B with Q4 quantization. The 616 GB/s bandwidth provides fast inference for its VRAM class. An excellent second-hand option for local AI experimentation.

Compatible Models

Model	Parameters	Best Quant	VRAM Used	Fit	Est. Speed
Qwen 3 0.6B	600M	Q4_K_M	2.5 GB	Runs	~246 tok/s
Qwen 3.5 0.8B	800M	Q4_K_M	1.5 GB	Runs	~411 tok/s
Gemma 3 1B	1B	Q8_0	2 GB	Runs	~308 tok/s
Llama 3.2 1B	1B	Q8_0	3 GB	Runs	~205 tok/s
DeepSeek R1 1.5B	1.5B	Q8_0	3 GB	Runs	~205 tok/s
SmolLM2 1.7B	1.7B	Q8_0	2.7 GB	Runs	~228 tok/s
Gemma 2 2B	2B	Q8_0	4 GB	Runs	~154 tok/s
Gemma 3n E2B	2B	Q4_K_M	3.3 GB	Runs	~187 tok/s
Gemma 4 E2B	2B	Q4_K_M	4 GB	Runs	~154 tok/s
Qwen 3.5 2B	2B	Q4_K_M	3 GB	Runs	~205 tok/s
Llama 3.2 3B	3B	Q8_0	5 GB	Runs	~123 tok/s
StarCoder2 3B	3B	Q4_K_M	3.5 GB	Runs	~176 tok/s
Phi-3 Mini 3.8B	3.8B	Q8_0	5.8 GB	Runs	~106 tok/s
Phi-4 Mini 3.8B	3.8B	Q4_K_M	4.5 GB	Runs	~137 tok/s
Gemma 3 4B	4B	Q4_K_M	5 GB	Runs	~123 tok/s
Gemma 3n E4B	4B	Q4_K_M	4.5 GB	Runs	~137 tok/s
Gemma 4 E4B	4B	Q4_K_M	6 GB	Runs	~103 tok/s
Qwen 3 4B	4B	Q4_K_M	4.5 GB	Runs	~137 tok/s
Qwen 3.5 4B	4B	Q4_K_M	4.5 GB	Runs	~137 tok/s
Yi 1.5 6B	6B	Q4_K_M	5 GB	Runs	~123 tok/s
Codestral Mamba 7B	7B	Q4_K_M	6.9 GB	Runs	~89 tok/s
DeepSeek R1 7B	7B	Q8_0	9 GB	Runs	~68 tok/s
Falcon 3 7B	7B	Q4_K_M	6.8 GB	Runs	~91 tok/s
InternLM 2.5 7B	7B	Q4_K_M	5.5 GB	Runs	~112 tok/s
Mistral 7B	7B	Q8_0	9 GB	Runs	~68 tok/s
OpenChat 3.5 7B	7B	Q4_K_M	6.9 GB	Runs	~89 tok/s
Qwen 2.5 7B	7B	Q8_0	9 GB	Runs	~68 tok/s
Qwen 2.5 Coder 7B	7B	Q8_0	9 GB	Runs	~68 tok/s
Qwen 2.5 VL 7B	7B	Q4_K_M	7 GB	Runs	~88 tok/s
StarCoder2 7B	7B	Q4_K_M	5.5 GB	Runs	~112 tok/s
WizardLM 2 7B	7B	Q4_K_M	6.9 GB	Runs	~89 tok/s
Aya Expanse 8B	8B	Q4_K_M	6.5 GB	Runs	~95 tok/s
Cogito 8B	8B	Q4_K_M	7.5 GB	Runs	~82 tok/s
DeepSeek R1 8B	8B	Q4_K_M	7.5 GB	Runs	~82 tok/s
Dolphin 3 8B	8B	Q4_K_M	6 GB	Runs	~103 tok/s
Nemotron 3 Nano 8B	8B	Q4_K_M	7.5 GB	Runs	~82 tok/s
Nous Hermes 2 8B	8B	Q4_K_M	6 GB	Runs	~103 tok/s
Qwen 3 8B	8B	Q4_K_M	7.5 GB	Runs	~82 tok/s
Qwen 3.5 9B	9B	Q4_K_M	7.5 GB	Runs	~82 tok/s
Yi 1.5 9B	9B	Q4_K_M	6.5 GB	Runs	~95 tok/s
Yi Coder 9B	9B	Q4_K_M	8 GB	Runs	~77 tok/s
Falcon 3 10B	10B	Q4_K_M	8.5 GB	Runs	~72 tok/s
Llama 3.2 Vision 11B	11B	Q4_K_M	8.5 GB	Runs	~72 tok/s
Granite 3.3 8B	8B	Q8_0	10 GB	Runs (tight)	~62 tok/s
Llama 3.1 8B	8B	Q8_0	10 GB	Runs (tight)	~62 tok/s
Mistral Nemo 12B	12B	Q4_K_M	9.5 GB	Runs (tight)	~65 tok/s
DeepSeek R1 14B	14B	Q4_K_M	9.9 GB	Runs (tight)	~62 tok/s
Phi-4 14B	14B	Q4_K_M	9.9 GB	Runs (tight)	~62 tok/s
Qwen 2.5 14B	14B	Q4_K_M	9.9 GB	Runs (tight)	~62 tok/s
Gemma 2 9B	9B	Q8_0	11 GB	CPU Offload	~17 tok/s
Gemma 3 12B	12B	Q4_K_M	10.5 GB	CPU Offload	~18 tok/s
Phi-4 Reasoning 14B	14B	Q4_K_M	11 GB	CPU Offload	~17 tok/s
Qwen 2.5 Coder 14B	14B	Q4_K_M	12 GB	CPU Offload	~15 tok/s
Qwen 3 14B	14B	Q4_K_M	12 GB	CPU Offload	~15 tok/s
StarCoder2 15B	15B	Q4_K_M	10.5 GB	CPU Offload	~18 tok/s
InternLM 2.5 20B	20B	Q4_K_M	12 GB	CPU Offload	~15 tok/s
gpt-oss 20B	21B	MXFP4	15 GB	CPU Offload	~12 tok/s
Codestral 22B	22B	Q4_K_M	14.7 GB	CPU Offload	~13 tok/s
Qwen 3.5 35B A3B	35B	Q4_K_M	12 GB	CPU Offload	~15 tok/s

55 model(s) are too large for this hardware.