Skip to content

NVIDIA GeForce RTX 4070 Super

NVIDIA · 12GB GDDR6X · Can run 48 models

Buy Amazon
Manufacturer NVIDIA
VRAM 12 GB
Memory Type GDDR6X
Architecture Ada Lovelace
CUDA Cores 7,168
Tensor Cores 224
Bandwidth 504 GB/s
TDP 220W
MSRP $599
Released Jan 17, 2024

AI Notes

The RTX 4070 Super is a strong mid-range option with 12GB VRAM and 504 GB/s bandwidth. It runs 7B models at high speed and can handle 13B models with Q4 quantization. Offers a good balance of performance and power efficiency for local AI inference.

Compatible Models

Model Parameters Best Quant VRAM Used Fit Est. Speed
Qwen 3 0.6B 600M Q4_K_M 2.5 GB Runs ~202 tok/s
Qwen 3.5 0.8B 800M Q4_K_M 1.5 GB Runs ~336 tok/s
Gemma 3 1B 1B Q8_0 2 GB Runs ~252 tok/s
Llama 3.2 1B 1B Q8_0 3 GB Runs ~168 tok/s
DeepSeek R1 1.5B 1.5B Q8_0 3 GB Runs ~168 tok/s
Gemma 2 2B 2B Q8_0 4 GB Runs ~126 tok/s
Gemma 3n E2B 2B Q4_K_M 3.3 GB Runs ~153 tok/s
Gemma 4 E2B 2B Q4_K_M 4 GB Runs ~126 tok/s
Qwen 3.5 2B 2B Q4_K_M 3 GB Runs ~168 tok/s
Llama 3.2 3B 3B Q8_0 5 GB Runs ~101 tok/s
Phi-3 Mini 3.8B 3.8B Q8_0 5.8 GB Runs ~87 tok/s
Phi-4 Mini 3.8B 3.8B Q4_K_M 4.5 GB Runs ~112 tok/s
Gemma 3 4B 4B Q4_K_M 5 GB Runs ~101 tok/s
Gemma 3n E4B 4B Q4_K_M 4.5 GB Runs ~112 tok/s
Gemma 4 E4B 4B Q4_K_M 6 GB Runs ~84 tok/s
Qwen 3 4B 4B Q4_K_M 4.5 GB Runs ~112 tok/s
Qwen 3.5 4B 4B Q4_K_M 4.5 GB Runs ~112 tok/s
DeepSeek R1 7B 7B Q8_0 9 GB Runs ~56 tok/s
Falcon 3 7B 7B Q4_K_M 6.8 GB Runs ~74 tok/s
Mistral 7B 7B Q8_0 9 GB Runs ~56 tok/s
Qwen 2.5 7B 7B Q8_0 9 GB Runs ~56 tok/s
Qwen 2.5 Coder 7B 7B Q8_0 9 GB Runs ~56 tok/s
Qwen 2.5 VL 7B 7B Q4_K_M 7 GB Runs ~72 tok/s
Aya Expanse 8B 8B Q4_K_M 6.5 GB Runs ~78 tok/s
Cogito 8B 8B Q4_K_M 7.5 GB Runs ~67 tok/s
DeepSeek R1 8B 8B Q4_K_M 7.5 GB Runs ~67 tok/s
Llama 3.1 8B 8B Q8_0 10 GB Runs ~50 tok/s
Nemotron 3 Nano 8B 8B Q4_K_M 7.5 GB Runs ~67 tok/s
Qwen 3 8B 8B Q4_K_M 7.5 GB Runs ~67 tok/s
Qwen 3.5 9B 9B Q4_K_M 7.5 GB Runs ~67 tok/s
Falcon 3 10B 10B Q4_K_M 8.5 GB Runs ~59 tok/s
Llama 3.2 Vision 11B 11B Q4_K_M 8.5 GB Runs ~59 tok/s
Mistral Nemo 12B 12B Q4_K_M 9.5 GB Runs ~53 tok/s
DeepSeek R1 14B 14B Q4_K_M 9.9 GB Runs ~51 tok/s
Phi-4 14B 14B Q4_K_M 9.9 GB Runs ~51 tok/s
Qwen 2.5 14B 14B Q4_K_M 9.9 GB Runs ~51 tok/s
Gemma 2 9B 9B Q8_0 11 GB Runs (tight) ~46 tok/s
Gemma 3 12B 12B Q4_K_M 10.5 GB Runs (tight) ~48 tok/s
Phi-4 Reasoning 14B 14B Q4_K_M 11 GB Runs (tight) ~46 tok/s
Qwen 2.5 Coder 14B 14B Q4_K_M 12 GB CPU Offload ~13 tok/s
Qwen 3 14B 14B Q4_K_M 12 GB CPU Offload ~13 tok/s
StarCoder2 15B 15B Q8_0 17 GB CPU Offload ~9 tok/s
Codestral 22B 22B Q4_K_M 14.7 GB CPU Offload ~10 tok/s
Devstral 24B 24B Q4_K_M 17 GB CPU Offload ~9 tok/s
Magistral Small 24B 24B Q4_K_M 17 GB CPU Offload ~9 tok/s
Mistral Small 3.1 24B 24B Q4_K_M 18 GB CPU Offload ~8 tok/s
Gemma 2 27B 27B Q4_K_M 17.7 GB CPU Offload ~8 tok/s
Qwen 3.5 35B A3B 35B Q4_K_M 12 GB CPU Offload ~13 tok/s
36 model(s) are too large for this hardware.