What AI models can the RTX A6000 run?
The RTX A6000 has 48 GB of GDDR6 ECC at 768 GB/s. With an 8K-token context it fits 20 of 21 models we track at Q4, 18 at Q8 and 13 at FP16; the largest text model it runs at Q4 is Llama 3.3 70B Instruct. The cheapest price we found today is $4,470 (used, GPU Sold, 2026-11-10), or you can rent one from $0.33/hr on Runpod.
Prices updated Nov 12, 2026 · CUDA - best support in llama.cpp, vLLM, ComfyUI (blower workstation card, 300 W)Buy vs rent: at $4,470 to buy and $0.33/hr to rent, buying only pays off after about 13,545 GPU-hours (ignoring electricity and resale value). Run your own numbers →
Every model we track on the RTX A6000
| Model | Q4 | Q8 | FP16 |
|---|---|---|---|
| Mistral 7B Instruct v0.37.2B | ✓ ~84 tok/s6.1 GB | ✓ ~52 tok/s9.2 GB | ✓ ~29 tok/s15.5 GB |
| Llama 3.1 8B Instruct8.0B | ✓ ~77 tok/s6.5 GB | ✓ ~47 tok/s9.9 GB | ✓ ~26 tok/s17.0 GB |
| DeepSeek R1 Distill Llama 8B8.0B | ✓ ~77 tok/s6.5 GB | ✓ ~47 tok/s9.9 GB | ✓ ~26 tok/s17.0 GB |
| Qwen3 8B8.2B | ✓ ~74 tok/s6.7 GB | ✓ ~46 tok/s10.2 GB | ✓ ~26 tok/s17.4 GB |
| Gemma 3 12B IT12.2B | ✓ ~55 tok/s8.7 GB | ✓ ~33 tok/s13.9 GB | ✓ ~18 tok/s24.5 GB |
| Qwen3 14B14.8B | ✓ ~44 tok/s10.6 GB | ✓ ~27 tok/s16.9 GB | ✓ ~14 tok/s29.8 GB |
| DeepSeek R1 Distill Qwen 14B14.8B | ✓ ~43 tok/s10.8 GB | ✓ ~26 tok/s17.1 GB | ✓ ~14 tok/s30.0 GB |
| Mistral Small 24B Instruct (2501)23.6B | ✓ ~29 tok/s15.6 GB | ✓ ~17 tok/s25.6 GB | ✓ ~9 tok/s46.2 GB |
| Gemma 3 27B IT27.4B | ✓ ~25 tok/s17.5 GB | ✓ ~15 tok/s29.2 GB | ✗needs 53 GB |
| Qwen3 30B-A3B (MoE)30.5B | ✓ ~162 tok/s19.0 GB | ✓ ~105 tok/s32.0 GB | ✗needs 59 GB |
| Qwen3 32B32.8B | ✓ ~20 tok/s21.5 GB | ✓ ~12 tok/s35.4 GB | ✗needs 64 GB |
| Qwen2.5 Coder 32B Instruct32.8B | ✓ ~20 tok/s21.5 GB | ✓ ~12 tok/s35.4 GB | ✗needs 64 GB |
| DeepSeek R1 Distill Qwen 32B32.8B | ✓ ~20 tok/s21.5 GB | ✓ ~12 tok/s35.4 GB | ✗needs 64 GB |
| Mixtral 8x7B Instruct (MoE)46.7B | ✓ ~51 tok/s28.4 GB | ✗needs 48 GB | ✗needs 89 GB |
| Llama 3.3 70B Instruct70.6B | ✓ ~10 tok/s43.3 GB | ✗needs 73 GB | ✗needs 135 GB |
| Qwen3 235B-A22B (MoE)235.1B | ✗needs 135 GB | ✗needs 235 GB | ✗needs 440 GB |
| FLUX.1 [dev]11.9B · image | ✓ fits9.9 GB | ✓ fits15.0 GB | ✓ fits25.4 GB |
| FLUX.1 [schnell]11.9B · image | ✓ fits9.9 GB | ✓ fits15.0 GB | ✓ fits25.3 GB |
| Stable Diffusion XL 1.02.6B · image | ✓ fits5.2 GB | ✓ fits6.3 GB | ✓ fits8.6 GB |
| Wan 2.1 T2V 14B (video)14.3B · video | ✓ fits14.9 GB | ✓ fits20.9 GB | ✓ fits33.4 GB |
| Wan 2.1 T2V 1.3B (video)1.4B · video | ✓ fits4.6 GB | ✓ fits5.2 GB | ✓ fits6.4 GB |
✓ = fits in 48 GB with an 8K-token context (text models) and ~1 GB runtime overhead; tok/s is a rough single-user estimate (0.6 × bandwidth ÷ bytes per token), not a benchmark. Leave ~0.5–1 GB free if this card also drives your display. Method.
RTX A6000 head-to-head
Similar-VRAM cards, cheapest first
FAQ
How much VRAM does the RTX A6000 have?
48 GB of GDDR6 ECC with 768 GB/s of memory bandwidth on a 384-bit bus.
What is the biggest AI model the RTX A6000 can run?
Of the models we track, Llama 3.3 70B Instruct (71B) is the largest text model that fits at 4-bit (Q4) with an 8K context. Bigger models need a second GPU or CPU offload.
Can the RTX A6000 run Llama 3.3 70B Instruct?
Yes, at Q4 (about 43 GB needed at Q4). Estimated speed at Q4: about 10 tokens/s.
Can the RTX A6000 run Qwen3 32B?
Yes, at Q4, Q8 (about 21 GB needed at Q4). Estimated speed at Q4: about 21 tokens/s.
Can the RTX A6000 run FLUX.1 [dev]?
Yes, at Q4, Q8, FP16 (about 10 GB needed at Q4).
How much does the RTX A6000 cost right now?
About $4,470 (used, GPU Sold, observed 2026-11-10). Prices change daily; check the retailer.
Is it cheaper to rent the RTX A6000?
For occasional use, yes. The cheapest on-demand rental we found was $0.33/hour (Runpod), so $4,470 buys about 13,545 hours of rental before electricity and resale value are considered.
Best GPU for…
Get the weekly GPU price drops
Cheapest card per model, new and used. Free, 1 email a week.
Subscribe on beehiiv →