RigFit › Models › Mistral Small 24B Instruct (2501)

Best GPU for Mistral Small 24B Instruct (2501) (2026)

The cheapest GPU that runs Mistral Small 24B Instruct (2501) at 4-bit (Q4) is the Arc A770 16GB (16 GB), currently about $342 (used, GPU Sold, 2026-11-10). Mistral Small 24B Instruct (2501) needs roughly 15.6 GB of VRAM at Q4 with an 8K-token context. Don't want to buy? Rent a GeForce RTX 5060 Ti 16GB that fits it from $0.095/hr (Vast.ai).

Prices updated Nov 12, 2026 · model on Hugging Face

Cheapest card that fits, by quantization

Quant	VRAM needed	Cheapest GPU that fits	Price	Est. speed*	Shop
Q4	15.6 GB	Arc A770 16GB16 GB	$342 usedGPU Sold · 2026-11-10	~21 tok/s	Check priceused · eBay or new on Amazon
Q8	25.6 GB	RTX A600048 GBor rent one from $0.33/hrRunpod	$4,470 usedGPU Sold · 2026-11-10	~17 tok/s	Check priceused · eBay or new on Amazon
FP16	46.2 GB	RTX A600048 GBor rent one from $0.33/hrRunpod	$4,470 usedGPU Sold · 2026-11-10	~9 tok/s	Check priceused · eBay or new on Amazon

*Rough estimate from memory bandwidth, single user. Context for text models: 8K tokens. Model specs: 23.6B parameters; 40 layers, 8 KV heads × 128 head dim (from Hugging Face metadata, fetched 2026-10-18).

Or rent a GPU for Mistral Small 24B Instruct (2501)

Quant	VRAM needed	Cheapest rentable GPU that fits	Price	Est. speed*
Q4	15.6 GB	GeForce RTX 5060 Ti 16GB16 GB	$0.095/hrVast.ai · marketplace	~17 tok/s	Rent
Q8	25.6 GB	RTX A600048 GB	$0.33/hrRunpod · Community Cloud	~17 tok/s	Rent
FP16	46.2 GB	RTX A600048 GB	$0.33/hrRunpod · Community Cloud	~9 tok/s	Rent

Cheapest on-demand hourly prices found 2026-11-12 05:55 ET. Marketplace prices move by the minute. Compare all rental prices and the buy-vs-rent break-even →

Try other settings

Model

Quantization

Context length

FAQ

How much VRAM does Mistral Small 24B Instruct (2501) need?

About 16 GB at Q4, 26 GB at Q8, 46 GB at FP16 (including a 8K-token KV cache and ~1 GB runtime overhead).

What is the best budget GPU for Mistral Small 24B Instruct (2501)?

Arc A770 16GB is the cheapest card in our tracker that fits it at Q4.

Can I rent a GPU to run Mistral Small 24B Instruct (2501) instead?

Yes. The cheapest single rented GPU with enough VRAM for Q4 we found was a GeForce RTX 5060 Ti 16GB at $0.10/hour on Vast.ai (2026-11-12). Marketplace prices change constantly.

How fast will it run?

Token generation on a single GPU is limited mostly by memory bandwidth. Our tokens/sec figures are estimates (0.6 × bandwidth ÷ bytes read per token) and real results vary by backend and settings.

Other models