Quantization
Türkçe: Kuantizasyon
A technique that lowers memory and speed cost by storing model weights with fewer bits (for example 8 or 4 instead of 16).
// EXPLANATION
Reducing weights to 4 bits cuts model memory to about a quarter, so it fits in smaller VRAM. The loss of precision is usually small, but answer quality can degrade at very low bit widths.
// ALSO KNOWN AS
quantizationquantisationquantized
// RELATED TERMS
VRAMThe high-bandwidth memory on the graphics card itself; an AI model's weights have to fit into it.
LLM (large language model)A neural network trained on very large amounts of text that generates text by probabilistically predicting the next piece (token) of a sequence.
InferenceThe stage in which a trained model produces output for new input; the phase that serves users after training.
