How to estimate the materialized model size
When we heard about Large Language Models, we always hear about the parameter size of the model. E.g. GPT-3.5 has 175 billion parameters, Deepseek R-1 has 671 billion parameters and GPT-4 is rumored to even have 1.8 trillion parameters. What exactly does that mean? A fundamental knowledge is, for most part, the model is composed of the model’s parameters (often called weights and biases). Each parameter is a numerical value that the model learned during training and the precision of these numbers dictates how much space they occupy. ...