Compressing LLMs below 1 bit per parameter without retraining
Samsung's LittleBit compresses language models to 0.6 bits per parameter while keeping them usable. No fine-tuning required.
Read the note 1 min read
From the notebook
quantization.
1 note on this topic.