Training LLMs on an 8GB GPU is now a weekend project

Someone released a minimal repo for post-training LLMs on an 8GB GPU. Three techniques: supervised fine-tuning, direct preference optimisation, and group relative policy optimisation. All fit on the same hardware you use for gaming. The repo is called nano-llm-posttraining. The pitch is simplicity. Most post-training guides assume you have a cluster or at least a chunky A100. This one assumes you have a 3070 or equivalent and want to understand what is happening without wading through framework abstractions. DPO is the interesting one here. Direct preference optimisation trains a model to prefer one output over another without needing a separate reward model. You give it pairs of responses where one is better, and it learns to push probabilities toward the good one. The math is simpler than reinforcement learning from human feedback, which is why it fits on 8GB. GRPO is newer. Group relative policy optimisation. Instead of pairwise comparisons, you sample multiple outputs from the model, rank them, and use that ranking as the training signal. It sits between supervised fine-tuning and full RLHF in complexity. I had not heard of it until this repo. The code is minimal because it skips the parts that do not matter for learning. No multi-node setup. No custom CUDA kernels. Just PyTorch and a clear training loop. If you wanted to understand how preference tuning actually works, this is the repo to read on a Sunday. The timing is good. A year ago you needed institutional resources to fine-tune anything past toy models. Now the barrier is a weekend and a gaming GPU. That shift matters more than most benchmarks.

---

Source: Show HN: Minimal LLM Post-Training Experiments on an 8GB GPU (SFT, DPO, GRPO)