Someone ran a trillion-parameter LLM on consumer hardware with Optane memory
An enthusiast loaded a 1T-parameter model into 768GB of Intel Optane DIMMs and got 4 tokens per second on a single GPU. Slow, but it worked.
Read the note 1 min read
From the notebook
memory.
1 note on this topic.