Vikrant
Someone fit a reasoning model into 14MB
Needle2 runs on phones, smartwatches, and Raspberry Pis. The entire model is smaller than a single photo.
Read post
hardware
3 posts tagged.
- machine-learning
-
amdVikrant
AMD removed memory encryption from consumer Ryzen CPUs and nobody noticed
A firmware update silently killed Transparent Memory Encryption on consumer chips. The engineers stopped replying when users asked why.
Read post
-
llmVikrant
Someone ran a trillion-parameter LLM on consumer hardware with Optane memory
An enthusiast loaded a 1T-parameter model into 768GB of Intel Optane DIMMs and got 4 tokens per second on a single GPU. Slow, but it worked.
Read post