AI models are starting to ship as torrents
Distributed file sharing might be the only way to serve 70GB models without bankrupting yourself on egress fees.
From the notebook
6 notes on this topic.
Distributed file sharing might be the only way to serve 70GB models without bankrupting yourself on egress fees.
OpenAI and Anthropic slashed API prices by up to 80 percent in a few months. The pricing war is not about market share.
Project Suncatcher is Google's proposal to deploy GPU clusters in low Earth orbit. The pitch is simple: space is cold, solar power is free, and latency to ground stations is acceptable for batch jobs.
One database to rule them all. Raphael Bauer makes the case for ditching Redis, Elasticsearch, and message queues in favour of PostgreSQL extensions.
A GitHub repo sketches how Europe could pool scattered compute across universities and research labs to train a GPT-4 class model without buying a new datacenter.
The Norwegian University of Science and Technology is running LLM workloads on Chinese storage hardware. The performance numbers are interesting.