llm A field note by Vikrant Sharma
Someone finally built a Rosetta Stone for LLM pricing
Models.dev is an open-source database that tracks pricing, context windows, and rate limits across every major LLM provider. No more tab-sprawl to compare GPT-4 versus Claude costs.
Comparing LLM costs is a nightmare. You have OpenAI’s pricing page in one tab, Anthropic’s in another, Google’s somewhere else, and you are doing mental arithmetic to convert per-million-token rates into what your actual workload will cost. Models.dev is a GitHub repo that solves this. It is a structured database of AI model specs: pricing, context window sizes, rate limits, modalities. Everything in JSON. OpenAI, Anthropic, Google, Cohere, Mistral, all in one place. The interesting bit is that it is not just a static list. The repo is structured so you can query it programmatically. You could build a cost estimator that pulls live data from the repo, or a model router that picks the cheapest provider for a given input length. The data is permissively licensed, so you can fork it, embed it, whatever. I have been manually checking pricing pages every time I prototype something new. Last week I spent twenty minutes figuring out whether Claude 3.5 Sonnet was cheaper than GPT-4o for a 50k token summarisation job. This would have taken thirty seconds with a local clone of this repo. The Hacker News thread is full of people asking for features: embeddings models, multimodal pricing, vision token costs. All valid. But the core problem it solves is already useful. You can compare apples to apples without opening fifteen browser tabs. I am cloning this repo today. Next time I need to pick a model for a project, I will query the JSON instead of hunting through marketing pages.
Source: Models.dev: open-source database of AI model specs, pricing, and capabilities
