LLM coding assistants have a variance problem
Dan Luu measured the same coding task twenty times with the same prompt. The variance in output quality was higher than the difference between model versions.
Read the note 1 min read
From the notebook
claude.
2 notes on this topic.
