Monospace fonts for LLM tokens, not characters
Someone built a tool that generates fonts where every tokenizer chunk takes the same visual width. Suddenly prompt length is readable.
Read the note 1 min read
From the notebook
tokenization.
1 note on this topic.