Back to the notes

DuckDB ditched yacc for a PEG parser and the error messages got readable

DuckDB 2.0 switched from yacc to a hand-written PEG parser. The goal was not speed. It was error messages that do not make you want to quit programming.

Close-up of a monitor displaying green and blue programming text
Rob Wingate / Unsplash Unsplash License

DuckDB 2.0 replaced their yacc-based SQL parser with a PEG parser. The move was not about performance. It was about error messages that actually help. Yacc parsers are fast. They are also terrible at explaining what went wrong. You get “syntax error near unexpected token” and a line number. PEG parsers track context. They know what the parser expected at the point of failure, which means they can say “expected a column name, got a semicolon” instead of dumping a state machine number on you. The DuckDB team hand-wrote the PEG parser instead of generating it from a grammar. That means they control the error recovery. When you forget a closing parenthesis in a subquery, the parser can guess where you meant to close it and keep going. It can report multiple errors in one pass instead of dying at the first problem. This is the kind of polish that does not show up in benchmarks. It shows up when you are debugging a CTE at 11pm and the error message points you to the actual mistake instead of three lines below it. Most SQL engines still use yacc because it works and nobody wants to rewrite a parser. DuckDB did it anyway. The tradeoff is maintenance. Hand-written parsers are more code to test and more surface area for bugs. But if your project is a developer tool, error messages are the UI. A confusing parser error is a support ticket waiting to happen. DuckDB chose debuggability over tradition and that is worth noticing.


Source: DuckDB V2 PEG-based SQL parser

Back to all notes

Behind the notes

Vikrant
Sharma.

Artificial Intelligence Engineer intern at Voxon Photonics in Adelaide. Studying a Master of Information and Communications Technology at UniSC, with a focus on data, machine learning and security.

Meet the person behind the work