Stack overflows in production are humbling. Here's the iterative tree-sitter walker we landed on, what we got wrong, and why the fix was less code, not more.
Stack overflows in production are humbling. Here's the iterative tree-sitter walker we landed on, what we got wrong, and why the fix was less code, not more.
tree-sitter gives you a cursor API for walking the AST. It is iterative by design. We promptly wrote a recursive wrapper around it, because recursion is what you reach for when you've spent a decade writing compilers.
Then a customer indexed a generated TypeScript file with a 12,000-line union type. Stack depth: 12,000. Every Rust thread has an 8 MB stack. We blew it twice on the same file, in two different ways. Once in the parser, once in our walker. The extracted error path itself recursed.
We rewrote the walker as an explicit stack of cursors. Less code. No risk of overflow. Same behaviour. The lesson, again: the hard part of iteration isn't the iteration; it's giving up on recursion.
Beta is open. Five-minute setup. Source never leaves your network.