Signals vs. Query-Based Compilers

Summary

📖 tl;dr: With the rise of LSPs, query-based compilers have emerged as a new architecture. That architecture is much more similar and also different to Signals than I initial assumed them to be. Over the winter holidays curiosity got the better of me and I spent a lot of time reading up on how modern compilers achieve interactivity in the days of LSPs and tight editor integrations. And it turns out that modern compilers are built around the same concept as Signals in UI rendering, with some interesting different design choices. Old Days: The Pipeline Architecture The classic teachings about compilers present them as a linear sequence of phases that code runs through until you end up with the final binary. Writing a compiler this way is pretty straight forward, assuming that the language is reasonably simple (JavaScript is ridiculously complex and the opposite of simple). Source Text -> AST -> IR -> Assembly -> Linker -> Binary First, source code is transformed into an Abstract Syntax Tree (=AST) which transforms the input text into structured objects/structs. It's the place where syntax errors, grammar errors and those kinds of things are caught. For example: This JavaScript source code... const a = 42; ...is transformed into something similar to this AST: { "type": "VariableDeclaration", "kind": "const", "declarations": [ { "type": "VariableDeclarator", "id": { "type": "Identifier", "name": "a" }, "init": { "type": "NumericLiteral", "value": 42 } } ]} The AST is then pushed through a few more steps until we end up with the final binary. The actual details of these steps are not important for this blog post and what we described here is very oversimplified. But it's important to know that compilers typically run code through a bunch of different stages until code can be executed. Doing all of this takes time. Too much time to run on every key stroke. Modern: Query Based Compilers When a developer changes a single file by typing a single letter, there is a lot of work...