Architecture
All of taintwire is in src/index.ts, about 700 lines. There are three stages, and only the last one touches the database.
parseAst() flatten() load()
source ──────────────────> Babel AST ──────────────> rows + edges ───────────────> LadybugDB
cs-mast | babel (+ hash per per-type node rows, CREATE/ALTER tables,
node) per-type-pair edges COPY in batches
add(code, filename)
-
Parse.
parseAst()runs cs-mast or@babel/parserand returns the BabelFilenode. See Parsing and hashing. -
Flatten.
flatten()walks the tree once and produces:nodes:Map<type, Row[]>, one row per AST node, grouped by node type (one group per table).scopes: oneScoperow per lexical environment.rels.SON,rels.DECLARES,rels.CREATES_SCOPE,rels.PARENT_SCOPE,rels.IN_SCOPE,rels.REFERS_TO,rels.READS,rels.WRITES,rels.FLOWS_TO,rels.CALLS,rels.ARGUMENT_TOandrels.RETURNS_TO:Map<"fromType\0toType", Edge[]>, edges grouped by the pair of endpoint tables.rootId: theFilenode's id, whichadd()returns.
The last seven come from a short post-pass at the end of
flatten()that resolves names, then calls, once every binding in the file is known. See Flattening, DECLARES, Scopes, References, Value flow and Calls. -
Store the source.
CREATE (:Source {file, code}). This runs before any AST rows are written, so a duplicatefilenamefails onSource's primary key while the graph is still untouched. -
Load.
load()creates any missing node tables and edge-table pairs, then bulk-loads each group, including theScoperows, withCOPY ... FROM (UNWIND $rows ...)in batches of 5,000. See Storage.
Code map
| Symbol | Role |
|---|---|
PARSER_OPTIONS | @babel/parser options, identical to JS Recon's |
CS_MAST_CONFIG | cs-mast config: all scat categories, every other Babel type in sinc |
parseAst() | Picks the parser and wraps cs-mast ParseErrors |
COLUMNS | The column set shared by every AST node table |
SKIP_PROPS | Babel fields that never go into props |
DEFAULT_PAIRS | The FROM/TO pair each edge table is created with in open() |
RELS | Edge tables and their property columns (SON: key, idx; READS, WRITES: access, access_signature; CALLS: candidates; ARGUMENT_TO: arg_index, callsite; DECLARES, CREATES_SCOPE, PARENT_SCOPE, IN_SCOPE, REFERS_TO, FLOWS_TO, RETURNS_TO: none) |
SCOPE_COLUMNS | The columns of the Scope table |
scalar() | Turns value into a string for the value column |
bindingIds() / declared() | Which identifiers a node binds (the DECLARES targets) |
scopeKind() | Which kind of scope a node creates, if any |
hashOf() | The digest part of a node's cs-mast signature |
scopeRow() | A Scope row with its deterministic id and signature |
isRef() | Whether an Identifier child is a lexical name lookup |
facts() | A node's value flows and binding writes, before resolution |
flatten() | Iterative AST walk into rows and edges, then the resolution and call post-pass (callables(), paramName()) |
TaintGraph.open() | Opens the DB, creates Source and Scope, discovers existing tables and pairs |
TaintGraph.add() / load() | Parse, flatten, store source, bulk load |
ensureNodeTable() / ensureRelPair() | Lazy DDL |
batched() | One prepared COPY, run per 5,000-row slice |
TaintGraph.code() | Id to source text, via the Source table |
TaintGraph.save() | Row-by-row copy into a new database |
importCode (exported as import) | open() + add() |
Design choices
- One file, few abstractions. There's no plugin system and no per-relation classes. Edge tables are driven by the
RELSconst, so a new edge type means aRELSentry plus the code that emits its edges inflatten().open(),save()andload()already loop overRELS. - Rows are built in JS, then bulk-copied. Grouping is done in memory, so the database only sees large homogeneous
COPYs. The design notes sketched oneMERGEper node and edge, butCOPYis much faster. See Storage. - The AST is the schema. Table names,
SON.keyvalues andpropsfields come straight from Babel. Nothing is renamed, so Babel's docs and AST Explorer describe the graph exactly. - The source is stored. Every node can be turned back into source text from the database alone, including after
save()and reopening.