041d7dc291
Adds tokenizeWithState(line, openComment) -> {tokens, commentLeftOpen}
and tokenizeLines(allLines) which threads the open-comment flag across
lines.
Viewer now tokenizes from line 0 (not just the visible slice) so the
state going into the visible window is correct, then renders the
slice. A line wholly inside a (* block is emitted as a single
'comment' token; PROGRAM and other keywords on those lines no longer
get falsely highlighted.
tokenize(line) kept as a thin wrapper for the single-line callers
(tests, future use).
|
||
|---|---|---|
| .. | ||
| integration | ||
| tui | ||
| unit | ||
| bench-results-v5.json | ||
| bench-results.json | ||
| bench.mjs | ||
| mock_watcher.py | ||
| test-fixes.mjs | ||
| TEST_OVERVIEW.md | ||