ewilliams28/turnstone
1
0
Fork 0
mirror of https://github.com/turnstonelabs/turnstone.git synced 2026-08-12 23:12:23 -06:00
Code Issues Packages Projects Releases Wiki Activity

fix(renderer): mermaid streaming parser errors + progressive hljs (#510)

Browse Source 
* fix(renderer): mermaid streaming parser errors + progressive hljs

Live streaming was rendering mermaid diagrams with `Parse error,
got 'PS'` messages — bare `(`, `[`, `{` inside unquoted edge / node
labels re-entered Mermaid's shape parser. Two unrelated streaming-
specific issues in the renderer pile-up here; this commit addresses
both plus a follow-on UX improvement for code highlighting.

## Mermaid label autoquoter

`_normalizeMermaidSource` wraps two label forms that Mermaid rejects
when they contain bare shape-delimiter chars:

  1. Edge labels:  `|content|`  →  `|"content"|`
  2. Rectangle node labels:  `ID[content]`  →  `ID["content"]`

Shapes whose syntax already nests delimiters — cylinders `[(...)`,
subroutines `[[...]]`, trapezoids `[/.../]` `[\...\]`, circles
`((...))`, hexagons `{{...}}`, diamonds `{...}` — are intentionally
left alone (their inner delimiters are part of the shape syntax;
quoting would corrupt them). Labels already wrapped in `"..."` are
also left alone. The rewrite is idempotent and runs before the
mermaid SVG cache lookup so identical malformed input hits the
cache on re-render rather than re-quoting per tick.

## Markdown fence-pair regex

The old fence regex `/(```+)([^\s`]*)\n([\s\S]*?)\1/g` would, mid-
stream, pair an unclosed ```mermaid open with the OPENING backticks
of a later ```python fence as the "close", handing mermaid a
truncated source. New regex:

    /(```+)([^\s`]*)\n((?:(?!\1)[\s\S])*?)\1[ \t]*(?=\n|$)/g

Two constraints close the gap:

  - `(?!\1)` inside the content quantifier blocks the lazy matcher
    from extending across another N-backtick run. Smaller inner
    counts (e.g. 3-backtick inner inside a 4-backtick outer) still
    pass since `\1` is the open's actual count.
  - `[ \t]*(?=\n|$)` after `\1` forces the close to a line
    boundary, so ```python (open with a language tag) can't
    masquerade as a previous fence's close.

Together: an unclosed fence stays as plain markdown until its true
close arrives, so neither mermaid nor hljs ever sees a mid-stream
truncated source.

## Progressive hljs

Extracted `postRenderHljs` from `postRenderMarkdown` with a source-
keyed `_hljsCache` (FIFO, cap 64, keyed on `language:source`) and
wired it into `_streamingRenderApply`. Closed code fences are now
syntax-highlighted as they stream in, matching the progressive
mermaid pattern from #426. Per-tick cost stays cheap because the
cache returns the pre-tokenized HTML synchronously on hit; only
unique (language, source) pairs pay `hljs.highlightElement`.

## Internal cleanup from the review pipeline

  - `_cacheFifoEntry(cache, key, value, max)` replaces the duplicated
    `_cacheHljsEntry` and `_cacheMermaidEntry`. Single tested
    implementation across four caches (hljs, mermaid svg, mermaid
    error, mermaid normalize memo). The "don't evict on overwrite"
    invariant is pinned per-cache in tests.
  - `_mermaidNormalizeCache` memoizes raw textContent → normalized
    output so the per-rAF-tick autoquoter split + regex doesn't
    repeat for unchanged diagrams. Eviction shares
    `_MERMAID_CACHE_MAX` with the SVG cache it feeds.

## Tests

The fake DOM in tests/test_renderer_js.py grew a few capabilities
to drive these paths:
  - `classList` is now array-like (length + indexed access) so the
    hljs language-extraction loop works.
  - `textContent` setter mirrors the real-DOM side effect of
    entity-escaping into innerHTML, so `escapeHtml()` round-trips
    (otherwise every `renderMarkdown` returns empty `<p>` tags).
  - `querySelectorAll` handles both `pre code.language-mermaid`
    and `pre code[class*='language-']`.

Added: 6 fence-pairing regression cases, 9 hljs-progressive cases
(cache hit / distinct sources / language separation / NO_HIGHLIGHT
langs / terminal class / eviction / overwrite / postRenderMarkdown
wraps hljs / _streamingRenderApply invokes hljs), 11 autoquoter
cases including both diagram sources from the live screenshot
encoded verbatim as parametrized regressions, and 3 normalize-memo
cases (populates on first call, consulted before normalize via
sentinel pre-seed, distinct sources cache separately).

Total: 104 renderer tests pass (was 67).

* fix(renderer): apply Copilot review feedback on #510

Two doc / harness adjustments from the PR review — no behavior
change in production code.

- The `_mermaidNormalizeCache` comment claimed eviction "stays in
  lockstep with the SVG cache". That was misleading: the two
  caches key on different things (raw textContent vs normalized
  source) and evict independently. Updated the comment to describe
  what they actually share (the cap, for memory footprint) and
  what they don't (positional coupling), and to note that the memo
  deliberately survives `_initMermaid` since normalize output is
  theme-independent.

- The fake DOM in tests/test_renderer_js.py had `innerHTML` setter
  clear `children` but leave `_textContent` intact, so subsequent
  `textContent` reads could return stale data after an innerHTML
  mutation (real DOM invalidates textContent on innerHTML write).
  No current test triggered this, but it would mask future bugs
  that depend on innerHTML/textContent consistency. Setter now
  clears `_textContent`; the children-derived fallback in the
  getter returns `''` after the wholesale replace.

All 104 renderer tests still pass; ruff + mypy clean.
This commit is contained in:
Patrick Buckley
2026-05-11 15:57:46 -07:00
committed by GitHub
parent 8a847f5288
commit f8f076cf20
2 changed files with 920 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_renderer_js.py
+668 -19
View File
@@ -255,12 +255,17 @@ function makeEl(tag) {
setAttribute(k, v) { this._attrs[k] = v; },
getAttribute(k) { return this._attrs[k] !== undefined ? this._attrs[k] : null; },
get classList() {
// Real DOMTokenList is array-like (length + indexed access) AND
// exposes add/remove/contains. The hljs language-extraction
// loop reads .length + [j], so we return a fresh Array snapshot
// each get + bolt the mutator methods on. add/remove operate on
// the live _classes set so subsequent reads see updates.
const self = this;
return {
add(...c) { c.forEach(x => self._classes.add(x)); },
remove(...c) { c.forEach(x => self._classes.delete(x)); },
contains(c) { return self._classes.has(c); },
};
const arr = Array.from(self._classes);
arr.add = (...c) => c.forEach((x) => self._classes.add(x));
arr.remove = (...c) => c.forEach((x) => self._classes.delete(x));
arr.contains = (c) => self._classes.has(c);
return arr;
},
get className() { return Array.from(this._classes).join(' '); },
set className(v) {
@@ -269,9 +274,33 @@ function makeEl(tag) {
get textContent() {
return this._textContent || this.children.map(c => c.textContent || '').join('');
},
set textContent(v) { this._textContent = v; this.children = []; },
set textContent(v) {
// Real DOM: assigning textContent ALSO replaces innerHTML with
// an entity-escaped representation of the same text. escapeHtml
// (utils.js) round-trips via this side effect — without it,
// every escapeHtml() call returns '' and renderMarkdown emits
// empty <p> tags.
this._textContent = v;
this.children = [];
this._innerHTML = String(v)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;');
},
get innerHTML() { return this._innerHTML; },
set innerHTML(v) { this._innerHTML = v; this.children = []; },
set innerHTML(v) {
// Real DOM invalidates the previous textContent when innerHTML
// is replaced — leaving _textContent intact would return stale
// data from subsequent textContent reads and mask bugs that
// depend on innerHTML/textContent consistency. We don't HTML-
// parse here, so the cheap correct behavior is to clear
// _textContent and let the children-derived fallback in the
// textContent getter (which is empty after this children = [])
// take over.
this._innerHTML = v;
this.children = [];
this._textContent = '';
},
get isConnected() {
// In real DOM this checks attachment to the document; for the
// test harness we approximate via the parent chain. After
@@ -304,17 +333,28 @@ function makeEl(tag) {
this.parent = null;
},
querySelectorAll(selector) {
// Only supports the literal "pre code.language-mermaid"
// selector that postRenderMermaid uses.
// Supports the two selectors the post-render passes use:
// "pre code.language-mermaid" (postRenderMermaid)
// "pre code[class*='language-']" (postRenderHljs)
const out = [];
const wantsMermaid = selector === "pre code.language-mermaid";
function matchesLangAttr(el) {
for (const cls of el._classes) {
if (cls.startsWith('language-')) return true;
}
return false;
}
function walk(node) {
for (const c of (node.children || [])) {
if (
const isCodeInPre =
c.tagName === 'CODE' &&
c.parent && c.parent.tagName === 'PRE' &&
c._classes.has('language-mermaid')
) {
out.push(c);
c.parent && c.parent.tagName === 'PRE';
if (isCodeInPre) {
if (wantsMermaid) {
if (c._classes.has('language-mermaid')) out.push(c);
} else if (matchesLangAttr(c)) {
out.push(c);
}
}
walk(c);
}
@@ -352,6 +392,20 @@ global.mermaid = {
},
};
// hljs stub. highlightElement mutates the element in place: replaces
// innerHTML with a deterministic synthetic span keyed by the source,
// and adds the hljs class — same surface postRenderHljs depends on.
// hljsHighlightCallCount lets tests assert "ran N times" semantics.
let hljsHighlightCallCount = 0;
global.hljs = {
configure: () => {},
highlightElement: (el) => {
hljsHighlightCallCount++;
el._classes.add('hljs');
el._innerHTML = '<span class="hljs-tok">' + el._textContent + '</span>';
},
};
vm.runInThisContext(fs.readFileSync(%(utils)s, 'utf8'));
vm.runInThisContext(fs.readFileSync(%(renderer)s, 'utf8'));
@@ -514,7 +568,7 @@ def test_mermaid_cache_evicts_oldest_at_cap() -> None:
scenario = """
const cap = _MERMAID_CACHE_MAX;
for (let i = 0; i < cap + 5; i++) {
_cacheMermaidEntry(_mermaidSvgCache, 'src-' + i, {svg: 'svg-' + i, bindFunctions: null});
_cacheFifoEntry(_mermaidSvgCache, 'src-' + i, {svg: 'svg-' + i, bindFunctions: null}, cap);
}
process.stdout.write(JSON.stringify({
size: _mermaidSvgCache.size,
@@ -536,10 +590,10 @@ def test_mermaid_overwrite_does_not_evict() -> None:
const cap = _MERMAID_CACHE_MAX;
// Fill exactly to cap.
for (let i = 0; i < cap; i++) {
_cacheMermaidEntry(_mermaidSvgCache, 'src-' + i, {svg: 'svg-' + i, bindFunctions: null});
_cacheFifoEntry(_mermaidSvgCache, 'src-' + i, {svg: 'svg-' + i, bindFunctions: null}, cap);
}
// Overwrite an existing entry — must not evict src-0.
_cacheMermaidEntry(_mermaidSvgCache, 'src-5', {svg: 'svg-updated', bindFunctions: null});
_cacheFifoEntry(_mermaidSvgCache, 'src-5', {svg: 'svg-updated', bindFunctions: null}, cap);
process.stdout.write(JSON.stringify({
size: _mermaidSvgCache.size,
hasOldest: _mermaidSvgCache.has('src-0'),
@@ -558,8 +612,8 @@ def test_mermaid_cache_cleared_on_init() -> None:
— the rendered output depends on themeVariables which change
on init."""
scenario = """
_cacheMermaidEntry(_mermaidSvgCache, 'src-1', {svg: 'old', bindFunctions: null});
_cacheMermaidEntry(_mermaidErrorCache, 'src-bad', 'old error');
_cacheFifoEntry(_mermaidSvgCache, 'src-1', {svg: 'old', bindFunctions: null}, _MERMAID_CACHE_MAX);
_cacheFifoEntry(_mermaidErrorCache, 'src-bad', 'old error', _MERMAID_CACHE_MAX);
_initMermaid();
process.stdout.write(JSON.stringify({
svgSize: _mermaidSvgCache.size,
@@ -624,3 +678,598 @@ def test_streaming_render_invokes_mermaid_post_render() -> None:
"_streamingRenderApply must call postRenderMermaid for "
"progressive diagram rendering during streaming"
)
# ---------------------------------------------------------------------------
# _normalizeMermaidSource — autoquote labels with bare shape-delimiter
# chars. Mermaid rejects unquoted ( ) [ ] { } inside other labels with
# a "got 'PS'" parse error (paren-start in shape context). The two
# diagrams in the screenshot regression case are encoded here verbatim.
# ---------------------------------------------------------------------------
def _run_normalize(source: str) -> str:
"""Drive _normalizeMermaidSource against the JS harness and return
its output. The function is pure, so no container / mermaid stub
setup is required."""
scenario = f"""
const input = {json.dumps(source)};
const output = _normalizeMermaidSource(input);
process.stdout.write(JSON.stringify({{ output: output }}));
"""
out = _run_mermaid_scenario(scenario)
return str(out["output"])
# Diagram 1 from the screenshot regression — unquoted edge labels with
# parens and <br/> markers. Mermaid rejects both edge labels with
# "got 'PS'"; quoting them resolves it.
_SCREENSHOT_DIAGRAM_1_IN = (
"flowchart LR\n"
' A["vllm-openai:nightly<br/>commit 5536fc0c0<br/>2026-05-11 11:59"]'
" -->|22 upstream<br/>main commits<br/>(10 csrc, but<br/>no new bindings)|"
' B["fork merge_base<br/>7863fff6e5<br/>2026-05-12 00:27"]\n'
" B -->|13 jasl patches<br/>(Python only:<br/>tunings, kernels,"
"<br/>warmup, etc.)|"
' C["ds4-sm120-preview-dev<br/>acc3455b1e"]'
)
_SCREENSHOT_DIAGRAM_1_OUT = (
"flowchart LR\n"
' A["vllm-openai:nightly<br/>commit 5536fc0c0<br/>2026-05-11 11:59"]'
' -->|"22 upstream<br/>main commits<br/>(10 csrc, but<br/>no new bindings)"|'
' B["fork merge_base<br/>7863fff6e5<br/>2026-05-12 00:27"]\n'
' B -->|"13 jasl patches<br/>(Python only:<br/>tunings, kernels,'
'<br/>warmup, etc.)"|'
' C["ds4-sm120-preview-dev<br/>acc3455b1e"]'
)
# Diagram 2 from the screenshot regression — unquoted RECTANGLE node
# label `D[untouched<br/>(.so, _version.py,<br/>install-vendored)]`.
# Same parser failure mode; quoting the bracket label fixes it.
_SCREENSHOT_DIAGRAM_2_IN = (
"flowchart LR\n"
" A[nightly's vllm/<br/>installed package] --> B{tar -xf<br/>fork-vllm.tar}\n"
" B -->|in archive| C[overwritten with<br/>fork's version]\n"
" B -->|not in archive| D[untouched<br/>(.so, _version.py,"
"<br/>install-vendored)]\n"
" E[explicit rm of 1 file<br/>deleted upstream] --> B"
)
_SCREENSHOT_DIAGRAM_2_OUT = (
"flowchart LR\n"
" A[nightly's vllm/<br/>installed package] --> B{tar -xf<br/>fork-vllm.tar}\n"
" B -->|in archive| C[overwritten with<br/>fork's version]\n"
' B -->|not in archive| D["untouched<br/>(.so, _version.py,'
'<br/>install-vendored)"]\n'
" E[explicit rm of 1 file<br/>deleted upstream] --> B"
)
@pytest.mark.parametrize(
("source", "expected"),
[
(_SCREENSHOT_DIAGRAM_1_IN, _SCREENSHOT_DIAGRAM_1_OUT),
(_SCREENSHOT_DIAGRAM_2_IN, _SCREENSHOT_DIAGRAM_2_OUT),
],
)
def test_mermaid_autoquote_fixes_screenshot_diagrams(source: str, expected: str) -> None:
"""The two exact diagrams from the screenshot regression. If
these stop being rewritten with quoted labels, mermaid will
again reject them with `Expecting ... got 'PS'` during live
streaming."""
assert _run_normalize(source) == expected
@pytest.mark.parametrize(
"source",
[
# Clean diagram — no shape delimiters in any label.
"graph TD\n A[foo] --> B[bar]",
# Edge label with no special chars.
"A --> B\nA -->|plain text| B",
# Already-correctly-quoted node label.
'A["already (quoted)"] --> B',
# Already-correctly-quoted edge label.
'A -->|"already (quoted)"| B',
# Cylinder shape — inner () is part of the shape syntax.
"A[(database)] --> B",
# Subroutine shape — inner [] is part of the shape syntax.
"A[[subroutine]] --> B",
# Trapezoid shape — inner / is part of the shape syntax.
"A[/trapezoid/] --> B",
# Reverse trapezoid.
"A[\\trap\\] --> B",
# Mermaid directive — braces here are config, not a label.
'%%{init: {"theme": "dark"}}%%\ngraph TD\n A --> B',
# <br/> tags on their own don't trip quoting.
"A[line1<br/>line2] --> B",
# Sequence diagram — different grammar; we only target labels
# in shape/edge syntax that match the regex anchors.
"sequenceDiagram\n A->>B: hello",
],
)
def test_mermaid_autoquote_leaves_valid_source_alone(source: str) -> None:
"""The autoquoter must not rewrite syntactically valid Mermaid —
a false positive here would break a working diagram. Each case
covers a syntax form whose delimiters are intentional and must
not be wrapped."""
assert _run_normalize(source) == source
def test_mermaid_autoquote_edge_label_with_parens() -> None:
"""Bare-parens edge label gets wrapped. The bare `(` would
otherwise re-enter Mermaid's shape parser."""
src = "A -->|note (with parens)| B"
assert _run_normalize(src) == 'A -->|"note (with parens)"| B'
def test_mermaid_autoquote_node_label_with_parens() -> None:
"""Bare-parens node label gets wrapped."""
src = "D[label (foo, bar)]"
assert _run_normalize(src) == 'D["label (foo, bar)"]'
def test_mermaid_autoquote_node_label_with_braces() -> None:
"""Bare-braces in a rectangle label get wrapped. (Diamond {}
shapes are left alone — only single-bracket [] labels are
rewritten.)"""
src = "A[config {key: value}]"
assert _run_normalize(src) == 'A["config {key: value}"]'
def test_mermaid_autoquote_preserves_br_tag_with_parens() -> None:
"""`<br/>` inside a label that also has parens stays — only the
quoting needs to be added around the whole label."""
src = "A[line1<br/>(line2)] --> B"
assert _run_normalize(src) == 'A["line1<br/>(line2)"] --> B'
def test_mermaid_autoquote_skips_label_with_internal_quote() -> None:
"""If a label contains a literal `"`, wrapping would produce
nested unescaped quotes. The autoquoter must punt — leaving the
parse error to surface, rather than silently producing a worse
one."""
src = 'A[he said "hi" (lol)]'
assert _run_normalize(src) == src
def test_mermaid_autoquote_multiple_edges_on_one_line() -> None:
"""Both edge labels on a single line get rewritten independently."""
src = "A -->|first (paren)| B -->|second (paren)| C"
expected = 'A -->|"first (paren)"| B -->|"second (paren)"| C'
assert _run_normalize(src) == expected
def test_mermaid_autoquote_normalized_source_hits_cache() -> None:
"""The SVG cache keys on the normalized source — same malformed
input that the LLM streamed earlier still hits the cache on
re-render rather than re-invoking mermaid.render every tick."""
bad = "A[label (with parens)] --> B"
scenario = (
_build_mermaid_container_js([bad])
+ _MERMAID_DRAIN_JS
+ """
postRenderMermaid(container);
setTimeout(() => setTimeout(() => {
const container2 = buildContainer(sources);
postRenderMermaid(container2);
setTimeout(() => {
process.stdout.write(JSON.stringify({
renderCalls: renderCallCount,
normalized: container.children[0]._attrs['data-mermaid-source'],
}));
}, 0);
}, 0), 0);
"""
)
out = _run_mermaid_scenario(scenario)
assert out["renderCalls"] == 1, "second render bypassed the cache"
assert out["normalized"] == 'A["label (with parens)"] --> B'
def test_mermaid_normalize_memo_populates_on_first_call() -> None:
"""First postRenderMermaid call populates _mermaidNormalizeCache
with a raw→normalized entry. A second call on identical raw
textContent then hits the memo (size stays at 1, no second
normalize call), which is the perf-1 fix — avoids re-running
split + per-line regex per rAF tick when the diagram hasn't
changed."""
bad = "A[label (with parens)] --> B"
scenario = (
_build_mermaid_container_js([bad])
+ _MERMAID_DRAIN_JS
+ """
postRenderMermaid(container);
const sizeAfterFirst = _mermaidNormalizeCache.size;
const cachedNorm = _mermaidNormalizeCache.get(sources[0]);
// Re-render on a fresh container with the same source.
const container2 = buildContainer(sources);
postRenderMermaid(container2);
setTimeout(() => setTimeout(() => {
process.stdout.write(JSON.stringify({
sizeAfterFirst: sizeAfterFirst,
cachedNorm: cachedNorm,
sizeAfterSecond: _mermaidNormalizeCache.size,
}));
}, 0), 0);
"""
)
out = _run_mermaid_scenario(scenario)
assert out["sizeAfterFirst"] == 1, "first call didn't populate normalize memo"
assert out["cachedNorm"] == 'A["label (with parens)"] --> B'
assert out["sizeAfterSecond"] == 1, (
"second call added a new entry — memo missed on identical source"
)
def test_mermaid_normalize_memo_is_consulted_before_normalize() -> None:
"""Pre-seed _mermaidNormalizeCache with a sentinel value for a
raw source. postRenderMermaid must use the sentinel rather than
re-running _normalizeMermaidSource. Catches a regression where
the memo gets populated but the lookup path is skipped."""
bad = "A[label (with parens)] --> B"
sentinel = "SENTINEL_FROM_MEMO --> X"
raw_js = json.dumps(bad)
sentinel_js = json.dumps(sentinel)
scenario = (
_build_mermaid_container_js([bad])
+ _MERMAID_DRAIN_JS
+ f"""
_mermaidNormalizeCache.set({raw_js}, {sentinel_js});
postRenderMermaid(container);
setTimeout(() => setTimeout(() => {{
process.stdout.write(JSON.stringify({{
sourceAttr: container.children[0]._attrs['data-mermaid-source'],
}}));
}}, 0), 0);
"""
)
out = _run_mermaid_scenario(scenario)
assert out["sourceAttr"] == sentinel, (
"postRenderMermaid bypassed the normalize memo and re-ran normalize"
)
def test_mermaid_normalize_memo_distinct_sources_cache_separately() -> None:
"""Two distinct raw sources produce two memo entries. Confirms
the memo keys on raw textContent, not on something coarser like
container identity."""
bad1 = "A[label (with parens)] --> B"
bad2 = "C[other (label)] --> D"
scenario = (
_build_mermaid_container_js([bad1, bad2])
+ _MERMAID_DRAIN_JS
+ """
postRenderMermaid(container);
setTimeout(() => setTimeout(() => {
process.stdout.write(JSON.stringify({
size: _mermaidNormalizeCache.size,
hasBad1: _mermaidNormalizeCache.has(sources[0]),
hasBad2: _mermaidNormalizeCache.has(sources[1]),
}));
}, 0), 0);
"""
)
out = _run_mermaid_scenario(scenario)
assert out["size"] == 2
assert out["hasBad1"] is True
assert out["hasBad2"] is True
# ---------------------------------------------------------------------------
# Code-fence pairing — close requires \n / EOS, content can't cross
# another close-pattern. Repros the streaming bug where ```mermaid +
# later ```python were paired by the regex, handing mermaid a
# truncated source.
# ---------------------------------------------------------------------------
def _render_md(source: str) -> str:
"""Drive renderMarkdown against the JS harness and return the
rendered HTML. The function is a pure string transform; no DOM
container scaffolding is required."""
scenario = f"""
const input = {json.dumps(source)};
const output = renderMarkdown(input);
process.stdout.write(JSON.stringify({{ output: output }}));
"""
out = _run_mermaid_scenario(scenario)
return str(out["output"])
_FENCE = "```"
def test_fence_partial_open_emits_no_code_block() -> None:
"""While a fence is still open and there's no other ``` later in
the buffer, no <code> block is emitted — the open fence stays as
plain markdown text until the real close arrives."""
src = "Intro\n" + _FENCE + 'mermaid\nA["x"] -->|note (with parens)| B["y"]\nstill streaming'
html = _render_md(src)
assert "<code" not in html, f"open fence should not emit <code> mid-stream: {html!r}"
def test_fence_partial_with_later_open_does_not_pair_wrongly() -> None:
"""Before the fence-pair fix: an unclosed ```mermaid followed by
a ```python (also unclosed) would have paired up as
<code class=mermaid>...</code>python..., handing mermaid a
truncated source. With the new regex, neither fence emits a
block until its OWN closing line arrives."""
src = "Intro\n" + _FENCE + "mermaid\nA --> B\n" + _FENCE + 'python\nprint("hi")'
html = _render_md(src)
assert 'class="language-mermaid"' not in html, (
f"mermaid fence should not emit while open: {html!r}"
)
assert 'class="language-python"' not in html, (
f"python fence should not emit while open: {html!r}"
)
def test_fence_close_paired_with_next_open_is_rejected() -> None:
"""Repro of the live-streaming failure: mermaid fence open, then
```python opens and ``` closes the python block. Without the
fix, the regex paired mermaid's open with python's *open* (or
backtracked all the way to python's close), producing
<code class=mermaid>truncated</code>. With the fix mermaid stays
open (content can't cross another \\1 run; close must be at line
boundary) and only python's pair matches."""
src = (
"Intro\n"
+ _FENCE
+ 'mermaid\nA["x"] -->|note (with parens)| B["y"]\n'
+ _FENCE
+ 'python\nprint("hi")\n'
+ _FENCE
)
html = _render_md(src)
assert 'class="language-mermaid"' not in html, f"mermaid fence misparing reintroduced: {html!r}"
assert 'class="language-python"' in html, f"python fence on its own should match: {html!r}"
def test_fence_closed_emits_code_block() -> None:
"""Baseline: a properly closed fence with its close on its own
line emits the <code> block as expected — the anchor doesn't
break the normal case."""
src = "Intro\n" + _FENCE + "python\nimport os\n" + _FENCE + "\nAfter"
html = _render_md(src)
assert 'class="language-python"' in html
assert "import os" in html
def test_fence_close_at_end_of_buffer_emits() -> None:
"""A fence that closes at the very end of the buffer (no trailing
newline) still emits — the anchor accepts end-of-string as a
valid line boundary, so the rehydration / static-render path
where the buffer ends cleanly at ``` still works."""
src = "Intro\n" + _FENCE + "python\nimport os\n" + _FENCE
html = _render_md(src)
assert 'class="language-python"' in html
assert "import os" in html
def test_fence_close_with_trailing_whitespace_emits() -> None:
"""A close followed only by spaces / tabs before \\n still counts
— CommonMark allows trailing whitespace on the close line."""
src = "Intro\n" + _FENCE + "python\nimport os\n" + _FENCE + " \nAfter"
html = _render_md(src)
assert 'class="language-python"' in html
# ---------------------------------------------------------------------------
# postRenderHljs — progressive syntax highlighting + source-keyed cache
# ---------------------------------------------------------------------------
def _build_hljs_container_js(blocks: list[tuple[str, str]]) -> str:
"""Build a container with <pre><code class="language-LANG"> blocks.
``blocks`` is a list of ``(language, source)`` tuples — the language
becomes the ``language-X`` class, the source becomes textContent."""
arr = "[" + ", ".join(f"[{json.dumps(lang)}, {json.dumps(src)}]" for lang, src in blocks) + "]"
return f"""
function buildHljsContainer(blocks) {{
const container = document.createElement('div');
for (const [lang, src] of blocks) {{
const pre = document.createElement('pre');
const code = document.createElement('code');
code.classList.add('language-' + lang);
code.textContent = src;
pre.appendChild(code);
container.appendChild(pre);
}}
return container;
}}
const blocks = {arr};
const container = buildHljsContainer(blocks);
"""
def test_hljs_cache_hit_skips_highlight_call() -> None:
"""Two postRenderHljs calls on identical source must invoke
hljs.highlightElement exactly once — the second call hits the
cache and applies the stored markup synchronously. Mirrors the
mermaid SVG-cache invariant that lets streamingRender fire on
every rAF tick without re-tokenizing every code block."""
scenario = (
_build_hljs_container_js([("python", "import os")])
+ """
postRenderHljs(container);
const container2 = buildHljsContainer(blocks);
postRenderHljs(container2);
process.stdout.write(JSON.stringify({
highlightCalls: hljsHighlightCallCount,
cacheSize: _hljsCache.size,
firstHtml: container.children[0].children[0]._innerHTML,
secondHtml: container2.children[0].children[0]._innerHTML,
secondHasHljsClass: container2.children[0].children[0]._classes.has('hljs'),
}));
"""
)
out = _run_mermaid_scenario(scenario)
assert out["highlightCalls"] == 1, (
"second postRenderHljs call invoked highlightElement — cache miss"
)
assert out["cacheSize"] == 1
assert out["firstHtml"] == out["secondHtml"]
assert out["secondHasHljsClass"] is True
def test_hljs_distinct_sources_highlight_independently() -> None:
"""Distinct sources each trigger one highlight and cache one entry.
Cache key includes the source string, not e.g. just the language."""
scenario = (
_build_hljs_container_js([("python", "import os"), ("python", "print('hi')")])
+ """
postRenderHljs(container);
process.stdout.write(JSON.stringify({
highlightCalls: hljsHighlightCallCount,
cacheSize: _hljsCache.size,
}));
"""
)
out = _run_mermaid_scenario(scenario)
assert out["highlightCalls"] == 2
assert out["cacheSize"] == 2
def test_hljs_cache_separates_by_language() -> None:
"""Same source text under different language fences must NOT
collide in the cache — language is part of the key. Otherwise a
`python` block of `foo` and a `ruby` block of `foo` would share
a single (wrongly-highlighted) cache entry."""
scenario = (
_build_hljs_container_js([("python", "foo"), ("ruby", "foo")])
+ """
postRenderHljs(container);
process.stdout.write(JSON.stringify({
highlightCalls: hljsHighlightCallCount,
cacheSize: _hljsCache.size,
}));
"""
)
out = _run_mermaid_scenario(scenario)
assert out["highlightCalls"] == 2
assert out["cacheSize"] == 2
def test_hljs_skips_no_highlight_langs() -> None:
"""language-mermaid / language-text / language-plaintext etc. must
get the `nohighlight` class without invoking hljs.highlightElement.
Highlighting plaintext or mermaid source would be both wasteful
and ugly."""
scenario = (
_build_hljs_container_js(
[("mermaid", "graph TD\\nA-->B"), ("text", "plain"), ("plaintext", "p")]
)
+ """
postRenderHljs(container);
process.stdout.write(JSON.stringify({
highlightCalls: hljsHighlightCallCount,
cacheSize: _hljsCache.size,
mermaidNoHighlight: container.children[0].children[0]._classes.has('nohighlight'),
textNoHighlight: container.children[1].children[0]._classes.has('nohighlight'),
plaintextNoHighlight: container.children[2].children[0]._classes.has('nohighlight'),
}));
"""
)
out = _run_mermaid_scenario(scenario)
assert out["highlightCalls"] == 0
assert out["cacheSize"] == 0
assert out["mermaidNoHighlight"] is True
assert out["textNoHighlight"] is True
assert out["plaintextNoHighlight"] is True
def test_hljs_terminal_lang_marks_pre_for_terminal_styling() -> None:
"""Shell-family languages (bash / sh / zsh / console / terminal)
must add the `code-terminal` class to the parent <pre>, so the
stylesheet can give them the terminal look-and-feel."""
scenario = (
_build_hljs_container_js([("bash", "echo hi")])
+ """
postRenderHljs(container);
process.stdout.write(JSON.stringify({
highlightCalls: hljsHighlightCallCount,
preHasTerminalClass: container.children[0]._classes.has('code-terminal'),
}));
"""
)
out = _run_mermaid_scenario(scenario)
assert out["highlightCalls"] == 1
assert out["preHasTerminalClass"] is True
def test_hljs_cache_evicts_oldest_at_cap() -> None:
"""FIFO eviction at _HLJS_CACHE_MAX. Mirrors the mermaid cache —
prevents unbounded growth on long sessions with many distinct
code blocks."""
scenario = """
const cap = _HLJS_CACHE_MAX;
for (let i = 0; i < cap + 5; i++) {
_cacheFifoEntry(_hljsCache, 'key-' + i, 'val-' + i, cap);
}
process.stdout.write(JSON.stringify({
size: _hljsCache.size,
hasOldest: _hljsCache.has('key-0'),
hasNewest: _hljsCache.has('key-' + (cap + 4)),
}));
"""
out = _run_mermaid_scenario(scenario)
assert out["size"] == 64
assert out["hasOldest"] is False
assert out["hasNewest"] is True
def test_hljs_overwrite_does_not_evict() -> None:
"""Overwriting an existing key is an in-place update, not a new
insertion — must not evict the oldest unrelated entry. Same
invariant as the mermaid cache."""
scenario = """
const cap = _HLJS_CACHE_MAX;
for (let i = 0; i < cap; i++) {
_cacheFifoEntry(_hljsCache, 'key-' + i, 'val-' + i, cap);
}
_cacheFifoEntry(_hljsCache, 'key-5', 'val-updated', cap);
process.stdout.write(JSON.stringify({
size: _hljsCache.size,
hasOldest: _hljsCache.has('key-0'),
updated: _hljsCache.get('key-5'),
}));
"""
out = _run_mermaid_scenario(scenario)
assert out["size"] == 64
assert out["hasOldest"] is True, "overwrite evicted oldest unnecessarily"
assert out["updated"] == "val-updated"
def test_post_render_markdown_invokes_hljs() -> None:
"""postRenderMarkdown is the public end-of-stream entry point and
must still run syntax highlighting after the postRenderHljs
refactor — regression guard for the public API surface that
app.js / coordinator code already call."""
scenario = (
_build_hljs_container_js([("python", "import os")])
+ """
postRenderMarkdown(container);
process.stdout.write(JSON.stringify({
highlightCalls: hljsHighlightCallCount,
hasHljsClass: container.children[0].children[0]._classes.has('hljs'),
}));
"""
)
out = _run_mermaid_scenario(scenario)
assert out["highlightCalls"] == 1
assert out["hasHljsClass"] is True
def test_streaming_render_invokes_hljs() -> None:
"""_streamingRenderApply must call postRenderHljs so closed code
fences appear progressively (syntax-highlighted) during streaming,
not only at stream_end via streamingRenderFinalize. The cache
keeps the per-tick cost down to a synchronous lookup."""
body = _RENDERER_JS.read_text(encoding="utf-8")
start = body.index("function _streamingRenderApply")
hljs_call = body.find("postRenderHljs(el)", start, start + 4000)
assert hljs_call != -1, (
"_streamingRenderApply must call postRenderHljs for progressive "
"syntax highlighting during streaming"
)
turnstone/shared_static/renderer.js
+252 -54
View File
@@ -317,9 +317,34 @@ function renderMarkdown(text) {
// regex treated the outer-open and inner-open as a single fence
// pair, stranding the rest of the content with visible
// \x00CB{n}\x00 sentinels.
//
// Two constraints below close the gap that mid-stream buffers
// expose:
//
// 1. Content can't contain its own close pattern — `(?!\1)`
// inside the content quantifier blocks the lazy matcher
// from extending across another N-backtick run. Without
// this, a buffer like ```mermaid\n<partial>\n```python\n
// <partial>\n``` would extend mermaid's content all the
// way to the FINAL ```, swallowing python and handing
// mermaid a wrong (and incomplete-looking) source. With
// the lookahead, content stops at the first matching run
// and the open simply doesn't match anything until a true
// close arrives. Inner backticks of a SMALLER count (e.g.
// 3-backtick inner inside a 4-backtick outer) still pass
// since `\1` is the OPEN count, not just three.
//
// 2. The close must live at a line boundary — `[ \t]*(?=\n|$)`
// after `\1` forbids the close from being immediately
// followed by a language tag, so ```python opening another
// fence can't masquerade as the previous fence's close.
//
// Together these mean an unclosed fence stays as plain markdown
// until its true close arrives — no intermediate parse errors
// flash through mermaid / hljs while a stream is in flight.
var codeBlocks = [];
text = text.replace(
/(```+)([^\s`]*)\n([\s\S]*?)\1/g,
/(```+)([^\s`]*)\n((?:(?!\1)[\s\S])*?)\1[ \t]*(?=\n|$)/g,
function (m, _open, lang, code) {
var cssLang = _langToCssClass(lang);
codeBlocks.push(
@@ -718,38 +743,98 @@ var _TERMINAL_LANGS = {
};
var _hljsConfigured = false;
function postRenderMarkdown(containerEl) {
// Syntax highlighting (skip if highlight.js unavailable)
if (typeof hljs !== "undefined") {
if (!_hljsConfigured) {
hljs.configure({ ignoreUnescapedHTML: true });
_hljsConfigured = true;
// Source-keyed highlight cache. Mirrors _mermaidSvgCache: streamingRender
// replaces innerHTML wholesale on every rAF tick, so the <code> elements
// inside come up FRESH each tick — they don't carry the hljs class, and
// nothing on them carries forward. Without a cache, running hljs per
// tick would re-tokenize every code block every paint cycle on long
// streamed responses with many fences. With the cache, identical
// (language, source) pairs reuse the highlighted innerHTML synchronously.
//
// Cache miss runs hljs.highlightElement(el) (which mutates the element
// in place: replaces its innerHTML with highlighted span markup and
// adds the hljs class) and stores the resulting markup. Cache hit
// assigns that stored markup to el.innerHTML and re-adds the hljs
// class manually — semantically equivalent to a fresh highlightElement
// call without paying for re-tokenization.
//
// The cached value is the structured span markup that hljs itself
// produced from already-escaped text content, so re-assigning it to
// innerHTML doesn't widen the XSS surface beyond what hljs.highlight
// Element already does.
//
// FIFO-bounded so a long session with many distinct code blocks can't
// grow unbounded.
var _hljsCache = new Map();
var _HLJS_CACHE_MAX = 64;
// Shared FIFO eviction helper for the source-keyed caches in this
// file (_hljsCache, _mermaidSvgCache, _mermaidErrorCache, plus the
// raw→normalized mermaid memo). Only evicts the oldest when inserting
// a NEW key — overwriting an existing key is an in-place update and
// must not pay the eviction cost (which would drop an unrelated
// cached entry). The `cache_overwrite_does_not_evict` tests pin this
// invariant per cache.
function _cacheFifoEntry(cache, key, value, max) {
if (!cache.has(key) && cache.size >= max) {
var firstKey = cache.keys().next().value;
cache.delete(firstKey);
}
cache.set(key, value);
}
function _applyCachedHljs(el, cachedHtml) {
el.innerHTML = cachedHtml;
el.classList.add("hljs");
}
function postRenderHljs(containerEl) {
if (typeof hljs === "undefined") return;
if (!_hljsConfigured) {
hljs.configure({ ignoreUnescapedHTML: true });
_hljsConfigured = true;
}
var codeEls = containerEl.querySelectorAll("pre code[class*='language-']");
for (var i = 0; i < codeEls.length; i++) {
var el = codeEls[i];
// Already-highlighted element (e.g. postRenderMarkdown called twice
// on the same DOM with no intervening innerHTML replace). The
// streaming path replaces innerHTML wholesale per tick, so this
// guard primarily protects the non-streaming render path.
if (el.classList.contains("hljs")) continue;
// Extract language name from class
var langClass = "";
for (var j = 0; j < el.classList.length; j++) {
if (el.classList[j].startsWith("language-")) {
langClass = el.classList[j].substring(9);
break;
}
}
var codeEls = containerEl.querySelectorAll("pre code[class*='language-']");
for (var i = 0; i < codeEls.length; i++) {
var el = codeEls[i];
if (el.classList.contains("hljs")) continue;
// Extract language name from class
var langClass = "";
for (var j = 0; j < el.classList.length; j++) {
if (el.classList[j].startsWith("language-")) {
langClass = el.classList[j].substring(9);
break;
}
}
// Skip plaintext variants
if (_NO_HIGHLIGHT_LANGS[langClass]) {
el.classList.add("nohighlight");
continue;
}
// Apply highlighting
// Skip plaintext variants
if (_NO_HIGHLIGHT_LANGS[langClass]) {
el.classList.add("nohighlight");
continue;
}
// Cache key: language + separator + source. ":" isn't part of a
// language identifier so the prefix is unambiguous across keys.
var source = el.textContent;
var cacheKey = langClass + ":" + source;
if (_hljsCache.has(cacheKey)) {
_applyCachedHljs(el, _hljsCache.get(cacheKey));
} else {
hljs.highlightElement(el);
// Add terminal styling class for shell languages
if (_TERMINAL_LANGS[langClass]) {
el.closest("pre").classList.add("code-terminal");
}
_cacheFifoEntry(_hljsCache, cacheKey, el.innerHTML, _HLJS_CACHE_MAX);
}
// Add terminal styling class for shell languages
if (_TERMINAL_LANGS[langClass]) {
var pre = el.closest("pre");
if (pre) pre.classList.add("code-terminal");
}
}
}
function postRenderMarkdown(containerEl) {
postRenderHljs(containerEl);
// Render mermaid diagrams (lazy-loads mermaid.js on first use)
postRenderMermaid(containerEl);
}
@@ -826,6 +911,92 @@ function _getMermaidTheme() {
};
}
// Mermaid label autoquoter.
//
// Mermaid's flowchart parser treats ( ) [ ] { } as shape delimiters
// EVERYWHERE — including inside other labels — unless the label is
// wrapped in "...". LLM-emitted diagrams routinely produce things
// like A["x"] -->|note (with parens)| B or D[label (foo, bar)]
// and Mermaid then rejects them with "Parse error, got PS" (paren-
// start in shape context — the parser entered a nested shape parse
// at the bare `(` and ran out of expected closing tokens).
//
// We can't fix every malformed diagram, but the two patterns above
// are easy to spot syntactically and quote:
//
// 1. Edge labels: |content| → |"content"|
// 2. Plain rectangle node labels: ID[content] → ID["content"]
//
// Shapes whose syntax already nests delimiters — cylinders [(...)],
// subroutines [[...]], trapezoids [/.../] [\...\], circles ((...)),
// double circles (((...))), hexagons {{...}}, diamonds {...} — are
// intentionally left alone. The inner delimiters are part of the
// shape, and our regex would corrupt valid syntax. Authors using
// those shapes must quote the label manually.
function _normalizeMermaidSource(source) {
if (!source) return source;
// Fast path: no shape delimiters anywhere → nothing to quote.
if (
source.indexOf("(") === -1 &&
source.indexOf("[") === -1 &&
source.indexOf("{") === -1
) {
return source;
}
var lines = source.split("\n");
for (var i = 0; i < lines.length; i++) {
var line = lines[i];
// %% directives and comments — never rewrite. The %%{init:...}%%
// form contains braces that would otherwise look like a label.
if (/^\s*%%/.test(line)) continue;
line = _quoteMermaidNodeLabels(line);
line = _quoteMermaidEdgeLabels(line);
lines[i] = line;
}
return lines.join("\n");
}
function _quoteMermaidNodeLabels(line) {
// ID[content] → ID["content"] when content needs quoting.
//
// The first character of content is restricted to NOT be [ ( / \
// so we skip [[subroutine]], [(cylinder)], [/trap/], [\trap\].
// The rest of content is restricted to NOT contain [ ] so the
// regex can't run away past a legitimate ].
return line.replace(
/([A-Za-z_][\w-]*)\[([^[(/\\\n][^[\]\n]*?)\]/g,
function (m, id, content) {
if (_mermaidLabelNeedsQuoting(content)) {
return id + '["' + content + '"]';
}
return m;
},
);
}
function _quoteMermaidEdgeLabels(line) {
// |content| → |"content"| when content needs quoting.
// Edge labels can't contain a literal | (it's the delimiter), so
// [^|\n] is exhaustive.
return line.replace(/\|([^|\n]+)\|/g, function (m, content) {
if (_mermaidLabelNeedsQuoting(content)) {
return '|"' + content + '"|';
}
return m;
});
}
function _mermaidLabelNeedsQuoting(content) {
// Any literal " in content would produce nested unescaped quotes
// when we wrap. Punt to manual fix. This also short-circuits the
// already-correctly-quoted "..." case (which has " at the bounds).
if (content.indexOf('"') !== -1) return false;
// <br/> and <br> are part of Mermaid's allowed HTML in labels and
// don't on their own require quoting.
var stripped = content.replace(/<br\s*\/?>/gi, "");
return /[()[\]{}]/.test(stripped);
}
// Source-keyed SVG cache. Identical mermaid source produces identical
// SVG, so we can swap in cached output synchronously without re-running
// mermaid.render. Crucial for streaming markdown: streamingRender does
@@ -845,16 +1016,24 @@ var _mermaidSvgCache = new Map();
var _mermaidErrorCache = new Map();
var _MERMAID_CACHE_MAX = 64;
function _cacheMermaidEntry(cache, source, value) {
// Only evict the oldest when inserting a new key — overwriting an
// existing source is an in-place update and should not pay the
// eviction cost (which would drop an unrelated cached entry).
if (!cache.has(source) && cache.size >= _MERMAID_CACHE_MAX) {
var firstKey = cache.keys().next().value;
cache.delete(firstKey);
}
cache.set(source, value);
}
// Raw-textContent → normalized memo. _normalizeMermaidSource splits +
// regex-replaces line by line; on a 50-line flowchart that's ~57 µs.
// The SVG cache short-circuits mermaid.render once we have the
// normalized key, but the *normalize step itself* runs on every rAF
// tick (postRenderMermaid always calls it before the SVG-cache
// lookup, since the normalized output IS the lookup key). Memoizing
// raw → normalized avoids repeating the split + regex for diagrams
// whose source hasn't changed between ticks.
//
// Bounded by _MERMAID_CACHE_MAX so its memory footprint stays in the
// same order of magnitude as the SVG cache it feeds, but the two
// queues evict INDEPENDENTLY: this memo keys on raw textContent while
// the SVG cache keys on normalized source, so a single diagram can
// occupy one slot in each with no positional coupling. The memo also
// deliberately survives `_initMermaid` (which clears the SVG / error
// caches on theme change) — normalization output is purely a function
// of input text, independent of mermaid theme / config.
var _mermaidNormalizeCache = new Map();
function _applyMermaidSvg(container, svg, bindFunctions) {
container.innerHTML = svg;
@@ -921,10 +1100,12 @@ function _renderMermaidBlock(container, callback) {
var id = "mermaid-" + ++_mermaidIdCounter;
return mermaid.render(id, source).then(
function (result) {
_cacheMermaidEntry(_mermaidSvgCache, source, {
svg: result.svg,
bindFunctions: result.bindFunctions,
});
_cacheFifoEntry(
_mermaidSvgCache,
source,
{ svg: result.svg, bindFunctions: result.bindFunctions },
_MERMAID_CACHE_MAX,
);
for (var i = 0; i < pending.length; i++) {
var c = pending[i];
if (c.isConnected) {
@@ -936,7 +1117,7 @@ function _renderMermaidBlock(container, callback) {
var orphan = document.getElementById(id);
if (orphan) orphan.remove();
var msg = err && err.message ? err.message : "Diagram error";
_cacheMermaidEntry(_mermaidErrorCache, source, msg);
_cacheFifoEntry(_mermaidErrorCache, source, msg, _MERMAID_CACHE_MAX);
for (var i = 0; i < pending.length; i++) {
var c = pending[i];
if (c.isConnected) _applyMermaidError(c, source, msg);
@@ -962,7 +1143,20 @@ function postRenderMermaid(containerEl) {
for (var i = 0; i < codeEls.length; i++) {
var pre = codeEls[i].closest("pre");
if (!pre) continue;
var source = codeEls[i].textContent;
// Autoquote labels with bare shape-delimiter chars before
// caching / rendering. Identical malformed input maps to identical
// normalized output, so the SVG cache still hits on repeated
// streams of the same diagram. _mermaidNormalizeCache skips the
// split + per-line regex when the raw textContent hasn't changed
// between ticks — only on a fresh source does normalization run.
var raw = codeEls[i].textContent;
var source;
if (_mermaidNormalizeCache.has(raw)) {
source = _mermaidNormalizeCache.get(raw);
} else {
source = _normalizeMermaidSource(raw);
_cacheFifoEntry(_mermaidNormalizeCache, raw, source, _MERMAID_CACHE_MAX);
}
var div = document.createElement("div");
div.setAttribute("data-mermaid-source", source);
// Use cache.has (not truthiness) so a future cached value of
@@ -1022,11 +1216,12 @@ function reRenderAllMermaid() {
// cycle. renderMarkdown tolerates mid-stream partial fences / lists
// (they render as literal text and resolve once the closing tokens
// arrive), and the per-element buffer cache skips identical redundant
// renders (SSE retries / resumes). hljs syntax highlighting stays
// deferred to streamingRenderFinalize, but mermaid runs inline on
// every render so closed diagram fences appear progressively as they
// complete (the source-keyed SVG cache makes re-renders cheap; only
// the first encounter with a given source pays mermaid.render).
// renders (SSE retries / resumes). Both hljs syntax highlighting
// and mermaid run inline on every render so closed code / diagram
// fences appear progressively as they complete; their source-keyed
// caches (_hljsCache, _mermaidSvgCache) make subsequent rAF ticks
// that re-extract the same closed fence hit synchronously without
// re-invoking hljs.highlightElement / mermaid.render.
// renderMarkdown escapes HTML internally (see escapeHtml in
// utils.js); it is the trust boundary for the markup written to el
// below.
@@ -1036,11 +1231,14 @@ function _streamingRenderApply(el, buffer) {
el._lastRenderedBuffer = buffer;
var html = renderMarkdown(buffer);
el.innerHTML = html;
// Progressive mermaid render — see comment above. postRenderMermaid
// is no-op when the element has no language-mermaid code blocks,
// and the source-keyed cache avoids re-invoking mermaid.render
// for blocks we've already rendered. Subsequent rAF ticks that
// Progressive hljs + mermaid render — see comment above. Both are
// no-ops when the element has no matching code blocks, and their
// source-keyed caches avoid re-tokenizing / re-rendering for
// sources we've already processed. Subsequent rAF ticks that
// re-extract the same closed fence hit the cache synchronously.
if (typeof postRenderHljs === "function") {
postRenderHljs(el);
}
if (typeof postRenderMermaid === "function") {
postRenderMermaid(el);
}