Skip to content

Fix nested code fence parsing in lists #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nathan-contino wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix nested code fence parsing in lists #103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 14 additions & 5 deletions src/checks/observability/markdown-content-parity.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -332,12 +332,21 @@ function extractMarkdownText(markdown: string): string {
// on a run of >=N. Capture the opener so the close-side backreference
// matches; otherwise nested example fences (4-backtick outer, 3-backtick
// inner) get mis-paired and inner markers leak out as text.
//
// Per CommonMark §4.5, a fence may be indented 0–3 spaces (list items
// produced by Turndown indent fences by 4 spaces, which is still a valid
// CommonMark indented code fence inside a list continuation). The leading
// spaces are stripped from the opener match so the backreference on the
// closer still works.
const codeBlocks: string[] = [];
text = text.replace(/^(`{3,})[^`\n]*\n([\s\S]*?)^\1`*\s*$/gm, (_match, _opener, content) => {
const idx = codeBlocks.length;
codeBlocks.push(content);
return `\x00BLOCK${idx}\x00`;
});
text = text.replace(
/^ {0,3}(`{3,})[^`\n]*\n([\s\S]*?)^ {0,3}\1`*\s*$/gm,
(_match, _opener, content) => {
const idx = codeBlocks.length;
codeBlocks.push(content);
return `\x00BLOCK${idx}\x00`;
},
);

// Step 2: Protect inline code spans from subsequent stripping.
// Replace `...` with placeholders so link/emphasis regexes don't
Expand Down
82 changes: 82 additions & 0 deletions test/unit/checks/markdown-content-parity.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2658,4 +2658,86 @@ This test guards against regressions that special-case any single backtick run l
const pageResults = result.details?.pageResults as Array<{ missingSegments: number }>;
expect(pageResults[0].missingSegments).toBe(0);
});

it('protects fenced code blocks indented inside list items', async () => {
// Turndown indents fenced code blocks by 4 spaces when they appear inside
// numbered list items. Without this fix, the code fence regex (which
// required backticks at column 0) failed to match, leaving the YAML list
// lines unprotected. The list-marker stripper then removed the leading
// "- " from lines like "- db_data:/var/lib/postgresql/data", so the HTML
// segment "- db_data:/var/lib/postgresql/data" had no match in the
// stripped markdown text.
const html = `<html><body><main>
<h1>Install with Docker</h1>
<p>Use the steps below to install FusionAuth using Docker Compose.</p>
<ol>
<li>
<p>Download the configuration files from the repository.</p>
<p>Full docker-compose.yml example:</p>
<pre><code class="language-yaml">services:
db:
image: postgres:16.0-bookworm
volumes:
<span class="token punctuation">-</span> db_data<span class="token punctuation">:</span>/var/lib/postgresql/data
fusionauth:
image: fusionauth/fusionauth-app:latest
volumes:
<span class="token punctuation">-</span> fusionauth_config<span class="token punctuation">:</span>/usr/local/fusionauth/config
<span class="token punctuation">-</span> \${FUSIONAUTH_LOCAL_KICKSTART_DIRECTORY}<span class="token punctuation">:</span>/usr/local/fusionauth/kickstart
volumes:
db_data<span class="token punctuation">:</span>
fusionauth_config<span class="token punctuation">:</span>
</code></pre>
</li>
<li>
<p>Start the services with docker compose up.</p>
</li>
</ol>
</main></body></html>`;

const markdown = `# Install with Docker

Use the steps below to install FusionAuth using Docker Compose.

1. Download the configuration files from the repository.

Full docker-compose.yml example:

\`\`\`yaml
services:
db:
image: postgres:16.0-bookworm
volumes:
- db_data:/var/lib/postgresql/data
fusionauth:
image: fusionauth/fusionauth-app:latest
volumes:
- fusionauth_config:/usr/local/fusionauth/config
- \${FUSIONAUTH_LOCAL_KICKSTART_DIRECTORY}:/usr/local/fusionauth/kickstart
volumes:
db_data:
fusionauth_config:
\`\`\`

2. Start the services with docker compose up.`;

const url = 'http://mcp-indented-fence.local/docs/docker';

server.use(
http.get(
url,
() =>
new HttpResponse(html, {
status: 200,
headers: { 'Content-Type': 'text/html' },
}),
),
);

const ctx = makeCtx([{ url, markdown, htmlBody: html }], 'mcp-indented-fence.local');
const result = await check.run(ctx);
expect(result.status).toBe('pass');
const pageResults = result.details?.pageResults as Array<{ missingSegments: number }>;
expect(pageResults[0].missingSegments).toBe(0);
});
});