You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A demo video published in the documentation is a claim that the command it records runs clean. Nothing currently checks that claim, so a recording that shows an error or a warning can be rendered and published unnoticed. This happened: the lint demo published an ESLint warning that stayed in the documentation until someone read the video.
The check belongs at recording time, in ahoy update-videos, so a bad recording never becomes an artifact in the first place.
Details
While recording, decode each cast to plain text and refuse to render the SVG when that text reports an error, a warning or a failure.
Decode the cast to plain text, stripping ANSI escape sequences, including sequences split across two recorded chunks.
Match the decoded text for reported errors, warnings and failures.
Match failures explicitly. A command that fails inside a recording still leaves asciinema exiting 0, so the exit code proves nothing.
Exempt fragments that carry a marker word without reporting anything, before matching: paths, URLs, package names such as symfony/error-handler, command line options such as --no-error-on-unmatched-pattern, and zero counts such as 0 errors, warnings: 0 and --max-warnings=0.
Stop the render when anything is found, and report every offending line with its line number so the source is identifiable.
Write the decoded text to .artifacts/videos/<name>.txt for inspection. It is working material, not a published artifact, so it is not committed.
Group repeated identical lines in the report rather than printing each occurrence.
Scope and constraints:
The check runs during recording only. It does not run in CI and does not re-check committed casts.
No override flag. Output that cannot be fixed in this repository is fixed at the command that emits it, not waved through.
Every existing recording passes the check once the sources of its output are fixed.
An earlier implementation was built and validated against all nine recordings in #3047, then dropped when that branch was rebuilt around the ESLint 9 migration. That work can be used as a starting point.
Summary
A demo video published in the documentation is a claim that the command it records runs clean. Nothing currently checks that claim, so a recording that shows an error or a warning can be rendered and published unnoticed. This happened: the
lintdemo published an ESLint warning that stayed in the documentation until someone read the video.The check belongs at recording time, in
ahoy update-videos, so a bad recording never becomes an artifact in the first place.Details
While recording, decode each cast to plain text and refuse to render the SVG when that text reports an error, a warning or a failure.
asciinemaexiting0, so the exit code proves nothing.symfony/error-handler, command line options such as--no-error-on-unmatched-pattern, and zero counts such as0 errors,warnings: 0and--max-warnings=0..artifacts/videos/<name>.txtfor inspection. It is working material, not a published artifact, so it is not committed.Scope and constraints:
An earlier implementation was built and validated against all nine recordings in #3047, then dropped when that branch was rebuilt around the ESLint 9 migration. That work can be used as a starting point.