Run the engine's offline test suite on Windows#25
Closed
xroche wants to merge 8 commits into
Closed
Conversation
The engine ships ~90 tests and not one has ever run on Windows: they only run under `make check` on Linux and macOS. Start with the 50 offline ones -- no network, no server, just httrack.exe driven from PATH. They are the engine's tests, but ours is the binary that ships, and this code has never executed on this platform. Pin core.autocrlf: the scripts are #!/bin/bash and the engine has no .gitattributes, so a converting checkout would rewrite them and bash would die on $'\r'. Assert tests actually ran, rather than that none failed. Every gate in these scripts exits 77 -- no python3, no HTTPS, no codec -- so a suite that quietly degrades to "everything skipped" would otherwise report success having tested nothing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A 25-line tail cannot tell a real Windows bug from a POSIX assumption in the test, and 16 of the 50 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
httrack.exe is a native binary, so MSYS rewrites any argument shaped like a POSIX path -- and a URL path is shaped exactly like one. '-#test=mime /a/b.html' reached the engine as 'C:/Program Files/Git/a/b.html', and four tests failed silently on it. Switch the rewriting off and give the tests a temp dir that is already a Windows path, so nothing needs translating in either direction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A shell heredoc expanded the backticks in the previous commit and substituted the example away, leaving "# into ." Same code, readable reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
They pipe stderr to /dev/null and print one word, so a Windows failure reads "FAIL" and nothing more. The engine side needs the assertion, not the verdict. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both sides added a step at the same point and neither replaces the other: the version-metadata check from master, then the engine test suite. Tests run on the staged binaries, before signing rewrites them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Several of these assert with `test "$(httrack ...)" == "..." || exit 1`, which prints nothing at all when it fails. Re-run the test under a trace and keep it, so the log holds the call that failed and what it actually printed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
It prints nothing and the test only sees an empty string, which is what a process that died silently looks like. The self-test drives the matcher deep on purpose and Windows gives a thread 1MB of stack where Linux gives 8MB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Owner
Author
|
Closing: the engine adopted this, which is where it belongs.
What the exercise turned up, all now fixed upstream:
Plus the two test-portability issues ( The GUI has the same ANSI/UTF-8 mismatch by a different door — it builds its own argv and calls Tier 2 (the local Python HTTPS crawl suite, ~40 tests) is still unrun on Windows, and now belongs in the engine repo too. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The engine ships ~90 tests and not one has ever run on Windows — they only run under
make checkon Linux and macOS. This is Tier 1: the 50 offline ones (no network, no server), driven fromPATHagainst thehttrack.exewe now ship. They are the engine's tests, but ours is the binary that reaches users, and this code has never been executed on this platform.Two things worth calling out.
core.autocrlfis pinned off, because the scripts are#!/bin/bashand the engine has no.gitattributes— a converting checkout rewrites them and bash dies on$'\r'. And the step asserts tests actually ran, not merely that none failed: every gate in these scripts exits 77 (no python3, no HTTPS, no codec), so a suite that quietly degrades to "everything skipped" would otherwise report success having tested nothing.Expect this to go red first. Any failure here is either a real Windows bug in code that has never run on Windows, or a POSIX assumption in a test — both are worth knowing. Tier 2 (the local Python HTTPS server crawls, ~40 more) follows once this is stable.