WIP: ast-fuzz assertion detector, mutation kinds, and an ast_verify check for base-class nodes - #3862
Open
aleksisch wants to merge 4 commits into
Open
WIP: ast-fuzz assertion detector, mutation kinds, and an ast_verify check for base-class nodes#3862aleksisch wants to merge 4 commits into
aleksisch wants to merge 4 commits into
Conversation
The jit driver compiles a .das file through the LLVM backend without running it - prewarming the dll cache, building a standalone -exe, or emitting a native .o for static linking. All three are things an SDK user does, so the tool moves out of utils/internal/, where nothing installs. Making a utils/ directory external is a contract utils/REVIEW.das machine-checks: an install rule in CMakeLists.txt, an rst page plus its toctree row, and a layout row in install/CLAUDE.md. A row in ci/smoke_test_bundle.sh compiles the installed entry point, so a missing peer file in the bundle fails the release lane instead of the user's first run. The two build-side references (the DAS_LLVM_AOT_LIB batches in CMakeLists.txt, skills/internal/aot_testing.md) follow the path.
aleksisch
force-pushed
the
aleksisch/more-fuzzer
branch
from
August 28, 2026 07:35
879d020 to
e7365a8
Compare
Typed mode builds each expression bottom-up from a requested type, so the program is well-typed by construction and inference runs to the end instead of stopping at the first error. Run mode gives the victim a main that calls every synthesized function, so the same program can be executed twice and its output compared. probe.das is the source-level counterpart to the AST generator: it writes ordinary source text, so a crash it finds has a repro anyone can paste. It carries the mismatch, cycle, garbage and corpus-mutation matrices, reports a probe that compiles but takes too long - judged against its own file's baseline, not a global one - and stores a corpus mutant as a recipe rather than a copy of the file. The mismatch matrix found the const-initializer folding crash and is gated now that the crash is fixed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The generator builds trees from nothing, which is why its findings are shapes only a macro can build. Mutation starts from a real parsed program and applies edits a person could have typed - a different operator, literal, argument order, name, declared type, signature or nesting - so inference sees an ordinary program and a crash is a compiler bug. Every mutation stays source-expressible for that reason. --mutate <file> reuses the whole existing driver: the seed range sweeps the candidate node index, and subprocess isolation, --threads and the crash/verifier/timeout classification are unchanged. --mutate-dir sweeps a corpus and calibrates each file; --mut-stride samples it; --mut-count applies several edits at once and skips calibration; --mut-depth bounds the nesting kinds; --synth-skip now applies to every generator. An assertion banner counts as a crash, victim and log names are per-run, and a mutation timeout is timed against the wrapper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ExprConst and ExprMakeLocal are bases a concrete node derives from. Neither overrides visit, so a walk that reaches one asserts inside Expression::visit on an assertions-enabled build, with no diagnostic - and the walk it kills includes this verifier's own. The scan therefore reads the module's gc list instead of descending, and returns before the visitor starts. All three entry points check: verify_module, verify_module_after_infer, and verify_function, which has no module of its own and asks the program being compiled for one. The pair is what a macro can build and a visitor cannot walk, measured rather than read off the header: of the 43 classes without their own visit, 39 inherit one and 2 cannot be constructed from daslang at all. Of 31 generator repros that asserted, 26 now report a located error. The rest reach the compiler's own Program::visit before any verifier entry runs. The make-pr ast-verify gate skips the ast-fuzz fixtures, which are deliberately malformed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
aleksisch
force-pushed
the
aleksisch/more-fuzzer
branch
from
August 28, 2026 07:36
e7365a8 to
c6cb39c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The AST fuzzer could not see the bugs it was built to find. Every sweep ran against a Release binary, where
DAS_NO_ASSERTIONScompiles out all 318 internal assertions, so a run was only detectable when it segfaulted. This branch adds an assertions-enabled path, five mutation kinds, and a verifier check for the one defect class the work turned up.The verifier now reports a base class used as a node.
ExprConstandExprMakeLocalare bases that a concrete node derives from, and neither overridesvisit, so a walk reaching one asserts insideExpression::visitwith no diagnostic. The walk it kills includes the verifier's own, so the scan reads the module's gc list instead of descending. This matters outside the fuzzer: a macro that builds anExprMakeLocalcrashes a Debug build today and gets a located error instead.The fuzzer gained mutation kinds for function signatures and for arbitrarily deep type or statement nesting, and a
--mut-depthknob. Four defects in the fuzzer's own instrumentation were fixed, each of which had been producing numbers that measured nothing: two concurrent sweeps shared one victim file,--synth-skipwas inert for the main generation path, mutations aimed at a required module were never re-checked, and timeouts were judged against a baseline that excluded the macro module's own load cost.No compiler bug was found. Roughly 11,000 source-expressible mutants across Release and Debug produced zero crashes and zero assertion failures, with the detector independently proven able to fire.
Ledger
Validation
Full preflight and the AOT sweep were not run - this is a WIP branch and CI is the first full check. Run locally: rebase onto origin/master, lint and format over all changed
.das, the ast-fuzz suite (19/19), and the make-pr sync and ast-verify gates. The Debug binary used for the assertion work was built in a separate worktree, since CMake pins output to<src>/bin.The verifier check was negative-controlled: 60 real test files compiled under
--ast-verifyproduced 0 false positives, and 26 of 31 generator repros that previously asserted now report a located error.Claims - stated, not tested
The remaining 5 repros reach the compiler's own
Program::visitbefore any verifier entry runs. No daslib-side check can precede that walk; closing it would need a compiler-side pre-pass, which is not attempted here. A break would look like a Debug assert inExpression::visitwith noAST verifyline before it.The ast-verify gate skipping
utils/internal/ast-fuzz/selftest/is a pre-existing gap, not one this branch introduced:cycle.dasandalias.dasemit the same reports under the gate's own command.