Conversation
Domains with a wildcard record (*.example.com) resolve every predicted
subdomain to the same catch-all IPs, producing huge false-positive counts
that compound through the recursive inference loop.
This detects wildcards once per apex, up front (before the expensive
transformer inference), by probing random subdomains. When a wildcard is
present, resolution keeps only predictions that resolve to at least one IP
outside the catch-all set; predictions resolving solely to the wildcard IPs
are dropped as noise. A new --wildcard {filter,skip} flag (default: filter)
lets the user skip wildcard apexes entirely instead.
- resolve.py: add detect_wildcard(), _resolve_ips() helper; get_registered_domains()
gains an optional wildcard_ips filter.
- main.py: detect wildcard per apex in run(), warn, and thread wildcard_ips
through to resolution; add wildcard param + validation.
- cli.py: add --wildcard flag.
- tests: cover wildcard detection and wildcard-aware filtering.
- README: regenerate CLI help block (adds --wildcard; reflects current argparse).
- model.py: black formatting only (no logic change).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
4 tasks
klaasmeinke
approved these changes
Sep 10, 2026
klaasmeinke
left a comment
Collaborator
There was a problem hiding this comment.
Looks good to me! Maybe just remove the inline comments? I think they are a bit verbose.
Addresses review feedback on hadriansecurity#38. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Author
|
Thanks! Removed the verbose inline comments in 5cb4538 (kept the docstrings, since those match the rest of the codebase). Ready when you are. |
Author
|
Hey @klaasmeinke — this should be good to go now: your review comments are addressed and it's approved. The CI checks on the latest commit are waiting on "Approve and run workflows" (fork-PR gate) — could you kick those off (and merge) when you get a chance? Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to the discussion on #37. Domains with a wildcard record (
*.example.com) resolve every predicted subdomain to the same catch-all IPs, producing huge false-positive counts (2,199 in the original repro) that compound through the recursive inference loop.Per @klaasmeinke's review suggestion, this moves wildcard detection to the start, before the expensive transformer inference. Rather than exiting with an error when a wildcard is found, it warns and filters — keeping the real subdomains that resolve outside the catch-all set — with an opt-out flag to skip instead. This preserves the tool's usefulness on the many domains fronted by CloudFront/Cloudflare while still killing the noise.
Approach
detect_wildcard(apex)probes a few random subdomains. If they all resolve, the apex has a catch-all record and the union of their IPs is the wildcard set. Runs per-apex inrun()before inference — the probe is only a handful of DNS lookups.--wildcard {filter,skip}flag (defaultfilter).skipdrops the wildcard apex entirely. With--multi-apex, this is per-apex — a wildcard on one apex never affects the others.Why not exit early on detection
The motivating domain still had 21 real subdomains with distinct A records. Exiting would return none of them, and under
--multi-apexa run-level exit would also kill the clean apexes. Filtering recovers strictly more true positives than exiting.Known tradeoff: a genuine subdomain sharing the wildcard's IP (same CDN distribution) is filtered out as a false negative. Filtering still recovers far more than exiting (which recovers zero).
Changes
subwiz/resolve.pydetect_wildcard()and_resolve_ips();get_registered_domains()gains an optionalwildcard_ipsfiltersubwiz/main.pyrun(), warn, threadwildcard_ipsthrough resolution; addwildcardparam + validationsubwiz/cli.py--wildcardflagtests/test_resolve.pyREADME.md--wildcard; also reflects current argparse output)subwiz/model.pyblack --checklint green under the current stable Black)Backward compatibility
wildcarddefaults to"filter";get_registered_domains(wildcard_ips=None)is unchanged, so existing callers and the non-wildcard fast path are unaffected.Test plan
pytest tests/test_resolve.py— existing regression + new wildcard tests passblack --check ./subwiz ./testscleanpython -m subwiz.cli -h(test_readme_switches)🤖 Generated with Claude Code