Skip to content

fix: detect wildcard DNS up front and filter false positives - #38

Open
0xParth wants to merge 2 commits into
hadriansecurity:mainfrom
0xParth:feat/wildcard-detection
Open

0xParth wants to merge 2 commits into
hadriansecurity:mainfrom
0xParth:feat/wildcard-detection

Conversation

@0xParth

@0xParth 0xParth commented Sep 7, 2026

Copy link
Copy Markdown

Summary

Follow-up to the discussion on #37. Domains with a wildcard record (*.example.com) resolve every predicted subdomain to the same catch-all IPs, producing huge false-positive counts (2,199 in the original repro) that compound through the recursive inference loop.

Per @klaasmeinke's review suggestion, this moves wildcard detection to the start, before the expensive transformer inference. Rather than exiting with an error when a wildcard is found, it warns and filters — keeping the real subdomains that resolve outside the catch-all set — with an opt-out flag to skip instead. This preserves the tool's usefulness on the many domains fronted by CloudFront/Cloudflare while still killing the noise.

Approach

  1. Detect once, up front. detect_wildcard(apex) probes a few random subdomains. If they all resolve, the apex has a catch-all record and the union of their IPs is the wildcard set. Runs per-apex in run() before inference — the probe is only a handful of DNS lookups.
  2. Warn + filter (default). When a wildcard is detected, resolution keeps only predictions resolving to at least one IP outside the wildcard set; predictions resolving solely to the catch-all IPs are dropped.
  3. --wildcard {filter,skip} flag (default filter). skip drops the wildcard apex entirely. With --multi-apex, this is per-apex — a wildcard on one apex never affects the others.

Why not exit early on detection

The motivating domain still had 21 real subdomains with distinct A records. Exiting would return none of them, and under --multi-apex a run-level exit would also kill the clean apexes. Filtering recovers strictly more true positives than exiting.

Known tradeoff: a genuine subdomain sharing the wildcard's IP (same CDN distribution) is filtered out as a false negative. Filtering still recovers far more than exiting (which recovers zero).

Changes

File Change
subwiz/resolve.py Add detect_wildcard() and _resolve_ips(); get_registered_domains() gains an optional wildcard_ips filter
subwiz/main.py Detect wildcard per apex in run(), warn, thread wildcard_ips through resolution; add wildcard param + validation
subwiz/cli.py Add --wildcard flag
tests/test_resolve.py Cover wildcard detection and wildcard-aware filtering
README.md Regenerate the CLI help block (adds --wildcard; also reflects current argparse output)
subwiz/model.py Black formatting only — one blank line; no logic change (kept the black --check lint green under the current stable Black)

Backward compatibility

  • wildcard defaults to "filter"; get_registered_domains(wildcard_ips=None) is unchanged, so existing callers and the non-wildcard fast path are unaffected.
  • No new dependencies.

Test plan

  • pytest tests/test_resolve.py — existing regression + new wildcard tests pass
  • black --check ./subwiz ./tests clean
  • README help block matches python -m subwiz.cli -h (test_readme_switches)

🤖 Generated with Claude Code

Domains with a wildcard record (*.example.com) resolve every predicted
subdomain to the same catch-all IPs, producing huge false-positive counts
that compound through the recursive inference loop.

This detects wildcards once per apex, up front (before the expensive
transformer inference), by probing random subdomains. When a wildcard is
present, resolution keeps only predictions that resolve to at least one IP
outside the catch-all set; predictions resolving solely to the wildcard IPs
are dropped as noise. A new --wildcard {filter,skip} flag (default: filter)
lets the user skip wildcard apexes entirely instead.

- resolve.py: add detect_wildcard(), _resolve_ips() helper; get_registered_domains()
  gains an optional wildcard_ips filter.
- main.py: detect wildcard per apex in run(), warn, and thread wildcard_ips
  through to resolution; add wildcard param + validation.
- cli.py: add --wildcard flag.
- tests: cover wildcard detection and wildcard-aware filtering.
- README: regenerate CLI help block (adds --wildcard; reflects current argparse).
- model.py: black formatting only (no logic change).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@klaasmeinke klaasmeinke left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Maybe just remove the inline comments? I think they are a bit verbose.

Addresses review feedback on hadriansecurity#38.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@0xParth

0xParth commented Sep 15, 2026

Copy link
Copy Markdown
Author

Thanks! Removed the verbose inline comments in 5cb4538 (kept the docstrings, since those match the rest of the codebase). Ready when you are.

@0xParth

0xParth commented Sep 15, 2026

Copy link
Copy Markdown
Author

Hey @klaasmeinke — this should be good to go now: your review comments are addressed and it's approved. The CI checks on the latest commit are waiting on "Approve and run workflows" (fork-PR gate) — could you kick those off (and merge) when you get a chance? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants