Skip to content

Add pytest-cov coverage reporting (informational, not a CI gate) - #266

Merged
yakew7 merged 2 commits into
yakew7:mainfrom
Circout-sudo:patch-3
Aug 16, 2026
Merged

Add pytest-cov coverage reporting (informational, not a CI gate)#266
yakew7 merged 2 commits into
yakew7:mainfrom
Circout-sudo:patch-3

Conversation

@Circout-sudo

Copy link
Copy Markdown
Contributor

Closes #249

  • Add a test extra to pyproject.toml (pytest, pytest-cov)
  • Add a make coverage target; make test output is unchanged
  • CI's profiler job installs the new test extra and runs a new, separate "Coverage report (informational)" step via make coverage, right after the existing make test step (which is untouched)

No --cov-fail-under is set anywhere, so this can never fail a PR on a coverage threshold - it's visibility only, as requested in the issue.

Baseline on this run: 82% overall coverage across faircode/. Weakest spots: figures.py (0%), cli.py (51%), report.py (64%), benchmark.py (72%) - good candidates for a follow-up issue.

README coverage badge (mentioned as optional in #249) is not included here since it needs a live coverage service (Codecov/Coveralls) set up by the maintainer first.

⏸️ Paper freeze active. Do not modify anything under paper/results-frozen/, results/, the faircode/ analysis core, any audit.yaml / dataset CSV, or the reproducibility parameters (random_state, split, iteration counts, metrics) - these are frozen for a paper in peer review (see CLAUDE.md). New audits cannot merge to main yet (timing hold - they will be parked post-paper). Explainers, docs, website, and tooling are unaffected. If you think you found a bug in the analysis code, flag it in an issue - do not silently fix it.

Summary

Type

  • Audit
  • Explainer
  • Bug fix
  • Other

Audit checklist

  • I opened or linked a corresponding issue first
  • The folder is named after the domain, not the dataset
  • unfair.py includes protected attributes and prints the required output format
  • fair.py removes protected attributes and identified proxy variables
  • Both scripts use random_state=42 and an 80/20 train/test split
  • Proxy variables were actually tested, not just guessed
  • unfair.png and fair.png are included as PNG screenshots
  • The dataset is public and accessible without login or payment
  • The dataset file is included, or DATA.md is included if the file is too large
  • README.md includes the new results row and audit section
  • A notebook was added if the audit benefits from one

Before fairness gap:

After fairness gap:

Reduction:

Protected attribute(s):

Proxy variables dropped:


Explainer checklist

  • The file is in explainers/ and uses lowercase hyphenated naming
  • It includes a plain-language definition
  • It uses a real example from this repo or a documented real-world case
  • It includes runnable Python detection or measurement code
  • It acknowledges limitations or trade-offs
  • It links to related explainers or repo projects
  • It includes 2-3 primary sources
  • The Explainers table in README.md was updated

Linked issue

Closes #

Closes yakew7#249

- Add a `test` extra to pyproject.toml (pytest, pytest-cov)
- Add a `make coverage` target; `make test` output is unchanged
- CI's profiler job installs the new `test` extra and runs a new,
  separate "Coverage report (informational)" step via `make coverage`,
  right after the existing `make test` step (which is untouched)

No --cov-fail-under is set anywhere, so this can never fail a PR on a
coverage threshold - it's visibility only, as requested in the issue.

Baseline on this run: 82% overall coverage across faircode/.
Weakest spots: figures.py (0%), cli.py (51%), report.py (64%),
benchmark.py (72%) - good candidates for a follow-up issue.

README coverage badge (mentioned as optional in yakew7#249) is not included
here since it needs a live coverage service (Codecov/Coveralls) set up
by the maintainer first.
@vercel

vercel Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

@Circout-sudo is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown

@yakew7 @ahmdkaml - new PR to review.

@yakew7
yakew7 merged commit 105744b into yakew7:main Aug 16, 2026
15 of 16 checks passed
yakew7 added a commit that referenced this pull request Aug 16, 2026
#263/#264/#265/#266 added make coverage (pytest --cov=faircode) and
wired it into the profiler job as an informational, non-gating step -
but the job's install line was never updated to include pytest-cov,
so the step has failed on every run since it merged. continue-on-error:
true masked it as an overall job success, so nobody noticed - verified
by simulating the exact install line in a clean venv: pytest errors
with "unrecognized arguments: --cov=faircode --cov-report=term-missing"
without the plugin installed.

Add pytest-cov to the profiler job's install line. Verified in a fresh
venv that make coverage now actually reports real numbers instead of
erroring out. Also gitignore .coverage, the local artifact make
coverage produces.
yakew7 added a commit that referenced this pull request Aug 16, 2026
Bump snapshot to cover #262 (Shreyash0712, 5 more explainers),
#263/#264 (ahmdkaml, make coverage + non-gating CI coverage report),
and #265/#266 (new contributor Circout-sudo/Yojeet, pyproject.toml
test extra - same coverage-reporting effort as ahmdkaml's PRs, both
closing #249). Adds Circout-sudo as a new Contributors entry, the
Contributor tooling & docs area row, and the git identity map (git
author "Yojeet" vs GitHub handle).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No test-coverage measurement in CI

3 participants