Skip to content

adding HDP based coverage test - #19

Merged
ConnorStoneAstro merged 4 commits into
mainfrom
hdp
Aug 6, 2026
Merged

adding HDP based coverage test#19
ConnorStoneAstro merged 4 commits into
mainfrom
hdp

Conversation

@ConnorStoneAstro

Copy link
Copy Markdown
Owner

If you have a posterior density function, that is probably more sensitive than pted. We now provide a consistent implementation for the sake of comparison.

Copilot AI lite review requested due to automatic review settings August 6, 2026 08:40
@ConnorStoneAstro ConnorStoneAstro added the enhancement New feature or request label Aug 6, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The two_tailed_p semantic change conflicts with existing tests/docs and the new HDP additions need small correctness/doc fixes (example code + input handling) before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR adds an HDP-based coverage test (hdp_coverage_test) intended to provide a more sensitive calibration check when posterior density values are available, alongside documentation and test coverage.

Changes:

  • Add hdp_coverage_test utility and export it from the package.
  • Add unit test coverage for hdp_coverage_test.
  • Document the new HDP workflow in the README.
File summaries
File Description
tests/test_utils.py Adds a new test for hdp_coverage_test and updates utils imports.
src/pted/utils.py Reworks two_tailed_p and introduces hdp_coverage_test.
src/pted/__init__.py Exposes hdp_coverage_test at the package level.
README.md Adds usage docs for hdp_coverage_test and updates footnotes.
Review details

Suppressed comments (2)

src/pted/utils.py:319

  • The leftover commented-out block should be removed entirely instead of kept inline (it’s dead code after the two_tailed_p rewrite, and makes the surrounding logic harder to scan).
#     assert df > 2, "Degrees of freedom must be greater than 2 for two-tailed p-value calculation."
#     alpha = chi2_dist.pdf(chi2, df)
#     mode = df - 2

#     if np.isclose(chi2, mode):

src/pted/utils.py:497

  • The docstring describes “higher density than the ground truth”, but the code uses >= (ties counted as higher). Also, input shapes aren’t validated, and importing chi2_dist inside the function is redundant since it’s already imported at module scope.

Consider normalizing inputs with np.asarray, validating shapes, using strict > to match the wording, and handling the Nsim==1 case explicitly to avoid an ill-defined chi² combination step.

    from scipy.stats import chi2 as chi2_dist

    Nsamp, Nsim = posterior_samples.shape
    q = np.sum(posterior_samples >= ground_truth[None], axis=0)
    chi2_hdp = -2 * np.sum(np.log((q + 1) / (Nsamp + 1)))
  • Files reviewed: 4/4 changed files
  • Comments generated: 4
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Comment thread src/pted/utils.py
Comment thread src/pted/utils.py
Comment thread tests/test_utils.py
Comment thread README.md Outdated
@codecov-commenter

codecov-commenter commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.65%. Comparing base (51e568d) to head (fddf599).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #19      +/-   ##
==========================================
- Coverage   97.66%   97.65%   -0.01%     
==========================================
  Files           4        4              
  Lines         342      341       -1     
==========================================
- Hits          334      333       -1     
  Misses          8        8              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ConnorStoneAstro
ConnorStoneAstro merged commit ec4a5c8 into main Aug 6, 2026
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants