Internal parameter covariance functionality.

[HTJense]

Add functionality for covariance between internal parameters.

Looking at the importance of covariance between calibration parameters for the LAT, this branch adds the ability to load in a Gaussian mean + covariance for internal parameters.

I've added an example for the LAT sims, it currently works by having the option

parameter_covariance:
  mean: calib_mean.txt
  cov: calib_cov.txt

in the yaml file.

The covariance file should have a structure like:

# A B C
1.0 0.0 0.0
0.0 1.0 0.0
0.0 0.0 1.0

where the first line specifies the parameters used, and the numbers after specify the full covariance matrix (in this example, this is an uncorrelated unit covariance).

The means can be provided as either a file in a similar format, as an array of numbers, or as a single number (which will be broadcast into an array of numbers).

[codecov-commenter]

Codecov Report

❌ Patch coverage is 22.22222% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.12%. Comparing base (3fdafbf) to head (88d5334).

Files with missing lines	Patch %	Lines
mflike/mflike.py	22.22%	28 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3fdafbf) and HEAD (88d5334). Click for more details.

HEAD has 3 uploads less than BASE

Flag	BASE (3fdafbf)	HEAD (88d5334)
	4	1

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #144       +/-   ##
===========================================
- Coverage   87.24%   72.12%   -15.13%     
===========================================
  Files           3        3               
  Lines         541      574       +33     
  Branches      106        0      -106     
===========================================
- Hits          472      414       -58     
- Misses         59      160      +101     
+ Partials       10        0       -10     

Files with missing lines	Coverage Δ
mflike/mflike.py	75.68% <22.22%> (-8.10%)	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sgiardie]

This looks great. One thing to add is a comment in _MFLike.yaml to explain the formatting of the covmat file and also explaining this feature in the documentation

[HTJense]

Will do - I'm now checking whether it works as intended on full test runs.

[xgarrido]

Also adding unit tests for this new functionality will be good (you can either add a basic text file within tests directory or write on disk a covmat inside the unit test)

[HTJense]

I have added a docstring and unit test. The latter is simply checking whether the functionality works (there's an example minimally covariant covmat included in this file, that would be equivalent to setting 1% uncertainties on the calibration parameters with some small correlation between the different channels). Since the simulations all have perfect calibration, the unit test simply checks that it can load the file and that keeping cal = 1 for all channels doesn't affect the chi square (it shouldn't).

[sgiardie]

Question: If we use this parameter covariance block, then we should not use the standard Cobaya priors on the calibrations, right?
Can you check that the two things are equivalent in the case of a diagonal covmat between the calibrations?

[HTJense]

Okay, I added a unit test that imposes three independent 1% calibration priors using either this covmat method or standard Cobaya priors, and then compares the difference in log-posterior when these calibrations are identically varied within this distribution.

The test itself is a bit clunky, as the standard cobaya methods end up being a contribution to the prior that cobaya computes, whereas with the covmat method they end up being a likelihood contribution (and the normalization factor gets cancelled out via the logp_const value that we include in MFLike). I ended up making the test compare the difference between the two chi squares and confirm that it equals the normalization factor you would expect.

This gives us two tests for this code now (the first checks whether a file can be loaded, the second tests whether the resulting log-likelihood differs from cobaya's log-prior only up to a normalization factor).