Implementing tension statistics #333
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #333 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 36 37 +1
Lines 3069 3097 +28
=========================================
+ Hits 3069 3097 +28 ☔ View full report in Codecov by Sentry. |
williamjameshandley
added
enhancement
|
Hi @DilyOng, many thanks for taking charge of incorporating this. Let's get it plumbed into anesthetic first, and then get feedback from others on if anything is missing. At the moment, this code is specialised to a specific naming scheme (which is what the union and intersection functions are doing), and for a wider grid. I think we should re-organise this so that in the first instance it is more similar to @AdamOrmondroyd's suspiciousness package, but retaining the class/cacheing structure of Tasks:
|
|
I think after that it would also be good to implement a function in addition to (or possibly in place of!) the class for producing a |
|
Please remember to remove ( |
…ation and testing it with correlated gaussian likelihoods. Found a problem with the function anesthetic.examples.perfect_ns.correlated_gaussian. The generated likelihood gaussian in the parameters is not normalised and the evidence is not unity. Need to take into account the LogLmax.
…ed_gaussian. Within the correlated_gaussian function, changed logLike function. Changed the function's description to match the fact that evidence is not unity.
…kelihood test case with the tests folder.
…nction tension_stats() for calculating tension statistics. Rewrote the test_tension_stats.py in tests to match the format of other files. It tests mock datasets with guassian likelihood. Both compatiable and incompatiable datasets have passed the test.
…dd a file for datasets pairwise_comparison, but not completed
…the theoretical logR, logS and logI values sit within 3 std of the numerical solution's distribution from anesthetic, instead of testing between minimum and maximum values of the distribution.
|
@AdamOrmondroyd needs to approve the changes in order for this to be merged |
tests/test_tension.py
Outdated
| covAB = inv(inv(covA) + inv(covB)) | ||
| meanAB = covAB@(solve(covA, meanA)+solve(covB, meanB)) | ||
| dmeanAB = np.array(meanA)-np.array(meanB) | ||
| logLmaxAB = -1/2 * dmeanAB@solve(covA+covB, dmeanAB) + logLmaxA + logLmaxB |
There was a problem hiding this comment.
Missing spaces around @
tests/test_tension.py
Outdated
|
|
||
| logS_std = samples_stats.logS.std() | ||
| logS_mean = samples_stats.logS.mean() | ||
| logS_exact = d/2 - 1/2*dmeanAB@solve(covA+covB, dmeanAB) |
tests/test_tension.py
Outdated
| bounds = [[-1, 1], [0, 3], [0, 1]] | ||
|
|
||
| meanA = [0.1, 0.3, 0.5] | ||
| covA = np.array([[.01, 0.009, 0], |
There was a problem hiding this comment.
Missing 0 in 0.01 (there are a few of these, I'm technically on holiday and using my phone so you can find the rest)
… computation to save computing time for high-nsamples runs
…ng stats to tension stats
| samples['logI'] = statsA['D_KL'] + statsB['D_KL'] - statsAB['D_KL'] | ||
| samples.set_label('logI', r'$\ln\mathcal{I}$') |
There was a problem hiding this comment.
@williamjameshandley: The notation with logI matches the one in eq. (9) of Quantifying tensions in cosmological parameters. However, the Shannon information I_S = log(P/pi) already carries a logarithm in its notation, so I find this logI notation confusing. Wouldn't just I be more appropiate?
The notation logR = logS - I also matches better the Occam equation logZ = logL_P - D_KL...
Thoughts?
Description
This is a work in progress pull request aiming to address #325 and as a learning exercise on how to do pull request.
Checklist:
flake8 anesthetic tests)pydocstyle --convention=numpy anesthetic)python -m pytest)