Skip to content

Fix to p-value calculation for permutation and exact#150

Open
mjacketti wants to merge 2 commits into
mainfrom
JT-pval
Open

Fix to p-value calculation for permutation and exact#150
mjacketti wants to merge 2 commits into
mainfrom
JT-pval

Conversation

@mjacketti

Copy link
Copy Markdown
Collaborator

The discrepancy was caused by a tail-direction error in the exact and permutation p-value calculations. The JT statistic increases as the evidence for an increasing trend gets stronger. Therefore, the increasing alternative should use the upper tail of the null distribution, P(J ≥ observed J), while the decreasing alternative should use the lower tail, P(J ≤ observed J). The code was effectively using the opposite tail for the permutation method, and the exact method was similarly misaligned with the statistic direction. As a result, datasets with a very strong increasing trend produced p-values near 1 instead of near 0. The approximate method used the expected direction, which is why it produced the statistically sensible result.

@mjacketti mjacketti marked this pull request as ready for review May 7, 2026 17:43
@mjacketti mjacketti requested a review from adavis10 May 7, 2026 17:43

@adavis10 adavis10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good. I think merging into main is fine b/c we're not changing any of the documentation so the monthly scans won't pull anything not ready for public release into the scheduled updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants