Fix one-tailed Wilcoxon p-values in compute_pvals_wilcoxon

[azrabano23]

Fixes #1176

What was wrong

compute_pvals_wilcoxon built the one-tailed p-value from SciPy's two-sided signed-rank p-value: halve it, then put it on the "pipe1 > pipe2" side if the mean paired difference is positive, else on the other side (1 - p/2). The signed-rank test does not test the mean; its direction comes from the rank sums W+/W-, which disagree with the sign of the mean whenever a few large differences point one way and many small ones the other.

Example, 7 subjects, pipeline 1 wins on 6 by 0.02 and loses on one by 0.20 (SciPy 1.15.3):

	current develop	scipy.stats.wilcoxon(x, y, alternative=...)
pvals[pipeline_1, pipeline_2]	0.8203	"greater": 0.1797
pvals[pipeline_2, pipeline_1]	0.1797	"less": 0.9453

So the matrix said pipeline 2 was better (p = 0.18) when the signed-rank test says the opposite (p = 0.18 for pipeline 1). The values feed compute_dataset_statistics (every dataset with >= perm_cutoff subjects), find_significant_differences and the summary / meta-analysis plots. When mean and rank sums agree, the halved value still differs from the exact one-sided p by the point mass at the observed statistic (0.82 vs 0.95 above).

The fix

Ask SciPy for the one-sided test directly:

p = stats.wilcoxon(df.loc[:, pipe1], df.loc[:, pipe2], alternative="greater")[1]

alternative exists since SciPy 1.5; moabb requires scipy>=1.9.3. The all-zero-differences guard (0.5) and the (0, 1) clipping from #1134 are unchanged. In the common case where mean and rank sums agree and the exact method is used, results change only by the point-mass term; with the normal approximation used for larger n, the two agree whenever the rank sums and the mean point the same way.

Verification (CPU only, Python 3.12, macOS arm64)

python -m pytest moabb/tests/test_analysis.py -k "TestStats or Corrected"   # 36 passed
# the new test test_wilcoxon_tail_follows_rank_sums_not_mean fails on develop:
#   assert np.isclose(pvals[0, 1], expected_greater)  ->  0.8203 vs 0.1797
ruff check / ruff format --check on the two changed files                  # clean

Also added a "Bugs" entry to docs/source/whats_new.rst.

🤖 Generated with Claude Code

[bruAristimunha]

@qbarthelemy, i was wondering, can you please review once you have time please 🙏🏽

[Copilot]

🟢 Approval recommended

The change directly aligns implementation with SciPy’s one-sided Wilcoxon API and is backed by a focused regression test reproducing the reported failure mode.

Pull request overview

This PR fixes the computation of one-tailed Wilcoxon signed-rank p-values in compute_pvals_wilcoxon by using SciPy’s native one-sided test (alternative="greater") instead of deriving a one-sided value from the two-sided p-value and the mean paired difference (which can disagree with the signed-rank statistic’s direction).

Changes:

• Update compute_pvals_wilcoxon to call scipy.stats.wilcoxon(..., alternative="greater") directly for the “pipe1 > pipe2” direction.
• Add a regression test covering the case where mean difference and signed-rank direction disagree (issue #1176 reproducer).
• Add a changelog entry documenting the bug fix.

File summaries

File	Description
moabb/tests/test_analysis.py	Adds a regression test validating that one-tailed p-values follow the signed-rank statistic direction (via SciPy alternatives), not the mean difference.
moabb/analysis/meta_analysis.py	Fixes one-tailed Wilcoxon p-value computation by delegating tail selection to SciPy’s alternative="greater" logic.
docs/source/whats_new.rst	Documents the Wilcoxon one-tailed p-value fix in the Bugs section.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[qbarthelemy]

The original code tests whether the two pipelines differ, specifically checking if "pipe1 is greater than pipe2" or "pipe2 is greater than pipe1", using a two-sided test;
it then divides the p-value by 2 and retains only the relevant side (which is a surprising approach).

The proposed correction tests only the hypothesis that "pipe1 is greater than pipe2" at out[i, j], given that "pipe2 is greater than pipe1" will be tested on the other side of the matrix, at out[j, i]. This alters the current behavior but is cleaner.

Ultimately, we need to consider what exactly we want to test with this function.

In any case, the documentation should advise the user not to overlook corrections for multiple testing.

[qbarthelemy]

Suggested change

	order: list of length (n_pipelines)
	Names corresponding to df columns

[qbarthelemy]

Suggested change

	pvalues

[qbarthelemy]

Suggested change

	Returns a square matrix of p-values computed via the Wilcoxon rank-sum test,

[azrabano23]

Thanks for the careful read, @qbarthelemy. Applied all three wording suggestions in e8ebf95.

On what the function tests: I read the original intent as directional, because the caller already treats it that way. compute_dataset_statistics reads [i, j] and [j, i] as two different hypotheses, and find_significant_differences keeps only the direction it reports, so the matrix has always been consumed as "pipe_i > pipe_j at [i, j]". The old code approximated that by halving the two-sided p and picking the side from the mean difference, which is only equal to the one-sided signed-rank p-value when the mean and the rank sums agree. The change keeps the meaning the callers assume and computes it exactly. If you would rather the function be symmetric (a two-sided test at both [i, j] and [j, i]), that is a one-line change, but then the downstream direction filter would need its own one-sided test, so I left it directional.

The docstring now states that convention explicitly ([i, j] tests order[i] > order[j], reverse direction at [j, i]) and advises that the k * (k - 1) p-values are uncorrected, with Bonferroni/Holm as examples. pytest moabb/tests/test_analysis.py -k "TestStats or Corrected": 36 passed; ruff clean.