Add extensible copula hypothesis testing framework

[Santymax98]

Hi Oskar,

After some time working on this, I think this is probably the last "big" PR I send for a while hahaha.

I had worked on this some time ago because of some academic needs, and I ended up adapting it to the API style we have been using in Copulas.jl.

The main idea is to introduce a generic hypothesis-testing framework for copulas, following a structure very similar to the fitting API.

Roughly, the design is:

Hypothesis × Statistic × Calibration
                  ↓
              CopulaTest

So instead of having every test implemented as an isolated object, the hypothesis declares which statistics and calibration methods are available, and the implementation is selected through Val dispatch.

It is intentionally similar to:

_available_fitting_methods(...)
_fit(..., Val(method))

with the testing side using:

_available_statistics(...)
_available_calibrations(...)
_teststatistic(..., Val(statistic))
_calibrate(..., Val(calibration), ...)

The first available statistic/calibration is used as the default, as we already do for fitting methods.

The PR currently includes tests for:

• independence,
• exchangeability,
• radial symmetry,
• extreme-value dependence,
• goodness of fit for a specified copula,
• goodness of fit for a fitted copula model.

There are a few reusable calibration engines as well: simulation, randomization, multiplier bootstrap, and parametric bootstrap.

For composite GOF, the model is refitted inside every bootstrap sample.

The main reason I structured it this way is extensibility: adding another hypothesis or another statistic should mostly require implementing the mathematical part and declaring its capabilities, without changing the generic CopulaTest machinery.

I also added a manual section with the mathematical definitions of the tests and references, plus a section in the developer guide explaining how to extend the framework.

I tried to keep this PR limited to the testing API itself. Automatic copula family selection is something I worked on separately, so I will keep that in another independent PR rather than mixing both ideas here.

[codecov]

Codecov Report

❌ Patch coverage is 95.24887% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.58%. Comparing base (ad60873) to head (f6f694a).

Files with missing lines	Patch %	Lines
src/CopulaTest.jl	95.12%	19 Missing ⚠️
src/show.jl	96.15%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #435      +/-   ##
==========================================
+ Coverage   86.07%   86.58%   +0.50%     
==========================================
  Files          91       92       +1     
  Lines        7764     8198     +434     
==========================================
+ Hits         6683     7098     +415     
- Misses       1081     1100      +19     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lrnv]

This is a huge piece of work, once again ;)

let me try to finish #434 and then i'll take a look at this. Or maybe the other way around so that i formalize this too in the tests ? not sure...

I will definitely merge it, its a great idea but i need a bit of time :)

[lrnv]

I like the overall Hypothesis × Statistic × Calibration architecture: it fits well with the dispatch-oriented design already used elsewhere in Copulas.jl, avoids a large central dispatcher, and gives us a clean common result type.

A few changes required before merging. The most important ones are about composite GOF refitting and the treatment of ties.

1. Composite GOF refitting is no longer generic enough on current main

The current implementation refits a composite GOF model through:

function _gof_refit(M::CopulaModel, U::AbstractMatrix)
    return Distributions.fit(
        CopulaModel,
        typeof(_copula_of(M)),
        U;
        method=M.method,
        quick_fit=false,
        derived_measures=false,
        vcov=false,
    )
end

This assumes that typeof(C) contains all the information needed to reconstruct the fitted model.

That is no longer true for all copulas on current main.

For example, NestedArchimedeanCopula is now fitted from a template instance because the tree structure and the families of the inner nodes are runtime data, not information encoded in the type. Likewise, SurvivalCopula now stores the flip pattern at runtime rather than in the concrete type.

As a result, composite GOF can either fail to refit such models or silently refit a different model than the one that was originally estimated.

I think the refit itself should be dispatched from the fitted copula instance rather than unconditionally from its type, e.g. along the lines of:

_gof_refit(M::CopulaModel, U) =
    _gof_refit(_copula_of(M), M, U)

function _gof_refit(C::Copula, M, U)
    fit(
        CopulaModel,
        typeof(C),
        U;
        method=M.method,
        derived_measures=false,
        vcov=false,
    )
end

function _gof_refit(C::NestedArchimedeanCopula, M, U)
    fit(
        CopulaModel,
        C,
        U;
        method=M.method,
        derived_measures=false,
        vcov=false,
    )
end

and similarly preserve runtime configuration such as the SurvivalCopula flip mask.

At minimum, I would add composite GOF tests for a nested Archimedean model and a partially flipped survival copula.

More generally, if fitting methods gain additional runtime configuration in the future, GOF should reproduce the original fitting specification rather than only preserve M.method.

2. Ties are currently accepted silently

_test_pseudos accepts any finite matrix and then calls pseudos(U) unless pseudo_values=true.

However, pseudos currently uses ordinal ranks. Therefore tied observations are artificially assigned different ranks.

This matters for these tests, in particular for empirical-copula multiplier procedures. The asymptotic procedures implemented here are generally derived for continuous margins / no ties, while dedicated tie-aware bootstrap procedures are needed in the discrete or tied case.

I would not try to implement all tie-aware variants in this PR. For a first implementation, I would prefer a conservative behavior:

• detect ties in the relevant input margins;
• throw a clear ArgumentError for procedures that do not currently support them;
• document that continuous margins / absence of ties are required for now.

I think that is substantially safer than returning a plausible p-value without the theoretical guarantees advertised by the documentation.

3. Please verify the normalization of each published statistic

For GOF, the current statistic is implemented as:

s += abs2(cdf(Cn, u) - cdf(C, u))
return s / size(U, 2)

For the usual empirical-copula Cramér--von Mises statistic written in terms of
√n (Cₙ - Cθ), integrating with respect to the empirical copula gives a sum of squared discrepancies rather than their mean.

The same question appears in the extreme-value statistic.

This does not currently change the bootstrap p-value if both the observed and bootstrap statistics are multiplied by the same constant, but it does change the value returned by teststatistic(test). Since the public statistic is explicitly named :Sn and the documentation ties it to published definitions, I think the reported value should follow the conventional normalization.

I would check each statistic independently rather than applying a global change: for example, the normalization used by the radial-symmetry statistic may legitimately differ.

4. Keep extension hooks private unless we truly want to support them as public API

I like the extensibility of the implementation, but I would prefer not to make all of the internal hooks part of the public SemVer contract.

The developer guide currently presents functions such as

_available_statistics
_available_calibrations
_teststatistic
_calibrate
_simulation_sample
_randomization_sample
_multiplier_representation
_bootstrap_copula
_bootstrap_hypothesis

as an extension contract for contributors and downstream code.

These are all implementation-oriented _... functions, and I would rather keep as many of them private as possible. Making them public now would significantly enlarge the API surface and would constrain future changes to the testing machinery.

My preference would be:

• keep the generic public user-facing API small:
  • CopulaHypothesis
  • CopulaTest
  • the concrete public test constructors
  • pvalue
  • teststatistic
• keep most routing/calibration helpers internal;
• describe the developer-guide extension mechanism explicitly as an internal contributor API that may evolve;
• only promote an internal hook to public later if we find a real downstream extension use case that requires a stability guarantee.

In other words, I would preserve the extensible internal architecture without automatically turning every dispatch seam into a supported external interface.

This would also be consistent with the recent public/internal API split in Copulas.jl.

5. The multiplier implementation can become very memory-heavy

The multiplier representation currently materializes full n × n matrices, one for every permutation or every EV power.

For example, the default EV test stores three dense matrices simultaneously. At n = 5000, one Float64 matrix is already about 200 MB.

The more concerning case is:

permutations = :all

where the number of permutations grows factorially and each one currently receives its own n × n matrix.

I do not think this needs a full algorithmic rewrite before the first merge, because the default :G2 path is much more reasonable. But I would at least add a cost guard for :all.

Longer term, it would be preferable to stream/process one representation at a time rather than retain every matrix simultaneously.

6. Rebase the tests into the current test-suite structure

The PR was written against the previous test layout. Current main now organizes tests into API contracts, correctness tests, operations, etc.

I would integrate the new tests into something like:

test/operations/hypothesis_testing.jl

rather than restoring another top-level monolithic test file.

The tests I would especially like to see after the rebase are:

• ties rejected/documented where unsupported;
• composite GOF for NestedArchimedeanCopula;
• composite GOF for a partially flipped SurvivalCopula;
• exact statistic normalization checks on small deterministic samples;
• RNG reproducibility for each stochastic calibration engine.

The existing CI is green on the PR branch, which is a good sign, but it predates the recent API/test-suite refactors on main.

7. Small cleanup items

A few smaller things are worth fixing during the rebase:

• show.jl contains unrelated CRLF/LF churn;
• references.bib appears to have large line-ending churn for a small number of added references;
• the new documentation still contains a !!! info block, whereas the current VitePress docs now preserve custom/native containers through ::: syntax;
• in the exchangeability documentation, the displayed formula appears to add the weight to the squared discrepancy, while the implementation multiplies by the weight.

I would clean those up rather than carrying unrelated diff noise into this already large PR.

What I would keep

A few parts look good to me and I would not change them just for the sake of changing them:

• using StatsAPI.HypothesisTest / StatsAPI.pvalue directly is appropriate; there is no need to depend on HypothesisTests.jl;
• the common CopulaTest result type is a good design choice;
• the Val-dispatch architecture is consistent with the rest of the package;
• centralizing the mechanics of simulation/randomization/multiplier/bootstrap calibration is a good separation of concerns;
• the Monte Carlo p-value correction used here is intentional and should not be replaced casually.

Suggested priority

For me the blockers before merge are:

1. preserve runtime model structure/configuration in composite GOF refitting;
2. handle ties conservatively instead of silently applying continuous-margin procedures.

Then I would:

3. verify the exact normalization of the published statistics;
4. keep the internal extension hooks private and adjust the developer guide accordingly;
5. add a guard around obviously explosive multiplier configurations;
6. rebase the tests and docs onto the current repository structure.

With those changes, I think the PR would fit the package architecture well without unnecessarily expanding the stable internal API surface.