The Corrector class

Here we take a look at multiple comparisons correction in meta-analyses.

from pprint import pprint

import matplotlib.pyplot as plt
import seaborn as sns
from nilearn.plotting import plot_stat_map

Download data

from nimare.extract import download_nidm_pain

dset_dir = download_nidm_pain()

Load Dataset

import os

from nimare.dataset import Dataset
from nimare.utils import get_resource_path

dset_file = os.path.join(get_resource_path(), "nidm_pain_dset.json")
dset = Dataset(dset_file)
dset.update_path(dset_dir)

mask_img = dset.masker.mask_img

Multiple comparisons correction in coordinate-based meta-analyses

Tip

For more information multiple comparisons correction and CBMA in NiMARE, see Multiple comparisons correction.

from nimare.meta.cbma.ale import ALE

# First, we need to fit the Estimator to the Dataset.
meta = ALE(null_method="approximate")
results = meta.fit(dset)

# We can check which FWE correction methods are available for the ALE Estimator
# with the ``inspect`` class method.
from nimare.correct import FWECorrector

print(FWECorrector.inspect(results))

Out:

['bonferroni', 'montecarlo']

Apply the Corrector to the MetaResult

Now that we know what FWE correction methods are available, we can use one.

The “montecarlo” method is a special one that is implemented within the Estimator, rather than in the Corrector.

corr = FWECorrector(method="montecarlo", n_iters=50, n_cores=2)
cres = corr.transform(results)

DISTS_TO_PLOT = [
    "values_desc-size_level-cluster_corr-fwe_method-montecarlo",
    "values_desc-mass_level-cluster_corr-fwe_method-montecarlo",
    "values_level-voxel_corr-fwe_method-montecarlo",
]
XLABELS = [
    "Maximum Cluster Size (Voxels)",
    "Maximum Cluster Mass",
    "Maximum Summary Statistic (ALE Value)",
]

fig, axes = plt.subplots(figsize=(8, 8), nrows=3)
null_dists = cres.estimator.null_distributions_

for i_ax, dist_name in enumerate(DISTS_TO_PLOT):
    xlabel = XLABELS[i_ax]
    sns.histplot(x=null_dists[dist_name], bins=40, ax=axes[i_ax])
    axes[i_ax].set_title(dist_name)
    axes[i_ax].set_xlabel(xlabel)
    axes[i_ax].set_xlim(0, None)

fig.tight_layout()

Out:

  0%|          | 0/50 [00:00<?, ?it/s]
  2%|2         | 1/50 [00:06<05:34,  6.83s/it]
  6%|6         | 3/50 [00:07<01:43,  2.19s/it]
 10%|#         | 5/50 [00:08<00:59,  1.31s/it]
 14%|#4        | 7/50 [00:10<00:42,  1.00it/s]
 18%|#8        | 9/50 [00:11<00:33,  1.24it/s]
 22%|##2       | 11/50 [00:12<00:28,  1.39it/s]
 26%|##6       | 13/50 [00:13<00:23,  1.56it/s]
 30%|###       | 15/50 [00:14<00:21,  1.59it/s]
 34%|###4      | 17/50 [00:15<00:19,  1.65it/s]
 38%|###8      | 19/50 [00:16<00:17,  1.75it/s]
 42%|####2     | 21/50 [00:17<00:16,  1.77it/s]
 46%|####6     | 23/50 [00:18<00:14,  1.84it/s]
 50%|#####     | 25/50 [00:19<00:13,  1.83it/s]
 52%|#####2    | 26/50 [00:19<00:11,  2.12it/s]
 54%|#####4    | 27/50 [00:20<00:12,  1.80it/s]
 56%|#####6    | 28/50 [00:20<00:10,  2.08it/s]
 58%|#####8    | 29/50 [00:21<00:12,  1.75it/s]
 60%|######    | 30/50 [00:21<00:09,  2.16it/s]
 62%|######2   | 31/50 [00:22<00:10,  1.75it/s]
 64%|######4   | 32/50 [00:23<00:08,  2.05it/s]
 66%|######6   | 33/50 [00:23<00:09,  1.71it/s]
 68%|######8   | 34/50 [00:24<00:07,  2.14it/s]
 70%|#######   | 35/50 [00:24<00:08,  1.78it/s]
 72%|#######2  | 36/50 [00:25<00:06,  2.02it/s]
 74%|#######4  | 37/50 [00:25<00:07,  1.70it/s]
 76%|#######6  | 38/50 [00:26<00:05,  2.15it/s]
 78%|#######8  | 39/50 [00:27<00:06,  1.63it/s]
 80%|########  | 40/50 [00:27<00:04,  2.03it/s]
 82%|########2 | 41/50 [00:28<00:05,  1.77it/s]
 84%|########4 | 42/50 [00:28<00:04,  1.97it/s]
 86%|########6 | 43/50 [00:29<00:04,  1.75it/s]
 88%|########8 | 44/50 [00:29<00:02,  2.07it/s]
 90%|######### | 45/50 [00:30<00:02,  1.87it/s]
 92%|#########2| 46/50 [00:30<00:02,  1.94it/s]
 94%|#########3| 47/50 [00:31<00:01,  1.87it/s]
 96%|#########6| 48/50 [00:31<00:00,  2.03it/s]
 98%|#########8| 49/50 [00:32<00:00,  1.92it/s]
100%|##########| 50/50 [00:32<00:00,  2.10it/s]
100%|##########| 50/50 [00:32<00:00,  1.53it/s]

You can also look at the description of the Corrector.

print("Description:")
pprint(cres.description_)
print("References:")
pprint(cres.bibtex_)

Out:

Description:
('An activation likelihood estimation (ALE) meta-analysis '
 '\\citep{turkeltaub2002meta,turkeltaub2012minimizing,eickhoff2012activation} '
 'was performed with NiMARE 0.2.1+0.g2b1c23a.dirty (RRID:SCR_017398; '
 '\\citealt{Salo2023}), using a(n) ALE kernel. An ALE kernel '
 '\\citep{eickhoff2012activation} was used to generate study-wise modeled '
 'activation maps from coordinates. In this kernel method, each coordinate is '
 'convolved with a Gaussian kernel with full-width at half max values '
 'determined on a study-wise basis based on the study sample sizes according '
 'to the formulae provided in \\cite{eickhoff2012activation}. For voxels with '
 'overlapping kernels, the maximum value was retained. ALE values were '
 'converted to p-values using an approximate null distribution '
 '\\citep{eickhoff2012activation}. The input dataset included 267 foci from 21 '
 'experiments, with a total of 334 participants. Family-wise error rate '
 'correction was performed using a Monte Carlo procedure. In this procedure, '
 'null datasets are generated in which dataset coordinates are substituted '
 'with coordinates randomly drawn from the meta-analysis mask, and maximum '
 'values are retained. This procedure was repeated 50 times to build null '
 'distributions of summary statistics, cluster sizes, and cluster masses. '
 'Clusters for cluster-level correction were defined using edge-wise '
 'connectivity and a voxel-level threshold of p < 0.001 from the uncorrected '
 'null distribution.')
References:
('@article{Salo2023,\n'
 '  doi = {10.52294/001c.87681},\n'
 '  url = {https://doi.org/10.52294/001c.87681},\n'
 '  year = {2023},\n'
 '  volume = {3},\n'
 '  pages = {1 - 32},\n'
 '  author = {Taylor Salo and Tal Yarkoni and Thomas E. Nichols and '
 'Jean-Baptiste Poline and Murat Bilgel and Katherine L. Bottenhorn and Dorota '
 'Jarecka and James D. Kent and Adam Kimbler and Dylan M. Nielson and Kendra '
 'M. Oudyk and Julio A. Peraza and Alexandre Pérez and Puck C. Reeders and '
 'Julio A. Yanes and Angela R. Laird},\n'
 '  title = {NiMARE: Neuroimaging Meta-Analysis Research Environment},\n'
 '  journal = {Aperture Neuro}\n'
 '}\n'
 '@article{eickhoff2012activation,\n'
 '  title={Activation likelihood estimation meta-analysis revisited},\n'
 '  author={Eickhoff, Simon B and Bzdok, Danilo and Laird, Angela R and Kurth, '
 'Florian and Fox, Peter T},\n'
 '  journal={Neuroimage},\n'
 '  volume={59},\n'
 '  number={3},\n'
 '  pages={2349--2361},\n'
 '  year={2012},\n'
 '  publisher={Elsevier},\n'
 '  url={https://doi.org/10.1016/j.neuroimage.2011.09.017},\n'
 '  doi={10.1016/j.neuroimage.2011.09.017}\n'
 '}\n'
 '@article{turkeltaub2002meta,\n'
 '  title={Meta-analysis of the functional neuroanatomy of single-word '
 'reading: method and validation},\n'
 '  author={Turkeltaub, Peter E and Eden, Guinevere F and Jones, Karen M and '
 'Zeffiro, Thomas A},\n'
 '  journal={Neuroimage},\n'
 '  volume={16},\n'
 '  number={3},\n'
 '  pages={765--780},\n'
 '  year={2002},\n'
 '  publisher={Elsevier},\n'
 '  url={https://doi.org/10.1006/nimg.2002.1131},\n'
 '  doi={10.1006/nimg.2002.1131}\n'
 '}\n'
 '@article{turkeltaub2012minimizing,\n'
 '  title={Minimizing within-experiment and within-group effects in activation '
 'likelihood estimation meta-analyses},\n'
 '  author={Turkeltaub, Peter E and Eickhoff, Simon B and Laird, Angela R and '
 'Fox, Mick and Wiener, Martin and Fox, Peter},\n'
 '  journal={Human brain mapping},\n'
 '  volume={33},\n'
 '  number={1},\n'
 '  pages={1--13},\n'
 '  year={2012},\n'
 '  publisher={Wiley Online Library},\n'
 '  url={https://doi.org/10.1002/hbm.21186},\n'
 '  doi={10.1002/hbm.21186}\n'
 '}')

Show corrected results

MAPS_TO_PLOT = [
    "z",
    "z_desc-size_level-cluster_corr-FWE_method-montecarlo",
    "z_desc-mass_level-cluster_corr-FWE_method-montecarlo",
    "z_level-voxel_corr-FWE_method-montecarlo",
]
TITLES = [
    "Uncorrected z-statistics",
    "Cluster-size FWE-corrected z-statistics",
    "Cluster-mass FWE-corrected z-statistics",
    "Voxel-level FWE-corrected z-statistics",
]

fig, axes = plt.subplots(figsize=(8, 10), nrows=4)

for i_ax, map_name in enumerate(MAPS_TO_PLOT):
    title = TITLES[i_ax]
    plot_stat_map(
        cres.get_map(map_name),
        draw_cross=False,
        cmap="RdBu_r",
        threshold=0.5,
        cut_coords=[0, 0, -8],
        figure=fig,
        axes=axes[i_ax],
    )
    axes[i_ax].set_title(title)

Multiple comparisons correction in image-based meta-analyses

from nimare.correct import FDRCorrector
from nimare.meta.ibma import Stouffers

meta = Stouffers(resample=True)
results = meta.fit(dset)
print(f"FWECorrector options: {FWECorrector.inspect(results)}")
print(f"FDRCorrector options: {FDRCorrector.inspect(results)}")

Out:

FWECorrector options: ['bonferroni']
FDRCorrector options: ['indep', 'negcorr']

Note that the FWECorrector does not support a “montecarlo” method for the Stouffers Estimator. This is because NiMARE does not have a Monte Carlo-based method implemented for most IBMA algorithms.

Apply the Corrector to the MetaResult

corr = FDRCorrector(method="indep", alpha=0.05)
cres = corr.transform(results)

Show corrected results

fig, axes = plt.subplots(figsize=(8, 6), nrows=2)
plot_stat_map(
    cres.get_map("z"),
    draw_cross=False,
    cmap="RdBu_r",
    threshold=0.5,
    cut_coords=[0, 0, -8],
    figure=fig,
    axes=axes[0],
)
axes[0].set_title("Uncorrected z-statistics")
plot_stat_map(
    cres.get_map("z_corr-FDR_method-indep"),
    draw_cross=False,
    cmap="RdBu_r",
    threshold=0.5,
    cut_coords=[0, 0, -8],
    figure=fig,
    axes=axes[1],
)
axes[1].set_title("FDR-corrected z-statistics")

Out:

/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/0.2.1/lib/python3.8/site-packages/nilearn/_utils/niimg.py:61: UserWarning: Non-finite values detected. These values will be replaced with zeros.
  warn(

Text(0.5, 1.0, 'FDR-corrected z-statistics')