The Corrector class

Here we take a look at multiple comparisons correction in meta-analyses.

from pprint import pprint

import matplotlib.pyplot as plt
import seaborn as sns
from nilearn.plotting import plot_stat_map

Download data

from nimare.extract import download_nidm_pain

dset_dir = download_nidm_pain()

Load Dataset

import os

from nimare.dataset import Dataset
from nimare.utils import get_resource_path

dset_file = os.path.join(get_resource_path(), "nidm_pain_dset.json")
dset = Dataset(dset_file)
dset.update_path(dset_dir)

mask_img = dset.masker.mask_img

Multiple comparisons correction in coordinate-based meta-analyses

Tip

For more information multiple comparisons correction and CBMA in NiMARE, see Multiple comparisons correction.

from nimare.meta.cbma.ale import ALE

# First, we need to fit the Estimator to the Dataset.
meta = ALE(null_method="approximate")
results = meta.fit(dset)

# We can check which FWE correction methods are available for the ALE Estimator
# with the ``inspect`` class method.
from nimare.correct import FWECorrector

print(FWECorrector.inspect(results))

['bonferroni', 'montecarlo']

Apply the Corrector to the MetaResult

Now that we know what FWE correction methods are available, we can use one.

The “montecarlo” method is a special one that is implemented within the Estimator, rather than in the Corrector.

corr = FWECorrector(method="montecarlo", n_iters=50, n_cores=2)
cres = corr.transform(results)

DISTS_TO_PLOT = [
    "values_desc-size_level-cluster_corr-fwe_method-montecarlo",
    "values_desc-mass_level-cluster_corr-fwe_method-montecarlo",
    "values_level-voxel_corr-fwe_method-montecarlo",
]
XLABELS = [
    "Maximum Cluster Size (Voxels)",
    "Maximum Cluster Mass",
    "Maximum Summary Statistic (ALE Value)",
]

fig, axes = plt.subplots(figsize=(8, 8), nrows=3)
null_dists = cres.estimator.null_distributions_

for i_ax, dist_name in enumerate(DISTS_TO_PLOT):
    xlabel = XLABELS[i_ax]
    sns.histplot(x=null_dists[dist_name], bins=40, ax=axes[i_ax])
    axes[i_ax].set_title(dist_name)
    axes[i_ax].set_xlabel(xlabel)
    axes[i_ax].set_xlim(0, None)

fig.tight_layout()

  0%|          | 0/50 [00:00<?, ?it/s]
  2%|▏         | 1/50 [00:03<02:28,  3.03s/it]
  6%|▌         | 3/50 [00:03<00:46,  1.02it/s]
 10%|█         | 5/50 [00:03<00:26,  1.72it/s]
 14%|█▍        | 7/50 [00:04<00:18,  2.35it/s]
 18%|█▊        | 9/50 [00:04<00:14,  2.74it/s]
 22%|██▏       | 11/50 [00:05<00:11,  3.36it/s]
 26%|██▌       | 13/50 [00:05<00:09,  3.91it/s]
 28%|██▊       | 14/50 [00:05<00:08,  4.35it/s]
 30%|███       | 15/50 [00:06<00:08,  3.95it/s]
 32%|███▏      | 16/50 [00:06<00:08,  4.23it/s]
 34%|███▍      | 17/50 [00:06<00:07,  4.58it/s]
 36%|███▌      | 18/50 [00:06<00:07,  4.50it/s]
 38%|███▊      | 19/50 [00:06<00:05,  5.21it/s]
 40%|████      | 20/50 [00:07<00:06,  4.64it/s]
 42%|████▏     | 21/50 [00:07<00:05,  4.85it/s]
 44%|████▍     | 22/50 [00:07<00:06,  4.10it/s]
 48%|████▊     | 24/50 [00:07<00:05,  4.39it/s]
 52%|█████▏    | 26/50 [00:08<00:05,  4.58it/s]
 56%|█████▌    | 28/50 [00:08<00:04,  4.73it/s]
 58%|█████▊    | 29/50 [00:08<00:04,  5.23it/s]
 60%|██████    | 30/50 [00:09<00:04,  4.92it/s]
 62%|██████▏   | 31/50 [00:09<00:03,  5.38it/s]
 64%|██████▍   | 32/50 [00:09<00:03,  4.50it/s]
 68%|██████▊   | 34/50 [00:09<00:03,  4.97it/s]
 70%|███████   | 35/50 [00:10<00:03,  4.79it/s]
 72%|███████▏  | 36/50 [00:10<00:02,  5.28it/s]
 74%|███████▍  | 37/50 [00:10<00:02,  4.83it/s]
 76%|███████▌  | 38/50 [00:10<00:02,  4.79it/s]
 78%|███████▊  | 39/50 [00:10<00:02,  4.89it/s]
 80%|████████  | 40/50 [00:11<00:01,  5.35it/s]
 82%|████████▏ | 41/50 [00:11<00:02,  4.30it/s]
 86%|████████▌ | 43/50 [00:11<00:01,  4.65it/s]
 90%|█████████ | 45/50 [00:12<00:01,  4.80it/s]
 94%|█████████▍| 47/50 [00:12<00:00,  4.93it/s]
 96%|█████████▌| 48/50 [00:12<00:00,  5.36it/s]
 98%|█████████▊| 49/50 [00:13<00:00,  4.55it/s]
100%|██████████| 50/50 [00:13<00:00,  4.92it/s]
100%|██████████| 50/50 [00:13<00:00,  3.78it/s]

You can also look at the description of the Corrector.

print("Description:")
pprint(cres.description_)
print("References:")
pprint(cres.bibtex_)

Description:
('An activation likelihood estimation (ALE) meta-analysis '
 '\\citep{turkeltaub2002meta,turkeltaub2012minimizing,eickhoff2012activation} '
 'was performed with NiMARE 0.5.0 (RRID:SCR_017398; \\citealt{Salo2023}), '
 'using a(n) ALE kernel. An ALE kernel \\citep{eickhoff2012activation} was '
 'used to generate study-wise modeled activation maps from coordinates. In '
 'this kernel method, each coordinate is convolved with a Gaussian kernel with '
 'full-width at half max values determined on a study-wise basis based on the '
 'study sample sizes according to the formulae provided in '
 '\\cite{eickhoff2012activation}. For voxels with overlapping kernels, the '
 'maximum value was retained. ALE values were converted to p-values using an '
 'approximate null distribution \\citep{eickhoff2012activation}. The input '
 'dataset included 267 foci from 21 experiments, with a total of 334 '
 'participants. Family-wise error rate correction was performed using a Monte '
 'Carlo procedure. In this procedure, null datasets are generated in which '
 'dataset coordinates are substituted with coordinates randomly drawn from the '
 'meta-analysis mask, and maximum values are retained. This procedure was '
 'repeated 50 times to build null distributions of summary statistics, cluster '
 'sizes, and cluster masses. Clusters for cluster-level correction were '
 'defined using edge-wise connectivity and a voxel-level threshold of p < '
 '0.001 from the uncorrected null distribution.')
References:
('@article{Salo2023,\n'
 '  doi = {10.52294/001c.87681},\n'
 '  url = {https://doi.org/10.52294/001c.87681},\n'
 '  year = {2023},\n'
 '  volume = {3},\n'
 '  pages = {1 - 32},\n'
 '  author = {Taylor Salo and Tal Yarkoni and Thomas E. Nichols and '
 'Jean-Baptiste Poline and Murat Bilgel and Katherine L. Bottenhorn and Dorota '
 'Jarecka and James D. Kent and Adam Kimbler and Dylan M. Nielson and Kendra '
 'M. Oudyk and Julio A. Peraza and Alexandre Pérez and Puck C. Reeders and '
 'Julio A. Yanes and Angela R. Laird},\n'
 '  title = {NiMARE: Neuroimaging Meta-Analysis Research Environment},\n'
 '  journal = {Aperture Neuro}\n'
 '}\n'
 '@article{eickhoff2012activation,\n'
 '  title={Activation likelihood estimation meta-analysis revisited},\n'
 '  author={Eickhoff, Simon B and Bzdok, Danilo and Laird, Angela R and Kurth, '
 'Florian and Fox, Peter T},\n'
 '  journal={Neuroimage},\n'
 '  volume={59},\n'
 '  number={3},\n'
 '  pages={2349--2361},\n'
 '  year={2012},\n'
 '  publisher={Elsevier},\n'
 '  url={https://doi.org/10.1016/j.neuroimage.2011.09.017},\n'
 '  doi={10.1016/j.neuroimage.2011.09.017}\n'
 '}\n'
 '@article{turkeltaub2002meta,\n'
 '  title={Meta-analysis of the functional neuroanatomy of single-word '
 'reading: method and validation},\n'
 '  author={Turkeltaub, Peter E and Eden, Guinevere F and Jones, Karen M and '
 'Zeffiro, Thomas A},\n'
 '  journal={Neuroimage},\n'
 '  volume={16},\n'
 '  number={3},\n'
 '  pages={765--780},\n'
 '  year={2002},\n'
 '  publisher={Elsevier},\n'
 '  url={https://doi.org/10.1006/nimg.2002.1131},\n'
 '  doi={10.1006/nimg.2002.1131}\n'
 '}\n'
 '@article{turkeltaub2012minimizing,\n'
 '  title={Minimizing within-experiment and within-group effects in activation '
 'likelihood estimation meta-analyses},\n'
 '  author={Turkeltaub, Peter E and Eickhoff, Simon B and Laird, Angela R and '
 'Fox, Mick and Wiener, Martin and Fox, Peter},\n'
 '  journal={Human brain mapping},\n'
 '  volume={33},\n'
 '  number={1},\n'
 '  pages={1--13},\n'
 '  year={2012},\n'
 '  publisher={Wiley Online Library},\n'
 '  url={https://doi.org/10.1002/hbm.21186},\n'
 '  doi={10.1002/hbm.21186}\n'
 '}')

Show corrected results

MAPS_TO_PLOT = [
    "z",
    "z_desc-size_level-cluster_corr-FWE_method-montecarlo",
    "z_desc-mass_level-cluster_corr-FWE_method-montecarlo",
    "z_level-voxel_corr-FWE_method-montecarlo",
]
TITLES = [
    "Uncorrected z-statistics",
    "Cluster-size FWE-corrected z-statistics",
    "Cluster-mass FWE-corrected z-statistics",
    "Voxel-level FWE-corrected z-statistics",
]

fig, axes = plt.subplots(figsize=(8, 10), nrows=4)

for i_ax, map_name in enumerate(MAPS_TO_PLOT):
    title = TITLES[i_ax]
    plot_stat_map(
        cres.get_map(map_name),
        draw_cross=False,
        cmap="RdBu_r",
        symmetric_cbar=True,
        threshold=0.5,
        cut_coords=[0, 0, -8],
        figure=fig,
        axes=axes[i_ax],
    )
    axes[i_ax].set_title(title)

Multiple comparisons correction in image-based meta-analyses

from nimare.correct import FDRCorrector
from nimare.meta.ibma import Stouffers

meta = Stouffers(resample=True)
results = meta.fit(dset)
print(f"FWECorrector options: {FWECorrector.inspect(results)}")
print(f"FDRCorrector options: {FDRCorrector.inspect(results)}")

/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
/home/docs/checkouts/readthedocs.org/user_builds/nimare/checkouts/stable/nimare/meta/ibma.py:118: FutureWarning: 'force_resample' will be set to 'True' by default in Nilearn 0.13.0.
Use 'force_resample=True' to suppress this warning.
  else resample_to_img(nib.load(img), mask_img, **self._resample_kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/image/resampling.py:805: FutureWarning: From release 0.13.0 onwards, this function will, by default, copy the header of the input image to the output. Currently, the header is reset to the default Nifti1Header. To suppress this warning and use the new behavior, set `copy_header=True`.
  return resample_img(
FWECorrector options: ['bonferroni']
FDRCorrector options: ['indep', 'negcorr']

Note that the FWECorrector does not support a “montecarlo” method for the Stouffers Estimator. This is because NiMARE does not have a Monte Carlo-based method implemented for most IBMA algorithms.

Apply the Corrector to the MetaResult

corr = FDRCorrector(method="indep", alpha=0.05)
cres = corr.transform(results)

Show corrected results

fig, axes = plt.subplots(figsize=(8, 6), nrows=2)
plot_stat_map(
    cres.get_map("z"),
    draw_cross=False,
    cmap="RdBu_r",
    symmetric_cbar=True,
    threshold=0.5,
    cut_coords=[0, 0, -8],
    figure=fig,
    axes=axes[0],
)
axes[0].set_title("Uncorrected z-statistics")
plot_stat_map(
    cres.get_map("z_corr-FDR_method-indep"),
    draw_cross=False,
    cmap="RdBu_r",
    symmetric_cbar=True,
    threshold=0.5,
    cut_coords=[0, 0, -8],
    figure=fig,
    axes=axes[1],
)
axes[1].set_title("FDR-corrected z-statistics")

/home/docs/checkouts/readthedocs.org/user_builds/nimare/envs/stable/lib/python3.9/site-packages/nilearn/plotting/img_plotting.py:1416: UserWarning: Non-finite values detected. These values will be replaced with zeros.
  safe_get_data(stat_map_img, ensure_finite=True),

Text(0.5, 1.0, 'FDR-corrected z-statistics')