LDA topic modeling

Trains a latent Dirichlet allocation model with scikit-learn using abstracts from Neurosynth.

import os

import pandas as pd

from nimare import annotate
from nimare.dataset import Dataset
from nimare.utils import get_resource_path

Load dataset with abstracts

dset = Dataset(os.path.join(get_resource_path(), "neurosynth_laird_studies.json"))

Initialize LDA model

model = annotate.lda.LDAModel(n_topics=5, max_iter=1000, text_column="abstract")

Run model

new_dset = model.fit(dset)

View results

This DataFrame is very large, so we will only show a slice of it.

id study_id contrast_id Neurosynth_TFIDF__001 Neurosynth_TFIDF__01 Neurosynth_TFIDF__05 Neurosynth_TFIDF__10 Neurosynth_TFIDF__100 Neurosynth_TFIDF__11 Neurosynth_TFIDF__12
0 17029760-1 17029760 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
1 18760263-1 18760263 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
2 19162389-1 19162389 1 0.0 0.0 0.0 0.000000 0.0 0.176321 0.0
3 19603407-1 19603407 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
4 20197097-1 20197097 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
5 22569543-1 22569543 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
6 22659444-1 22659444 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
7 23042731-1 23042731 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
8 23702412-1 23702412 1 0.0 0.0 0.0 0.061006 0.0 0.000000 0.0
9 24681401-1 24681401 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0


Given that this DataFrame is very wide (many terms), we will transpose it before presenting it.

model.distributions_["p_topic_g_word_df"].T.head(10)
LDA5__1_connectivity_functional_functional connectivity LDA5__2_connectivity_macm_methods LDA5__3_connectivity_functional_anterior LDA5__4_cortex_motor_literature LDA5__5_cortex_functional_cbp
10 0.001000 0.001000 1.000894 0.001000 1.001106
abstract 2.001000 0.001000 0.001000 0.001000 0.001000
action 0.001000 1.001461 1.000539 0.001000 0.001000
active 0.001000 0.001000 1.000541 3.001459 0.001000
addition 0.001000 0.001000 1.059705 2.001422 1.941873
additionally 1.000778 0.001000 1.001222 0.001000 0.001000
affective 2.660249 0.001000 2.341711 0.001000 1.001040
affective processes 0.001000 0.001000 2.001000 0.001000 0.001000
ale 1.000852 0.001000 0.001000 1.001148 0.001000
altered 4.001000 0.001000 0.001000 0.001000 0.001000


LDA5__1_connectivity_functional_functional connectivity LDA5__2_connectivity_macm_methods LDA5__3_connectivity_functional_anterior LDA5__4_cortex_motor_literature LDA5__5_cortex_functional_cbp
Token
0 connectivity connectivity connectivity cortex cortex
1 functional macm functional motor functional
2 functional connectivity methods anterior literature cbp
3 structural connectivity patterns human published lateral
4 social patterns networks prefrontal parcellation
5 modeling behavioral cognitive identified dorsal
6 posterior connections insula talairach frontal
7 task mapping functional networks error cognition
8 method stimulation cortex stimulation ventromedial
9 correlations database memory prefrontal cortex clusters


Total running time of the script: ( 0 minutes 3.358 seconds)

Gallery generated by Sphinx-Gallery