LDA topic modeling

Trains a latent Dirichlet allocation model with scikit-learn using abstracts from Neurosynth.

import os

import pandas as pd

from nimare import annotate
from nimare.dataset import Dataset
from nimare.utils import get_resource_path

Load dataset with abstracts

dset = Dataset(os.path.join(get_resource_path(), "neurosynth_laird_studies.json"))

Initialize LDA model

model = annotate.lda.LDAModel(n_topics=5, max_iter=1000, text_column="abstract")

Run model

new_dset = model.fit(dset)

View results

This DataFrame is very large, so we will only show a slice of it.

id study_id contrast_id Neurosynth_TFIDF__001 Neurosynth_TFIDF__01 Neurosynth_TFIDF__05 Neurosynth_TFIDF__10 Neurosynth_TFIDF__100 Neurosynth_TFIDF__11 Neurosynth_TFIDF__12
0 17029760-1 17029760 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
1 18760263-1 18760263 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
2 19162389-1 19162389 1 0.0 0.0 0.0 0.000000 0.0 0.176321 0.0
3 19603407-1 19603407 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
4 20197097-1 20197097 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
5 22569543-1 22569543 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
6 22659444-1 22659444 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
7 23042731-1 23042731 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0
8 23702412-1 23702412 1 0.0 0.0 0.0 0.061006 0.0 0.000000 0.0
9 24681401-1 24681401 1 0.0 0.0 0.0 0.000000 0.0 0.000000 0.0


Given that this DataFrame is very wide (many terms), we will transpose it before presenting it.

model.distributions_["p_topic_g_word_df"].T.head(10)
LDA5__1 LDA5__2 LDA5__3 LDA5__4 LDA5__5
10 1.000687 0.001000 1.001313 0.001000 0.001
abstract 0.001000 0.001000 2.001000 0.001000 0.001
action 1.000640 0.001000 1.001360 0.001000 0.001
active 2.999361 1.002639 0.001000 0.001000 0.001
addition 1.999514 1.001881 0.001000 2.001605 0.001
additionally 1.000699 0.001000 0.001000 1.001301 0.001
affective 4.000566 0.001000 2.001434 0.001000 0.001
affective processes 2.001000 0.001000 0.001000 0.001000 0.001
ale 0.001000 0.001000 0.001000 2.001000 0.001
altered 0.001000 0.001000 0.001000 4.001000 0.001


LDA5__1 LDA5__2 LDA5__3 LDA5__4 LDA5__5
Token
0 connectivity stimulation social functional likelihood
1 functional identified human cortex robust
2 functional connectivity methods cortex network important
3 macm prefrontal functional literature brodmann
4 anterior published motor control additionally
5 cognitive use maps error ventromedial
6 human magnetic cognitive altered differential
7 posterior resting task lateral estimation ale
8 networks transcranial magnetic functional segregation cortices morphometry vbm
9 analytic transcranial segregation likelihood working memory


Total running time of the script: ( 0 minutes 5.115 seconds)

Gallery generated by Sphinx-Gallery