LDA topic modeling

Trains a latent Dirichlet allocation model with scikit-learn using abstracts from Neurosynth.

import os

import pandas as pd

from nimare import annotate
from nimare.nimads import Studyset
from nimare.utils import get_resource_path

Load Studyset with abstracts

studyset = Studyset(
    os.path.join(get_resource_path(), "neurosynth_laird_studyset.json"),
    target="mni152_2mm",
)

Initialize LDA model

model = annotate.lda.LDAModel(n_topics=5, max_iter=1000, text_column="abstract")

Run model

new_studyset = model.fit(studyset)

View results

This DataFrame is very large, so we will only show a slice of it.

new_studyset.annotations_df[new_studyset.annotations_df.columns[:10]].head(10)

	id	study_id	contrast_id	LDA5__1_literature_talairach_estimation	LDA5__2_connectivity_functional_human	LDA5__3_cortex_lateral_frontal	LDA5__4_control_error_network	LDA5__5_posterior_anterior_identified	Neurosynth_TFIDF__001	Neurosynth_TFIDF__01
0	17029760-1	17029760	1	0.004247	0.004182	0.004221	0.004201	0.983149	0.0	0.0
1	18760263-1	18760263	1	0.008014	0.008016	0.967889	0.008042	0.008038	0.0	0.0
2	19162389-1	19162389	1	0.007484	0.007190	0.970830	0.007265	0.007231	0.0	0.0
3	19603407-1	19603407	1	0.003467	0.986099	0.003462	0.003486	0.003486	0.0	0.0
4	20197097-1	20197097	1	0.979892	0.005021	0.005022	0.005043	0.005022	0.0	0.0
5	22569543-1	22569543	1	0.002685	0.989137	0.002698	0.002689	0.002792	0.0	0.0
6	22659444-1	22659444	1	0.001500	0.640450	0.001514	0.001509	0.355028	0.0	0.0
7	23042731-1	23042731	1	0.002443	0.682402	0.175601	0.002475	0.137079	0.0	0.0
8	23702412-1	23702412	1	0.002198	0.783366	0.210029	0.002222	0.002186	0.0	0.0
9	24681401-1	24681401	1	0.005150	0.005170	0.005155	0.979378	0.005147	0.0	0.0

Given that this DataFrame is very wide (many terms), we will transpose it before presenting it.

model.distributions_["p_topic_g_word_df"].T.head(10)

	LDA5__1_literature_talairach_estimation	LDA5__2_connectivity_functional_human	LDA5__3_cortex_lateral_frontal	LDA5__4_control_error_network	LDA5__5_posterior_anterior_identified
10	0.001000	2.000845	0.001000	0.001000	0.001000
abstract	0.001000	2.000879	0.001000	0.001000	0.001000
action	0.001000	2.000864	0.001000	0.001000	0.001000
active	0.001000	0.999869	2.001531	0.001000	1.001389
addition	1.002324	1.999470	1.000826	0.001000	1.001191
additionally	0.001000	0.998772	0.001000	1.002981	0.001000
affective	0.001000	6.000872	0.001000	0.001000	0.001000
affective processes	0.001000	0.001000	0.001000	0.001000	2.000914
ale	2.000962	0.001000	0.001000	0.001000	0.001000
altered	0.001000	2.999675	0.001000	1.002193	0.001000

n_top_terms = 10
top_term_df = model.distributions_["p_topic_g_word_df"].T
temp_df = top_term_df.copy()
top_term_df = pd.DataFrame(columns=top_term_df.columns, index=range(n_top_terms))
top_term_df.index.name = "Token"
for col in top_term_df.columns:
    top_tokens = temp_df.sort_values(by=col, ascending=False).index.tolist()[:n_top_terms]
    top_term_df.loc[:, col] = top_tokens

top_term_df

	LDA5__1_literature_talairach_estimation	LDA5__2_connectivity_functional_human	LDA5__3_cortex_lateral_frontal	LDA5__4_control_error_network	LDA5__5_posterior_anterior_identified
Token
0	literature	connectivity	cortex	control	posterior
1	talairach	functional	lateral	error	anterior
2	estimation	human	frontal	network	identified
3	likelihood estimation	macm	motor	functional	stimulation
4	likelihood	cognitive	cognition	additionally	seed
5	coordinate	functional connectivity	prefrontal	role	task
6	estimation ale	social	prefrontal cortex	properties	functional connectivity
7	ale	networks	lobe	tasks	neuropsychiatric
8	suggest	analytic	frontal pole	plays	implicated
9	quantitative	maps	pole	hemisphere	cluster