🙌 Again, another round of applause and huge thanks to the greatest mentor @sam, who kicked off this exciting project with his dissertation. Your guidance was pivotal. I sincerely would not have survived my PhD journey without @samnastase.bsky.social and @haxbylab.bsky.social ...!
🤯 We were surprised at how well these behavioral-arrangement models capture cortical representational geometry, including in areas like VT—our findings suggest that high-level, behaviorally-relevant features of action understanding occupy a privileged role in cortical representation.
🥧 Variance partitioning revealed that behavioral models of transitivity and sociality captured a large portion of unique variance throughout the action observation network, and extending into ventral temporal cortex.
💃 We found that, out of nine models, the behavioral models capturing the meaning of the actions depicted in the stimuli—the transitivity and sociality models—best captured neural representational geometry throughout much of the action observation network.
🧠 We tested all nine of these models against neural representational geometries (with hybrid hyperalignment based on a separate movie stimulus!) using both a searchlight analysis and in regions of interest.
🎞️ Finally, we constructed semantic RDMs from word embeddings based on verbs and nonverbs in an annotation of the stimulus, a gaze RDM from a separate eye-tracking sample, and a visual motion energy RDM. This amounted to 3+ hours of fMRI data and 5+ hours of behavioral data per participant (N = 23)!
🖼️ We also included three other behavioral arrangement tasks where participants organized static images from the video stimuli according to their visual content: scene, person, and object features.
👆 To capture behaviorally-relevant action features, we had participants perform two behavioral arrangement tasks where they organized the action videos according to their object-/goal-related features (transitivity) or their social features (sociality).
🎥 We developed a condition-rich fMRI design with 90 real-world action videos spanning a variety of social and nonsocial action categories. What are the organizing features of observed action representation across cortex? We built several different kinds of models to find out…
Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision available at bioRxiv!
🚨 New paper out with @samnastase.bsky.social and @haxbylab.bsky.social! We use representational similarity analysis to test how well behavioral, semantic, and visual models capture cortical representational geometries when viewing naturalistic action videos: doi.org/10.1101/2024...
