herrick fung

Stimulus reliability but not boundary distance manipulations violate the folded-X pattern of confidence The folded-X pattern has been identified as a critical signature of confidence: as conditions become easier, confidence increases for correct trials b…

Our new paper is out in Cognition! What determines whether confidence follows the classic "folded-X" pattern vs. the "double-increase" pattern? The answer lies in the type of stimulus manipulation. Big thanks to my advisor Doby @dobyrahnev.bsky.social and co-first author @herrickfung.bsky.social !

More evidence for ‘task-defining’ vs. ‘auxiliary’ stimulus manipulations, each with distinct effects on confidence.

Check out my new paper with @dobyrahnev.bsky.social and @kaixue98.bsky.social, now in Cognition. Also check out our earlier sister paper on this matter!

Kai's thread 👇

Turns out that individual differences in accuracy, confidence, and RT among ANNs that only differ in their random initialization mimic the individual differences in humans.

It may be time for NeuroAI to take individual differences even more seriously.

Check out Herrick's thread 👇

Huge thanks to my two wonderful mentors @apurvaratan.bsky.social & @dobyrahnev.bsky.social

Preprint: www.biorxiv.org/content/10.6...
Data and code: github.com/herrickfung/...
8/8

Our work highlights an important but often overlooked dimension in human-model evaluations. We propose that a truly human-like model should not only exhibit strong alignment with the average human behavior, but also capture the structured idiosyncrasies that characterize individual humans.

7/8

We then conducted a preregistered human experiment (N = 60) and showed generalization of these findings to a more complex 10-choice object task, though the human-ANN mappings are generally less robust than the human-human benchmark, demonstrating partial mapping of individual differences.

6/8

Critically, these human-ANN mappings are consistent not only across splits of the data but also across behavioral metrics (accuracy, confidence, RT). This means that if an ANN instance is more similar to you in accuracy, it’s also more similar to you in confidence and RT.

5/8

To ensure robustness, we performed 1,000 bootstrap iterations of data splits and found that human–ANN mappings exhibit variability and consistency comparable to the human–human benchmark.

4/8

We found substantial individual differences in both humans and ANNs. Crucially, different ANN instances varied in how well they aligned with individual humans, with each instance providing a distinct fit to different subjects.

3/8

We reanalyzed a dataset of 60 humans performing an 8-choice digit task and, in parallel, trained 60 randomly initialized instances of each ANN architecture (RTNet, AlexNet, and ResNet18) on the same task. We then quantified human–ANN alignment at the individual level.

2/8

🚨 New preprint on individual differences in artificial neural networks and human behavior.

We show that individual differences among ANN instances trained with different random initializations capture the individual differences in human behavior.

1/8

Using artificial neural networks to reveal the human confidence computation Author summary Human decisions are accompanied by a sense of confidence which reflects the decision accuracy. Conventionally, human confidence has been studied using two-choice tasks with simple stimu...

How do people compute a sense of confidence? This question is usually addressed using very simple images because we don't know how complex stimuli are represented internally. In a new paper, we addressed this question using artificial neural networks (ANNs).

journals.plos.org/ploscompbiol...

Using artificial neural networks to reveal the human confidence computation Author summary Human decisions are accompanied by a sense of confidence which reflects the decision accuracy. Conventionally, human confidence has been studied using two-choice tasks with simple stimu...

Glad to have played a small part in this work led by Medha Shekhar, now out in PLOS Comp Bio. Using neural networks, we show that in multi-alternative perceptual task, humans compute confidence by the difference between the internal evidence of the top two choices.

journals.plos.org/ploscompbiol...

New preprint: Confidence-accuracy dissociations in perceptual decision making. A review I was supposed to write 3 years ago for my VSS Young Investigator Award. Better late than never 😅 I tried to organize the literature and explore the likely mechanisms. Feedback welcome!

osf.io/preprints/ps...

"How participants create illusory experiences to help experimenters." A new preprint of a multi-study manuscript w/
@zoltandienes.bsky.social, @anilseth.bsky.social & Ryan Scott, extending previous work on demand characteristics & phenomenological control in psychological effects. osf.io/n2ukj

Excited to share my first paper: Model–Behavior Alignment under Flexible Evaluation: When the Best-Fitting Model Isn’t the Right One (NeurIPS 2025). link below.

Need more fMRI data (beyond the amazing NSD)? Introducing MOSAIC! Incredible effort led expertly by Ben Lahner, with help from grad student Mayukh Deb. Work in collaboration with the amazing Aude Oliva! @neurosky.bsky.social. More below..

Human-like individual differences emerge from random weight initializations in neural networks Much of AI research targets the behavior of an average human, a focus that traces to Turing’s imitation game. Yet, no two human individuals behave exactly alike. In this study, we show that artificial...

"ANN instances showed consistent variation in their alignment with specific human subjects." www.biorxiv.org/content/10.1... This particular network is just like me fr!

Perceptual confidence has near perfect access to the existence of discrete representations, but only weak access to precision - PubMed Perceptual confidence describes the degree to which we believe our internal perceptual representations reflect the external stimuli that caused them. How confidence is derived from internal representations is currently debated, but answering this question is made difficult because the nature of inte …

Clever behavioural experiment exposing how confidence is formed in visual perception

pubmed.ncbi.nlm.nih.gov/41138620/

Hi Tahereh—Just in case you missed my recent post, here's the link to my preprint: www.biorxiv.org/content/10.1...

So lucky to have two wonderful mentors behind this project @apurvaratan.bsky.social & @dobyrahnev.bsky.social

Human-like individual differences emerge from random weight initializations in neural networks Much of AI research targets the behavior of an average human, a focus that traces to Turing's imitation game. Yet, no two human individuals behave exactly alike. In this study, we show that artificial...

No two humans behave exactly alike. But what about neural networks? We found early evidence that human-like individual differences in behavior emerge from networks trained with different initializations. Here’s a peek at our results—to be presented at UniReps & DBM @NeurIPS. Full paper on the way!

Hi Tahereh — Good to hear from you. We haven't posted it yet, but I'll put it on arXiv in the next few days. I'll send you the link once it's up, in case you'd like to add it now or later.

It looks like stimulus manipulations can be divided into "task-defining" and "auxiliary". The manipulations from each group have very different effects on accuracy vs. confidence. And all auxiliary manipulations seem to work in basically the same way. Really cool stuff by @herrickfung.bsky.social.

Huge thanks to Medha Shekhar, @kaixue98.bsky.social, @manurausch.bsky.social, & @dobyrahnev.bsky.social for their guidance and collaboration on this project.

Full article: www.sciencedirect.com/science/arti...

Data and code: github.com/herrickfung/...

n/n

These findings show a clear behavioral difference between manipulating the task-defining feature (orientation) and other auxiliary manipulations that only affect stimulus clarity, offering a way to predict how novel manipulations affect confidence and accuracy.

4/n

Manipulating other features—size, duration, spatial frequency, and noise—though visually distinct, showed strikingly similar effects: they (1) affected confidence more strongly than accuracy, (2) showed sub-additive or no interaction with other features, & (3) violated the Folded-X pattern.

3/n

In an orientation discrimination task, manipulating the “task-defining” tilt offset of the Gabor demonstrated three unique effects: it (1) affected accuracy more strongly than confidence, (2) showed supra-additive interaction with other features, & (3) exhibited the Folded-X pattern.

2/n

Similarities and differences in the effects of different stimulus manipulations on accuracy and confidence Visual stimuli can vary in multiple dimensions that affect accuracy and confidence in a perceptual decision-making task. However, previous studies hav…

Glad to see my first-year project is out!

In two experiments, we manipulated multiple stimulus features in a perception task, yet their effects on confidence and accuracy fell into just two distinct behavioral patterns, offering a way to predict the effects of novel stimulus manipulations.

1/n