Weβre launching a bi-weekly m-Path blog on #ESM / #EMA measurement, new papers & showcases.
ποΈ Every 2 weeks (Wed).
First up: item quality.
A paper by @gudruneisele.bsky.social introduces ESM-Q: a consensus checklist for good momentary items.
π Read the blog: blog.m-path.io/blog/our-blo...
Nice writeup of our new study in @jamapsychiatry.com by Kyle Mittan, a colleague here at @uarizona.bsky.social
news.arizona.edu/news/confuse...
1/n Out today in @jamapsychiatry.com: Interpretation Issues With the Patient Health Questionnaire Instructions. We find troubling variability in whether people think they should respond based on the frequency of the Sx or the frequency of being bothered by the Sx.
jamanetwork.com/journals/jam...
Sebastian Castro-Alvarez and I are organizing a symposium on Computerized Adaptive Testing and other techniques to reduce burden in EMA studies. Please reach out if youβre interested in joining, or if you know someone who might be!
The first preprint from the SHARE study is out! π₯³ We compared the effects of three different incentives (a bulk payment, a bulk payment with personalized feedback, and payment per beep) on data quantity, data quality, and participant experiences in a student sample.
Had a great time at the ESM Expert Network Meeting in Rotterdam! I presented the first SHARE study results: we found that paying participants per beep boosts compliance as compared to bulk payment (with or without personalised feedback)βand without harming data quality.
Poster: tinyurl.com/47asuaxj
Are you an ESM item measuring the quality of social experiences looking for validation? π
Because if you are, I've got some bad news for you. Our new preprint is out: osf.io/preprints/ps...
@gudruneisele.bsky.social @ginettelafit.bsky.social @lisapeeters.bsky.social @oliviajkirtley.bsky.social
Interested in ESM/EMA research?
Check out @lisapeeters.bsky.social's review of methodological variation in recent ESM studies.
It provides detailed numbers on current practices across the entire research pipelineβfrom research questions and study design to reporting.
π¨ New preprint: We compared 13 methods for detecting momentary careless responding in the WARN-D data (206k+ obs.). Tutorials guide you through each method. The takeaway? Diverging results, inherent subjectivity (to varying degrees), and a clear need for further validation.
osf.io/preprints/ps...
with @inezgermeys.bsky.social @robin8.bsky.social @lisapeeters.bsky.social Joana De Calheiros Velozo, Thomas Vaessen & Aleksandra Lachowicz
New preprint!
Does data quality drop when participants respond to #EMA / #ESM surveys in the presence of others?
We explored this in a pooled analysis of 4 ESM studies and found (subtle) shifts in burden & response behavior depending on participantsβ social context:
doi.org/10.31234/osf...
How Accurate & Consistent are Self-Report responses via Visual Analogue Scales (VAS) in Ecological Momentary Assessment & Digital Studies?
Examined in a massive EMA study (N = 3,761), our new #OpenAccess piece on this is just out:
online.ucpress.edu/collabra/art...
#PsychSciSky #AcademicSky
π§΅:
Experienced, bothered by, or both: People differ in how they understand the Patient Health Questionnaire (PHQ) items: https://doi.org/10.31234/osf.io/jcsq8_v1
Treemap showing measurement fragmentation across subfields in psychology. Hill-Shannon Diversity π·=1626.05
How often measures in the APA PsycTESTS database are (re)used according to the APA PsycInfo database: rarely, the majority are never reused.
Our fragmentation index (Hill-Shannon diversity) over time across subdisciplines shows fragmentation rising.
Our paper "A fragmented field" has just been accepted at AMPPS. We find it's not just you, psychology is really getting more confusing (construct and measure fragmentation is rising).
We updated the preprint with the (substantial) revision, please check it out.
osf.io/preprints/ps...
While I was unfortunately unable to travel to Belgium for the conference, I have uploaded the recording+slides+code for my #SAA2025 talk on multilevel multiple imputation for EMA research here: osf.io/xjg24/files/...
Happening now at #SAA2025! @gudruneisele.bsky.social leading a symposium on ESM data quality & quantity. Kicking us off @ccp-kuleuven.bsky.social PhD student, @millapihlajamaki.bsky.social on careless responding. Preprint: osf.io/preprints/ps... #ProudSupervisor
Do you like coffee β, ambulatory assessment π±and #OpenScience π? We are hosting a contributorsβ cafΓ© during lunch breaks & poster sessions @saa2025leuven.bsky.social. Come find the cafΓ© to have a cup of coffee and contribute your items to the repository! #SAA2025 𧡠β¬οΈ 1/4
Deviations from preregistrations seem to be the norm, not the exception. Great to see more guidance on how to navigate them thoughtfully.
Emily Willroth and @oliviaatherton.bsky.social's paper is another helpful resource:
doi.org/10.1177/2515...
New article in MBRβ: www.tandfonline.com/doi/full/10.... We show that latent Markov factor analysis (LMFA) can detect different types of careless responding in experience sampling data while also accounting for changes in attentive responding. With Esther Ulitzsch and @joranjongerling.bsky.social
It was a pleasure to present some work on improving ESM/EMA assessments at the Center for Self-Report Science at USC yesterday. What a bunch of great people with so many interesting perspectives β Iβve already learned a ton!
Are you heading to @saa2025leuven.bsky.social? Fancy extending your stay in Leuven to learn more about #OpenScience for Experience Sampling Method research with @ginettelafit.bsky.social & I? We are running a one-day course after the conference! More info here: tinyurl.com/334rytta
π£ New blog post! What if ESM data grew on trees? π³ Read more about the EMOTE database in our latest blog by Imogen Smith, Pete Koval, @elisekalo.bsky.social, & Daniel Russo-Batterham! esmitemrepositoryinfo.com/blog-posts#E... #ESM #EMA #OpenScience
The first preprint from my PhD is out: osf.io/preprints/ps...! π₯³
We explored the temporal dynamics of four careless responding indicators (response time, within-beep standard deviation, an inconsistency index, occasion-person correlation) in ESM data across different samples.
Thread belowπ§΅
Effects were robust across various model specifications but contrast with previous findings on ESM data quality and compliance. We discuss possible explanations and implications for study design.
With Wolfgang Viechtbauer, Oliver LΓΌdtke, Inez Myin-Germeys, Gabriel Nagy, & Steffen Nestler
Effect of higher sampling frequencies (3 vs 6 vs 9 beeps per day) became more pronounced over time. See below for attentiveness probabilities over study days and beeps.
We applied mixture modeling of response times to existing experimental ESM data & found that higher sampling frequencies led to more careless responding, while longer questionnaires did not. Co-led by Esther Ulitzsch. Accepted at Psychol Assess, preprint: doi.org/10.31234/osf.io/2s38a_v1
With an amazing team of participating experts and the ESM item repository team: Anu Hiekkaranta, Yoram Kunkels, Inez Myin-Germeys, & @oliviajkirtley.bsky.social
@esmrepository.bsky.social
What makes a good ESM item?
We asked 42 ESM/EMA experts this question in a Delphi study, and the result is the ESM-Q: A consensus-based quality assessment tool for ESM items. Now accepted at #psynomBRM. Updated preprint: osf.io/preprints/ps...
Thread below.
Im also not aware of systematic investigations of this. In our qualitative work, participants complain about repetitiveness and some say they answer in autopilot mode-> mixing up the item order may help. But as Leonie says, this could have other unwanted effects. Someone should look into this!
Well-deserved SIPS commendation for the ESM Preprocessing Gallery! Since many errors happen during preprocessing, we can't stress its importance enough! This excellent tool helps applied researchers navigate all the steps they have to take before analyzing their ESM/EMA data
