The first preprint from the SHARE study is out! 🥳 We compared the effects of three different incentives (a bulk payment, a bulk payment with personalized feedback, and payment per beep) on data quantity, data quality, and participant experiences in a student sample.
🚨Model Checking for Vector Autoregressive Models 🚨
In a new preprint, @joranjongerling.bsky.social, @bsiepe.bsky.social, @sachaepskamp.bsky.social, Lourens Waldorp and I provide a tutorial on model checking for Vector Autoregressive (VAR) models: osf.io/preprints/ps...
Happy to share more — and more work on handling missing EMA data is on the way!
Key takeaways:
🤔 A deeper understanding of the time series is crucial before interpreting dynamic change.
💻 Data-driven model selection is limited in detecting the true form of nonstationarity.
📴 Missing EMA data remains a major barrier to reliable inference.
We then applied all candidate models to the rich 239-day EMA dataset from Kossakowski et al. and confronted the practical challenges of empirical data — especially missingness. Missing EMA data makes model fitting unstable, complicates cross-validation, and can obscure meaningful changes.
The main finding: the “correct” model is far from guaranteed to be selected. Performance depends heavily on the degree of nonstationarity, time-series length, and the selection method used.
In our two-part study, we first simulated a range of nonstationary time series and conducted model selection (information criteria, cross-validation, out-of-sample prediction) among all candidate models.
⏳Changes/nonstationarity in time series data
We’re fascinated by how dynamics change in time-series data — and how hard it is to model those changes when we don’t fully understand the underlying processes. Data-driven model selection approaches can help, but how well does it actually work?
A new publication from my PhD project on model selection for nonstationary time-series data is out, co-authored with an amazing team (Anja Franziska Ernst, @ginettelafit.bsky.social , Ward B. Eiling, @bringmannlaura.bsky.social)!
Link: doi.org/10.1111/bmsp...
🏡 Key take-home message:
Before building very large EMA-based networks, it’s often better to start smaller, build interpretable models, and keep evaluating whether the model matches the theory being tested (Hoekstra et al.: often they don’t, see osf.io/preprints/ps...).
🪜 We walk through both applications step-by-step and review existing single-case network studies to highlight typical choices in variables and timepoints.
We take two perspectives:
- ensuring sufficient power for testing individual edges, and
- ensuring good predictive accuracy of the whole network to avoid overfitting.
In this paper, we show how to plan the required length of a single-case EMA study if the goal is to estimate a reliable VAR network. These same tools can also be used to retrospectively assess the quality of previously published single-case networks based on the time-series length.
🆒 Networks! Networks?
Dynamic symptom networks hold great promise for understanding temporal processes in mental health, but their complexity (from VAR model) raises an important question: how can we ensure we are accurately recovering these dynamics? We argue it is a must to have enough timepoints.
From my master’s thesis to my first PhD project — excited to share that this work (together with @jordanrvl.bsky.social, @ginettelafit.bsky.social, Anja Franziska Ernst, Josip Razum, Eva Ceulemans, and @bringmannlaura.bsky.social) is now published in AMPPS!
Link: doi.org/10.1177/2515...
We built the openESM database:
▶️60 openly available experience sampling datasets (16K+ participants, 740K+ obs.) in one place
▶️Harmonized (meta-)data, fully open-source software
▶️Filter & search all data, simply download via R/Python
Find out more:
🌐 openesmdata.org
📝 doi.org/10.31234/osf...
ICYMI: decision to cut 5 international bachelor psychology programs in NL poses serious threat to psych science nationally & internationally, given how much research conducted here has historically punched above its weight in terms of per-capita output.
eiko-fried.com/nl-psycholog...
Image with three main boxes with recommendations for applied researchers on conceptualization, estimation, and interpretation when using network features. Conceptualization: - Discuss why a network feature should be relevant for a given treatment or outcome - Consider plausible effect size and sample size needed to detect an association with a distal outcome Estimation: - Carefully choose preprocessing and network estimation methods - Follow good practices for predictive models (e.g., cross-validation, out-of-sample validation) Interpretation: - Consider uncertainty in node selection and network feature regression - Compare network features with simpler time series features (e.g., person-specific mean or SD)
Can we use features of dynamic networks (e.g. centrality) to improve treatment selection and outcome prediction?
New preprint on the topic: We highlight the role of uncertainty & introduce a Bayesian multilevel approach for uncertainty quantification of network features 🧵
osf.io/preprints/ps...
[1/3] Cool new work by @bringmannlaura.bsky.social and colleagues on digital feedback tools in psychotherapy with m-Path.io! 📊
Link below 👇
www.sciencedirect.com/science/arti...
📣New preprint 📣
@leonieschorrlepp.bsky.social, @domimaciejewski.bsky.social, @bringmannlaura.bsky.social, Mithra Hesselink and I wrote a paper illustrating the value of qualitative methods for checking the validity of your ESM data.
doi.org/10.31219/osf...
New paper alert. And the first paper for our PhD-student Klazien 🥳🎈
Not open access but final author version available from osf.io/uwjb2 (and #ihazpdf).
In psychological test norming, nonrepresentativeness can lead to biased estimates.
1/4
The first preprint from my PhD is out: osf.io/preprints/ps...! 🥳
We explored the temporal dynamics of four careless responding indicators (response time, within-beep standard deviation, an inconsistency index, occasion-person correlation) in ESM data across different samples.
Thread below🧵
✨ Excited to share our new preprint: osf.io/preprints/os... ✨
In this paper, Nick Jacobson and I dive into the challenges researchers face when developing prediction models for just-in-time adaptive interventions (JITAI). 🧵👇
#EMA #JITAI #CrossValidation #PassiveSensing
Are psychometric networks sufficiently supported by data such that one can be confident when interpreting its results? We analysed 294 psychometric networks from 126 papers with the Bayesian approach to address this question @jmbh.bsky.social Sara Ruth van Holst @maartenmarsman.bsky.social 🧵
🚨Preprint alert! 🚨
In this one, @mieronen.bsky.social and I argue that using network psychometrics, DAGs or BNs to study predictions of the network theory of psychopathology (NT) is conceptually incoherent.
Why? NT either directly contradicts or cannot support causal sufficiency!
1/11
Excited to share that my first PhD paper got published! We explored the effects of correctly vs. failing to model true interactions in data. Can model misspecification lead to reversed conclusions? Which model generalizes better to a larger sample?
Read the article here: doi.org/10.3758/s134...
Big thanks to my wonderful coauthors @jordanrvl.bsky.social @ginettelafit.bsky.social Anja Ernst, Josip Razum, Eva Ceulemans, @bringmannlaura.bsky.social and other people that make this paper as it is today❤️
With the novel method introduced by @jordanrvl.bsky.social and team (pubmed.ncbi.nlm.nih.gov/38717682/), we assessed the risk of overfitting and having low predictive accuracy for N=1 temporal networks in published studies. Let us start to rethink when, how, and why we use the VAR-based networks.
New preprint from our team. We reflect on the usage of VAR-based person-specific temporal networks with a key question:
do we have sufficiently long time-series to estimate a VAR model without overfitting the data? Say no to models that "find meanings in random patterns".
osf.io/preprints/os...
#Rstats `MatchIt` v4.6.0 is out! `MatchIt` implements propensity score matching and other matching methods for causal effect estimation. This isn't a major release, but here are the main updates: 🧵
#causalsky #econsky #episky #statsky
