Amelia Zein

Transparent and comprehensive statistical reporting is critical for ensuring the credibility, reproducibility, and interpretability of psychological research. This paper offers a structured set of guidelines for reporting statistical analyses in quantitative psychology, emphasizing clarity at both the planning and results stages. Drawing on established recommendations and emerging best practices, we outline key decisions related to hypothesis formulation, sample size justification, preregistration, outlier and missing data handling, statistical model specification, and the interpretation of inferential outcomes. We address considerations across frequentist and Bayesian frameworks and fixed as well as sequential research designs, including guidance on effect size reporting, equivalence testing, and the appropriate treatment of null results. To facilitate implementation of these recommendations, we provide the Transparent Statistical Reporting in Psychology (TSRP) Checklist that researchers can use to systematically evaluate and improve their statistical reporting practices (https://osf.io/t2zpq/). In addition, we provide a curated list of freely available tools, packages, and functions that researchers can use to implement transparent reporting practices in their own analyses to bridge the gap between theory and practice. To illustrate the practical application of these principles, we provide a side-by-side comparison of insufficient versus best-practice reporting using a hypothetical cognitive psychology study. By adopting transparent reporting standards, researchers can improve the robustness of individual studies and facilitate cumulative scientific progress through more reliable meta-analyses and research syntheses.

Our paper on improving statistical reporting in psychology is now online 🎉

As a part of this paper, we also created the Transparent Statistical Reporting in Psychology checklist, which researchers can use to improve their statistical reporting practices

www.nature.com/articles/s44...

My favorite comment on the FT story

ChatGPT Harm Reduction for Writing Assignments ChatGPT Harm Reduction for Writing Assignments Because this document has escaped containment, a couple points of explanation. I wrote this for myself and a few colleagues as we work out how to handle...

Whatever strategy you’re about to suggest, we’ve thought about it. docs.google.com/document/d/1...

Historical and experimental evidence that inherent properties are overweighted in early scientific explanation

💖This paper has been ~11 years in the making - and probably my favorite project of all time. Thrilled to see it in @pnas.org! I'm so lucky that Zach decided to do a second PhD and join my lab @psychillinois.bsky.social back in 2014 - a fabulous scientist & human being! www.pnas.org/doi/10.1073/...

Lucky Coincidences: Experiencing Serendipity in Museums and Beyond Serendipity is the unintentional, accidental discovery of something new or surprising that feels positive and meaningful for the individual. Four studies (N1 = 1638; N2 = 279; N3 = 520; N4 = 452) exa...

✨ LUCKY COINCIDENCES ✨ Have you ever come across a surprising, accidental discovery that felt meaningful and motivated you to further engage with it?
In our new paper now out in JASP (doi.org/10.1111/jasp...), we explore such serendipitous experiences in museums and beyond. 1/5 🧵

Shannon's slides are always so unbelievably clear and helpful!!!

github.com/shannonpileg...

I'm having "Ohhhhh that's what that means" moments every 10 seconds here.
#positconf2025

I'm afraid we've been at it again

Doughnut academia. Adapting the “doughnut” model of economics to the academic world enables us to visualize the inner social foundations that universities should provide, and the outer human and planetary boundaries that universities need to avoid overshooting. Note that the ordering of elements within the inner and outer rings is random; there is no direct pairing between foundations and ceilings. Adapted from Raworth, 2017 under a CC-BY-SA license.

Seven ways to think like a 21st century scientist. 1. Change the goal: from a business that produces papers and graduated students, towards a university that works towards the inside space of the academic doughnut. 2. Get savvy with systems: from feeling like a cog in the university machine, towards being gardeners of our academic system. 3. See the big picture: from academics who look out over the world from their ivory tower, towards scholarship which accepts its own embeddedness in (and dependence on) society and the planet. 4. Create to regenerate: from a rat race where we tread water, towards “slow scholarship” that values community building, deep thinking and rest crucial for intellectual work. 5. Nurture human nature: from the lone genius, towards team science. 6. Design to distribute: from a funding system where the rich get richer, towards a fair distribution of opportunities and resources. 7. Be agnostic about growth: from a focus on increasing numbers of papers, citations and students, towards rebuilding trust in our own academic communities and with society.

Feeling like academia is in pretty bad shape? You're not alone.

@clarekelly.bsky.social and I previously wrote about the need to collectively rethink and reshape scientific practice: the academic doughnut. Read more at elifesciences.org/articles/84991

But, have these ideas changed anything? 👇

Models as Prediction Machines: How to Convert Confusing Coefficients into Clear Quantities Abstract Psychological researchers usually make sense of regression models by interpreting coefficient estimates directly. This works well enough for simple linear models, but is more challenging for more complex models with, for example, categorical variables, interactions, non-linearities, and hierarchical structures. Here, we introduce an alternative approach to making sense of statistical models. The central idea is to abstract away from the mechanics of estimation, and to treat models as “counterfactual prediction machines,” which are subsequently queried to estimate quantities and conduct tests that matter substantively. This workflow is model-agnostic; it can be applied in a consistent fashion to draw causal or descriptive inference from a wide range of models. We illustrate how to implement this workflow with the marginaleffects package, which supports over 100 different classes of models in R and Python, and present two worked examples. These examples show how the workflow can be applied across designs (e.g., observational study, randomized experiment) to answer different research questions (e.g., associations, causal effects, effect heterogeneity) while facing various challenges (e.g., controlling for confounders in a flexible manner, modelling ordinal outcomes, and interpreting non-linear models).

Figure illustrating model predictions. On the X-axis the predictor, annual gross income in Euro. On the Y-axis the outcome, predicted life satisfaction. A solid line marks the curve of predictions on which individual data points are marked as model-implied outcomes at incomes of interest. Comparing two such predictions gives us a comparison. We can also fit a tangent to the line of predictions, which illustrates the slope at any given point of the curve.

A figure illustrating various ways to include age as a predictor in a model. On the x-axis age (predictor), on the y-axis the outcome (model-implied importance of friends, including confidence intervals). Illustrated are 1. age as a categorical predictor, resultings in the predictions bouncing around a lot with wide confidence intervals 2. age as a linear predictor, which forces a straight line through the data points that has a very tight confidence band and 3. age splines, which lies somewhere in between as it smoothly follows the data but has more uncertainty than the straight line.

Ever stared at a table of regression coefficients & wondered what you're doing with your life?

Very excited to share this gentle introduction to another way of making sense of statistical models (w @vincentab.bsky.social)
Preprint: doi.org/10.31234/osf...
Website: j-rohrer.github.io/marginal-psy...

Copy of plaque at Peirce's birthplace in Cambridge MA with words "CHARLES SANDERS PEIRCE SCIENTIST. MATHEMATICIAN LOGICIAN, PHILOSOPHER WAS BORN IN THIS HOUSE ON SEPTEMBER 10,1839 ONE OF AMERICA'S MOST ORIGINAL AND VERSATILE INTELLECTS"

A photo of a middle aged Charles Peirce standing legs crossed with hand on hip in a 19thC suit.

Happy birthday to the #19thC geoscientist & pragmatist philosopher Charles Sanders Peirce
"Science . . . [is] a living historic
entity. . . .As such, it does not consist so much in knowing, nor even in 'organized
knowledge,' as it does in diligent inquiry into truth" CP 1.44
#philsci #philsky ⚒️

surveydown: An open-source, markdown-based platform for programmable and reproducible surveys This paper introduces the surveydown survey platform. With surveydown, researchers can create surveys that are programmable and reproducible using markdown and R code, leveraging the Quarto publicatio...

Excited to announce that our paper with @pingfanhu.bsky.social and Bogdan Bunea on our #rstats package {surveydown} is now published in @plosone.org

The paper compares the benefits of using a code-based approach to survey design, leveraging #quarto and #shiny

journals.plos.org/plosone/arti...

{truffle} is an R package for teaching users to process data.

Semi-realistic psychological datasets with predetermined effects (via `truffles_` functions) are then hidden in common data processing headaches (via `dirt_` functions) for students to clean and analyze.

mmmdata.io/posts/2025/0...

Yes, that's on point - very possible that it's related to the fact that the debate is so polarized in the US. It has been long argued that people's perceptions between sci and rel differ between countries/religious affiliations, and now we have empirical evidence for this.

Measuring How Individuals Relate Science to Religion When trying to make sense of what is going on in the world and their personal lives, people often refer to scientific and religious explanations. Based on pertinent literature, we introduce a novel...

Link to paper here: www.tandfonline.com/doi/full/10.... (not open access) but the post-print is available here: osf.io/preprints/ps...

..the more religious US participants perceived a more intense conflict between science and religion above and beyond the more secular German participants. So perhaps country-level ≠ individual-level perceptions (?)

We ran DIF analysis comparing Germans (ref) and US Americans (foc) and found an interesting pattern. We found that while a higher level of religiosity was generally associated with higher perceptions of compatibility...

This means that people most likely to agree with statements that closely match their specific preference/viewpoint, not just more compatibility perceptions = more agreement to the items.

...compartment (separate and independent domains), complementary (they fill each other's gaps), and consonance (perfectly compatible/unified). Most importantly, people's responses follow an unfolding response process (Thurstone) rather than dominance response process (Likert).

..that turns out to be *not really* a typology, but rather, categories representing different "regions" on the unidimensional, bipolar conflict-compatibility continuum. We define these regions as; conflict, context-switch (flexible depending on situation)...

In a newly published registered report by @mariogollwitzer.bsky.social, Moritz Heene, and me, we found that people perceive the relationship between science and religion differently, as described by Ian G. Barbour in his typology. We made a tool to scrutinize Barbour's typology...

"In essence, the study of social psychology is primarily an historical undertaking. We are essentially engaged in a systematic account of contemporary affairs. We utilize scientific methodology, but the results are not scientific principles in the traditional sense.."
psycnet.apa.org/record/1973-...

I remember the early days of my bachelor, Gergen's "Social Psychology as History" was the first scientific article I read. Was shoved down my throat in a socpsych intro class. I barely understood English back then, let alone psych theory. 18 yrs later, I find it super useful for my upcoming project.

Cover page for the manuscript: Morey, R. D., & Davis-Stober, C. P. (2025). On the poor statistical properties of the P-curve meta-analytic procedure. Journal of the American Statistical Association, 1–19. https://doi.org/10.1080/01621459.2025.2544397

Abstract for the paper: The P-curve (Simonsohn, Nelson, & Simmons, 2014; Simonsohn, Simmons, & Nelson, 2015) is a widely-used suite of meta-analytic tests advertised for detecting problems in sets of studies. They are based on nonparametric combinations of p values (e.g., Marden, 1985) across significant (p < .05) studies and are variously claimed to detect “evidential value”, “lack of evidential value”, and “left skew” in p values. We show that these tests do not have the properties ascribed to them. Moreover, they fail basic desiderata for tests, including admissibility and monotonicity. In light of these serious problems, we recommend against the use of the P-curve tests.

Paper drop, for anyone interested in #metascience, #statistics, or #metaanalysis! @clintin.bsky.social and I show in a new paper in JASA that the P-curve, a popular forensic meta-analysis method, has deeply undesirable statistical properties. www.tandfonline.com/doi/full/10.... 1/?

Psychologists Have Been Wrong About Death For 40 Years Smart people are sometimes the last to realize that the cognitive ship they are captaining is about to sink.

For decades, psychologists believed reminding people of death would make them fry themselves in tanning booths, recommend harsher punishments for prostitutes, support martyrdom, and spend more on luxury goods. Rigorous testing revealed: nope. But dead theories have a way of refusing to stay buried.

Victims of Conspiracies? An Examination of the Relationship Between Conspiracy Beliefs and Dispositional Individual Victimhood Conspiracy beliefs have been linked to perceptions of collective victimhood. We adopt an individual perspective on victimhood by investigating the relationship between conspiracy beliefs and the indi....

🚨 Fresh off the press 🚨 Our #TISP spin-off paper on the relationship between #ConspiracyBeliefs and individual #victimhood is now out! doi.org/10.1002/ejsp...
1/8 🧵

Today, I received a thank-you email from someone in Australia. They said our tutorial paper was exceptionally helpful and asked if I was available for a brief chat. Academia can be sometimes… brutal, but something like this is enough to keep me going🤗

🚨 New study alert! We're excited to share our "Bestiary of Questionable Research Practices in Psychology" in #AMPPS. We tackle the credibility crisis in research by defining, collecting, and categorizing QRPs using a community consensus method. 🧵#OpenScience #QRPs @psychscience.bsky.social

Can Science and Religion Coexist? | Rizqy Amelia Zein A modern, beautiful, and easily configurable blog theme for Hugo.

Over the past few weeks, I was invited to talk about a bit of my research interest at Universitas Airlangga and Universitas Indonesia. I discussed what past evidence says about how people use scientific and religious explanations. Slides are accessible here rameliaz.github.io/talk/2025-ivs/

Which Kind of Science Reform What hope is there for science reform, if we can't agree on what to reform? Right now, principles are more important than practices.

How can we reform science? I have some ideas. But I am not sure you’ll like them, because they don’t promise much. elevanth.org/blog/2025/07...

A Tale of Two Science Reform Movements Reflections from meetings of Metascience 2025 and the Society for the Improvement of Psychological Science

New post! "A Tale of Two Science Reform Movements," in which I compare the recent #Metascience2025 and #SIPS2025 conferences, and find that I am much more at home at one than the other. getsyeducated.substack.com/p/a-tale-of-...