Dr Mircea Zloteanu 🌺🌞🍃

#statstab #501 Prediction intervals for GLMs

Thoughts: Sometimes the prediction of the next dat point can be [0,1]. Not very useful.

#prediction #uncertainty #predictionintervals #glm #binomial #probability

fromthebottomoftheheap.net/2017/05/01/g...

While I enjoy diatribes on these issues, I also see it as a lack of fit with empirical reality. We take for often forget just how grounded our hypotheses are. They are (most often) compatible with the world and just ask a simple tangential question. Parapsychology does not.

Great paper right? I thought I would cap 500 of my #statstab with it. I have a few of these older papers open somewhere in my browsers that really shaped how I view science and what we do.

#statstab #500🥳 How Parapsychology Could Become a Science

Churchland asks: why has parapsychology failed to become a successful science despite decades of experiments?
A: the absence of a coherent theoretical framework.

#critique #philosophy #science

thehangedman.com/teaching-fil...

This applies to "true" (prospective) pre-post designs. But (as I found out recently), you also can have retrospective pre-post (how you feel now vs x time ago) which both happen after treatment. Here the manipulation drives the pre-post change (alongside response shift), called thentests.

Maybe in 10 or 20 Y from now I will conclude that "a t-test is all I need", but for now I want to be more critical with my analyses and learn what can even be measured.

I don't disagree with any of what you said. I've even been more open to the notion that parsimony shouldn't be seen as a the "best or more natural" path. But I'm currently leaning heavily on the design school (like what Senn does) and also on relaxing assumptions (like what the ROBMA package does)

I fall into the camp of "try to best capture your data generating process". Often this means fancy models.

I also fall into the other camp of trying multiple approaches and capturing your uncertainty about assumptions.

I have a post coming soon on the benefits of "complex" models for ordinal data.

Oops. Didn't catch the link issue. Although it did look suspicious :l

I dislike the mentality of "simpler model = better communication". We can just do better to present results. Also "complex" is a loaded term. How much stats theory do you need to understand asymptotic testing?

#statstab #499 SIMPLICITY AND COMPLEXITY IN ECOLOGICAL DATA ANALYSIS

Thoughts: I dislike this paper, but I don't mind sharing views that disagree with my own. Effective communication matters.

#analysis #critique #critical #ANOVA #ttest #simplicity #communication

doi.org/10.1890/0012...

#statstab #498 FReD Annotator:

Thoughts: "Check among citations whether one of the cited studies has been replicated" cool tool.

#OpenScience #replication #integrity #metascience #reproducibility #tool

metaanalyses.shinyapps.io/replicationd...

It's odd we decided to specify the Frequentist properties of CIs, but not the other stuff.

"the 80% power t-test for a pop ES= 0.5, finds an estimate lower than at the 5% alpha decision criterion, with a 95%CI of..."

Maybe if we did it would make more (less?) sense on what these do

Sage Journals: Discover world-class research Subscription and open access journals from Sage, the world's leading independent academic publisher.

#statstab #497 On the Statistical Analysis of Experiments
With Manipulation Checks

Thoughts: All psychologists reading this title will panic. Yes, you can't just delete data and assume all is well.

#assumptions #QRPs #estimand #causalinference #ITT #ATE #bias

journals.sagepub.com/doi/pdf/10.1...

That said, with small samples, under freq, you can assume your model is correct even if your data looks "not great". Like how some still treat likert data with a t-test if the believe the DGP of the effect is normally distributed and unbounded.

Thanks! I guess I was thinking of these as visual tools. I find most frequentist assumption checks to be not fit for purpose in my field (e.g. Normality tests with small samples). I would still think a ppc plot can make you rethink.

Can you say more on this? I've not seem ppc discussed for freq in my field (but I'm sure there are probably decades-long debates on this, as everything else in stats)

Yes. I've been meaning to integrate it into my frequentist stuff. But for now brms has me covered.

I'll tinker a bit then post something. I think ppl are right and an ordinal RM ANOVA type is enough for my question. I'm just looking at "change score" differences between groups.

Posterior predictive checks — check_predictions Posterior predictive checks mean "simulating replicated data under the fitted model and then comparing these to the observed data" (Gelman and Hill, 2007, p. 158). Posterior predictive checks can be u...

#statstab #496 Posterior predictive checks {performance}

Thoughts: Idk why more frequentist don't use ppc for their models. I can diagnose so many issues visually this way.

#error #posterior #ppc #modelfit #diagnostics #model #r #rstats #easystats

easystats.github.io/performance/...

I think it is a cool paragraph. It would put at ease many non-experts on causal inference that some severity testing was done and how strong the findings are.

Would this just be an ordinal "RM ANOVA"?

fit_stack <- brm(
rating ~ time * group + (1|id),
data = dat,
family = cumulative("logit")
)

If you have other formulations or ideas for models plz share. For now I might contrasts: rank test (as Frank Harrell mentioned), RM ANOVA, multivariate latent, and maybe ancova (mostly to argue why here it might not be appropriate for pre).

Ideally I'd like to run competing models that address dif sources of error to get a sense of things. I'm using this as a learning opportunity - what can I say, and not say, with different models?

This is my first time seeing a multivariate model used this way, so even that is cool to know about.

3/3 imagine: group 1 - just fired, group 2 - still employed. Now I ask both groups "how do you feel about your QoL now" then "what about 5y ago" (randomized)? Maybe the scale interpretation for one group changes (life seems much better when I had a job) compared to group2 that is more consistent.

But, since in this study the ratings order was randomized (counterbalanced between participants), it may be that the rescaling is not happening. Idk. I think the shift in scale perceptionaspect needs to be captured here, alongside the shift itself. (but I could be wrong as I can't visualise these)

The RM-ANOVA makes a measurement invariance assumption, no? Participants use the scale the same way for both ratings.

The multivariate model permits different thresholds (Participants may reinterpret scale categories). To me this allow 1) scale recalibration, 2)reinterptation of quality of life 1/

Idk what you mean. Are you saying the multivariate model above is the same as anRM ANOVA w/ time*group?

I will take any help on this. I find it interesting that I've not come across this analysis before, even though clearly in psych you have such designs often. 🤷‍♂️