Your letter very clearly reads as saying that there are bots, and you use that to question the integrity of online sampling... but you agree with the notion that your measures are not sufficiently establishing to make such a claim. Do you not think an amendment to your published letter is required?
Recently, van der Stigchel and colleagues posted a provocative commentary suggesting that we should be wary of bots in online behavioral data collection (🧵by @cstrauch.bsky.social here: bsky.app/profile/cstr...). But should we? Here is my response letter osf.io/preprints/ps.... 1/5
On Thursday I'll be taking part in a roundtable “How real is the LLM threat to online research in academia?” for @joinprolific.bsky.social alongside @davmicrot.bsky.social, Michael Nicholas Stagnaro, and Raluca Rilla.
Sign up here: lnkd.in/em8-MpjN
Interval strongly predicted retention: 1-week → 80% completed all sessions; 4-week → 50%.
Payment had no significant effect.
Participants higher in routine showed better retention; those higher in automaticity showed worse
Read the paper here: osf.io/preprints/ps...
What drives retention in online longitudinal research? We conducted an experiment (N=1,798) on @joinprolific.bsky.social, orthogonally manipulating payment rate (£6–£9/hr + bonus) and session interval (1, 2, or 4 weeks) across five sessions. The findings challenge some common assumptions 👇
Hey @rory-stewart.bsky.social @alastaircampbell2.bsky.social @therestpolitics.bsky.social I asked your favourite question to 1,936 US Adults "Who do you believe is the biggest threat to global order and security?"
33.8% of Americans rate the US as the biggest threat.... 🤯
@michaeljkane.bsky.social pop me over a message and will do what I can to help!
Second, and most importantly, the economics of it don't make sense, even at $0.05/response. For someone to scale this approach would require multiple user accounts (which we have robust guards against), meaning the break-even point for a bad actor is likely impossible to reach
First, the barrier to entry here is really high. Creating a bot like this is not trivial, and required an academic team to design and implement. Current 'naive' agents such as that offered by ChatGPT are simple to catch - track mouse moves, typing speed, or even simple reverse shibboleths
Lots of chatter about this paper currently. Its a stark warning, but at present I see this as a stark warning of what might come, not what is happening now. As a research community we need to see it as a call-to-arms to develop new strategies, NOT a call to abandon online sampling. Reasoning below
Agreed, its a stark warning, and should be a call-to-arms in terms of the community finding ways to detect such bots. Keen to work together with anyone who's interested to figure that out
Not to diminish this at all - I think its a critical warning of what is coming and therefore something all researchers need to start thinking about how to protect against. But its highly unlikely to be a current threat. At least on panels with safeguards in place
Remember that (a) the barrier to entry here is high (not simple to make a bot like this), and (b) the economics of it don't make a lot of sense, even at $0.05/response, to scale it would require multiple user accounts (which we have guards against) the break-even point is likely impossible to reach
Physical touch might be one of our best behavioural indicators of social connection. Unlike loneliness surveys, you can't be hugged without another person choosing to embrace you...
Full data: www.prolific.com/polling
The patterns are stark:
Men: 21.1% vs Women: 12.6%
African Americans: 24.9% (highest)
White Americans: 15.2%
Political independents: 18.7%
Republicans: 15.1%
1 in 6 Americans can't remember the last time someone hugged them.
@joinprolific.bsky.social surveyed 1,917 US adults: 24.6% were hugged today, 31.7% in the past few days. But 16.7% couldn't remember, it had been so long, the memory was gone. 🧵
There's not just a generational wealth gap anymore. There's a wellbeing gap too.
And right now, it's hitting younger people the hardest.
(Data from our latest @joinprolific.bsky.social poll - November 2024)
Full data: www.prolific.com/polling
Millennials/older Gen Z are carrying the heaviest load:
- Student loans
- Locked out of housing market
- Childcare costs through the roof
- Job market being disrupted by AI
Financial crisis → Pandemic → Cost-of-living crisis
25-34 year-olds: Just 23% doing well (29% actively struggling)
35-54 year-olds: Stuck around 31% for two decades
65+ year-olds: 48% doing well
That's more than double their younger counterparts.
We asked 1,917 Americans: "Are you OK?"
Only 38% said yes.
41% said things are difficult. 21% are actively struggling.
But the age breakdown is wild 🧵
AI will soon expose that most published research is mediocre, irrelevant (or simply bullsh*t). Most scientists know that, but the public doesn't... What's it going to do for trust in science when they find out - how do we plan for it? 🤔
Great article by @naomioreskes.bsky.social
shorturl.at/i2Hn5
What is one question you'd dearly love to ask a representative sample of 2,000 US adults?
Going to try this again because it was super interesting last time. I'll stick the best question in our next @joinprolific.bsky.social poll that goes live at the start of November.
#polling #research
Plot twist: What if AI's best future might not be the insanely profitable one 🤔
Great article arguing that instead of trillion-dollar empires, we could get free, open-source models that are just "good enough" for most people. Would that be so bad?
theconversation.com/generative-a...
#AI #tech
The Double Squeeze hitting Trump's base.
9 months of @joinprolific.bsky.social tracking show voters report surging costs AND growing fears of US global decline.
They voted for prosperity & strength. They got strain & decline.
Full analysis: www.prolific.com/resources/tr...
#poll #geopolitics
Super excited to share this work with the world!
We wanted to put human preference at the heart of our AI model leaderboard, and to do so in a principled and representative manner.
The result: HUMAINE
Check out the data here and get in touch with any questions: huggingface.co/spaces/Proli...
Bodies like the AISI get this right by combining industry knowledge with academic independence. We need more of that, not less, especially when the stakes are so high to. We're essentially in a winner-takes-all race for AGI, but we shouldn't sacrifice public trust to get there faster. [3/3]
The current approach is backwards. We let companies lead on policy while academics and researchers get sidelined. Meanwhile, Trump's deregulatory push is creating a free-for-all that prioritises market cap over public interest. Hybrid governance models are the only sensible path forward. [2/3]
Sharing a piece I wrote for The AI Journal on who should govern AI: aijourn.com/a-fine-balan...
Our recent @joinprolific.bsky.social polling shows 69.7% of people think AI investment will primarily benefit corporations not the public. They're probably right. [1/3]
#AI #AIGovernance #TechPolicy
📊 Our new @joinprolific.bsky.social AI User Experience Leaderboard is live!
AI systems ranked by real human experience, not just technical metrics. Using Census-based sampling + MRP for results that represent the general public.
Check it out here: huggingface.co/spaces/nlpet...
bsky.app/profile/andr...
Affecting even the Republican base
