Hong Chen

🤣

Thanks for the interest! We use several models in the pipeline, including one existing model for information change from Wright et al. 2022. You can find the model in their paper. We’re will release other needed models, so stay tuned

Appreciate the deep read! great point - engagement can vary based on citations of cited paper. we have paper citation count and publication year also included in regression. one interesting finding (in the appendix!) is fidelity decreases as the citation count of the cited paper increases.

Thanks for sharing this! yeah, medicine is a major example where this effect can have real consequences. not hard to imagine how some clinical practices based on distorted or unfounded information. definitely something worth further investigation!

The Noisy Path from Source to Citation: Measuring How Scholars Engage with Past Research Academic citations are widely used for evaluating research and tracing knowledge flows. Such uses typically rely on raw citation counts and neglect variability in citation types. In particular, citati...

Thank you to @davidjurgens.bsky.social and @innovation.bsky.social for advising this project!

Check out the full paper here: arxiv.org/abs/2502.20581

Relying on intermediary sources in citations carries risks! While intermediaries serve as common tools for authors to navigate the literature, they can also introduce information loss or even misrepresentation, compounding distortions and amplifying misinformation over time.

1️⃣ Citation fidelity decreases when authors cite an intermediary source as well as the original claim.
2️⃣The fidelity of the intermediary source affects the fidelity of subsequent citations.

Do authors truly engage with what they cite? We find that exposure to others’ interpretations may influence how claims are reported, which establish a “telephone effect📞” in citations:

We find that citation fidelity is NOT random. It’s higher when:
✅ authors cite papers that are more recent and intellectually close
✅ the cited paper is open-access
✅ the first author has a lower H-index and the author team is medium-sized!

Analyzing a multi-disciplinary 42M paper dataset with full-text, we identify 13M pairs of sentences with a citation and the sentence with the corresponding claim in the original paper.

We use supervised models to measure fidelity between these two sentences.

Not all citations are equal!
They vary in fidelity – citations may paraphrase, summarize, or even misrepresent original knowledge.

How accurately do citations reflect the original research? Do authors truly engage with what they cite?

In a new study, we analyze millions of citation sentence pairs to measure citation fidelity and how it varies depending on authors’ engagement with prior literature.

arxiv.org/abs/2502.20581

⬇️