Many thanks to all co-authors: @kat-heller.bsky.social , @nenadtomasev.bsky.social , Chintan Ghate, Tiya Tiyasirichokchai, @adoubleva.bsky.social , Oluwatosin Akande, Geoffrey Siwo, Steve Adudans, Sylvanus Aitkins, Odianosen Ehiakhamen, and Eric Ndombi.
We find that including context such as risk factors and location in addition to symptoms also improves model performance. Additionally we assemble a panel of human experts to set a human expert baseline score on the dataset and to provide ratings of data quality, usefulness, etc
Results show that LLMs perform worse on TRINDs than they do on reported US-based health QAs, indicating a distribution shift in the data and the need for further optimization for global diseases. We also find that LLMs more accurately identify diseases that are common or specific.
We develop and expand the TRopical and INfectious Diseases (TRINDs) dataset to evaluate LLMs for these contexts, and demonstrate through systematic experimentation, the effect of contextual information on LLM outputs for disease classification.
Check out our latest work on Contextual Evaluation of Large Language Models for Tropical and Infectious Diseases (openreview.net/forum?id=yXe...), accepted at two NeurIPs workshops: GenAI4health (genai4health.github.io) and AIM-FM (aim-fm-24.github.io/NeurIPS/). #llms #genai4health #globalhealth
Hi! 👋🏿😊
We (+ @dr-nyamewaa.bsky.social) are excited to present two recent works detailing the landscape of AI in health in Africa from a fairness and equity angle:
The Nteasee Study arxiv.org/abs/2409.12197
and
The Case for Globalizing Fairness dl.acm.org/doi/10.1145/... (1/11)
