All news

SI screened nearly 4,000 medical papers: only one in eight reported publishing its code

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

A journal's stated policy does not by itself guarantee code availability: the share of publications with code ranges from zero to more than 70%.

Researchers from MIT and Dana-Farber used large language models to assess how code publication is handled in clinical prediction models. The algorithms extracted code availability statements from nearly 4,000 articles in the open PubMed Central collection. The researchers then went through hundreds of linked repositories, assessing documentation, dependencies and reuse terms. Only about one in eight studies reported publishing its code. By journal, the share ranged from zero to more than 70%, and low rates appeared even at publications with a formal disclosure policy. The results were published in Nature Medicine and are being used in developing the TRIPOD-Code recommendations. The quality of the automated labeling was checked by people: two researchers independently reviewed 500 articles and a separate sample of repositories. Agreement was higher for concrete features (such as the presence of a license) than for assessments of documentation sufficiency. The stance of journals with a formal code disclosure policy was not found in the sources checked.

Sources 1