OpenAI releases MentalHealthBench: a test of 1,215 conversations for checking how SI responds in crisis conversations
Prepared by Sintelligent with the help of SI. Publication time restored from the build log. . Updated: .
A common exam is emerging for SI in the most sensitive conversations; the quality of help can be compared rather than taken on faith.
The open benchmark consists of 1,215 synthetic dialogues and 5,262 evaluation criteria written by more than 80 psychologists and psychiatrists from 22 countries in 19 languages. The dialogues are divided into routine (53.5%), high-acuity (18.2%) and emergency (28.3%); the interlocutors include adults, teenagers, clinicians and caregivers. It evaluates clinical accuracy, the ability to clarify context, respect for a person's autonomy, empathy, recognition of urgency and avoidance of harm. OpenAI is publishing the methodology so that others can repeat and extend the checks.
Sources 2