TY - JOUR
T1 - Understanding effect size
T2 - an international online survey among psychiatrists, psychologists, physicians from other medical specialities, dentists and other health professionals
AU - Heimke, Ferdinand
AU - Furukawa, Yuki
AU - Siafis, Spyridon
AU - Johnston, Bradley C.
AU - Engel, Rolf R.
AU - Furukawa, Toshi A.
AU - Leucht, Stefan
N1 - Publisher Copyright:
© Author(s) (or their employer(s)) 2024.
PY - 2024/1
Y1 - 2024/1
N2 - Background and objective Various ways exist to display the effectiveness of medical treatment options. This study examined various psychiatric, medical and allied professionals’ understanding and perceived usefulness of eight effect size indices for presenting both dichotomous and continuous outcome data. Methods We surveyed 1316 participants from 13 countries using an online questionnaire. We presented hypothetical treatment effects of interventions versus placebo concerning chronic pain using eight different effect size measures. For each index, the participants had to judge the magnitude of the shown effect, to indicate how certain they felt about their own answer and how useful they found the given effect size index. Findings Overall, 762 (57.9%) participants fully completed the questionnaire. In terms of understanding, the best results emerged when both the control event rate (CER) and the experimental event rate (EER) were presented. The difference in minimal importance difference units (MID unit) was understood worst. Respondents also found CER and EER to be the most useful presentation approach while they rated MID unit as the least useful. Confidence in the risk ratio ranked high, even though it was rather poorly understood. Conclusions and clinical implications For dichotomous outcomes, presenting the effects in terms of the CER and EER could lead to the most correct interpretation. Relative measures including the risk ratio must be supplemented with absolute measures such as the CER and EER. Effects on continuous outcomes were better understood through standardised mean differences than mean differences. These can also be supplemented by dichotomised CER and EER.
AB - Background and objective Various ways exist to display the effectiveness of medical treatment options. This study examined various psychiatric, medical and allied professionals’ understanding and perceived usefulness of eight effect size indices for presenting both dichotomous and continuous outcome data. Methods We surveyed 1316 participants from 13 countries using an online questionnaire. We presented hypothetical treatment effects of interventions versus placebo concerning chronic pain using eight different effect size measures. For each index, the participants had to judge the magnitude of the shown effect, to indicate how certain they felt about their own answer and how useful they found the given effect size index. Findings Overall, 762 (57.9%) participants fully completed the questionnaire. In terms of understanding, the best results emerged when both the control event rate (CER) and the experimental event rate (EER) were presented. The difference in minimal importance difference units (MID unit) was understood worst. Respondents also found CER and EER to be the most useful presentation approach while they rated MID unit as the least useful. Confidence in the risk ratio ranked high, even though it was rather poorly understood. Conclusions and clinical implications For dichotomous outcomes, presenting the effects in terms of the CER and EER could lead to the most correct interpretation. Relative measures including the risk ratio must be supplemented with absolute measures such as the CER and EER. Effects on continuous outcomes were better understood through standardised mean differences than mean differences. These can also be supplemented by dichotomised CER and EER.
UR - http://www.scopus.com/inward/record.url?scp=85185900796&partnerID=8YFLogxK
U2 - 10.1136/bmjment-2023-300978
DO - 10.1136/bmjment-2023-300978
M3 - Article
C2 - 38388002
AN - SCOPUS:85185900796
SN - 2755-9734
VL - 27
JO - BMJ mental health
JF - BMJ mental health
IS - 1
ER -