Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study

Tessa E. Hendriks; Alberto Balduzzi; Susan Van Dieren; J. Annelie Suurmeijer; Roberto Salvia; Thomas F. Stoop; Marco Del Chiaro; Sven D. Mieog; Mark Nielen; Sabino Zani; Daniel Nussbaum; Thilo Hackert; Jakob R. Izbicki; Ammar A. Javed; D. Brock Hewitt; Bas Groot Koerkamp; Roeland F. De Wilde; Yi Miao; Kuirong Jiang; Kohei Nakata; Masafumi Nakamura; Jin Young Jang; Mirang Lee; Cristina R. Ferrone; Shailesh V. Shrikhande; Vikram A. Chaudhari; Olivier R. Busch; Ajith K. Siriwardena; Oliver Strobel; Jens Werner; Bert A. Bonsing; Giovanni Marchegiani; Marc G. Besselink; Nynke Michiels; Valerie Rebattu; Fabio Casciani; Salvatore Paiella; Serena Mele; Christopher Wolfgang; Sarah Kaslow; Peter Allen; Dan Blazer; Oskar Franklin; Salvador Rodriguez Franco; Michael Kirsch; Toshitaka Sugawara; Rutger Theijse; Marie Capelle; Roel Haen; Martina Nebbia; Louisa Bolm; Zhi Ven Fong; Amit Chopde; Aditya Kunte; Kaival Gundavda; Gurudutt Varty; Naoki Ikenaga; Toshiya Abe; Zipeng Lu; Baobao Cai; Mara Götz; Faik G. Uzunoglu; Jan Bardenhagen; Fiete Gehrisch; Won Gun Yun; Youngmin Han; Savio George A.Da P. Barreto; Horacio Asbun; Charles Vollmer; Massimo Falconi; Werner Hartwig; Mustapha Adham; Abe Fingerhut; Maximillian Bockhorn; Nicholas Zyromski; Ugo Boggi; Asahi Sato; Christopher Halloran; Giovanni Butturini; Giuseppe Kito Fusai; Helmut Friess; Keith D. Lillemoe; Kevin Conlon

doi:10.1097/SLA.0000000000006473

Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study

Tessa E. Hendriks, Alberto Balduzzi, Susan Van Dieren, J. Annelie Suurmeijer, Roberto Salvia, Thomas F. Stoop, Marco Del Chiaro, Sven D. Mieog, Mark Nielen, Sabino Zani, Daniel Nussbaum, Thilo Hackert, Jakob R. Izbicki, Ammar A. Javed, D. Brock Hewitt, Bas Groot Koerkamp, Roeland F. De Wilde, Yi Miao, Kuirong Jiang, Kohei NakataMasafumi Nakamura, Jin Young Jang, Mirang Lee, Cristina R. Ferrone, Shailesh V. Shrikhande, Vikram A. Chaudhari, Olivier R. Busch, Ajith K. Siriwardena, Oliver Strobel, Jens Werner, Bert A. Bonsing, Giovanni Marchegiani, Marc G. Besselink, Nynke Michiels, Valerie Rebattu, Fabio Casciani, Salvatore Paiella, Serena Mele, Christopher Wolfgang, Sarah Kaslow, Peter Allen, Dan Blazer, Oskar Franklin, Salvador Rodriguez Franco, Michael Kirsch, Toshitaka Sugawara, Rutger Theijse, Marie Capelle, Roel Haen, Martina Nebbia, Louisa Bolm, Zhi Ven Fong, Amit Chopde, Aditya Kunte, Kaival Gundavda, Gurudutt Varty, Naoki Ikenaga, Toshiya Abe, Zipeng Lu, Baobao Cai, Mara Götz, Faik G. Uzunoglu, Jan Bardenhagen, Fiete Gehrisch, Won Gun Yun, Youngmin Han, Savio George A.Da P. Barreto, Horacio Asbun, Charles Vollmer, Massimo Falconi, Werner Hartwig, Mustapha Adham, Abe Fingerhut, Maximillian Bockhorn, Nicholas Zyromski, Ugo Boggi, Asahi Sato, Christopher Halloran, Giovanni Butturini, Giuseppe Kito Fusai, Helmut Friess, Keith D. Lillemoe, Kevin Conlon

Medicine and Health

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

Objective: To determine the interobserver variability for complications of pancreatoduodenectomy as defined by the International Study Group for Pancreatic Surgery (ISGPS) and others. Background: Good interobserver variability for the definitions of surgical complications is of major importance in comparing surgical outcomes between and within centers. However, data on interobserver variability for pancreatoduodenectomy-specific complications are lacking. Methods: International cross-sectional multicenter study including 52 raters from 13 high-volume pancreatic centers in 8 countries on 3 continents. Per center, 4 experienced raters scored 30 randomly selected patients after pancreatoduodenectomy. In addition, all raters scored 6 standardized case vignettes. This variability and the "within centers"variability were calculated for 2-fold scoring (no complication/grade A vs grade B/C) and 3-fold scoring (no complication/grade A vs grade B vs grade C) of postoperative pancreatic fistula, postpancreatoduodenectomy hemorrhage, chyle leak, bile leak, and delayed gastric emptying. Interobserver variability is presented with Gwet AC-1 measure for agreement. Results: Overall, 390 patients after pancreatoduodenectomy were included. The overall agreement rate for the standardized cases vignettes for 2-fold scoring was 68% (95% CI: 55%-81%, AC1 score: moderate agreement), and for 3-fold scoring 55% (49%-62%, AC1 score: fair agreement). The mean "within centers"agreement for 2-fold scoring was 84% (80%-87%, AC1 score; substantial agreement). Conclusions: The interobserver variability for the ISGPS-defined complications of pancreatoduodenectomy was too high even though the "within centers"agreement was acceptable. Since these findings will decrease the quality and validity of clinical studies, ISGPS has started efforts aimed at reducing the interobserver variability.

Original language	English
Pages (from-to)	728-733
Number of pages	6
Journal	Annals of Surgery
Volume	280
Issue number	5
DOIs	https://doi.org/10.1097/SLA.0000000000006473
State	Published - 1 Nov 2024

Keywords

complications
interobserver variability
ISGPS
morbidity
pancreatic surgery
pancreatoduodenectomy

Access to Document

10.1097/SLA.0000000000006473

Cite this

Hendriks, T. E., Balduzzi, A., Van Dieren, S., Suurmeijer, J. A., Salvia, R., Stoop, T. F., Del Chiaro, M., Mieog, S. D., Nielen, M., Zani, S., Nussbaum, D., Hackert, T., Izbicki, J. R., Javed, A. A., Hewitt, D. B., Koerkamp, B. G., De Wilde, R. F., Miao, Y., Jiang, K., ... Conlon, K. (2024). Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study. Annals of Surgery, 280(5), 728-733. https://doi.org/10.1097/SLA.0000000000006473

@article{e208bd633b8e42dbb71c02e44ba52611,

title = "Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study",

abstract = "Objective: To determine the interobserver variability for complications of pancreatoduodenectomy as defined by the International Study Group for Pancreatic Surgery (ISGPS) and others. Background: Good interobserver variability for the definitions of surgical complications is of major importance in comparing surgical outcomes between and within centers. However, data on interobserver variability for pancreatoduodenectomy-specific complications are lacking. Methods: International cross-sectional multicenter study including 52 raters from 13 high-volume pancreatic centers in 8 countries on 3 continents. Per center, 4 experienced raters scored 30 randomly selected patients after pancreatoduodenectomy. In addition, all raters scored 6 standardized case vignettes. This variability and the {"}within centers{"}variability were calculated for 2-fold scoring (no complication/grade A vs grade B/C) and 3-fold scoring (no complication/grade A vs grade B vs grade C) of postoperative pancreatic fistula, postpancreatoduodenectomy hemorrhage, chyle leak, bile leak, and delayed gastric emptying. Interobserver variability is presented with Gwet AC-1 measure for agreement. Results: Overall, 390 patients after pancreatoduodenectomy were included. The overall agreement rate for the standardized cases vignettes for 2-fold scoring was 68% (95% CI: 55%-81%, AC1 score: moderate agreement), and for 3-fold scoring 55% (49%-62%, AC1 score: fair agreement). The mean {"}within centers{"}agreement for 2-fold scoring was 84% (80%-87%, AC1 score; substantial agreement). Conclusions: The interobserver variability for the ISGPS-defined complications of pancreatoduodenectomy was too high even though the {"}within centers{"}agreement was acceptable. Since these findings will decrease the quality and validity of clinical studies, ISGPS has started efforts aimed at reducing the interobserver variability.",

keywords = "complications, interobserver variability, ISGPS, morbidity, pancreatic surgery, pancreatoduodenectomy",

author = "Hendriks, {Tessa E.} and Alberto Balduzzi and {Van Dieren}, Susan and Suurmeijer, {J. Annelie} and Roberto Salvia and Stoop, {Thomas F.} and {Del Chiaro}, Marco and Mieog, {Sven D.} and Mark Nielen and Sabino Zani and Daniel Nussbaum and Thilo Hackert and Izbicki, {Jakob R.} and Javed, {Ammar A.} and Hewitt, {D. Brock} and Koerkamp, {Bas Groot} and {De Wilde}, {Roeland F.} and Yi Miao and Kuirong Jiang and Kohei Nakata and Masafumi Nakamura and Jang, {Jin Young} and Mirang Lee and Ferrone, {Cristina R.} and Shrikhande, {Shailesh V.} and Chaudhari, {Vikram A.} and Busch, {Olivier R.} and Siriwardena, {Ajith K.} and Oliver Strobel and Jens Werner and Bonsing, {Bert A.} and Giovanni Marchegiani and Besselink, {Marc G.} and Nynke Michiels and Valerie Rebattu and Fabio Casciani and Salvatore Paiella and Serena Mele and Christopher Wolfgang and Sarah Kaslow and Peter Allen and Dan Blazer and Oskar Franklin and Franco, {Salvador Rodriguez} and Michael Kirsch and Toshitaka Sugawara and Rutger Theijse and Marie Capelle and Roel Haen and Martina Nebbia and Louisa Bolm and Fong, {Zhi Ven} and Amit Chopde and Aditya Kunte and Kaival Gundavda and Gurudutt Varty and Naoki Ikenaga and Toshiya Abe and Zipeng Lu and Baobao Cai and Mara G{\"o}tz and Uzunoglu, {Faik G.} and Jan Bardenhagen and Fiete Gehrisch and Yun, {Won Gun} and Youngmin Han and Barreto, {Savio George A.Da P.} and Horacio Asbun and Charles Vollmer and Massimo Falconi and Werner Hartwig and Mustapha Adham and Abe Fingerhut and Maximillian Bockhorn and Nicholas Zyromski and Ugo Boggi and Asahi Sato and Christopher Halloran and Giovanni Butturini and Fusai, {Giuseppe Kito} and Helmut Friess and Lillemoe, {Keith D.} and Kevin Conlon",

year = "2024",

month = nov,

day = "1",

doi = "10.1097/SLA.0000000000006473",

language = "English",

volume = "280",

pages = "728--733",

journal = "Annals of Surgery",

issn = "0003-4932",

publisher = "Wolters Kluwer Health",

number = "5",

}

Hendriks, TE, Balduzzi, A, Van Dieren, S, Suurmeijer, JA, Salvia, R, Stoop, TF, Del Chiaro, M, Mieog, SD, Nielen, M, Zani, S, Nussbaum, D, Hackert, T, Izbicki, JR, Javed, AA, Hewitt, DB, Koerkamp, BG, De Wilde, RF, Miao, Y, Jiang, K, Nakata, K, Nakamura, M, Jang, JY, Lee, M, Ferrone, CR, Shrikhande, SV, Chaudhari, VA, Busch, OR, Siriwardena, AK, Strobel, O, Werner, J, Bonsing, BA, Marchegiani, G, Besselink, MG, Michiels, N, Rebattu, V, Casciani, F, Paiella, S, Mele, S, Wolfgang, C, Kaslow, S, Allen, P, Blazer, D, Franklin, O, Franco, SR, Kirsch, M, Sugawara, T, Theijse, R, Capelle, M, Haen, R, Nebbia, M, Bolm, L, Fong, ZV, Chopde, A, Kunte, A, Gundavda, K, Varty, G, Ikenaga, N, Abe, T, Lu, Z, Cai, B, Götz, M, Uzunoglu, FG, Bardenhagen, J, Gehrisch, F, Yun, WG, Han, Y, Barreto, SGADP, Asbun, H, Vollmer, C, Falconi, M, Hartwig, W, Adham, M, Fingerhut, A, Bockhorn, M, Zyromski, N, Boggi, U, Sato, A, Halloran, C, Butturini, G, Fusai, GK, Friess, H, Lillemoe, KD & Conlon, K 2024, 'Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study', Annals of Surgery, vol. 280, no. 5, pp. 728-733. https://doi.org/10.1097/SLA.0000000000006473

Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study. / Hendriks, Tessa E.; Balduzzi, Alberto; Van Dieren, Susan et al.
In: Annals of Surgery, Vol. 280, No. 5, 01.11.2024, p. 728-733.

Research output: Contribution to journal › Article › peer-review

TY - JOUR

T1 - Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy

T2 - An International Cross-Sectional Multicenter Study

AU - Hendriks, Tessa E.

AU - Balduzzi, Alberto

AU - Van Dieren, Susan

AU - Suurmeijer, J. Annelie

AU - Salvia, Roberto

AU - Stoop, Thomas F.

AU - Del Chiaro, Marco

AU - Mieog, Sven D.

AU - Nielen, Mark

AU - Zani, Sabino

AU - Nussbaum, Daniel

AU - Hackert, Thilo

AU - Izbicki, Jakob R.

AU - Javed, Ammar A.

AU - Hewitt, D. Brock

AU - Koerkamp, Bas Groot

AU - De Wilde, Roeland F.

AU - Miao, Yi

AU - Jiang, Kuirong

AU - Nakata, Kohei

AU - Nakamura, Masafumi

AU - Jang, Jin Young

AU - Lee, Mirang

AU - Ferrone, Cristina R.

AU - Shrikhande, Shailesh V.

AU - Chaudhari, Vikram A.

AU - Busch, Olivier R.

AU - Siriwardena, Ajith K.

AU - Strobel, Oliver

AU - Werner, Jens

AU - Bonsing, Bert A.

AU - Marchegiani, Giovanni

AU - Besselink, Marc G.

AU - Michiels, Nynke

AU - Rebattu, Valerie

AU - Casciani, Fabio

AU - Paiella, Salvatore

AU - Mele, Serena

AU - Wolfgang, Christopher

AU - Kaslow, Sarah

AU - Allen, Peter

AU - Blazer, Dan

AU - Franklin, Oskar

AU - Franco, Salvador Rodriguez

AU - Kirsch, Michael

AU - Sugawara, Toshitaka

AU - Theijse, Rutger

AU - Capelle, Marie

AU - Haen, Roel

AU - Nebbia, Martina

AU - Bolm, Louisa

AU - Fong, Zhi Ven

AU - Chopde, Amit

AU - Kunte, Aditya

AU - Gundavda, Kaival

AU - Varty, Gurudutt

AU - Ikenaga, Naoki

AU - Abe, Toshiya

AU - Lu, Zipeng

AU - Cai, Baobao

AU - Götz, Mara

AU - Uzunoglu, Faik G.

AU - Bardenhagen, Jan

AU - Gehrisch, Fiete

AU - Yun, Won Gun

AU - Han, Youngmin

AU - Barreto, Savio George A.Da P.

AU - Asbun, Horacio

AU - Vollmer, Charles

AU - Falconi, Massimo

AU - Hartwig, Werner

AU - Adham, Mustapha

AU - Fingerhut, Abe

AU - Bockhorn, Maximillian

AU - Zyromski, Nicholas

AU - Boggi, Ugo

AU - Sato, Asahi

AU - Halloran, Christopher

AU - Butturini, Giovanni

AU - Fusai, Giuseppe Kito

AU - Friess, Helmut

AU - Lillemoe, Keith D.

AU - Conlon, Kevin

PY - 2024/11/1

Y1 - 2024/11/1

N2 - Objective: To determine the interobserver variability for complications of pancreatoduodenectomy as defined by the International Study Group for Pancreatic Surgery (ISGPS) and others. Background: Good interobserver variability for the definitions of surgical complications is of major importance in comparing surgical outcomes between and within centers. However, data on interobserver variability for pancreatoduodenectomy-specific complications are lacking. Methods: International cross-sectional multicenter study including 52 raters from 13 high-volume pancreatic centers in 8 countries on 3 continents. Per center, 4 experienced raters scored 30 randomly selected patients after pancreatoduodenectomy. In addition, all raters scored 6 standardized case vignettes. This variability and the "within centers"variability were calculated for 2-fold scoring (no complication/grade A vs grade B/C) and 3-fold scoring (no complication/grade A vs grade B vs grade C) of postoperative pancreatic fistula, postpancreatoduodenectomy hemorrhage, chyle leak, bile leak, and delayed gastric emptying. Interobserver variability is presented with Gwet AC-1 measure for agreement. Results: Overall, 390 patients after pancreatoduodenectomy were included. The overall agreement rate for the standardized cases vignettes for 2-fold scoring was 68% (95% CI: 55%-81%, AC1 score: moderate agreement), and for 3-fold scoring 55% (49%-62%, AC1 score: fair agreement). The mean "within centers"agreement for 2-fold scoring was 84% (80%-87%, AC1 score; substantial agreement). Conclusions: The interobserver variability for the ISGPS-defined complications of pancreatoduodenectomy was too high even though the "within centers"agreement was acceptable. Since these findings will decrease the quality and validity of clinical studies, ISGPS has started efforts aimed at reducing the interobserver variability.

AB - Objective: To determine the interobserver variability for complications of pancreatoduodenectomy as defined by the International Study Group for Pancreatic Surgery (ISGPS) and others. Background: Good interobserver variability for the definitions of surgical complications is of major importance in comparing surgical outcomes between and within centers. However, data on interobserver variability for pancreatoduodenectomy-specific complications are lacking. Methods: International cross-sectional multicenter study including 52 raters from 13 high-volume pancreatic centers in 8 countries on 3 continents. Per center, 4 experienced raters scored 30 randomly selected patients after pancreatoduodenectomy. In addition, all raters scored 6 standardized case vignettes. This variability and the "within centers"variability were calculated for 2-fold scoring (no complication/grade A vs grade B/C) and 3-fold scoring (no complication/grade A vs grade B vs grade C) of postoperative pancreatic fistula, postpancreatoduodenectomy hemorrhage, chyle leak, bile leak, and delayed gastric emptying. Interobserver variability is presented with Gwet AC-1 measure for agreement. Results: Overall, 390 patients after pancreatoduodenectomy were included. The overall agreement rate for the standardized cases vignettes for 2-fold scoring was 68% (95% CI: 55%-81%, AC1 score: moderate agreement), and for 3-fold scoring 55% (49%-62%, AC1 score: fair agreement). The mean "within centers"agreement for 2-fold scoring was 84% (80%-87%, AC1 score; substantial agreement). Conclusions: The interobserver variability for the ISGPS-defined complications of pancreatoduodenectomy was too high even though the "within centers"agreement was acceptable. Since these findings will decrease the quality and validity of clinical studies, ISGPS has started efforts aimed at reducing the interobserver variability.

KW - complications

KW - interobserver variability

KW - ISGPS

KW - morbidity

KW - pancreatic surgery

KW - pancreatoduodenectomy

UR - http://www.scopus.com/inward/record.url?scp=85201083071&partnerID=8YFLogxK

U2 - 10.1097/SLA.0000000000006473

DO - 10.1097/SLA.0000000000006473

M3 - Article

C2 - 39087327

AN - SCOPUS:85201083071

SN - 0003-4932

VL - 280

SP - 728

EP - 733

JO - Annals of Surgery

JF - Annals of Surgery

IS - 5

ER -

Interobserver Variability in the International Study Group for Pancreatic Surgery (ISGPS)-Defined Complications after Pancreatoduodenectomy: An International Cross-Sectional Multicenter Study

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this