Skip to main content

Table 1 Reviewed methods and their total scores

From: A survey on artificial intelligence assurance

Year First author's last name and citation Publishing venue AI subarea Total score
2020 D'Alterio [50] FUZZ-IEEE XAI 10
2019 Tao [208] IEEE Access Generic 10
2020 Anderson [11] ACM TIIS RL 9
2020 Birkenbihl [29] EPMA ML 9
2020 Checco [39] JAIR DS 9
2020 Chen [40] IEEE Access XAI 9
2020 Cluzeau [43] EASA DL 9
2019 Kaur [109] WAINA XAI 9
2020 Kulkarni [117] Academic Press DS 9
2020 Kuppa [118] IEEE IJCNN XAI 9
2020 Kuzlu [120] IEEE Access XAI 9
2021 Massoli [145] CVIU DL 9
2020 Spinner [201] IEEE TVCG XAI 9
2016 Veeramachaneni [226] IEEE HPSC DS 9
2018 Wei [230] AS RL 9
2020 Winkel [236] EJR RL 9
2014 Ali [8] GISci DS 8
2018 Alves [9] NASA ARIAS ABS 8
2019 Batarseh [24] EDML DS 8
2016 Gao [71] SEKE DS 8
2020 Gardiner [72] Nature Sci Rep ML 8
2016 Gulshan [81] JAMA CV 8
2020 Guo [82] IEEE ICCVW XAI 8
2020 Han [87] IET JoE XAI 8
2016 Heaney [93] OD GA 8
2019 Huber [97] KI AAI RL 8
2019 Keneni [112] IEEE Access XAI 8
2020 Kohlbrenner [116] IEEE IJCNN XAI 8
2019 Maloca [134] PLoS ONE DL 8
2020 Malolan [136] IEEE ICICT XAI 8
2020 Payrovnaziri [168] JAMIA ML 8
2008 Peppler [170] OASJ DS 8
2020 Sequeira [196] SciDir AI RL 8
2020 Sivamani [199] IEEE LCS DL 8
2020 Tan [207] IEEE IJCNN XAI 8
2020 Tao [209] IEEE CoG XAI 8
2020 Welch [231] PhysMedBiol DL 8
2020 Xiao [239] IS DL 8
2016 Aitken [6] UC ABS 7
2019 Barredo-Arrieta [19] IEEE ITSC XAI 7
2013 Batarseh [20] IEEE TSMCS KBS 7
2001 Berndt [27] COMP DS 7
2010 Bone [30] CEUS RL 7
2016 Celis [38] PrePrint ML 7
2019 Chittajallu [42] IEEE ISBI XAI 7
2018 Elsayed [67] NIPS CV 7
2019 Ferreyra [69] FUZZ-IEEE XAI 7
2006 Forster [70] Uni of South Africa AGI 7
1985 Ginsberg [74] IJCAI KBS 7
2018 Go [76] ACM CCS DL 7
2020 Halliwell [86] PrePrint DL 7
2015 He [90] MPE GA 7
2020 Heuer [94] ACM UMAP ML 7
2016 Jiang [102] PMLR RL 7
2020 Kaur [111] AINA XAI 7
2016 Kianifar [113] SC GA 7
2019 Lee [121] IEEE ICTC XAI 7
2017 Liang [126] MILCOM DS 7
2020 Mackowiak [132] PrePrint CV 7
2018 Mason [143] AHIM RL 7
2018 Murray [157] FUZZ-IEEE XAI 7
2019 Naqa [66] MedPhys ML 7
2019 Prentzas [174] IEEE BIBE XAI 7
2018 Pynadath [177] Springer HCIS ML 7
2020 Ragot [179] CHI ML 7
2020 Rotman [184] PrePrint RL 7
2015 Rovcanin [185] WN RL 7
2020 Sarathy [188] IEEE SISY XAI 7
2018 Stock [203] ECCV CV 7
2009 Tadj [206] SCI KBS 7
1999 Thomas [214] AAAI GA 7
2020 Uslu [220] AINA XAI 7
2018 Xu [240] PrePrint DL 7
2019 Bellamy [26] IBM JRD XAI 6
2019 Beyret [28] IEEE IROS RL 6
2018 Cao [35] JAIHC ML 6
2020 Cruz [47] PrePrint RL 6
2001 Halkidi [85] JIIS ML 6
2020 He [91, 92] PrePrint RL 6
2020 Islam [98] IEEE TFS XAI 6
2005 Liu [128] AI2005 DL 6
2019 Madumal [133] PrePrint RL 6
1996 Martin [138] ERIC DS 6
2007 Martín-Guerrero [141] AJCAI RL 6
2000 Mosqueira-Rey [155] ESA KBS 6
2020 Mynuddin [160] IETITS RL 6
2020 Puiutta [175] CD-MAKE RL 6
2018 Ruan [187] IJCAI DL 6
2019 Schlegel [193] IEEE ICCVW XAI 6
2020 Toreini [216, 217] ACM FAT ML 6
2020 Toreini [216, 217] PrePrint ML 6
2019 Vabalas [222] PLoS ONE ML 6
2010 Winkler [237] IEEE SUTC CV 6
2002 Wu [238] IJHCS KBS 6
2019 Zhu [246] ACM PLDI RL 6
1992 Andert [12] IJM KBS 5
2018 Antunes [14] IEEE DSN-W ML 5
1989 Becker [25] NASA KBS 5
2019 Chen [41] CS RL 5
2019 Cruz [46] AI 2019 AAI RL 5
2020 Diallo [57] IEEE ACSOS-C XAI 5
2010 Dong [62] IEEE ICWIIAT GA 5
2019 Dupuis [64] UoG XAI 5
2015 Goodfellow [79] PrePrint ML 5
2020 Guo [82] IEEE CM XAI 5
2020 Haverinen [89] Uni of Jyväskylä XAI 5
1997 Jones [104] JMB GA 5
2019 Joo [106] IEEE CoG RL 5
2020 Katell [107] ACM FAT XAI 5
2007 Knauf [115] IEEE TSMC KBS 5
1995 Lockwood [130] AES KBS 5
2000 Marcos [137] IEE Proc KBS 5
2017 Mason [144] WhiteRose RL 5
1988 Morell [154] IEA/AIE KBS 5
2020 Murray [158] IEEE TETCI XAI 5
2010 Niazi [162] SpringSim ABS 5
2000 Onoyama [166] JETAI KBS 5
2019 Ren [182] PrePrint DL 5
2013 Sargent [189] JoS ABS 5
2003 Schumann [195] EANN DL 5
1995 Singer [198] POQ DS 5
2019 Srivastava [202] AAAI AIES NLP 5
2006 Taylor [211] Springer DL 5
2020 Taylor [213] IEEE CVPRW XAI 5
2020 Tjoa [215] IEEE TNNLS ML 5
2020 Uslu [221] BWCCA XAI 5
2020 Varshney [225] IEEE CISS ML 5
2018 Volz [228] IEEE CIG XAI 5
2020 Wieringa [234] ACM FAT XAI 5
2020 Wing [235] PrePrint ML 5
2019 Yoon [242] IEEE ICCVW XAI 5
2019 Zhou [245] IJCAI XAI ML 5
1994 Zlatareva [249] ESA KBS 5
2018 AI Now (Algorithmic Accountability Policy Tooklit) [7] AI Now XAI 4
2015 Arifin [15] Springer ABS 4
2015 Batarseh [21] AIR KBS 4
2007 Brancovici [31] IEEE CEC XAI 4
1987 Castore [37] NASA STI KBS 4
2013 Cohen [45] EternalS NLP 4
2020 Das [51] PrePrint XAI 4
2013 David [52] UCS ABS 4
2018 Došilović [63] MIPRO ML 4
2000 Edwards [65] Oxford DS 4
2018 EY (Assurance in the Age of AI) [17] EY ML 4
2019 Guidotti [80] ACM CS XAI 4
2018 Jilk [103] PrePrint ABS 4
2017 Leibovici [123] ISPRS Int J. Geo-Inf DS 4
2020 Li [125] IEEE TKDE XAI 4
2019 Mehrabi [146] PrePrint ML 4
2019 Meskauskas [150] FUZZ-IEEE XAI 4
1998 Miller [151] MS GA 4
2019 Nassar [161] WIREs DMKD XAI 4
1992 Preece [173] ESA KBS 4
2019 Qiu [178] AS Generic 4
1984 Sargent [190] IEEE WSC ABS 4
2003 Taylor [212] SPIE DL 4
1999 Tsai [218] IEEE TKDE KBS 4
1991 Vinze [227] IM KBS 4
2019 Wang [229] ACM CHI XAI 4
1993 Wells [232] AAAI KBS 4
2018 Zhu [247] IEEE CIG XAI 4
1998 Zlatareva [248] DBLP KBS 4
2018 Abdollahi [1] Springer ML 3
1997 Abel [2] FLAIRS Conference KBS 3
2018 Adadi [4] IEEE Access XAI 3
2018 Agarwal [5] PrePrint Generic 3
2016 Amodei [10] PrePrint ML 3
2019 Breck [32] SysML ML 3
1996 Carley [36] CASOS KBS 3
2000 Coenen [44] CUP KBS 3
1987 Culbert [48] NASA SOAR KBS 3
2020 Dağlarli [49] ADL XAI 3
1992 Davis [53] RAND ABS 3
2020 Dodge [61] ExSS-ATEC XAI 3
2018 Everitt [68] IJCAI AGI 3
1991 Gilstrap [73] TI KBS 3
2019 Glomsrud [75] ISSAV XAI 3
1996 Gonzalez [78] EAAI KBS 3
1997 Harmelen [88] EUROVAV KBS 3
2019 He [91, 92] PrePrint DL 3
2020 Heuillet [95] PrePrint RL 3
2009 Hibbard [96] AGI AGI 3
2019 Israelsen [99] ACM CSUR Generic 3
2019 Jha [101] NeurIPS DL 3
2002 Knauf [114] IEEE TSMC KBS 3
2017 de Laat [54] PhilosTechnol ML 3
1994 Lee [122] IEEE TSMC KBS 3
2004 Liu [127] IEEE MLC ABS 3
1997 Lowry [131] ISMIS Generic 3
2012 Martinez-Balleste [139] IEEE SIPC CV 3
2020 Martinez-Fernandez [140] PrePrint XAI 3
2017 Mason [142] DCAART RL 3
1993 Mengshoel [148] IEEE exp KBS 3
2005 Menzies [149] AC Generic 3
2007 Min [152] WSC KBS 3
1997 Murrell [159] DSS KBS 3
1987 O'Keefe [164] IEEE exp KBS 3
2020 Putzer [176] PrePrint XAI 3
1991 De Raedt [55] JWS KBS 3
2020 Raji [180] ACM FAT XAI 3
2004 Sargent [191] IEEE WSC ABS 3
1990 Suen [204] ESA KBS 3
2019 Sun [205] IEEE VTC XAI 3
2006 Yilmaz [241] CMOT ABS 3
1997 Zaidi [243] Automatica KBS 3
1996 Abel [3] FLAIRS Conference KBS 2
2016 Aitken [6] PrePrint ABS 2
1998 Antoniou [13] AI Magazine KBS 2
2019 Arrieta [16] SciDir IF XAI 2
2018 Bride [34] ICFEM XAI 2
2020 Dghaym [56] AU SSAV XAI 2
2015 Dobson [59] JCLS ML 2
2018 Hagras [83] IEEE Comp XAI 2
1999 Hailu [84] IEEE SMC RL 2
2020 He [91, 92] IEEE IRCE XAI 2
2016 Janssen [100] GIQ DS 2
2020 Kaur [110] NBiS XAI 2
2008 Liu [129] IEEE SSSC ABS 2
2006 Min [153] ICMLC KBS 2
2019 Mueller [156] PrePrint XAI 2
1996 Nourani [163] ACM SIGSOFT Generic 2
2020 Pawar [167] IEEE CyberSA XAI 2
2009 Pèpe [169] JCG GA 2
2013 Pitchforth [171] ESA DL 2
2017 Protiviti (Validation of Machine Learning Models) [223] Protiviti ML 2
2010 Sargent [192] WSC ABS 2
2019 Spada [200] AIAI XAI 2
2005 Taylor [210] IEEE IJCNN DL 2
2016 Zeigler [244] JDMS ABS 2
2001 Barr [18] ACL NLP 1
2020 Brennen [33] ACM CHI EA XAI 1
2006 Dibie-Barthélemy [58] KBS KBS 1
2020 European Commission (A European Approach to Excellence and Trust) [165] European Commission XAI 1
2000 Gonzalez [77] JETAI Generic 1
2018 Kaul [108] ACM AIES ML 1
2003 Kurd [119] SAFECOMP DL 1
2017 Lepri [124] PhilosTechnology ML 1
2018 Mehri [147] ACM ARES DL 1
2019 Pocius [172] AAAI-19 RL 1
2019 Rossi [183] JIA XAI 1
2010 Schumann [194] NASA SCI DL 1
2018 Sileno [197] PrePrint XAI 1
2019 Varshney [224] ACM XRDS ML 1
2016 Wickramage [233] FTC DS 1