The effects of class rarity on the evaluation of supervised healthcare fraud detection models

Herland, Matthew; Bauder, Richard A.; Khoshgoftaar, Taghi M.

doi:10.1186/s40537-019-0181-8

Journal of Big Data

Table 11 Tukey’s HSD: ratio

From: The effects of class rarity on the evaluation of supervised healthcare fraud detection models

Part B				Part D				DMEPOS				Combined
Pos count	Ratio	AUC	Group	Pos count	Ratio	AUC	Group	Pos count	Ratio	AUC	Group	Pos count	Ratio	AUC	Group
(a) Train_Test
All	[10:90]	0.80998	a	All	[10:90]	0.78501	a	All	[10:90]	0.78675	a	All	[10:90]	0.84661	a
All	[25:75]	0.80945	a	400	[10:90]	0.75851	a	400	[1:99]	0.77359	a	200	[1:99]	0.81010	a
1000	[10:90]	0.80927	a	200	[10:90]	0.72831	a	400	[10:90]	0.77152	a	100	[1:99]	0.76000	a
All	[35:65]	0.80777	a	200	[1:99]	0.72403	a	400	[All:all]	0.77021	a	100	[10:90]	0.75085	a
400	[10:90]	0.79846	a	100	[10:90]	0.70322	a	400	[25:75]	0.76203	a	All	[1:99]	0.84512	ab
200	[10:90]	0.78454	a	400	[25:75]	0.75088	ab	400	[35:65]	0.76083	a	200	[10:90]	0.80394	ab
1000	[25:75]	0.80416	ab	200	[25:75]	0.72089	ab	200	[1:99]	0.74776	a	100	[25:75]	0.73603	ab
400	[25:75]	0.79332	ab	100	[1:99]	0.69788	ab	200	[10:90]	0.74532	a	100	[All:all]	0.73240	abc
400	[1:99]	0.79105	ab	100	[25:75]	0.68656	abc	200	[All:all]	0.74494	a	All	[All:all]	0.83554	bc
200	[1:99]	0.77736	ab	All	[25:75]	0.77651	b	200	[25:75]	0.73703	a	200	[All:all]	0.78787	bc
All	[50:50]	0.80290	b	All	[35:65]	0.77168	b	100	[1:99]	0.71795	a	100	[35:65]	0.71594	bc
1000	[35:65]	0.80145	bc	400	[1:99]	0.74272	b	100	[10:90]	0.71655	a	All	[25:75]	0.83167	c
400	[35:65]	0.78776	bc	400	[35:65]	0.74009	b	100	[All:all]	0.71162	a	All	[35:65]	0.82591	c
200	[25:75]	0.76937	bc	200	[35:65]	0.70677	bc	All	[1:99]	0.78608	ab	200	[25:75]	0.78622	c
All	[1:99]	0.79715	c	100	[All:all]	0.67992	bc	100	[25:75]	0.70045	ab	200	[35:65]	0.77587	c
200	[All:all]	0.76255	c	All	[1:99]	0.76392	c	All	[25:75]	0.78179	abc	100	[50:50]	0.70722	c
200	[35:65]	0.75844	c	All	[50:50]	0.76248	c	400	[50:50]	0.74619	b	All	[50:50]	0.81434	d
1000	[1:99]	0.79823	cd	400	[50:50]	0.72678	c	200	[35:65]	0.71374	b	200	[50:50]	0.75780	d
400	[All:all]	0.77886	cd	400	[All:all]	0.72336	c	200	[50:50]	0.70118	b	–	–	–	–
1000	[50:50]	0.79504	d	200	[All:all]	0.70054	cd	all	[All:all]	0.77825	bc	–	–	–	–
All	[All:all]	0.78832	d	100	[35:65]	0.67357	cd	100	[35:65]	0.68687	bc	–	–	–	–
400	[50:50]	0.77793	d	All	[All:all]	0.74612	d	All	[35:65]	0.77490	c	–	–	–	–
200	[50:50]	0.74526	d	200	[50:50]	0.68893	d	100	[50:50]	0.67462	c	–	–	–	–
1000	[All:all]	0.78758	e	100	[50:50]	0.65839	d	All	[50:50]	0.76333	d	–	–	–	–
(b) Train_CV
All	[10:90]	0.82319	a	All	[10:90]	0.77090	a	All	[10:90]	0.73988	a	All	[10:90]	0.82112	a
1000	[10:90]	0.81748	a	All	[25:75]	0.76972	a	400	[10:90]	0.72602	a	200	[1:99]	0.78428	a
400	[10:90]	0.79391	a	400	[10:90]	0.75133	a	200	[10:90]	0.68450	a	100	[1:99]	0.73084	a
400	[25:75]	0.79186	a	400	[1:99]	0.74374	a	200	[1:99]	0.68391	a	200	[10:90]	0.78131	ab
200	[10:90]	0.76815	a	200	[10:90]	0.71388	a	100	[1:99]	0.66837	a	100	[10:90]	0.72700	ab
200	[1:99]	0.76623	a	200	[1:99]	0.71341	a	100	[10:90]	0.66221	a	All	[1:99]	0.81300	b
All	[25:75]	0.82113	ab	100	[1:99]	0.69233	a	100	[All:all]	0.65738	a	All	[25:75]	0.81154	b
1000	[25:75]	0.81538	ab	100	[10:90]	0.68280	a	400	[1:99]	0.72322	ab	200	[25:75]	0.77045	bc
400	[1:99]	0.78671	ab	100	[25:75]	0.67886	a	400	[25:75]	0.72010	ab	100	[All:all]	0.70978	bc
200	[25:75]	0.75931	ab	100	[All:all]	0.67779	a	400	[All:all]	0.71707	ab	100	[25:75]	0.70695	bc
All	[35:65]	0.81799	b	400	[25:75]	0.74283	ab	200	[All:all]	0.67961	ab	All	[35:65]	0.80248	c
400	[35:65]	0.78149	b	200	[25:75]	0.70243	ab	200	[25:75]	0.67483	ab	All	[All:all]	0.79995	c
200	[35:65]	0.74914	b	All	[35:65]	0.76372	b	100	[25:75]	0.65625	ab	200	[35:65]	0.76136	c
200	[All:all]	0.74885	b	All	[1:99]	0.75973	b	All	[25:75]	0.73331	b	200	[All:all]	0.75921	c
1000	[35:65]	0.81078	bc	200	[All:all]	0.69948	b	All	[1:99]	0.73307	b	100	[35:65]	0.69683	cd
All	[50:50]	0.81137	c	100	[35:65]	0.65408	b	400	[35:65]	0.71446	b	All	[50:50]	0.78849	d
All	[1:99]	0.81066	c	100	[50:50]	0.64716	b	200	[35:65]	0.66684	b	200	[50:50]	0.74497	d
400	[All:all]	0.77018	c	400	[35:65]	0.73359	bc	100	[35:65]	0.63598	bc	100	[50:50]	0.68324	d
400	[50:50]	0.76851	c	200	[35:65]	0.69507	bc	All	[35:65]	0.72656	c	–	–	–	–
200	[50:50]	0.73313	c	All	[50:50]	0.75384	c	All	[all:all]	0.72649	c	–	–	–	–
1000	[1:99]	0.80692	cd	200	[50:50]	0.68167	c	400	[50:50]	0.70160	c	–	–	–	–
1000	[50:50]	0.80325	d	400	[All:all]	0.72706	cd	200	[50:50]	0.65281	c	–	–	–	–
all	[All:all]	0.79896	d	All	[All:all]	0.74634	d	100	[50:50]	0.63076	c	–	–	–	–
1000	[All:all]	0.79083	e	400	[50:50]	0.72126	d	All	[50:50]	0.71734	d	–	–	–	–

Back to article page