A non-parametric maximum for number of selected features: objective optima for FDR and significance threshold with application to ordinal survey analysis

Ghaseminejad Tafreshi, Amir Hassan

doi:10.1186/s40537-018-0128-5

Journal of Big Data

Table 2 Sorted p-values and associated \(\updelta{\text{s}}\), R_max and breakpoints

From: A non-parametric maximum for number of selected features: objective optima for FDR and significance threshold with application to ordinal survey analysis

Column:	2	3	4	5	6	7	8	9	10
Set	p-value of the set (alpha if this set is rejected)	Frequency in set	Less than cumulative frequency hypotheses in rejected sets	Cumulative expected number of false discoveries (CEFD) if this set is rejected	Expected false discovery contribution of this set	FDR	Expected cumulative number of true discoveries (CETD) if this set is rejected	Expected true discovery contribution of this set	\({\varvec{\updelta}}\) = CETD − CEFD
1	0	7	7	0	0	0	7	7	7
2	0.000001	1	8	0.001044	0.001044	0.000131	7.998956	0.998956	7.997912
3	0.000005	1	9	0.00522	0.004176	0.00058	8.99478	0.995824	8.98956
4	0.000008	1	10	0.008352	0.003132	0.000835	9.991648	0.996868	9.983296
5	0.000009	1	11	0.009396	0.001044	0.000854	10.9906	0.998956	10.98121
6	0.000015	1	12	0.01566	0.006264	0.001305	11.98434	0.993736	11.96868
7	0.000021	1	13	0.021924	0.006264	0.001686	12.97808	0.993736	12.95615
8	0.000022	1	14	0.022968	0.001044	0.001641	13.97703	0.998956	13.95406
9	0.000034	1	15	0.035496	0.012528	0.002366	14.9645	0.987472	14.92901
10	0.000037	1	16	0.038628	0.003132	0.002414	15.96137	0.996868	15.92274
11	0.0001	1	17	0.1044	0.065772	0.006141	16.8956	0.934228	16.7912
…	…	…	…	…	…	…	…	…	…
34	0.00183	1	40	1.91052	0.061596	0.047763	38.08948	0.938404	36.17896
35	0.001841	1	41	1.922004	0.011484	0.046878	39.078	0.988516	37.15599
36	0.001944	1	R @ FDR 0.05 42	2.029536	0.107532	0.048322	39.97046	0.892468	37.94093
37	0.002081	1	43	2.172564	0.143028	0.050525	40.82744	0.856972	38.65487
38	0.00213	1	44	2.22372	0.051156	0.050539	41.77628	0.948844	39.55256
39	0.002369	1	45	2.473236	0.249516	0.054961	42.52676	0.750484	40.05353
…	…	…	…	…	…	…	…	…	…
104	0.010835	1	110	11.31174	0.363312	0.102834	98.68826	0.636688	87.37652
105	0.010966	1	R @ Break Point 111	11.4485	0.136764	0.10314	99.5515	0.863236	88.10299
106	0.011041	1	112	11.5268	0.0783	0.102918	100.4732	0.9217	88.94639
107	0.01112	1	113	11.60928	0.082476	0.102737	101.3907	0.917524	89.78144
108	0.011198	1	114	11.69071	0.081432	0.10255	102.3093	0.918568	90.61858
109	0.011256	1	115	11.75126	0.060552	0.102185	103.2487	0.939448	91.49747
110	0.01126	1	116	11.75544	0.004176	0.10134	104.2446	0.995824	92.48912
111	0.012177	1	117	12.71279	0.957348	0.108656	104.2872	0.042652	91.57442
112	0.012225	1	R @ Break Point 118	12.7629	0.050112	0.10816	105.2371	0.949888	92.4742
113	0.014141	1	119	14.7632	2.000304	0.124061	104.2368	-1.0003	89.47359
114	0.014273	1	120	14.90101	0.137808	0.124175	105.099	0.862192	90.19798
115	0.014334	1	121	14.9647	0.063684	0.123675	106.0353	0.936316	91.07061
…	…	…	…	…	…	…	…	…	…
174	0.038832	1	181	40.54061	1.092024	0.223981	140.4594	-0.09202	99.91878
175	0.039077	1	182	40.79639	0.25578	0.224156	141.2036	0.74422	100.4072
176	0.039224	1	183	40.94986	0.153468	0.22377	142.0501	0.846532	101.1003
177	0.0393	1	R _max 184	41.0292	0.079344	0.222985	142.9708	0.920656	101.9416
178	0.04194	1	185	43.78536	2.75616	0.236678	141.2146	-1.75616	97.42928
179	0.042014	1	186	43.86262	0.077256	0.235821	142.1374	0.922744	98.27477
180	0.042584	1	187	44.4577	0.59508	0.237742	142.5423	0.40492	98.08461
181	0.042642	1	188	44.51825	0.060552	0.236799	143.4818	0.939448	98.9635
…	…	…	…	…	…	…	…	…	…

If we set p-value threshold at 0.001944, we reject 42 null hypotheses, and we can report FDR less than 0.05
If we set p-value threshold at 0.010966, we reject 111 null hypotheses, and we can report FDR less than 0.11
If we set p-value threshold at 0.012225, we reject 118 null hypotheses, and we can report FDR less than 0.11
If we set p-value threshold at 0.0393, we reject 184 null hypotheses, and we can report FDR less than 0.23

Back to article page