Evaluation of maxout activations in deep learning across several big data domains

Journal of Big Data

Table 3 Number of trainable parameters per dataset

Dataset	ReLU	ReLU 2x	ReLU 3x	ReLU 6x	MFM 2-1	MFM 3-1	MFM 6-1	MFM 3-2
Fashion-MNIST	21,840	85,670	191,500	760,990	43,170	64,500	128,490	128,000
MNIST	21,840	85,670	191,500	760,990	43,170	64,500	128,490	128,000
CIFAR-10	31,340	122,670	274,000	1,087,990	62,170	93,000	185,490	183,500
Medicare Part B	96,450	258,434	485,954	1,561,730	192,770	289,090	578,050	387,522
Medicare Part D	97,474	260,482	489,026	1,567,874	194,818	292,162	584,194	390,594
DMEPOS	105,666	276,866	513,602	1,617,026	211,202	316,738	633,346	415,170
Combined CMS	120,002	305,538	556,610	1,703,042	239,874	359,746	719,362	458,178
CIFAR-100	156,130	572,160	1,248,190	4,836,280	287,160	418,190	811,280	833,190
IDMT-SMT-Audio-Effects	825,996	3,294,476	7,405,452	29,593,356	1,648,908	2,471,820	4,940,556	4,938,636
IRMAS	1,808,779	7,226,123	16,252,043	64,981,259	3,614,731	5,420,683	10,838,539	10,836,363
Sentiment140	2,743,234	10,966,914	24,671,042	98,666,114	5,485,442	8,227,650	16,454,274	16,449,346
MS-Celeb	2,997,432	11,469,704	25,417,816	100,117,192	5,737,864	8,478,296	16,699,592	16,948,056
LFW	4,286,434	17,137,602	38,553,506	154,189,634	8,572,354	12,858,274	25,716,034	25,705,890
Google Speech Commands	6,767,267	27,028,771	60,784,547	243,017,507	13,516,579	20,265,891	40,513,827	40,525,219
Amazon1M	8,641,474	34,559,874	77,755,202	311,002,754	17,281,922	25,922,370	51,843,714	51,838,786
Yelp500K	8,641,474	34,559,874	77,755,202	311,002,754	17,281,922	25,922,370	51,843,714	51,838,786
Yelp1M	8,641,474	34,559,874	77,755,202	311,002,754	17,281,922	25,922,370	51,843,714	51,838,786
Amazon4M	17,030,082	68,114,306	153,252,674	612,992,642	34,059,138	51,088,194	102,175,362	102,170,434