Dimensionality reduction and class prediction algorithm with application to microarray Big Data
 Fadoua Badaoui^{1}Email author,
 Amine Amar^{2},
 Laila Ait Hassou^{3},
 Abdelhak Zoglat^{3} and
 Cyrille Guei Okou^{4}
Received: 24 July 2017
Accepted: 26 September 2017
Published: 10 October 2017
Abstract
The recent technology development in the concern of microarray experiments has provided many new potentialities in terms of simultaneous measurement. But new challenges have arisen from these massive quantities of information qualified as Big Data. The challenge consists to extract the main information containing the sense from the data. To this end researchers are using various techniques as “hierarchical clustering”, “mutual information” and “selforganizing maps” to name a few. However, the management and analysis of the millions resulting dataset haven’t yet reached a satisfactory level, and there is no clear consensus about the best method/methods revealing patterns of gene expression. Thus, many efforts are required to strengthen the methodologies for optimal analysis of Big Data. In this paper, we propose a new processing approach which is structured on feature extraction and selection. The feature extraction, is based on correlation and rank analysis and leads to a reduction of the number of variables. The feature selection, consists in eliminating redundant or irrelevant variables, using some adapted techniques of discriminant analysis. Our approach is tested on three type of cancer gene expression microarray and compared with concurrent other approaches. It performs well, in terms of prediction results, computation and processing time.
Keywords
Introduction
In recent years, technological innovations have led to a massive amount of data with relatively low cost. These massive and highthroughput data is commonly called Big Data. However, there is no universally agreedupon definition of Big Data, but the more widely accepted explanations tend to describe it in terms of challenges that it presents. In terms of computational efficiency and time processing, Big Data motivate the development of new computational tools and data storage methods [17, 20, 27, 29]. Regarding this issue, [38] evokes three principal challenges which are related to dimensions of Big Data and which address volume, velocity and variety. Other authors have proposed additional dimensions such as veracity, validity or value [15]. The volume, one of the famous five Vs that characterize Big Data, is the main challenge that interests the statistician when analyzing high dimension datasets. The other Big Data dimensions, interest particularly computer scientists and data investigators.
Besides challenges, Big Data give many opportunities in terms of results analysis and information extraction in different fields such as genomics and biology [30], climatology and water research [19], geosciences [44], neurology [18], spam detection and telecom [6, 13], Cybersecurity [56], Software engineering [14, 40], social media analysis [37, 46], biomedical imaging [53], economics [21, 35], high frequency finance and marketing strategies [7]. The goal of using Big Data in the aforementioned fields is to develop accurate methods to predict the future, to gain insight into the relationship between the features and responses, to explore the hidden structures and to extract important common features across subpopulations.
The main problem with Big Data is still how to efficiently process it. To handle this challenge, we need new statistical thinking and computational methods. In fact, many statistical approaches that perform well for low dimension data, are inadequate when analyzing Big Data. Thus, to design effective statistical procedures for the exploration and prediction in this context, new needs will be identified, aside classical issues such as heterogeneity, noise accumulation, spurious correlations [23], incidental endogeneity [39], and [26], and sure independence screening [25], Hall and Miller [32, 33], and [12]. In terms of statistical accuracy, dimension reduction and variables selection play pivotal roles in analyzing high dimension data. For example, in high dimension classification, [48], and [22] showed that conventional classification rules using all features perform no better than random guess due to noise accumulation. This motivates new regularization methods [9, 10, 24, 54, 55].
The aim of dimension reduction procedures is to summarize the original pdimensional data space in a form of a lower kdimensional components subspace \((k \ll p)\). To achieve this goal, statistical and mathematical theory provide many approaches. Based on frequency use, the most commonly applied methods are still principal component analysis (PCA) [2], and [34], partial least squares (PLS) [4, 5] and [45], linear discriminant analysis (LDA) [8], and sliced inverse regression (SIR) [3]. Rash Model (RM) is another recent efficient way for feature extraction which provides an appealing framework for handling highdimensional datasets [36].
For all aforementioned considerations, and given the growing importance of alternative statistical approaches, we propose a new approach to reduce a dataset dimension, especially for classification purposes. The approach addresses the case where the number of variables p largely exceeds the sample size n \((p \gg n)\), which is common in the Big Data context. To handle high dimension datasets in the prediction framework, we propose to proceed in five steps. The first three steps seek to reduce the number of variables using correlation arguments. The fourth and fifth steps consist in eliminating redundant or irrelevant variables, using adapted techniques of discriminant analysis. The performance of our approach is evaluated by measuring its accuracy of class prediction and processing time.
Before introducing a detailed description of our approach, it is worth to have a good understanding of state of the art in the concern of extraction and selection methodologies, especially for Big Data. Thus, the following section proposes to conduct a review of published studies to identify key trends with respect to the types of used methods.
Background and statistical review
For p smaller than n, classical methods of classification (LDA, PCA, \(\ldots\)) can be applied. In this work, we consider the case where p is much larger than n. This data structure has been used in special cases of gene expression data [11], to characterize different types of cancers [31] and the Lymphoma dataset [1].
The analysis of a high dimension dataset is primarily based on comparison of variables or observations, using a variety of similarity measures. The correlation, can be used as a measure of association between variables. To measure correlation between categorical and numerical variables, “the statistic \(\eta\)” can be used [28, 47] and [51, 52]. This statistic represents the ratio of variability between groups to the total variability.
In this paper, we elaborate a new approach to deal with the large dimension challenge presented by the Big Data framework. Our approach is summarized in an algorithm in five steps. The first three steps lead to the reduction of the number of columns (variables) in a dataset, the two others identify pertinent variables for building an accurate classifier. We apply our techniques to publicly available microarray datasets and compare our results with findings discussed in [36]. Our approach can clearly be used in many other areas (economy, finance, environment...etc.) where “high dimension” is a Big Data challenge.
A dimension reduction algorithm

Step 1: Calculate the correlation ratio between each variable \(\mathbb {X}_j\) and nominal response (\(\mathbb {Y}\)) defined as:where \({X}_{ij}^k\) is the value of variable \(\mathbb {X}_j\) measured on the ith individual belonging to the kth class, \(\bar{ X}_j^k\) is the mean of the restricted \(\mathbb {X}_j\) to the kth class, and \(\bar{{X}}_j\) is the (unrestricted) mean of \(\mathbb {X}_j\).$$\begin{aligned} \eta ^2_j=\frac{\sum \limits_{k=1}^K n_{k}(\bar{X}_j^k\bar{{X}}_j)^2}{\sum \limits_{k=1}^K \sum \limits_{i=1}^{n_k} ({X}_{ij}^k\bar{X}_j^k)^2} \end{aligned}$$(2)

Step 2: For \(j= 1,\ldots , p\), sort \(\mathbb {X}_j\) in descending order according to \(\eta ^2_j\) values, and extract a basis of the first \({p}^\prime\) linearly independent variables, following the process of GramSchmidt ([49]).

Step 3: For j and \(j^\prime\) in \(\{1,\ldots , p^\prime \}\) with \(j<j^\prime\), calculate \(\tau (\mathbb {X}_j,\mathbb {X}_{j^\prime })\), the Kendall rank correlation coefficient between the \(\mathbb {X}_j\) and \(\mathbb {X}_{j^\prime }\). If \(\tau (\mathbb {X}_j,\mathbb {X}_{j^\prime }) \ge 0.5\), eliminate \(\mathbb {X}_{j^\prime }\) (because \(\eta ^2_{j^\prime }<\eta ^2_{j}\)). Otherwise keep \(\mathbb {X}_j\) and \(\mathbb {X}_{j^\prime }\).

Step 4: For \(\ell\) ranging from 2 upto \({p}^{\prime \prime }\), perform the LDA to subsets, of the dataset resulting from Step 3, involving the \(\ell\) first variables. For the classification purpose, retain the variables that maximize the cross validation percentage.

Step 5: Repeat the steps 1 to 4 with different sample sizes. The final set of retained variables contains those proven reliable predictors at least \(m\%\) of the time (m may be set as 70%).
Application and results
In this section we consider the application of our approach to some real datasets recently used in cancer gene expression studies by several authors. The first dataset has been obtained from acute leukemia patients at the time of diagnosis [31]. This dataset comes from a study of gene expression in two types of acute leukemias, acute lymphoblastic leukemia (ALL) and acute myeloid leukemia (AML). The data consist of 47 cases of ALL (38 Bcell ALL and 9 Tcell ALL) and 25 cases of AML of \(p = 3571\) human genes. The second dataset concerns the Prostate cancer that contains 52 prostate tumor observations and 50 nontumor prostate observations of \(p = 6033\) genes.
Both data sets are from Affymetrix highdensity oligonucleotide microarrays and are publicly available [16].
Application to Leukemia dataset
Reduction of number of genes for different sample size in dataset
Sample size  Rank of the extracted basis (steps 1 and 2)  Nb of genes with Kendall rank correlation \(<0.5\) (step 3)  Number of final selected genes (step 5) 

60  60  33  3 
55  55  31  4 
50  50  28  22 
45  45  13  7 
40  40  11  9 
Final retained classifiers
Frequency  Number of genes  Cross validation (%)  Final retained genes 

At least 4 times  2 genes  91.7  gene376, gene456, gene626, gene672, gene874, gene907, gene918, gene951, gene956, gene979, gene1001 
At least 3 times  4 genes  96.7  
At least 2 times  11 genes  98.3 
Class prediction for the test sample
Observations  Observed class  Scores  Predicted class 

1  0  1797  0 
2  1  − 5044  1 
3  1  − 4543  1 
4  0  3743  0 
5  0  0243  0 
6  1  − 5924  1 
7  1  − 5016  1 
8  1  − 4070  1 
9  0  − 0222  0 
10  1  − 5733  1 
11  1  − 4967  1 
12  1  − 3395  1 
Performances comparison
RMLDA  Our approach  

Number of selected genes  ERrandom selection  ERsupervised selection  Number of selected genes  Random sample size  Error rate (%) 
50  0.31  0.04  11  60  0 
100  0.29  0.04  11  58  0 
200  0.27  0.05  11  46  0 
Application to prostate cancer state
Reduction of number of genes for different sample size in dataset
Sample size  Rank of the extracted basis (steps 1 and 2)  Nb of genes with Kendall rank correlation \(<0.5\) (step 3)  Final selected genes (step 5) 

89  89  31  5 
80  80  26  5 
75  75  31  3 
70  70  34  10 
60  60  34  10 
50  50  24  8 
Final retained classifiers
Frequency  Number of genes  Cross validation (%)  Final retained genes 

At least 4 times  1 genes  92.1  gene2619, gene1495, gene2425, gene2746, gene4849, gene1788, gene1897, gene2848, gene4155 
At least 3 times  5 genes  91  
At least 2 times  9 genes  95.5 
Class prediction for testing sample
Observations  Observed class  Scores  Predicted class 

1  0  − 0.857  0 
2  0  − 1.064  0 
3  0  − 0.614  0 
4  0  − 2.846  0 
5  0  − 1.593  0 
6  0  − 1.933  0 
7  1  1.035  1 
8  1  2.149  1 
9  1  2.806  1 
10  1  2.751  1 
11  1  0.584  1 
12  1  0.722  1 
13  1  0.048  1 
Performance comparison
RMLDA  Our approach  

Number of selected genes  ERrandom selection  ERsupervised selection  Number of selected genes  Random sample size  Error rate (%) 
50  0.46  0.18  9  90  0 
100  0.45  0.19  9  80  0 
200  0.45  0.21  9  67  0 
It is worth noting that the use of the developed approach is not restricted to binary prediction problems. It can be extended to cover multiclass prediction. Indeed, we applied the approach on a third dataset which concerns the small blue cell tumors (SRBCTs) presented as a matrix of 2308 genes (columns) and 83 samples (rows), from a set of microarray experiments. The SRBCTs are 4 different childhood tumors classified into four major types: BL (Brkitt lymphoma), EWS (Ewings sarcoma), NB (neuroblastoma), and RMS (rhabdomyosarcoma). After applying the same approach described above for (2308 × 83) dataset, 8 genes are selected. Even if, we have 4 different classes, our approach performs well. It gives a mean accuracy rate of 90%.
Conclusions
Big Data is a highly topical issue of major importance in healthcare research. In fact, the role of Big Data in medicine consists to better build health profiles and predictive models around individual patients, so that we can better diagnose and treat disease. Big data comes into play an important role to overcome major challenges posed by cancer which represents an incredibly complex disease. The cancer disease is always changing, evolving, and adapting, where a single tumor can have more than 100 billion cells, and each cell can acquire mutations individually. To best understand evolution of cancer or to best distinguish tumor classes, we need advanced modeling by integrating Big Data. Different techniques are available, but it suffers from a lack of accuracy or processing complexity.
The purpose of this article is to present methods to reduce the number of variables and keep those that contain more information for reliable and informative classification. The article proposes methods for dimensionality reduction and classification, in several stages, using gene expression data from two recent studies. This way of proceeding, allows to retrieve the variables that contain most information for proper classification according to type of cancer. The retained model is the one that guarantees the best classification by crossvalidation. The final model is then used to predict the class samples of the test set.
A comparative study was developed, for binary problems, between the results of our approach and that of the model developed by Rash [36]. The main conclusion is that our approach performs well the RMLDA based approach with a null error rate and a 100% of accuracy.
It is worth to note that our approach can be compared with other multiclass prediction problems by integrating multiple ROC analysis and can be used to analyze other prediction problems in different fields such as, finance and banking, marketing and environment.
Declarations
Authors' contributions
All mentioned authors contribute in the elaboration of the article. All authors read and approved the final manuscript.
Acknowlegements
A particular acknowledgement is for the scientific and the editorial committee. Acknowledgement is also for the providers the used Data.
Competing interests
All authors confirm that there are no competing interests. The article is not under any other review process and is not subject of any other submission.
Availability of data and materials
All data used are publically available. Source of the used data is mentioned in the article.
Consent for publication
Authors approve the consent for publication.
Ethics approval and consent to participate
All authors confirmed the ethics approval and consent to participate.
Funding
No funding exists. We have ask for free charge processing and we have a confirmation for the Big data Journal, that we not need to pay the article processing charge, because we are based in a lowincome country.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Alizadeh AA, Eisen MB, Davis RE, Ma C, Lossos IS. Distinct types of diffuse large Bcell lymphoma identified by gene expression profiling. Nature. 2000;403:503–11.View ArticleGoogle Scholar
 Alter O, Brown PO, Botstein D. Singular value decomposition for genomewide expression data processing and modeling. PNAS. 2000;97(18):10101–6.View ArticleGoogle Scholar
 Antoniadis A, LambertLacroix S, Leblanc F. Effective dimension reduction methods for tumor classification using gene expression data. Bioinformatics. 2003;19(5):563–70.View ArticleGoogle Scholar
 Boulesteix AL. PLS dimension reduction for classification with microarray data. Stat Appl Genet Mol Biol. 2004;3(1):1–30.MathSciNetView ArticleMATHGoogle Scholar
 Boulesteix AL, Strimmer K. Partial least squares: a versatile tool for the analysis of high dimensional genomic data. Brief Bioinf. 2008;8:24–32.Google Scholar
 Bughin J. Reaping the benefits of big data in telecom. J Big Data. 2016;3:14.View ArticleGoogle Scholar
 Casaca JA, da Gama AP. Marketing in the Era of Big data, human and social sciences at the common conference. 2013.Google Scholar
 Cai T, Liu WD. A direct estimation approach to Sparse linear discriminant analysis. J Am Stat Assoc. 2011;106:1566–77.MathSciNetView ArticleMATHGoogle Scholar
 Candes E, Tao T. The Dantzig selector: statistical estimation when p is much larger than n. Ann Stat. 2005;35(6):23132351.MathSciNetGoogle Scholar
 Chen S, Donoho D, Saunders M. Atomic decomposition by basis pursuit. SIAM J Sci Comput. 1998;20(1):3361.MathSciNetView ArticleMATHGoogle Scholar
 Chiaromonte F, Martinelli J. Dimension reduction strategies for analyzing global gene expression data with a response. Math Biosci. 2002;176:123144.MathSciNetView ArticleMATHGoogle Scholar
 Christopher G, Jiashun J, Wasserman L, Yao Z. A comparison of the lasso and marginal regression. J Mach Learn Res. 2011;13:21072143.MathSciNetGoogle Scholar
 Crawford M, Khoshgoftaar M, Prusa D, Richter N, Al Najada H. Survey of review spam detection using machine learning techniques. J Big Data. 2015;2:23.View ArticleGoogle Scholar
 Depeige A, Doyencourt D. Actionable knowledge as a service (AKAAS): leveraging big data analytics in cloud computing environments. J Big Data. 2015;2:12.View ArticleGoogle Scholar
 Demchenko Y, Grosso P, de Laat C, & Membrey P. Addressing Big Data issues in scientific data infrastructure. Proceedings of the international conference on collaboration technologies and systems, May 20–24. San Diego: IEEE Xplore Press; 2013. p 485. DOI: 10.1109/CTS.2013.6567203.Google Scholar
 Dettling M. BagBoosting for tumor classification with gene expression data. Bioinformatics. 2004;20(18):3583–93.View ArticleGoogle Scholar
 Donoho DL, Elad M. Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization. Proc Natl Acad Sci. 2013;100(5):2197–202.MathSciNetView ArticleMATHGoogle Scholar
 Kondziolka Benjamin T C, Lunsford LD, Silverman J. Development, implementation, and use of a local and global clinical registry for neurosurgery. Big Data. 2015;3(2):80–9.View ArticleGoogle Scholar
 DongGuo H, Zhang L, WeiZhu L. Earth observation big data for climate change research. Adv Clim Change Res. 2015;6(2):108–17.View ArticleGoogle Scholar
 Efron B, Hastie T, Johnstone I, Tibshirani R. Least angle regression. Ann Stat. 2003;32:407451.MathSciNetMATHGoogle Scholar
 Einav L, Levin J. Economics in the age of big data. Science. 2014;346(6210):1243089.View ArticleGoogle Scholar
 Fan J, Fan Y. High dimensional classification using features annealed independence rules. Ann Stat. 2008;36:260537.MathSciNetView ArticleMATHGoogle Scholar
 Fan J, Guo S, Hao N. Variance estimation using refitted crossvalidation in ultrahigh dimensional regression. J R Stat Soc Ser B. 2012;74(1):3765.MathSciNetView ArticleGoogle Scholar
 Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc. 2001;96(456):13481360.MathSciNetView ArticleMATHGoogle Scholar
 Fan J, Lv J. Sure independence screening for ultrahigh dimensional feature space (with disussion). J R Stat Soc Ser B. 2007;70(5):849911.Google Scholar
 Fan J, Liao Y. Endogeneity in ultrahigh dimension, technical report. New Jersey: Princeton University; 2014.Google Scholar
 Fan J, Samworth R, Wu Y. Ultrahigh dimensional feature selection: beyond the linear model. J Mach Learn Res. 2009;10:20132038.MathSciNetMATHGoogle Scholar
 Fisher R. Statistical methods for research workers. ISBN 0050021702; 1926.Google Scholar
 Friedman J, & Popescu B. Gradient directed regularization for linear regression and classification. Technical report. 2004.Google Scholar
 Gesing S, Connor T, & Taylor I. Genomics and biological Big Data: facing current and future challenges around data and software sharing and reproducibility. Position paper at BDAC15 (Big Data Analytics: Challenges and Opportunities), workshop in cooperation with ACM/IEEE SC15, Austin; 2015.Google Scholar
 Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD, Lander ES. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science. 1999;286:531–7.View ArticleGoogle Scholar
 Hall P, Miller H. Using generalized correlation to effect variable selection in very high dimensional problems. J Comp Graph Stat. 2009;18(3):533550.MathSciNetView ArticleGoogle Scholar
 Hall P, Miller H. Modeling the variability of rankings. Ann Stat. 2010;38(20):2652–77.MathSciNetView ArticleMATHGoogle Scholar
 Holter NS, Mitra M, Maritan A, Cieplak M, Banavar JR, Fedoro NV. Fundamental patterns underlying gene expression profiles: simplicity from complexity. Proc Natl Acad Sci USA. 2000;97:8409–14.View ArticleGoogle Scholar
 Husain S, Kalinin A, Truong A, Dinov D. SOCR data dashboard: an integrated big data archive mashing medicare, labor census and econometric information. J Big Data. 2015;2:13.View ArticleGoogle Scholar
 Kastrin A, Peterlin B. Raschbased highdimensionality data reduction and class prediction with applications to microarray gene expression data. Exp Syst Appl. 2010;37(7):5178–85.View ArticleGoogle Scholar
 Kramer A, Guillory J, Hancock J. Experimental evidence of massive scale emotional contagion through social networks. Proc Natl Acad Sci USA. 2014;111(24):8788–90.View ArticleGoogle Scholar
 Laney D. 3D Data management: controlling data volume, velocity and variety. 2001.Google Scholar
 Liao Y, Jiang W. Posterior consistency of nonparametric conditional moment restricted models. Ann Stat. 2011;39(6):30033031.MathSciNetView ArticleMATHGoogle Scholar
 Loureno JR, Cabral B, Carreiro P, Vieira M, Bernardino J. Choosing the right NoSQL database for the job : a quality attribute. J Big Data. 2015;2(1):1–26.Google Scholar
 Mardia KV, Kent JT, Bibby JM. Multivariate analysis. San Diego: Academic Press Inc; 1979.MATHGoogle Scholar
 McLachlan GJ. Discriminant analysis and statistical pattern recognition. New York: Wiley; 1992.View ArticleMATHGoogle Scholar
 Meulman JJ, Heiser JW. IBM SPSS Categories 20. 2011. pp. 233–248Google Scholar
 Narock TW, & Hitzler P. Crowdsourcing semantics for Big Data in geosciences applications. In: AAAI 2013 Fall symposium series, semantics for Big Data, November 15–17. Arlington; 2013.Google Scholar
 Nguyen DV, Rocke DM. On partial least squares dimension reduction for microarraybased classification: a simulation study. Comput Stat Data Anal. 2004;46(3):407–25.MathSciNetView ArticleMATHGoogle Scholar
 Pääkkönen P. Feasibility analysis of AsterixDB and spark streaming with Cassandra for streambased processing. J Big Data. 2016;3:6. doi:10.1186/s4053701600418.View ArticleGoogle Scholar
 Pearson ES. Review of statistical methods for research workers (R. A. Fisher). Sci Prog. 1926;20:733–4.Google Scholar
 Pittelkow PH, Ghosh M. Theoretical measures of relative performance of classifiers for high dimensional data with small sample sizes. J R Stat Soc B. 2008;70:15973.MathSciNetMATHGoogle Scholar
 Pursell L, Trimble SY. GramSchmidt orthogonalization by Gauss elimination. Am Math Month. 1991;98(6):544549. doi:10.2307/2324877.MathSciNetView ArticleMATHGoogle Scholar
 Ripley BD. Pattern recognition and neural networks. Cambridge: Cambridge University Press; 1996.View ArticleMATHGoogle Scholar
 Santos F. Le rapport de corrlation : mesurer la liaison entre une variable qualitative et une variable quantitative. CNRS, UMR 5199 PACEA. 2015.Google Scholar
 Shaldehi AH. Using Eta (η) correlation ratio in analyzing strongly nonlinear relationship between two variables in practical researches. J Math Comput Sci. 2013;7(3):213–20.Google Scholar
 Toga W, Dinov D. Sharing big biomedical data. J Big Data. 2015;2:7.View ArticleGoogle Scholar
 Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B. 1996;58(1):267288.MathSciNetMATHGoogle Scholar
 Zhang C. Nearly unbiased variable selection under minimax concave penalty. Ann Stat. 2010;38(2):894942.MathSciNetView ArticleGoogle Scholar
 Zuech R, Koshgoftaar M, Wald R. Intrusion detection and big heterogeneous data: a survey. J Big Data. 2015;2:3.View ArticleGoogle Scholar