Effectiveness analysis of machine learning classification models for predicting personalized context-aware smartphone usage

Abstract

Due to the increasing popularity of recent advanced features and context-awareness in smart mobile phones, the contextual data relevant to users’ diverse activities with their phones are recorded through the device logs. Modeling and predicting individual’s smartphone usage based on contexts, such as temporal, spatial, or social information, can be used to build various context-aware personalized systems. In order to intelligently assist them, a machine learning classifier based usage prediction model for individual users’ is the key. Thus, we aim to analyze the effectiveness of various machine learning classification models for predicting personalized usage utilizing individual’s phone log data. In our context-aware analysis, we first employ ten classic and well-known machine learning classification techniques, such as ZeroR, Naive Bayes, Decision Tree, Random Forest, Support Vector Machine, K-Nearest Neighbors, Adaptive Boosting, Repeated Incremental Pruning to Produce Error Reduction, Ripple Down Rule Learner, and Logistic Regression classifiers. We also present the empirical evaluations of Artificial Neural Network based classification model, which is frequently used in deep learning and make comparative analysis in our context-aware study. The effectiveness of these classifier based context-aware models is examined by conducting a range of experiments on the real mobile phone datasets collected from individual users. The overall experimental results and discussions can help both the researchers and applications developers to design and build intelligent context-aware systems for smartphone users.

Introduction

Nowadays, smartphones are considered as the most useful and essential devices of our daily life, in which individuals’ around the world communicate with one another for various purposes. Around 96.8% people in the current world use mobile devices, and this coverage even increases up to 100% in many developed countries [1]. Recently, with the rapid advances in context-aware mobile technologies and increasing popularity of data science research, data-driven personalized mobile services and systems are emerging as an important technology for designing and developing user-centric smart mobile applications [2]. People use smartphones for various purposes, such as voice communication by phone call activity, Internet browsing, mobile applications (Apps) using, e-mailing, online social networking, instant messaging etc. in their various day-to-day situations [3]. Individuals’ such activities with their phones and corresponding contextual information—for example, temporal, spatial, or social contexts, are captured by their own devices through the device logs, such as phone call logs [4, 5], app usage logs [6, 7], notification logs [8], web logs [9] etc.

The main principle of such sort of phone log information is that it contains reliable information of users’ mobile phone activities, as for most of the day, these devices are with their owners. Thus, modeling individual’s diverse activities with their phones, analyzing corresponding usage patterns in multi-dimensional contexts, and eventually predict their future usage utilizing such phone log data, can be used for building various context-aware personalized systems, such as smart interruption management system, intelligent mobile recommendation system, context-aware smart searching and reminder system, intelligent notification management system, smart application management system etc. that intelligently assist the end mobile phone users in their daily activities. For instance, individual’s phone call activity [10, 11] (e.g., answer, decline, missed, or making outgoing call) model based on relevant contexts utilizing phone call log data, can be used to provide personalized phone call services to intelligently manage the call interruptions. Such call interruptions mostly cause by the incoming calls in an inappropriate time (e.g., professional meeting in the morning). Moreover, a context-aware phone call activity model can play a role to intelligently remind an individual to make a phone call to another individual according to their interpersonal relationship (e.g., a close friend) in a particular location (e.g., in a restaurant), and to intelligently searches the desirable contact from the large contact list in that contexts. Similarly, mobile applications (e.g., Facebook, News, Whatsapp, Microsoft Outlook etc.) usage prediction model utilizing apps usage data, mobile notifications (e.g., promotional emails, various notifications from social networks, etc.) response model utilizing notification logs data, or other phone usage models, can play a role to provide the relevant context-aware mobile services for individuals in the real world. In a forensic context, the personalization also can play a critical role in the recovery of data about the misuse of telecommunications devices [12], or in the detection of usage patterns that may be consistent with a malware infection [13]. In order to provide such context-aware personalized services intelligently, a machine learning classifier based usage prediction model utilizing the relevant contextual phone log data is the key. In this paper, we take into account individual’s phone call activities in relevant contexts, and corresponding experiments using machine learning classifiers throughout the paper.

Let’s consider a particular phone call activity example of a smartphone user, Alice, who works in a corporate office as an executive officer. In each Monday somewhere in the range of 09:00 A.M. and 11:00 A.M., she attends in a ‘professional meeting’ at her office to talk about various authority assignments with her associates. Typically, she declines the incoming phone calls during that time period, as she wouldn’t like to be interrupted with incoming phone calls in the meeting. The reason is that it might cause intrusions that make unsettling influence for herself as well as may make aggravation for encompassing different associates and make humiliating circumstance in that meeting. However, if the phone call originates from her manager, she needs to answer the call as it prone to be more important for her, despite the fact that she is in a meeting. In this manner, the phone call activity of an individual cell phone user may change, dependent on her current contexts. An effective machine learning classifier based prediction model may anticipate Alice’s call exercises, in which she answers, declines, or misses the incoming phone calls in her day-by-day work schedules. Such call handling model based on relevant contexts (temporal, spatial, or interpersonal social relationship) utilizing Alice’s phone log data, could be used to build a smart call interruption management system for the user Alice, in order to provide context-aware personalized services intelligently.

In the area of mining mobile phone data, classification is the most popular machine learning technique to model and predict individual’s such activities with their phones. The goal of classification is to accurately classify the activity class labels of instances whose contextual features or attribute values are known, but class values are unknown [14]. Effectively modeling and predicting user phone call activities from phone log data using machine learning techniques is challenging as different classifiers give different prediction results in different contexts. Thus, in this work, we take into account a vast experimental analysis on different machine learning technique based models utilizing contextual mobile phone data. In order to analyze the effectiveness of a machine learning based prediction model, we employ ten most popular classic classification techniques [14, 15], such as ZeroR, Naive Bayes (NB), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Adaptive Boosting (AdaBoost), Repeated Incremental Pruning to Produce Error Reduction (RIPPER), Ripple Down Rule Learner (RIDOR), and Logistic Regression (LR) classifiers. We choose these techniques as they are well-known classifiers in machine learning and frequently used in the area of context-aware mobile services and systems for different purposes, which are summarized briefly in “Machine learning classifiers: background and related work” section. As deep learning has become very popular in recent days and used in various application areas, briefly discussed in “Machine learning classifiers: background and related work” section, we also present empirical evaluations with Artificial Neural Network (ANN) learning model, which is frequently used in deep learning for making comparative analysis in our context-aware study. Once the machine learning classifier based model has been built, the effectiveness of each model is examined by conducting experiments on the real mobile phone datasets consisting individual’s phone call activities and corresponding contextual information, collected from different individual mobile phone users. In our analysis, we calculate the prediction results in terms of precision, recall, F-measure, kappa, receiver operating characteristic (ROC) value, and error rate measured by mean absolute error (MAE) and root mean squared error (RMSE) of all these classifier based models, in order to provide better personalized mobile services in relevant context-aware systems for the benefit of end mobile phone users.

The contributions of this work can be summarized as follows:

• We highlight the importance of machine learning classifiers in modeling and predicting individual’s smartphone usage for personalized context-aware mobile services and systems.

• We do modeling and predicting personalized diverse activities with their phones in relevant multi-dimensional contexts using various machine learning classification techniques utilizing their own phone log data.

• We have conducted a range of experiments on the real phone usage datasets of individual users to evaluate the effectiveness of each classifier based prediction model on unseen contextual test cases.

The rest of the paper is organized as follows. “Machine learning classifiers: background and related work” section provides a summary of machine learning classification techniques and related works in the area of context-aware mobile services and systems. In “Contexts and context-aware mobile services” section, we briefly discuss about multi-dimensional contexts and corresponding context-aware mobile services with examples. In “Methodology: context-aware predictive modeling” section, we present the methodology for modeling personalized smartphone usage based on contexts. We report the experimental results on real mobile phone datasets and discuss about the effectiveness of the classification models in “Evaluation and experimental results” section. Some key observations of our machine learning based context-aware analysis are summarized in “Discussion” section, and finally “Conclusion and future work” section concludes this paper and highlights the future work.

Machine learning classifiers: background and related work

Classification is one of the most popular machine learning technique to predict the class of new samples, using a model inferred from training data. In general, classification is defined as a learning method that maps or classifies data instances into the corresponding class labels that are predefined in the given dataset. According to Han et al. [14], data classification is a two-step process; first one is the learning step where a classification model is constructed from a given dataset; the data from which a classification function or model is learned is known as the training set, and second one is a classification step where the model is used to test or predict the class labels for a separate unseen given data; the data set that is used to test the classifying ability of the learned model or function is known as the testing set. Research that relies on contextual smartphone data collected from individual mobile phone users is mostly application-specific. Many classification algorithms have been proposed with the ability of making predictions in the data mining literature. In order to predict smartphone usage, a number of researchers use different classification techniques for various context-aware mobile services and systems. In addition to context-aware mobile services, data-driven techniques can be used in other domains like in the context of cyber security with relevant data [16, 17], context-aware IoT (Internet of Things) or health care services utilizing relevant data [18,19,20,21]. Hence, we mainly focus and summarize a number of well-known and most popular classification techniques relevant to our target area, in the following subsections.

ZeroR

Zero-R is the simplest classification approach among the classification techniques in the area of data mining [15]. In general, this technique relies mainly on the target class and ignores all the predictors. Zero-R classifier simply predicts the majority category class available in the dataset. For instance, “$phone \; call \rightarrow answer$” can be a Zero-R model according to the majority call responses of a sample user utilizing her own phone log dataset. Although, there is no particular predictive intelligence in a Zero-R classifier based model, it is useful for deciding a standard execution as a benchmark for other classification techniques [15]. For instance, Fetter et al. [22] use ZeroR classifier in their analysis to compare with the base prediction probabilities while predicting selective availability for instant messaging.

Ripple Down Rule learner (RIDOR)

Ripple Down Rule learner (RIDOR) [15] is a direct classification method. This algorithm produces two types of rules such as default rule and the additional rules of that default rule. According to this principal, the algorithm first generates a default rule by analyzing the dataset. After that, it also produces a set of additional rules. To do this, this algorithm calculates the error rate and takes into account the least error rate while selecting the rules. In addition to the default rule, these generated additional rules are also used to predict the unseen classes for a particular condition. For instance, if a user is at home then she answers the incoming calls, except the caller is an unknown person, could be an example of such rule. In the area of context-aware mobile service, Anagnostopoulos et al. [23] use RIDOR classification technique in their machine learning based analysis while predicting the location of mobile users. Poppinga et al. [24] use RIDOR classifier for sensor-based identification of opportune moments for triggering notifications. In [25], Ayu et al. also use RIDOR classification technique in their activity recognition study using mobile phone data.

Repeated incremental pruning to produce error reduction (RIPPER)

Repeated incremental pruning to produce error reduction (RIPPER) [26] is a propositional rule learner proposed by William W. Cohen. This strategy is based on incremental reduced error pruning (IREP). In this machine learning classification technique, a set of pruning operators are used to repeatedly simplify the generated rule set. At each stage of simplification, the pruning operator chosen based on the error value, i.e., the one that yields the greatest reduction of error on the pruning set. For instance, in the morning, if a user is at home, and calls come from her friends or colleagues then she answers the incoming calls, could be an example of such rule. In the area of context-aware mobile services, Anagnostopoulos et al. [23] use RIPPER classification technique in their machine learning based analysis while predicting the location of mobile users.

K-nearest neighbors (KNN)

Another classification algorithm, K-nearest neighbors (KNN) [27] is one of the simplest classification technique in machine learning. It is a type of instance-based learning, or lazy learning. In this classification technique, it takes into account local approximation and all the computation is deferred until classification. It stores all the available cases in the given dataset and classifies new cases based on distance functions like Euclidean distance as a similarity measure. After that the K most comparable occasions, called the neighbors, are controlled via seeking through the whole preparing set for another test information point. Finally, the expectation result is made by abridging the yield variable for those K cases dependent on larger part casting a majority voting of the neighbors. A number of researchers use KNN classifier in the area of context-aware mobile services. For instance, Swati et al. [28] have used KNN technique in their personalizing mobile search engine. Bozanta et al. [29] use KNN technique in their analysis while developing a contextually personalized hybrid recommender system. Anagnostopoulos et al. [23] use KNN classification technique in their machine learning based analysis while predicting the location of mobile users. In [25], Ayu et al. use KNN classification technique in their activity recognition study using mobile phone data. In [30], the authors use KNN classifier while building their recommendation system. Fisher et al. [31] also use KNN technique in their smartphone interruptibility analysis.

Naive Bayes (NB)

Naive Bayes (NB) [32] in one of the most popular classification algorithms in the area of data mining. A Naive Bayes classifier is a basic probabilistic based technique, which can foresee the class membership probabilities [14]. As it calculates the likelihood of membership for each class, it also can easily handle the missing attribute values by simply omitting the corresponding probabilities for those attributes. In a Naive Bayes classifier, the effect of an attribute on a given class is also independent of those of other attributes, which is known as class conditional independence. This classifier easily calculates the probability to classify or predict the class in a given dataset. Although, it may cause the zero probability problem, it can play an important role while predicting mobile user activity classes in multi-dimensional contexts [33]. Pejovic et al. [3] have considered Naive Bayes classifier in their approach for the purpose of intelligently handling mobile interruptions. Bedogni et al. [34] use Naive Bayes classifier in their context-aware mobile applications through transportation mode detection techniques. In Ayu et al. [25], use Naive Bayes classification technique in their activity recognition study using mobile phone data. Fogarty et al. [35] use Naive Bayes classifier for predicting human interruptibility with sensors. Fisher et al. [31] use this classifier in their smartphone interruptibility analysis. Recently, Sarker et al. [36] propose a robust prediction model based on Naive Bayes classifier for context-aware mobile services. Turner et al. [37, 38] also use Naive Bayes classifier in their interruptibility prediction and management analysis.

Logistic regression (LR)

Logistic regression (LR) [39] is another popular probabilistic based statistical model used to solve the classification problems. Typically, logistic regression estimates the probabilities using a logistic function, which is also referred to as sigmoid function. The hypothesis of logistic regression tends it to limit the function between 0 and 1. This classifier measures the relationship between the categorical dependent variable and one or more independent variables for a given dataset. The dependent variable is the target class, we are going to predict. However, the independent variables are the attributes or contextual features, we are going to use to predict the target class. In our work, contextual information can be used as independent variables and individual’s diverse mobile phone activities are represented as dependent variable. A number of researchers use logistic regression classifier in the area of context-aware mobile services for different purposes. For instance, Riboni et al. [40] use logistic regression classifier for the purpose of context-aware activity recognition. In [41], the authors use this classifier in order to predict users’ demographics utilizing mobile data. Wang et al. [42] use logistic regression in their mobile recommendation system. In [43], the authors use this technique while predicting user activity level in social networks. In [44], Ernsting et al. use logistic regression in their population-based survey to change and manage health behaviors using smartphones and health apps. Ayu et al. [25] use this classifier in their activity recognition study using mobile phone data. Turner et al. [37, 38] also use logistic regression technique classifier in their interruptibility prediction and management analysis.

Support Vector Machines (SVM)

Support Vector Machines (SVM) [45], is another popular classification technique used widely for various predictive analytics. In most cases, SVM is utilized as a binary classifier, i.e., to isolate the information focuses in the information variable space by their class, either class 0 or class 1. To do this, a hyperplane is chosen in the vector machine, which is a line that can take part into the variable space. Utilizing this hyperplane, the vector machine learning calculation finds the coefficients bringing about the best detachment of the classes. Researchers use SVM classifier in the area of context-aware mobile services for various purposes. For instance, Pielot et al. [46], have used Support Vector Machines to analyze contextual mobile phone data. Bedogni et al. [34] use SVM classifier in their context-aware mobile applications through transportation mode detection techniques. Bayat et al. [47] use SVM classifier in their study on human activity recognition using accelerometer data from smartphones. In [25], Ayu et al. use SVM classification technique in their activity recognition study using mobile phone data. In [22], Fetter et al. use this classifier in their analysis while predicting selective availability for instant messaging. Fogarty et al. [35] use SVM classifier for predicting human interruptibility with sensors. Fisher et al. [31] use this technique in their smartphone interruptibility analysis. Turner et al. [37, 38] also use SVM classifier in their interruptibility prediction and management analysis.

Decision tree (DT)

A very well-known and mostly discussed technique for classification and then used for prediction is decision trees [48]. The core algorithm for building decision trees called ID3 proposed by Quinlan [48]. ID3 algorithm constructs a decision tree by employing a top-down approach in which a greedy searching through the given training dataset is used to test each attribute or context at every node. It calculates the entropy and information gain which is a statistical property that is used to select which attribute to test at each node in the tree [48]. Based on the ID3 algorithm, an extension namely C4.5 algorithm is proposed by Quinlan [49]. C4.5 builds decision trees from a training dataset in the similar procedure as ID3. This algorithm generates contextual decision rules to predict mobile user activity. C5.0 is another modified decision tree algorithm used for predictive modeling [50]. C5.0 decision tree is significantly faster and more memory efficient than C4.5. Researchers use different techniques for generating decision trees in their analysis. In the area of context-aware mobile services and systems, decision tree classifier is mostly used to analyze contextual mobile phone data. For instance, in [51], Hong et al. have proposed a decision tree based approach to build a context-aware system utilizing context history, for the purpose of providing personalized services. In [52], Lee et al. propose an approach for deploying personalized mobile services using decision tree algorithm. An intelligent mobile interruption management system based on decision tree model has been built by Zulkernain et al. [53]. In Fetter et al. [22], use decision tree classifier in their analysis while predicting selective availability for instant messaging. Fogarty et al. [35] use decision tree classifier for predicting human interruptibility with sensors. Fisher et al. [31] use decision tree classifier in their smartphone interruptibility analysis. Turner et al. [37, 38] use decision tree classifier in their interruptibility prediction and management analysis. Recently, Sarker et al. [36] also use decision tree in their robust prediction model based on multi-dimensional contexts for context-aware mobile services.

Random forest (RF)

Random forest classifier proposed by Breiman et al. [54] is an ensemble machine learning method that takes into account multiple learning algorithms together while generating a prediction result. Random forest combines bootstrap aggregation (bagging) [55] and random feature selection [56] to construct a collection of decision trees exhibiting controlled variation. Overall, random forest generates a number of decision trees rather than a single decision tree, to predict the final output activity class of the mobile phone users utilizing their phone log data. By generating multiple decision trees for a given dataset, it reduces the over-fitting problem causes in the single decision tree discussed earlier. A number of researchers use random forest classifier in the area of context-aware mobile services. For instance, Pielot et al. [57] have used random forest classifier in their prediction model. In another study, Pielot et al. [46] have used this classifier in their analysis for mobile services. Bedogni et al. [34] use random forest classifier in their context-aware mobile applications through transportation mode detection techniques. Bayat et al. [47] use this classifier in their study on human activity recognition using accelerometer data from smartphones. Ayu et al. [25] use random forest classification technique in their activity recognition study using mobile phone data. Turner et al. [37, 38] also use random forest classifier in their interruptibility prediction and management analysis.

Adaptive boosting (AdaBoost)

Adaptive boosting is a machine learning meta-algorithm formulated by Freund et al. [58]. This technique can also be used for the purpose of prediction. The concept of this algorithm is to combine the output of other learning algorithms, named as ‘weak learners’ to produce an effective classifier for getting the final output of the boosted classifier. AdaBoost is called adaptive classifier in that sense by dramatically improving the classifier performance, however, in some cases it may cause overfits. AdaBoost is sensitive to noisy data and outliers. A number of researchers use AdaBoost classifier in the area of context-aware mobile services for various purposes. For instance, Pejovic et al. [3] have considered AdaBoost classifier in their approach for the purpose of intelligently handling mobile interruptions. Anagnostopoulos et al. [23] use this classification technique in their machine learning based analysis while predicting the location of mobile users. Fogarty et al. [35] use AdaBoost classifier for predicting human interruptibility with sensors. In [30], the authors use AdaBoost classification technique while building their recommendation system. Turner et al. [37, 38] also use AdaBoost classifier in their interruptibility prediction and management analysis.

Artificial neural network (ANN) and deep learning

An Artificial Neural Network (ANN) is also a popular machine learning technique frequently used in the area of deep learning, which is a computational model that is inspired by the biological neural networks in human brain. The most popular neural network algorithm is backpropagation [14] that performs learning on a multilayer feed-forward neural network. During the learning phase, the network learns by adjusting the weights iteratively for predicting the correct class label of the given input features. There are several variants of neural network models, such as Single Layer Perceptron, Multilayer Perceptron, Recurrent Neural Network, Long Short-Term Memory Network, and Convolutional Neural Network. A multilayer backpropagation feed-forward neural network consists of an input layer having input neurons or nodes that provide information from the outside world to the network, one or more hidden layers having hidden neurons or nodes that perform computations and transfer information from the input nodes to the output nodes, and an output layer having output neurons or nodes that are responsible for computations and transferring information from the network to the outside world. Typically, network design is a trial-and-error process and may affect the accuracy of the resulting trained network. Researchers use such neural networks for different purposes. For instance, Alawnah et al. [59] design an approach for modeling smartphones’ power using neural network utilizing power-related records. Leong et al. [60] propose a neural network approach for predicting the determinants of mobile credit card acceptance in contactless payment for the future generation. Chong et al. [61] propose a neural network approach for predicting mobile commerce adoption determinants, i.e., buying and selling of goods and services through wireless handheld devices. Tan et al. [62] empirically investigates on the elements that affect the user’s intention to adopt mobile learning (m-learning) using a feed-forward back-propagation multi-layer perceptron model. Rajashekar et al. [63] have conducted a neural network based study on smart phone user behaviour characterization based on autoencoders and self organizing maps. Recently, in the area of machine learning and data science research and applications, deep neural network learning has become very popular and applied to various fields including computer vision, speech recognition, natural language processing, face recognition, bioinformatics, medical diagnosis, blood cell classification, stock market prediction, weather forecasting etc. to solve the complex classification problems.

Table 1 A summary of machine learning classification techniques used for various context-aware mobile services and systems

All of these classification techniques discussed above are able to solve the classification and prediction tasks in the area of context-aware mobile services and systems, which are summarized in Table 1. Thus, it is very difficult to assume an effective classification model in order to predict individual’s samrtphone usage based on multi-dimensional contexts. Therefore, in this paper, we analyze the effectiveness of these popular machine learning classifier based prediction models considering relevant multi-dimensional contexts, for the purpose of achieving better context-aware personalized mobile services.

Contexts and context-aware mobile services

In general, context is defined as any information that can be used to characterize the situation of an entity [64]. In this work, individual smartphpone user is taken into account as an entity. As we aim to analyze individual’s phone call activities using machine learning classification techniques, the relevant contexts, such as temporal, spatial, or social information, are needed to take into account, which have the influence on individuals for such mobile phone activities. In the following, we briefly discuss about such contextual information with relevant examples in the area of context-aware mobile services and systems. These are:

• Temporal context Temporal context is one of the primary context having influence on phone call activities of an individual user. For instance, one’s mobile phone activities in the morning may not be similar in the evening or night [65]. As each user activity is associated with a particular timestamp (e.g., YYYY-MM-DD hh:mm:ss) in the real world, temporal context plays a primary role to model individuals’ mobile phone usage behavior. Halvey et al. [66] have shown through the analysis of a large sample of user data that time-of-the-week is an important factor in modeling mobile user behavior. For instance, the specific date (e.g., YYYY-MM-DD) of an activity, days-of-the-week (Monday, Tuesday, ..., Sunday) to occur that activity, the specific or exact time-of-the-day (e.g., hh:mm:ss) are considered as the temporal context. In addition to these primary temporal information, the additional temporal information such as public holidays, weekdays, weekends, might have the influence on mobile phone users to make decisions.

• Spatial context Spatial context is another significant user’s context that can be used to model and predict individual’s mobile phone activities more effectively [67, 68]. The reason is that phone call activities of an individual can also be treated as a location based service. For instance, one’s activity at office may not be similar with her activities at home. Thus, understanding user mobility in their daily life is a fundamental issue for the context-aware applications that is able to provide location based mobile services for the benefit of individuals. With the increasing pervasiveness and context-awareness of smart mobile phones over the past few years, a number of emerging location based mobile applications are adopted by the users. The reasons that makes user current location or spatial context in mobile phone applications so popular are; location based mobile services rely on the knowledge about the user’s geographical availability to obtain relevant information on the area or spot, and the user behaves accordingly under that particular spatial context [69]. Thus, we take into account individual’s location as a spatial context in our analysis.

• Social context In addition to the above spatio-temporal, social contexts have also an influence on individual mobile phone users to take decisions [70]. In the real world, individuals are engaged in various social activities such as professional meeting, seminar, lecture etc. People are well differ from each other in how they behave with their mobile phones during various events, which has been shown in Sarker et al. [71]. For instance, one individual mobile phone user may be happy to accept the incoming phone calls during a professional meeting, whereas another individual may not be wanted to do the similar activities during that event [72]. Even the activity of a particular individual may be different subject to what type of event is occurred [73]. For example, an individual’s phone call activity or response during a ‘professional meeting’ may be well different from her activity during a ‘lunch-break’ event. The social context, such as the relationships between individuals, e.g., family, friend, work or professional, romantic, or unknown, have a strong influence on individual mobile phone users to take such phone call handling decision in such events [70, 74]. For instance, an individual typically ‘declines’ an incoming phone call during an event official meeting, however, she ‘answers’ if the incoming call comes from her ‘boss’. Thus the social relational bonding between the caller and the callee has a strong influence to handling call response decision. We also use such social relational context to make the prediction model more effective.

Methodology: context-aware predictive modeling

In this section, we present our approach for modeling mobile user activity using various classification techniques. It comprises of three steps: Data set exploration, Contextual data processing, and Machine learning classifier-based predictive modeling. In the following, we briefly discuss these steps one by one.

Dataset exploration

A mobile phone dataset (e.g., context logs) with multi-dimensional contexts is a collection of records, where each record is a set of pairs (context, value) [e.g., (location, office)], called items, which identify specific data features (e.g., location) and their values in the corresponding domains (e.g., office). The multi-dimensional contexts are temporal, spatial or social, discussed above. In the following, the notion of mobile phone dataset with multi-dimensional contexts is formally stated [36].

Let $Con = \{con_1,con_2, ..., con_m\}$ be a set of nominal contexts and $Q = \{q_1,q_2, ..., q_m\}$ the set of corresponding domains. A mobile phone dataset DS is a collection of records, where:

(i) :

Each record r is a set of pairs $(con_i,value_i)$ , where $con_i \in Con$ , and $value_i \in Q$ . For example, if $con_i$ represents as the context ‘location’, then an example of $value_i$ is ‘at office’ or ‘market’.

(ii) :

Each $con_i \in Con$ , also called attribute (contextual feature), may occur at most once in any record or instance, and

(iii) :

Each record has a particular user activity with mobile phones (e.g., decline the incoming phone call).

Let’s consider an example of a sample information coming from a phone call log that records user phone call activities with corresponding multi-dimensional contextual information. For instance, a sample record represents that the user declines her friend’s phone call (Incoming call from 047XXX234) on 2016-09-19 10:03:15 at her office. This example composed of the associated contexts, such as date-time represents temporal context, contact number may represent the interpersonal social relationship between the caller and callee, e.g., friend, user location represents spatial context, for the user phone call activity ‘decline’. Such multi-dimensional contextual data can be used to analyze the effectiveness of different machine learning classifier based prediction models related to user’s activities with their phones.

Contextual data processing

Real-world smartphone data collected from individual users usually comprise a set of features whose interpretation depends on some contextual information. These are temporal, spatial, or social context discussed in the earlier section. Such contextual features and related patterns are of high interest to be discovered from the mobile phone data in order to predict smartphone user behavior for unseen test cases. In this work, we have used users’ real-world phone log dataset in order to model individual smartphone user phone call activity in different contexts. In order to build the prediction model, first we extract the contextual information discussed above from the dataset. However, the raw contextual data is not applicable to build the prediction model using machine learning techniques. To make such phone log contextual data applicable for building the prediction model, we process the contextual information into a meaningful category. In the following, we discuss how we process these contextual information to build the prediction model using machine learning classifiers.

• Context discretization Real-world mobile phone data contains continuous time-series data. However, such data is not directly applicable to build the prediction model. Thus, in order to convert such continuous data into nominal values, we use our earlier behavior-oriented time-series segmentation (BOTS) approach [75] to process this temporal context. In this approach, we first generate initial time slices using a base period as the finest granularity required to distinguish day-to-day activities of an individual user. After that, we aggregate adjacent slices dynamically with the same dominant characteristics in order to generate behavior-oriented segments. We also measure the corresponding applicability that takes into account not only the support value but also the temporal coverage to finalize the optimal segments. Thus, by using this approach, in this work, we generate a number of optimal temporal segments based on the similar activities of each individual user at various time-of-the day, and days-of-the-week. We also take into account day-wise segmentation as individual’s activity may not similar in all days-of-the-week, may differ from day-to-day. For instance, one’s Monday’s activities may not be similar with her Friday’s activities because of various day-to-day situations in the real world life. The total number of time segments generated by this approach is dynamic, depends on the number of the similar activity patterns during 24-h a day. An example of such generated segments are Friday [09:00–11:15], Monday [12:30–13:30], Saturday [15:30–18:45] etc. In addition to this temporal context, we also use the nominal values of user locations that are available in the dataset [76]. An example of such spatial contexts are home, office, market, MIT, Harvard, John Street etc.

• Data-centric social context generation In addition to above spatio-temporal context, we also use user individual’s unique phone number available in the dataset as a social relational context. The reason is that it is very difficult to make the device understandable about the semantic relational context, such as family, friend or others. Moreover, users activities may not be similar for all her friends or all the family members or others. For instance, one’s phone call response behavior may vary between ‘best friend’ and ‘close friend’, even though both persons represent same relationship category ‘friend’. Thus, it may differ from person to person in the real world. Therefore, we generate the data-centric social relational context from the mobile phone data to achieve our goal. The main principle to generate the value of data-centric social context is “each unique mobile phone number represents a particular one-to-one relationship” [10]. For instance, best friend’s phone number (03...034567) represents one relation (Rel01), while close friend’s phone number (03...044587) represents another relation (Rel02) etc. Similarly, boss’s phone number (02...034211) represents another different relation (Rel03), which is also unique comparing to others. Such data-centric context values can play a significant role for personalized context-aware mobile services.

Machine learning classifier-based predictive modeling

In the area of mining mobile phone data, classification is a supervised learning method and can be used to model mobile phone user activity based on multi-dimensional contexts. As mentioned earlier, in our analysis, we employ ten classic and most popular classification techniques that are frequently used in the area of context-aware mobile services for various purposes, which are briefly discussed in “Machine learning classifiers: background and related work” section. These techniques are ZeroR, Naive Bayes (NB), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Adaptive Boosting (AdaBoost), Repeated Incremental Pruning to Produce Error Reduction (RIPPER), Ripple Down Rule Learner (RIDOR), and Logistic Regression (LR) classifiers. In addition to these popular classifiers in machine learning, we also take into account Artificial Neural Network (ANN) deep learning model in our context-aware analysis. In the following, we discuss our classification process that is carried out on four main components of contextual data set:

• Smartphone usage or activity class label The standard methodology comprises in taking in a classifier from a named dataset, to anticipate the class of new examples [14]. In this work, we take into account users’ diverse phone call activities as different class labels to achieve our goal. For this, we take into account all the diverse phone call activities or usage, as different distinct class labels to model individual’s mobile phone activity. For instance, individual’s phone call activities, such as answering the call, decline the call, missed call, and making an outgoing call, are represented as distinct classes; Class 1, Class 2, Class 3, and Class 4 respectively.

• Non-class attributes or contextual features These are independent in the data set, which are used to predict user’s such mobile phone activities. These are also known as contexts such as temporal context (e.g., day, time), spatial context (e.g., user location), and social context (e.g., social relationship), discussed earlier, in which the class levels are dependent.

• Training dataset The training data set in machine learning is the actual dataset used to train the classification model for performing various predictive services relevant to the features involved. We use a major part of contextual mobile phone data discussed above for building the activity model. In order to build the model, we randomly partition the initial data into k mutually exclusive subsets or “folds”, $d_1,d_2,...,d_k$, each of which has an approximately equal size. Then, in each iteration i, we use all the instances of all partitions except the partition $d_i$ as the training dataset to build the activity model.

• Testing dataset This is the data typically used to provide an unbiased evaluation of the activity prediction model that is learned using the training dataset mentioned above. Actually, testing data is used for testing the model whether it is predicting the class label appropriately or not, for a particular context. This is done by using a k fold cross validation. In k fold cross-validation, we use the partitioned data $d_i$ as testing dataset in each iteration i.

Evaluation and experimental results

To evaluate the effectiveness of each machine learning classifier based model, we have conducted a range of experiments on the real mobile phone datasets for predicting individual mobile phone users’ activity in different contexts. In the following subsections, we briefly describe the datasets, and present the experimental results and corresponding discussions. As we focus on personalized usage prediction model, we illustrate the experimental results of two different individuals selected randomly, in details, and finally report the average prediction results by taking into account all the datasets.

Smartphone datasets

We have done experiments on ten phone log datasets, each of which contains individual’s diverse phone call activities and corresponding contextual information. These datasets are collected from different types of users including faculty, staff, and students, over the period of 9 months by Massachusetts Institute of Technology (MIT) for their Reality Mining project [76]. The datasets incorporate individuals with various kinds of calling examples and call disseminations in multi-dimensional contexts, e.g., temporal, spatial and social contexts, discussed above and corresponding phone call activities, e.g., incoming call responses that include answer or decline calls by the users, missed and outgoing calls of every individuals. An example of such data is represented as ‘device ID (e.g., 111e6f2b3894—John’s Phone), time series (e.g., 2019-10-12 11:13:25—time format YYYY-MM-DD hh:mm:ss), cell area (e.g., 37137), cell tower ID (e.g., 222—Market), contact phone number (e.g., 047345762—John’s number), call directions (incoming, missed, and outgoing), call duration (e.g., 35 s). We distinguish between the answer and decline call responses from the incoming calls by taking into account the call duration. For instance, if the call duration is more than 0, then the call has been answered by the user; if it is equal to 0 then the call has been declined [65]. In our experiments, the phone log datasets of individuals are represented as DS01, DS02, ..., DS10 that contain 55,105 phone call records with contexts.

Evaluation metric

In order to measure the prediction accuracy, we compare the predicted response with the actual response (i.e., the ground truth) and compute the accuracy of each classifier based model in terms of:

• Precision, recall/sensitivity and ROC Precision is a measure of exactness, which represents the ratio between the number of phone call activities that are correctly predicted and the total number of activities that are predicted (both correctly and incorrectly). On the other hand, recall is a measure of completeness, which represents the ratio between the number of phone call activities that are correctly predicted and the total number of activities that are relevant. Recall also represents the Sensitivity [26]. These are calculated using the values of true positive rate, false positive rate, and false negative rate in prediction results. receiver operating characteristic (ROC) [26] can be used as another metric that also features on true positive rate, and false positive rate, to evaluate the classifier output quality. If TP, FP, and FN denote true positives, and false positives, and false negatives, then the formal definition of precision and recall are [26]:

  $$\begin{aligned} Precision = \frac{TP}{TP + FP} \end{aligned}$$

  (1)
  $$\begin{aligned} Recall= \frac{TP}{TP + FN} \end{aligned}$$

  (2)

• F-measure and kappa F-measure is a metric that combines precision and recall in a single score, which is the harmonic mean of precision and recall. Kappa is a metric that compares an observed accuracy with an expected accuracy (random chance). The formal definition of F-measure and Kappa are [26]:

  $$\begin{aligned} F\hbox {-}measure= 2 * \frac{Precision * Recall}{Precision + Recall} \end{aligned}$$

  (3)
  $$\begin{aligned} Kappa= \frac{(observed \; accuracy - expected \; accuracy)}{(1 - expected \; accuracy)} \end{aligned}$$

  (4)

• MAE and RMSE mean absolute error (MAE) and Root Mean Squared Error (RMSE) are used to calculate the error rate of each classifier based model, which represents the success of prediction. If the predicted values on the test instances are $p_1, p_2, ...,p_n$, and the actual values are $a_1, a_2, ...,a_n$, for n data points, then MAE and RMSE are formally defined as below [26]:

  $$\begin{aligned} MAE= \frac{1}{n}\sum _{i=1}^n \left| p_i-a_i \right| \end{aligned}$$

  (5)
  $$\begin{aligned} RMSE= \sqrt{\frac{1}{n}\sum _{i=1}^n \left( p_i - a_i \right) ^{2}} \end{aligned}$$

  (6)

In order to evaluate the mobile phone usage prediction model based on machine learning classifiers, we take into account all these metrics to calculate utilizing individual’s mobile phone data.

Experimental results and effectiveness analysis

In this experiment, we show the effectiveness of each machine learning classifier based prediction model for individuals’ mobile phone activities in relevant contexts. In the following, we have shown the experimental results of ZeroR, Naive Bayes (NB), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Adaptive Boosting (AdaBoost), Repeated Incremental Pruning to Produce Error Reduction (RIPPER), Ripple Down Rule Learner (RIDOR), Logistic Regression (LR), and Artificial Neural Network (ANN) classifiers based smartphone usage prediction models. The details of these classifiers are discussed in “Machine learning classifiers: background and related work” section. For each model, we utilize the same datasets in order to compare the techniques fairly. For the purpose of evaluating our approach, we employ the tenfold cross validation technique on each mobile phone dataset. The 10-fold cross validation technique splits individual’s phone log data into 10 sets of size N/10. After that, it trains each model on 9 sets and tests it using the remaining one set. According to the procedure of cross validation, this repeats 10 times and we take a mean prediction results for each model. To show the effectiveness of each classifier based model, we calculate and compare the prediction results in terms of precision, recall, F-measure, kappa, receiver operating characteristic (ROC) value, and error rate measured by mean absolute error (MAE) and root mean squared error (RMSE), defined above.

In Tables 2 and 3, we have shown the prediction results of each classic classifier based model in terms of CCI (correctly classified instances) rate, ICI (incorrectly classified instances) rate, mean absolute error (MAE) rate, root mean squared error (RMSE) rate, and receiver operating characteristic (ROC) value of two different individual mobile phone users utilizing their own mobile phone datasets. Prediction results of ANN model for these datasets are shown in different tables discussed later. For example, as shown in Table 2, we see that the correctly classified instances of RF and DT are 85.37% and 85.13% respectively that are higher than other classifier based models. Similarly, according to the experimental results shown in Table 3, we see that the correctly classified instances of RF and DT are 84.66% and 82.28% respectively that are also higher than other classifier based models. The ROC values that represent true positive rate against the false positive rate of these tree-based classification models also give better results than other classifier based models, shown in Tables 2 and 3. In addition to these measurements, tree based models also give lower error rate in terms of ICI, MAE, and RMSE values. Overall, by taking into account all the context-aware prediction results shown in Tables 2 and 3, we can conclude that the tree based classifiers, i.e., DT (decision tree) and RF (random forest) based context-aware models for each individual smartphone user perform better than other classifier based models.

Table 2 Prediction results using various machine learning classification models utilizing dataset DS02

Table 3 Prediction results using various machine learning classification models utilizing dataset DS05

Effectiveness comparison

To show the model effectiveness based on machine learning classifiers for individual users, Figs. 1 and 2 show the relative comparison of precision, recall, f-measure and kappa for two different individuals. For each prediction model, we use the same train and testing data for the purpose of fair evaluation. In addition to individual’s comparison, we also show the relative comparison of average results for a collection of datasets shown in Fig. 3. For this, we calculate the average results of precision, recall, f-measure and kappa of all ten datasets described above.

Fig. 1

Effectiveness comparison results for different machine learning classifier based context-aware models utilizing dataset DS02

Fig. 2

Effectiveness comparison results for different machine learning classifier based context-aware models utilizing dataset DS05

Fig. 3

Effectiveness comparison results (average) for different machine learning classifier based context-aware models utilizing a collection of datasets

If we observe Figs. 1, 2, and 3, we find that tree-based classification models give higher prediction results than other context-aware models, in terms of precision, recall, f-measure and kappa statistic, while applying on mobile phone data consisting of multi-dimensional contexts. In particular, RF (random forest) based context-aware model generating multiple decision trees gives the prediction results with the highest values of precision, recall, f-measure and kappa statistic, shown in Figs.1, 2, and 3. In the following, we discuss the comparing results of random forest based model with other classifier based models.

Random forest vs. baseline classifier

As shown in Figs. 1, 2 and 3, the values of precision, recall, f-measure, and kappa of ZeroR (baseline) classifier based model, are relatively poor than other classifier based models. The reason for getting this low value is that, this classifier simply predicts the majority class without taking into account computing intelligence. As individual’s daily mobile phone activities vary in different contexts, only predicting this majority activity class can not ensure the effectiveness of the model. Random forest classifier based context-aware model resolves this issue and provides comparatively better prediction results shown in Figs. 1, 2 and 3. Although, ZeroR based model gives lower prediction results, this classifier is used for determining a baseline performance as a benchmark for other classifier based models.

Random forest vs. decision tree

Both the decision tree (DT) and random forest (RF) are known as tree based classifiers, i.e., both classifiers generate trees based on contexts, while building a prediction model in the area of context-aware mobile services and systems, discussed briefly in “Machine learning classifiers: background and related work” section. In a decision tree based model, only a single decision tree is generated from the training dataset. On the other hand, in a random forest ensemble learner based model, multiple decision trees are generated from the training dataset. Our experimental results on individual’s real-life mobile phone datasets having multi-dimensional contexts, show that both these tree based context-aware classification models give better prediction results than other classifier based models in terms of precision, recall, f-measure, and kappa, shown in Figs. 1, 2 and 3. In particular, the random forest classifier based model is far better than a single decision tree in predicting users’ mobile phone activities based on multi-dimensional contexts. The reason is that decision tree classifier generates a few number of logic rules from the tree, in which the prediction accuracy might be less. On the other hand, random forest generates a set of logic rules from a number of separate decision trees generated in the forest. Each tree in a random forest model behaves as a separate classifier and thus it provides more logic rules by taking into account the majority voting of these trees. As a result, we get higher performance in terms of prediction accuracy with higher values of precision, recall, f-measure, and kappa statistic, that measure the effectiveness of a prediction model.

Random forest vs. other ML classifiers

Besides the prediction results of tree-based classification models, discussed above, other classic machine learning classifiers like logistic regression based model, Support Vector Machine based model, k-nearest neighbor based model, and Naive Bayes classifier based model, also give significant personalized prediction results, shown in Figs. 1 and 2. In addition to precision, recall, f-measure, and kappa values, the correctly classified instances and ROC values that represent true positive rate against the false positive rate of these models give good personalized results, shown in Tables 2 and 3. Besides such personalized results utilizing individual’s usage data, the average prediction results by considering a collection of datasets of different individuals of these classifier based models are also significant in terms of average values of precision, recall, f-measure, and kappa statistic that have been shown in Fig. 3. However, in the area of context-aware mobile services and systems, the random forest ensemble learner based context-aware model improves the overall prediction results, shown in Figs. 1, 2 and 3. In order to improve the results, the random forest model fits multiple decision trees with the subset of smartphone data, and averages all the single tree results. It also controls over-fitting and biasing while building the prediction model. As a result, the random forest ensemble learning based context-aware model gives better prediction results than all other classifier based models discussed in this paper, and can be considered such model as an effective prediction model while taken into account smartphone usage data having multi-dimensional contexts.

Effectiveness comparison with neural network classification model

One interesting observation based on the above classic machine learning analysis is the effectiveness of the tree-based classification models, particularly the random forest learning based context-aware smartphone usage model. One may ask how does this compare with neural network learning? In general, building an effective model in terms of higher accuracy with deep neural network learning traditionally requires very large datasets [77]. For instance, face recognition with deep neural network is usually trained with millions of data samples, argued by Hu et al. [78]. Neural network based other application areas are discussed in “Machine learning classifiers: background and related work” section. In our case, individual’s smartphone usage like daily phone call activities in different contexts recorded in phone log may not be much in our real world life. Thus, to get huge amount of training data of individuals’ phone usage, collecting contextual data year after year might not be meaningful to reflect one’s present behavior. The reason is that individual’s behavior changes over time and a period of recent data are more likely to be interesting and significant than older ones for predicting individual’s future behavior in a particular context [79]. Consequently, individual’s phone usage log data does not have too many samples and contextual features, requiring re-engineering of feature extraction in the deep neural network learning model. For instance, we take into account three dimensions of contextual features such as temporal, spatial, and social context relevant to individual’s phone usage behavior in our context-aware analysis. Thus, the relevant contextual features and corresponding phone log data is very small, in comparison to image, audio, text logs, medical records or similar data that has been used in neural network learning experiments. Therefore, based on the relevancy of contextual features and individual’s phone usage log data, our hypothesis is that neural network learning is not going to work well for our context-aware analysis compared to the random forest ensemble learning model discussed above.

Table 4 Prediction results using neural network learning based context-aware classification model utilizing dataset DS02

Table 5 Prediction results using neural network learning based context-aware classification model utilizing dataset DS05

Fig. 4

Effectiveness comparison results of neural network based context-aware prediction model for different number of epochs utilizing dataset DS02

Fig. 5

Effectiveness comparison results of neural network based context-aware prediction model for different number of epochs utilizing dataset DS05

To test this hypothesis, we have also conducted experiments on a backpropagation based multilayer neural network model to classify individual’s smartphone usage. In our neural network based context-aware model, input layer consists of three contextual features: temporal, spatial, and social context, and output layer represents four different behavioral classes; answering, declining, missed, and making outgoing call. The hidden layer consists of neurons as the average of the input contextual features and the output behavioral classes. We also use different number of epochs for the purpose of comparing the prediction results, in order to build an effective neural network model. For instance, in Tables 4 and 5, we have shown the prediction results of this neural network model in terms of CCI (correctly classified instances) rate, ICI (incorrectly classified instances) rate, mean absolute error (MAE) rate, root mean squared error (RMSE) rate, and receiver operating characteristic (ROC) value, by varying the number of epochs starting from 10 to 100, for two different individual mobile phone users utilizing their own mobile phone datasets. As shown in Table 4, we see that the results vary depending on the number of epochs, and this model gives higher prediction results, when the number of epoch is 20. Similarly, as shown in Table 5, we see that this neural network model gives higher prediction results, when the number of epoch is 30. However, for these contextual datasets, the random forest learning model gives more significant results, i.e., higher accuracy rate in terms of CCI and ROC value, and lower error rate in terms of ICI, MAE, and RMSE values than this neural network model, shown in Tables 2 and 3. Overall, by taking into account all the context-aware prediction results shown in Tables 2, 3, 4, and 5, we can conclude that the tree based classifier, particularly, RF (random forest) based context-aware model performs better than other machine learning classification models.

In addition to the above experimental results, Figs. 4 and 5 show the prediction results of neural network model in terms of precision, recall, f-measure and kappa statistic. The results are also shown by varying the number of epochs starting from 10 to 100 for these two different individuals utilizing their smartphone datasets. If we observe Figs. 4 and 5, we also find that these prediction results of neural network context-aware model are not constant for different number of epochs, gives highest results when the number of epoch is 20 and 30 respectively for these two datasets. However, random forest based model gives more significant prediction results with higher values of precision, recall, f-measure and kappa statistic, shown in Figs. 1 and 2. In addition to these individual’s results, we also calculate the prediction results in terms of average precision and recall of 0.68 and 0.71 respectively by considering all the ten datasets mentioned earlier. However, more significant results with higher values of average precision and recall are found by random forest based context-aware model, shown in Fig. 3. Therefore, based on the overall experimental results discussed above, we can conclude that the random forest ensemble learning based context-aware model performs significantly better, and consequently more effective than other machine learning classification models, for predicting individual’s smartphone usage behavior. The reason is that random forest ensemble model combines a number of single decision tree learners to form a strong learner in order to make the context-aware model more effective.

Discussion

Overall, our machine learning classifier based context-aware analysis is fully personalized and individual’s smartphone usage data-oriented that reflects their behavioral patterns according to their phone log data. In our context-aware analysis, we first employed ten classic and well-known machine learning classification techniques, such as ZeroR, Naive Bayes (NB), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Adaptive Boosting (AdaBoost), Repeated Incremental Pruning to Produce Error Reduction (RIPPER), Ripple Down Rule Learner (RIDOR), and Logistic Regression (LR) classifiers. We have also presented the empirical evaluations of Artificial Neural Network (ANN) based classification model, which is frequently used in deep learning. All these techniques are briefly discussed in “Machine learning classifiers: background and related work” section. In the following, we highlight a number of key observations of our experimental study.

To predict individual’s smartphone usage for personalized context-aware mobile services and systems, different contextual information such as temporal, spatial or social contexts, can have the influence on individual’s usage. Various machine learning classification techniques mentioned above, can play a vital role to achieve our goal. However, the raw smartphone data of individuals’ are not directly applicable to build such prediction model. The multi-dimensional contextual information relevant to individual’s usage are needed to process for modeling and predicting personalized diverse activities with their phones using these machine learning classification techniques, which has been discussed briefly in “Methodology: context-aware predictive modeling” section.

One important finding of our study is that tree-based classification models, i.e., decision tree and random forest, give higher prediction results than other context-aware models, while applying on mobile phone data consisting of multi-dimensional contexts. In particular, random forest ensemble learner based context-aware model gives better prediction results than the decision tree model. The reason is that decision tree classifier based context-aware model generates a few number of contextual logic rules from the tree. On the other hand, random forest takes into account multiple decision trees while building a context-aware model, and consequently generates a set of contextual logic rules. Thus it provides more logic rules and significant performance in terms of prediction accuracy with higher values of precision, recall, f-measure, and kappa statistic that measure the effectiveness of a prediction model, shown in Figs. 1, 2, and 3.

We have observed poor prediction results when using the baseline ZeroR classifier based model. The reason is that this classifier simply predicts the majority class without taking into account computing intelligence. However, it can be used as a benchmark for other classifier based models. Compared to this baseline classifier, logistic regression based model, Support Vector Machine based model, k-nearest neighbor based model, and Naive Bayes classifier based model, give significant prediction results, shown in Figs. 1, 2 and 3, as they have their own predictive power in computing. Moreover, these classifier based context-aware models also give significant results in terms of ROC value and lower error rate for each individual user, shown in Tables 2 and 3, which is much better than the baseline classifier.

Although artificial neural network is more popular and frequently used in deep learning model, in order to solve the complex classification problems, it does not give significant results compared to the tree based context-aware model. The reason is that individual’s phone usage log data does not have too many samples to train the neural network model properly, and limited contextual features such as temporal, spatial, and social context, as required re-engineering of feature extraction in the deep neural network learning model. Thus, the relevant contextual features and corresponding phone log data is very small, in comparison to image recognition, signal processing, medical records or similar data that has been used in neural network learning experiments. The performance of neural network based deep learning model might be better in solving the complex classification problems with relevant huge datasets, where the classic machine learning techniques discussed in “Machine learning classifiers: background and related work” section may not be useful. Our neural network based context-aware model also shows that the prediction results differ by varying the number of epochs while training the model. Large number of epochs is not always good in terms of prediction results like precision, recall, f-measure, kappa statistic, shown Figs. 4 and 5. The reason is that it causes the model to overfit the training data and consequently decreases the accuracy. Overall, based on the characteristics of contextual features, and corresponding experimental results utilizing individual’s smartphone datasets, we can conclude that the random forest ensemble learning based context-aware model is more effective than other machine learning classification models for predicting individual’s usage behavior.

Conclusion and future work

Context-aware personalized smartphone usage modeling and prediction can be used for building various data-driven smart applications in order to intelligently assist the end mobile phone users. For providing such context-aware personalized services intelligently, we have analyzed the effectiveness of various popular machine learning classifier based usage prediction models including neural network learning model utilizing individual’s real-life mobile phone data. In our machine learning based effectiveness analysis, we take into account multi-dimensional contexts such as temporal, spatial, or social contexts, that have influences, either separately or combinedly, on individual’s activities with their phones. Although, we choose phone call contexts as examples, this effectiveness analysis is also applicable to other mobile application domains based on relevant multi-dimensional contexts and corresponding smartphone usage of the users. We believe that our effectiveness analysis on various machine learning classification based context-aware models can help both the researchers and application developers as a reference point to work in the area of context-aware mobile services and systems.

Based on our effectiveness analysis, to develop a context-aware real-life application utilizing individual’s smartphone usage data, in order to provide intelligent personalized services for the end mobile phone users, and to asses our work in application level, can be a future work.