Learning topic description from clustering of trusted user roles and event models characterizing distributed provenance networks: a reinforcement learning approach

Provenance graph, computational trust and hierarchy of roles

The issues of trust and provenance have been described in [37, 38] where every state in the provenance graph has been associated with an activity. The security or access restrictions or storage constraints of data base access and the concept of associating cost with trust has been described in [3]. The use of information cascades for tracing the path associated with a piece of news has been described in [6]. A news article has been described as having been influenced or transformed with edit activities which define the path of provenance to the final version of news as described in [7]. The concepts of dependency provenance for describing event dependencies have been described in [39]. The modelling of a database view as a query tree evaluated in a bottom up fashion has been described in [40]. The identification of trusted groups of users with clustering has been described in [41]. A Hierarchical Solution of Markov Decision Process (MDP) has been described in [30, 42]. The comparison of same edge set in the query proposal graph and the provenance graph has been described with the SEC (same edge contribution) in [31]. A group operator has been described in [43] for defining groups based on actor and its invocation granularity. The situation trust values which have been averaged over all situations to determine the trust in an agent of interest as has been described in [44]. A provenance system has been defined [4, 45] on collection of subjects s, objects o, and attributes a. A subject s1 with attribute set a1 performs an action. A migration policy decision has been taken based on input object o1 and attribute set a1 associated with subject s1 and the state reached as a result of migration. The information transfer model developed in our work which includes network issues has relevance to the work described in [4]. The issue of network congestion and measuring its relevance has been described in [46–48]. The above description summarizes related work in the news reports provenance graph representation and modelling issues.

The application of markov decision process to decision theoretic planning has been described in [10]. The Q Learning based approach for construction of provenance based plan of records has strength of relevance with the reinforcement learning procedure as described in [10]. The concept of tokens received from event and or background topic or other originating topic [49] for producing a new theme describing events or topics [49] has been described in [50].

This method of reward calculation described in this paper has similarity with the precision measure of accurately classifying a target from a source as has been described in [23]. This mathematical model of cost or reward value calculation which has been described in this paper has similarity with a model of action as has been described in [42]. A networked model structure and distributed exchange of information have been considered in [5, 47, 51, 52]. The provenance trust scores of all data passing through a node “A” has been averaged to produce a measure of trust associated with the node “A” as has been described in [53]. A trust measure has been updated for every data or message passing through this link in its provenance path which provides a context for trust calculation. This trust is calculated as the average of the trust values computed from the context provided by every data or message passing through this link. The concept of averaging trust values over contexts has been described in [54]. A procedure of average trust calculation along a provenance or policy path learned with reinforcement learning Algorithm has been described in the context of calculating average reward along a policy path in [55, 56]. A trust value can be calculated [57] as the weighted average of the trust values computed along alternative provenance paths where the weights have been determined from the path length. The Q value has been expressed as a weighted sum of extracted information for each feature as has been described in [58]. The concept of trust computation from record weight and trust of agents has been described in [59]. The above description summarizes related work in the trust or reward value calculation issues.

The evolution of events over time stamps has been used to describe event threads [60] as has been described in [61]. The documents have been selected based on a pre-selected set of features based on time, and then partitioned into clusters of events which have been organized into a hierarchical structure as has been described with threads within [61]. A DAG defining a provenance graph has been utilized to model topic correlations as has been described in [62]. The correlations between few interpretable super topics and their sub topics have been examined in [62]. The above description summarizes the related work in representation of event or topic thread.

The distinctive structural features associated with a social role have been interpreted as “structural signatures” [63]. A concept of group context variable has been described in [64] for determining admissibility of users to a trusted group role with a representation of trust or risk variable corresponding to the role. A distance from ideal measures or the same-as chains property suggested in [65] can also be utilized for setting thresholds in performing clustering of nodes or actor roles. The concept of associating a semantic role for tracking [66] a topic or sub-topic has been described in [67]. The above description summarizes related work in the representation of role within our work.

Related work in story, topic, event

The detection of previously unspecified events using feature pivot techniques has been described in [17]. The cluster centres describe the initial news reports cluster [68]. The different structures or models [68] have been sourced from news reports that have higher degree of “contribution”. Aspect level classification or clustering news reports has been described in [18, 23] which categorizes the news reports into classes or categories for exploring at multiple depths of detail. A likelihood of a particular aspect word in a snippet of information has been identified and the probabilities of words that describe the aspect have been described in [69]. The counts of a shared aspect [70] describing snippets in the corpus of documents have been collected for processing in [69]. The above description summarizes related work in the aspect based topic representation property.

The use of information cascades has been used for tracing the path a piece of news has traversed in the social media graph initiating from the source and traversing the information influencers [6] in the relevant provenance path. Multiple sources can produce stories about the same event which may be grouped for summarization and mining [71] purposes as has been described in [72]. The dependencies can be defined between aspects from which a relative weight-age can be calculated for every aspect describing its importance as has been described in [72]. The theory of additive compositionality for combining several words with an element wise sum of vectors for comparing similarities between whole tweets has been described in [16]. The above description summarizes related work in the dependencies between aspects and the relative weight-age calculation issues of aspects.

A topic has been represented with many sub-topics and as the granularity of an event is too small to describe a topic, and a less granular event has been used to describe only a sub-topic in [73]. The event indexing models has been used for characterizing a topic or story narrative with a plan of event executions as has been described in [74]. A similarity measure between news articles has been defined using weighted similarity of the persons and keywords describing the articles [75]. The topic splitting and merging research where an event causally influences another for establishing links between stories or topics or storylines have been described in [18, 76–80]. There can be multiple storylines related to higher level topics, people, location and time [71]. A storyline has been described [71] on a subset of the relevant topics, for instance a storyline has been described as to cover only the political or the economic aspect of Lehman Brother collapse. A cost has been associated with detection of false link between document pairs where the strength of a link has been described as representative of a measure of cohesion between these documents [76] and this cost or reward value can be utilized for applying reinforcement learning approach. A topic has been described as a seminal event or activity alongside occurrences of other secondary events and activities [81]. The concept of evolutionary discovery of theme which has been interpreted as a semantically coherent topic or subtopic transition appears within [82]. Many themes can be active at an interval of time, and a theme evolution graphs has been represented with arcs connecting a theme to another across the time intervals [82]. The theme evolution arcs can represent threads of themes describing lessons from the event, aids, concerts for the event, personal experience from the event (s), or donation match [60], utilized for describing the topic of Asian Tsunami. Topic specific words can be identified by removing words which appear exceeding a threshold measure defined based on ratio of frequency count of documents in which this word appears and frequency count of documents describing the topic of interest [83]. The above description summarizes related work in the hierarchical topic sub topic and threads of themes or sub-topics representation property.

The weight calculated for an aspect has been described based on frequency of occurrences of terms associated with this aspect interpreted as importance measure as has been described in [12, 18, 72]. The incremental merging of acquired topic tree model for more recent times uses a procedure for detecting the attachment point of new model as has been described in [18]. A Gaussian random markov field approach has been adapted to model correlations between different corpora or document and markov topic model uses this approach to describe topic structure within and across corpora of documents [84]. The ART model [85] describes the per message topic distribution based on author and recipient pair. The high level general topics have been discussed for the entire event, unlike specific topics [86] that have been discussed during specific segment of the event [15]. Thus a general tweet has a weak topical influence from the event, unlike a specific tweet which has a strong topical influence from one segment of the event [15]. A hierarchical clustering from more abstract topics to more concrete topics based on time and conditions defined on aspect attributes has been described in [87]. An event has been described as a series of stories where these stories has been described as having been formed from core stories and their related secondary stories [88]. The development and evolution of these core stories can be described by a number of branches which have been detected from denseness of connected nodes in the neighbourhood [88]. The above description summarizes related work in the dynamics of story or topic representation.

Related work in clustering of nodes in the provenance graph

A clustering based on weight of link or edge connecting event pair as has been described in [89]. A MMHP (Marked Multivariate “Hawkes” Process) algorithm utilizes textual information cluster for activation of events into different clusters and the HTM (“Hawkes” Topic Model) algorithm models the evolution of textual information with Correlated Topic Model (CTM) through cascade of topics as has been described in [90]. A concept of role of macro action has been described in [91] where this macro action satisfies a sub-goal. The clustering of documents based on shared aspect or topic or event or sub-topic, or sub-event has been described in [18, 92–94].

The nodes of high centrality importance has been described as bottleneck states connecting the minimum cut arcs which are members of another cluster or component [95]. The “betweenness” centrality of a node or document can be important in detecting variation of the changes impacted by the cited sources as has been described in [96]. The clustering objective measures like MQ (module quotient) or the Quality Cut Measure has been examined for inclusion of newly identified nodes in a cluster of nodes in a graph. The cluster node expansion and cluster edge expansion measures have been considered in [24] for growing a cluster using a derived cluster node expansion capability measure. A concept of group context variable has been described in [64] for determining admissibility of users to a trusted group role with a representation of trust or risk variable corresponding to the role. A distance from ideal measures or the same-as chains property suggested in [65] can also be utilized for setting thresholds in performing clustering of nodes or actor roles. A concept of role of macro action has been described in [91] where this macro action satisfies a sub-goal with a reward function particular to the sub-goal or sub-topic [97]. The concept of associating a semantic role for tracking a topic or sub-topic has been described in [67]. The paper on clustering of news reports [18] provide a description of tree model construction and merging where the split or merge points have been described as providing a link from past story to more recent story description. Aspect based regression tree clustering described in [18] produces clusters of documents as leaf nodes, where every leaf node can be associated with a news topic or a news category [21] or a news issue [22]. The above description summarizes related work in the clustering of topic or sub topic or event nodes, both within a provenance graph model or across a pair of such models.

A recursive partitioning approach has been adopted instead of a mixture model for satisfying the conditions of interactions of variables, or for detecting these interactions, or for generating scarce interaction patterns, or for selecting these variables without requiring prior specification of these variables [98]. A recursive splitting of cluster nodes describing a regression tree as has been described in [18] reiterates these advantages. The documents have been analyzed for identifying a list of candidates for a target document based on titles similarity, content similarity, unique words and frequents words [99]. The importance measure of a term can be utilized for computing the similarity or distinctive relation between documents which signifies that these documents are describing a collection of similar or discerning topics. The “betweenness” centrality measures the importance of a node in the cluster. The degree centrality has been considered as not sufficient as a measure for the whole network and closeness centrality has not always proven as the ideal measure for measuring centrality. The importance of a node or story or event depends on the interpretation provided in describing the topic graph model and has been measured with the use of between-“ness” centrality in this paper. The key phrases which have been represented in these centrally important nodes have been considered as more relevant in constructing a policy path plan routed through these nodes. Similar articles or articles describing similar events and which have been clustered to produce an event class trigger have been used for describing events in the same class [100]. A Hierarchical Topic Detection algorithm has been described in [101] where a theme area comprising of the title and the initial paragraph of the document has been defined and the detail area has been used for additional details about the topic or subtopic of the topic. The network metrices have been evaluated in [102]. The centrality has been described as highly characteristic of a hierarchical network, where patterns have been described as centralized on few individuals who attract attention from other network nodes [102]. The high density networks have been distinguished in terms of level of modularity which is the measure of interconnectedness of clusters [103]. However the clusters defined based on degree centrality measuring the interconnectedness between network nodes does not prove as sufficient for the entire cluster. The relevant algorithm examine large data sets and efficiently find subgroups and the algorithm uses edge “betweenness” as a metric for identifying boundaries of communities as has been described in [103]. The in-group social media network structure has users with unified interests, while networks with clustered communities [104] limit information flow to small silos of users which have been described as stable over time [103]. The distinctive structural features associated with a social role have been interpreted as “structural signatures” [63]. The above description summarizes related work in the “centrality” issues in clustering.

A cluster has been defined for news stories occurring at a snapshot interval of time which describe the same sub goal or sub topic of the goal topic [105] or whole story [106]. A clustering of events has been defined on the commonality of attributes describing events, where exact match of attributes causes events to be positioned in the same cluster, whereas a partial match causes a link to be described between events across their corresponding clusters as has been described in [106]. The story clusters have been defined for their corresponding snapshot time intervals and have been linked based on their similarity measures like Jaccard index, Sorensen-Dice coefficient or similarities defined based on measure of inclusion or exclusion. A k-modes clustering algorithm has been described in [32] for categorical data. A number of cluster analysis methods has been described in [32] for identifying the number of clusters, which have been defined based on gap statistics, and other measures based on information criteria such as AIC (Akaike Information Criteria) or BIC (Bayes Information Criteria). The above description summarizes related work in the other clustering methods applicable to both within and across provenance graph models.

Related work in classifier learning

A reinforcement learning procedure has been adopted for identifying the policy describing the narrative path for the whole story which can also represent the path from start topic or event to goal topic [107] or event. Machine learning algorithm such as reinforcement learning has been utilized in the task of Text Mining [108] applications for learning aspect or topic of document or story [109] as tasks in computing long term rewards and thus making these methods applicable to Big Data Analytics [71]. A reinforcement learning procedure has been applied to an aspect based representation of a data base of stories which have been considered as relevant to the news reports analysis or econometric problem domain as has been described in [18, 110]. Reinforcement learning method has been utilized for finding the stochastic shortest path in scheduling of tasks for satisfying the quality of service constraints [46] in the presence of drift of concept [111] associated with the incoming data in Big-Data [71] Streaming Applications as has been described in [112]. Q Learning procedure can be applied to produce a plan [10] of event or topic executions [74] from the start topic or event [107] to the goal topic or event in [107]. Q Learning is a model free reinforcement learning method. The Q Learning procedure has been described as a topic in reinforcement learning in [43]. Q Learning procedure considers the weighted average of rewards present in a policy path as compared to reinforcement learning approach which considers a simple average of reward in a policy path [43]. The above description summarizes the related work in reinforcement learning, Q Learning and Big Data Applications.

The main difference between Bayesian and non Bayesian methods is the use of priors. The prior for a Bayesian network structure can be converted to the priors for an equivalent Bayesian structure by application of change of variables using a method of Jacobian Transformation as has been described in [113]. The concept of score for belief network has been presented in [114, 115]. The scores of two isomorphic belief networks must be considered as equal. A Bayesian metric with Dirichlet priors (BD) has been presented in [116] for calculating a score. A score equivalent BD metric has been presented (BDe) for identifying score equivalent Bayesian network structures. Such score equivalent metrics can be used for identifying equivalent task structure network for provenance path learning. A method based on scoring equivalent class operators has been described in [116]. A search algorithm has been described that moves along structures with the application of these equivalence class operators. A greedy algorithm has been applied over a local subspace which produces better results [117]. A unit task or activity node which has been added to or removed from a network produces the maximum improvement in this score [118]. However the greedy hill climbing approach can result in search getting trapped in local maxima thus requiring backtracking and restarts. An adaptive simulated annealing approach may also be applicable here [119]. The above description summarizes the relevance of Bayesian learning issues to our work.

The process of adapting classifier to the situation of change has been described in [120]. A concept drift can happen if prior probabilities of classes defined on target variables change, or if conditional probabilities associating the target class variable and input variable changes, with a corresponding impact of change on posterior probabilities [120]. The ensemble learning methods produce models that are either homogenous or heterogeneous. Homogenous algorithms which use the same algorithm with different parameter either by introducing randomness or by manipulating input attributes, model outputs or training instances with a process of “bagging” or “boosting” as has been described in [121]. The explore vs exploit strategies for selecting actions has been described in [120]. Reinforcement learning algorithm has been described as where all state action pairs have been observed and top updated Q values have been retained for future processing as has been described in [122]. The reward function values impact distance measures used by the clustering algorithm [122]. The three heuristics for credit assignments [27] have been described in [123]. Constraint Satisfaction and Emerging Algorithm based on Set Covering Problem Solutions have been described in [124]. A survey of set covering problem solutions has been provided in [33]. A combination of partitioning and agglomerative clustering algorithm has been described in [35] where the data points have been partitioned and then clustered maintaining the total number of identified partitions. A method of merging models using agglomerative clustering approach where models have been merged based on a maximum classification likelihood measure has been described in [36]. The above description summarizes the related work in merging graph models.

The ideas described in this paper where the graph models are either first partitioned using a partitioning approach like PAM or CLARA and the representative models in each cluster are merged using either a Bayesian scoring approach or from using their property of markov equivalence thus producing an essential model have similarities with ideas discussed in [35] and in [36]. The distance between a pair of graph models has been described using a graph edit distance measure in [125].

A cluster has been defined for news stories occurring at a snapshot interval of time which describe the same sub goal or sub topic of the goal topic or the whole story [106]. The concept of Structured Stories has been described in [126] where a semantic zoom provides drill down into detail of a specific event in the story. An event within the story can be linked to another story containing a detailed narrative of the event [126]. A story has been described as a sequence of story fragments where a story fragment can act as a bridge based on term context dependent attractiveness between start topic and the end topic [107]. The concept of bridging topic has been described in the story of Lehman Brother Collapse which has been described using the Political Aspect or Topic and Economic Aspect or Topic and this can provide an example to our approach of describing splitting or merging using bridging topic in story description.

A window length has been defined for training the topic model which has been specified using a time range and a refresh rate of length less than the window length for considering if the training instances are old [127]. The model used for event similarity calculation lags an interval of refresh rate in minutes [127]. As the volume of tweets has been considered as not uniform the refresh rate compensates for non uniformity of the count of words. Alternatively, a sliding window has been updated every 15 min, and the model has been retrained with tweets of the past 24 h [127]. These are relevant in considering lengths of incremental time intervals for constructing model.

Clustering and topic models have been integrated into representing a storyline where there is a probability computed for assigning a new document to an existing storyline or a new storyline [128]. A storyline can represent a higher level topic where a link has been established to this topic for preserving this storyline [128]. A cluster has been identified for a newly acquired document based on either a hard decision where the Bayesian cluster models have been updated with assignment of a document to a cluster with the highest probability or a soft probabilistic updating has been performed. A cosine similarity value between topic pairs has been utilized to identify a bump in the cosine similarity graph for interpreting event topics [129]. The localized high cosine similarity bumps have been used for interpreting event topics, where a random change in the similarity value can indicate events that are not time specific [129]. An event topic present in another event topic’s hot zone or with high similarity value have been linked together in the graphical representation and grouped together to represent list of sets of event topics [129]. In the genomics collection, divergence of topic stories from the general collection, can indicate that subsequent stories have been associated with a new topic within a new introduced time window [130]. The candidate models participating in the procedure for merging have policy path classifiers which have been described as either only member of any set of similar classifiers or are members in the minimum cover set [33] of such similar classifiers that are present in the cluster. The above description summarizes related work in the Bayesian Model Merging and minimum cover set merging methods.

Problem statement and solution

News reports describing event(s) are associated with or are routed to an intermediate node in a network of nodes describing a provenance graph where each node in this graph is associated with an activity. Every news report is presented by a user whose trustworthiness is measured with a computational trust value. This activity describes a “match and extract” procedure for transforming these input reports to the destination activity or event or topic node thus providing a more detailed version of the narrative. This search and extract procedure which uses the importance measure associated with individual attributes or records describing the news reports and the computational trust measure associated with their originators have been described in “Development of relevance of computational trust” section. The relevance of the clustering to this work has been described in “Related work in clustering of nodes in the provenance graph” section and our approach to clustering of nodes has been described in “Our solution to clustering nodes in provenance graph” section. This procedure of learning the path in a provenance graph of activity or event nodes has been described in “Rationale for application of Q Learning algorithm in this trust based representation” section. The rationale for application of Q Learning algorithm to the Trust Provenance Graph Representation has been described in “Application of Q Learning algorithm for learning provenance path” section. The provenance graph models thus constructed from identified dependencies between events or activities is relevant for an interval of time window. These graph models are merged to produce consistent models spanning larger intervals of time windows which apply also to incrementally acquired models for more recent time interval windows. The narrative of splitting and merging for producing the goal topic or whole story description in these identified merged models have been described in “Our approach to provenance graph structure and classifier learning” section.

A block diagram or the design of the solution steps has been presented in Fig. 1. The examples in support of the design have been presented in this paper in Figs. 2 and 3. The Theme or Event or Topic or Subtopic evolution threads describe the Asian Tsumami depicted in Fig. 1. Here for example, Lessons from Event has been described with the Theme evolution Theme 1 → Theme 1 → Theme 4. Possible threads for Aids include Theme 4 → Theme 3 → Theme 5 and thread for Personal Experience include Theme 5 → Theme 6 → Theme 3 → Theme 3. The confluence or merging of Theme 4 and Theme 5 into Theme 6 and then evolving into Theme 3 described the Personal Experience thread as Theme 4 → Theme 6 → Theme 3 → Theme 3 or as Theme 5 → Theme 6 → Theme 3 → Theme 3. Theme 1 at time interval 4 splits into and evolves towards Theme 1 at time 5 and Theme 6 at time 5, corresponding to Lessons from the event thread and Criticism on Iraq thread. The learning or construction of provenance path to a goal path discovery has been described in Fig. 3 as exemplified in [131].

Development of relevance of computational trust

The value is normalized with a division by the summation of such probabilities identified for all attributes of the source record.

A measure can be associated with any field value based on the occurrence count of this value in the news report from where this record is represented. This can be calculated based on the iteratively developed conditional probability of a field token value given an event as a maximization step of EM algorithm solution as described in [132]. The EM algorithm maximizes the posterior probability of occurrence of the event.

This probability value can also be calculated by a beta binomial method. This can also be calculated as an optimal score approach where the conditional probability of the field token value is calculated based on defined number of past observations, and observations from news reports with application of appropriate weight-age to each that minimize a posterior log likelihood measure as described in [133].

This w value determines the posterior probability of entity of interest.

The reward received at node B is COMPT_AB where the composite trust in A after interaction with “B” [40] which has been derived using Eq. 10 from a choice condition determined from application of Eq. 9.

The unit step reward for corresponding p_AB(n) is valid for n = 1.

However multi step reward calculated for such node pair (A,B) = p_AB(n) for n ≥ 1, which is symbolized as p_ABt and is derived using application of Eqs. 13, 14, 15 and 16 appearing later in this document.

A similarity measure Sim(i, j) between two records i and j is defined where each of these records are described by k attributes and attr_in_merge(j) = 1, indicates if the jth attribute values of the records have similarity value > 0, else attr_in_merge(j) = 0.

The procedure of cumulating occurrence counts of token or evidence for merging information from multiple sources has also been discussed in [135]. The choice of routing or forwarding of information based on reward calculated on aggregation based on data correlation and gain in reward or diminishment of distance measure to the goal node has been described in [136].

This net similarity measure calculated using Eq. 9, is used for determining the appropriate probability disjunction strategy. Here the probabilistic trust values pA and pB associated with individual source records participating in the merge are calculated from applying Eqs. 1 and 7.

Where if the net similarity measure > high threshold, probabilistic disjunctive strategy [137] is adoped for the positively correlated case and the Net_trust is min(1, pA + pB). else if the net similarity measure = 0.5, the probabilistic disjunctive strategy [12] is adopted for the independent case and the Net_trust is \( pA + pB\;{-}\;pA\;*\;pB. \) else if the net similarity measure < low threshold and > minimum threshold, probabilistic disjunctive strategy is adopted for the negatively correlated case [12] and the Net_trust is min(1, pA + pB). else If net similarity measure < minimum threshold, probabilistic disjunctive strategy is adopted for the ignorance case and the Net trust is max(pA, pB).

This net trust value COMPT_AB or the reward received at node A is the composite or net trust in the event after merging the message records from pair of sources using method as described in Eq. 10 of this section. This measure considers the importance of similar sources in a cluster alongside sources with the news report “contribution” [138] or measure of distinctive relevance of sources in the reward value computation.

Rationale for application of Q Learning algorithm in this trust based representation

The agent selects an action probabilistically based on Boltzmann distribution defined using Q value probabilities associating states with actions [139]. The Q values are associated with policy of actions and thus justifying the use of provenance path probability values in determining action probabilities.

The transition probability p_ij between states i and j = exp ((f(j) − f(i))/t) has been described in [55]. The cost functions associated with states i and j are represented as f(i) and f(j) representing their energy states. The value of a policy is the sum of rewards obtained from execution of actions described in the policy. The Q Learning algorithm selects a policy with a higher reward. The difference (f(j) − f(i)) has been calculated from the optimum value of the policy connecting these states i and j. As a policy with higher reward (dominant policy) has always been selected over less rewarding policy [121] the procedure described for calculating reward value (Eq. 16) from p_AB (d) is appropriate here. Thus this procedure favours moves between similar or relevant nodes or states.

Only sub-topic sub-goal states of each cluster have been identified using application of method described in [24] and are retained for the re-computation of probabilistic weight connecting pair of nodes in the reduced clustered state space. The detected node pairs defining the provenance graph in the reduced cluster space are utilized to learn the policy path with application of Q Learning method. The Q Learning approach adopted is unique to our work in the news reports modelling application as the approach provides more weight-age to rewards from moves taken when the search space is less localized and distant from the goal while calculating a discounted sum of rewards present in the trust path to goal. The rewarding moves between nodes or states in a reduced clustered space using a Q Learning approach provides more weight-age to early moves based on relevance or contribution of information associating or linking these nodes as this information is of more value as compared to that in proximity with the goal where the learning or the search has already been localized. Also final steps or edges in the path are associated with excessive granular representation of events which are discounted or attached a lesser weight in computing the weighted average value of reward from start node to goal node.

Application of Q Learning algorithm for learning provenance path

The Q Learning procedure is described as follows. Initialize V(s) for all states using initial network congestion conditions and using procedures described in Eqs. 1, 7 and 10. The network nodes are considered to be connected by links based on probabilities determined from current congestion situation as described in Eq. 8. Here s is the set of nodes, that are base database [140] relation sites, where record merging operation is applied on records arriving from two or more network sites.

Here the transition probability T(s,a,s’) is determined from application of Eq. 16, and R(s,a) is the trust in the component of information contributed by state s in performing the transformation step at a state s’ reached from forwarding action of data or information packet and this is determined from application of Eq. 16 after reaching state s’ as described earlier.

The policy learned is considered to be “good” if the magnitude of the updated Value attributes are less than an acceptable threshold different from the magnitude of these attribute value prior to update. The greedy policy finds the optimal policy in finite number of steps sometimes earlier than convergence of iterated value function. Here on arriving at a better optimal policy, all the previously learned policy paths including the latest that are covered are ignored and a new policy which is sub-optimal but is presently optimal is learned. This procedure continues until all policy paths satisfying the goal are learned.

Thus a message is transferred through a selected policy path of nodes in a provenance network for maximizing the iterated value described for a node in the graph.

A composite trust value of a node “A” is calculated from its content importance, which includes the trust of the subject associated with the node and importance of the node. The change of importance measure between a node “A” of interest and active node “B” in the provenance path after traversing n steps in the provenance path is calculated where node B is active after executing n provenance steps.

A provenance trust score is calculated for every data “d” passing through this node “A”. This is calculated as minimum or average of the trust values associated with nodes in the provenance path. The provenance trust scores of all data passing through this node “A” are averaged to produce a measure of trust associated with node “A”.

The subject node “B” is included in the same cluster as the cluster identified for subject node “A” if the following conditions are satisfied.

The weight of source record at “A” after satisfying constraints associated with merging with record at “B” is w_AB and is calculated using Eq. 7. The weight measure is further normalized by maximum w_AB calculated for all links describing the provenance graph.

Alternatively p_AB(2) can also be calculated as the average or the weighted average which is of more relevance in the adopted of Q Learning procedure of the trust transitions p_AC(1) and p_CB(1) if we adopt a procedure described in [44].

If P_ABt > threshold value these nodes are positioned in the same cluster P_ABt which is the derived trust value of “A” to “B” obtained using distinct provenance paths (path_i) for a data input “d” to the stream.

Alternatively this trust value can be calculated as the weighted average of the trust values computed along the alternative paths where the weights are determined from the path length. Here this may be noteworthy that the links nearer to the start node contribute heavily to the net path trust value connecting any pair of start and goal nodes in the provenance path.

This is also described as to the procedure of combination of path trust values in [44].

After every updated policy identification round, the revised trust values of agents/actors associated with nodes in the provenance network are propagated and updated according to the trust propagation rules described for social networks in [141].

Our solution to clustering nodes in provenance graph

The adjacent clusters/modules are merged based on the following procedure. A module score or a cluster score is defined for each module group in a workflow [45] based on distances of module element pairs contained in the module group. The workflow score is defined as the sum of the module group scores. A greedy strategy is adopted which merges two adjacent module groups with best workflow score. This process continues until only a predefined number of module groups remain. The adjacent cluster components are merged if these identified sub-goal nodes corresponding to the clusters are the same. A concept of role of macro action is applicable where this macro action satisfies a sub-goal and where a reward function may be used which is particular to the sub-topic sub-goal. The roles corresponding to the macro actions satisfies the same sub-topic or sub-goal role and have correlated topic models describing the connected event nodes present in the sub-goal or sub-topic cluster. This role can be interpreted as the semantic role or users of the same role having the same or similar “structural signatures” or users having a trusted group [41] role corresponding to the sub-topic cluster associated with tracking a particular topic or sub-topic or sub-goal of the story. All agents associated with a macro action may be interpreted as playing an informing role to the evaluating agent associated with the sub-topic sub-goal agent. As a consequence of merging of clusters, a hierarchical clustering of sub-topic sub-goal nodes is achieved where sub-goal nodes inferred from recursively defined merged clusters may indicate split or merge points in the goal topic or whole story description. Here a merging of clusters may aim at forming a merged cluster which have nodes in the merged cluster of similar Q values or Reward values as calculated from application of Eq. 16. The actions connecting the sub-topic sub-goal nodes of finally produced clusters are the macro actions on which the Q Learning procedure is re-applied. The distance measure defined on agent roles and trust links in combination with the reward based Q value approach produces a clustering where every cluster has nodes with same or similar agent roles [64].

This condition for merging of clusters is applied with the module score approach to obtain the final clustering. The probabilistic trust weight connecting the sub-goal nodes has been described as a macro action for identifying clusters which are updated using principles described earlier. A role is associated with interpreting a story theme or topic or subtopic or aspect of a story which may be derived from input vector of aspects.

An alternative clustering or classification approach has been described as follows. The examples associated with a state can be classified into one or more identified categories. The full set of states is subdivided into smaller set of states to reduce the entropy measure defined on categories. An action is described on a state with an attribute which is used to further classify the subset of examples in the state. The Information Gain functions that are used are Entropy based, Gini Index based and Discriminant power function based [expected count/(total count)]. The leaf nodes of this decision tree must satisfy the constraint that all activity or tasks nodes which are members of any leaf cluster node are executed by agents with the same role.

Here for measuring the applicability of our approach, a module group score can be defined for a group of module element nodes based on Joint Entropy distance [139] separating every pair of nodes (A, B) defined on this derived probabilistic weight P_ABc as discussed in Eq. 16.

This distance measure is further normalized by the maximum distance value max (D_ij) for the provenance graph model. Thus D_ij = D_ij/max(D_ij).

Thus quality of identified grouping or clustering of nodes in a module group can be represented using this distance measure and the difference from the weighted average distance calculated between every pair of nodes in the identified cluster. If this difference is lesser than a threshold value, the nodes are considered as elements of the same cluster. The adjacent groups of modules can be merged that satisfies a workflow score constraint as discussed earlier for a pre-specified limit on the number of clusters/groups in workflow.

Rationale for provenance graph structure and classifier learning

Only those events have been considered that have one or more of the necessary seed terms that have been used to describe a topic of interest to the user. Here the topic specific words that remove other words commonly occurring across topics have been identified using procedure described in [83]. The identification of story clusters for snapshot intervals of time and procedure for linking these clusters have been described in [106]. The models thus identified may be merged based on their importance in description of the story. Here interpreting clustering of documents for representing model cluster and Bayesian probabilistic approach [142] of assignment of document to a model cluster as has described in [143] are relevant. The application driven text data source can be static or dynamic where a static source implies that the document collection as not having frequent updates, while other text streams can be characterized as having many updates [144]. The topic modelling techniques have also been adapted with respect to the temporal scale for narrowing down events to fine granularity [144]. Hierarchical topics provide an overview of topics from one corpora [34]. The method suggested in [34] provides a full picture of topics from multiple corpora which can represent time updated versions of earlier corpora, where the hierarchical topic models have been merged based on graph matching methods [34], such as graph edit distance and other such methods [34]. A phrase reinforcement learning has been proposed in [145] where a starting phrase represents the topic for which generating a summary of tweets has been proposed, and this best partial summary represents the selection of path with the maximum sum of weights along the path [145]. The above description summarizes related work in the merging of topic models and representing summary policy path.

The classifier policy path has been detected from this structure by application of Q Learning algorithm for learning provenance path as has been described in an earlier section. Q Learning procedure has earlier been described to produce a plan [10] of event or topic executions [74] from the start topic or event [107] to the goal topic or event [107].

Provenance graph model(s) have been constructed for time window (s) which have been determined from window length [127] and refresh rate [127]. The time windows can also be defined based on a concept of sliding window [127] with overlaps between the models defined for these time windows.

The integer linear programming, constraint satisfaction, and other emerging algorithm based set covering problem solutions have been described as relevant for generating a cover set of classifiers for the models. A classifier in a model is compared with all classifiers in every other model learned and the best match is considered for calculating significance of a classifier [146]. Each one of these significant classifiers present in the minimum cover set and which are associated with one or more model (s) are compared with all such significant classifiers which are associated with other such model (s) and a similarity measure between these models has been computed from the classifiers present in these models using a correlation ratio measure as described in [147]. Alternatively, this similarity measure can be calculated as Pearson Product moment correlation measure. A principle of covariance measure defined between a single value and a vector of values is the sum of the covariance measures calculated between the single value and each element of the vector. Alternatively a distance or similarity measure can be described between a pair of these models using a graph edit distance measure as has been described in [125]. The models are clustered using pair-wise similarity or distance measures between a model pair described by their representative classifiers as described earlier. A hierarchical agglomerative Approach [148] can be adopted for this with merging of most similar model pairs at every hierarchy where a merged model can been derived using procedures as has been described later in Steps 9.2, 9.3 and 11. The above description summarizes the relevance of covariance or correlation based clustering methods to our work.

A partitioning approach can be adopted for this model merging using the K-medoids approach where a model has been selected as representative for the cluster of models and this clustering method is realized with a Partitioning around “medoids” (PAM) or Clustering Large Applications (CLARA). The algorithm described as PAM starts with randomly collected seed models and improves the clustering with a greedy strategy by randomly selecting a model as a “medoid” which reduces the measure associated with the absolute error criterion [148] representing the sum of dissimilarities. Here all models present in a cluster are merged. Alternatively, models are merged from using their property of markov equivalence producing an essential model. These have similarities with ideas appearing in [35, 36]. The algorithm described in Step 11 has been utilized to complete the definition of merged model. PAM (partitioning around “medoids”), a medoid based clustering algorithm which has been cited in our work is less influenced by outliers. Our representation of record instances for a time interval with a provenance graph model reduces the cluster space and makes it feasible for application of PAM. For large data sets, a sampling based method called CLARA can be used where after sampling the clustering methodology PAM has been applied to detect the best “medoid”. A “medoid” based approach to clustering of points described in PAM, CLARA or CLARANS where two clusters has been be merged based on the farthest distance between two points in the cluster pair and this tested merged cluster satisfies less than a certain threshold value for the diameter measure. The revised centroid point has been identified from merging of clusters. This centroid point has a maximum distance measure value to a point in the merged cluster and satisfies a magnitude less than a threshold value in radius measure. The above description summarizes the database clustering methods such as PAM, CLARA, CLARANS which are relevant to our work.

Also for the cause of merging models, the “medoid” model in a cluster has been selected as the initial model. The other present models in the cluster has been merged iteratively such that the intermediate model minimally increases the error defined based on a maximum likelihood measure as has been described in [149]. Alternatively a Bayesian Model Merging principle has been adopted such that product of model prior and likelihood measure associated with the models has a maximum value [149]. Also the models can be ordered based on their relevance to a subject or topic using principles of Bayesian Factor or Bayesian sampling approaches as has been described in [150] and the models can be merged in an appropriate order within a cluster. Here a unified provenance graph model is thus constructed where model unification procedure has been applied for merging element models from incrementally acquired information obtained at more recent time intervals as has been described earlier. A merging of time series data using principles of Dynamic Time warping (DTW) has been described in [151] where the time series pair participating in the merge have been optimally aligned using principles of Dynamic Programming if the lengths of the time series pair have not been observed as same. A cluster has been represented by time series data which have not been considered as similar in nature and a representative time series has been derived from the time series data present in the cluster that considers the DTW distance to identify the closest time series [151]. The above description summarizes related work in merging models within a cluster using Bayesian Factor, or using methods for Time Series Merging based on Dynamic Time warping.

A hierarchical agglomerative clustering approach appearing in [11] has been adopted with merging of most similar model pairs at every hierarchy where a merged model has been derived using a procedure described later in Steps 9.2, 9.3 and 11. BOAT uses attribute selection method like Gini index which has been used for constructing for Regression Trees.

Boosting is a method of combining ensemble classifiers created from a weighted version of learning sample, where weights have been adjusted at each step to provide increased weight to cases misclassified earlier. Adapting re-sampling has been identified as the key to success with misclassified cases receiving larger weights in the next step. Bagging has been applied to larger trees in contrast to boosting that works well with stumps or slightly larger trees [152]. The idea of combining ensemble of classifiers using a weighted version of each as has been described in boosting and this has interpretation of relevance to our work where earlier steps of learning have been provided more weight-age than those derived later and the significant classifiers thus derived have more weight-age value. Also the start nodes describing a topic in the Phrase Reinforcement Learning in learning of topic description as has been described in [145]. The above description summarizes the relevance of Boosting over Bagging for combining ensemble classifiers.

A reinforcement learning approach has been described as providing a balance between pruning for generalization and growing deeper trees for accuracy [153]. A continuous U tree algorithm transfers traditional U tree algorithm to reinforcement learning and this U tree algorithm can be viewed as a Regression Tree algorithm for storing state values [154]. The regression clustering (CART) with splits satisfying a maximum gain measure as modelled by ginni coefficient describes a probabilistic measure modelling the fraction of points of the predecessor nodes that are present in the one or the other successor node. The above description summarizes the relevance of U tree algorithm and CART algorithm to our work.

Structural Regression Trees integrates the regression method of learning into inductive logic programming [155]. This method however produces a solution to the Relational Regression Problems which have been difficult to understand, and assumes that all features are equally relevant to all parts of instance space, and also does not have easy utilization of domain space [155]. The SRT method has a simple method of tree selection based on Minimum Description Length (MDL) [113] principle. The MDL algorithm measures the simplicity and accuracy of the theory and data. The theory description length has been derived from encoding of literals and encoding of the predicted values in leaves [155]. The data length has been derived from encoding of errors [155]. A model has been selected with minimum message length associated with the sum of theory message length and data message length of the model [155]. The balance provided by Regression Tree approach [153] with the methodology of error complexity pruning and growing deeper tree for accuracy has similarity with the MDL based approach to learning as has been described in [155]. The interaction network summarization has been described with independent topical events that are temporally and topically coherent and this interaction network has been summarized by large events [156]. A collection of k-events has been selected that maximizes the node coverage and this task maps to the finding the maximum set cover solution [156]. The above description summarizes related work on the MDL principle and maximum cover set solutions for representing models.

Our approach to provenance graph structure and classifier learning