DEvIANT : Discovering significant exceptional (dis)agreement within groups
Abstract
We consider any type of data featuring individuals (e.g., parliamentarians , customers) performing observable actions (e.g., votes, ratings) on entities (e.g., legislative procedures, movies). In such data, we aim to find contexts (i.e. subgroup of entities) for which an exceptional (dis)agreement is observed among a group of individuals. To this end, we introduce the novel problem of discovering statistically significant exceptional contextual intra-group agreement patterns. To handle the data sparsity, we use the Krippendorff's Alpha measure to assess the agreement among individuals. We devise a branch-and-bound algorithm, named DEvIANT, to discover such patterns. DEvIANT exploits both closure operators and tight optimistic estimates. We derive analytic approximations for the confidence intervals (CIs) associated to patterns for a computationally efficient significance assessment. We prove that these approximate CIs are nested along specialization of patterns. This makes it possible to incorporate pruning properties in the algorithm to early discard non-significant patterns. Empirical study on several datasets demonstrates the efficiency and the usefulness of DEvIANT.
Origin | Files produced by the author(s) |
---|
Loading...