Clustering Short Text and Its Evaluation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2012

Clustering Short Text and Its Evaluation

Résumé

Recently there has been an increase in interest towards clustering short text because it could be used in many NLP applications. According to the application, a variety of short text could be defined mainly in terms of their length (e.g. sentence, paragraphs) and type (e.g. scientific papers, newspapers). Finding a clustering method that is able to cluster short text in general is difficult. In this paper, we cluster 4 different corpora with different types of text with varying length and evaluate them against the gold standard. Based on these clustering experiments, we show how different similarity measures, clustering algorithms, and cluster evaluation methods effect the resulting clusters. We discuss four existing corpus based similarity methods, Cosine similarity, Latent Semantic Analysis, Short text Vector Space Model, and Kullback-Leibler distance, four well known clustering methods, Complete Link, Single Link, Average Link hierarchical clustering and Spectral clustering, and three evaluation methods, clustering F-measure, adjusted Rand Index, and V. Our experiments show that corpus based similarity measures do not significantly affect the clusters and that the performance of spectral clustering is better than hierarchical clustering. We also show that the values given by the evaluation methods do not always represent the usability of the clusters.
Fichier non déposé

Dates et versions

hal-01147116 , version 1 (29-04-2015)

Identifiants

Citer

Prajol Shrestha, Christine Jacquin, Beatrice Daille. Clustering Short Text and Its Evaluation. 13th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing), Mar 2012, New-Delhi, India. pp.169-180, ⟨10.1007/978-3-642-28601-8_15⟩. ⟨hal-01147116⟩
80 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More