As Computer Supported Collaborative Learning (CSCL) gains a broader usage in multiple educational scenarios facilitated by the use of technology, the need for automated tools capable of detecting and stimulating collaboration increases. We propose a computational model using ReaderBench that assesses collaboration from a dialogical perspective. Accordingly, collaboration emerges from the intertwining of different points of view or, more specifically, from the inter-animation of voices pertaining to different speakers. Collaboration is determined from the intertwining or overlap of voices emitted by different participants throughout the ongoing conversation. This study presents a validation of this model consisting of a comparison between the output of our system and human evaluations of 10 chat conversations, selected from a corpus of more than 100 chats, in which Computer Science students debated on the advantages and disadvantages of CSCL technologies (e.g., chat, blog, wiki, forum, or Google Wave). The human evaluations of the degree of collaboration between the participants and the automated scores showed good overlap as measured by precision, recall, and F1 scores. Our overarching conclusion is that dialogism derived from the overlapping of voices can be perceived as a signature for collaboration.