In this work, we present a new method for clustering words into equivalence classes within a bigram word (which is extensible to a trigram word). This method is based upon an improved approach of the simulated annealing technique. The main idea consists in defining a transition function instead of a random transition function as usually used in classical approaches. Our transition function specifies the way the classes may evolve during the simulating annealing process. It is based on a set of stochastic rules based on linguistic criteria which select the set of words that can be moved from one class to another. This method allows to palliate the main drawbacks of the classical approach: time complexity and random transition function. In this paper, we compare the performances of both approaches: the classical simulated annealing and the proposed one and we report the results of an evaluation on two French databases. The classical approach is 2,5 times slower than our approach and despite of a low value of perplexity the classification obtained is not as good as the one given by our approach.