This paper deals with clustering applications, it proposes an on the fly algorithm for distributing graphs. The proposed algorithm is based on the meta-heuristic "Particle Swarm Optimization (PSO)". For the sake of presentation, the idea of adapting the PSO algorithm for graphs distribution is developed firstly by assuming that the graph is already generated. After that a distributed algorithm is proposed for concurrently generating and distributing graphs. This distribution should ensure the workload balancing property and the minimization of the number of distributed inter-site edges.