Neural networks that are able to retrieve store and retrieve information constitue an old but still active area of research. Among the different existing architectures, recurrent networks that combine as-sociative memory with error correcting properties based on cliques have recently shown good performances on storing arbitrary random messages. However, they fail in scaling up to large dimensions data such as images, mostly because the distribution of activated neurons is not uniform in the network. We propose in this paper a new penalization term that alleviates this problem, and shows its efficiency on partially erased images reconstruction problem.