Non-binary LDPC codes are now recognized as a potential competitor to binary coded solutions, especially when the codeword length is small or moderate. More and more works are reported with good performance/complexity tradeoffs, which make non-binary solutions interesting for practical applications, such as 4G-wireless systems or DVB-like systems. In this paper, we show that proposing non-binary LDPC codes built on finite fields is actually a limitation, both from performance and implementation points of view. By considering non-binary codes on the general linear group, we show in particular that a slight performance improvement can be obtained, compared to Galois Field codes, with reasonable additional cost in the hardware implementation. The performance gain is quite small, but comes at a slight extra decoding cost, and is obtained by proper generalization of the code optimization techniques that are standard for non-binary LDPC codes on fields.