With Lattice-based cryptography (LBC), ciphertexts are represented as points near a lattice, and Babai’s round-off algorithm allows to decrypt them when one knows the secretkey. Recently, an accelerated variant of the round-off, based on Residue Number Systems (RNSs), has been proposed. Herein, we combine this technique with the use of lattices of Optimal Hermite Normal Form (OHNF) and propose further refinements, so as to reduce the decryption complexity. This approach lends itself largely to data-level parallelism, allowing for low latency decryption operations on multi-core CPUS with Single Instruction Multiple Data (SIMD) extensions, and achieves high-throughput on GPUs. Finally, we are able to perform decryptions up to 20 times faster than the most efficient implementation in related art, which exploits the Mixed-Radix System (MRS), in an Intel i7 6700K CPU, and we are able to decrypt up to 11832 messages/s in a Titan X GPU.