Force Fields (FFs) are an established tool for simulating large and complex molecular systems. However, parametrizing FFs is a challenging and time-consuming task that relies on empirical heuristics, experimental data, and computational data. Recent efforts aim to automate the assignment of FF parameters using pre-existing databases and on-the-fly \textit{ab-initio} data. In this study, we propose a Graph-Based Force Fields (GB-FFs) model to directly derive parameters for the Generalized Amber Force Field (GAFF) from chemical environments and research into the influence of functional forms. Our end-to-end parameterization approach eliminates the need for expert-defined procedures and enhances the accuracy and transferability of GAFF across a broader range of molecular complexes. The GB-FFs model, which is only grounded on ab initio data, is implemented in the highly parallel Tinker-HP GPU package. Simulation results are compared to the original GAFF parameterization and validated on various experimentally and computationally derived properties, including free energies.