Long Range (LoRa) is a proprietary modulation technique that uses Chirp Spread Spectrum (CSS) modulation for low power and wide area communications. Despite the advantages of LoRa technology, the reconfiguration of transmission parameters such as Spreading Factor (SF) and Transmission Power ( ), remains limited to maximize the uplink traffic. In this paper, we look upon additional parameters such as the Bandwidth (BW) and the Coding Rate (CR). We apply Fuzzy C-Means (FCM) algorithm to acquire knowledge about the quality of each transmission setting. Then, we use this knowledge in Q-learning and Markov Decision Process (MDP) algorithms as a state transition matrix to converge better and faster to the set of transmission settings that maximize the uplink data rate. As the solution should cope with different scenarios, we vary the number of End Devices (EDs), Base Stations (BSs), Packet Sizes (PSs) and Packet Rates (PRs). In addition, we compare our solution with many algorithms such as EXP3, ADR and EXPLoRaTS. Simulation results show that MDP with FCM clustering preprocessing improves better several Quality of Service (QoS) metrics including the Data Rate (DR), Packet Delivery Ratio (PDR), Time on Air (ToA) and Transmission Energy ( ). Thus, the PDR and the DR were improved by 25%, the ToA was reduced by 40% and was reduced by 20%.