The next generation of WLANs will be, for the most part, ubiquitous in urban areas, densely deployed, and implementing the latest amendment of IEEE 802.11 standard, namely 802.11ax also known as Wi-Fi 6. Among the main purposes of 802.11ax is the improvement of the spatial reuse of radio channels by allowing the dynamical update of the sensitivity threshold and the transmission power at each node. In this regard, our contributions are twofold. First, we investigate the performance improvement resulting from a more efficient spatial reuse of radio channels with 802.11ax. Second, we introduce a centralized solution based on the Multi-Armed Bandit (MAB) framework and a sub-sampling technique to quickly discover an appropriate configuration of the sensitivity threshold and transmission power at each access point. We evaluate our solution with the network simulator ns-3 on different network topologies. The simulation results show the ability of our solution to quickly and robustly adjust these parameters of access points in order to significantly improve the behavior of WLANs.