In this letter, the outage performance of a two user downlink non-orthogonal multiple access (NOMA) time varying network without any knowledge on the channel state or distribution at the base station (BS) is investigated. Building on reinforcement learning techniques and, more precisely, on multi-armed bandits (MAB), we propose a novel adaptive NOMA scheme that optimally tunes which user should perform successive interference cancellation (SIC) jointly with the power allocation at the BS. Remarkably, our proposed scheme requires only a single bit (ACK-type) of feedback from each user and is still able to outperform OMA, as demonstrated by the numerical results in many settings of interest including stochastic and even non-stationary (adversarial) ones.