We present preliminary results from our calculation of the pseudoscalar transition form factors of the $\eta$ and $\eta'$ mesons using staggered quarks on $N_f=2+1+1$ gauge ensembles generated by the Budapest-Marseille-Wuppertal collaboration. These transition form factors are an important input for the hadronic light-by-light contribution to the muon $(g-2)$. We first elaborate on the extraction of the masses of the $\eta$ and $\eta'$ mesons, that mix under the dynamics of QCD. Thereafter, we show our preliminary results for the pseudoscalar transition form factors, focusing on the $\eta$ meson in the absence of mixing.