We prove a version of the weight part of Serre's conjecture for mod $p$ Galois representations attached to automorphic forms on rank 2 unitary groups which are non-split at $p$. More precisely, let $F/F^+$ denote a CM extension of a totally real field such that every place of $F^+$ above $p$ is unramified and inert in $F$, and let $\overline{r}: \textrm{Gal}(\overline{F^+}/F^+) \longrightarrow {}^C\mathbf{U}_2(\overline{\mathbb{F}}_p)$ be a Galois parameter valued in the $C$-group of a rank 2 unitary group attached to $F/F^+$. We assume that $\overline{r}$ is semisimple and sufficiently generic at all places above $p$. Using base change techniques and (a strengthened version of) the Taylor-Wiles-Kisin conditions, we prove that the set of Serre weights in which $\overline{r}$ is modular agrees with the set of Serre weights predicted by Gee-Herzig-Savitt.