Bimetallic nanoparticles (BNPs) are promising candidates for fundamental research and applications as their physico-chemical properties can in many cases be tuned continuously or enhanced with respect to the mono-metallic particles. Here we investigate the possibility of fabricating silver-indium and silver-aluminium BNPs in the range of 4-5 nm diameter and of varying stoichiometry by laser vaporization and gas condensation in the gas phase. We analyse their crystalline structure and segregation behaviour using transmission electron microscopy and probe their oxidation state in optical absorption measurements by tracking spectral changes of the localized surface plasmon resonance (LSPR). These complementary techniques show that, despite the small size and the high reactivity especially of aluminium, the BNPs form a silver-rich alloyed core surrounded by an oxide shell. Exposure to air leads to consecutive oxidation, whereas annealing the BNPs in a reducing atmosphere stabilizes the alloyed particle cores, as demonstrated by a narrow and blue-shifted LSPR. This is a first step towards the stabilization of non-oxidized bimetallic nanoparticles combining a noble and a trivalent metal.