We numerically examine the mechanisms involved in nanoparticle formation by laser ablation of metallic targets in vacuum and in liquid. We consider the very early ablation stage providing initial conditions for much longer plume expansion processes. In the case of ultrashort laser ablation, the initial population of primary nanoparticles is formed at this stage. When a liquid is present, the dynamics of the laser plume expansion differs from that in vacuum. Low compressibility of the ambient liquid results in strong confinement conditions. As a result, ablation threshold rises drastically, the ablated material is compressed, part of it becomes supersaturated and the backscattered material additionally heats the target. The extension of a molten layer leads to the additional ablation at a later stage also favoring nanoparticle formation. The obtained results thus explain recent experimental findings and help to predict the role of the experimental parameters. The performed analysis indicates ways of a control over nanoparticle synthesis.