The etiology of neurodevelopmental disorders is linked to defects in parvalbumin (PV)-expressing cortical interneurons and to prenatal immune challenges. Mouse models of maternal immune activation (MIA) and microglia deficits increase the postnatal density of PV interneurons, raising the question of their functional integration. Here, we show that MIA and embryonic depletion of macrophages including microglia have a two-step impact on PV interneurons wiring onto their excitatory target neurons in the barrel cortex. In adults, both challenges reduced the inhibitory drive from PV interneurons, as reported in neurodevelopmental disorders. In juveniles, however, we found an increased density of PV neurons, an enhanced strength of unitary connections onto excitatory cells, and an aberrant horizontal inhibition with a reduced lateral propagation of sensory inputs in vivo. Our results provide a comprehensive framework for understanding the impact of prenatal immune challenges onto the developmental trajectory of inhibitory circuits that leads to pathological brain wiring.