In the last three decades it has become clear that cortical regions, interconnected via white-matter fibers, form a modular and hierarchical network. This organization, which has also been seen at the microscopic level in the form of interconnected neural assemblies, is believed to support the coexistence of segregation (specialization) and integration (binding) of information. A fundamental open question is to understand how this complex structure can emerge in the brain. Here, we made a first step to address this question and propose that adaptation to various inputs could be the key driving mechanism for the formation of structural assemblies. To test this idea, we develop a model of quadratic integrate-and-fire spiking neurons, trained to stimuli targetting distinct sub-populations. The model is designed to satisfy several biologically plausible constraints: (i) the network contains excitatory and inhibitory neurons with Hebbian and anti-Hebbian STDP; and (ii) neither the neuronal activity nor the synaptic weights are frozen after the learning phase. Instead, the network continues firing spontaneously while synaptic plasticity remains active. We find that only the combination of the two inhibitory STDP sub-populations allows for the formation of stable modular organization in the network, with each sub-population playing a distinct role. The Hebbian sub-population controls for the firing rate and the anti-Hebbian mediates pattern selectivity. After the learning phase, the network activity settles into an asynchronous irregular resting-state, resembling the behaviour typically observed in-vivo in the cortex. This post-learning activity also displays spontaneous memory recalls, which are fundamental for the long-term consolidation of the learned memory items. The model introduced represents a starting point for the joint investigation of neural dynamics, connectivity and plasticity.