Radio-frequency interference (RFI) raises a challenging issue confronted by radio astronomy. This challenge is particularly pregnant when recording extremely faint signals such as those associated with pulsar observations. Indeed, generally of higher energy, RFI significantly degrades the quality of the measurements which makes astronomical data more difficult to interpret and analyze. The current solutions to tackle this problem usually consist in performing RFI flagging, i.e., localizing the time-frequency bins in the dynamic spectrum affected by interference. Then the RFI-corrupted data, i.e., the measurements associated with these identified bins, is generally discarded before any subsequent data processing, which unavoidably leads to a loss of information. Alternatively, this paper proposes to formulate RFI mitigation as a joint detection and restoration task to allow parts of the dynamic spectrum affected by RFI to be not only identified but also recovered. The proposed method relies on a particular instance of a recent architecture of deep convolutional networks. This network is trained on a large data sets generated within a simulation framework specifically designed according to physically-inspired and statistical models of the pulsar signals and of the RFI. Through extensive numerical experiments, the proposed approach is shown to reach competitive performance in terms of RFI detection and dynamic spectrum restoration.