Industrial Internet of Things (IIoT) systems can leverage deep learning (DL) models to provide intelligent applications. However, the training process of such DL models tends to leak privacy when the training data contain sensitive operational and management information. Most studies about privacy-preserving DL require a trusted data collector and thus are not suitable in an untrusted environment. In this paper, we propose a distributed privacy-preserving framework for DL with edge-cloud computing, where direct privacy leakage and indirect privacy leakage of training data are both considered. First, a pre-trained convolutional neural network (CNN) is partitioned into two parts for limiting direct privacy leakage of raw data, namely the feature module and the classification module. The feature module consisting of the lower layers of the CNN is deployed on an edge server, which is used to attract the feature of raw data. The feature data is fed to the classification module consisting of the higher layers of the CNN, which is deployed on the cloud server to compute image classification tasks. Second, a perturbation module between the feature module and the classification module is designed for limiting indirect privacy leakage during feature data transmission. The perturbation module uses local differential privacy (LDP) to produce noise in the feature data. Third, to further improve the accuracy, we retrain the classification module with the randomized feature data from edge servers. Experimental results show that the proposed framework achieves high accuracy under low privacy budgets.