Remote sensing techniques are increasingly employed for early detection of ground deformation and landslide warning systems. The combination of vast remote sensing data and advancements in machine learning algorithms has led to significant progress in landslide detection and mapping.This study proposes an innovative neural network architecture for landslide detection. The network utilizes a fusion of optical images (RGB) and Digital Elevation Models (DEMs) to enhance accuracy. Additionally, attention layers and Mixup techniques are incorporated to further improve the model’s performance. Given the limited availability of training data, the proposed network was trained on a publicly accessible dataset, specifically the established Bijie landslide dataset located in China. We demonstrate that using an efficient training strategy on this dataset allows us to pretrain a network that can be easily fine-tuned for a different site exhibiting landslides at various scales. In practice, we introduce a newly developed dataset covering the southwestern part of Hokkaido, Japan, which experienced a landslide event in 2018. This dataset is freely available on the internet. The results demonstrate the effectiveness of the proposed architecture compared to existing methods, highlighting the benefits of our pretrained model.