Accurate and timely monitoring of forest canopy height is critical for assessing forest dynamics, biodiversity, carbon sequestration as well as forest degradation and deforestation. Recent advances in deep learning techniques, coupled with the vast amount of spaceborne remote sensing data offer an unprecedented opportunity to map canopy height at high spatial and temporal resolutions. Current techniques for wall-to-wall canopy height mapping correlate remotely sensed information from optical and radar sensors in the 2D space to the vertical structure of trees using lidar's 3D measurement abilities serving as height proxies. While studies making use of deep learning algorithms have shown promising performances for the accurate mapping of canopy height, they have limitations due to the type of architectures and loss functions employed. Moreover, mapping canopy height over tropical forests remains poorly studied, and the accurate height estimation of tall canopies is a challenge due to signal saturation from optical and radar sensors, persistent cloud cover, and sometimes limited penetration capabilities of lidar instruments. In this study, we map heights at 10 m resolution across the diverse landscape of Ghana with a new vision transformer (ViT) model, dubbed Hy-TeC, optimized concurrently with a classification (discrete) and a regression (continuous) loss function. This model achieves significantly higher accuracy than previously employed convolutional-based approaches (ConvNets) optimized with only a continuous loss function. Hy-TeC results show that our proposed discrete/continuous loss formulation significantly increases the sensitivity for very tall trees (i.e., > 35 m). Overall, Hy-TeC has significantly reduced bias (0.8 m) and higher accuracy (RMSE = 6.6 m) over tropical forests for which other approaches show poorer performance and oftentimes a saturation effect. The height maps generated by Hy-TeC also have better ground sampling distance and better sensitivity to sparse vegetation. Over these areas, Hy-TeC showed an RMSE of 3.1 m in comparison to a reference dataset while the baseline ConvNet model had an RMSE of 4.3 m. Hy-TeC, which was used to generate a height map of Ghana using free and open access remotely sensed data with Sentinel-2 and Sentinel-1 images as predictors and GEDI height measurements as calibration data, has the potential to be used globally.