In this work, we investigate a new deep learning reconstruction method of blood flow velocity within deformed vessels from contrast enhanced X-ray projections and vessel geometry. The principle of the method is to perform linear or nonlinear dimension reductions on the Radon projections and on the mesh of the vessel. These low dimensional projections are then fused to obtain the velocity field in the vessel. The accuracy of the reconstruction method is proved using various neural network architectures with realistic unsteady blood flows. The approach leverages the vessel geometry information and outperforms the simple PCA-net.