Time-of-flight (ToF) camera technology provides a real-time depth map of a scene with adequate frequency for the monitoring of physiological patient motion. However, dynamic surface motion estimation using a ToF camera is limited by issues such as the raw measurement accuracy and the absence of fixed anatomical landmarks. In this work we propose to overcome these limitations using surface modeling through B-splines. This approach was assessed in terms of both motion estimation accuracy and associated variability improvements using acquisitions of an anthropomorphic surface phantom for a range of observation distances (0.6-1.4 m). In addition, feasibility was demonstrated on patient acquisitions. Using the proposed B-spline modeling, the mean motion estimation error and associated repeatability with respect to the raw measurements decreased by a factor of 3. Significant correlation was found between patients' surfaces motion extracted using the proposed B-spline approach applied to the ToF data and the one extracted from synchronized 4D-CT acquisitions as the ground truth. ToF cameras represent a promising alternative for contact-less patient surface monitoring for respiratory motion synchronization or modeling in imaging and/or radiotherapy applications.