Understanding the controlling factors of fruit quality development is challenging, because fruit quality is the result of the interplay between physical and physiological processes that are under the control of genes and the environment. Although process-based models have been used to make significant progress in understanding these factors, they to a large extent ignore the shape and internal structure of the fruit. To help investigate effects of fruit shape and internal structure on quality, the aim of this work is to create a 3D virtual fruit model that integrates fruit structure and function with growth governed by environmental inputs. For this purpose, we are creating a modelling pipeline that includes the following steps: creation of a 3D volumetric mesh of the internal and external fruit structure, calculation of the fruit’s physical properties from the resulting mesh, and integration of aspects of fruit physiology into the 3D structure. We have applied this pipeline to study tomato fruit (Solanum lycopersicum) by constructing 3D volumetric meshes from two images of perpendicular fruit slices and from MRI data, and integrating water and carbon transport processes into one of these meshes. To illustrate the tomato model, we performed a simulation of one season’s of the fruit’s growth, and compared the results with an already published process-based tomato fruit model. The results of the two models were in general agreement, but our model provided additional information on the internal properties of the fruit, such as a gradient in sugar concentration. Once the model is calibrated and evaluated, our approach will be suitable for studying the effects of internal fruit heterogeneity and overall shape on fruit quality development.