How can we evaluate the spatio-temporal performance of virtual environments (VE) for research use? Here we show that end-to-end latency (ETEL) of VE can strongly damage users' perceptual and perceptuo-motor behaviors and that it can be considered to be the key factor for evaluating face and functional fidelity of a VE. We used a virtual ball-bouncing task as a paradigmatic example. Ball bouncing is known to exhibit attractive and repelling states whose localization in the racket cycle is sufficiently thin to be changed by small variations of ETEL. We first present a simple test-bed to measure the intrinsic ETEL of research-related VE systems. We then report results of a psychophysical ball-bouncing experiment in which ETEL was manipulated. While face validity (i.e., subjective experience) was maintained with relatively high values, the results reveal that the perception-action behavior (performance) was damaged with smaller ETEL values. These results call for action-perception variables in order to test the fidelity of VE systems.