Users would experience individually different sickness symptoms during or after navigating through an immersive virtual environment, generally known as cybersickness. Previous studies have predicted the severity of cybersickness based on physiological and/or kinematic data. However, compared with kinematic data, physiological data rely heavily on biosensors during the collection, which is inconvenient and limited to a few affordable VR devices. In this work, we proposed a deep neural network to predict cybersickness through kinematic data. We introduced the encoded physiological representation to characterize the individual susceptibility; therefore, the predictor could predict cybersickness only based on a user’s kinematic data without counting on biosensors. Fifty-three participants were recruited to attend the user study to collect multimodal data, including kinematic data (navigation speed, head tracking), physiological signals (e.g., electrodermal activity, heart rate), and Simulator Sickness Questionnaire (SSQ). The predictor achieved an accuracy of 97.8% for cybersickness prediction by involving the pre-computed physiological representation to characterize individual differences, providing much convenience for the current cybersickness measurement.