This research proposed a method for adaptive Lagrange multiplier determination for rate-distortion optimization with dynamic texture in High Efficiency Video Coding (HEVC). Inspired by the experimental results of the Lagrange multiplier selection test, the presented approach adaptively predicts the optimum Lagrange multiplier for different dynamic texture sequences, based on the features of the dynamic texture sequences such as normal flow and spatial-temporal information. The Lagrange multiplier among the given values will be chosen based on the Bjontegaard delta measurements. After that, the data of training dynamic texture will be used for Support Vector Machine (SVM) in machine learning for getting the predicting results. The proposed algorithm has been fully integrated into HEVC reference codec. The result shows that the proposed method can improve 0.5 in Structural Similarity Metric (SSIM) and 2 in Peak Signal-to-Noise Ratio (PSNR).