Hidden Markov models (HMMs) are a standard tool in many applications, including change-point (or segmentation) data analysis. Since HMMs are intrinsically heterogeneous, the detection of outliers in data modeled by HMMs is a challenging problem. This problem can be modeled by an ad hoc model which extends the HMM by explicitly taking into account variables for the outlier status of the observations. We suggest a novel and model free method based on relative entropy and show a dynamic programming algorithm to implement it in linear time. We validate the two methods on simulated data. We apply our method based on relative entropy on Copy Number Variation (CNV) data and show its effectiveness.