The Constrained Application Protocol (CoAP) is a lightweight communication protocol designed by the Internet Engineering Task Force (IETF) for wireless sensor networks and Internet-of-Things (IoT) devices. The reliability mechanism in CoAP is based on retransmissions after timeout expiration and on an exponential backoff procedure which is designed to be simple and adapted to constrained devices. In this research work, we propose a new exact analytical model to analyze the performance of CoAP in lossy wireless networks modeled by the well-known Gilbert-Elliott two-state Markov process. We also show how to compute several performance metrics using closed form expressions such as the observed loss ratio, goodput, and the delay before success with a time complexity no more than O(r) with r is the maximum re-transmission limit. This study provides insights about improving CoAP recovery mechanism and highlights the properties -- including the limitations -- of CoAP. Also, it presents guidelines to tune CoAP parameters dynamically in order to adapt to network losses caused by interference and mobility. The model is validated using the realistic environment Cooja/Contiki OS where theoretical and experimental results match very well.