The Datagram Transport Layer Security (DTLS) protocol is the IETF standard for securing the Internet of Things. The Constrained Application Protocol, ZigBee IP, and Lightweight Machine-to-Machine (LWM2M) mandate its use for securing application traffic. There has been much debate in both the standardization and research communities on the applicability of DTLS to constrained environments. The main concerns are the communication overhead and latency of the DTLS handshake, and the memory footprint of a DTLS implementation. This paper provides a thorough performance evaluation of DTLS in different duty-cycled networks through real-world experimentation, emulation and analysis. In particular, we measure the duration of the DTLS handshake when using three duty cycling link-layer protocols: preamble-sampling, the IEEE 802.15.4 beacon-enabled mode and the IEEE 802.15.4e Time Slotted Channel Hopping mode. The reported results demonstrate surprisingly poor performance of DTLS in radio duty-cycled networks. Because a DTLS client and a server exchange more than 10 signaling packets, the DTLS handshake takes between a handful of seconds and several tens of seconds, with similar results for different duty cycling protocols. Moreover, because of their limited memory, typical constrained nodes can only maintain 3-5 simultaneous DTLS sessions, which highlights the need for using DTLS parsimoniously.