The transportation system is gradually migrating toward autonomous, electric and intelligent vehicles. Wireless-enabled vehicles along with infrastructure units on the road are connected with traffic management centers that use intelligent data analysis tools to efficiently manage city's traffic. However, such wireless connectivity can make the ITS networks vulnerable to security threats; thus, impacting the application's reliability. On the other hand, the use of robust security techniques could hamper applications' quality of service (QoS). To understand the interplay between these two conflicting requirements, this article reviews the security and QoS design challenges in the ITS aspect of smart cities. Using an experimental test-bed, we evaluate the standard compliant security processing delays, develop an on-line tool that presents detailed security benchmark results, and study the impact of security on QoS using simulation results. We also discuss how machine learning based adaptive signature verification techniques can enhance QoS in ITS. We further present future opportunities to optimize the security-QoS balance for ITS applications.