Attracted by their easy-to-use interfaces and captivating benefits, conversational systems have been widely embraced by many individuals and organizations as side-by-side digital co-workers. They enable the understanding of user needs, expressed in natural language, and on fulfilling such needs by invoking the appropriate backend services (e.g., APIs). Controlling the conversation flow, known as Dialogue Management, is one of the essential tasks in conversational systems and the key to its success and adoption as well. Nevertheless, designing scalable and robust dialogue management techniques to effectively support intelligent conversations remains a deeply challenging problem. This article studies dialogue management from an in-depth design perspective. We discuss the state of the art approaches, identify their recent advances and challenges, and provide an outlook on future research directions. Thus, we contribute to guiding researchers and practitioners in selecting the appropriate dialogue management approach aligned with their objectives, among the variety of approaches proposed so far.