Recent technical advances introduced drones into the consumer market. Thus, past research explored drones as levitating objects that provide in-situ interaction and assistance. While specific use cases and feedback scenarios have been researched extensively, technical and social constraints prevent drones from proliferating into daily life. In this work, we present past research in the area of human-drone interaction we conducted. We present technical boundaries and user-based considerations that arose during our research. We discuss our lessons learned and conclude how to deal with current challenges in the area of human-drone interaction.