Is there something peculiar in our appreciation of improvised music? How does knowing that the music we are listening to is improvised affect our experience? As a first step in answering these questions, I have conducted an experiment in which an audio recording of the very same piece of music-a saxophone/clarinet freely improvised duet-was presented to 16 listeners, either as an improvisation ("IMPRO" condition), or as the live performance of a composition for saxophone and clarinet ("COMPO" condition). Listeners were encouraged both to reflect on their listening experience and to describe in their own words the music they heard. First, evaluative judgments were strongly different in the two listening conditions: listeners approached the piece with specific sets of values in mind, by relying on different features or different kinds of criteria (aesthetic ones in the COMPO condition vs ethical ones in the IMPRO condition) to ground their appreciative judgments. Second, and maybe more importantly, listening experiences were quite different in the two conditions: in the COMPO condition, the piece was more commonly experienced as a sonic product, with listeners paying great attention to the various acoustical effects achieved by the musicians and to the overall structure (or lack thereof); in the IMPRO condition, the music was often described as a kind of communicational or relational process, with descriptions that largely interweaved music-specific terms and more broadly social terms. Overall, this experiment shows that our listening experience can be dramatically affected by modal considerations, i.e., by how we think the music was produced. More specifically, it sheds some light on what constitutes the core of the aesthetic experience of improvisation by exhibiting what is centrally at play (and what is not) when we listen to collectively improvised music.