Music is a communication signal and the estimation of music structure is essentially an information-theoretic problem. The structure of a music content M can be understood as the “proper type” and “right quantity” of latent side information S which provides an economical explanation of M and Q jointly. Two philosophies can support the definition of Q : Shannon’s Information (SI), also called lossy source-coding scheme, which relates information Q to the distortion of M with respect to a prototypical structure itself derived from a probabilistic model of all possible structures, and Kolmogorov’s Complexity (KC), sometimes referred to as algorithmic compressibility, which considers M as the output of a short, standalone program (within a class of valid structure generating programs), whose size is related to Q. Shannon’s approach is fundamentally a knowledge-based (inter-opus) approach, where statistically typical forms provide templates that guide the recognition of music content organization (stylistic structure). Kolmogorov’s framework is rather based on a data-driven (intra-opus) viewpoint and focuses on internal redundancy as a primary criterion for grouping musical material into consistent structural patterns (“semiotic” structure). Both conceptions of information are meaningful, but understanding and exploiting their interaction remains a fundamental scientific bottleneck – in MIR, in Computational Musicology, and also in many other scientific domains. The duality between SI and KC in music is for instance illustrated by Schenker’s versus Narmour’s conceptions of music structure, and KC approaches are becoming increasingly popular in MIR. However, current approaches in Music Structure Analysis fail in explicitly accounting for both aspects simultaneously, even though they are presumably present with a different balance across musical genres (this could be one of the causes of ambiguities in human perception of structure). Note that, even though, neither SI nor KC can actually be calculated exactly, they can be estimated using models, i.e. family of distributions for SI such as Hidden Markov Models and classes of programs for KC (as prefigured by the System & Contrast Model). Approaching the diverse views of music structure within the common framework of Information Theory appears as a relevant move towards a better understanding of what music structure is, but also as a key for a more efficient use of music structure in computational contexts. Though music is not a uniquely decodable code, it can be assumed that the number of reasonable structural hypothesis is sufficiently limited so as to be tackled in a relatively unified framework, encompassing the two main facets of data compression (SI and KC). By understanding the interaction between the “two sides of a same coin,” music could become a case study that would help bridging a fundamental gap in Information Sciences.