On Making Music with Heartbeats

Representation and analysis of musical structures in heart signals can benefit understanding of cardiac electrophysiology aberrations such as arrhythmias, which can in turn aid in the diagnosis and treatment of cardiac arrhythmias.


Introduction
Representation and analysis of musical structures in heart signals can benefit understanding of cardiac electrophysiology aberrations such as arrhythmias, which can in turn aid in the diagnosis and treatment of cardiac arrhythmias.
The typical time-frequency analysis of electrocardiographic recordings of cardiac arrhythmias yield descriptive statistics that provide useful features for classification, but fail to capture the actual rhythms of the physiological phenomena.Here, I propose to use music notation to represent beat-to-beat and morphological feature-to-feature durations of abnormal cardiac rhythms, using articulation markings when emphasis is warranted.The rhythms and articulations captured in these representations may provide cues to differentiate between individual experiences of cardiac arrhythmia, with potential impact on personalising diagnostics and treatment decisions.
Music generation is presented as an application of these rhythm transcriptions.The physiological origins ensure that the music based on heart rhythms, even abnormal ones, sound natural.The two-part music creation process draws inspiration from music collage practices, and comprises of a retrieval component followed by transformation processes, which can be applied at the melody or block levels, and complex combinations thereof.
The music thus created can not only be used to identify distinct heart signatures and what they mean for different cardiac conditions, they can also provide a visceral record of the experience of an arrhythmic episode.The pounding and fluttering of arrhythmia can often be physically uncomfortable.The music created from arrhythmic traces is not easy listening; it is often provocative, but potentially instructive.
Music generated from arrhythmic hearts can have interesting implications for development of intelligent heart-brain computer interfaces.Music has been shown to have direct impact on cardiac response [1][2].This heart-brain response offers a window into the listener's true state of emotion, with potential for integration into emotive interfaces for music performance.Like respiration, and with the appropriate sensors, heart rhythms can also provide a feedback mechanism during music performance, driven by, and influencing autonomic response [3].
An understanding of how abnormal heart rhythms can be precisely and organically rendered into music thus opens up new opportunities and challenges for music and Artificial Intelligence (AI).The goal here is to propose some ideas and examples of these opportunities and challenges, and to offer some possible solution approaches.

Why Cardiac Arrhythmias
Worldwide prevalence of cardiovascular disease (CVD) is high, and is expected to increase substantially in the future.In the United States, nearly half (48 percent, 121.5 million in 2016) of all adults have some form of cardiovascular disease [4].According to the World Health Organization, of the 56.9 million global deaths in 2016, 40.5 million (71%) were due to NCD, and the leading cause of NCD is CVD (17.9 million, or 44% of NCD deaths) [5].CVD is the cause of more than half of all deaths in the European Region [6].Hence, there is great interest in characterizing heart disease so as to benefit CVD diagnostics and therapeutics.
Like other conditions that affect the heart, abnormalities in heart rhythms (arrhythmias) are associated with morbidity and substantial economic costs.A study using over half a million records in the UK Biobank ascertained that abnormalities of cardiac rhythm affect >2% of adults, and the incidence rate of 0.5% per year is similar to that of stroke, myocardial infarction, and heart failure [7].Another study based on the Swedish National Study on Aging and Care (SNAC) data of adults aged 60 and older, shows that the prevalence and incidence of arrhythmias rapidly increase with age [8].Baseline prevalence of atrial fibrillation (AF) was 4.9%, other arrhythmias including premature ventricular complexes or ventricular ectopics (VEs), supraventricular tachycardia (SVT), and supraventricular extrasystole or ectopic (SVEs) were 8.4%, first-or second-degree atrioventricular (AV) block was 7.1%.AF is the most common arrhythmia, with a global prevalence of 33.5 million in 2010 [9].
There have been significant advances in the diagnosis and management of cardiac arrhythmias over the past two decades [10], but the classification and description of arrhythmias remain crude and often bear little relation to the symptoms and treatment outcomes.Arrhythmias are classified according to source (atrial / supra-ventricular, ventricular, junctional), rate (bradycardia is slow and tachycardia is fast), and regularity (fibrillations are fast and irregular, and tachycardias are fast and regular).
Consider the case of AF.AF is irregular and often rapid, and is subdivided according to duration: paroxysmal AF is sporadic, lasting more than 30 seconds but less than a week; early persistent AF is continuous for more than a week but less than a year; longstanding persistent AF is continuous for more than a year; and, permanent AF is chronic and ongoing.These coarse descriptions fail to capture the actual rhythms and feature variations in individual occurrences of AF.The classifications also have little bearing on patients' symptoms or the likelihood of success of ablations or other treatments.
With the move towards precision medicine and the customisation of health care, it will become increasingly important to be able to distinguish between individual experiences of different arrhythmias.

Why Music Representation
Arrhythmias and other heart conditions are highly musical, with their innate periodicity and time varying musical structures.The rhythms of arrhythmia closely resemble those of music rhythms, both the natural ones encountered in performance and more stylised ones found in musical scores.The episodic nature of many arrhythmias, their time evolution, and the local patterns, have direct musical equivalents.The musical language we use to describe these structures therefore offers tools to describe cardiac anomalies in far greater detail than currently practiced.
Musicians have over the centuries developed a rich vocabulary for describing frequency (pitch) and time (rhythm), and how these attributes change over time, notating them so that they can be reproduced with high fidelity [11].Over time, music notation has gained in sophistication to represent almost any imaginable rhythmic pattern, pushing the limits of rhythm notation, rendering, and perception.Virtuosic uses of notation include Haydn's incongruous metric and harmonic groupings [12], Brahms' creative use of hemiolas and metrical shifts [13], Stravinsky's playing with metrical changes while the rhythm remained untouched [14](p.61),Elliott Carter's invention of metric modulation [15], and Brian Ferneyhough's breathtakingly complex notations [16], just to name a few.
This rich vocabulary has been used to describe birdsong.In the 19th century, Lunn showed brief notated examples of cuckoo and blackbird calls in a short piece about the history of musical notation [17].Composer Messiaen incorporated many birdsongs into his compositions like [18].For bird enthusiasts, Saunders systematically transcribed the songs of 201 birds from the Eastern United States [19], representing pitch on the y-axis, duration on the x-axis, intensity with thick and thin lines, pronunciation with loops for consonant sounds and wavy lines to indicate trills, and quality, but ignoring rhythm and repetitions of accented syllables [20].These are features that can be captured using music notation.
Music has also been used to capture linguistic prosody.Joseph Steele was one of the earliest protagonist of this idea.He notated accent (pitch inflection), duration, pause, cadence, and force (loudness) using symbols on a five-line staff inspired by that used in music [21].In the early 1900s, Scripture notated the general melody and length of sounds using the height placement and lengths of symbols and by the number and heaviness of marks [22].In opera, Arnold Schoenberg and Leoš Janáček used speech melody, not has an accurate representation of natural speech, but as an approximate guide to singers to follow the pitch inflexions of speech.Research developments in representing speech rarely used music notation.Noting that work in speech rhythm is predominantly driven by a desire to classify languages than to elucidate actual rhythms of spoken language, recent work by Brown, Pfordresher, and Chow builds on that of Steele to represent spoken meter using music notation [23].Independently, Simões and Meireles have been using music notation to transcribe speech prosody in Spanish, Portuguese, and English, but with only a 4/4 meter [24].
Given its flexibility and power to render strange and varied rhythms, using music to represent the fine variations in arrhythmia is not a far-fetched idea.Scholars and physicians have long noted the close connections between music and the heart.In the Middle Ages, academic physicians wrote about the music of the human pulse [25].The first use of music notation to describe cardiovascular anomalies was applied to heart murmurs, first by the inventor of the stethoscope Réne Laennec [26], and more recently by nephrologist Michael Field [27], reflecting the close listening necessary for cardiac auscultation.These first instances of representing cardiac disorders using music notation will be described in greater detail in later sections.As far as we know, music representation of heart rhythm disorders like that in cardiac auscultation has been shown only recently by Chew in [28].The focus on details of actual rhythms could potentially provide cues to the physiological phenomena of arrhythmias, and make a difference in how arrhythmia is viewed, described, and discussed, beyond simple categories.
Music representation of heart rhythm disorders can give a detailed description of the actual rhythms, and can potentially provide tools for characterising individual cases of the arrhythmias and the forms they take within a person.For example, music notation can provide quick, visual information about differences between different kinds of ventricular premature beats, or specific kinds of rhythmic irregularities encountered in experiences of atrial fibrillation that could potentially be linked to severity of symptoms or treatment outcomes.
To make music representation scalable, transcriptions will need to be automated.While many rhythm quantisation techniques exist, the specifics of the methods will need to be tailored and fine-tuned for the new applications.The music representations can also form the basis for further comparisons such as similarity assessment and classification.These problems present new opportunities and challenges for music and AI.

Hearts Driving Music
Taking the parallels between music and heart signals one step further, the strong similarities between the human pulse and music means that heart data can be readily mapped to music.
Using heart rate time series data calculated over 300-beat heartbeat windows and mapping the numbers to 18 notes on a diatonic scale, Goldberger (alias Davids) generated a set of melodies that, augmented with improvised accompaniment, were recorded in the Heartsongs CD [29].More complex mapping techniques have followed in the sonification of cardiac data.An example is Yokohama [30]'s work, which maps cardiac data to melodic pitches: each heartbeat interval corresponds to a Musical Instrument Digital Interface (MIDI) note, with intervallic changes such as premature beats triggering more significant pitch changes.In [31], Orzessek and Falkner passed heartbeat intervals through a bandpass filter and mapped them to MIDI note onsets, pitch, and/or loudness to sonify heart rate variability (HRV).Ballora et al. [32] further maps HRV data to pitch, timbre, and pulses over a course of hours for medical diagnosis.
Heart rate variability parameters have been used to guide music generation.In [33], Fukumoto et al. used the high frequency component of HRV, linked to autonomic nervous activity-as a fitness value in their Interactive Evolutionary Computation system to generate music chord progressions.Heart data has also been used to shape interactive performances in real-time.In Votava and Berger [34]'s Heart Chamber Orchestra, interpretations of its twelve musicians' heartbeats, detected through ECG monitors, and relationships between them influence a real-time score that is then read and performed by the musicians from a computer screen.Related to this, physiological measures like respiration, blood pressure and heart rate have been shown to increase with music tempo [1], and decrease with lower tempo [3].Prior work has mainly focused on non-arrhythmic hearts.
Arrhythmia rhythms arise naturally in music; it is but a small step from rhythm notation of cardiac pathologies to turning them into collage music.Since arrhythmia rhythms arise naturally in music, finding music that matches the rhythms of a segment of recorded arrhythmia then becomes a matter of retrieval.And the task of creating a musical piece then becomes one of re-combining these retrieved segments in elegant and interesting ways to form a collage composition.
Collage is a common compositional technique, one that is commonly used in AI systems for music generation.David Cope's EMI (Experiments in Musical Intelligence) [35] is a classic example using the idea of recombinancy [36].EMI takes fragments of existing music and recombines them into new logical sequences to produce music in the style of composers ranging from Bach to Scott Joplin.In [35], Cope argues that recombinancy transcends music, citing that great books in English comprise of recombinations of the twenty-six alphabet letters, and Western art music consist of recombinations of twelve equal-tempered scale pitch classes.The quality of the work then depends on the "subtlety and elegance of their recombination." The OMax family of human-machine co-improvisors by Gérard Assayag et al. [37] provides another example of recombinant music.The OMax systems generate music sequences stylistically similar to that produced by the human co-improvisor.The systems use factor oracles and create new music by recombining subsequences from the original material.Pachet's Continuator similarly generates stylistically consistent music from a human improvisor, but using variable Markov models [38].Mimi [39] is another factor oracle-based system, with the addition of visual feedback.In Mimi4x [40], the user can structurally engineer four recombination processes by simultaneously controlling four Mimi instances.This was inspired, in part, by John Zorn's Cobra, where a composition consists of a set of cue cards with rules instructing players what to do; a prompter decides on the combination sequence for the performance.
In the same vein of musical material re-use, the RhythmCAT system [41] creates new loops that mimick the rhythmic pattern and timbral qualities of a target loop using a corpus of existing sound material.Audio inpainting is a specific instance of collaging.Long audio signal gaps are patched using segments from other parts of a recording that can be smoothly inserted using similarity graphs [42]; deep neural networks are used to restore missing audio content at a smaller scale [43].At the extreme end, small grains of sonic elements are joined together to form larger acoustic events in granular synthesis [44].
Collage music created based on heart rhythms will adhere to the extracted rhythms.Music generation to fit a rhythm template has been practiced in Her-remans and Chew's MorpheuS music generation system [45], which randomly assigns notes to the rhythms of an existing piece, then alters them iteratively to more closely fit the tension profile.Dabby's chaos theory driven music generation technique also systematically transforms an existing composition by degrees [46].Transcriptions of sight reading consist of faithfully notating flawed performances of pitches and rhythms [28,47], not unlike the accurate transcription of abnormal heart rhythms.
Music based on arrhythmias provides a visceral way to communicate the experience of an arrhythmia.The interrupted rhythms, the skipped beats, the sharp transitions, they can all be captured and rendered through music.For some listeners, this might be pleasurable.For others, it might be discomforting and disorienting.But it will hopefully provide a lasting impression of what an arrhythmia feels like.The upcoming sections will present historic notations of heart murmurs, followed by an introduction to music notation of cardiac arrhythmias; then, music created from the rhythm transcriptions, and a short conclusion.

Music Notation in Cardiac Auscultation
Heart sounds form important cues for diagnosing valvular disorders.The opening and closing of heart valves and the flow of blood through the valves produce the rhythmic lubdub and periodic swooshing sounds of the beating heart.Here, we review and discuss some of the earliest examples of applying music notation to representing heart murmurs heard in the process of auscultation, before introducing notation for cardiac arrhythmias in the next section.

Venous Hum
The inventor of the stethescope, Réne Laennec, who was both a physician and a flute player, provided one of the earliest examples for using music notation to describe a cardiac disorder, in this case, a benign heart murmur called the venous hum [26].
(a) Laennec (1837)'s original notation for the venous hum [26]  Laennec integrated music notation into the description of the venous hum.His original notation is shown in Figure 1(a), which will be adapted to Figure 1(b) based on his remarks.In his text, Laennec describes the undulating sequence of tones as passing through three notes over a range of a major third: the highest note was a little too low for a major third, but not quite low enough to warrant a flat.Duration-wise, the notes were of roughly the same length, with the tonic being a little longer, but of variable duration.The relative durations were denoted by dots atop the empty noteheads.The revised transcription in Figure 1(b) is at a lower register, ascribes a quarter flat to the third, and assigns actual note values to the longer tonic.It further approximates the note values from Laennec's notation, taking into account the fact that he mentioned that the notes were of roughly equal but variable durations.

Heart Murmurs
More recently, another physician-musician Michael Field, a nephrologist and flautist, also used music notation to describe heart murmurs, as well as regular heart sounds [27].Figure 2 re-creates Field's transcriptions of four left-sided heart murmurs.In contrast to Laennec's notation, Field's transcriptions are unpitched, but include fine details such as grace notes, articulations, and dynamic markings to more precisely indicate the shaping and evolution of the sounds.Acciacaturas mark the initial snap in mitral stenosis; trills capture the rumbling quality of aortic and mitral stenosis; decrescendo signs indicate the diminishing sound of aortic regurgitation.

Aortic Stenosis
Fig. 2. Representation of heart murmurs by Field [27] using music notation Field observes that the close listening to musical rhythms, articulation, and dynamics essential to deep enjoyment of musical performance are the same skills needed to identify the characteristic patterns of heart sounds and valvular murmurs.The music notation designed to represent performance variations such as articulations and dynamics convey the variations audible in common heart murmurs.Motivated by a desire to teach student doctors how to identify these heart murmurs, Field uses the music transcriptions in his teaching of cardiac auscultation.He further proposes that the exercise be extended to other murmurs and cardiac conditions.

Music Notation of Cardiac Arrhythmias
Pathologic heart rhythms map to recognisable musical rhythms that serve as defining characteristic of different arrhythmias.This section demonstrates the feasibility of using music notation to represent different cardiac arrhythmias.In [28], I presented a few such examples.Here, these examples are further expanded and the notation of other cardiac rhythm disorders introduced.

Premature Ventricular and Atrial Contractions
Premature contractions, a.k.a.ectopics, are some of the most basic and common arrhythmias that produce a distinctive abnormal heart rhythm.
(a) Ectopics (V) followed by fully compensatory post-extrasystolic pauses Figure 3 shows examples of rhythms produced by premature contractions.Figure 3(a) presents a transcription of sinus rhythm, normal beats marked N, with premature ventricular contractions, also called ventricular ectopics marked VE.Ectopics can be atrial, ventricular or junctional, but the musical rhythm representation, apart from the labels, does not distinguish between the three.
In the figure, each ectopic is followed by a characteristic prolonged postextrasystolic pause, the time between the VE and the next N.The pause is fully compensatory as the time between the preceding N to the subsequent N after the ectopic is double the length of the preceding NN interval-the crotchet plus minim equals three beats, which is two times the dotted crotchet.
In these examples, the onsets have also been marked as upbow (a V above the notehead) or downbow (a square bracket above a notehead).A premature contraction is shown with an upbow because its onset is often imperceptible; whereas, the normal beats are marked downbow to indicate that they are strong onsets.The ones following an ectopic has an even stronger emphasis, marked by the sforzandos, because the post-extrasystolic pause allows the heart to fill up with more blood than normal and the next beat is especially strong.
Figure 3(b) shows a transcription of premature atrial contractions.The ectopics are marked SVE (supraventricular ectopic) and the normal constractions are marked N. Because the time between the N onset just before and after each SVE is less than double the length of the preceding NN (normal) interval, the pause is not compensatory.Note that post-extrasystolic pauses following VEs can be non-compensatory and those following SVEs can be compensatory.
Premature contractions can recur in periodic patterns: every other beat (bigeminy), every third beat (trigeminy), every fourth beat (quadrigeminy), etc.; they can also occur in quick succession, in couplets or triplets.Figure 3(b) shows a bigeminy rhythm.Figure 4 shows the electrocardiographic trace of a trigeminy with compensatory pauses and two possible ways to transcribe the rhythm: one in simple triple time, and the other in compound triple time.

A Theory of Beethoven and Arrhythmia
The characteristic syncopated rhythms of premature contractions can be found in composed music, such as that of Beethoven.Because of this striking similarity, cardiologists Lüderitz [48][49], Cheng [50], and more recently Goldberger et al. [51] amongst others, have speculated that Beethoven may have suffered from arrhythmia, and that distinctive rhythms in his compositions represent musical electrograms of his abnormal heart rhythms.The commonly cited example is the dotted rhythm that features prominently in the opening to his piano Sonata in E ("Les Adieux"), Op.81a, shown in Figure 5.An interpretation of this dotted rhythm as that of premature beats would suggest that the main beats are the ectopic beats, marked 'E' in the figure, followed by (non-compensatory) pauses; normal beats are marked 'N'.If the normal beat is three sixteenth notes' duration, then the extra-systolic pause, at five sixteenth notes, is not quite twice the normal beat, and so is not fully compensatory.
Another example of such a dotted rhythm can be seen in an earlier sonata, Op.31 No. 3. Again, the normal and ectopic beats are marked 'N' and 'E' in the score.It is worth noting that many rhythms of arrhythmia can be found in music, so ascribing such rhythms to a composer's possible cardiac condition may lead to false positive conclusions.

Ventricular and Supraventricular Tachycardias
Tachycardia is an abnormal heart rhythm where the heart beats regularly but faster than normal, even when the body is at rest.Tachycardias are labeled according to their source: ventricular or supraventricular.
Tachycardias can be triggered by ectopics, or induced with short bursts of fast pacing, as was the case for the recording shown in Figure 7. Figure 7(b) shows only the electrocardiographic (ECG) signal from Lead II; Figure 7(a) displays the same ECG segment with an added layer showing the short burst of fast pacing (the three vertical lines) that triggered the ventricular tachycardia.The steady crotchet rhythm at about 71 bpm breaks into a fast trot with quavers at about 85 bpm, i.e. a pulse rate of nearly 170 bpm, as shown in Figure 7(c).The tachycardia, the fast rhythm, continues for over 30 seconds before it is terminated with anti tachycardia pacing (not shown).Supraventricular tachycardia produces a similar kind of rhythm, but the ECG trace has a different morphology.

Atrial Fibrillation
Atrial fibrillation (AF) is a common condition characterised by fast and irregular rhythms.On the ECG, an additional clue is the lack of P waves.Music transcriptions of AF rhythms require many more metric and tempo changes as a result of this irregularity.Figure 8 contains a metric modulation, a proportional change in tempo, between bars one and two, like that used by Elliot Carter.To capture the rhythmic variation, the tempo went from 94 bpm to 126 bpm, a 3:4 tempo ratio; as notated, a dotted quaver in the previous tempo (94 bpm) is equivalent to a crotchet in the new tempo (126 bpm).All three examples contain frequent meter changes, like in the music of Stravinsky.The first transcription, Figure 8, goes from 3  4 to 4 4 to 2 4 to 3 8 .The second transcription, Figure 9, alternates between 4  4 and 7 8 .And, finally, Figure 10, goes from   .There are high degrees of variability in the duration contrast, but also in the underlying tempo of the transcriptions.The notated tempi shown range from 94 bpm in Figure 8 to 125 bpm and 188 bpm in Figure 9 to 214 bpm in Figure 10.
Because these are subsequences retrieved from a long recording for human inspection, they also embed anomalous behaviours, such as strings of broad complex beats labeled 'V' (ventricular) in the ECG strip.A series of broad complex beats is usually labeled VT (ventricular tachycardia), but they can also arise in AF.

Atrial Flutter
Atrial flutter is an arrhythmia caused by a re-entry circuit in the right atrium that causes the atria to pulse at a rapid rate.The atrial rate is determined by the size of the atrium.As a result, it settles reliably around 300 bpm.Only some of these impulses are conducted to the lower chambers of the heart, the ventricles, due to the heart's own gating mechanism, the atrioventricular (AV) node.The ventricular rate is determined by the AV conduction ratio-whether every other atrial beat is conducted to the ventricles, every third beat, or every fourth beat, etc.For example, a 2:1 AV conduction ratio leads to a ventricular rate of 150 bpm, and a 4:1 ratio leads to a rate of 75 bpm.This concludes the discussion on notating heart rhythms.The next section describes the collage and transformation processes involved in generating music from abnormal heartbeats.

Music Generation From Abnormal Heartbeats
Drawing from a growing collection of collage music based on cardiac electrophysiology aberrations, this section will introduce two main ideas underlying the creation of these pieces.The examples are selected from a collection of seven Little Etudes for beginner-to-intermediate piano players released for World Heart Rhythm Week 2020, the Arrhythmia Suite (2017-2018), and the Holter Highlights introduced in [28].
The following sections discuss the retrieval task involved in finding appropriate source material for the arrhythmia pieces, and the matter of musical transformation to make the retrieved segments fit smoothly.

A Retrieval Task
Selected pieces created from ECG recordings of cardiac arrhythmias are catalogued in Table 1.Alongside the name of each arrhythmia piece is the music source for the piece, when one exists.The pieces were collaged by Chew; rhythm for the Arrhythmia Suite were transcribed using a combination of automated tools and manual revisions by Krishna, Soberanes, and Ybarra, and Orini and Lambiase provided the source sequence.Included in the table are url's, links
The very first task in the process of creating the pieces is the identification of an appropriate source piece from which to draw music material for recombination and transformation.The choice of pieces not only relied on matching the most salient rhythmic patterns, the pitch patterns in the chosen pieces must also fit the kinds of duration prolongations and reductions found in the particular ECG sequence.Only one piece is used in order to ensure stylistic coherence through a consistency of musical language use.Thus, it is important that the source piece can encompass the variations in the rhythms of the arrhythmia ECG.
While transcribing the rhythm sequence for the first piece in the Arrhythmia Suite, Ashwin Krishna noted that the short bursts of fast pacing used to induce the ventricular tachycardia in ECG sequence 161122 produced the same rhythm as that in Mars in The Planets, Op.32, by Holst-see Figure 12.Thus, Mars became the source piece for Arrhythmia Suite: I. 161122 VT before during after ECG.The militant regularity and ratcheting intensity of Mars provided good material suitable for adapting to the ventricular tachycardia episode in the piece.12. Sources for Arrhythmia Suite: I. 161122 VT before during after ECG-ECG showing fast pacing bursts to trigger ventricular tachycardia (signal from Lead II and RV3-4) and beginning of Holst's Mars for two pianos (condensed to two staffs).
In the Little Etudes, beginning with the ectopics, the 2:3 pattern of the ventricular bigeminy in No. 3 is captured by the 5  8 rhythm of Samuel Barber's Sea-snatch from his Hermit Songs, Op.29, see Figure 13(a).Extra emphasis, a sforzando (sf), is put on the forceful regular beat following the early ventricular beat and pause.Recall that during the pause, the heart fills up with more blood than usual, causing the next heartbeat to be particularly forceful.The 2+2+2+3 rhythm of Dave Brubeck's Blue Rondo A La Turk renders perfectly the rhythmic sequence of ventricular ectopics with compensatory post extra-systolic pauses in Little Etude No. 6.This rhythmic ostinato corresponds exactly to the middle portion of Figure 3(a).
It is not common that a pre-existing musical rhythm fits the arrhythmia rhythm exactly, as in the above examples.Often, rhythmic adjustments or rearrangements have to be made.The next section presents some of the transformations required and how this impacts the choice of the source music.

A Matter of Transformation
Here, we explore a few of the techniques that were used to adapt existing musical rhythms to those extracted from the ECGs.

Melodic Transformations
The Siciliane in the Holter Highlights, Figure 15, is based on Bach's Siciliane from the Flute Sonata BWV 1031.The flute melody Little Etudes by Elaine Chew is licensed under a CC BY-SA 4.0 international license -https://creativecommons.org/licenses/by-sa/4.0/ 5 (a) adaptation of Samuel Barber's Sea-snatch to highlight ventricular bigeminy rhythm, with emphasis on the more forceful normal beat following a pause is particularly adept at fitting to the rhythm of the ECG sequence of Figure 8.This is because the note at the top of the upward leap of a fourth lends itself to flexible elongation in order to fit the couplet (two wider ventricular beats) in the ECG, which is followed by a pause.The Wenckebach Lullaby, Little Etude No. 7, presents another example of a warped melody.The Wenckebach block is a second degree atrio-ventricular heart block where there is some obstruction of the conduction from the atria to ventricles.It is characterised by progressively elongating PR intervals that reset when a complete beat is dropped.The Brahms Lullaby is chosen because the melody not only fits rhythmically, but also lends itself to the intervallic prolongation of the Wenckebach block, leading to a dramatic dropped beat at the end of the phrase, as shown in Figure 16.
Little Etudes by Elaine Chew is licensed under a CC BY-SA 4.0 international license -https://creativecommons.org/licenses/by-sa/4.0/(a) bars 5-9 of the Wenckebach Lullaby (Little Etude 7) In the unmetered Little Etude No. 1, the irregular rhythms of a slow atrial fibrillation fits the elastic rhythms of romantic playing styles, which tend to flexibly bend time through musical rubato.To depict the fibrillatory waves of AF, Chopin's Nocturne Op.62 No.1, which has many trills, is used as source material.Figure 17 shows (a) the original Chopin nocturne excerpt, and (b) its transformed version.Although the two sound very similar, they are visually quite different.To simplify the physical movements, the right hand melody with the trills has been split between the right and left hands in the little etude.
The original nocturne has regular notated rhythms.The written melodic notes of the little etude are explicitly of irregular lengths.When rendered as written, the little etude actually closely resembles how the original nocturne might be performed.Block Recombination Holter Highlights: I. Mixed Meters, Figure 18, is based on the third of Libby Larsen's Penta Metrics.The frequent 7 8 and 5 8 meters in the ECG sequence shown in Figure 10 fit naturally into Larsen's pedagogical piece, which is written in 7  8 time.The Holter Highlights piece is based on a recombination of the elements of Larsen's original music shown in Figure 19: (a) the 7  8 motif (truncated in the

Conclusions and Discussion
The preceding sections have given an introduction to the representation of cardiac rhythms using music notation, beginning by motivating the study of abnormal heart rhythms, and proposing that music representation of actual heart rhythms may offer insights into arrhythmia variations and subtypes.After a short introduction to uses of music notation to represent heart mumurs, examples of musical representations of arrhythmias followed, covering ectopics, tachycardias, atrial fibrillation, and atrial flutter.These transcriptions, deployed at scale, could potentially yield cues for arrhythmia symptoms and treatment.Music generation from abnormal heartbeats was then described as a twopart process of music retrieval followed by musical transformation, which can be applied at the melodic or block levels, and complex combinations thereof.The generation process draws inspiration from AI techniques for music generation by sampling from and transforming existing compositions.It also presents interesting new problems not yet a staple of music information research.
Together, the challenges of representing arrhythmias with music and those in turning rhythm transcriptions to music include: accurate, reproducible, yet flexible and comparable representations of ECG features of abnormal heartbeats using music; stratification of transcriptions into subclasses; elegant solutions to combining and transforming music to fit arrhythmia rhythms; creative matching of arrhythmia sequences to music sources.The success of these tasks rely on understanding the nuances of these new problems in a new domain, and finding appropriate and efficient solutions to them.
Being able to render abnormal heart rhythms precisely and accurately into music has far reaching consequences not only for gaining insights into cardiac conditions, but also for expanding the scope of biologically sourced music.The physiology of the heart constrains music generated from heartbeats to natural sounding rhythms, even in states of arrhythmia, thus ensuring satisfactory and often provocative musical results.Furthermore, the importance of heart-brain interactions in cardiac arrhythmias [68] suggests future possibilities for integrating affective considerations into the making of music with heartbeats.

Fig. 1 .
Fig. 1.The first music representation of a heart murmur, the venous hum, (a) as proposed by physician-musician Laennec; and, (b) following Laennec's text description accompanying the initial notation.

Figure 11
Figure 11  gives an example of atrial flutter; notes depicting the atrial contractions have stems pointing up, and notes marking ventricular contractions have stems pointing down.The first bar demonstrates a 3:1 block; the second bar a 4:1 block, and so on.The underlying atrial rate of 300 bpm is indicated in the tempo marking.This concludes the discussion on notating heart rhythms.The next section describes the collage and transformation processes involved in generating music from abnormal heartbeats.

5 8
bar); (b) the descending octaves; and, (c) right hand repeated chord pairs with step-wise moving left hand octaves.
80-120Little Etudes by Elaine Chew is licensed under a CC BY-SA 4.0 international license -https://creativecommons.org/licenses/by-sa/4.0/(b) adaptation of Dave Brubeck's Blue Rondo A La Turk to emphasise the ventricular ectopics with compensatory pause embedded in the rhythm Fig. 13.Excerpts from Little Etude No.3: Bigeminy Sea-snatch and Little Etude No.6: Ventricular Ectopics incorporating different ventricular ectopic rhythms.