Using Lilypond in the Platonic Music Engine

I have a music generating software project, written in Lua, called the Platonic Music Engine (PME). It started its life as a simple tool to be used by several other projects I’m working on but has gotten so out of hand that it has taken on a life of its own. It aims to do everything musical that can ever be done. Part of this goal is to be able to generate sheet music in a variety of styles which is where Lilypond comes in.

The PME operates by taking an initial input from the user like a name, a number, or any random string of characters and turns that string into a piece of music, a single melodic line. This music is not constructed with any conventional aesthetic ideas in mind. In fact it’s basically random. Technically it’s pseudo-random which means that the music is deterministically generated such that while the initial seed produces the same music each time, the results appear to be random. In the original incarnation of this project the PME was only going to quantize the score to make it playable on the instrument of the user’s choice while altering the musical qualities of the original music as little as possible (ie, it would still sound random).

A friend convinced me that producing random sounding music probably was not the way to gain a large enough following that would be willing to pay the bills. I reluctantly agreed and began adding what I call style algorithms to the software which allow the user to manipulate that initial piece into sounding like any style of music ever devised. Or it will, right now I’ve only programmed in a few style algorithms like one that creates a very simple Bach-like Invention, one that creates serial music, another that creates guitar chords, and others which you can see here.

In addition to producing music in a wide variety of styles it also generates sheet music for those pieces. It does several styles of graphic notation (using LaTeX) as well as standard notation using Lilypond. You can see examples of all of these here and below.

An example score produced by the PME using minimal quantization, ie, maximal random-soundingness. (click to enlarge)

An example score produced by the PME using minimal quantization, ie, maximal random-soundingness. (click to enlarge)

Audio for the sheet music above.

An example of a graphical score. This one is an adaptation of Feldman's graph notation based on the sheet music above. It makes use of an entirely optional dynamic shading feature where the the darkness (light, medium, and dark) matches the volume (soft, medium, and loud).

An example of a graphical score. This one is an adaptation of Feldman’s graph notation based on the sheet music above. It makes use of an entirely optional dynamic shading feature where the the darkness (light, medium, and dark) matches the volume (soft, medium, and loud). (click to enlarge)

In the first example above a “minimal amount” of quantization was used. But that isn’t exactly accurate. What this means is that I told the computer to take a piece of music that has more notes (128 possible notes vs the 88 that are available on a piano), greater range of durations (32767 possible values vs the 15 in use in the software), and more gradations in volume (128 vs 12 used in standard notation—ppppp to fffff) and squeeze it down to fit what an actual human can play on an actual instrument. For this example I told the computer to use the full range of values that it had available for this instrument and to treat them all equally. A more “aggressive quantization” would have been telling the software to “weight” some of those values like only using the notes in the C-Major scale and of those putting extra emphasis on the tonic and dominants and not use the supertonic at all. And then only using p, mp, mf, and ff along with just eighth, quarter, and half notes and then on top of all that only using the middle two octaves of the piano. The user has a near limitless variety of options that they can pass to the PME in order to quantize the initial music into sounding like whatever they desire and that’s before applying any actual style algorithms to it.

Using Lilypond with the PME

The PME, when called upon to, can generate a Lilypond file representing the music that it has generated. Getting it to do this hasn’t been an easy process. We are all used to having complete control over our Lilypond files in order to get the exact look we want. In this project the software has to produce perfect or nearly perfect results without any intervention at all. Users cannot be expected to edit the resulting Lilypond files themselves. This is especially the case when the PME goes live in an online version that allows anyone without any music experience to generate music.

The good news is that Lilypond is up to the task. In fact I cannot imagine any other program being able to meet this requirement. There are other text-based engraving programs but none that produce such polished output. And of course the usual suspects of programs that have graphical interfaces are of no use here.

In order to produce good results I have had to put quite a bit of effort into the process of generating Lilypond files. The rest of this post will discuss some of the details of this process.

An interesting default feature of Lilypond is that it does not automatically split notes and rests across bar lines. It is expected that you, as the person creating the Lilypond file, will manually indicate how and when notes are split. In the case of the PME, many of the style algorithms do not generate music that is actually written with a time signature in mind—it’s basically long cadenzas. But I decided that it looks better, and makes it easier to read, if there are bar lines anyway. The user is allowed to choose any time signature they want in most style algorithms, but because most of these style algorithms do not generate music with any time signature in mind this creates a problem. Lilypond will just put a note at the end of a measure, with its full duration, regardless of how little sense it makes.

Fortunately Lilypond also gives you the means to disable this feature and will split notes and rests across bar lines in a fairly sophisticated manner. You simply add the following code in your \layout section:

\layout {
  \remove "Note_heads_engraver" 
  \consists "Completion_heads_engraver"
  \remove "Rest_engraver" 
  \consists "Completion_rest_engraver" 
}

Now Lilypond will split your notes across bar lines.

An example score produced by the PME using minimal quantization, ie, maximal random-soundingness

The first two measure of the score above. The snippet on the left shows the default Lilypond behavior, the snippet on the right shows Lilypond automatically breaking notes across bar lines.

The PME will generate music for any instrument (that it has a definition for—right now this comprises the instruments making up general midi plus a few others) and tries its best to make sure the sheet music is appropriate for that instrument. Besides making sure the music fits the range of the instrument it also attempts to generate the sheet music using whatever is standard for that instrument including the proper clef and transpositions.

One aspect of making music look good and natural for an instrument is the judicious use of ottava markings. Because the PME does not allow us the privilege of manually altering the Lilypond code, we either have to do without ottava markings or figure out a way to automatically add them from within the software. Originally I wrote a lot of code to calculate good spots to insert ottava markings based on the octave of the pitch. Unfortunately it was very difficult to generate good results, either the marking would come in too late (the pitch too high above/below the staff) or too early and there was little I could do about it without devoting a massive amount of time and code to tracking additional data.

Fortunately the amazing Lilypond community came to the rescue. I posted a question about automatic ottava markings to the Lilypond list and while Lilypond has no such built in capability it does provide enough of a customizable framework via its Scheme scripting engine to add these kinds of features. In fact David Nalesnik very generously created a Scheme function that would automatically insert ottava markings based on the number of ledger lines used instead of the octave of the note which had been my approach. This Scheme function allowed me to rip out a significant amount of code that I no longer have to maintain and update when changes are made and the results look even better. You can find a download link for this function here and see it in action in the first image above.

Similarly, dealing with enharmonic spellings has been a huge headache. This involved multiple tables for different keys and root notes. And whenever fundamental changes were made to the underlying infrastructure, I had to update all the tables and the code that processed them. It was an even larger and more unwieldy mess than the ottava situation above.

And then while looking through the mail list I came across a post by Peter Gentry describing some work he had done extending an existing Scheme function dealing with automatic enharmonic spellings. We discussed the issue, with several other people, and he ended up creating a function that handles automatically forcing the appropriate enharmonic spelling for whatever key you’re in. Here’s a link to my gitlab project page where you can download his code.

With this function in hand I was able to rip out all of my code which would try to use the appropriate enharmonic spellings in the sheet music. As it stands now the PME only generates Lilypond code with sharps (and naturals) and the enharmonic.ly function determines if they should be flats instead and makes that change. This has improved my code tremendously and produces perfect results.

In this example the notes sent to Lilypond were all naturals and sharps and the enharmonic.ly script converted the sharps to the enharmonically equivalent flats. This example also shows more aggressive quantizing starting from the same source material as above. (click to enlarge)

In this example the notes sent to Lilypond were all naturals and sharps and the enharmonic.ly script converted the sharps to the enharmonically equivalent flats. This example also shows more aggressive quantizing starting from the same source material as above. (click to enlarge)

One final example of Lilypond’s functionality and then I’ll wrap this up. One of the style algorithms I created is basically a musical version of John Cage’s Mesostic (also see below for an example from Cage where he found the spine “James Joyce” in Joyce’s Finnegans Wake) method of generating poetry from a source text. My style algorithm creates a standard score and then “finds” famous melodies within it like the opening theme to Beethoven’s Fifth Symphony or the theme from Bach’s Toccata. The sound file plays the source music very softly and when the found melody notes occur it plays them loudly providing a kind of aural analogue to the typographic method Cage employed. But then I also wanted to have the score make the found melody clear. Lilypond colors the melody note red and depending on the style chosen either makes the melody note always appear at the beginning of its own line (acrostic), be somewhere in the middle of its own line (mesostic), or just be displayed following standard notation practice (standard). Implementing this was trivially easy and did not involve much more than adding the line

red_note = { 
  \once \override NoteHead.color = #red 
  \once \override Stem.color = #red 
}

to my Lua code and then inserting the command,\red_note, into the appropriate spot while the software generates the Lilypond file.

whase on the Joint
           whAse
          foaMous
          oldE
            aS you

             Jamey
             Our
       countrY   
   is a ffrinCh
        soracEr this is

(Mesostic created by John Cage using “James Joyce” as the spine and Finnegans Wake as the source text to search through.)

The Musical Mesostic style algorithm using the acrostic formatting. The spine is a rather famous motif. (click to enlarge)

The Musical Mesostic style algorithm using the acrostic formatting. The spine is a rather famous motif. (click to enlarge)

A rather famous musical motif should be audible above the rest of the music.

The Future and Final Thoughts

I am already anticipating certain challenges to come. One of which is a generic method of indicating microtones using any tuning including those that divide the octave into any number of divisions (eg, 128-EDO, 1,000-EDO, 3,728.31-EDO). My thoughts are that once we’re past what can be done with the normal accidentals (quarter sharps, triple quarter sharps, and ignoring enharmonic spellings) that there will need to be a new approach. One that I’ve successfully experimented with is creating accidentals that show the percentage distance from the base note to what it sounds like. So if a note is 18.4% of the way between its root note, G, and the next note, A, then the accidental would be something like “184” arranged vertically and attached to the G. Not that this would be particularly useful for a musician or that any instrument exists that can play in 1,000-EDO tuning, but that the sheet music will be accurate and still look good. This is the Lilypond code I used in my tests:

{
  \override Accidental #'stencil = #ly:text-interface::print 
  c' e' 
  \once \override Voice.Accidental.text = 
  \markup { 
    \override #' (baseline-skip . 1.2) \teeny 
    \column {1 8 4} 
  } 
  gis' e'
}
Proof of concept for a new accidental style for microtones. This is a G that is 18.4% of the way between a G and an A.

Proof of concept for a new accidental style for microtones. This is a G that is 18.4% of the way between a G and an A.

More work will need to be done to make this method work really well but I feel like I’ve demonstrated that it can be done. (Now no one should be surprised when a post to the mail list shows up with some questions about how to tweak this code.)

I am happy with the graphical scores that I am producing with LaTeX but feel confident that I could also produce them in Lilypond using postscript (or some other method?). This would make formatting the headers and title pages much easier and if done in a general enough fashion could be useful for other Lilypond users. If someone wants to help me understand postscript and how to integrate it into Lilypond then I’d love to tackle that issue.

Additionally, any composer who would like to see their works adapted for the PME or any musician who has an interest in seeing the works of any composer of any style from anywhere in the world from any time similarly adapted, please contact me. I want to include every musical idea that has ever existed or will ever exist and I really enjoy the process of collaborating with others in making these things happen. An example here would be to take the very simple Bach-like Invention style algorithm I’ve done and replace it with algorithms that more closely match what Bach actually did, like an algorithm for each Invention. Basically if there’s any piece of music or musical style you’d like to see algorithmized I am up for it.

At the end of the day Lilypond has proven itself capable of generating beautiful scores automatically without any human intervention. My software creates a Lilypond file and with some specific choices made by me and plenty of help from the community (both answering my questions and generously providing new Scheme functions), the resulting sheet music is both accurate and beautiful.

I remain further convinced that there is simply no other engraving product available that can produce these results at this level of quality. And I also remain certain that no matter what issue I have a solution is available thanks to the wonderful Lilypond community and the hard work all the developers have put into making Lilypond the powerful and flexible product that it is.


David Bellows studied composition at various schools in East Tennessee. He has written music for the concert stage and gallery openings and of late has been taking private commissions. He is currently in Bellingham Washington, US, and is looking for a place to live.

4 thoughts on “Using Lilypond in the Platonic Music Engine

  1. Michael Collins

    Excellent post, Dave. While this is an outstanding summary of the PME’s technical side, I’d also like to briefly discuss the philosophy behind the work. This will help people understand your motivations, goals, and view of the PME’s place in the Western musical tradition.

    In our email correspondences, you repeatedly reference your belief in the “post-modern dilemma”. Since the compositional experiments in the early to mid 20th century have pushed the definition of music as far as it’s capable of going, the best we post-moderns can do is put our own spin on older styles. True novelty is no longer possible.

    This may help newcomers understand *why* you want to code style algorithms or create midi tables for alternative tunings. You’re basically collating the “building blocks” of 2500+ years of musical culture. The user is then free to combine these pieces however he wishes. He’s not creating anything new per se, just recycling the past like all post-moderns must do.

    Reply
    1. David Bellows Post author

      And for some context, Michael Collins is a composer whose works I am in the process of adapting for use in the Platonic Music Engine. You can see an example of what I’ve done with his piece Lex here and his original version here.

      Adapting the work of other living composers presents interesting challenges but I always work to capture the soul of the original while preserving some rather vaguely defined aesthetic of the PME.

      I should also note that as per his comments above while it is true that I do not believe there is anything fundamentally new to do (works that challenge at a fundamental level how we think about art/music), I do believe there are still infinitely many more works to be done in infinitely more styles yet to be created. The distinction is subtle, perhaps, but important.

      Reply
  2. Urs Liska

    I like this concept very much!

    While I’m not completely sure if you’ll ever arrive at a point where this can be considered “ready” (I think we don’t have to bring in “finished” here) I think it is a worthwile artistic adventure. For me it has a beautiful balance between author’s responsibility, randomness, “user” input and aesthetic variety. In particular I have the impression that it is a very strong thought to have the user trigger an “ideal” but unheard – “Platonic” – score, thus creating something personal (i.e. generated through something personal) and then have multiple ways to create audible music in a theoretically endless variety of styles. The “user” is thus capable of producing anything he considers right or that simply pleases him, but still has the foundation of a strictly thought-through, algorithmic process, so the result is not completely arbitrary.

    Reply
    1. David Bellows Post author

      I like this concept very much!

      Thank you! I’ve devoted my life to it so I hope at least a few people are interested in it!

      While I’m not completely sure if you’ll ever arrive at a point where this can be considered “ready” (I think we don’t have to bring in “finished” here) I think it is a worthwile artistic adventure.

      Part of the beauty is that it will never be finished. There will always be new musical ideas to be added and explored. But at least the infrastructure can be in place.

      In particular I have the impression that it is a very strong thought to have the user trigger an “ideal” but unheard – “Platonic” – score, thus creating something personal (i.e. generated through something personal) and then have multiple ways to create audible music in a theoretically endless variety of styles. The “user” is thus capable of producing anything he considers right or that simply pleases him, but still has the foundation of a strictly thought-through, algorithmic process, so the result is not completely arbitrary.

      You’ve nailed it. The results will always preserve that intimate connection to whatever the user first used as a seed making the result unique and personal. And then being able to explore that Ideal Platonic music through nearly limitless possibilities of quanitizing and applying style algorithms will hopefully make for a compelling and pleasurable experience for the user.

      And of course composers are free to, and are encouraged to, add their own musical ideas to the PME (with them being given full credit with links to their websites or whatever they want).

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *