LilyPond and the World of Digital Music Editing

Ever heard of MEI? As a LilyPond user and regular reader of our blog you may have not, but let me assure you that MEI is one of the most exciting prospects LilyPond has had in recent years. Embracing the Music Encoding Iinitiative and interfacing with its data format will open up new horizons for us. LilyPond can be incorporated into Digital Music Edition, a realm where Finale and Sibelius fail by design. This post goes into some length introducing aspects of digital music edition and how LilyPond may (and will) fit into that system. As a surplus there’s a PDF essay attached that gives even more examples and thoughts on the subject, although more directed towards MEI-related readers.

What Is MEI?

Grossly simplified you can see MEI as the academic and free companion of MusicXML. Where the latter was created from the perspective of music software exchanging (mostly pop) music data, MEI was invented as a tool for the (scholarly) editor to comprehensively encode music and its sources. Although much less widely known than MusicXML MEI is actually at the core of, and the de-facto standard within, current trends in (digital) (scholarly) music edition.

Analog Versus Digital Musicology

These days musicology stands at the crossroads between the “analog” and the “digital” approaches to music edition. Significant efforts, thought and discussion have been spent on the topic of exploiting the potential that digital data processing offers over mere printed editions. Clearly the tendency is to move away from monolithic concepts with one authoritative text and its justification in a separate critical apparatus. Instead readers are now considered “users” who can interact with the “edition”, following hyperlinked reading paths through a web of information or compiling their personal version from choices offered by the editors.

Encoding and Editing

At the core of such concepts is the encoding of the material to be edited, some even go as far as to say “The encoding is the edition”. Although it is perfectly possible to “only” encode the musical content most projects are in one way or the other interested in encoding much more information from the physical sources. I have seen one project where even the shapes of all “objects” on the manuscripts are retraced and encoded as SVG objects. This way the encoding of a musical object “knows” where that object is on the original page (i.e. the scanned image). You can see this in action on the demo page of “Beethoven’s Werkstatt”. When you look at one of the finished pages you can click on elements in the scans to see them colored. Or you can check boxes to highlight material that represent a given stage of the “genetic process” leading to the final page. And when you open the “Variantenabfolge” tab on the right side of the screen you’ll see the rendered music where you can click on items and have them colored in the scan! All this is extremely fascinating but it also has to be admitted that the digital edition concepts are still in an experimental stage, and not everything that is just feasible may prove really useful in the long run and return significant musical insights.

Anyway, MEI aims at being a comprehensive format to accommodate all these encoding goals for possibly all conceivable music. To achieve this it is not simply an XML Schema but rather an extensible framework with a number of default Schemas which can be extended at will to incorporate targets that are not covered by the standard yet.

But MEI is not only that specification, it is also the community that has evolved around it, and this is actually the core of any future oriented (academic) music editing. If you are interested in public funding for musical edition projects you have to be “digital” today, and if you’re going the digital way there are few options besides MEI.

Visualize the Encoding

However, there is – still – one drawback to note. So far there is no proper way to create professional engravings from MEI data. Engraving quality has not been the main focus of digital edition concepts so far, simply because the inherent complexity forced the concentration on the core set of challenges. And maybe that shiny interactive stuff is not only more sexy but also the genuine progress in digital music edition. Producing printed and carefully engraved scores from digitally encoded data may seem totally old-fashioned, and it is definitely not the most exciting area of digital edition. But I am deeply convinced that beautifully crafted scores are a tradition that should not easily be discarded. A day will come where an edition will be performed from, and then it will make a huge difference. So tools are necessary that take the flexibility of an XML encoded music edition and bring it to the virtues of candcrafted music notation.

In recent years astonishing progress has been made with an MEI rendering tool, Verovio, that is a really clever piece of software. It allows real-time rendering, for example in a browser, with automatic reflow upon resize, some graphical editing tools that will instantly update the XML source files and re-render it in the browser. However, its focus is on being fast, and its approach to music layout is (consequently) rather simplistic, so I don’t really expect this tool to reach professional engraving quality anytime soon – and probably this isn’t even an appropriate goal for it.

Don’t Visualize — Engrave!

I wouldn’t be surprised if you already guess what I’m after 😉 . Talking about professional engraving quality? Talking about editions encoded as text files? Well, then LilyPond should be the perfect solution, shouldn’t it? Well, in a way it is, but there is this “small” nuisance, namely: LilyPond can’t make any use of MEI files so far.

Therefore I decided to take over some responsibility and make the idea of interfacing LilyPond with MEI a personal matter as I’m convinced it will be a tremendous asset for both the MEI community and LilyPond. I have already talked about the value of high-quality engraving in digital edition, and for LilyPond such an integration would open up a whole new range of potential users and applications. And these users, namely academic institutions and projects, are quite likely to contribute back significant amounts of improvements and enhancements to LilyPond itself.

Neatly Interfacing History and Future

Neatly Interfacing History and Future

As a first major step I gave a presentation at this year’s Music Encoding Conference which took place in a beautiful historic university building in Florence. And it turned out my perspective on the situation was mostly true. More or less everyone involved in digital music edition knows LilyPond, many even use it personally, but very few are actively monitoring its development. So far LilyPond can’t be used for MEI projects, implementing such an integration is a non-trivial and presumably large task, and nobody has spare time and resources. So there just is no real link between the two.

Adequate recreational space for coffee breaks

Adequate recreational space for coffee breaks

Consequently my plan was to introduce myself to the MEI community and give an up-to-date report of LilyPond’s current state and development. Showing how much has recently been done for embracing scholarly application I also wanted to make obvious that there now is an active interest and some developer potential for the integration within the LilyPond community.

Heading Towards the Future

It seems my anticipation had been slightly wrong in one respect: I had been afraid that “digital music edition” generally doesn’t care anymore about beautiful scores. But this seems not to be the case, rather scholars do have that inherent desire but simply don’t see the option. In a way it felt like showing the audience a golden path. So feedback was somewhat overwhelming, and I had plenty of opportunity for very interesting and partially also promising conversations during the conference days. It seems there are numerous edition projects that would benefit from this development and which could support it in one way or another. At least two development projects have/had already started working in that direction, and there are currently plans to apply for a properly funded research project.

So while talking about engraving MEI editions with LilyPond is still talking about the future there clearly is a road to that future. Now we have to join efforts and work towards it. If you are interested in this subject – in general or from the perspective of a concrete project don’t hesitate to get in touch: Write me an email if you know how or drop a comment and I’ll find you. Clearly there’s more to it than can be disclosed publicly yet …
Attached you’ll find a written and extended version of my Florence presentation, giving a breakdown of the idea of interfacing LilyPond with MEI. It is written with MEI users and developers in mind so as a LilyPond user you may find some information missing or superfluous. On the other hand, if you are new to concepts of plain text, semantic markup and version control you may consider first having a look at my earlier fundamental essay about the benefits of using plain text tools in music.

Download PDF

5 thoughts on “LilyPond and the World of Digital Music Editing

  1. Abraham Lee

    Beautifully presented essay! I loved reading through the PDF. I am always amazed at LilyPond’s engraving quality, but I am getting more and more excited at the future prospects of data exchange with LilyPond, whether it be MEI or MusicXML or whatever. I think these converters should be given a lot of love so they work as correctly as possible for non-LilyPond users so they can be more comfortable with getting their feet wet than they would otherwise be willing to do. Keep up the amazing work!

  2. Pingback: Music Encoding Conference 2016 | Scores of Beauty

  3. Nuno João Casteleira

    The essay you produced is beautifully typeset. Could you point to the source file? I’d be very interested in studying it as I’m writing my master’s thesis in Music Teaching with Pandora and I’d like to include musical examples as you have in your document!



Leave a Reply

Your email address will not be published. Required fields are marked *