Annotating scores

(This is a post in a series about the “Crowd engraving” project “Oskar Fried: Das trunkne Lied”)

For quite some time I had been thinking (and to some extent writing) about the potential LilyPond’s plain text approach offers for implementing in-score annotations. The idea got stuck in a state of initial drafts because it requires Scheme skills that I don’t really have, but our crowd engraving project with the huge score was an incentive for me to implement at least a “working draft”, and as it turned out this was a priceless addition already in this initial state.

The Idea

LilyPond’s music input is stored in plain text files and therefore it is possible (and recommended) to add source comments in the files to document them and make them easier to read and maintain. The most obvious targets for these comments are usually formal hints (measure numbers or structural items like “coda”, “3rd verse” etc.) but one can of course also annotate the content (“is this really a g' or is there a flat missing?”) or issues with LilyPond coding (“I don’t see how to align these two notes properly”). A typical use case are also “todo” comments (“Check this again when line breaking is fixed”). However, the concept of code annotation can be extended by implementing comments that actually do something.

The Implementation

A characteristic of code comments (actually that’s their inherent benefit) is that they are completely ignored by the “compiler” (i.e. LilyPond), and therefore the user can insert arbitrary text there without worrying what the compiler will think about it. But in a way this is also a limitation. Code comments are good to document code for someone who is reading through the specific spot in the code, but they are not useful when you want to actively point others to an annotation.

For this purpose we had already created a few functions in our earlier Oskar Fried project and called the library “in-source-communication”. Basically these were commands that highlight something in the score and produce a message in the console output of LilyPond. That way we had a convenient way to announce things to the collaborators and have these items be tied directly to the music input. For the current project I then implemented some functionality that doesn’t do much more yet but that can hopefully be developed into a really useful feature for scholarly use of LilyPond in the future.

There is a function \annotate that can be used like this:

  \annotate \with { type = "musical-issue"
    context = "bassoon 1"
    source = "OE"
    author = "Urs Liska"
    date = "2014-09-13"
    message = "Hairpin missing in OE but present in contrabassoon"
  f2. ~ \< |
  f1 \!

This will highlight the affected grob (here: the hairpin on the f) through coloring (if switched on) and produce a clickable message in the console (if switched on). Through this the compiled score immediately reveals problematic spots in the score, and the compilation output in the console “invites” to browse through issues. There is much more this should/could do, but for now I’m happy that we can at least enter annotations and have these comments as obvious as e.g. an issue tracker while being directly tied to the actual music input.

Practical Implications

As mentioned earlier the original score of Oskar Fried’s “Das trunkne Lied” contains numerous errors or questionable notations. While we are not commissioned a scholarly review our contributors are encouraged to annotate and/or fix anything they notice along the way. This means: If someone notices an obvious error he should fix it – but document that with a critical-remark annotation. If a dubious notation is encountered (e.g. a hairpin that is missing from a voice but present in others) a musical-issue annotation can be added, so the question can be discussed later. In the end all musical-issue annotations should either be turned into critical remarks or discarded. Usually in the workflow the above musical-issue annotation would be processed during review and probably be changed to a critical-remark annotation.

So far (that is with about 60 % of the instrumental parts and none of the vocal parts entered and reviewed) we already have more than 880 annotations in our files! Among these 515 are labeled critical-remark and 300 refer to open musical questions – all of them already peer-reviewed. This means we will probably identify far more than 1.000 errors in the score during music entry alone. I find this is a remarkable number, given that our team has few professional musicologists and that we didn’t even start doing something like an explicit scholarly review.
I’m quite sure these wouldn’t help us as much if they were entered as plain code comments. Having them encapsulated in explicit commands instead makes them ready for further elaboration and processing.

Need for a GUI

Apart from many new functions that I would like to have there is one thing that makes working with \annotate still somewhat unsatisfactory: it’s quite verbose. I mean as you can see above adding an annotation implies writing quite some code, a lot of which is actually redundant. The musical context could be determined from the file, the author and date could be retrieved from the version control commit. In theory one could even write the annotation text in the commit message. Not all of this would be equally useful or practical, but in the end it’s tedious and somewhat error-prone to enter lots of annotations. I’m sure if it were more straightforward to add an annotation we’d have an even lower threshold and higher number.

One solution to this which I want to develop (but I doubt I’ll be ready with it in our current project) is a graphical interface to annotations to be used in Frescobaldi. This will then provide right-click access to an annotation editor where things like annotation type or author can be chosen from a dropdown, while date and context will be inferred from the actual environment. Having annotations directly in the music input is a conceptual advantage over other approaches where one has to maintain the music and the scholarly review separately, but having graphical access directly from the score view will make that unbeatably natural.


I will only touch the potentials of \annotate, but if someone is interested in this kind of functionality and has the necessary Scheme skills available at his fingertips I’d be more than happy about assistance in developing this further.

One of the next things to do is making the function aware of the annotation type. Depending on that it could then respond individually. E.g. musical-issue annotations would be colored only while developing the score while critical-remark annotations would call an “editorial function” on the affected grob, so that for example slurs annotated with a critical remark are dashed automatically. It would also be great if annotations could directly create footnotes or print annotations directly in the score, possibly on a separate PDF layer.

Another wish is outputting annotations to separate files, together with the position in the score which should be retrieved by LilyPond when it compiles the score. This would result in viewable files with annotations, grouped by type and sorted by measure, which provide point-and-click support to directly navigate to the annotated music.

Finally the output of critical remarks should possibly be done in a way that it can directly be used from a LaTeX document and used in a critical report. If that’s possible I could finally edit the critical report directly in the score, with everything documented and maintained in version control’s safety net. This would (will?) really be a killer application for scholarly edition!

[Edit (2014-11-14):]
In the meantime the usability of the function has been improved. The musical context is determined automatically, and there are a number of wrapper functions that
remove the need to specify the type separately. The function can now be used like this, which is much less redundant and more informative:

\criticalRemark \with {
  author = "Malte Meyn"
  message = "There is only one hairpin plus pp between bcl and horn 1.
            Probably it is meant for both instruments."
e4. \>

Now the different types of annotation cause different highlighting colors so it will be much easier to spot those annotations that still have to be discussed. And an additional improvement is that annotations can now also be used as \tweaks so that it is possible to annotate one note head or one accidental from within a chord or similar things. The other wishes are still open, though …

BTW: we’re already at 680 critical remarks and 350 musical issues …

One thought on “Annotating scores

  1. Philippe Massart

    It sounds promising. 🙂

    Outside the crowd engraving application, the idea of implementing annotation and giving it an output is also interesting for “non-LilyPond” people working on a music score: proofreader, composer, any people needing to access those annotations but who are not always using LilyPond itself.


Leave a Reply

Your email address will not be published. Required fields are marked *