Using LaTeX for a Musical Edition

This post is part of a series analyzing LilyPond’s performance during the preparation of a new edition of Oskar Fried’s songs.

Of course we had known from the very beginning that we wanted to prepare the scores of our edition with LilyPond. But at one point we decided to use yet another plain text tool to put the pieces together into one complete volume: LaTeX.
I will show you some LaTeX features I learned to love during this project.
And as the question of integrating text and music pops up regularly on the lilypond-user mailing list it may be interesting to see some of the tricks we used.

I don’t know for sure about the chronology:
Did I dive into version control to manage the LaTeX file or did I decide to learn LaTeX because it is a natural choice when using version control?
Well, I’ll probably dig into that topic for another post because it might be relevant for others too.
No matter why, I did the switch, made our Fried song book my first real-world LaTeX document – and I’m very happy with the decision because LaTeX offers quite a number of advantages for such a task.

Using Version Control

This time I won’t go into detail about the advantages of version control, but of course using LaTeX we did benefit from them without any drawbacks. And it does make a difference when both the scores and the complete volume can be versioned. Add to this the bonus of being able to edit files online with most Git service providers – in fact I could even encourage a few people who didn’t have any prior knowledge to review parts of the book and apply their corrections themselves through the web interface.

Handling Large Documents

Recalling earlier projects with considerably simpler tasks I wouldn’t want to imagine editing this 124 page document with heavy graphics (scores) in InDesign. It would probably slow down to the level of being completely unresponsive. LaTeX on the other hand is simply shrugging its shoulders about such dimensions because it doesn’t have to keep such a document “open” in memory, calculate layout in real-time and still offer editing features. The file size of the “main” document is just 8 kilobytes – note that if you create an empty document with a graphical program, it will probably be bigger! (Of course this isn’t a fair comparison but it may show you what I mean.) Reordering content – e.g. moving a paragraph, adding an image etc. – doesn’t add anything to the CPU or memory usage because it’s simply a matter of copying a few bytes of plain text – whereas a graphical DTP program would instantly have to reflow the whole document.
Well, recompiling the whole book admittedly does take its time, but thanks to include-files one can work on short segments most of the time – and what’s of particular importance it doesn’t interrupt me immediately and automatically as soon as I want apply a change.

But apart from these general features of LaTeX there are some particular characteristics that make your life as a document author and typesetter more enjoyable.

LaTeX commands

LaTeX provides predefined “commands” – but you can also create custom ones. These can be used as formatting styles – as not using manual formatting should be a matter of course even with word processors – like for example \opus{3, 2} which would be defined to assign a certain font face to the content “3, 2”. But LaTeX commands are much more powerful than word processor stylesheets. I could for example define that \opus command differently, so that \opus[2]{3} would add the comma and a typographically correct space between the two arguments while \opus{3} would not print the comma and the space. Actually I can define commands like a programmer.

For the revision report of our edition I created a command \revEntry (for “revision entry”) which takes four arguments: a measure number, a position in measure, the affected instrument, and the entry body text. The separators and space between the different entries are intelligently handled so I could have entries such as

\revEntry{45}{3. \crotchet}{\lh}{Missing slur in OE.}
\revEntry{46}{}{\lyrics}{OE prints “sie” instead of “er”.}
\revEntry{}{last \semiquaver}{\rh}{Wrong pitch c\oct[2].}

which would result in

reventry-original

Here you can see several commands in action: some simple shorthands guaranteeing consistent formatting and typography for abbreviations (\rh, \lh, \lyrics), a nice little function typesetting octave ticks (\oct), and finally the revision entry itself that formats the entries differently according to the presence or absence of the different arguments.
While actively working on the report I used a modified version of the command to typeset the entries in a much more tabular and generously spaced form that simplified working with them. Note that I’m not only talking about character formatting but of a complete layout with different spacer characters, order of elements etc.

So, the exact same input code as in the previous example:

\revEntry{45}{3. \crotchet}{\lh}{Missing slur in OE.}
\revEntry{46}{}{\lyrics}{OE prints “sie” instead of “er”.}
\revEntry{}{last \semiquaver}{\rh}{Wrong pitch c\oct[2].}

produced

reventry-draft

lilyglyphs

One more type of command you see in these examples are the rhythmical notes (\crotchet and \semiquaver). Using notational symbols in continuous text was something I had been looking for since I began writing about music, but I’d never found a simple and satisfying solution. Having switched to LaTeX I gave it a new try but I didn’t find a proper solution there either, each one was limited in some way or the other: some were restricted to a certain set of symbols, some couldn’t scale with the text size, some simply looked awful.
So I took our edition project as an incentive to create a really good solution once and for all: lilyglyphs. This LaTeX package makes glyphs from LilyPond’s Emmentaler font available through predefined or generic commands, and provides an infrastructure for easily creating commands with arbitrary LilyPond notation (the notes for example aren’t part of the OpenType font but are drawn directly by LilyPond, and lilyglyphs includes them as small PDF image files).
You can get a first impression and see how easily it can be extended in my previous posts.
By now lilyglyphs is included in TeXLive (and presumably other LaTeX distributions), so users can simply type

\usepackage{fontspec}
\usepackage{lilyglyphs}

in their document preamble and start using LilyPond’s beautiful notation in their text documents right away.

Script the sources

Another nice feature of plain text files is their accessibility to automatic processing. Above you’ve seen the command \oct that typesets octave ticks. I noticed that this command didn’t work well with the note f because the bowl at the upper extender of the f collided with the first tick. The solution was simple (it isn’t the only possible solution but I went this way because it seemed the most straightforward one): I created a new command \fOct which adds just that necessary amount of space between the f and the ticks. Propagating this change throughout all the files of the revision report was a matter of one call to
rpl "f\oct" "\fOct" *. Oh, of course I first had to install this tool, which involves the complicated and tedious process 😉 of typing sudo apt-get install rpl, entering my password and hitting enter. Compare this to the traditional approach with normal style sheets…

Tyopgraphy

One last “LaTeX advantage” I’m going to cover is its extraordinary typesetting quality. After all we’re interested in “beautiful scores”, so why should we make compromises in the typesetting of text?
LaTeX (or rather the underlying TeX engine) has been developed for typesetting texts, with a strong focus on books, particularly scientific ones. One of the major consequences of this is a paragraph formatting and page layout quality that one shouldn’t even start to compare with that of word processors, and that is completely up to par with any competitor in the DTP segment.
But for the sake of making the point I will make a comparison with output from LibreOffice, copying two paragraphs from our edition to empty documents. Both were set to the same font size and page layout, I chose justified paragraph alignment with automatic hyphenation and that was all. In LibreOffice I stripped everything that can’t be realized (e.g. lilyglyphs 🙂 ), and in LaTeX I used the package microtype.

Two paragraphs typeset with LibreOffice

Two paragraphs typeset with LibreOffice (click to view PDF)

The same paragraphs typeset with LuaLaTeX and the microtype package

The same paragraphs typeset with LuaLaTeX and the microtype package (click to view PDF)

Even looking at these downscaled preview images you can see that the LaTeX rendition is more evenly spaced while the LibreOffice version has many holes in it – between the words but occasionally even between the lines.
If you now open both full-size images in new tabs to easily compare them more thoroughly you can see more details.
One aspect to note in particular is the shape of the left and right borders of the text block. In LibreOffice the elements are aligned to a perfectly straight line. In theory this should be what we want, but in reality you’ll notice that particularly the hyphens make the visual impression of that border quite jagged. LaTeX, on the other hand, makes hyphensprotrude in the right page margin (it would also do the same with quotation marks and even with lighter parts of letters such as “A” or “T” but this just didn’t happen here) – which makes the overall impression much better (if you have trouble seeing this effect, try looking at the samples from a distance).
In old books – back when people cared about the things they did more than they do now – protrusion really was the standard, while today it is a rather rare feature.
The other technique is “font expansion” that uses tiny bits of squeezing and pulling at the character shapes and the space between them. This allows LaTeX to distribute the text more evenly on a line, to make the whitespace between words more consistent and – perhaps most important – to reduce the number of hyphenated lines.
This is professional typesetting out-of-the-box.


Tips and Tricks for Merging Text and Scores

There are several ways to mix text and music in a document, but in a case like ours where we have scores and text neatly separated on individual pages it’s quite natural to create one LaTeX document for the whole compilation and include the scores’ PDF documents through \includepdf from the pdfpages package.
While this is basically very straightforward there are issues to consider, namely page layout, headers and footers and finally page numbering.

I think page layout in music books heavily differs from that of text books and therefore of LaTeX’s default behaviour. In general you will have much smaller margins on A4 or similarly large paper. In text typesetting you are confronted with the issue of reducing the linewidth in order to improve legibility – an issue that is completely different with scores. Therefore you should happily ignore LaTeX’s inclination to calculated type areas and hardcode it with the geometry package to fit your scores’ margins.

The next topic to consider are page headers and footers – who should print them, LilyPond or LaTeX? We decided to do it on the LaTeX side because this automatically takes care of page numbering. Page numbers remain consistent even if there are changes, for example through inserting pages with illustrations, and of course it would have been difficult to get the formatting of footers and page numbers identical. Fortunately it’s easily possible to include the PDF documents as a whole while still printing page headers on top of them.
But when working on a single song you normally work directly with the score, and you may consider it impractical to have a printout without page numbers. So what to do about this situation?
Our solution was part of a concept we used to call Draft Mode, which I can only recommend to adapt in one way or the other.
Basically we had include commands in each file that could load either a “draftMode”, a “prePubMode” or no style sheet.

% Optionally activate draft Mode or prePublication mode
% Both modes print a "progress bar" in the header line.
% draft mode additionally applies coloring to many manually applied tweaks

\include "../includes/config/draftMode.ily"
%\include "../includes/config/prePubMode.ily"

These modes did a lot which I may write about another time (just one note: adding the Layout Control Modes to Frescobaldi was a direct consequence of this project). But in particular they printed a “status bar” and page numbers. One little noteworthy trick was that if no mode-stylesheet was included the status bar was printed anyway but with an empty string. This way it didn’t affect page layout and we could safely switch modes on and off until the last day.

Managing such a song book with alternating text and score parts in a LaTeX document is also very straightforward because the “sources“ are always only a single compilation away. Updating a score usually means simply to first compile the score with LilyPond and then compile the volume with LaTeX, and you’re done. And when we started versioning certain states of the PDF files too incorporating Janek’s latest changes in the main volume was a matter of

git pull
lualatex -interaction=nonstopmode "main-volume.tex"
evince main-volume.tex

which can conveniently be wrapped up in a one line script, so (e.g.) update-fried will get any updates from the server, compile and display the updated volume with one command!

“The Future of Scholarly Editing”

To finish off this long post I’ll outline one tool that I missed dearly during the final stages. It isn’t a deficiency of our set-up that annoyed me, quite the contrary. It’s rather that I see a potentially revolutionary perspective, and not already having that just hurts. 😉

Within our “Draft Mode” we implemented a few editorial tools for “in-source communication”. With them we could insert commands like \todo in the LilyPond input file – which would highlight the affected notational element with a color and print a message to the console output. But I intend to vastly improve this functionality by a full-blown \annotate package. With this it will be possible to enter arbitrary annotations to a score, for example general remarks, TODOs, technical questions or critical notes. Apart from being used for visual annotations in the score (not affecting layout) these annotations are output to separate files as sorted lists – like LilyPond’s music this will support point-and-click so you can directly switch to the corresponding position in the input file when reading your TODO list. This will greatly increase efficiency when dealing with open tasks and questions during the preparation of scores, especially since this is a convenient way to pass along issues between collaboators.

But what I’m really looking forward to is having this functionality combined with a new LaTeX package that can read the output of LilyPond’s \annotate and retrieve entries for a critical report from it. Of course it will be non-trivial to implement this in a way that it can become generally useful. But I’m confident that it will provide a quantum leap for dealing with scholarly editions:
When I notice something in a musical source I’ll add an annotation to the LilyPond input file. Once I’m decided about it I can simply say that it now is a critical remark and use it in the critical report of the LaTeX document. When proof-reading the score I don’t have to tediously cross-check with another document to see if there already is a critical remark for any given issue because I can immediately see it in the score and its input file. I can even make any improvements to the remark within my score editing environment. Maybe it will even be possible to realize these annotations in a way that they affected items are slightly highlighted and the annotation is displayed as a tooltip when hovering over the note.
The ultimate step to “editor’s heaven” will finally be the implementation of a graphical annotation editor in Frescobaldi – then it will be possible to right-click on notes or other elements to directly enter or edit the critical remarks for the printed report!

OMG, why can’t I simply fast-forward to the state when this is already available…

4 thoughts on “Using LaTeX for a Musical Edition

  1. Pingback: Power of TeX - music scores | There and back again

  2. Michael

    Very interesting indeed.

    I’m typesetting an editorial paper myself with Lilypond and LaTeX at the moment and wonder whether you could put the LaTeX preamble somewhere online.
    Especially I’m interested in how you dealt with the editorial remarks. At the moment I have markups with boxed numbers within the lilypond score and would list the remarks in the paper afterwards.

    Another layout would be footnote-ish. From your exampe it looks as if you put them in a kind of table. I’d like to know how that looks like on a page.

    Could you please contact me for some chatter?

    Thanks,

    Michael

    Reply
    1. Urs Liska

      I’m not fully clear what you are referring to: which of the examples do you mean? And are you more interested in the visual aspect (how can one adequately visualize that stuff?) or in the TeX coding?

      In general I don’t have any problems making the preamble and any “programming” part available, but I’ll have to look if I can just copy it or if it should be processed before (which would make it slightly less probable to happen any time soon).

      Reply
  3. Pingback: Introducing ScholarLY | Scores of Beauty

Leave a Reply

Your email address will not be published. Required fields are marked *