When we began working on the Fried project, LilyPond’s current development version was 2.15.37. Since that time a new stable version – 2.16 – was released, followed by a steady flow of 2.17.x development releases. How did the changes in the program affect our workflow?
Trying to keep up
Initially we tried to keep up with the updates and always use the latest development version. This was partly due to the fact that some new releases contained fixes for bugs that affected us – for example, we reported a problem with the formatting of hairpins (issue 2532), which was fixed in 2.15.39 release. However, this proved to be too much work: for each new version of LilyPond, Janek had to compile all beautified scores and make sure there are no regressions (i.e. check if the new version didn’t introduce any unwanted changes). Doing this meant visually comparing dozens of pages, system-after-system. And it wasn’t enough to check if the new version was the same as the old – most of the time there were subtle, harmless changes in spacing. Note that changes not necessarily are problematic – in general you can say that new LilyPond versions make the layout of your existing scores even better – but still you have to carefully check whether individual changes in the new version aren’t for the worse.
Our worst enemy here was the butterfly effect that Janek already mentioned in his post about slurs: some tiny difference in the positioning of a dynamic may influence LilyPond’s decisions regarding line breaks, and when line breaking changes everything becomes different. It’s almost impossible to swiftly compare two versions of a piece that have different line-breaking, because you cannot use “Alt-Tab comparison”.
Turning to a stable approach
At one point we therefore decided not to change the LilyPond version unless really necessary. Nevertheless, the situation wasn’t so simple: for quite a long time Mike’s skyline patch, which we used, was in active development and produced uneven results – some scores looked better with it, some worse. Because of that we used different LilyPond versions for different songs, which was very inconvenient. So we made one last change when Mike’s skyline patch was finished and merged into the official LilyPond codebase – since it was stable and consistent enough to be used in all songs, we upgraded all source files to 2.17.3, which once more involved some extra work to fix regressions.
We kept to this for a whole year – with one exception: For the beautification of Heiterkeit, güldene, komm! op. 7, No. 1 Janek wanted to use a special function he wrote and didn’t want to miss in this extremely complex song. Since the function required LilyPond 2.17.25 he decided to upgrade just this song to the latest development version available at that moment (2.17.27). Everything moved smoothly, but we wouldn’t want to go back to the earlier situation described at the beginning of this post. Particularly not if you consider that due to some internal changes it wasn’t possible to compile this song with LilyPond 2.17.29 (of course that’s not anything to say against LilyPond devs – it’s just the normal way of software development that in development versions anything can change at any time to be in the best possible shape for the next stable release).
As mentioned at the outset of this article checking for regressions was a significant time-sink. Unfortunately this problem appeared not only with changed LilyPond version – it appeared all the time: since any modification (to the music or e.g. to a function in the library) could change the layout of a score, we had to check for regressions after every big change to be absolutely sure that nothing broke. There was really not much that we could do about it, apart from being cautious. Fortunately using version control meant that if anything became broken, it wasn’t difficult to get back to the last known good state, even if the offending change wasn’t the last one. This allowed us to decrease the frequency of checking the scores – in the end it seemed enough to check every score once, after finishing beautification.
Returning to custom and volatile builds
The situation got a last turn right before the end. When seeing all those scores in their “beautified glory” Urs expressed a few more wishes because some design elements didn’t fit perfectly in the “style” of the scores. A characteristic of the style sheet that we had developed is that most of the lines are somewhat thicker than default – which makes it very enjoyable to play from the scores. Unfortunately there are elements that can’t easily be made thicker and consequently were now somewhat inconsistent – a fact that only became that obvious in the context of the beautified scores. In particular Urs wished to have portato dashes and chord brackets heavier but didn’t find any settings how to achieve that.
Janek quickly found out that the thickness of the chord brackets (actually they’re bracketed arpeggios) is for some unknown reason hardcoded in LilyPond’s C++ source files and that it would be a breeze for him to change that. As a LilyPond developer he could easily modify the source code and compile a custom build with which he then could compile affected scores (i.e. those containing chord brackets). Luckily he didn’t simply change the hardcoded thickness but added a settable property for it, so we can hope to see this improvement in default LilyPond soon. Unfortunately, since Urs didn’t have access to custom LilyPond builds, he couldn’t compile the scores containing chord brackets properly 🙁 .
Now, portato dashes are part of LilyPond’s notation font Emmentaler and therefore naturally don’t have any properties that could be overridden in LilyPond input files. But as the source code of the fonts is also part of LilyPond’s source repository, and the fonts are directly compiled from them when building LilyPond, Janek could simply modify Emmentaler’s source files to make the portato somewhat thicker and compile another custom build containing this improvement too. Again, without access to custom LilyPond builds Urs was dependent on letting Janek produce new PDFs.
Our final judgement on the situation? Well, it’s really ambivalent.
- It is really nice that LilyPond generally improves, and that these improvements
automatically affect existing scores. It is also nice that you can live
“at the cutting edge of technology” by using the latest development versions –
something you don’t have with big commercial products.
But it’s way less nice to experience the trouble that is tied to changing LilyPond versions.
It is also exciting that LilyPond is accessible to actual improvements from within
an edition project – by either extending it with Scheme code in the input files
or by actually making modifications to LilyPond’s source code. This is really a unique
feature of Open Source Software.
Of course, not everyone has an actual LilyPond developer in the team, so some of these
opportunities may not be available to you. But if you are able to compile LilyPond
from source code (and Janek’s work on a special building script should make doing this
easier), you may ask for assistance on the LilyPond mailing list and try using some
We are now in a situation very similar to that of the beginning: the scores in our
edition have to be compiled with specific, custom versions of LilyPond, ones
that aren’t easily available but have to be built from LilyPond’s
And our conclusions, tips and suggestions?
- Make a version freeze at the latest possible moment and keep any work following that moment as concise and short as possible.
- Try to determine all possible engraving issues before making the version freeze and before starting beautification.
- If custom modifications to LilyPond have to be done, do them once and at this point.
We already mentioned version control in this article, but in his next post Urs will write much more about this and how it saved our day more than once 😉 .