Honestly: Is LilyPond Good Enough?

Recently Daniel Spreadbury (formerly senior product manager at Sibelius, currently product marketing manager for Steinberg‘s upcoming notation software) published a post documenting the development process of Steinberg’s scoring application, which ignited a discussion both in the comments under his post, and on the LilyPond-user mailing list.

As I watched the discussion and read various comments from LilyPond users (explaining why they believe LilyPond is the best notation software around), I once again asked myself this question: are we fair in our claims about LilyPond?

You see, quite often I have an impression that some of us get too enthusiastic when advocating LilyPond – we sometimes say that “LilyPond’s output is perfect”, “you don’t have to do any manual adjustments to get publication-quality scores”. These are, unfortunately, overstatements – and thus it would be good to avoid them, and stay realistic. Don’t get me wrong – I believe that LilyPond is great and that, in many cases, there’s no better alternative: that’s why I continue to use Lily myself and recommend her to other people! And of course you don’t get publication-quality scores out-of-the-box by any other notation software. My point is that in the sea of marketing bullshit – when everyone claims that their product “produces perfect results”, “is the industry standard”, “is the only one allowing you to create anything you can imagine”, etc. (do these claims sound familiar? Very similar ones appear on the website of one of the major notation packages) – having realistic claims is like fresh air, and can win us trust of the other people.

Of course, I have to plead guilty myself: I was an over-enthusiastic user initially, saying “look at how perfect this LilyPond-made score is” and “all other software is crap” to everybody around for some time. Since I’ve realized that, I became sensitive in this area – that’s why I write this post.

So, back to the question from the post title: is LilyPond honestly good enough for everyday and production use?

Firstly, let me say that I’ve never seen a score that was engraved by LilyPond perfectly out-of-the-box, with no tweaks. Not even a short one containing just a dozen measures – there’s always some kind flaw: a wrongly placed beam, an ugly tie (hopefully Tie Crusade will help here – in the next post I’ll give you an update about the situation), or the page layout is off.

Again, don’t get me wrong: in many, many cases the default output is very good or even excellent – Urs Liska posted an example some time ago. But when I say perfect, I mean perfect: flawless, with nothing that could be corrected. We haven’t reached that level yet (if you have a counterexample, please post it!).

However, the percentage of scores that are engraved perfectly out-of-the-box is not a particularly good metric. What really matters is the average effort needed to bring a score to desired quality: do you have to manually adjust numerous objects and change lots of settings, or is it just a matter of inserting an occasional linebreak or inserting a few tweaks?

The answer is: it heavily depends on the music. I would say that:

François-Joseph Gossec - 1st Symphony

An example of music that LilyPond handles extraordinarily well by default- 1st Symphony by François-Joseph Gossec, taken from the Mutopia Project.

“Classical” music (before XXth century) is usually easy to engrave. LilyPond’s out-of-the-box results are usually better (quite often a lot better) than the default output from other notation programs, and improving it to publication quality in most cases requires a low or moderate amount of work. Advanced LilyPond knowledge is rarely needed here, but usually you have to go beyond note-entry (i.e. you need to use simple overrides). In general, LilyPond usually is a winner here.

Zauberbuch excerpt (Mike Solomon)

An example of contemporary notation (Zauberbuch by Mike Solomon).

Modern music containing unusual notations requires a lot of LilyPond skills, and for many people without basic programming background it will be very difficult. On the other hand, you can take advantage of LilyPond’s programmability (this can be a big advantage for algorithmic composers). So, depending on computer skills of the user, producing “advanced” notations using LilyPond can range from being “insanely difficult” to “awesomely powerful”.

Vocal music generally requires more work than instrumental music, because lyrics often cause spacing problems, and there is no solution for automatic combining separate parts on one staff (it can be done, but it requires some work). I don’t think that other programs fare better in this regard, but there is much room for improvement (I’m trying to fix some issues in this area, but the work is moving forward extremely slowly). In this area LilyPond is neither worse nor much better than other software.

Eja Mater (Antonín Dvořák) choir

A fragment from Antonín Dvořák’s “Stabat Mater”. A handful of overrides were necessary to ensure good page layout, because the staves were spaced too far apart by default.

Parts for monophonic instruments quite often are engraved very good out-of-the-box, and with a moderate amount of quite simple tweaks they can be brought to perfection. Here other programs usually fall behind, as more notation elements have to be manually adjusted for an outstanding result.

Eja Mater (Antonín Dvořák) clarinet

A clarinet part from the same oratorio. It was easy to tweak it near perfection (in fact, the default output was already very good).

fried 4-1 unbeautfied

An excerpt from “Die Mauer entlang” by Oskar Fried (op.4 No1). A lot of work was necessary to bring this score to publication quality.

On the other hand, polyphonic instruments such as guitar or piano may require a lot more work. Again, much depends on the kind of music, but especially if there’s a lot of slurs, unusual tuplets, cross-voice ties and voices jumping around between staves, getting everything right can require a lot expertise and effort. This is particularly true for complex piano music, where many elements have to be repositioned manually – which gets cumbersome with LilyPond’s text input. While I haven’t used graphical notation software for quite a while, so my judgement might be significantly off, I suppose that in some cases producing good-looking results may be faster in these programs. At any rate, LilyPond isn’t a very convenient tool for such work – although it is definitely possible to create professionally-looking piano scores, default output is rarely sufficient (it is usable, but contains obvious problems).

Die Mauer entlang (Oskar Fried, op.4 No1)

The above fragment after “beautifying” – slurs were reshaped, distance between staves was specified manually, beam slope was corrected, dynamics moved around etc.

In any case, achieving perfect results (beyond “basic publication quality”) is usually tedious, because it involves moving many objects around, and the offsets for that have to be determined by trial-and-error. On the other hand, having tweaks explicitly written in the source file is very helpful as one can clearly see what has happened to the music and easily go back to the default state. Still, the fact that one has to recompile the score every now and then to check the results is especially problematic in case of bigger projects.

Of course, there are other factors that make LilyPond attractive: you can modify it according to your wishes and you don’t have to pay lots of money to use it (since it’s Free Software); it supports more platforms than major commercial packages, text input allows effective use of version control, etc. This makes me sure that despite the problems mentioned above LilyPond is a great piece of software definitely worth using, and in many cases it probably is the best choice available.

But there is a lot to be improved. And we shouldn’t forget that.

14 thoughts on “Honestly: Is LilyPond Good Enough?

  1. Carl

    Regarding automatic part combining…I use a modified version of automatic part combining scheme file in my efforts. I adjusted the parameters to fit what I’ve seen in most hymnals that use combined stems. I’m only seeing a couple of issues, mostly in the area of ties and slurs. If I have “c2~c8” in the bass and “c’2(b8)” in the tenor, I would want the notes combined with a slur above the notes for the tenor. Right now, I think it separates them.

    To achieve the lyrics association, I include the individual lines in the score block as hidden voices. The thing I had to do beyond what I’ve seen most of the places that talk about hiding notes say is remove the dot engraver.

    Reply
  2. David

    As software improves, and computers improve, and consumer-grade printers improve, and knowledge of the possibilities becomes more widespread, standards for “good enough” also continue to go up. The first generation of computer music typesetting was clumsy, ugly, and wrong. As a proof of concept, it inspired others to do better in various ways. (The same general pattern can be seen in the history of just about every music-printing technology, even the hand-written variety; clumsy, rudimentary, low-quality first attempts are followed by various improvements.)

    Be that as it may, I would argue that there is another clear answer to the question, beyond “It heavily depends on the kind of music” – and that answer is “Exactly because it heavily depends on the kind of music, the answer is a definite ‘No, Lilypond isn’t good enough’.” (…remembering that none of the examples are musically obscure or unexpected; “anything with lyrics” and “piano music with a dense texture” certainly don’t reasonably qualify as “corner cases”.) Obviously, if unknown or little-used notation is difficult to typeset, there’s a good excuse for that. “Songs in general” and “difficult piano music in traditional notation” are not good areas to be stuck with making excuses. New users of any music-printing software, on hearing “Exotic types of musical notation may require special skill and a lot of work”, will not be surprised. But the same people, on hearing “Lyric spacing doesn’t come out very well yet” or “You may run into trouble with hard piano music” will seriously consider switching to something that works.

    And yet, as you already pointed out, Lilypond IS a great piece of software and worth using, despite the fact that its weaknesses are in some very inconvenient and prominent places.

    Reply
    1. Urs Liska

      I think Janek has been just too honest with his judgements.
      What you say, David, is perfectly right. But I’m not sure that other software is actually better suitable for the tasks Janek describes as problematic. He is obviously concerned about perfection. What he doesn’t consider is the performance LilyPond shows for the average use case. The examples he displays are not corner cases, but (at least in my experience) wysiwyg tools do everything except “just work” for such scores. It definitely is more convenient to tweak the kind of issues you have with e.g. the shown Oskar Fried score graphically with a mouse. But you usually have many more items to tweak just to make such a score readable in the first place.
      Anybody who wants to engrave scores like these in publication quality will have to invest significant amounts of time and skill with any software tools.

      Reply
  3. Urs Liska

    It is very honorable to point out some of the sore points with LilyPond. But I think I’ll add some more aspects which seem important to me. In the post Janek asks whether LilyPond is “good enough for everyday and production use“, but in his explanations and examples he very much concentrates on the “production use” part of the question. I think one could sum up his results with “LilyPond is good enough in so far it is able to produce production quality output, but it should be improved with regard to the amount of work that has to be spend to achieve that goal.” Well, I think this conclusion could be made for about any notation software on the market, and I don’t want to do any comparison this time.
    But I would like to add a few more questions one should raise:

    • What is LilyPond’s performance with regard to “everyday use”, that is (as I think of it) the preparation of scores for anything except publication, when the level of perfection that Janek aspires isn’t a real requirement?
    • How much work do you have to spend if you are preparing scores for use and not for publication?
    • What will be the quality of scores that aren’t tweaked at all?

    While these may seem strange questions I think they are very real ones. In fact I have to do with both regularly:

    1. I’m preparing scores as performance material for myself or ensembles I’m playing in
      It is an important factor how much work I have to put into the scores to make them usable for that puropose
    2. I’m playing in ensembles or orchestras and have to play from parts that someone (e.g. the composer) has prepared without bringing them to publication quality through a major publishing house. How do LilyPond scores compare to such scores prepared with other programs?

    I think both cases would somewhat modify the judgments significantly.
    When I prepare scores for performance I have very little to be tweaked manually. While the result isn’t publication quality at all it is mostly usable out-of-the-box. If you consider this example (please excuse the broken links to the full-size images) you’ll see that the LilyPond rendering is practically usable to play from while the “commercial” try has to be considerably tweaked to be usable at all. If you consider the result of our (very unrepresentative) poll you see that this seems to be valid for the majority of LilyPond use cases.

    On the other hand, as a performer I feel that the average “default LilyPond” score is really better to play from than the “default Finale/Sibelius” score one often sees.

    Of course I don’t want to trivialize LilyPond’s deficiencies when it comes to producing perfect output without significant user intervention. But one also shouldn’t forget is that the competitors don’t achieve this automatically either. And that this perfect output isn’t what most users actually need.

    Reply
    1. Janek Warchoł Post author

      I agree, we shouldn’t miss that aspect of LilyPond use, and indeed, producing usable scores is usually between “very simple” and “not particularly difficult”.

      I think that when talking about quality of LilyPond output and it’s productivity as a tool, we should clearly state these differencies, so that new users will know what to expect – some things are easy, other things not. That’s why I wrote this post (and the next one, in which i’ll present my opinions graphically).

      Reply
  4. David

    Urs Liska and U L: those are very good points.

    I do think comparing against other software, saying “I admit LilyPond has its problems, but the other guys are no better”, misses the point. It’s also an easy position to refute, by pointing out that after the tweaking is finished “the other guys are really no worse, either”.

    The examples in Janek’s article are situations in which Lilypond clearly doesn’t live up to its own (high and increasing) standards. Finding ways to make Lilypond just as good in these areas as it is in others might turn out to be a better goal than “keeping up with the Joneses”.

    In terms of a text-entry-only piece of software, a situation where simple & expected results require complex & unexpected input is a usability defect, not part of a learning curve.

    … adding a reminder that my perhaps-negative-sounding comments are coming from a happy user of Lilypond – but a happy Lilypond user who hesitates to introduce Lilypond to others, for the kinds of reasons that are the topic of this discussion.

    Reply
    1. Michael Rivers

      I believe that no amount of tweaking fixes Finale’s output. It’s so lacking in weight that it’s always difficult to read, and its slurs are comically thin at the ends and thick in the middle. When I take piano scores “professionally” typeset in Finale and typeset them in Lilypond, even before any tweaks, the readability is greatly improved. The original post is definitely correct that things like cross-voice ties and voices popping in and out of existence are more difficult to deal with in Lilypond than they should be, but I never found those things to be easy in Finale or Sibelius either.

      Reply
      1. Janek Warchoł Post author

        I also don’t like Finale output in general – in my opinion, the default font is ugly, and things like pointy slur ends also look bad. However, this is mostly a matter of style – what’s important are the differencies in “hard” features, like collisions between objects.

        Reply
      2. Corey

        I’m admittedly a very recent Lilypond user and far more experienced with Finale. While I completely agree that the default style of Finale is way less than pleasing, adjusting things like the weight of the output is incredibly easy. And once you make those adjustments and save it as a template, you never again have to even think about or look at another light-weight score.

        For me, this is the rub with Lilypond. I think it’s an extraordinarily powerful program, but finding that power and applying it is its biggest uphill challenge. While I’m certain that more experience will make this hill less steep, as I currently see it, yes the default output is light years better than either Fin or Sib, I even think the text based note entry is far more efficient, but even though there are more tweaks to make in the other programs, those tweaks are (for me) easier to both find and execute than with Lilypond.

        Reply
        1. David

          In a way, this particular “rub” means the people who do the hard work on Lilypond have become victims of their own success – because they continue to do such a great job at making Lilypond “do the right thing” automatically in so many situations, the places where they haven’t yet been able to do so are highlighted more and more.

          Reply
  5. David Looney

    Open source software in general lacks documentation of the basics. I fall in the category of “power user” or “light programmer” (DOS days to Windows 7) and have moved on from a lot of cool stuff for lack of documentation. In some cases you’re not even instructed on how to install the software. With LilyPond that wasn’t the case and I had no installation problems. It got up and running no problem.

    This program has very extensive documentation, a surprise frankly, but it’s an absolute MUST for a program with a lot of strength like LilyPond, otherwise one must move on; not because you don’t want to use the program but because without detailed doc’s you’ll never figure out how to use it (assuming you’re not a heavy programmer.)

    That said, the documentation lacks cohesion and is over-ready for a major overhaul. The content is there, the organizational structure is there and A+++ for those who have maintained and updated it. As much as we hate to “waste time and resources” on things like performing a “documentation overhaul” I feel strongly that this program would reach more “new users” with a retool of the doc’s more so than by adding cool new features. Simply stated, the documentation does not communicate, it’s more like a dictionary; an invaluable resource/reference for the long-term but not a good way to learn the English language.

    The biggest astonishment for me researching notation software is the complete and total lack of an offering for a gigantic market segment: musicians that don’t read; they use tabs, lead sheets, chord sheets and the Nashville system. Huh? That’s like saying the brand new sports car you just bought reaches 100mph in five seconds but you can’t drive it to the corner store because it doesn’t drive well at 35mph. Huh? My research suggest that LilyPond has reached its high stature based upon impeccable output and sophistication and its thrust should continue to reach for the top. However, unlike my example, if you can reach 100mph then 35mph should be a no-brainer.

    In closing, my suggestion is twofold.
    Take the great documentation you have and retool it. Consolidate, re-organize and then flesh it out. It will provide value to LilyPond year-after-year (at little cost after the initial overhaul.) I think you’ll be surprised at the increase of users. Sounds like a good project for a music college or two.

    Second, a concentrated effort to facilitate the newbies effort. Split the documentation up between traditional formats and contemporary formats (for lack of a better term.) Good tutorials with step-by-step instructions (and links to other reference material) along with well commented, COMPLETED code is best. Completed code is critical for the newbie and provides the dual purpose of functioning as function as a template.

    Even without the tutorial, well commented completed code job is fantastic for newbie even if it doesn’t contain a tutorial. Once the newbie gets his feet wet, the well documented ready to go code is easy to customization and use, gets the job done while teaching the user how to do it.

    Just my 2 cents.

    Reply
    1. Janek Warchoł Post author

      Very interesting insights, thanks! I have similar feelings with regard to our documentation: it’s very detailed and comprehensive, but for a newcomer it can easily be overwhelming.

      I was thinking about ways to improve this situation, but such a project (documentation overhaul) would require a huge discussion, which is dificult to organize and can easily become a time-sink.

      Reply
    2. David

      David Looney: I understand and agree with the bulk of what you’ve said. However, I don’t think there’s any point in a piece of music-printing software trying to support musicians who don’t read music. Plus there are already enough music-related things to work on (e.g. get irregular polyphony [which means an extremely large chunk of existing piano, harp, and guitar music that Lilypond is currently not very good at] and lyrics working properly before even thinking of tackling non-notation projects).

      Reply
  6. ptoye

    I agree on the documentation issue. The Learning document is fine, but has too much unexplained detail in some of the examples. But the Notation document isn’t complete – some things are only in Learning, so you have to have three windows open at all times – 2 documents and one Lilypond editor. Takes up a lot of real state on a small monitor.

    I can’t compare with Sibelius or Finale as I don’t run them, but I’ve just set a song which the singer said would have taken half the time with Sibelius. Certainly getting piano music right is very hard work (the vocal line was trivial), as the concept of the “voice” doesn’t really work for anything post-Bach. Which is all of the piano repertoire.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *