Since I’ve started writing about plain text and version control and the like I’ve occasionally been asked about my opinion. Usually these questions circle around collaborative workflows, but recently I was asked by a composer whom I tried to “convert”:
Since I work totally alone, I honestly don’t yet see the benefit —
please tell me what I’m missing, as a solo engraver.
My immediate reaction to such a question has something quasi-religious about it, along the lines of “Believe me, when you’re there you’ll see the light.” But as I wouldn’t consider this a satisfactory answer, the question turned out to be quite intriguing. So I made up my mind and did find a number of reasons why any LilyPond user should at least seriously consider using version control, even when she’ll never work with someone else. And since the composer who had asked considered them “compelling” I thought it would be a good idea to pour it into a blog post for a broader audience.
Actually I consider it a deficiency that the use of version control isn’t endorsed more prominently in the LilyPond community or its website. I’m sure if I had heard about the concept right from my first contact with LilyPond I would have started to actually use it several years earlier. From my impression there are users who don’t know about it and users who already knew about it from other contexts (e.g. programming work) – but those who know it take it for granted and don’t consider it necessary to talk about it…
In a second post I’ll then take my personal experience – my doubts before and my enthusiasm after the “conversion” – as an opportunity for some thoughts on possible ways into that new world, addressing the composer’s follow-up question about an estimate on the learning curve.
It’s right there, so why not simply use it?
LilyPond works on plain text input files; that’s not exactly a secret. And therefore it’s readily accessible to workflows with version control – but this is much less known.
Not using version control as a tool would seem comparable to deliberately setting aside the localization of your operating system. You can do without, but if the tools are available it’s usually a good idea to use them.
Of course this isn’t a strong and logical argument that would convince someone in doubt, but just like the tools it’s simply there 😉 .
Restoring (and surpassing) the “unsaved file” concept
One thing that had annoyed me since I started working with compiled systems (be it LilyPond scores or programming languages) is that you can’t work in the context of an “unsaved file”. With graphical programs you can simply start hacking away and at the end of your working session decide to either save your document, discard it or save your work to another file (keeping the previous state as a copy). With compiled systems you have to save the file as soon as you want to compile it – which you usually do quite often during a working session. If you want to keep track of the state of your document before starting a session you’d either have to rely on your editor’s Undo capabilities or always work on a copy of your project and replace your previous version only at the end of the working session. (However, Frescobaldi does provide an option to preview the output from the current state of a document without having to save it.)
Version control provides a concept that is roughly equivalent to the idea of an “unsaved document” and that lets you graciously overcome this limitation of compiled systems. In a versioned project you’re always working on top of the latest commit, and this is like the last saved state of the document (or rather project). You can save and compile to your heart’s delight without compromising the ability to go back to that last known state.
But this approach is actually even more powerful than the unsaved document:
- You can save your file much more often, so if your computer crashes you’ll only lose a small bit of work.
- When making your next commit you can selectively add your changes. This means after hours of tweaking you can freely decide which portions of your work you want to keep and which not. No need to decide between “keep” or “discard” your whole work. Or you can organize your work in coherent sets of changes by committing them in groups tailored to your needs.
- This isn’t dependent on a phyiscal working session on your running computer. You can decide to make a new commit after a few minutes or after a week of intermittent work. To give an extreme example – just to blow your mind – you could remove the hard drive from your computer, send it on a trip around the world, and resume and commit your work on another computer.
Having access to the full project history
When you work with version control you can always inspect any previous state your project had been in. For example if you managed to break something and noticed it only somewhat later you can go back in history to determine the exact spot where it happened – there you can selectively revert or modify the offending addition.
Or you may wonder why you did something the way you did – you can go back to that point in history and examine the context the last change was made in. (Actually a version control system can tell you when a given line of your input has been changed.) This may help you recalling what you may have thought then.
It may not be easy to put your fingers on, but from my experience this is an option you don’t want to miss any more once you’ve gotten used to it. Nowadays if I encounter an old and complex project from the time before I used version control I feel uneasy and somewhat helpless. Even if I won’t need it at all in any given case it’s very good to know that I might access, inspect and modify the full history of a project if necessary.
Keeping the project consistent with branches
The progress of a project under version control is by convention reflected by the progress of its
master branch. There are many more or less formalized workflows around, but it’s always a good idea to keep that
master branch in a consistent state at any time. For example, when editing a score it should – at any given point in the history of the
master branch – be compilable without errors.
To achieve this you can do all work of transitional or experimental character on dedicated branches which are somewhat like isolated sessions. Once you’re finished with the given task and decide it’s to be kept you merge it into the
master branch, and this will proceed from one consistent state to another, short-circuiting all the intermediate states in between. This way you can always keep your project in a consistent state – even when you’re working at several issues at a time, in independent branches.
Some use cases for this are:
- Adding a logical layer, e.g. articulations and dynamics.
- Entering a complete part
- Reviewing or overworking a finished score
- Incorporating the scholarly review
Being able to work in such isolated sessions – without compromising the previous state – can significantly make you more comfortable with retouching “finished” work.
Working on multiple computers
If you – like me – have to work on different computers you actually are collaborating – with yourself. In that case you can benefit from using version control with a remote repository hosted at a service provider’s server.
If you are synchronizing your work using USB devices and forget to synchronize your working directory before switching computers you’re heading for trouble: You have to scrupulously avoid editing files that also have been modified on the other computer.
But if you’re having the same situation with a project under version control (i.e. you forgot to commit and/or push your changes) you can simply continue to work on the other computer (of course without being able to access the latest changes) and simply merge your work later. If you should happen to create conflicting modifications version control provides concise tools to resolve them.
Use web interfaces to edit project files
If you are hosting your repository on a server (which may make sense even when you only use one computer at home) you have the possibility to edit your files directly there through the web interface of the hosting provider. This is a very nice option for quick ideas you may come across when you’re somewhere else. Simply log in through a web browser, edit your file and apply the quick fixes.
(And although we started with the assumption of someone working on his own this is a great way to let others participate and review your work. I experienced several occasions when people who didn’t know about the technical details but were somehow involved in the project added their remarks or additions directly that way.)
Indirect impact of using version control
Apart from these concrete aspects using version control may have an indirect impact on your way of working, in fact it can change your whole thinking about tools and workflows. What I have written several times isn’t a buzz phrase but a deeply felt reality: I can’t imagine anymore how working was before I started using version control.
Version control acts like a safety net and greatly reduces the risk of losing work once it has been committed to the repository. This makes you feel much more comfortable working at different edges at the same time or trying out things that may or may not be persistent or that risk breaking something.
Version control also makes you more aware of structuring your work in coherent units. Once you start working with repositories you will be much more careful to organize your work in sensible units that will form the chain of commits in your project history.
That way version control makes you more free and more structured at the same time. And all this applies equally whether you’re working on your own or collaboratively with tens or hundreds of collaborators.
So, if it’s all that terrific why isn’t everybody using version control?
Well, as I’ve already said there are too many people who simply don’t know about version control. Or they have heard about it but would only think of it as a programmer’s tool. Throughout this blog and also on the LilyPond mailing list I try to do something about that 😉 and it’s also part of my first visible contribution to LilyPond – the “Features” page on lilypond.org (English only so far) now mentions version control.
But more importantly, version control actually comes at a cost: you have to learn a completely new concept and toolkit.
I think it’s a more than worthwhile investment, but I also can imagine why it looks daunting for people who haven’t used it at all. I recall that just before I actually started with it I was very unsure if it would work out and if it’d be worth the trouble.
In an upcoming post I’ll collect some old emails from that time, try to describe how I got talked into trying it and how I experienced the learning curve.