Restoring Deleted LilyPond Files

There are plenty of opportunities to delete important files, and even cases where you deliberately do it and only some time later notice that it was a stupid mistake. However, if you are using version control you’re lucky because that doesn’t make you worry at all. No need to hope that some undelete tool can be applied in time before something else irreversibly overwrites the bytes on disk etc.

Well, this is neither new nor specific to music editing, but I thought it a good idea to write a short post about it anyway. It will increase the chance that someone involved in music stumbles over the information, and it is yet another spotlight on the potential of versioned workflows for music editing.

Oops

In a previous post I explained that in our crowd engraving challenge to collaboratively engrave a huge orchestral score we delete files for segments of the music where instruments actually play nothing. Going through a part deleting such files makes you feel good because it’s the fastest way to proceed the “completion status” of our status reports. But I have to admit that it’s also a quite error-prone process where you can easily do too much and delete files which you shouldn’t. For example there are numerous segments that only contain a finishing note of a phrase, after which the instrument disappears in frenched (suppressed) staves. It’s easy to overlook that fragment … The next person who is then working on that part sill stumble over it and wonder why there is no empty segment template in which to enter the music.

Fortunately it is dead easy to restore that file, no matter how much work has been done in the meantime. Actually this involves only two steps.

Identifying Where the File Has Been Deleted

git log has plenty of options that you can use to start trying to pin down the problem. However, the seemingly natural approach won’t work:

.../das-trunkne-lied$ git log parts/clarinetI/01.ily
fatal: ambiguous argument 'parts/clarinetI/01.ily': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions

Well, it’s somewhat logical that you cannot retrieve the history of something that isn’t there anymore, isn’t it? But Git wouldn’t be Git if it wouldn’t assist you gracefully with such trivial tasks. And actually it tells you immediately what to do: use --. This will tell LilyPond that yes, it is a file you are looking for and asks it to look in its history if there isn’t such a file in the current working tree. You can even tell Git to return only the commit that deleted the file:

.../das-trunkne-lied$ git log --diff-filter=D -- parts/clarinetI/01.ily
commit 50e62d87cf2dc3e1b0c39812f0b001d85e388712
Author: Urs Liska <git@ursliska.de>
Date:   Tue Sep 30 14:19:15 2014 +0200
 
    clarinetI: Remove empty segments

Well, oops, it was myself. Good that it’s me who noticed 😉

OK, now we have identified the commit where the file was deleted, but what next? Basically we need to find the commit that immediately precedes that one because that is the last commit where the interesting file had been present, so it also represents the file’s last state before deletion. To do this we first modify the previous request to only show the commit hash:

.../das-trunkne-lied$ git log --pretty=format:%H --diff-filter=D -- parts/clarinetI/01.ily
50e62d87cf2dc3e1b0c39812f0b001d85e388712

Using the $(...) notation we will now retrieve the Git log from that commit backwards, restricting it to the previous commit:

/das-trunkne-lied$ git log -1 --pretty=format:%H $(git log --pretty=format:%H --diff-filter=D -- parts/clarinetI/01.ily)~1
8da631a9d92129479867a2c876bbdd9fe1ccfbe0

What does this do?
Well, first we retrieve (as above) the commit hash deleting the file. This is enclosed in $(...) and used as input to yet another git command. Actually it’s the same git log command with two modifications: The first option is -1, restricting the output to just one commit while the ~1 at the end of the command tells Git to use “one commit before” the one we retrieved from inside the $(). So eventually our call returns the commit ID of the last commit where our file was present.

Restoring the File

Restoring the file is surprisingly simple:

.../das-trunkne-lied$ git checkout 8da631a9d92129479867a2c876bbdd9fe1ccfbe0 parts/clarinetI/01.ily
.../das-trunkne-lied$ git status
# On branch staging
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#   new file:   parts/clarinetI/01.ily

Why on earth does this work, isn’t git checkout used for switching between branches? Well, yes, but not exclusively. According the man page git checkout “Updates files in the working tree to match the version in the index or the specified tree”. When you use it for switching branches this is actually what it does. But you can also use it to “update” any file in the working tree to any state that can be referenced by a commit, branch or tag. So what we did was update parts/clarinetI/01.ily to the last state of the repository that contained that file, effectively restoring it from the Git history.

Now the file is present in the working tree and staged, so we can simply commit it and have it ready for music entry. I won’t do this now because the file I used for this example has actually been deleted correctly and I don’t want it to be restored. So I simply do what Git suggest: git reset HEAD parts/clarinetI/01.ily and then delete it from disk. This last exercise can be taken as yet another example for how easy your life can be with version control when you don’t have to worry about messing up your working directory through processes such as deleting or restoring arbitrary files.

Summary

Maybe this looks overly complicated and even daunting to someone not very familiar with Git. But basically you can work your way towards such a solution by building it step by step. There are many solutions out there that give you parts of the solution from which you can work your way onwards.

If we’d expect to have this problem regularly we could even go one step further and don’t call that last git checkout command manually but feed it with the output of the previously developed commands. But I think that would be overkill – because then we’d run into the 80/20 rule saying that the last 20% of a programming task require 80% of the time. I wouldn’t feel pretty safe having such a nested command that would try to restore a file with one call of, say, git-restore parts/clarinetI/01.ily without implementing quite some safety infrastructure.

2 thoughts on “Restoring Deleted LilyPond Files

  1. Garrett Fitzgerald

    Interesting post, but I’m still partial to Subversion and TortoiseSVN. I like being able to do everything with the right-click menu. 🙂

    Reply
    1. Urs Liska

      You’re welcome. My posts are actually in favour of version control itself, not specific tools. But I have to admit Git is the only one I actually used so I always write about that.

      But as I don’t have a history with Subversion or even CVS I wouldn’t ever consider using a centralised VCS because I think the advantages of decentralized systems are so significant. The user interface is a different thing, and you’ll have pretty GUIs for Git too. OTOH I usually recommend learning the concepts of version control through the command line. GUIs hide away too much of the complexity IMO. Even now that I’m quite comfortable with everything I regularly check back on the command line if I’m not completely sure about what my GUI is going to do.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *