There are plenty of opportunities to delete important files, and even cases where you deliberately do it and only some time later notice that it was a stupid mistake. However, if you are using version control you’re lucky because that doesn’t make you worry at all. No need to hope that some
undelete tool can be applied in time before something else irreversibly overwrites the bytes on disk etc.
Well, this is neither new nor specific to music editing, but I thought it a good idea to write a short post about it anyway. It will increase the chance that someone involved in music stumbles over the information, and it is yet another spotlight on the potential of versioned workflows for music editing.
In a previous post I explained that in our crowd engraving challenge to collaboratively engrave a huge orchestral score we delete files for segments of the music where instruments actually play nothing. Going through a part deleting such files makes you feel good because it’s the fastest way to proceed the “completion status” of our status reports. But I have to admit that it’s also a quite error-prone process where you can easily do too much and delete files which you shouldn’t. For example there are numerous segments that only contain a finishing note of a phrase, after which the instrument disappears in frenched (suppressed) staves. It’s easy to overlook that fragment … The next person who is then working on that part sill stumble over it and wonder why there is no empty segment template in which to enter the music.
Fortunately it is dead easy to restore that file, no matter how much work has been done in the meantime. Actually this involves only two steps.
Identifying Where the File Has Been Deleted
git log has plenty of options that you can use to start trying to pin down the problem. However, the seemingly natural approach won’t work:
.../das-trunkne-lied$ git log parts/clarinetI/01.ily fatal: ambiguous argument 'parts/clarinetI/01.ily': unknown revision or path not in the working tree. Use '--' to separate paths from revisions
Well, it’s somewhat logical that you cannot retrieve the history of something that isn’t there anymore, isn’t it? But Git wouldn’t be Git if it wouldn’t assist you gracefully with such trivial tasks. And actually it tells you immediately what to do:
use --. This will tell LilyPond that yes, it is a file you are looking for and asks it to look in its history if there isn’t such a file in the current working tree. You can even tell Git to return only the commit that deleted the file:
.../das-trunkne-lied$ git log --diff-filter=D -- parts/clarinetI/01.ily commit 50e62d87cf2dc3e1b0c39812f0b001d85e388712 Author: Urs Liska <firstname.lastname@example.org> Date: Tue Sep 30 14:19:15 2014 +0200 clarinetI: Remove empty segments
Well, oops, it was myself. Good that it’s me who noticed 😉
OK, now we have identified the commit where the file was deleted, but what next? Basically we need to find the commit that immediately precedes that one because that is the last commit where the interesting file had been present, so it also represents the file’s last state before deletion. To do this we first modify the previous request to only show the commit hash:
.../das-trunkne-lied$ git log --pretty=format:%H --diff-filter=D -- parts/clarinetI/01.ily 50e62d87cf2dc3e1b0c39812f0b001d85e388712
$(...) notation we will now retrieve the Git log from that commit backwards, restricting it to the previous commit:
/das-trunkne-lied$ git log -1 --pretty=format:%H $(git log --pretty=format:%H --diff-filter=D -- parts/clarinetI/01.ily)~1 8da631a9d92129479867a2c876bbdd9fe1ccfbe0
What does this do?
Well, first we retrieve (as above) the commit hash deleting the file. This is enclosed in
$(...) and used as input to yet another git command. Actually it’s the same
git log command with two modifications: The first option is
-1, restricting the output to just one commit while the
~1 at the end of the command tells Git to use “one commit before” the one we retrieved from inside the
$(). So eventually our call returns the commit ID of the last commit where our file was present.
Restoring the File
Restoring the file is surprisingly simple:
.../das-trunkne-lied$ git checkout 8da631a9d92129479867a2c876bbdd9fe1ccfbe0 parts/clarinetI/01.ily .../das-trunkne-lied$ git status # On branch staging # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: parts/clarinetI/01.ily
Why on earth does this work, isn’t
git checkout used for switching between branches? Well, yes, but not exclusively. According the man page
git checkout “Updates files in the working tree to match the version in the index or the specified tree”. When you use it for switching branches this is actually what it does. But you can also use it to “update” any file in the working tree to any state that can be referenced by a commit, branch or tag. So what we did was update
parts/clarinetI/01.ily to the last state of the repository that contained that file, effectively restoring it from the Git history.
Now the file is present in the working tree and staged, so we can simply commit it and have it ready for music entry. I won’t do this now because the file I used for this example has actually been deleted correctly and I don’t want it to be restored. So I simply do what Git suggest:
git reset HEAD parts/clarinetI/01.ily and then delete it from disk. This last exercise can be taken as yet another example for how easy your life can be with version control when you don’t have to worry about messing up your working directory through processes such as deleting or restoring arbitrary files.
Maybe this looks overly complicated and even daunting to someone not very familiar with Git. But basically you can work your way towards such a solution by building it step by step. There are many solutions out there that give you parts of the solution from which you can work your way onwards.
If we’d expect to have this problem regularly we could even go one step further and don’t call that last
git checkout command manually but feed it with the output of the previously developed commands. But I think that would be overkill – because then we’d run into the 80/20 rule saying that the last 20% of a programming task require 80% of the time. I wouldn’t feel pretty safe having such a nested command that would try to restore a file with one call of, say,
git-restore parts/clarinetI/01.ily without implementing quite some safety infrastructure.