Towards templating with la-lily-pond

In my previous post “Trees, Music and Lilypond”, I introduced a hierarchical naming scheme to store music in a structured manner and then to use template functions on it. Now I will explain how template functions can also be organized hierarchically (in a tree) and “normalized,” and what benefit we gain from this. In short, this is an effective way to separate different tasks — storing the music (content) is one task and laying it out in a score (presentation) is another. While I was previously writing about the theory grounding this technique, I will now shift these ideas towards the actual implementation and the main goals of this approach. This post will have three parts:

  • First I’ll make some adjustments to the code I shared in my previous post.
  • Second I’ll write about the hierarchical organization of templates.
  • Third I’ll show a “normalized” signature for template functions.

Coding adjustments

The following adjustments will improve our coding and avoid a pitfall when working with trees.

1. dot-notation

When addressing nodes in a tree, we use paths. In my previous post I entered these paths as strings, which were then split into lists. Now I want to show the benefits of a relatively recent LilyPond feature, namely the dot-notation, which provides a better way of working with paths. An argument to a music function with a predicate list? can be entered in dot-notation so that my.path.to.music will yield a list #'(my path to music). So instead of using strings that are split into lists to create such path lists (as in my previous post), from now on I will use the list? predicate and dot-notation for any path input, which avoids the quotes and is (IMO) easier to type.

2. relative paths

As we work with trees, we want to be able to use relative path lists. So say we are at path my.path.to.melody, and we want to access my.path.to.lyrics. We may want to use a relative path to get there, which would look like this list: #'(.. lyrics). The .. part of this relative path moves us one hierarchical step up in the path, and from there we can access the “lyrics” branch instead of “melody.” For this to work, we need a little function to normalize path lists:

#(define-public (normalize-path path)
   (let ((ret '()))
     (for-each
      (lambda (e)
        (set! ret
              (cond ((eq? e '..)(if (> (length ret) 1) (cdr ret) '()))
                (else `(,e ,@ret))))) path)
     (reverse ret)))

Next let’s see how we would use this function to “normalize” a relative path. (The following code also shows the function called “Path” that allows us to use the dot-notation discussed above.)

% get a list from dot-notation
Path = #(define-scheme-function (parser location p)(list?) p)
% a path to a melody
pathToMelody = \Path my.path.to.melody
% create path to lyrics with relative path
pathToLyrics = #(normalize-path (append pathToMelody '(.. lyrics)))

pathToLyrics is “normalized” from my.path.to.melody.'..'.lyrics to my.path.to.lyrics.

3. the tree-object

There is one issue with the association-list based tree that I discussed in my previous post, and this problem might occur while storing either music or templates using paths. If you have stored a template with a path vocal it would be overridden by a second template that was stored with the path vocal.group, which is not what we want. So instead of the association-list approach, I decided to use an object-oriented approach, which is also possible with Scheme. I won’t explain all the details of the Scheme code used to create the trees. Again, if you are not familiar with Scheme, just try to understand the basic idea of a class tree, which has methods get and put.

% OOP module in scheme
#(use-modules (oop goops))

% a class for one tree-node
#(define-class <tree> ()
   ; the children of this node
   (children #:accessor children #:setter set-children! #:init-value '())
   ; the node-name
   (key #:accessor key #:init-keyword #:key #:init-value 'node)
   ; the node-value
   (value #:accessor value #:setter set-value! #:init-value #f)
   )

% set value "val" at "path" in "tree"
#(define-method (tree-set! (tree <tree>) (path <list>) val)
   (if (= (length path) 0)
       ; if we reached the addressed node, set value
       (set! (value tree) val)
       ; else look for child
       (let ((ckey (car path))
             (cpath (cdr path)))
         ; get child tree
         (let ((child (assoc-get ckey (children tree)) ))
           ; if it is not a tree, create one and store it
           (if (not (is-a? child <tree>))
               (begin (set! child (make <tree> #:key ckey))
                 (set! (children tree) (assoc-set! (children tree) ckey child ) )))
           ; ascending the tree recursively
           (tree-set! child cpath val)
           ))
       ))
% get value at "path" in "tree"
#(define-method (tree-get (tree <tree>) (path <list>))
   (if (= (length path) 0)
       ; if we reached the addressed node, return value
       (value tree)
       ; else get next child
       (let* ((ckey (car path))
              (cpath (cdr path))
              (child (assoc-get ckey (children tree)) ))
         (if (is-a? child <tree>)
             ; ascending the tree recursively
             (tree-get child cpath)
             #f)
         )))

With this Scheme code we can create objects of class tree, add objects to it with a method tree-set! and retrieve them with a method tree-get. We can do this at any point in our path without overriding any child values. For example, our tree can now store separate objects under vocal and vocal.staff at the same time.

A template tree

Now that we have these code adjustments in place, we can move on to our next topic which is the hierarchical organization of templates. So far I have mainly written about the tree representing the musical content of a score, but now I’d like to shed some light on another tree: the template tree.

First a brief word about the inherent design pattern that I am using here, from a software development perspective. (This is a sidenote, if you are not familiar with these software development terms, just take note that there are established design patterns that describe what we are doing here.) In my previous post I outlined template functions. Those functions are also hierarchically related: vocalStaff is called by SATBGroup, which might be called in a file combining the five movements of a composition. The functions call each other (roughly) following the composite design pattern or may be seen as the “view” part in the model-view-controller design pattern.

So now we will create a tree to hold our templates and define the functions \registerTemplate and \callTemplate to work with this template tree.

templateTree = #(make <tree>)

registerTemplate =
#(define-void-function (parser location path templ)(list? ly:music-function?)
   (tree-set! templateTree path templ))

A template in this tree can call its children, siblings, or parents, by using a relative path that starts from wherever it is located in the path. So a template with a path vocal can call its child vocal.staff with a relative path staff. A template vocal.group.SATB might call its parent vocal.group with a path ‘..‘. In this way multiple templates can work together and delegate work to other related templates in order to assemble a composite piece of engraved music. This lets us engrave a piece by calling a base template, which then delegates different parts of the work to its child templates. The “leaf” templates may only need to engrave one single context — a Lyrics context, a Staff, or even just a Voice. Each template will only get the musical content that it is to handle or display.

Now that we have a tree for our templates, we store the templates, which are just music functions, in the tree with a function \registerTemplatepath.to.template #(define-music-function ...). The music function we are defining here is our template, and the path.to.template is where it is being stored in the tree. We can then call our templates with a path to the template and a path to the musical content that it will take: \callTemplatepath.to.template path.to.music.

What shall happen during this call?

  1. call the function stored at path path.to.template with the music stored at path "path.to.music".

This looks reasonable. But do we want to do the path-appending (path-unfolding) in a template? No, it would result in a lot of recurring code. But we can avoid this with a global current template path and a global current music path. The functions \getMusic, \putMusic and \callTemplate use those global paths like the current working directory in a file system. Any given template path or music path is taken to be a relative path and is unfolded against the globally stored current base path.

Let’s update our list of what happens during a template call:

  1. set the current template path and the current music path
  2. call the function stored in the current template path
  3. reset the current template path and the current music path to their previous values
callTemplate =
#(define-music-function (parser location template-path music-path)(list? list?)
   ; get template from path
   ; set current template-path and current music-path
   (set! return-value ((ly:music-function-extract templ) parser location music-path)')
   ; reset current template-path and current music-path to previous value
   return-value
   )

This snippet is shortened for readability. I prepared an example file based on the one from my previous post, where the code is complete.

Let us line up, what is happening now:

  1. we call \callTemplate vocal.group.SATB massInC.kyrie to create the music for our score — like \SATBGroup in the previous post
  2. this call sets the current music path to massInC.kyrie and the current template-path to vocal.group.SATB
  3. the template vocal.group.SATB calls the template vocal (its grandparent) with the relative path #'(.. ..), for the four voices with the relative music paths soprano, alto, tenore, basso
  4. the current template path is set to vocal for all four calls, while the current music path is set to massInC.kyrie.soprano, massInC.kyrie.alto, massInC.kyrie.tenore and massInC.kyrie.basso, respectively for each call
  5. the template vocal creates a Staff, a Voice and a Lyrics context using the music from the given path appended with melody and lyrics respectively (i.e. massInC.kyrie.soprano.melody, massInC.kyrie.soprano.lyrics, etc.) — like \vocalStaff in the previous post

Normalizing the signature

What advantage do we gain with this tree of templates? To answer this, we will first add an argument to the signature of all the template functions. Any template must define two arguments, music-path and options, both with a predicate list?:

#(define-music-function (parser location music-path options)(list? list?) ...)

The music-path is not essentially needed as it is available as #(current-music-path), but it is still given for convenience (and backward-compatibility).

Now we can give an association list with an arbitrary set of options to the templates. So vocal.group.SATB can call vocal with additional arguments (options) to set the clef, the instrumentName or other needed information. And even better, we can register a template vocal.group wich receives a list of voices as one of its options. That way, we can easily define other choral group setups like SAATBB, for example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
\registerTemplate vocal.group
#(define-music-function (parser location my-path-to-choir options)(list? list?)
   (let ((staff-list (assoc-get 'staff-list options '())))
     #{
       \new ChoirStaff $(make-music 'SimultaneousMusic 'elements
        (map (lambda (staff) #{ \callTemplate #'(..) #(list (car staff)) #(cdr staff) #}) staff-list))
     #}))

% create vocal staffs for Soprano, Alto, Tenore and Basso
\registerTemplate vocal.group.SATB
#(define-music-function (parser location my-path-to-choir options)(list? list?)
   (let ((opts (assoc-set! options 'staff-list '(
                                                  (soprano . ((clef . "G")))
                                                  (alto . ((clef . "G")))
                                                  (tenore . ((clef . "G_8")))
                                                  (basso . ((clef . "bass")))
                                                  ))))
     #{ \callTemplate #'(..) #'() #opts #}))

The vocal.group.SATB template calls its parent (vocal.group) with predefined options. As you can see in lines 13–16, the voices are defined as pairs of a voice name and an association list that contains all options given to the vocal template. But you can also call vocal.group directly with whatever options are needed.

If you are thinking that setting up options as nested association lists manually is quite tedious, then you are absolutely right! But you can use helper functions that are included in openLilyLib and lalily. For example \optionsAdd myOptions staff-list.tenore.clef "G_8" will set the tenor-clef option inside the association list myOptions. I will explain this in more detail when I actually write about lalily.

Consolidation

In the implementations discussed here, the trees and the current paths are hidden and secured inside a closure which is part of a scheme module. If you are not familiar with scheme, modules, or closures, the important part is the attribute “hidden and secured.” The trees are globally available and won’t be overridden, for example, if you include files twice. The trees and their fruits, namely the music and the templates, are globally available, so you can separate the tasks of music input (content) and engraving (presentation). You can first include multiple files containing \putMusic statements and then build a book with several scores in it.

What’s next?

Now we have defined the core functions which are used in openLilyLib and lalily. In a future post I will discuss how to actually use the lalily-templates and what templates are already available.

The lalily-templates come with quite a few wrapper functions that automate the score creation — in some cases conditionally. This lets you have a score for proof-reading that is only created when the file containing a score-creating command is not included. And the predefined templates already contain the edition-engraver, so you can include the music definition (\putMusic) and add tweaks to an edition (for example, to customize it for a special paper-format) without ever touching the music content.

One thought on “Towards templating with la-lily-pond

  1. Pierre-Luc Gauthier

    Thanks, Jan-Peter Voigt.

    Great series on a seemingly simple subject: storing and retrieving music expressions.

    I have yet to master or even understand (fully) the scheme code but I feel this series could be a great reference on how to practically achieve efficiency.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *