The translation process¶
Overview¶
Translation in LilyPond is the flow of making a graphical representation of music. It is the only stage where timing information is available. During translation, graphical objects are created and set up for the backend. Notably, every object has a parent on both axes (X and Y).
Translation happens through several kinds of objects.
Contexts are used to store data. They are typed, and correspond
to a certain portion of the score, vertically. They are created
following a hierarchy. Common context types include Score,
StaffGroup, Staff and Voice. All contexts types are
listed in the Internals Reference under Contexts.
Stream events are derivatives of music events. They have a different
class in C++ and a different Scheme type predicate: ly:stream-event?.
Most music event types are turned into stream events when their time
comes. This does not hold for music types that contain other
music events, such as SequentialMusic. As a rule of thumb,
music types ending in Event are slated to become stream events.
Events are Probs <probs>, hence the functions related to event properties:
ly:event-property and ly:event-set-property!.
Iterators are responsible for the timing. They advance in “container”
music expressions – those preferably ending in Music rather than
Event – and broadcast their contents as stream events or create
child iterators for nested container expressions.
Engravers react to stream events. They can create grobs, set
context properties, and process the grobs created by other engravers.
Overall, they constitute the main component of translation. There
is an engraver for every single bit of notation: Note_heads_engraver,
Stem_engraver, Beam_engraver, Staff_symbol_engraver, and
many others. In contrast to iterators, it is possible to write
engravers in Scheme.
Engravers have no properties, because they are generally not directly used as objects. They communicate through context properties.
By convention, iterator and engraver names are written Capitalized_with_underscores. Stream events do not have any particular type. They are based on music classes (also called event classes) rather than music types.
Grob basics¶
Grobs are covered more in-depth in the Backend programming section. This
is an introduction to what you should know before writing an engraver.
The term grob is short for graphical object. The functions
related to their properties follow the same conventions as
those described in Probs.
- (ly:grob-property grob property [default])
- (ly:grob-set-property! grob property value)
Whenever possible, a grob has an event or another grob as its cause,
available in its cause property. This is used in engravers as
well as the backend. It is also the way point-and-click can trace
back the origin of a grob in the source file. The event-cause
function traces back the cause of a grob in events.
Like music objects, grobs are grouped together in grob interfaces. In the Internals Reference, they are listed under Graphical Object Interfaces.
Like music types and music classes, respectively, grob types are
spelled in CamelCase, and grob interface names in
lowercase-with-dashes.
Contexts¶
Context properties¶
Contexts contain properties. In LilyPond syntax, the standard way to
modify properties of a context is the \set command:
\set [ContextName].propertyName = value
Behind the scenes, this inserts a PropertySet event in the music.
When iterated, this event causes the property to be set (through
the Property_iterator).
Contexts are Probs <probs>, with the associated functions:
- (ly:context-property context property [default])
- (ly:context-set-property! context property value)
Moreover, every context contains grob property settings. The command
to change them in the music is \override:
\override [ContextName].GrobName.property-name = value
In Scheme, the following functions deal with grob property defaults:
- (ly:context-grob-definition context grob)
Return an alist mapping property names (as symbols) to their defaults for the grob (a symbol) in context.
- (ly:context-pushpop-property context grob property [value])
If value is given, push it as a default for grob.property in context. This is the equivalent of a
\temporary \override.Without a value, pop the top of the stack. This is like
\revert.Note that for backwards compatibility reasons,
\overridewithout\temporarypops the stack before pushing. This means that after a sequence of\overridecommands,\revertdoes not reinstate the value before the last\overridebut the primary, global default value for the grob property in the context type.
The context hierarchy¶
Context follow a hierarchical structure. The top-most context is called
Global. Every context but Global is contained in a parent.
- (ly:context-find context name)
Search and return a context with the given name (a symbol) above context: its parent, or the parent of its parent, or …
Return the boolean false if no appropriate parent is found.
Aliases are a way for the more rarely used context types, like
TabStaff, to fit in the hierarchy of the more frequently used,
such as Staff. Contexts having the name as alias are also
considered by ly:context-find. Thus, a TabStaff may be found
as a Staff parent to a TabVoice. This allows engravers to
work in several context types.
Context IDs¶
Every context is known under a unique ID, accessible through the
function ly:context-id. The syntax \new Staff = "up" ...
sets this ID. By default, it is generated automatically in a way
guaranteeing uniqueness.
Defining new context types¶
The official documentation has excellent details on Defining new context types.
Useful translation hooks¶
Accessing a context¶
In the course of a music expression, the \applyContext command
can be used to apply a certain function to a context. This comes
in handy when the settings that are to be applied to the context
depend on its properties. The \applyContext function takes
a single parameter, a function taking a context argument. It applies
in the context the music expression is being iterated in. To specify
a different context, the \context command should be used. It
sends music to a context found above the current one by type.
For example, here is how to increment the bar number by one:
incrementBar =
\context Score
\applyContext
#(lambda (context)
(ly:context-set-property!
context
'currentBarNumber
(1+ (ly:context-property context 'currentBarNumber))))
{
c'1\break
c'1\break
\incrementBar
c'1
}
Tweaking grobs during translation¶
A later section examines methods to change the appearance and
positioning of grobs in the backend. The \applyOutput command is
a facility to apply a function to certain grobs created in a context
or any of its descendants, at a single moment. It is useful for
local tweaks depending on context settings. There are two forms for
\applyOutput:
\applyOutput Context #procedure
\applyOutput Context.Grob #procedure
In the latter form, grobs of the specified types are matched, while in the former, all grobs match.
The procedure is called with three arguments:
The grob,
The context it originates from,
The context
\applyOutputapplies to.
A motivating use case is to set the color for all grobs:
colorNotes =
#(define-music-function (color) (color?)
#{
\applyOutput Voice
#(lambda (grob origin-context context)
(ly:grob-set-property! grob 'color color))
#})
{
\colorNotes red
<e' g' bes'>4
<e' g' a'>4
\colorNotes green
<d' fis' a'>4
}
Overwriting grob properties set by engravers¶
The \override command has the effect of setting the context-wide
default for a grob property. Certain engravers, however, write
properties in the grobs they create. For instance, the
Note_heads_engraver sets the staff-position on freshly
instantiated NoteHead grobs based on the pitch in the note
event it reads. This makes \override ineffective. The
\overrideProperty command sets a property on a grob after
it has been created and initialized by its origin engraver.
Internally, \overrideProperty uses \applyOutput. The
\overrideProperty command takes the grob property path and
a value, like \override but without an equals sign.
{
\overrideProperty NoteHead.staff-position 50
c'1
}
Writing an engraver¶
Basics¶
Engravers reside in contexts. They are added to them in output definitions. The pattern is:
\layout {
\context {
\ContextName
\consists Some_engraver
}
}
Here, Some_engraver should be the name of a predefined engraver.
They are listed in the Internals Reference, under
Engravers and Performers.
The \remove command is the opposite of \consists and suppresses
an engraver from a context where it would have been active by default.
For engravers defined in Scheme and not registered, the argument is
no longer a string, but a procedure: #Some_engraver, where
Some_engraver takes a context and uses the make-engraver macro
to return an engraver 1.
Here is a template for writing an engraver:
#(define (My_engraver context)
(let (variables...)
(make-engraver
((initialize engraver)
...)
((start-translation-timestep engraver)
...)
(listeners
((event-class-1 engraver event)
...)
((event-class-2 engraver event)
...)
...)
((process-music engraver)
...)
(acknowledgers
((grob-interface-1 engraver grob source-engraver)
...)
((grob-interface-2 engraver grob source-engraver)
...))
(end-acknowledgers
((grob-interface-1 engraver grob source-engraver)
...)
((grob-interface-2 engraver grob source-engraver)
...)
...)
((process-acknowledged engraver)
...)
((stop-translation-timestep engraver)
...)
((finalize engraver)
...))))
\layout {
\context {
\SomeContext
\consists #My_engraver
}
}
Don’t panic! The rest of this section explains the hooks one by one.
The variables… are in a Scheme closure and therefore internal to the engraver.
The ellipsis in every method stands for arbitrary Scheme expressions. As with any Scheme function, they are evaluated in order. Their return values are not important because engravers rely on side-effects.
The time step cycle¶
The music is iterated following time. Time step is the term
for the basic unit of time. If there are simultaneous voices,
time steps are made so that every event is played in some time
step. Schematically (x represents a note):
4 4 x---------------- x----------------
\tuplet 3/2 { 4 4 4 } x---------- x---------- x----------
Time steps [ ][ ][ ][ ]
When a time step starts, the start-translation-timestep is called in all engravers. Naturally, this is paired with stop-translation-timestep at the end of the time step.
Furthermore, on its very first time step, the initialize method is called. It may be used to set up state.
Note that for this first time step specifically,
start-translation-timestep is not called. This is because
contexts are created through music (the \new command), and music
is iterated in time steps, so the context creation may actually
happen after the start of a time step.
The initialize method is paired with finalize, called in the end of the life of an engraver. This method is for bookkeeping, for example, warning about an unterminated spanner. The stop-translation-timestep method is called before finalize, as one would expect.
Processing stream events¶
In the listeners body of the engraver are methods specific to certain event classes. When a corresponding stream event is broadcast, these methods are called with the engraver itself and the event as arguments.
(listeners
((event-class-1 engraver event)
...)
((event-class-2 engraver event)
...))
Event broadcasting follows the context hierarchy. Events heard in a context are also heard in its parents. This means that an engraver residing in Voice context is appropriate to listen to note events, for example, whereas an engraver in Staff context is suited to listening key changes.
It is important to know that grobs should not be created in listeners. This is due to the unspecified order of events. In iterating, for example, simultaneous music, it is unclear what order the events should arrive in:
\new Staff <<
{ \time 3/4 c'2. }
\\
{ \override Staff.TimeSignature.color = red c2. }
>>
In the above example, the OverrideProperty event corresponding to
the \override command may well be heard after the TimeSignatureEvent.
Yet, we want it to affect the created time signature.
This is why the workflow of engravers is to use listeners to record the events they are interested in. Listeners may also set context properties. Then, when all events have been heard, the process-music method is called. This is where grobs should be created.
Creating grobs¶
- (ly:engraver-make-grob engraver grob-type cause)
Return a newly created graphical object.
The grob-type may be any name of a grob, given as symbol. The cause is either a stream event, or another grob. When no particular cause can be given for a grob (e.g., a bar line), it should be the empty list.
First engraver example¶
This engraver listens to rest events. When a rest was found in the previous time step but not in the current time step, it takes it as the end of a melody and adds a breathing sign to the left.
#(define (Auto_breathe_engraver context)
(let ((previous-rest-event #f)
(rest-event #f))
(make-engraver
(listeners
((rest-event engraver event)
(set! rest-event event)))
((process-music engraver)
(if (and previous-rest-event
(not rest-event))
(ly:engraver-make-grob engraver 'BreathingSign previous-rest-event)))
((stop-translation-timestep engraver)
(set! previous-rest-event rest-event)
(set! rest-event #f)))))
\layout {
\context {
\Voice
\consists #Auto_breathe_engraver
}
}
\relative {
\time 9/8
\partial 8
a8 b4 d8 f4. r
g,8 a c ees4 g8 r4.
}
Acknowledging grobs¶
With listeners and the process-music hook, many cases for grob
creation are covered. However, grobs also require links between
different types. Dots objects are associated to NoteHead,
for instance. For the sake of modularity, they are created by
separate engravers. This is why a mechanism exists for engravers
to process grobs created in other engravers. This is called
acknowledgers.
Every engraver can exclusively acknowledge grobs created by engravers whose enclosing context is a direct or indirect child of the engraver’s context. An engraver in Voice context acknowledges all grobs created from engravers in this Voice; an engraver in Staff acknowledges all those created from engravers in this Staff and engravers in children Voices; etc.
The body of acknowledgers in make-engraver resembles that of listeners.
(acknowledgers
((grob-interface-1 engraver grob source-engraver)
...)
((grob-interface-2 engraver grob source-engraver)
...)
...)
Every acknowledger is called with three arguments: the engraver it
is operating in, the grob, and the engraver where the grob has
been created. The function ly:translator-context can provide
the context in which the source engraver lives.
Just like with listeners, acknowledgers should generally be used to record the grobs, not process them directly to create other grobs, since other engravers may change properties of the acknowledged grobs in their own acknowledgers. The equivalent of process-music for acknowledgers is process-acknowledged.
((process-acknowledged engraver)
...)
New grobs may be created in process-acknowledged. This leads to a new cycle of acknowledgers, then process-acknowledged.
Engravers do not acknowledge their own grobs.
Second engraver example¶
This engraver adds a balloon on every note, indicating its pitch.
It sets a bunch of properties on the balloon. In particular,
X-offset and Y-offset, which are normally taken from an
AnnotateOutputEvent (from the call to \balloonText), are set
to values making the slopes increase vertically in a chord. The
annotation-balloon property is deactivated to suppress the
rectangle around every note head. The pitches are converted to
strings through the note-name->string function, using the
english input language.
Note the frequent pattern of resetting the list of acknowledged
grobs at the end of process-acknowledged, to avoid processing grobs
twice if there are further acknowledge cycles.
#(define (Balloon_notes_engraver context)
(let ((note-heads '()))
(make-engraver
(acknowledgers
((note-head-interface engraver grob source-engraver)
(set! note-heads (cons grob note-heads))))
((process-acknowledged engraver)
(for-each
(lambda (i note-head)
(let* ((note-event (event-cause note-head))
(pitch (ly:event-property note-event 'pitch))
(pitch-string
(string-capitalize
(note-name->string pitch 'english)))
(balloon (ly:engraver-make-grob engraver
'BalloonTextItem
note-head)))
(ly:grob-set-parent! balloon X note-head)
(ly:grob-set-parent! balloon Y note-head)
(ly:grob-set-property! balloon 'font-size -3)
(ly:grob-set-property! balloon 'font-series 'bold)
(ly:grob-set-property! balloon 'X-offset -2)
(ly:grob-set-property! balloon 'Y-offset (+ i 0.2))
(ly:grob-set-property! balloon 'annotation-balloon #f)
(ly:grob-set-property! balloon 'text pitch-string)
(ly:grob-set-property! note-head
'extra-spacing-width
'(-3.5 . 0))))
(reverse! (iota (length note-heads)))
note-heads)
(set! note-heads '())))))
\layout {
\context {
\Voice
\consists #Balloon_notes_engraver
}
}
{
<e' g' b'>1
<e' g' a'>1
}
End acknowledgers¶
Spanners are not finished in a single time step. Their bounds must
be set in two different time steps, and they may be terminated in
reaction to an event, like hairpins with \!.
In addition to acknowledgers, which is triggered at grob creation
time, the end-acknowledgers body contains acknowledgers triggered
at the time a spanner ends. This time is announced by the engraver
responsible for ending it. Custom engravers should also announce
spanner ends.
- (ly:engraver-announce-end-grob engraver spanner cause)
Announce the end of spanner for other engravers.
Like the one for
ly:engraver-make-grob, the cause argument should be an event, another grob, or the empty list.
Third engraver example¶
This engraver prevents line breaks from crossing ties. This is achieved
through setting the forbidBreak property of the Score context.
Note the logic with the tie-in-progress variable. The break should
be forbidden on the left of the note head even when the tie ends on
this head, and we do not want to disallow breaks on the left of
a head that starts a tie. This is why, as is commonly done, the
acknowledgers only record the ties. The final processing is done
in stop-translation-timestep (often this is rather in
process-acknowledged).
#(define (No_break_during_tie_engraver context)
(let ((tie-in-progress #f)
(starting-tie #f)
(ending-tie #f))
(make-engraver
(acknowledgers
((tie-interface engraver grob source-engraver)
(set! starting-tie grob)))
(end-acknowledgers
((tie-interface engraver grob source-engraver)
(set! ending-tie grob)))
((stop-translation-timestep engraver)
(if tie-in-progress
(ly:context-set-property!
(ly:context-find context 'Score)
'forbidBreak
#t))
(if ending-tie
(set! tie-in-progress #f))
(if starting-tie
(set! tie-in-progress #t))
(set! starting-tie #f)
(set! ending-tie #f)))))
% TODO: meaningful music
music = {
c'1 c'1~ 1~ 1~
1~ 1~ 1~ 1~ 1~
1~ 1~ 1~ 1~ 1~
1 1 1~ 1~ 1~
1~ 1~ 1~ 1~
1 1 1 1 1
}
\new Voice \music
\new Voice \with { \consists #No_break_during_tie_engraver } \music
Grob parents and spanner bounds¶
Every grob has parents for positioning. The functions related to parents are:
- (ly:grob-parent grob axis)
- (ly:grob-set-parent! grob axis)
Return or set the parent of grob on axis (X or Y).
Spanners are special however: they do not have one parent but two
bounds on the horizontal axis. A bound is an simply an item.
The distinction between items and spanners is explained at
Grob flavors and line breaks.
- (ly:spanner-bound spanner direction axis)
- (ly:spanner-set-bound! spanner direction item)
Return or set the bound of spanner in direction (LEFT or RIGHT).
Horizontal parents and bounds¶
Paper columns are special grobs used to group other grobs together.
At every point in time, a context has two important properties:
currentMusicalColumn and currentCommandColumn. They are
the default X parents for all items created in the time step. The
musical column is used for all musical items and vice versa.
A non-musical item is defined as an item having the non-musical
property set to true, or a child of such an item. All other items
are considered musical.
For spanners, there is no default and the bounds must be set explicitly. For spanners that have no particular items as bounds, either the musical or the non-musical columns should be used.
Vertical parents¶
When no particular Y parent is set, the Axis_group_engraver adds
the grob in a VerticalAxisGroup corresponding to the context.
The effect is that all grobs created in the same context are placed
relative to the same vertical base line.
Fourth engraver example¶
This engraver aligns all dynamics on the same vertical position. This
is equivalent to using a separate Dynamics context, except that the
dynamics can be entered in the main music input without resorting to
spacer rests.
This is achieved by creating a single DynamicLineSpanner for the
entire score, unlike the default Dynamic_align_engraver, which makes
one for every sequence of consecutive dynamics.
A context property is declared to control whether the dynamic line
goes above or below the staff. Refer to Adding properties and types.
The dynamics are added to the spanner using
ly:axis-group-interface::add-element, which sets pointers in the
object to its axis group in addition to making the group parent of the
object.
The bounds of the spanner are non-musical columns.
#(set-object-property! 'dynamicAlignDirection 'translation-type? ly:dir?)
#(define (Align_all_dynamics_engraver context)
(let ((line-spanner #f)
(direction #f))
(make-engraver
((initialize engraver)
(set! line-spanner
(ly:engraver-make-grob engraver 'DynamicLineSpanner '()))
(ly:spanner-set-bound! line-spanner
LEFT
(ly:context-property context
'currentCommandColumn))
(ly:grob-set-property! line-spanner
'direction
(ly:context-property context
'dynamicAlignDirection)))
(acknowledgers
((dynamic-interface engraver grob source-engraver)
(ly:axis-group-interface::add-element line-spanner grob)))
((finalize engraver)
(ly:spanner-set-bound! line-spanner
RIGHT
(ly:context-property context
'currentCommandColumn))))))
\layout {
\context {
\Voice
\remove Dynamic_align_engraver
\consists #Align_all_dynamics_engraver
dynamicAlignDirection = #DOWN
}
}
<<
\new Staff \relative {
c'2\< d4 e |
c4 e e,2\f |
g'4\dim a g a |
c1\p |
}
>>
Killing grobs¶
Time is sequential, with no look-ahead possibilities. This is why
engravers must create all grobs in the time step they belong to.
It it is later realized that a grob is not needed, the grob should
be killed using the dedicated function ly:grob-suicide!.
Fifth engraver example¶
This engraver adds lines between note heads in a melody. Observe the
use of ly:grob-suicide!: the voice follower must be started
whenever a note head is found, without knowing if the next time step
is going to contain a note or a rest. Afterwards, if there is no
note head in the time step, the previously created voice follower
is killed.
In addition, the engraver recognizes slurs with both acknowledgers and end acknowledgers. It kills the grobs it just created when a slur is in progress. Note that slurs are not announced in the time step where they end but in the following time step. Therefore, some trickery is required to suicide the right grobs and keep state properly.
This example has pretty unclear musical interest. There is some fun to it, however.
#(define (Voice_line_engraver context)
(let ((previous-follower #f)
(next-follower #f)
(note-head #f)
(note-head-found #f)
(accidental #f)
(slur-in-progress #f)
(slur-start-now #f)
(slur-start-before #f)
(slur-end-before #f))
(make-engraver
(acknowledgers
((note-head-interface engraver grob source-engraver)
(set! note-head grob)
(set! note-head-found #t))
((slur-interface engraver grob source-engraver)
(set! slur-start-now #t))
((accidental-interface engraver grob source-engraver)
(set! accidental grob)))
(end-acknowledgers
((slur-interface engraver grob source-engraver)
(set! slur-end-before #t)))
((process-acknowledged engraver)
(if note-head
(begin
(if previous-follower
(begin
(ly:spanner-set-bound! previous-follower RIGHT note-head)
(ly:engraver-announce-end-grob engraver
previous-follower
note-head)))
(set! next-follower
(ly:engraver-make-grob engraver
'VoiceFollower
note-head))
(ly:spanner-set-bound! next-follower LEFT note-head)))
(if (and accidental previous-follower)
(ly:spanner-set-bound! previous-follower RIGHT accidental))
(set! note-head #f)
(set! accidental #f))
((stop-translation-timestep engraver)
(if (and previous-follower
(not note-head-found))
(ly:grob-suicide! previous-follower))
(if slur-end-before
(set! slur-in-progress #f))
(if slur-start-before
(set! slur-in-progress #t))
(if (and previous-follower
slur-in-progress)
(ly:grob-suicide! previous-follower))
(if slur-start-now
(set! slur-in-progress #t))
(set! previous-follower next-follower)
(set! next-follower #f)
(set! note-head #f)
(set! note-head-found #f)
(set! slur-start-before slur-start-now)
(set! slur-start-now #f)
(set! slur-end-before #f)))))
\layout {
\context {
\Voice
\consists #Voice_line_engraver
\override VoiceFollower.style = #'dashed-line
\slurDashed
}
}
\relative {
\override Score.SpacingSpanner.spacing-increment = 4
d'16( e f8) e d a' d4 a8 |
bes8 g16( e) a8 f16( d) g8 e16( cis) a8 bes'8~ |
bes8 g16( e) a,8 cis'8~ cis bes16( g) a,8 e''~ |
e8 cis16( a) bes( g a f) g( e f d) e( cis d b) |
cis( a b cis) d( e f g) a( bes c8) c16( d ees8) |
fis,8 g cis, d gis, r a r |
}
- 1
Technically, the
make-engravermacro builds an associative list containing procedures. This is why you can sometimes see engravers written in the old way:#(define (My_old_style_engraver context) `((start-translation-timestep . ,(lambda (engraver) ...)) (listeners (event-class-1 . ,(lambda (engraver event) ...)) ...) ...))
Also note that it is not strictly necessary to pass a function taking the context and returning the engraver. Thus,
\consists #(let (...) (make-engraver ...))is accepted. However, this has different semantics: because the methods are defined unconditionally, the same variables are shared by all instances of the engraver in parallel contexts. This is generally not what you want.