Mathias Bjare, Stefan Lattner, Gerhard Widmer,
"Differentiable Short-Term Models for Efficient Online Learningand Prediction in Monophonic Music"
, in Transactions of the International Society for Music Information Retrieval, Vol. 5, Nummer 1, 11-2022
Differentiable Short-Term Models for Efficient Online Learningand Prediction in Monophonic Music
Sprache des Titels:
As pieces of music are usually highly self-similar, online-learning short-term models are well-suited for musical sequence prediction tasks. Due to their simplicity and interpretability, Markov chains (MCs) are often used
for such online learning, with Prediction by Partial Matching (PPM) being a more sophisticated variant of simple MCs. PPM, also used in the well-known IDyOM model, constitutes a variable-order MC that relies on exact matches between observed n-grams and weights more recent
events higher than those further in the past. We argue that these assumptions are limiting and propose the Differentiable Short-Term Model (DSTM) that is not limited to exact matches of n-grams and can also learn the relative importance of events. During (offline-)training,
the DSTM learns representations of n-grams that are useful for constructing fast weights (that resemble an MC transition matrix) in online learning of intra-opus pitch prediction. We propose two variants: the Discrete Code Short-Term Model and the Continuous Code Short-Term Model.
We compare the models to different baselines on the ?The Session? dataset and find, among other things, that the Continuous Code Short-Term Model has a better performance than Prediction by Partial Matching, as it adapts faster to changes in the data distribution.
We perform an extensive evaluation of the models, and we discuss some analogies of DSTMs with linear transformers. The source code for model training and the experiments is available at https://github.com/muthissar/diffstm.
Sprache der Kurzfassung:
Transactions of the International Society for Music Information Retrieval
Anzahl der Seiten:
Aufsatz / Paper in sonstiger referierter Fachzeitschrift