Download Evolving Indirectly Represented Melodies with Corpus

Evolving Indirectly Represented Melodies
with Corpus-Based Fitness Evaluation
Jacek Wolkowicz, Malcolm Heywood, and Vlado Keselj
Faculty of Computer Science, Dalhousie University, 6050 University Ave.,
B3H 1W5 Halifax NS, Canada
{jacek,mheywood,vlado}@cs.dal.ca
Abstract. The paper addresses the issue of automatic generation of music
excerpts. The character of the problem makes it suitable for various kinds of
evolutionary computation algorithms. We introduce a special method of indirect
melodic representation that allows simple application of standard search
operators like crossover and mutation with no repair mechanisms necessary. A
method is proposed for automatic evaluation of melodies based upon a corpus
of manually coded examples, such as classical music opi. Various kinds of
Genetic Algorithm (GA) were tested against this e.g., generational GAs and
steady-state GAs. The results show the ability of the method for further
applications in the domain of automatic music composition.
Keywords: Music generation, unigrams, MIDI, generational genetic algorithms,
Steady-state genetic algorithms, automatic fitness assessment.
1 Introduction
Music is a well-structured organization of notes, thus one can probably design
algorithms and systems to support the various processes of music composition.
Applications for music touch many fields of computer science, but evolutionary
computation seems to be especially suited for music; not least because music
melodies have a linear, sequential nature, which clearly lends itself to representation
in terms of a gene structure. Moving beyond representational issues, there are still
many outstanding problems that are worthy of study. A few major problems to
address were summarized by McCormack [6]. This paper proposes approaches to two
of them: the problem of creating a music representation especially suited for
evolutional techniques and developing an automated fitness function that is able to
select pleasant individuals.
Both the most important and the toughest one is the evaluation of melodies in
terms of their fitness. A judgment whether certain melody sounds ‘good’ or ‘bad’ is a
vague issue even for humans. Many solutions to this problem try to overcome this
problem by manual fitness assignment, but this compromises our ability to automate
the process of music creation. Other approaches include fitness-assigning methods
that result from some theoretical models of music. However, limiting melody creation
results in artificial constraints on the resulting composition. The third approach, which
M. Giacobini et al. (Eds.): EvoWorkshops 2009, LNCS 5484, pp. 603–608, 2009.
© Springer-Verlag Berlin Heidelberg 2009
604
J. Wolkowicz, M. Heywood, and V. Keselj
will be used in this paper, utilizes sample pieces from a corpus to create classifiers
able to judge the properties inherent in new melodies.
2 Previous Work
In early approaches, introduced by Biles [1] or Jacob [3], the system relied on human
judgment as feedback for fitness evaluation. Johanson and Poli [4] augmented the
method with a neural network that used human feedback from their previous
experiments to automatic evaluation of later melodies.
Papadopoulos and Geraint [7] pointed out that there is no formal approach for
establishing the automatic evaluation of music quality, proposing a method for
automatic fitness evaluation based on static melody features. Other approaches to the
problem of automatic fitness assignment involve Neural Networks [9], SOMs [8],
fixed musicological rules [11] or similarity to a target melody [2]. The most similar
approach to the one proposed in this paper is based on Zipf’s law for music pieces [5].
3 Methodology
The approach presented in this paper is a novel approach to representing melodies and
in so doing avoids utilizing human feedback during fitness evaluation. However, a
sample of MIDI files is necessary to provide the source from which music features are
extracted. Specifically, a complete set of Preludes and Fugues from Das
Wohltenperierte Klavier 1 by J.S. Bach establishes the source of music features,
although no special significance is associated with this selection.
The system operates on the Western music twelve-tone scale. However, unlike
previous approaches, where notes were represented using absolute note pitches and
timings [2], here the smallest melody component will be the information about pitch
change and duration change (as proposed in [10]). The genome is a sequence of
genes, dubbed unigrams – pairs of integer values indicating the interval between notes
in semitones and the rounded binary logarithm of the ratio of the corresponding note’s
duration. Figure 1 shows the process of obtaining unigrams from a simple melody.
Fig. 1. Unigram extraction process
Naturally, the same melody can be played in different scales or/and different
tempos and it still remains the same melody and preserves relations between notes.
By emphasizing the underlying relation between notes we fulfill the human
Evolving Indirectly Represented Melodies with Corpus-Based Fitness Evaluation
605
perception of rhythm and melody. Since the pitch and the tempo of a melody is not
determined or needed, the scale and the tempo are necessary for playback only.
The other advantage of this approach over direct representations is that the logic of
the melody is preserved. Moreover, application of classical one/two point crossover
does not result in lethals (Figure 2).
Fig. 2. Two children created using different representations for standard crossover
3.1 GA Design Decisions
The evaluation of fitness is automatic and reflects the likelihood that the given
melody is a good melody given a corpus of musical compositions. The features of the
corpus are calculated by counting all melodies of ‘n’ consecutive unigrams, i.e., ngrams, and taking statistics of n-grams in the corpus like n-gram frequencies or
document frequencies (the count of documents in which the n-gram occurs). Then,
fitness can be evaluated according to the following general fitness function:
⎛
⎞
Γ2
Fitness( melody ) = Γ1 ⎜ wn ⋅
ϕ (n-gram )⎟
1≤ n ≤ N ⎝
n-gram∈melody
⎠
(1)
where Γ1 and Γ2 are some aggregate functions e.g., sum, average, max; Γ1 has the
range over the various sizes of n and Γ2 has the range of all n-grams in an individual
with the size of n; wn is a weighting factor of certain n-gram size; φ(n-gram) is a
function of a certain n-gram that is derived from corpus statistics.
Two forms of selection operator will be considered: Standard generational
(roulette-wheel) and a steady-state tournament. In the case of search operators, the
representation presented in this paper allows several classical operators to be
supported directly without the need
One-point crossover is defined such that the crossover point is selected
independently in each individual. That is to say, a melody coded using the method
proposed in this paper does not have the limitation where a gene at a position has a
certain meaning according to its localization in the genome. Moreover, we do not
want to impose a priori limits on the evolution of melody length.
Mutation consists of replacing a gene with a new tuple. Since unigrams represent a
change of melody, drawing a tuple from the corpus probabilistically will change a
single inflection point of the melody leaving the rest of the melody (relatively)
unaffected.
4 Experiments
The following experiments were conducted to characterize various features of the
problem. The population consists of 100 individuals, each being a sequence of 10
unigrams drawn randomly according to the probabilities obtained from the corpus. All
606
J. Wolkowicz, M. Heywood, and V. Keselj
experiments were written as Perl scripts using Perl::Midi and Perl::Midi::Corpus
libraries for analyzing and managing n-gram features of MIDI files and individuals in
the populations.
The key point in this system is to define the details of the fitness function. The
simple and most obvious decision is using sum (or average) for Γ1 and Γ2. One will
then have ranked individuals according to the average fitness of their components.
The main challenge is how to evaluate n-grams (individual sub-sequences) for their
likelihood of being good or bad melodies. The simplest approach is to take the
probability of n-grams. However, at transpired that despite the fact that ‘good’
melodies are usually quite complicated, the majority of the corpus is built from very
simple elements. The melodies evolved from φ functions based on the probability
model resulted in passages of notes just going up, down or simple trills.
The simple and conceptually acceptable solution to this problem is to use a variant
of the tf-idf measure from Information Retrieval, where it determines term
importance. Most important terms are the ones that are frequent, but occur in few
documents. The following formula was used to express the usefulness of an n-gram:
ϕ (n-gram ) =
log(tf (n-gram ))
df (n-gram )
(2)
4.1 Generational GA
Having established a suitably informative fitness function, we are in a position to
apply a standard Generational GA (GGA). The fitness function represented the sum
(Γ1) of five averages (Γ2) associated with component n-grams (n from 2 to 6). The
average values for fitness was around 1.5 and maximum fitness was changing rapidly
reaching the level of 3 (average φ value of 0.6) without the ability of keeping these
good genes across generations. It was still far from the peak φ values (~ 4.0).
In later generations a new phenomenon occurs – ‘note bloat’, unlimited growth of
individuals, similar to code bloat known from GP. It may result from many factors.
The individuals are not limited in length. Crossover is known to have a destructive
influence on the offspring. The influence of a wound after crossing over some
‘valuable’ regions may be hidden by the length of individuals. Hence shorter children
are more likely to suffer from this destructive influence then longer ones. Populations
with larger individuals register lower fitness properties (average and maximal).
4.2 Steady-State GA
A Steady-state, elitist selection operator always maintains the best solutions. Only one
child is created per iteration and the worst solution is replaced. An illustrative sample
run of the system in this approach is shown in Figure 3a.
Unlike GGA, the variety in the population drops and the whole population
‘converges’ to a single individual. There is no note bloat and the quality of solutions
is higher compared to those identified by GGA. Figure 3b shows the average
behaviour over 100 runs. In the first 5-10 runs the algorithm searches for a niche to
converge (increasing size of individuals), thereafter, a single solution appears.
Evolving Indirectly Represented Melodies with Corpus-Based Fitness Evaluation
(a)
607
(b)
Fig. 3. (a) Steady-state GA run over 100 epochs. (b) Average results of 100 runs of the system.
Mutation can inject new genes, resulting in new component melodies, which may
be more or less fit than those currently in the population. Steady-state selection
insulates us from the destructive influences of mutation. Moreover, it appears that
mutation is important; the rate of 0.2 results in a fitness improvement of around 20%.
Table 1. The Sample of resulting melodies
fit.
pattern
fit.
pattern
fit.
9.55
8.37
7.98
8.83
8.21
7.48
8.46
8.05
7.17
pattern
4.3 Results
The melodies generated by the system are shown in Table 1. Each melody is preceded
by its fitness. They were taken as final results from 100 runs of the steady-state
algorithm. It is usually easy to put those melodies into a harmonic context and most of
them preserve the scale or the key in which they are being played. Most of them are in
major scale, but some of them are complementary: the melody with the fitness of 8.21
is the major version of the minor melody with fitness 7.98.
5 Conclusions and Future Work
The method for generating music melodies using a corpus of musical pieces as a
guide for evaluating individuals gives encouraging results without resorting to
sophisticated evolutionary frameworks. Applying some knowledge of music while
formulating the representation has provided the basis for an indirect encoding, thus
avoiding the need for specialist search operators. The problem of evolving melodies
also demonstrates some new artefacts such as ‘note bloat’ that may be very interesting
to investigate.
608
J. Wolkowicz, M. Heywood, and V. Keselj
The results of the approach to date are characterized by a quite simple rhythmic
scheme. Separating rhythm from melody may help in obtaining more diversified
results, but one cannot make rhythm and melody totally independent from each other.
Other music features like establishing a proper beginning and ending and maintaining
a common harmonic logic and structure of an excerpt also need to be investigated.
The evaluation of results is still an open-ended issue.
References
1. Biles, J.: GenJam: A Genetic Algorithm for Generating Jazz Solos. In: Proceedings of the
1994 International Computer Music Conference, Aarhus, Denmark (1994)
2. Gartland-Jones, A.: MusicBlox: A Real-Time Algorithmic Composition System
Incorporating a Distributed Interactive Genetic Algorithm. In: Raidl, G.R., et al. (eds.)
EvoMusArt 2003. LNCS, vol. 2611, pp. 490–501. Springer, Heidelberg (2003)
3. Jacob, B.L.: Composing with genetic algorithms. In: Proceedings of the 1995 International
Computer Music Conference, pp. 452–455 (1995)
4. Johanson, B., Poli, R.: GP-Music: An interactive genetic programming System for music
generation with automated fitness raters (Technical report CSRP-98-13). School of
Computer Science, The University of Birmingham, Birmingham, UK (1998)
5. Manaris, B., Machado, P., McCauley, C., Romero, J., Krehbiel, D.: Developing fitness
functions for pleasant music: Zipf’s law and interactive evolution systems. In: Rothlauf, F.,
et al. (eds.) EvoMusArt 2005. LNCS, vol. 3449, pp. 498–507. Springer, Heidelberg (2005)
6. McCormack, J.: Open Problems in Evolutionary Music and Art. In: Rothlauf, F., et al.
(eds.) EvoMusArt 2005. LNCS, vol. 3449, pp. 428–436. Springer, Heidelberg (2005)
7. Papadopoulos, G., Wiggins, G.: A Genetic Algorithm for the Generation of Jazz Melodies.
In: Proceedings of the Finnish Conference on Artificial Intelligence (SteP 1998),
Jyvaskyla, Finland, September 7-9 (1998)
8. Phon-Amnuaisuk, S., Law, E.H.H., Kuan, H.C.: Evolving Music Generation with SOMFitness Genetic Programming. In: Giacobini, M., et al. (eds.) EvoMusArt 2007. LNCS,
vol. 4448, pp. 557–566. Springer, Heidelberg (2007)
9. Oliwa, T., Wagner, M.: Composing Music with Neural Networks and Probabilistic FiniteState Machines. In: Giacobini, M., et al. (eds.) EvoMusArt 2008. LNCS, vol. 4974, pp.
503–508. Springer, Heidelberg (2008)
10. Wolkowicz, J.: N-gram-based approach to automatic composer recognition. Master’s
Thesis. Warsaw University of Technology (2007)
11. Worth, P., Stepney, S.: Growing Music: Musical Interpretations of L-Systems. In:
Rothlauf, F., et al. (eds.) EvoMusArt 2005. LNCS, vol. 3449, pp. 545–550. Springer,
Heidelberg (2005)