Evaluating participatory concerts

Regular concerts are evaluated by orchestras in different ways. Marketing departments ask audiences for their opinion, productional leaders and process-managers evaluate the decision-making process. Artistic programmers evaluate the artistic choices made, and the extend to which the artistic concept in the programming was realised.

For an artistic programmer, evaluation of concerts can include artistic criteria such as: the choice of repertoire, how a specific concert relates to the rest of the programming, and the quality of the performance of the orchestra, with special attention to the conductor and the soloists. Quality of the performance is usually evaluated in artistic and aesthetic criteria, such as technical excellence or refinement, virtuosity, balance, intonation, articulation, etc.

However, in participatory or experimental concerts, organisational aspects can become artistically relevant: the process leading up to such a concert, artistic criteria of collaborative partners, the contribution of the audience in the development, and stage-design and other productional choices that affect the concert situation (read also theme 1). This insight requires a broadening of evaluation criteria, which is especially important to evaluate experimental and participatory projects.

The People's Salon

Broadening evaluation criteria

In addition to musical and aesthetic criteria, there are a range of other criteria that could potentially be artistically relevant. For example, in marketing surveys, audiences use a variety of criteria for assessing and evaluating their concert experiences:

Social criteria, such as sociality of an evening out, the joy of sharing the concert experience with others, whether there was a coffee-break;

Personal criteria, such as whether the concert invoked certain emotions, whether it was intellectually challenging, whether it appealed to personal memories or appreciations, whether the concert experience was relaxing, or criteria pertaining to the journey to and from the concert (whether there was enough parking, long queues, etc.);

Societal criteria, such as the relevance of the musical work, the composer, or the programming (i.e. how the program was linked to actuality or to the local region).

Usually, this range of criteria is not considered artistically relevant: artistic criteria pertain only to the musical performance itself. In participatory or experimental concerts, however, what matters artistically cannot be easily located and separated from other criteria beforehand. Instead of locating artistic relevance only to the musical work and its performance, in experimental and participatory concerts we can consider artistic quality as pertaining to all aspects that contribute to the emerging musical situation. This means that everything that contributes to the artistic experience, can be evaluated artistically.

For example, in the experiment The People’s Salon, the programmer shared his artistic responsibility for selecting repertoire for the evening with a group of ‘friends’ of the orchestra: regular and loyal concert-goers. During focus groups these friends (together with researchers and the artistic programmer) selected the repertoire based on their personal memories, stories and anecdotes about favourite musical works. This became the programme of the concert-evening. The stories became an artistically relevant contribution to the experience of the concerts. In addition, that audiences used their own stories to come up with a programming of the evening, added to the quality of the concert evening. In experimental or participatory projects, the process can become just as important as the end-result (the concert).

When evaluating experimental concerts, standard criteria used for evaluating regular concerts are not necessarily suitable. Other aspects can become artistically relevant for the experience of the concert, which demands new ways for evaluating their artistic contribution. New concert formats can require new norms and criteria, specific to these experimental projects.

Friend of philharmonie zuidnederland during The People's Salon

Ways of evaluation

The question is not only what criteria to use, the form an evaluation has also matters – especially when experimental or participatory projects are evaluated. How and when to evaluate participatory or experimental concerts can be challenging, as the programmer of philharmonie zuidnederland explains:

“We often don’t take enough time to really and thoroughly look back and learn from what we did. You also have to realize that these kinds of projects [experimental or participatory projects] elicit a lot of different and wide-ranging opinions. Everybody looks at these concerts in a different way. And then there is no space between finding it fantastic or nothing at all.”

Evaluating experimental projects can take additional time, as it can be unclear when it should be evaluated, who should be involved, and what the evaluation should be about. The schedule and planning of orchestras can be such that there is less time made available for proper evaluations after concerts.

Furthermore, as the programmer mentions, evaluations often take the form of black and white judgements: a concert then is either ‘good’ or bad’, successful or a failure. Researchers encountered this form of evaluation – judgements in either positives and negatives – also when evaluating the experiments Empty Minds and Mahler am Tisch. Musicians and staff tend to prioritize the quality of the actual performance, judging it either successful or not.

Judgement is a narrow understanding of evaluation. The aim of evaluations is not only to merely assess an experimental or participatory concert, but especially to learn from it. For an artistic programmer, experimental and participatory projects are potential rich moments for learning. Taking the time for evaluation, and explicitly developing fruitful ways of doing so, is therefore an invaluable precondition for innovative programming. Evaluation of these projects should therefore not only take place when these projects are finished, but also throughout the development of such project. Intermediate evaluations can be valuable moments for articulating emerging artistic criteria, and to keep track of the project during the making process.

A broader understanding of what can matter artistically, also necessitates a broader and more inclusive form of evaluation. Evaluating an experimental concert ideally includes artistic partners, musicians, audiences, planning staff, and productional leaders. Such an integrated or holistic approach to evaluation means that mutual learning can take place, between artistic programmers, partners, audiences, and staff. Such a form of evaluating concerts does take time. But the potential insights and lessons drawn from such projects, make them indispensable for an orchestra that aims to innovate and experiment.

Rehearsals for Mahler am Tisch

Summary

Evaluation is an important way of leaning in practice. Experimental or participatory concerts can be rich resources for learning. In such concerts, a variety of aspects can come to matter artistically as they contribute to the emerging musical situation – including the process. Evaluating such concerts therefore necessitates a broad view on what relevant artistic criteria could be, specific for that project. A broader understanding of what can matter artistically, also necessitates a broader and more inclusive form of evaluation – not as judgement, but as an opportunity for mutual learning.

Do you want to read more about this theme? It is described in more detail in the following academic articles:

Peters, P., van de Werff, T., Benschop, R. & Eve, I. (2022) Artful innovation: How to experiment in symphonic music practice. In Chaker, Sarah & Petri-Preis, Axel (eds), Tuning Up – Innovative Potentials of Musikvermittlung. Bielefeld: Transcript Verlag.

Spronck, V., Peters, P., & van de Werff, T. (2021) Empty Minds: Innovating Audience Participation in Symphonic Practice, Science as Culture, DOI: 10.1080/09505431.2021.1893681

Spronck, V. (2022) Listen Closely: Innovating Audience Participation in Symphonic Music, PhD dissertation, Maastricht University.