In order to evaluate we need objectives

“What is quality in a national theatre?” was the title of an opinion article by Christoph Dammann, ex-artistic director of the São Carlos National Opera House, published in the Público on the 3rd of May. I found the article very interesting. On one hand, because Dammann presented a number of indicators that could be useful for evaluating his performance as director of the national opera house in the last three years. Among them, the amount invested by the State on each spectator, something rarely discussed here and thus rarely considered as a performance indicator in our evaluations. But also, the increase in spectator numbers, the number of performances by portuguese singers, the number of co-productions with other countries. On the other hand, because in this article Dammann raised the question of the objectives, which should be defined by the Ministry together with the directors of cultural institutions. 

“The Theatre proprietor – the State – must discuss the following questions with the Director: how many productions should be presented in a season, with how many performances, at what cost for the State for each ticket sold? How many shows, concerts, rehearsals should me done by each element of the orchestra and the chorus?”. The numbers presented by the ex-artistic director of São Carlos seem valid. But, before discussing evaluation, we need to know what the initial objectives were. Only then should we be able to tell if he was successful or not. 

A few months ago, I heard of a museum director who was suddenly warned by his superiors that visitor numbers were low in relation to their expectations. What can one do when receiving such a warning? Easy: accept more school bookings and numbers will rise instantly. But will the quality of the visit be the same? Will the education service be able to receive numerous students with the same efficiency and care?

It is not difficult to arrange some useful numbers when necessary. If we are to take evaluation seriously, though, it is obvious that there can only be evaluation when there are objectives. These objectives must be discussed between the Ministry and the Directors, accepted by both sides and then communicated to the respective teams. Thus, everybody is clear regarding the final aims and the role each one has in the collective effort. The next step is to define performance indicators for the final evaluation. We can then say that we are all speaking the same language.

There are two very useful documents regarding this subject, both published by the British Department for Culture, Media and Sports. The first dates from 2006 and is entitled Understanding the Future: Priorities for England´s Museums (available here). The second is Balancing the Scorecard: a review of DCMS Performance Indicator Framework (available here). Another interesting element is that in the UK there are funding agreements between the State and, for example, National Museums, available on the DCMS site (see the example of the British Museum). In this document, there are clearly defined objectives the museum is proposing to reach within the three years covered by the agreement and the way it proposes to monitor its performance. Most important of all, they are available to anyone wishing to consult them.