Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

Saturday, 4 February 2017

Looking for sandy ground


"Free access to museums for under 30s", one reads in portuguese newspapers. The measure was approved in parliament yesterday. 

"Can anyone explain to me the logic of under 30s?", asks a Brazilian colleague.

"Is it to stimulate young families, like couples with small children?", replies another colleague. "Is it because it was found that unemployment is higher among the under 30s?"

Is it worth looking for the logic? Was there a logic? Was the measure based on any management report? Was it based on some audience survey? Were the professionals of the sector consulted? Are there concrete objectives that can be evaluated in one or two years’ time?

Sunday, 25 September 2016

Naming the impact: it may be Telmo or Rafael or Gustavo…

Telmo Martins, member of the Orquestra Geração (Photo: Maria Vlachou)

A few years ago, I saw the documentary Waste Land. It is about the work the Brazilian visual artist Vic Moniz created together with garbage pickers at the world’s largest garbage dump, Jardim Gramacho, in the outskirts of Rio de Janeiro. Moniz said that he wished to change the lives of a group of people with the same materials they deal with every day. So, together they used garbage to create large-size portraits of the garbage pickers, which were later sold in auction and the money was distributed among the garbage pickers. The works were presented in exhibitions in a number of contemporary art museums.

Sunday, 24 July 2016

Managing museums: a portuguese case

"Panels of St. Vincent" at NMAA (image taken from the National Museum of Ancient Art Facebook page)

The claim of a new legal status, of a special status, by the National Museum of Ancient Art (NMAA) in Lisbon has resulted in a very healthy debate among museum professionals in Portugal, especially (and unfortunately) after the announcement of the Minister of Culture that this status will actually be given to the museum. Independent of our criticism, positive or negative, of this case and this process, there is no doubt that we owe this very necessary debate to the NMAA, its director, António Filipe Pimentel, and to the entire museum staff*.

Sunday, 29 May 2016

First in our hearts

Image courtesy of the National Museum Soares dos Reis
How could we define the 'first' museum? Is it the one that best fulfills its mission? Or the one thar comes first to mind when you hear the word 'museum' (every marketeer’s dream)? Is it the one that has the biggest collection or the one that has the best collection? Is it the one that makes more exhibitions? Is the 'first' museum the one that produces a lot of news for the media, but continues to work for the same elite? Or is it the one that rarely makes the news, but works to diversify its 'elites'? Which one deserves to be considered 'first'? And who assigns the 'primacy', the museum to itself or the audiences, actual and potential, that benefit from its action?

Monday, 11 May 2015

One good idea, two responses and some lessons



It’s 125 years since Vincent Van Gogh’s death. Starting May 3 and for 125 days, the Van Gogh Museum in Amsterdam will be answering 125 questions regarding the painter, his life and his work. The museum invites anyone interested to ask a question to send it through their website and a page especifically created to present the results of this Q&A (watch the promotional video and visit the webpage).

Monday, 2 February 2015

What we know and what we don't do about it



In the last few weeks, I had the chance to talk to a couple of colleagues regarding some accessibility issues in their exhibitions. Things like poorly illuminated labels, bad contrast between letters and background, labels placed too low, objects exhibited at a high level and without inclination, long and complicated texts. I believe that these are issues that can easily be solved, without any further investment in money, just with some forward planning and the concern not to exclude. Actually, when exhibitions are designed to be inclusive, not only do they not cost more, but they can actually bring more money in, as more people will be able to access them.

I felt a bit puzzled when the people I approached told me that they knew all about those problems. Why did things happen that way, then? Is it possible that we are consciouly creating barriers to our exhibitions’ content? What do we do them for, then, if not for people to enjoy them?

I feel the same kind of puzzlement in conferences or training courses, when we discuss issues of management, communications, marketing, visitor services, education, etc. Quite often, some colleagues approach me and say: “We’ve been telling our superiors what you’ve just said to for years and years.”

Thus, it seems that there’s no lack museum professionals (including museum guards) who are aware of a number of small and big management or communications problems.  We have also got feedack from visitors themselves, through visitor books, comment cards, visitor studies, etc. Finally, there is also the contribution of academics, thinkers, bloggers, such as Maria Isabel Roque - who recently reminded us of some of the things that are still to happen, in her insightful post Acerca do que (ainda) falta ao património - or Luís Raposo - one of the few museum professionals in Portugal who regularly share their views publicly, his latest opinion article concerning the opening of the new Coaches Museum and future plans for museums in Lisbon’s Belem area.

So, we can’t complain that we haven’t already got truly valuable feedback – both from insiders and outsiders - which can help build strategies, correct mistakes, make decisions, register trends, understand changes and developments. Why don´t decision makers and those directly responsible for museum management act on it? What´s stopping us, what kind of barriers are we dealing with? Why are we going after more studies, new studies, if we haven’t done anything yet about the things we already know? Why existing knowledge seems to have no impact whatsoever on museum management and practices?

Here’s my attempt to identify some reasons:

It might be because, despite politically correct statements that museums are at the service of society, they are rather at the service of those who manage them. People – those who come and those who don’t come – and their interests and needs are actually not our principal concern. Objects are and it’s enough that they look beautiful for those who know how to appreciate them.

It might be because in this field we work with very short-term plans, which follow the electoral circles and which may easily be abandoned, with no further explanation or responsibilities taken. Thus, big and small issues remain and their discussion is perpetuated without brining any concrete developments.

Finally, it might be because we tend to settle for what’s “good enough”. We know what the problems are, but there comes a moment when we cannot insist anymore: either because we can’t get our arguments across or because we feel that we cannot expect or demand more from other people. Only that “good enough” is not good enough and the argument of “one step at a time” doesn’t always take us as far as we should go. In fact, it often keeps us just where we are.


More on this blog




Monday, 4 October 2010

Who ‘deserves’ to be funded? (II) Some conclusions

The relationship of many people with the cultural sector, perhaps of the majority, is the one described by John Holden, in page 32 of his text Capturing Cultural Value, as 'non-use values'. That is, they appreciate the fact that it exists (existence value), regardless of using it or not; they keep open the possibility of using it in the future, although they do not use it in the present (option value); they think it is important to bequeath something to future generations (bequest value).

Thus, while I was reading a series of
texts on the value of culture and its funding, I kept asking myself: does it make sense to continue focusing the debate on how to prove the value of culture? Is this what we need to convince people – audiences and non-audiences, politicians, sponsors-, of the ‘value of culture’? Who´s questioning it?

Culture has an intrinsic value, largely intangible, unmeasurable. Culture touches us, marvels us, make us grow as people, help us become more tolerant and demanding, less ignorant and arrogant. It makes us think about ourselves and the world. Each one of us lives this relationship in a very personal way; and each one of us can speak for themselves. These testimonies, many times recorded, are not exactly a ‘proof’, but they help us understand, and show others, how the cultural offer is received, understood and felt.

When discussing cultural funding, rarely do we use this kind of arguments. Because they are not easy to ‘prove’ and because they do not seem to be sufficient. They are not the expected indicators. In the meantime, in our ‘apology’ we often refer to the results of culture´s ‘collateral effects’, that is the ones related to economy, urban regeneration, social and health problems, etc. These exist and have already been proved in various reports.

It seems to me, though, that we shouldn´t be evaluating culture in general, and the arts in particular, based on these indicators. Monitoring these results is the job of the agents who represent each one of those sectors, which interact with the cultural sector because they find the interaction to be beneficial. I think that the cultural sector should concentrate on proving that it aims and manages to create bridges with the other sectors, so that its offer can be more promoted, more accessible and more ‘used’, resulting in more and more people getting in touch with its intrinsic value – the one that is hard to prove and measure, but which each person feels and understands, in his/her own way, when experiencing it. In my opinion, it is in these terms that the value of the cultural sector should be debated and evaluated. And it is also based on these terms that funding criteria should be established.

When considering the distribution of money, it seems to me that, generally speaking, we could identify three types of beneficiaries: artists who work individually, who produce; artists with a supporting structure; cultural institutions.

The job of the artists is to create; to create art of excellence. And the State should guarantee the conditions for that to happen. Artists do not create because it is beneficial for the society. They create because this is their way of breathing, of communicating. It is not up to them to prove that their art helps to solve health or social or other problems. The decision to fund them should be based on the quality of their work. Nevertheless, it seems legitimate to me to expect that an artist financed by the State would be open for collaborating with the mediators (education, outreach, communications staff), who aim to open the way for the public to come to the encounter with his/her art. Audiences, politicians, sponsors find it difficult to attribute value to something they don´t know it exists or something that looks strange to them, incomprehensible, and, thus, apparently useless and at times frightening.

In this context, we should pay particular attention to the fear and discomfort the words ‘contemporary art’ exercise on people. Noone protests against the funding of museums, companies with classical repertories, artists whose work does not defy the established canons… These are proposals the importance of which the general public accepts, even if they never use them. On the other hand, people´s relationship with experimental contemporary art, the art that aims to question the canons, to create new ways of looking at ourselves and the world, is less peaceful. Why? Because the majority of the people do not possess the necessary tools in order to be able to attribute value and importance to it. Apart from the obligation to guarantee the conditions so that this art cab happen, there is another obligation: to ‘educate’ the public so that it can learn to appreciate it (and then accept it or discard it), to give them the necessary tools in order to discover and explore it. And this cannot happen without the artist´s collaboration.

In what concerns the structures supporting an artist or the companies or cultural institutions in general (museums, galleries, cultural centres, performance halls), I am not at all uncomfortable with the idea of setting objectives (some common to all and some specific, decided by the funder and the cultural organization, according to its mission – see
post on the establishment of objectives in the funding agreements between british museums and the government). It seems legitimate to me that the decision to fund gives priority to those interested in pursuing those objectives. If we value the establishment of education departments or the establishment of a relationship with under-represented target-audiences, should priority be given to those who aim to follow that direction? If we value the elimination of physical barriers, so that disabled citizens may enjoy the cultural offer, wouldn´t it be legitimate to expect that publicly funded institutions provide access to them? Just to give two examples.

I believe that development can only be achieved with the establishment of concrete and measurable objectives, in the medium and long term. This is how we can manage to eliminate, little by little, the barriers of access to culture - mental, physical and financial. This is not about the value of culture, it is about access to it.

These and other issued will be discussed on the 6th of October in a debate that aims to provoke and is entitled Just what are the arts good for?”, organized by the Institute of Ideas in partnership with Culturgest.


Note on the 6th of October:
Being this a very relevant issue for the cultural sector, it is being discussed in various forums in many countries. Artsblog published today the post Proving what we know is true, that informs us of a study that is going to be undertaken by Theatre Bay Area with the objective to create a service that will allow them to quantify the intrinsic impact of their work. More information on this here.

Finally, it´s worth reading an article by Nicholas Serota, director of the Tate and one of the most influential people in the british cultural sector, published in the Guardian on the 5th of October and entitled A blitzkrieg on the arts. Here´s an excerpt that I consider fundamental should we really wish to change the terms of the debate: "With the ruthlessness of a blitzkrieg the coalition is threatening the stability of an entire system for cultural provision that has been built up by successive Conservative and Labour governments: a mixed economy of public and private support that has made Britain a civilised place to live, where all have an opportunity to enjoy the arts or celebrate our heritage, and have been doing so in increasing numbers."

Monday, 28 June 2010

About the Culture Minister´s interview

Culture Minister, Gabriela Canavilhas, gave an interview on the 25th of June to the newspaper Público (read here in portuguese only), announcing significant cuts in the area of culture, that will affect all sectors.

I believe there are various points that should be better explained. The fact that the reactions of various cultural agents obliged the Ministry to issue a press release on the day the interview was published (read
here in portuguese only) indicates that there has apparently been little dialogue and sharing of information. Everybody has manifested solidarity, but they don´t like being taken by surprise and feel that they should be more involved in the search for solutions. It makes sense.

I believe that the Culture Minister is someone who knows what she wants, confident, courageous and focused. She was very fast briefed on the situation of the various cultural sectors and took initiatives in order to initiate processes aiming to tackle issues that have been on standby for a long time.

This is why I feel disappointed every time the Minister seems to avoid answering certain questions and even gives information to the general public that is not precise. She is using the ‘tricks’ of politicians, which do not work with people that have a more profound knowledge of the field and, I repeat, keep the general public misinformed.

There are two moments in the interview that caused me this feeling, both related to museums. First, when the Minister was questioned about the controversial transfer of the National Museum of Archaeology, she answered:

“The public visibility of the controversy, if we look carefully, has been motivated by two or three people. It doesn´t have a large dimension. It´s the restricted movement of a restricted group. The fact that it has gained a lot of visibility in the newspapers doesn´t mean that it emanates from a significant civil force”.

“Rarely in culture do we see movements from large civil forces”, insisted the journalist Vanessa Rato.

“It is not so”, answered the Minister. “Look at the case of the Popular Art Museum. This was really a movement, with a very interesting strength. In the case of the Museum of Archaeology, it is the director and two or three more people…”.

I don´t want to believe that the Minister is not aware of the disagreement of a large part of museum professionals, of the intense activity of the National Museum of Archaeology Friends Group, of the heated debate in this group´s
blog, of the declaration of the Assembly-General of the National Committee of ICOM (International Council of Museums), that originated a petition. All these people are not simply against the transfer of the museum. They are against the transfer of the museum while there is not a study that can guarantee that the Cordoaria has the necessary conditions in order to receive a collection of great national importance.

But I would say more. Even if there were only two or three people against the transfer of the museum, even if it was only one, the Director, the fact that these people are museum professionals should be enough for the Minister to pay more attention and not try to minimize or ignore the validity of their opinions an actions. Because Luís Raposo is an excellent professional. Because he is a man of dialogue, looking for consensus. Because he´s not just whining. He´s fulfilling his responsibilities, trying to guarantee, in the name of all of us, the conditions for the preservation of the National Museum of Archaeology collection.

The second point in the interview that left me feeling uncomfortable was when the Minister was questioned about the efficiency of the articulation between the Ministry of Culture and the Ministry of Economy, giving as example the case of the exhibition “Encompassing the Globe”.

“I didn´t follow that process. I was in the Azores”, answered Gabriela Canavilhas.

Well, we all know now that the exhibition´s objectives are far from having been achieved and that the costs for the State, for the tax payers, have been considerable. The details were published in the newspaper Público in the 21st of April (read
here in portuguese only), Gabriela Canavilhas being already Culture Minister. In my post of the 26th of May, entitled NMAA in the news, I was commenting on the fact that former Culture Minister, José António Pinto Ribeiro, the minister responsible for the exhibition coming to Portugal, declined to comment on the news. I considered that it was his obligation towards the citizens to assume his responsibilities and answer the questions. Similarly, I don´t think it was correct on behalf of the actual Culture Minister to avoid answering when questioned about the efficiency of the partnerships between her Ministry and the Ministry of Economy. Even if she was still in the Azores, she would have surely heard the news. The question asked is very valid and what I would expect from a Minister who knows what she wants, confident, courageous and focused, would be a clear answer, guaranteeing that cases of bad management of public money, as the one of the exhibition “Encompassing the Globe”, will not be repeated.

Monday, 10 May 2010

In order to evaluate we need objectives


“What is quality in a national theatre?” was the title of an opinion article by Christoph Dammann, ex-artistic director of the São Carlos National Opera House, published in the Público on the 3rd of May. I found the article very interesting. On one hand, because Dammann presented a number of indicators that could be useful for evaluating his performance as director of the national opera house in the last three years. Among them, the amount invested by the State on each spectator, something rarely discussed here and thus rarely considered as a performance indicator in our evaluations. But also, the increase in spectator numbers, the number of performances by portuguese singers, the number of co-productions with other countries. On the other hand, because in this article Dammann raised the question of the objectives, which should be defined by the Ministry together with the directors of cultural institutions. 

“The Theatre proprietor – the State – must discuss the following questions with the Director: how many productions should be presented in a season, with how many performances, at what cost for the State for each ticket sold? How many shows, concerts, rehearsals should me done by each element of the orchestra and the chorus?”. The numbers presented by the ex-artistic director of São Carlos seem valid. But, before discussing evaluation, we need to know what the initial objectives were. Only then should we be able to tell if he was successful or not. 


A few months ago, I heard of a museum director who was suddenly warned by his superiors that visitor numbers were low in relation to their expectations. What can one do when receiving such a warning? Easy: accept more school bookings and numbers will rise instantly. But will the quality of the visit be the same? Will the education service be able to receive numerous students with the same efficiency and care?

It is not difficult to arrange some useful numbers when necessary. If we are to take evaluation seriously, though, it is obvious that there can only be evaluation when there are objectives. These objectives must be discussed between the Ministry and the Directors, accepted by both sides and then communicated to the respective teams. Thus, everybody is clear regarding the final aims and the role each one has in the collective effort. The next step is to define performance indicators for the final evaluation. We can then say that we are all speaking the same language.

There are two very useful documents regarding this subject, both published by the British Department for Culture, Media and Sports. The first dates from 2006 and is entitled Understanding the Future: Priorities for England´s Museums (available here). The second is Balancing the Scorecard: a review of DCMS Performance Indicator Framework (available here). Another interesting element is that in the UK there are funding agreements between the State and, for example, National Museums, available on the DCMS site (see the example of the British Museum). In this document, there are clearly defined objectives the museum is proposing to reach within the three years covered by the agreement and the way it proposes to monitor its performance. Most important of all, they are available to anyone wishing to consult them.

Monday, 26 April 2010

NMAA in the news

The National Museum of Ancient Art (NMAA) made news three times last week. First, it was the interview with its new director, António Filipe Pimentel, to the newspaper Público (read here in portuguese only). Two days later, it was the news about the costs of the exhibition Encompassing the Globe (read here in portuguese only), complemented the next day with a new article (read here in portuguese only).

Starting by the news regarding the exhibition, they were commented by many readers of Público online. Some of those defending the investment for the presentation of the exhibition in Portugal made the ‘inevitable’ comparison to the money invested on football (really, I think we should give up on this argument that doesn´t serve any purpose; in any way, football, apart from being profitable, mobilizes, excites and entertains thousands of people, not just some “illiterate who have never visited a museum”). Other comments in favour referred to the benefit for the country from the fact that the exhibition was presented at the NMAA, to the fact that it attracted more than 70000 visitors, to the works undertaken that benefited the museum and are there to stay, to the opportunity for the museum staff to get in touch with international museums and for the museum to open up to international museography.

It´s all a question of options, priorities, objectives. So there are many ways of evaluating the impact of the exhibition. My comments are the following:

I didn´t like it. Because I have stopped enjoying exhibitions that simply present beautiful objects, but they don´t help us appreciate them, they don´t tell a story. Encompassing the Globe was a traditional exhibition in terms of museology and thus didn´t make any sense to many of the people who visited it. I confess, I was one of them. I guess, because I haven´t got any concrete data, that if it managed to attract so many visitors it was because it was heavily advertised and presented as an exhibition that could not be missed. Because the subject was Portugal and the Portuguese, which interests both the national population and foreign tourists. Because “Smithsonian” is an important name that was used, as it should, in the promotion. These are my empirical explanations. I would be interested to know, though, if the large number of visitors was also the result of word-of-mouth, a sign that people who visited had liked the exhibition and recommended it to family and friends. I didn´t recommend it to anyone and noone recommended it to me.

In relation to the investment, I would say that, although we lack experience and also tradition in what concerns cultural sponsorship, we should at least know that, if six months before the exhibition opening we haven´t got the necessary funding (according to a statement by the then Minister of Culture José António Pinto Ribeiro), we are not likely to get it. This kind of business takes place well in advance. It´s also difficult to undestand why organizations that had sponsored the exhibition in Washington were not contacted regarding the presentation at the NMAA. And if the works in the museum or staff contact with international museography are so valued, as they should be, then we should admit their importance and invest directly on them, instead of through the pretext of a very expensive and, really, not so distinguishing exhibition.

In the meantime, I ask myself once again why it´s never part of the plan a summative evaluation of the impact of such expensive initiatives and a visitor survey, instead of each one of us giving a hint about it. On the other hand, I feel very disappointed when, in moments like this, those responsible for the decisions taken claim the right to be unavailable for comments, instead of considering it an obligation towards the citizens to answer the questions they are being asked.

Moving on to the first piece of news, I liked the interview with NMAA´s new director, António Filipe Pimentel. Because he claims to believe in the “middle way”. The one that does not give preference to any of the museum´s five fundamental functions (to collect, preserve, study, exhibit and interpret), setting them as opponents, but aims to establish a balance between them (see also his opinion article in the ICOM.PT bulletin available here in portuguese only
). And I quote: “There is a need to bring into harmony the two sides: to have a strategic and instrumental view of preservation and study, but not limiting it to just that. It must be put to the service of the community, which, nevertheless, can never sacrifice it, cannot jeopardize its security and safety.” Bringing harmony is not easy and depends a lot on the resources available, human and financial, the lack of which makes it many times compulsory to make options and establish priorities. But is seems to me that it is also a question of mentality. António Filipe Pimentel also refers to the museum as a stage, “the space for mobilizing a community making use of the objects exhibited, behind which there are always stories”. And he goes on: “A museum is not the National Archive. In an archive we have a treasure of information which is kept, preserved and consulted. Ours must be exhibited and permanently shown in stories and narratives”. I was very pleased to read these words. I see behind them what I consider to be the right mentality. I hope NMAA´s new director and the museum staff can put them into practice, can manage to transform them into concrete actions. We are attentive and full of expectations.