Fighting fire with fire: On using evaluation to counter the impacts of evaluation

by Andreas Hafner

Amsterdam_protests_Laauwen Media
Protesters in Amsterdam – Photo by Laauwen Media

During the last few years a number of bigger student protests have taken place in various countries (e.g. Chile or Canada). Some are taking place at this very moment. Protesters in the UK, Canada and the Netherlands are trying to improve living-, working- and studying- conditions for students and staff of their universities while also attempting to turn their universities’ structures more transparent. Let us take a more detailed look at one specific country. For a few months now students and staff of Dutch universities have protested for more democratic and transparent university governance structures and against precarious working conditions, financial speculation with university funds and the cutting and/ or merger of departments. Many of the conditions under critique are either directly about or at least closely linked to practices of evaluation in contemporary academia. The protests are not targeted at evaluation methods as such, but at the goals to which ends they are employed. These goals are financial efficiency and hand in hand with that the streamlining of knowledge production and teaching to the exploitation logic inherent in the currently dominant economic paradigm. Most of this critique of the current situation is not new. Quite a few authors have pointed out the problematic impacts of some forms of evaluation in academia, e.g. Shore and Wright with ” Coercive accountability: the rise of audit culture in higher education” (2000, pp. 58-89).

Especially with regard to these problematic impacts, evaluation can be seen as a form of ontological politics. An instance of this concept at work is e.g. described by Neyland and Woolgar (2014, pp. 21-54) with the example of a bin bag. This example is used to show how a seemingly ordinary object like a bin bag can become highly political through its enactment in different ways. In practice this means that distinct properties are ascribed to the object by different actors and a struggle for interpretative sovereignty ensues. Measurement and evaluation do not only assess the current status or future potential of people, objects and processes. They also define them to a certain extent and limit possible perspectives on and developments of these evaluees. An example for that would be an evaluation method that puts every evaluated practice or person into a limited number of categories. This leaves out other aspects and can be used to enforce a certain perspective on the context in which the evaluation is taking place. This does not necessarily happen intentionally, but it is hard to imagine a situation in which this aspect of evaluation is not present in one form or another.

Taking this perspective on the critiqued use of evaluation practices and their results might be of some use to the students and staff in the context of desired structural changes in the university system. The protesters could use the same tool intentionally, either as a suggestion for a different viewpoint that opens up new possibilities in negotiations or applied directly at the instances that they perceive to be responsible for the conditions they oppose. Certain forms of evaluation directed at either the university boards, the institutions responsible for the legal framework or other actors perceived as critical might, figuratively speaking, give these actors a taste of their own medicine. These methods of evaluation could range from letting students and staff rate university managements’ efforts to include their perspectives in decision making to discussing and assessing legislation in public gatherings. For this, the protesters would of course have to establish these evaluation methods in a broader context within society so that they actually have an effect on the respective instances. If they manage that, this might put some pressure on the targets of such a strategy to take the demands made more seriously.

The exact design of these evaluation methods and their employment in public discourse would of course have to be tailor-made for the specific contexts in which they should be used. As university boards are at least partly agreeing with the demands made, they could possibly also join forces with the protestors in this endeavor and create a broader alliance that would increase the pressure on the political actors to actually fulfill their recent promises of making academia in the Netherlands more democratic. Even if this form of creative provocation does not immediately lead to the desired changes, it might still produce some interesting reactions and start a broader discourse about what aims evaluation should fulfill in an academic setting and what role universities should play in contemporary society.

 

References

Neyland, D., & Woolgar, S. (2014). Wrong Bin Bag: the Situated Ontology of Mundane Governance. In Mundane Governance. Ontology and Accountability (pp. 21-54). Oxford University Press.

Shore, C., & Wright, S. (2000). Coercive accountability: the rise of audit culture in higher education. In M. Strathern (Ed.) Audit Cultures: Anthropological Studies in Accountability, Ethics and the Academy (pp. 58-89). Routledge.


Andreas Hafner is currently in the process of finishing the master programme at the Department of Science and Technology Studies at the University of Vienna. Apart from ontological matters he is interested in topics like monetary reform and post-scarcity economics.