Making impact with a tea bag – How to create trust in citizen science data?

By Jóia Maria Boode, Lisa-Maria Chmarra, Teresa Heinz, and Anna-Maria Lipp

With human made climate change rapidly progressing on a global scale, leaders in policy, industry and academia are looking for answers on how to halt this threat. Increasingly, scholars are looking for solutions outside of academia, a hitherto under-used source of knowledge production, namely citizen science.

For a group assignment in the STS-department of Vienna, we were asked to write a research proposal for a project that would be possible, but not actually conducted. Our proposal focuses on the use of citizen science in environmental sciences, and specifically on the quality of the data collected through this method. In this blogpost we hope to share with you why we think this topic is an important venue for research, and hope to inspire you to think critically about your intentions as you conduct your own research. In this way we hope that our proposal can still contribute to actual research being done.

Projects in fields like biology, gathering data on everything from temperature, to plant cycles, to bird migration are more prevalent than ever, but these efforts are halted by the lack of one important factor: trust. The difficulty of ensuring data quality of data collected through citizen science, especially when the collected data depends on interpretations (which is often the case in environmental studies) explains why citizen science is often not considered (and therefore used) a serious research method in many fields, including STS. We therefore wondered why there was barely any literature available about policing the quality of CS data, especially in the field of STS. Merging our collective experiences from research, arts and social relations, we focus on exactly this research gap about trustworthiness of citizen science data.

To specify our research question, we decided to connect our questions to a specific project that used Citizen Science as a means to collect data. And since we all share an interest in environmental studies through a passion for making a positive impact on the world through scientific practices, we decided to pick a case where citizen science is applied for environmental monitoring. We chose “The Tea Bag Index” (TBI), an Austrian project that lets participants use tea bags to measure soil quality by weighing them before putting them in the ground and three months after. Since this project already finished and published its results, we could look at how it succeeded at producing multiple peer-reviewed articles, gaining international press coverage, and winning the 2016 Austrian Citizen Science Award.

Picture depicting the authors on their way to bury tea bags

For this project we interviewed Max Fochler, because he has experience in working on the “public understanding of science and the public engagement of citizens in science.” In his early research, Max expressed that he felt it was strange to only focus on the public side of the citizen-involvement debate, and by reflecting on that he started to focus more and more on the side of the scientist and the process of creating knowledge.

Since we wanted to look into data collection, and specifically ensuring the quality of this data, we chose to focus on the “scientist-side” of citizen science projects. The basis of our case was to investigate the excellence and relevance of specific scientific practices. In the interview, Max asked: “assuming you are interested in the particular social and environmental impact of your work, how does that relate to your actual practices in science?” It made us think about our research approach and the goal we wanted to reach with our research. Was our research contributing to making a bigger impact? Or were we putting more control and therefore pressure on scientists who wanted to use citizen science in their research, therefore confirming the biases around citizen science?

Interestingly Max pointed out that “the more you talk to scientists, the more you realise that in their perception, a lot of things do not work well in the current academic systems, on a lot of different levels.” And though we can emotionally agree with each other that the amount of competition found in the academic world is negatively influencing research practices as well as the personal well-being of scientists (Morrish, 2019) it is hard to actually prove the connection of this to both the relevance and excellence of research practices and publications. The noticeable effects are different: e.g. losing talented people who get fed up with the hierarchical, competitive structures within the scientific work-practice, or people feeling pressured and burned-out. The influence on the actual knowledge that is being produced is not investigated so much.

The information that Max provided us with supports the finding of the research gap on quality control of data collected through citizen science, and partly explains why this gap is there. Structures that push researchers to publish many articles make it a safer choice for researchers to pick a method that is already considered to be credible, rather than going for a long-term, time-consuming citizen science project of which the results might be questioned (if even) by colleagues and peer reviewers. Max claimed that the influence of these structures on produced knowledge, especially in projects that deal with environmental or socio-environmental issues, is one that should absolutely be questioned as it might explain why certain topics are structurally avoided or overrepresented. “It is crucial to investigate whether a researcher strives to make an actual impact with their project, by for example focusing the outcomes on practical tips for citizens or farmers on how to improve their soil quality. Or do they strive to get the results that will lead them to a professorship or a highly cited article?”

The question of knowledge production is central to the field of STS, but empirical research on citizen science is largely lacking. Therefore, we argue that our research is an appropriate supplement to the field of STS, as it aims to fill this gap. We hope that our research proposal for creating citizen sciences standards will at the very least inspire others to think critically about the intentions of their research and stand still for a moment when they choose between making an impact on the world around them, or reaching the next step in their own career.


Heigl, F. (2022, August 31). Tea Bag Index – Österreich forscht.
Morrish, L. (2019). Pressure Vessels: The Epidemic of Poor Mental Health among Higher Education Staff. Occasional Paper 20. Higher Education Policy Institute.

Jóia Maria Boode (she/her) is a master student in the STS program at the University of Vienna. Previously she studied Interdisciplinary Arts with a focus on documentary film. She currently works as a documentary filmmaker in academic research groups, where she communicates science in a creative way. Her fields of interests are creative science education, science as a practice and culture and inequality studies with a focus on feminist technoscience.

Lisa-Maria Chmarra (she/her) is a master student in the STS program at the University of Vienna. Previously she studied sociology and anthropology. She currently works as a project manager in the field of science communication. Her fields of interest are science engagement, science as a practice and culture, and inequality studies.

Teresa Heinz (she/her) is a master student in the STS Program at the University of Vienna. Previously she followed a Bachelors’ Degree in International Relations and Management. Her fields of interest are the role of politics in science and how the human mind and cognition are influenced by “things” like institutions.

Anna-Maria “Ria” Lipp (-/She) is a master student in the STS program at the University of Vienna. (-she) has a background in environmental sciences and engineering and currently works as a prae-doc research assistant at the TU Vienna. Ria is really interested in feminist science studies and the intertwined relations between science and society, with a focus on integration, communication and activism.