Start Submission

Reading: How Does Policy Conceptualise Citizen Science? A Qualitative Content Analysis of Internation...

Download

A- A+
Alt. Display

Research Papers

How Does Policy Conceptualise Citizen Science? A Qualitative Content Analysis of International Policy Documents

Authors:

Susanne Hecker ,

Helmholtz Centre for Environmental Research, UFZ; German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, DE
X close

Nina Wicke,

TU Braunschweig, DE
X close

Muki Haklay,

University College London, GB
X close

Aletta Bonn

Helmholtz Centre for Environmental Research, UFZ; German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig; Friedrich Schiller University Jena, DE
X close

Abstract

Policy and science show great interest in citizen science as a means to public participation in research. To recognize how citizen science is perceived to foster joint working at the science-society-policy interface, a mutual understanding of the term “citizen science” is required. Here, we assess the conceptualisation and strategic use of the term “citizen science” in policy through a qualitative content analysis of 43 international policy documents edited by governments and authorities. Our results show that most documents embrace the diversity of the research approach and emphasize the many benefits that citizen science may provide for science, society, and policy. These include boosting spatio-temporal data collection through volunteers, tapping into distributed knowledge domains, increasing public interest and engagement in research, and enhancing societal relevance of the respective research. In addition, policy documents attribute educational benefits to citizen science by fostering scientific literacy, individual learning, and skill development, as well as by facilitating environmental stewardship. Through active participation, enhanced ownership of research results may improve policy decision-making processes and possibly democratise research as well as public policy processes, although the latter is mentioned only in a few European Union (EU) documents. Challenges of citizen science mentioned in the analysed policy documents relate mainly to data quality and management, to organisational and governance issues, and to difficulties of the uptake of citizen science results into actual policy implementation due to a lack of citizen science alignment with current policy structures and agendas. Interestingly, documents largely fail to address the benefits and challenges of citizen science as a tool for policy development, i.e., citizen science is mainly perceived as only a science tool. Overall, policy documents seem to be influenced strongly by the citizen science discourse in the science sector, which indicates a joint advocacy for citizen science.

How to Cite: Hecker, S., Wicke, N., Haklay, M. and Bonn, A., 2019. How Does Policy Conceptualise Citizen Science? A Qualitative Content Analysis of International Policy Documents. Citizen Science: Theory and Practice, 4(1), p.32. DOI: http://doi.org/10.5334/cstp.230
141
Views
36
Downloads
25
Twitter
  Published on 02 Dec 2019
 Accepted on 07 Jul 2019            Submitted on 29 Jan 2019

Introduction

In the last decade, citizen science (CS) has received increasing attention in policy circles, along with greater recognition among scientific institutions and the wider public. Framing the multifaceted concept of CS is the discourse about different views, values, expectations, and assumptions of different actors. While acknowledging the complex development of the CS field, a baseline of mutual understanding of what is meant by the term “citizen science” can enhance present and future CS development and boost collaboration among actors.

Syntheses of scholarly discussion have shown the diversity and evolution of CS as a research approach, e.g., in ecological and environmental science (Pocock et al. 2017). CS in practice also includes projects in the humanities, social sciences, and arts or life sciences (Hecker et al. 2018b). A current scientometric meta-analysis of CS publications by Kullenberg and Kasperowski (2016) characterised CS in the following rank order: First, as a data collection tool for biological, conservation, and ecological research; second, as a data collection tool for geospatial data; and third, as a research approach employed by the social sciences. Within epidemiology, Eitzel et al. (2017) investigated CS terminologies and found that the choice of terminology (e.g., what to call participants in a CS project) influences the inclusion or exclusion of knowledge and the (self-) perception of participants, and therefore, the whole knowledge production process. Some in the CS community have called for a common usage of the term CS to show its impacts on a broad scale, create synergies, and allow for networking (Hecker et al. 2018a), while acknowledging that there is not and cannot be one CS definition (Ceccaroni et al. 2017).

The development, acceptance, and implementation of CS relies not only on the public and scientific perception (Riesch and Potter 2014), but also is strongly impacted by policy development and its framing of CS. While the CS community may have a self-understanding of its use of CS, the policy community may have differences in the motivations, expectations, and specifications towards the outcomes and the process of CS. Note that in this paper, “policy” refers to the content-related dimensions of politics or a course of action. Policy thereby can be the basic principles by which a government is guided or the declared objectives that a government wants to achieve and preserve in the interest of a community (Business Dictionary 2018).

Importantly, CS data have been a cornerstone for natural resource management and nature conservation (e.g., McKinley et al. 2017, Hyder et al. 2015; Chandler et al. 2017; Kelly et al. 2019). CS offers the possibility for interaction not only with science, but also with policy, e.g., by “offering enhanced levels of participation in assessing (and determining) the success of EU environment policies” (European Commission 2013a, p.4). Understanding what policy means when using the term “citizen science” is therefore essential for researchers and policy advisors for communication and joint development of the approach.

In this paper we restrict our research to the term “citizen science,” because it is becoming a legal term. For example, CS is employed in the US Crowdsourcing and Citizen Science Act (US Congress 2017) and in high-level strategic documents such as the statement of the Group of Seven (G7) Science Academies towards their governments, entitled “Citizen Science in the Internet Era,” which considers CS as one of the three most pressing current scientific issues (G7 Science Academies 2019). In addition, CS is used as a binding term in funding schemes for research and innovation. See, for example, the EU Horizon 2020 programme “Science with and for Society” (SwafS) with its sub-programmes dedicated to CS, although no definition of “citizen science” is given, as well as the Austrian “Top Citizen Science” scheme, the 2016 German CS funding scheme, or the Australian “Citizen Science Grants.” As no single definition of “citizen science” is commonly accepted, we aim to provide a better understanding of how policy as a steering actor conceptualises the term.

While funding of specific activities is an important policy outcome, in this paper we do not examine the use of the term within the documents that relate directly to funding scientific projects and the way that these framings influence the evolution of CS. The main reason to avoid this aspect of policy implementation is that it will obfuscate the high-level policy goals that are used to conceptualise CS. Funding is the result of pressures from multiple actors including scientists, politicians, civil servants, civil society organisations, and individuals. The specific outcome is not a result of a linear link to wider policy goals, but the outcome of complex social and political negotiations. Moreover, CS is funded not only through research funding schemes, but also through subscription fees to charities and through activities of natural history museums. Therefore, looking at scientific public funding schemes for CS may obscure the wider justification given by policy to support and promote CS as a policy objective as provided in the policy documents.

The aim of this paper is, therefore, to clarify policy expectations through conceptualization of attributed benefits and challenges of CS and thus to distil policy’s understanding and strategic use of the term “citizen science” as an umbrella. We examine the understandings of CS within 43 international policy documents edited by governments and authorities from across the globe. Through a qualitative assessment, we investigate the following research questions:

  1. How does policy conceptualise and contextualise the term “citizen science”?
  2. What potential or evidence-based benefits are described regarding CS and for whom?
  3. What challenges are perceived for CS in the policy perspective?

We link our analysis to theory and existing concepts of CS and policy to place our results in current policy perspective contexts.

Theoretical Background: Public Participation in Policy and Science

Public participation has been discussed in both science and policy for several decades (Gregory and Miller 1998; Stilgoe et al. 2014), providing the background for current discussions of CS and its potential benefits and challenges.

In democracies, public participation is considered necessary for the acceptance of policy and to maintain the political democratic system. Public participation is alive through, for example, participation in elections or referenda, public consultations and consensus conferences, citizen advisory committees, citizen focus groups (Rowe and Frewer 2000), or the engagement in and support of civil society organisations. As a majority of citizens would like to become more involved in political processes, the development of participation instruments has increased in recent years (Norris 2002; Cain, Dalton, and Scarrow 2006). At the same time, participation in policy has legal sources. For example, the United Nations Economic Commission for Europe (UNECE) Aarhus Convention of 1998 provides EU citizens with the right to participate in environmental decision making. Because most of the decisions in this area necessitate scientific information, public access and use of environmental (scientific) information is seen as a prerequisite to meaningful participation. Subsequent directives and regulations provide possibilities for public participation; for example, the input of CS recording efforts towards EU statistics such as the European Common Bird Index or the European Grassland Butterfly Index (van Swaay et al. 2016) or the Marine Strategy Framework Directive (Nascimento et al. 2018). In another example, governments that are signatories to the Convention on Biological Diversity must report on biodiversity monitoring regularly; this opens up possibilities, but often also necessitates the need to use and engage with CS (Danielsen et al. 2018) due to paucity of in situ data.

In addition to the attention to participation in the area of environmental policy, there is growing awareness of the need for deeper engagement of the public in science policy. As Stilgoe et al. (2014) noted, this can be seen in the growing attention to the concept of Responsible Research and Innovation (RRI), which calls for a more democratic approach to decisions about the directions in which scientific developments should take (Peter et al. 2018; European Commission 2013b). Participation also is included as a key element in research agendas of Open Science and Open Innovation (Nascimento et al. 2018; Groom, Weatherdon, and Geijzendorffer 2016). Even though there are parallels, a central difference between political and scientific participation should be noticed: The discussions on political participation usually concern the extent of participation, because participation is essential for a democracy (van Deth 2009). In the field of participation in science, however, the fundamental questions still need to be asked as to whether, how much, and what kind of participation science needs.

Although those questions remain open, in the field of practical science communication newly developed communication forms often aim to facilitate participation and dialogue in the sense of a Public Engagement with Science (Durant 1999; Irwin and Wynne 1996). The main aim of this model is to create an equal, open, and unbiased exchange between science and the public. It replaces the idea of the so-called “deficit model,” which assumes that the wider public has a deficit of knowledge, a lack of interest, and a cognitive deficit in scientific thinking in contrast to research experts (Brossard and Lewenstein 2009; Bauer 2016). A few decades ago, as described in the Bodmer Report (Royal Society and Bodmer 1985), science communication was seen as a means to overcome information and competence deficits and to educate the public to gain some kind of scientific literacy. One-sided dissemination of scientific knowledge was seen as a solution to raise the public’s acceptance and understanding of science (Jones 2014; Miah 2017). From the 1990s onwards, this model has been criticized as too simplistic, being understood as one-to-one transfer that ignores the social processes and structural determinants of science communication and knowledge exchange (Schiele 2008). Other models have emerged in response to this critique (Brossard and Lewenstein 2009), such as the model of a Public Engagement with Science, that reflect the relationship of science and the public on a more equal footing and as a two-way exchange.

In parallel, sociological researchers began to question traditional ways of knowledge production. Gibbons et al. (1994) proclaimed that the old paradigm of knowledge production was replaced by a new “mode” of transdisciplinary, application-oriented, and socially distributed way of gaining knowledge. Around the same time, both Alan Irwin and Rick Bonney independently coined the term “citizen science,” laying the foundation for two different interpretations of the term. Irwin (1995) anchored the term in the social sciences where CS was described as a means for democratization of science, public engagement, equity, and justice in the discourse of science and in setting the research agenda. Bonney (1996) linked the term to the natural sciences and public involvement in scientific research, with members of the public partnering with professional scientists to collectively gather, submit, or analyse large quantities of data. These two strands can still be found within scientific discussions and research practice with a prevalence of CS more frequently employed and welcomed in the natural sciences (e.g. Hecker et al. 2018b; Cooper and Lewenstein 2016). The social sciences sometimes take a more critical approach (e.g., by elaborating on the various practices and genealogies that the uniformity of term CS might obscure (Strasser et al. 2018)). Nevertheless, both approaches are valued for their potential benefits.

Current inner-scientific discourse critically assesses CS and provides frameworks and conceptual work on CS and policy (e.g., Hyder et al. 2015; Hollow 2015; McKinley et al. 2017; Walters et al. 2000), whereas strategic policy papers, such as the European green and white papers (Serrano Sanz et al. 2014) mainly endorse CS, promote its implementation, and include recommendations for its development, which has informed concrete policy actions in the form of funding schemes, such as in Germany (BMBF 2017) or in Austria (e.g., Top Citizen Science funding schemes). The CS community actively advocates for the development of CS through tailored communication towards policymakers and their networks in the form of policy briefs, for example, the European Citizen Science Association (ECSA) advocated for the inclusion of CS as part of EU Policy Delivery (ECSA 2016) and endorsed the German Green Paper on Citizen Science (ECSA 2015), and the European project Doing It together Science (DITOS) advocated for synergies between CS and Open Science (DITOS 2017). The development of CS also is supported by global associations, national organisations, and capacity programmes (Göbel et al. 2017; Richter et al. 2018), and is promoted by conferences of at least three CS associations (i.e., the US Citizen Science Association, the European Citizen Science Association, and the Australian Citizen Science Association) as hubs for the development of the field (e.g., Hecker et al. 2018a). At the same time, critical voices that question CS in its current conceptualization have emerged (Mirowski 2018).

Both policy and science provide opportunities and call for public participation, which finds a melting point in the CS approach. This leads to the question whether science and policy have a different understanding of what CS is, what benefits they expect or perceive, and what challenges they see.

Methodological Approach: Qualitative Content Analysis

To explore policy’s understanding and use of the term “citizen science,” we carried out a qualitative content analysis (Mayring 2015) for 43 policy documents from the United States, Australia, New Zealand, Japan, the European Commission, Germany, Austria, the United Kingdom, and the inter-governmental Organization for Economic Cooperation on Development (OECD). We chose these countries because CS has developed in these places to a considerable degree, including initiatives from governments and capacity building (e.g., through national and international associations such as the US Citizen Science Association (CSA), the US Federal Community of Practice on Crowdsourcing and Citizen Science, European Citizen Science Association (ECSA), Australian Citizen Science Association (ACSA), and CitizenScienceAsia.

Data collection and sample

To select relevant policy documents for content analysis, we followed a triangulation approach. First, we contacted the CS network associations in Europe, the United States, and Australasia who pointed us to representatives they considered most knowledgeable to advise us on policy documents for our sample. We asked them which documents they considered as having been influential and relevant for the development of CS. Second, we searched for policy documents cited in scientific and policy papers. Inclusion criteria were that documents had to be published by governments and their ministries or government related authorities. Due to coding researchers’ language skills (Hecker, Wicke) the sample was restricted to documents in English or German. No other exclusion criteria were given, e.g., regarding policy area, publication date, or accessibility.

The final sample includes 43 policy documents (Appendix A, supplementary files) containing the term “citizen science.” The documents are mainly from two policy areas: Environmental policy (17 documents) and policy supporting science in general, including national science strategies (25 documents). An additional document is on urban policy. The 43 documents also can be clustered into the following categories: (a) reports on existing or past activities or descriptions of a current state of developments in science or meetings (14 documents); (b) strategic documents pointing towards future possible and/or intended activities (14 documents); (c) policy frameworks and programmes (3 documents); (d) information and discussion papers, including policy briefs (7 documents); and (e) single uncategorised documents, such as two memorandums, one Act, one clearance, and one letter.

Not all documents name their authors, but examining the authorships for those that do, scientific input can be identified for several of the documents: One is written by employees of the Commonwealth Scientific and Industrial Research Organisation (CSIRO), a research institution of the Australian government [document number 2, see Appendix A], and four by employees of the Joint Research Council (JRC) [18]; [19]; [20]; [21], the research service of the European Commission. One document is explicitly written by researchers [4], and two others by the Science Communication Unit University of the West of England (UWE), Bristol [14]; [15]. Document 16 is a meeting report thanking academic participants for their input to the document. Document 22 is written by the High Level Group on maximising the impact of EU Research & Innovation Programmes, a group of business and primarily science representatives, and Document 3 states that it is “based on science” [3:5]. In addition, seventeen documents refer to scientific literature in direct citations or a reference list.

Qualitative content analysis

We conducted a qualitative content analysis (Mayring 2015) of policy documents, using a deductive as well as inductive approach to structure the content. For this purpose, we developed a category system to code the material based on aspects derived from the theoretical background (see Appendix B, supplementary files, with links to relevant scientific literature). Because no similar research papers on qualitative content analysis on other science approaches from a policy perspective could be used as an orientation, we developed further categories building on patterns discovered in the assessed documents. The category system was applied using the software MaxQDA (version 18) for the coding process.

We analysed those text passages that included the exact term “citizen science” within the documents and, where applicable, considered each text paragraph before and after the total 1,130 mentions of the term focusing on the conceptualization of CS or its benefits and challenges. We treat “citizen science” as a term of art and do not consider other related terms such as “public participation in research,” “participatory research,” or “crowdsourced information.”

For quality control of the coding process, we applied a consensual coding method (Kuckartz 2016) to determine whether coders agreed on the coding content while using the same coding scheme. Two researchers (Hecker, Wicke) each coded the text independently according to scientific standards (see Lombard et al. 2004). We first compared independent codings for 10% of the same data material for similarity in segment selection and codings. Here, we used the function “intercoder agreement” of the software MaxQDA to see the distribution of codes on a segment level. Results showed between 66% and 91% of intercoder agreement. Differences were discussed and a common understanding of the coding scheme was developed. Both researchers then coded half of the entire sample each.

Results

The policy documents mention the term “citizen science” between one and 184 times per document. CS was referred to either in a broad context of other topics (Appendix A, supplementary files), e.g., in Australia’s strategy for nature 2018–2030 [12], or it was the main focus of attention of large parts or the whole document, such as in Environmental Citizen Science [15]. Therefore, documents cover a variety of categories. While most documents provide information on the conceptualisation of CS, many cover benefits while fewer cover challenges.

Conceptualisation and contextualisation of citizen science in policy documents

Understanding of citizen science

Half of the documents mention CS as a tool for data collection and analysis mainly in the field of environmental research, some of which refer back to Bonney’s initial definition from 1996 (see Table 1 for main results on conceptualization and contextualization). Only ten documents refer to Irwin’s definition of CS (1995), acknowledging the added value of societal development through CS [e.g., 15:5; 19:6 – i.e., document 19, page 6, indicating the page number of the PDF document. Please note that this might differ from the numbering within the policy document due to different numbering of sections]. More than half of the documents enlarge the understanding of CS going beyond these definitions. Some refer to CS as a tool for collaboration between the wider community and scientists working together in scientific projects, though without providing further explanation (e.g., “scientists collaborating in various forms with members of the public” [8:75]; see also [4:1; 26:1; 33:17]). Others define the forms of collaboration or different existing understandings (e.g., “from being better informed about science, to participating in the scientific process itself by observing, gathering or processing data” [37:14]), which are based on the description of tasks or activities that citizens might accomplish in a CS project [e.g., 9:4; 21:48; 23:28; 27:66; 30:1]. In this way, the documents provide a more concrete understanding of the concept. Fourteen documents emphasise the variety of approaches, forms of collaboration, or aims [e.g. 11:1; 23:28; 38:8], such as the “degrees of agency with regards to the research process, very different relationships with professional scientists and very different degrees of influence on policy relevant scientific projects” [19:7]. The term CS also is understood to encompass projects that gather data without linkage to specific scientific processes: They “are about user-generated content, not necessarily addressing a scientific process or issues” [21:3], alongside CS projects that explicitly address scientific questions.

Table 1

Main results for conceptualization and contextualization identified in international policy documents.


CS is predominantly referred to as a data collection tool (Bonney et al 1996), while few policy documents relate to CS as an approach to democratise science (Irwin 1995)
Documents embrace the variety of CS approaches and different levels of engagement
Documents provide descriptive understandings of CS through describing tasks of participants in CS activities
Policy application areas are mainly biodiversity and environment related, e.g., with reference to environmental policies or health risk management
CS is linked to Open Science and Crowdsourcing
CS is viewed as an inclusive approach to joint research bridging academia and societal actors and linked to education
Digital technologies are perceived as main driver for facilitating CS

Policy documents also use a wide concept of CS regarding the form of collaboration and level of engagement of citizens and embrace this collaboration between science and society: “The engagement of citizens in CS activities can take different forms, from crowdsourcing (citizens as sensors) to collaborative science and policy (full engagement)—all forms can bring added value to EU policies” [15:5].

Policy documents rarely relate to the classification provided in Bonney et al. (2009) and Shirk et al. (2012), which is widely cited in scientific literature, although four documents do refer to “contributory,” “collaborative,” and “community-led” [15:7] or “co-created” CS projects [4]; [15]; [19]; [21]. The call for co-creation of CS projects is mentioned as a task for researchers [15:4] but is not mentioned as an added value for policy. The historical roots of CS [15:4–6; 30:1] and the interlinkage with lay, local, and traditional knowledge [11:1; 5:9] are mentioned rarely and are lacking in most documents. Linkages to other concepts are occasionally drawn, including the typologies of Wiggins and Crowston (2012) [15:8; 37:14], Haklay’s (2013) scheme of participatory levels of CS [15:8; 21:43], or definitions provided by the White Paper on Citizen Science in Europe (Serrano Sanz et al. 2014) [21:8].

Documents link CS to other science policy concepts, such as “Open Science” and its components of open data, open access, open source, and open innovation in almost half of the documents [7:34; 8:87; 11:3; 17:26; 19:40; 19:51; 20:5; 21:7; 23:5; 27:17; 28:6; 32:35–36; 42:7]. Crowdsourcing of data is described as being part of CS or defined as distinct from CS, as both concepts advance scientific knowledge production [7:34; 9:1–2; 25:1; 28:6; 37:15; 41:4]. Single documents refer to CS alongside “Responsible Science and Innovation” [7:30; 19:51] and “Citizen Cyberscience” [15:14], or they link CS to the overall topic of the document, such as “Do-it-Yourself-science” [19; 32:56] or “Smart Cities” [21]. Policy areas for CS show a main focus on environmental policy areas (Table 2).

Table 2

Policy areas of CS in policy documents (for each document only one passage is mentioned regardless of the frequency of mentioning).

Policy area Total Source

Astronomy, e.g. asteroid detection 3 19:12, 25:10, 27:20
Biodiversity assessment, management and strategies 9 2:189, 4:3, 15:3, 19:50, 24:2, 25:10, 26:21, 29:4, 34:21
Environmental monitoring and reporting 4 19:11, 21:8, 39:10, 40:15
   wildlife monitoring and management 3 4:3, 15:3, 30:1
Environmental science and policies/policymaking 6 4:1, 11:2, 15:3, 20:5, 36:2, 37:14
   health-related 4 6:33, 13:1, 15:7, 19:8
   natural resources management 1 29:4
   biological conservation 2 15:6, 19:12
Environmental health risks management 1 9:1
   pest and disease outbreaks 5 4:3, 10:57, 19:44, 25:6, 30:1
   biosecurity, pest animals 2 2:156, 33:17
   disaster mitigation/planning 1 25:8
   hazard mapping, pollution breaches 1 19:50
   littering 3 11:1, 16:10, 20:9
   noise, air quality/pollution 5 11:3, 15:4, 20:36, 25:8, 30:1
   discovery of new species 4 4:3, 10:11, 15:3, 36:4
   invasive species 3 14:11, 15:16, 20:12
   soil health 1 15:24
Medical research 2 5:49, 25:6
   epidemiology 3 19:8, 25:8, 37:14
   biomedicine 1 28:61
   public health risks 1 9:1
Open Science, Open Data, Big data 3 20:5, 21:34, 18:46
Weather information 4 4:3, 5:49, 25:7, 30:1
Others
   Cultural heritage digital social innovation, digital government 1 20:37
   urban life 1 20:9
   consumer strategies 1 26:21
   social sciences 1 37:14
   smart cities, incl. ICT, energy and transport infrastructures 1 21:3
   geographical information and mapping, e.g. school districts 2 37:14, 25:9
   environmental justice 1 15:17

Furthermore, CS is noted as an inclusive approach to science in nine documents. They address who should be included—all parts of society, large numbers of people, everybody, or specifically disadvantaged communities—but only three refer to reasons or aims for inclusiveness: Include “diverse sources of expertise” [17:8], “bring in new ideas and insights and contribute[s] to solutions” [25:6], and “help scientists and researchers deal with the flood of data that confronts them” [32:55].

Education as one aspect of CS projects is mentioned in eight documents, partly amongst other aims of CS projects [19:50; 20:25; 21:9; 21:34; 21:42; 18:46; 15:3]. Few present educational outcomes or aims as opposed to producing scientific data [15:4]. Potential effects of education in CS projects are described as behavioural change [11:1] or empowerment of citizens [15:12]. Educational benefits can be experienced in formal and informal settings [15:14; 19:14; 19:44; 21:34], and depend on adequate time and financial resources [15:15]; their outcomes might be hard to evaluate and quantify and depend on the level and quality of engagement of participants: “While some participants set up their own experiments and posed questions that helped scientists develop new studies, these outcomes were not captured in the evaluation survey, demonstrating that the educational value of citizen science can sometimes be difficult to quantify” [15:14].

Although potential advances for innovation of CS regarding the relationship of science and the public are recently discussed in the field of science communication, policy documents mention this aspect only in three documents: As a function of CS projects, i.e., communicating science, education, or promoting scientific literacy through CS [19:11; 19:13; 21:9], and as a means to encourage participation in CS projects [43:2].

Drivers for citizen science

Modern development of (digital) technologies, including smartphone applications and the availability of the Internet, are perceived as major drivers for CS, as mentioned by 50% of all documents: “Developments to communication technology, such as recent smartphone apps, coupled with widespread access, enable almost anyone to become a citizen scientist” [14:11]. They allow for recording and mediation of real-time, accurate GPS data [19:28], potentially engaging large numbers of citizens, increasing the temporal and geographical extent of areas of CS [19:8], and enabling a diversity of CS projects [30:1]. Websites can provide information on CS projects and link to relevant sources [28:48] and are designed, for example, to develop synergies between CS and more community-based initiatives in New Zealand [35:31]. Social media are mentioned along with new technologies and the availability of the Internet [9:3; 18:46; 37:15], but are not singled out as drivers for networking, communication, or marketing particularly supporting CS. Gamification as an approach of CS is mentioned only twice [30:2; 15:4].

Five documents point to the importance of motivation of participants in CS when calling for an understanding of individual motivations [9:11; 15:21], which might be grounded in the “fun, educational, engaging” nature of CS projects [9:6], local environmental threats [15:26] that require large data sets potentially provided by CS [24:2], or other sources of motivation [19:15]. Factors for long-term participation need consideration [21:28], which includes finding motivated participants and engaged communities [21:14] as a success factor.

Our analyses showed no perceivable differences in CS approaches or understanding of CS among different parts of the world in the assessed documents. In addition to the overall analyses, we searched for differences in conceptualization of the term “citizen science” by grouping documents into those from the United States, Australasia, and Europe for this specific question. Understandings of CS were predominantly consistent throughout the groups. For benefits and challenges, we highlight in our analysis where we found differences for regions.

Benefits of CS for science, society, and policy

Overall, policy documents portray CS as an influential and promising approach (Riesch and Potter 2014). Benefits identified in 36 documents refer to individual benefits for researchers, members of society, and policy-makers, and to science, society, and policy as parts of the social system (Table 3).

Table 3

Main benefits of CS for science, society, and policy analysed in international policy documents.

Science Members of Society Policy

Science Project level
  • Increase in amount and scale of data
  • Validation of data
  • Cost-effectiveness
Science – Society Interface
  • Increase of public engagement, interest in research, and public awareness of science
  • Inclusion of diverse sources of expertise, perspectives and experiences (broad knowledge domains)
Increase of
  • understanding of science, scientific principles, and scientific challenges (scientific literacy)
  • individual learning
  • topical knowledge
  • interest in a scientific career
Improvement of
  • policy decision-making processes
  • implementation of policy
Increase of
  • knowledge about policy issues (political literacy)
  • societal relevance of policy
  • interest in policy decision-making as well as the acceptance of policy measures
Support of (environmental) stewardship and activism (civic empowerment)

Benefits for science

Benefits of CS for science are referred to at (a) the project level and (b) the science-society interface.

Project level

The creation of large-scale data sets both in terms of geographical spread and over time and the increase of different data types are the two factors most cited as benefits of CS for science. This encompasses the possibilities of data collection by volunteers over a larger spatial extent and temporal scale, reaching a wider geographic coverage as well as “new or greater access to resources, access to private lands and information” [4:2]. This increases not only the amount of available data, but also the type of data that would otherwise be very difficult to collect [4:2; 9:1; 20:21; 29:3; 40:15; 30:1], such as monitoring of alien species in private surroundings. Volunteers also can be helpful regarding the analyses, validation, and interpretation of data, such as validation of images submitted by other volunteers [20:25; 31:4]: “the human eye and brain allow volunteers to categorize millions of objects (like galaxies) or find solutions to complex problems that computer algorithms may not be able to solve” [25:6].

The support of citizens also may improve data quality by helping “to gain more timely information or to cover a specific topic more deeply” [20:21]. This can be the case in emergency situations, when fast data are needed. These data can improve the understanding of and conversation about specific topics, such as biodiversity, and can be used to inform biodiversity and ecosystem management decisions, for example [15:7; 29:5].

Some policy documents assume a cost-effectiveness of CS, underlining that data are provided at lower cost compared with traditional research methods, and thus the financial cost of scientific studies is reduced, “potentially offering better value for public money” [11:1; 14:10, 14:18; 29:3; 15:16; 30:3]; this argument is discussed in the literature but in some circumstances has been contradicted by CS project leaders (Blaney et al. 2016; Hecker et al. 2018b).

Science-society interface

In general, CS is attributed to increase public engagement and interest in research of citizens [1:19]. CS can provide a means for raising public awareness for scientific issues and research [4:4; 15:5]. Research agendas can be redirected towards issues of concern to citizens [32:56]. Documents assert added value to science through CS by providing volunteers’ perspectives, experience, or information that professional scientists may not have [25:6; 32:56]. The skills, dedication, and ingenuity of citizens are described as “untapped resources” [25:6] that bring in new insights and contribute to solutions. New research methods can arise and new discoveries can be made possible, which lead to the production of new scientific knowledge [42:7]. CS is described as a form of distributed intelligence, which allows utilising citizens’ knowledge and monitoring capabilities [14:7]—a benefit that is slightly more emphasized in European and US than in Australasian documents. Furthermore, documents underline the value of practical know-how as well as local and traditional, specialist knowledge, for example of particular species, habitats, and skills, particularly in relation to local environments [15:8; 20:9; 7:31]. Thus, CS can improve scientific research by the inclusion of diverse sources of expertise and knowledge domains [17:8].

Benefits for members of society

Benefits for members of society are mainly formulated for individuals rather than for society as a whole. The policy documents take up the ascribed notion that CS promotes citizens’ scientific literacy (Durant 1993). They state that through this first-hand experience, scientific literacy can be built and improved, which in turn can boost participation in science projects [4:4]. Citizens learn to think scientifically and develop critical abilities. Their ability to generate scientific questions (often about the environment) and their knowledge about science as well as scientific methods and processes rises [19:14; 25:10], an argument that also is discussed critically in the scientific literature (Bonney et al. 2009; Crall et al. 2013; Cronje et al. 2011).

CS also is said to enhance individual learning. Educational benefits could be experiences in formal education, leading to new knowledge and skills, as well as in informal learning, leading to friendships and more active lifestyles [15:14; 4:2]. CS activities not only can increase knowledge about science but also about the topic, which can change attitudes and behaviour. The documents refer especially to the environmental understanding and engagement of individuals and communities [40:15]. A critical perspective is offered by only one document regarding learning effects: “If participants are only involved in simple activities, such as taking measurements or recording observations, they are likely to have few opportunities to learn” [15:15].

Two documents mention a growing interest in following a scientific career due to participation in CS. Valuable skills can be gained and young people can be motivated to aspire to a career in science [25:11]. On an overarching level, policies often assert that participating in a CS project will foster the understanding of science, scientific principles, and scientific challenges in the public [4:2; 15:14; 25:5].

Benefits for policy

CS is attributed to increase interest and benefits in policy in various ways. The approach is attributed to “help activating citizens, thrive democratic change, increase transparency and trust, and counter-play populism and post-truth politics” [11:1].

Fifteen policy documents state that CS improves policy decision-making processes. Policy makers may consult with the public and consider public contributions through CS [20:28]. CS also can serve to inform policy [4:2, 34:62]. Local and tacit knowledge provided through CS also may augment the creation of an evidence base to support regulatory implementation and compliance [15:16; 20:9]. Citizens thereby can help to guide the agenda for research and policy debate [15:3], increasing the societal relevance of research and policy. Moreover, as one document indicates, citizens’ interest in policy decision-making as well as the acceptance of policy measures could become greater, because they may trust the data more as the data are provided by themselves [20:25]. Another positive implication could be faster uptake of the evidence base and policy measures, i.e., the policy “may be better and faster implemented at local level because of the citizens’ interest” [20:31]. On the other hand, citizenship could be strengthened and members of society could become even more active if it is noted that those contributions count [20:30].

Policy can apply scientific data either directly, to support action planning, or indirectly, for example through highlighting impacts of environmental change [15:16]. Policy actors could get accurate pictures of current developments and conditions and thus identify need for action [30:2]. Moreover, the effectiveness of ongoing work may be assessed and future efforts directed [15:7].

Almost half of the documents are located in the context of environmental policy, therefore, a lot of benefits refer to concrete aspects, i.e., CS “can support council and regional natural resource management biodiversity strategies, local forest initiatives or assist in the delivery of wildlife management plans” [4:3].

Haklay (2013, 2015) emphasises the importance of geographical levels of projects for policy and introduces the category of “extreme citizen science” as a bottom-up approach addressing local needs and collaboratively finding solutions. Policy documents point to the geographical level of CS projects from local to national to international [9:1; 15:6; 16:10; 19:11; 21:19; 23:28; 25:5; 30:1; 34:17] and on the geographical distribution of CS networks and their implementation [e.g. 4:4; 8:72; 20:12; 21:23; 39:11], yet without prevalence for a specific approach to a geographical level. When it comes to benefits, policy documents mainly focus on benefits for local areas, because projects often have “a ‘direct’ interest at stake” [32:56]. Nevertheless, CS contributes to solutions of global issues such as climate change or food security, as one document mentions [32:56].

In addition, the analysed policy documents point out that CS can support (environmental) stewardship (Ballard et al. 2017). Participating in CS can encourage environmental activism, bring society closer to science and to nature, and create “a sense of ownership” [15:16]. This can have an impact on citizens’ appreciation of their natural environment and community [9:1]. This is an aspect that could be fostered, for example, through public–private partnerships and cross-sector collaborations [12:11]. Citizens should be empowered to be active stewards of nature [e.g. 3:107; 4:3; 12:11; 30:3].

Moreover, CS may strengthen democracy [e.g. 11:1; 15:12; 41:2]. It is seen in some policy documents as a participatory form that can democratise research processes as well as public policy processes [15:12]. This benefit to fostering democracy is mentioned especially in European documents, but only in four out of 24 EU documents, and thus is not a frequent argument.

Challenges for CS in policy

Challenges for CS in policy can be categorised into three main areas: (a) Data quality and management issues; (b) operational, organisational, and governance issues; and, (c) policy implementation issues (Table 4).

Table 4

Main challenges for CS in policy perspectives.

Data quality and management Organisation and governance Policy implementation

– Reliability and quality of CS data
– Re-usability of solutions
– Standardisation of data and meta-data
– Interconnection, knowledge exchange and synergies between CS projects and communities
– Access and interoperability of data
– Communication, motivation and volunteer collaboration
– Internal project standards and use of tools
– Exclusion through digital technology
– Recognition of CS by science and policy
– Evaluation of CS projects
– Uptake of CS data by policy
– Expectation management
– Participation bias
– Publication bias towards successful projects
– Temporal gaps between scientific process and policy needs
– Lag of management action on findings

Data issues

CS data issues, as described in the policy documents, mainly concern data quality assurance, data analysis, and data accessibility [15:13]; representativeness and ethical issues, such as privacy [9:4]; legal issues and related quality control [20:25] or ownership of data [15:12]; and interoperability with other data sets or policy reporting needs [18:46]. These are aspects that also are key topics in the scientific discourse about CS, which often suggest that the quality of scientific work and data may be less reliable than research and data collected by scientists who do not engage members of the public (e.g., Vantieghem et al. 2016; Kamp et al. 2016) and discuss similar aspects, such as reliability, credibility, and quality assurance (Feldman et al. 2018; Freitag et al. 2016; Fowler et al. 2013; Kosmala 2016; Wiggins 2011). Main points in the policy documents are the re-usability of solutions, such as data, tools, or apps [20:16; 21:3; 21:46; 21:50]; the interoperability of contents and platforms [20:36]; and the integration of CS data with traditional sources of information [11:2]. The standardisation of data and metadata is seen as a significant challenge [16:13; 21:46]. There is a call that information systems should be flexible and make the reuse of data possible [15:12] as well as provide possibilities for “long-term data archiving, curation, access, and re-use” [21:46]. Policy documents urge for collaboration on technical issues across networks [20:11] to create synergies while stating a lack of overview on existing tools [20:16] and fragmentation of data [21:46]. It is also mentioned that CS data should be open data [30:4], whereby an open data policy agenda needs further discussions in the CS community [21:47]. While single voices stress the fact that the use and value of CS data might be more a question of “perception of data quality, rather than the actual data quality” [15:14], others question the reliability of data as participants might not deliver data at all or may not strictly apply or adhere to protocols [20:22]. The assumption that the work of “amateur” experts may be perceived within the scientific community as substandard, unreliable, or of doubtful quality is portrayed as one of the biggest challenges for CS [4:3; 30:1; 15:4; 15:16]. This may hinder a widespread use of CS. Therefore, CS needs to gain the acceptance and the recognition of the scientific community as well as of “potential end-users of volunteer-gathered data” [15:11]. In addition, Chapman and Hodges (2017) emphasise that CS data need to be suitable not only for policy decision-making, but also available, fit-for-purpose, robust, and timely to be uptaken by policy.

As described in the documents, citizens’ role changes from information consumer to producer, which calls for dialogue between the public and administration that should be underpinned “by interoperability arrangements between the data published by public administrations and that published by the public either as individuals or as part of organised citizen science initiatives” [21:47]. Although some authorities embrace collaboration with the public, they do not have answers to many questions, such as how to handle CS data and feed them into existing information flows, or how to reconcile CS data with data from other sources [21:44]. CS data are often not easily accessible, as they are embedded in project websites [21:45], and they are sometimes not submitted to peer review and critical scientific appraisal [19:15].

Thus, policy documents call for the incorporation of data-quality assurance standards, data management, and ongoing project evaluation into CS projects [25:2; 15:13; 30:3]. Project leaders, according to some policy documents, should be equipped with the skills to manage large datasets and to use the data effectively [30:1; 30:4].

Operational, organizational, and governance issues

Operational, organisational, and governance issues are addressed twice as often as data or policy issues. They are mentioned for administration and CS projects. Overall, the need for knowledge exchange and interconnection of CS projects is expressed, for example, through information hubs and registers of projects [4:4] serving as connecting service for practitioners and researchers; or through fora “to foster a regular dialogue between policy makers, mediators and practitioners” [11:2]. Such interconnectedness would advance interoperability [11:2] and knowledge sharing “between currently disconnected communities of practice” [20:37], for example, in the form of guidelines, best practices, and methodologies [20:9]. To achieve this, collaboration across CS networks is recommended [20:36] to foster synergies and “improve the joined understanding of the value of CS for policy” [20:18].

A range of challenges is addressed for CS projects concerning communication and collaboration with participants on equal terms [19:25; 19:50], including timely and adequate feedback [21:3] and recognition of their engagement [20:22] to sustain participation [21:28] and to build and maintain trust [21:3]. Reasons that might prevent participation in CS are educational background or limited access to technology, such as digital devices as well as knowledge or affinity to use them or the inclusion of certain groups that excludes others, such as technically less-literate audiences [15:17]. Some policy documents state that project leaders should consider the overall suitability of CS as an approach to the given issue of concern; ensure proper project design, planning, and development [4:4]; and consider the incentives and motivation for participants [19:14] if they want to meet policy goals [29:4]. CS projects need flexibility to “accommodate the views and knowledge of citizens” [15:18]. Although the collection of data by citizens may reduce costs, features specific to CS projects may incur costs that might be escalating, such as attracting participants or the growth of a project in scale and number of volunteers [15:16]. Insufficient support and resources, for example for educators who want to encourage their students to participate, may pose threats for projects [15:14]. Policy documents call for training, guidance, and feedback for CS project participants in data acquisition methods [20:22], which need time and resources [15:14; 19:15], and which have to be provided in accessible formats for the targeted audience [20:22]. At the same time, when citizen science projects rely on voluntary contributions, there may be no guarantee that the data will be collected and delivered [20:22–23], and CS, therefore, may not always be reliable for supporting official reporting duties.

Concerns also relate to the aspect that the different existing and emerging communities, which deal with various thematic topics, apply specific internal standards, such as for data provision in specific formats [20:14]. Furthermore, the “high turnover of new apps” and “high development costs” [30:2] are mentioned critically. Moreover, there are concerns that due to Internet solutions, direct interactions between participants, project leaders, and policymakers may be reduced and thus the opportunities for and the quality of joint discussions diminished [15:3]. Other aspects that should be considered, according to some policy documents, especially when projects take place across larger spatial scales, are language, culture, and history of the participants [15:12; 20:22;]; for example, promotional material and training should be offered in a way that the target audience can understand [20:22].

Policy implementation

Central challenges can be characterised as reported concerns about CS. These include the lack of recognition of the value of CS by scientific communities as well as by the policy communities [e.g., 15:11; 15:14; 15:18; 19:15]. This lack of recognition might result in the lack of willingness to incorporate CS project results into policy [15:12] and calls for a better understanding of the current use of CS data as evidence for policy [15:18]. Tracing the influence of CS projects on decision-making [15:17] could be challenging, however, because policy decision making is a complex process. In addition, documents state that more study is needed about how CS data are translated into the design of new policies and, respectively, the revision of existing policies [e.g. 20:32]. Another reason for why it may be hard to assess the impact of citizen science efforts [2:189] on public policy is the relative lack of formal evaluations of CS projects. According to some policy documents, “evaluations of the monitoring of data quality and data collation are often not undertaken” [2:189]. The appropriateness of the CS approach for a specific research question should be undertaken [15:24]; changes in attitude of participants, such as “towards science and the environment, and environmental behaviours,” are “notoriously difficult to measure” [15:16]. Project evaluation is therefore suggested, such as through participation surveys [9:5].

Justification is recommended concerning why a CS approach is chosen for data collection in addition to professional data [40:4], and some call for a critical attitude towards results of CS “due to a lack of independent analysis by social scientists and a probable bias towards publishing about successful—and not unsuccessful—projects” [15:13], which is common also for other scientific projects. Policy calls for involvement of citizen scientists and that CS should take care to represent and involve all sections of society in CS projects [15:12] to avoid current participation bias in CS [15:17].

One document mentions a lack of spontaneous collaboration between authorities and CS on local concerns [15:26] and the need for sustainability not only of CS programmes and their outcomes but also in terms of “political and financial guarantees for action on findings” [15:12] of CS projects.

Moreover, “policy requirements and Citizen Science offers do not always coincide, for example, in terms of timing” [20:28]. Therefore, practicability is another challenge of implementing CS [20:28]. There also are temporal gaps between citizens’ contributions and the policy actions these contributions inform, leading to a loss of connections to the citizens [20:30]. Participants may become frustrated and demotivated by the time required to cause and implement a change in management or policy based on CS results [20:31]. Therefore, another challenging aspect is maintaining the motivation of participants’ engagement [30:4].

CS exists within a set of norms that assumes that science is the most reliable source of knowledge [19:12], while for policy decision-making, scientific evidence contributes to some but not all of the decision-making process [20:28]. Expectation management therefore may be needed when implementing CS, as both scientists and participants in CS projects may have high expectations about the policy impact of their engagement [11:2; 20:28].

General concerns are raised about whether CS is “open science” (i.e., shares data and results in a transparent way), or if it may be misused for raising funds without creating useful data and meaningful public participation [9:1; 15:17]. In addition, when applying a CS approach, there may be a risk to focus policies only on obvious issues, while not tackling more complicated or intangible issues. CS is assumed to work best on hands-on problems where many volunteers can contribute directly, while less obvious issues might not attract people to participate [20:32].

The analysed policy documents provide suggestions for different challenges, for example to more consistently use the term “citizen science” to make CS projects traceable, create synergies, and avoid duplication of efforts [15:26; 16:11]. The missing alignment of CS with current structures, programmes, and policy agendas [11:2] as well as the lack of CS data feeding into the policy processes [20:27] could be addressed by a better integration of CS in the policy cycle with the provision of guidelines [20:19] and supporting frameworks [20:14]. Legal frameworks [11:2] could address issues such as data privacy and ownership. Overall, there is a call for the creation of more synergies [16:13], such as between CS projects and smart cities initiatives [21:3].

Discussion and Conclusion

The Analysis of the 43 policy documents addressed three interrelated questions that provided insight into how policy conceptualises and contextualises the term “citizen science” and its perceived benefits and challenges. As the policy texts range from factual assessment to advocative strategic documents, the text corpus is heterogeneous. While this heterogeneity is a methodological challenge to our research, these texts represent important and relevant documents chosen by CS communities as relevant for their engagement at the science-society-policy interface. The formats of the documents also differ in their aims. A report has the main objective to assess success and achievements; a strategy is pointing to the future; and a policy brief most often includes a short assessment and may be advocative to influence current discussions or practices. In general, however, we did not find strong differences in the conceptualisation of CS between the respective formats.

Overall, given the breadth of understandings, referenced definitions, and interlinkages to other concepts, the assessed policy documents generally prove sound knowledge of existing scientific concepts of CS. They focus mostly on those definitions towards collaboration between the public and scientists, embracing the variety of CS, diverse understandings, and multiple aims. Instead of formally determining what CS is or classifying the level of collaboration (e.g., following Bonney et al. 2009; Haklay 2015; Shirk et al. 2012), they describe citizens’ diverse activities and thus provide a descriptive picture of CS collaboration. The documents reflect a variety of policy application areas. Given that 17 documents are related to environmental policy and that the CS landscape is dominated by environment-related projects and research (see Hecker et al. 2018b; Kullenberg and Kasperowski 2016; Pocock et al. 2017; Heiss and Matthes 2017), it is not surprising that they refer positively to CS as predominantly a data collection tool. Documents also relate CS to other science policy concepts, especially open science, open data, and open innovation. The documents reflect a variety of policy application areas related to CS with a focus on environmental policies, encompassing CS projects on all geographical levels with benefits especially at the local level (see Haklay 2013). Digital technologies are emphasised as drivers of CS application (see Brenton et al. 2018; Mazumdar et al. 2018).

The documents are quite enthusiastic about the benefits of CS. They promote CS as a tool to bring economic, social, scientific, and political added value to [EU] policies (Joint Research Centre 2017). CS is attributed to have great potential to shape and influence science and policy. It seems surprising though that only two documents mention the potential of CS to increase the societal relevance of policy and to raise acceptance of policy measures. Also, the assessed policy documents fail to emphasise the opportunity for policy organisations to promote dialogue through CS with those people most interested in the policy topic under consideration of CS projects, as for example Hollow et al. (2015) and Nascimento et al. (2018) point out, e.g., with reference to environmental protection agencies. Rather, CS is framed as a tool for science and as a means for collaboration between science and society, while the assessed policy documents do not perceive a clear function for policy to drive or participate in the CS processes itself. Documents point out different advantages of CS but often remain unclear about how exactly CS should be realized. Moreover, they seldom provide evidence for the diverse assumptions and do not relate to scientific findings.

The findings partially correspond to benefits that European researchers have stated towards Responsible Research and Innovation (RRI) (Smallman 2018), while emphasis differs between categories. Survey results reported by Peter et al. (2018) show that applying RRI brings enhanced visibility in the research community and supports the emergency of new research topics. The findings furthermore indicate a positive effect of RRI on both the relevance and quality of their scientific outputs. Other perceived RRI benefits, such as faster diffusion of knowledge and stimulation of innovation or empowerment of citizens, do not find equivalence in our sample of policy documents related to CS. Surprisingly, only cost-effectiveness and an increasing interest in science are mentioned as benefits in both this study and the RRI report (Peter et al. 2018).

Linking the findings of the present research to the reported CS benefits and challenges perceived by CS project managers in a comprehensive European survey of the CS landscape (Hecker et al. 2018b) shows some congruencies. When asked about the added value of CS, project coordinators across Europe (n = 147) reported as main benefits the provision of large datasets, additional expertise, and the increasing relevance of research through CS. Main challenges were seen in insufficient funding, data quality concerns, and the missing integration of CS in education. Alongside quality concerns, the lack of appreciation in science and policy is parallel to the findings in the present study, while it does not seem to be a concern in the RRI realm (Peter et al. 2018), which is valued for bringing enhanced visibility. The aspect of enhanced visibility of science through CS approaches would be a worthwhile research subject for further investigation.

Policy documents point to challenges especially regarding organisation and governance of CS projects along with technical issues and policy implementation challenges. In contrast to Mirowski (2017), however, they do not frame CS as a “threat” to the freedom of science, nor did they classify CS as a means of big business to render citizens more docile. Critical aspects, such as a possible misuse of human resources, their skills, and knowledge, were rarely mentioned and also were not reported in the European CS project manager survey (Hecker et al. 2018b). The policy documents did not explicitly address concerns regarding tensions between unpaid and paid labour of volunteers and scientists (Woodcock et al. 2017), or that CS could be inappropriately employed to deliver on obligatory governmental duties, such as on biodiversity monitoring and reporting.

Overall, we could identify that all policy documents advocate for CS. It is noteworthy that, although expected, we could not detect clear differences when comparing the documents with a regional focus between the United States, Europe, and Australasia. The policy documents provide a comprehensive, knowledgeable, and descriptive approach to what CS is and address potential benefits and challenges that mostly resonates with scientific debates about CS. Through co-authorship and consultancy of some policy documents there is at least partial influence of the scientific community through direct or indirect input. This influence could explain the high congruence between the conceptualization of CS in the assessed policy documents and the discussions of the scientific community. The boundaries between science and policy development in the field of CS are not sharp. The overall results show that policy is part of the process where CS establishes itself both as a research approach and as a field. It could be argued that the policy documents are part of the agenda-setting process for CS, and that both the scientific and the policy community contribute towards the establishment of CS.

In the scientific realm, CS is mostly discussed in a critical but positive manner with publications advocating for the development of CS (e.g., Bonn et al 2016; Serrano Sanz et al. 2014), or providing insights into the evolving field of CS with different foci (e.g., Cavalier and Kennedy 2016, Ceccaroni and Piera 2017; Hecker et al. 2018a). Critical discussions most often do not question the CS approach per se but point to challenges that need to be addressed, such as data quality (e.g., Lukyanenko et al. 2016; Kosmala et al. 2016), the increase of public understanding of science or scientific literacy (e.g., Bonney et al. 2015; Crall et al. 2013), or learning potential (e.g., Bela et al. 2016; National Academies 2018; Feldman et al. 2018). Policy documents in our analyses also are positive towards CS but mention fewer challenges than benefits. Neither scholarly articles nor policy documents reject CS as an approach to be further developed.

Outlook

Overall, policy documents all show a great interest and many expectations towards CS: “… the potential value of citizen science is high, but … this potential, particularly for citizens and policymakers, remains largely untapped” [15:3]. As CS is a relatively young field, in many cases, evidence for the impact of CS is only emerging. The CS community should therefore seek to evaluate their current CS projects for their potential for policy-related gains and in this way provide the scientific basis for current assumptions. It also might be enlightening to see how these expectations are met by policy’s steering activities. Future research could look at how CS is intended to be implemented, e.g., in the framing of CS through governmental funding schemes, and how it is perceived in the media. Not least, framing might be different in other socio-cultural contexts, e.g., in Africa or Asia with divergent communication and participation traditions.

To complement the present study on the conceptualisation of the term “citizen science” by policy, we recommend further analysis to look into how policy understands the concept of citizen science as part of a larger field of open science. Such a study might encompass the broad field of understandings and various related concepts, their similarities, overlaps, boundary points, and differentiations. Consequently, this would include closely related search terms such as “volunteer monitoring,” “crowdsourced science,” “civic science,” or “public participation in scientific research,” and thereby further enrich the current understanding of citizen science.

Next steps on the policy side could be a deeper assessment of CS challenges specific to policy and how to address them—both with respect to science and societal needs. A systematic analysis of the impacts of CS to policy and derived best practice could inform the improved integration of CS into the policy development cycle and enhanced usability of CS data for policy evaluation and implementation. A critical assessment of the effects of CS on policy design, delivery, and evaluation will enhance the development of CS as a field and its potential to offer policy support and development.

Supplementary Files

The supplementary files for this article can be found as follows:

Appendix A

Table of Policy Documents. DOI: https://doi.org/10.5334/cstp.230.s1

Appendix B

Table of Category Coding System. DOI: https://doi.org/10.5334/cstp.230.s2

Acknowledgements

The authors kindly acknowledge the support for Susanne Hecker by the German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig, funded by the German Science Foundation (FZT 118). Support to Muki Haklay was provided by the European Union’s H2020 research and innovation programme Doing It Together Science (under Grant Agreement No 709443), and by the Natural Environment Research Council (Grant Number NE/R012067/1 and NE/S017437/1).

Competing Interests

Muki Haklay is a member of the editorial board of this journal. Aletta Bonn is a guest editor for this special issue. Neither had involvement in the review process for this article other than addressing reviewer questions. All other authors have no competing interests.

References

  1. Ballard, HL, Dixon, CGH and Harris, EM. 2017. Youth-focused citizen science: Examining the role of environmental science learning and agency for conservation. Biological Conservation, 208: 65–75. DOI: https://doi.org/10.1016/j.biocon.2016.05.024 

  2. Bauer, MW. 2016. Kritische Beobachtungen zur Geschichte der Wissenschaftskommunikation. In: Bonfadelli, H, et al. (eds.), Forschungsfeld Wissenschaftskommunikation, 17–40. Wiesbaden, Germany: VS Verlag für Sozialwissenschaften. DOI: https://doi.org/10.1007/978-3-658-12898-2_2 

  3. Bela, G, Peltola, T, Young, JC, Balázs, B, Arpin, I, Pataki, G, Hauck, J, Kelemen, E, Kopperoinen, L, Van Herzele, A, Keune, H, Hecker, S, Suškevičs, M, Roy, HE, Itkonen, P, Külvik, M, László, M, Basnou, C, Pino, J and Bonn, A. 2016. Learning and the transformative potential of citizen science. Conservation Biology, 30: 990–999. DOI: https://doi.org/10.1111/cobi.12762 

  4. Blaney, R, Pocock, M and Jones, G. 2016. Citizen science and environmental monitoring: towards a methodology for evaluating opportunities, costs and benefits. Available at https://www.ceh.ac.uk/citizen-science-best-practice-guide [Last accessed 17 January 2019]. 

  5. BMBF. 2017. Federal Ministry of Education and Research, Germany. Richtlinie zur Förderung von bürgerwissenschaftlichen Vorhaben (Citizen Science). In: Research, FMoEa (eds.), Bundesanzeiger vom 01.08.2016. 

  6. Bonn, A, Richter, A, Vohland, K, Pettibone, L, Brandt, M, Feldmann, R, Goebel, C, Grefe, C, Hecker, S, Hennen, L, Hofer, H, Kiefer, S, Klotz, S, Kluttig, T, Krause, J, Küsel, K, Liedtke, C, Mahla, A, Neumeier, V, Premke-Kraus, M, Rillig, MC, Röller, O, Schäffler, L, Schmalzbauer, B, Schneidewind, U, Schumann, A, Settele, J, Tochtermann, K, Tockner, K, Vogel, J, Volkmann, W, von Unger, H, Walther, D, Weisskopf, M, Wirth, C, Witt, T, Wolst, D and Ziegler, D. 2016. Greenpaper Citizen Science Strategy 2020 for Germany. Available at https://www.buergerschaffenwissen.de/citizen-science/publikationen-ressourcen [Last accessed 27 November 2018]. 

  7. Bonney, R. 1996. Citizen science: A Lab tradition. Living Bird, 15(4): 7–15. 

  8. Bonney, R, Cooper, CB, Dickinson, J, Kelling, S, Phillips, T, Rosenberg, KV and Shirk, J. 2009. Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. Bioscience, 59(11): 977–984. DOI: https://doi.org/10.1525/bio.2009.59.11.9 

  9. Bonney, R, Phillips, TB, Ballard, HL and Enck, JW. 2015. Can citizen science enhance public understanding of science? Public Understanding of Science, 25(1): 2–16. DOI: https://doi.org/10.1177/0963662515607406 

  10. Brenton, P, von Gavel, S, Vogel, E and Marie-Elise Lecoq, M-E. 2018. Technology infrastructure for citizen science. In: Hecker, S, et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy (in press), 63–80. London: UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.12 

  11. Brossard, D and Lewenstein, BV. 2009. A Critical Appraisal of Models of Public Understanding of Science: Using Practice to Inform Theory. In: Kahlor, L and Stout, P (eds.), Communicating Science: New Agendas in Communication, 11–39. New York: Routledge. 

  12. Business Dictionary. 2018. Policy. Available at http://www.businessdictionary.com/definition/policy.html [Last accessed 15 January 2019]. 

  13. Cain, BE, Dalton, RJ and Scarrow, SE. 2006. Democracy transformed? Expanding political opportunities in advanced industrial democracies. New York: Oxford University Press. 

  14. Cavalier, D and Kennedy, EB. 2016. The Rightful Place of Science: Citizen Science. Tempe, AZ: Consortium for Science, Policy & Outcomes. 

  15. Ceccaroni, L, Bowser, A and Brenton, P. 2017. Civic education and citizen science: Definitions, categories, knowledge representation. In: Ceccaroni, L and Piera, J (eds.), Analyzing the Role of Citizen Science in Modern Research, 1–23. IGI Global. DOI: https://doi.org/10.4018/978-1-5225-0962-2.ch001 

  16. Chandler, M, See, L, Copas, K, Bonde, AM, López, BC, Danielsen, F, Legind, JK, Masinde, S, Miller-Rushing, AJ and Newman, G. 2017. Contribution of citizen science towards international biodiversity monitoring. Biological Conservation, 213: 280–294. DOI: https://doi.org/10.1016/j.biocon.2016.09.004 

  17. Chapman, C and Hodges, C. 2017. Can Citizen Science Seriously Contribute to Policy Development?: A Decision Maker’s View. In: Ceccaroni, L and Piera, J (eds.), Analyzing the Role of Citizen Science in Modern Research, 246–261. Hershey, PA, USA: IGI Global. DOI: https://doi.org/10.4018/978-1-5225-0962-2.ch012 

  18. Cooper, CB and Lewenstein, BV. 2016. Two meanings of citizen science. In: Cavalier, D and Kennedy, EB (eds.), The Rightful Place of Science: Citizen Science, 51–62. Tempe, AZ: Consortium for Science, Policy & Outcomes. 

  19. Crall, AW, Jordan, R, Holfelder, K, Newman, GJ, Graham, J and Waller, DM. 2013. The impacts of an invasive species citizen science training program on participant attitudes, behavior, and science literacy. Public Understanding of Science, 22(6): 745–764. DOI: https://doi.org/10.1177/0963662511434894 

  20. Cronje, R, Rohlinger, S, Crall, A and Newman, G. 2011. Does Participation in Citizen Science Improve Scientific Literacy? A Study to Compare Assessment Methods. Applied Environmental Education & Communication, 10(3): 135–145. DOI: https://doi.org/10.1080/1533015X.2011.603611 

  21. Danielsen, F, Burgess, ND, Coronado, I, Enghoff, M, Holt, S, Jensen, PM, Poulsen, MK and Rueda, RM 2018. The value of indigenous and local knowledge as citizen science. In: Hecker, S, et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy, 110–123. London: UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.15 

  22. DITOs consortium. 2017. Citizen Science and Open Science: Synergies and Future Areas of Work, February 2018. Available at http://www.togetherscience.eu/about/deliverables/policy-brief-on-do-it-yourself-biotechnology [Last accessed 01 October 2018]. 

  23. Durant, JR. 1993. What is scientific literacy? In: Durant, JR and Gregory, J (eds.), Science and culture in Europe, 129–137. London: Science Museum. 

  24. Durant, JR. 1999. Participatory technology assessment and the democratic model of the public understanding of science. Science and Public Policy, 26(5): 313–319. DOI: https://doi.org/10.3152/147154399781782329 

  25. ECSA. 2015. ECSA Policy Paper #1: Endorsing the German Green Paper on Citizen Science. Available at https://ecsa.citizen-science.net/documents [Last accessed 29 January 2019]. 

  26. ECSA. 2016. ECSA Policy Paper #2: Endorsing the White Paper on Citizen Science for Europe. Available at https://ecsa.citizen-science.net/documents [Last accessed 29 January 2019]. 

  27. Eitzel, MV, Cappadonna, JL, Santos-Lang, C, Duerr, RE, Virapongse, A, West, SE, Kyba, CCM, Bowser, A, Cooper, CB, Sforzi, A, Metcalfe, AN, Harris, ES, Thiel, M, Haklay, M, Ponciano, L, Roche, J, Ceccaroni, L, Shilling, FM, Dörler, D, Heigl, F, Kiessling, T, Davis, BY and Jiang, Q. 2017. Citizen Science Terminology Matters: Exploring Key Terms. Citizen Science: Theory and Practice, 2(1): 1. DOI: https://doi.org/10.5334/cstp.96 

  28. European Commission. 2013a. Commission Staff Working Document. EU Shared Environmental Information System Implementation Outlook. SWD (2013) 18 final. http://ec.europa.eu/environment/archives/seis/pdf/seis_implementation_en.pdf [Last accessed 11 September 2019] 

  29. European Commission. 2013b. Responsible Research and Innovation (RRI), Science and Technology, Special Eurobarometer 401. Available at http://ec.europa.eu/commfrontoffice/publicopinion/index.cfm/Survey/getSurveyDetail/yearFrom/1974/yearTo/2013/surveyKy/1096 [Last accessed 09 February 2018]. 

  30. Feldman, RE, Žemaitė, I and Miller-Rushing, AJ. 2018. How training citizen scientists affects the accuracy and precision of phenological data. International Journal of Biometeorology, 62(8): 1421–1435. DOI: https://doi.org/10.1007/s00484-018-1540-4 

  31. Fowler, A, Whyatt, JD, Davies, G and Ellis, R. 2013. How reliable are citizen-derived scientific data? Assessing the quality of contrail observations made by the general public. Transactions in GIS, 17(4): 488–506. DOI: https://doi.org/10.1111/tgis.12034 

  32. Freitag, A, Meyer, R and Whiteman, L. 2016. Strategies Employed by Citizen Science Programs to Increase the Credibility of Their Data. Citizen Science: Theory and Practice, 1(1): 1–11. DOI: https://doi.org/10.5334/cstp.6 

  33. G7 Science Academies. 2019. Citizen science in the Internet era. Available at: https://rsc-src.ca/en/release-2019-statements-g7-science-academies [Last accessed 07 June 2019]. 

  34. Gibbons, M, Limoges, C, Nowotny, H, Trow, M, Scott, P and Schwartzman, S. 1994. The New Production of Knowledge. London: Sage. 

  35. Göbel, C, Cappadonna, JL, Newman, GJ, Zhang, J and Vohland, K. 2017. More than just networking for citizen science: Examining core roles of practitioner organizations. In: Ceccaroni, L and Piera, J (eds.), Analyzing the role of citizen science in modern research, 24–49. Hershey, PA: IGI Global. DOI: https://doi.org/10.4018/978-1-5225-0962-2.ch002 

  36. Gregory, J and Miller, S. 1998. Science in Public, Communication, Culture and Credibility. Cambridge: Perseus. 

  37. Groom, Q, Weatherdon, L and Geijzendorffer, IR. 2016. Is citizen science an open science in the case of biodiversity observations? Journal of Applied Ecology, 54(2): 612–617. DOI: https://doi.org/10.1111/1365-2664.12767 

  38. Haklay, M. 2013. Citizen Science and Volunteered Geographic Information – overview and typology of participation. In: Sui, DZ, Elwood, S and Goodchild, MF (eds.), Crowdsourcing Geographic Knowledge, 105–122. Berlin: Springer. DOI: https://doi.org/10.1007/978-94-007-4587-2_7 

  39. Haklay, M. 2015. Citizen Science and policy: A European perspective. Washington, DC: Woodrow Wilson International Center for Scholars. 

  40. Hecker, S, Bonney, R, Haklay, M, Hölker, F, Hofer, H, Goebel, C, Gold, M, Makuch, Z, Ponti, M, Richter, A, Robinson, L, Iglesias, JR, Owen, R, Peltola, T, Sforzi, A, Shirk, J, Vogel, J, Vohland, K, Witt, T and Bonn, A. 2018a. Innovation in Citizen Science – Perspectives on Science-Policy Advances. Citizen Science: Theory and Practice, 3(1): 4. DOI: https://doi.org/10.5334/cstp.114 

  41. Hecker, S, Garbe, L and Bonn, A. 2018b. The European citizen science landscape – a snapshot. In: Hecker, S, et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy, 190–200. London: UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.20 

  42. Heiss, R and Matthes, J. 2017. Citizen Science in the Social Sciences: A Call For More Evidence. In: GAIA – Ecological Perspectives for Science and Society, 26(1): S.22–26. DOI: https://doi.org/10.14512/gaia.26.1.7 

  43. Hollow, B, Roetman, PEJ, Walter, M and Daniels, CB. 2015. Citizen science for policy development: The case of koala management in South Australia. Environmental Science & Policy, 47: 126–136. DOI: https://doi.org/10.1016/j.envsci.2014.10.007 

  44. Hyder, K, Townhill, B, Anderson, LG, Delany, J and Pinnegar, JK. 2015. Can citizen science contribute to the evidence-base that underpins marine policy? Marine policy, 59: 112–120. DOI: https://doi.org/10.1016/j.marpol.2015.04.022 

  45. Irwin, A. 1995. Citizen science: a study of people, expertise, and sustainable development. 1st ed. London: Routledge. 

  46. Irwin, A and Wynne, B. 1996. Misunderstanding Science: The public reconstruction of science and technology. Cambridge: University Press. DOI: https://doi.org/10.1017/CBO9780511563737 

  47. Joint Research Centre. 2017. Citizen Science in EU Policies – Policy Brief. 

  48. Jones, RA. 2014. Reflecting on Public Engagement and Science Policy. Public Understanding of Science, 23(1): 27–31. DOI: https://doi.org/10.1177/0963662513482614 

  49. Kamp, J, Oppel, S, Heldbjerg, H, Nyegaard, T and Donald, PF. 2016. Unstructured citizen science data fail to detect long-term population declines of common birds in Denmark. Diversity and Distributions, 22(10): 1024–1035. DOI: https://doi.org/10.1111/ddi.12463 

  50. Kelly, R, Fleming, A, Pecl, G, Richter, A and Bonn, A. 2019. Social license through citizen science: a tool for marine conservation. Ecology and Society, 24(1): 16. DOI: https://doi.org/10.5751/ES-10704-240116 

  51. Kosmala, M, Wiggins, A, Swanson, A and Simmons, B. 2016. Assessing data quality in citizen science. Frontiers in Ecology and the Environment, 14(10): 551–560. DOI: https://doi.org/10.1002/fee.1436 

  52. Kukartz, U. 2016. Qualitative Inhaltsanalyse. Methoden, Praxis, Computerunterstützung. Weinheim und Basel: Beltz Juventa. 

  53. Kullenberg, C and Kasperowski, D. 2016. What Is Citizen Science? – A Scientometric Meta-Analysis. PLoS ONE, 11(1): e0147152. DOI: https://doi.org/10.1371/journal.pone.0147152 

  54. Lukyanenko, R, Parsons, J and Wiersma, Y. 2016. Emerging problems of data quality in citizen science. Conservation Biology, 30(3): 447–449. DOI: https://doi.org/10.1111/cobi.12706 

  55. Mayring, P. 2015. Qualitative Inhaltsanalyse. Grundlagen und Techniken, 12., Neuausgabe, 12., vollständig überarbeitete und aktualisierte Aufl. Weinheim: Beltz Pädagogik. 

  56. Mazumdar, S, Ceccaroni, L, Piera, J, Hölker, F, Berre, AJ, Arlinghaus, R and Bowser, A. 2018. Citizen science technologies and new opportunities for participation. In: Hecker, S, et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy (in press), 303–320. London: UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.28 

  57. McKinley, DC, Miller-Rushing, AJ, Ballard, HL, Bonney, R, Brown, H, Cook-Patton, SC, Evans, DM, French, RA, Parrish, JK, Phillips, TB, Ryan, SF, Shanley, LA, Shirk, JL, Stepenuck, KF, Weltzin, JF, Wiggins, A, Boyle, OD, Briggs, RD, Chapin, SF, III, Hewitt, DA, Preuss, PW and Soukup, MA. 2017. Citizen science can improve conservation science, natural resource management, and environmental protection. Biological Conservation, 208: 15–28. DOI: https://doi.org/10.1016/j.biocon.2016.05.015 

  58. Miah, A. 2017. Nanoethics, Science Communication, And A Fourth Model For Public Engagement. NanoEthics, 11(2): 139–152. DOI: https://doi.org/10.1007/s11569-017-0302-9 

  59. Mirowski, P. 2017. Is ‘grassroots’ citizen science a front for big business? Available from: https://aeon.co/essays/is-grassroots-citizen-science-a-front-for-big-business [Last accessed: 07 June 2018]. 

  60. Mirowski, P. 2018. The future (s) of open science. Social studies of science, 48(2): 171–203. DOI: https://doi.org/10.1177/0306312718772086 

  61. Nascimento, S, Iglesias, JMR, Owen, R, Schade, S and Shanley, L. 2018. Citizen science for policy formulation and implementation. In: Hecker, S, et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy, 219–240. London: UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.23 

  62. National Academies of Sciences, Engineering, and Medicine. 2018. Learning Through Citizen Science: Enhancing Opportunities by Design. Washington, DC: The National Academies Press. 

  63. Norris, P. 2002. Democratic phoenix: Reinventing political activism. Cambridge, New York: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511610073 

  64. Peter, V, Maier, F, Mejlgaard, N, Bloch, C, Madsen, EB, Griessler, E, Wuketich, M, Meijer, I, Woolley, R, Lindner, R, Bührer, S, Jäger, A, Tsipouri, L and Stilgoe, J. 2018. Monitoring the evolution and benefits of responsible research and innovation. Brussels. Available at: https://publications.europa.eu/en/publication-detail/-/publication/fdd7dd10-c071-11e8-9893-01aa75ed71a1/language-en/format-PDF/source-86068597 [Last accessed 07 June 2019]. 

  65. Pocock, MJO, Tweddle, JC, Savage, J, Robinson, LD and Roy, HE. 2017. The diversity and evolution of ecological and environmental citizen science. PLoS ONE, 12(4): e0172579. DOI: https://doi.org/10.1371/journal.pone.0172579 

  66. Richter, A, Dörler, D, Hecker, S, Heigl, F, Pettibone, L, Sanz, FS, Vohland, K and Bonn, A. 2018. Benefits from capacity building in citizen science: insights from strategic development programs in Europe. In: Hecker, S, et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy, 269–283. London: UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.26 

  67. Riesch, H and Potter, C. 2014. Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions. Public understanding of science, 23(1): 107–120. DOI: https://doi.org/10.1177/0963662513497324 

  68. Rowe, G and Frewer, LJ. 2000. Public participation methods: A framework for evaluation. Science Technology & Human Values, 25: 3–29. DOI: https://doi.org/10.1177/016224390002500101 

  69. Royal Society (Great Britain) and Bodmer, WF. 1985. The public understanding of science. Report of a Royal Society ad hoc group endorsed by the Council of the Royal Society. London: Royal Society. 

  70. Schiele, B. 2008. On and about the deficit model in an age of free flow. In: Cheng, D, Claessens, M, Gascoigne, T, Metcalfe, J, Schiele, B and Shi, S (eds.), Communicating science in social contexts – New models, new practices, 91–117. Springer. 

  71. Serrano Sanz, F, Holocher-Ertl, T, Kieslinger, B, Sanz García, F and Silva, CG. 2014. White paper on citizen science for Europe. Available at http://www.socientize.eu/?q=eu/content/download-socientize-white-paper [Last accessed 27 November 2018]. 

  72. Shirk, JL, Ballard, HL, Wilderman, CC, Phillips, T, Wiggins, A, Jordan, R, McCallie, E, Minarchek, M, Lewenstein, BV, Krasny, ME and Bonney, R. 2012. Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society, 17(2): 29. DOI: https://doi.org/10.5751/ES-04705-170229 

  73. Smallman, M. 2018. Citizen Science and Responsible Research and Innovation. In: Hecker, S, et al. (eds.), Citizen Science – Innovation in Open Science, Society and Policy, 241–253. London: UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.24 

  74. Stilgoe, J, Lock, SJ and Wilsdon, J. 2014. Why should we promote public engagement with science? Public Understanding of Science, 23(1): 4–15. DOI: https://doi.org/10.1177/0963662513518154 

  75. Strasser, BJ, Baudry, J, Mahr, D, Sanchez, G and Tancoigne, E. 2018. “Citizen Science”? Rethinking Science and Public Participation. Science and Technology Studies. Available at: https://www.rri-tools.eu/-/-citizen-science-rethinking-science-and-public-participation. DOI: https://doi.org/10.23987/sts.60425 

  76. US Congress. 2017. Crowdsourcing and Citizen Science Act. 15 USC §3724. 

  77. van Deth, JW. 2009. Politische Partizipation. In: Kaina, V and Römmele, A (eds.), Politische Soziologie, 141–161. Wiesbaden: Springer Fachmedien. DOI: https://doi.org/10.1007/978-3-531-91422-0_6 

  78. van Swaay, CAM, van Strien, AJ, Aghababyan, K, Åström, S, Botham, M, Brereton, T, Carlisle, B, Chambers, P, Collins, S, Dopagne, C, Escobés, R, Feldmann, R, Fernández-García, JM, Fontaine, B, Goloshchapova, S, Gracianteparaluceta, A, Harpke, A, Heliölä, J, Khanamirian, G, Komac, B, Kühn, E, Lang, A, Leopold, P, Maes, D, Mestdagh, X, Monasterio, Y, Munguira, ML, Murray, T, Musche, M, Õunap, E, Pettersson, LB, Piqueray, J, Popoff, S, Prokofev, I, Roth, T, Roy, DB, Schmucki, R, Settele, J, Stefanescu, C, Švitra, G, Teixeira, SM, Tiitsaar, A, Verovnik, R and Warren, MS. 2016. The European Butterfly Indicator for Grassland species 1990–2015. Report VS2016.019, De Vlinderstichting, Wageningen. Available at http://nora.nerc.ac.uk/511714/. 

  79. Vantieghem, P, Maes, D, Kaiser, A and Merckx, T. 2016. Quality of citizen science data and its consequences for the conservation of skipper butterflies (Hesperiidae) in Flanders (northern Belgium). Journal of Insect Conservation, 21(3): 451–463. DOI: https://doi.org/10.1007/s10841-016-9924-4 

  80. Walters, LC, Aydelotte, J and Miller, J. 2000. Putting more public in policy analysis. Public Administration Review. 60(4): 349–359. DOI: https://doi.org/10.1111/0033-3352.00097 

  81. Wiggins, A and Crowston, K. 2012. Goals and tasks: Two typologies of citizen science projects. In: 45th Hawaii International Conference on System Science, 04.01.2012 – 07.01.2012. Wailea, HI. 3426–3435. DOI: https://doi.org/10.1109/HICSS.2012.295 

  82. Wiggins, A, Newman, G, Stevenson, RD and Crowston, K. 2011. Mechanisms for Data Quality and Validation in Citizen Science. In 2011 Seventh IEEE International Conference on e-Science Workshops, 14–19. Piscataway, NJ: IEEE. DOI: https://doi.org/10.1109/eScienceW.2011.27 

  83. Woodcock, J, Greenhill, A, Holmes, K, et al. 2017. Crowdsourcing citizen science: Exploring the tensions between paid professionals and users. Journal of Peer Production, 10. 

comments powered by Disqus