Positionality statements

Our team comprises three experts in environmental social science: a graduate student studying conservation social science, an ecologist who studies citizen science programs, and a social scientist who studies ecological and scientific literacy. Our team has collaborated for the past three years and knows that our individual perspectives have helped shape the analysis of this study. All three of us are formally trained in ecology and have worked at some point in our lives as research assistants (or technicians) and as citizen science volunteers. Our research experience in discourse analysis varies, but we have worked collaboratively throughout the research process to ensure that we adhere to the standards of research practices.

Research setting

CitSci.org is an online platform that supports citizen science projects (Newman et al. 2012). It is a free resource and therefore helps to alleviate some of the financial difficulties that citizen science projects often face by providing cost-effective data management, documentation, and sharing (Wang et al. 2015). It has been used by more than 750 projects since it was first developed in 2007. Project managers create project pages where they provide a general overview of the project and define goals and tasks. They can also add links to other webpages that often further describe projects. Project managers can recruit and communicate with volunteers on CitSci.org. There is also a means by which data collection protocols can be developed and shared with other citizen science projects. Once a volunteer joins a project, they upload data to CitSci.org using the protocols developed by the project managers. Project managers and volunteers can also perform preliminary analyses of their data on CitSci.org. Because CitSci.org has these different functions and governance capabilities, it can support a diverse array of projects (Lynn et al. 2019) and in turn can collect metadata across projects to conduct informative meta-analyses (Newman et al. 2012). CitSci.org is an appropriate platform for such a study because it can support studies across an array of projects with different types of volunteer tasks.

Data collection

In this study we included environmental and ecological projects on CitSci.org that were active between June 2016 and June 2018. Any project that temporarily ceased activity between these dates but resumed after data were extracted for the study was excluded. This resulted in 165 projects that were included in our analysis. Of these 165 projects, some projects were created under the same umbrella organization and shared the same descriptions. In these instances, they were counted as a single project. This reduced our total number of unique project descriptions to 152. Each project on CitSci.org has a profile page on which a primary project description is provided by the creators. Project managers also have the option to provide defined metadata attributes such as goals, tasks, external website, study extent, status, privacy, sampling design, QA/QC (quality assurance/quality control), QA/QC description, and organization. Project descriptions and described metadata attributes were extracted for analysis.

Some projects include hyperlinked external websites that further describe relevant metadata attributes of the project; these were included in our analysis. Only websites that were hyperlinked in the main project descriptions or within the metadata attribute fields described above were included. Websites often contain information other than what pertains to the project, so only information about the project within the hyperlinked domain was included. Although some websites provide links to other website domains with more information on the project, these were excluded from analysis. Within the website domain, specific data collection protocol documents and data portals that require a login were also excluded. Of the 165 projects in the study, 34 had website links that met the criteria for additional inclusion. Again, the projects that shared the same website were analyzed as a single project. This resulted in a total of 23 unique external websites that were included in our analysis.

Data analysis

We used an inductive coding process to identify emergent themes, while using Bloom’s taxonomy as a sensitizing concept (Bowen 2006; Charmaz 2006). In other words, we recognized that citizen science tasks could be classified by levels of cognition using Bloom’s taxonomy; however, we did not anticipate all the tasks that would be described, nor how often they would appear in our analysis. We started by identifying potential codes for random subsets of 10% of the project descriptions. As we considered these data, it became clear that tasks for volunteers varied across levels of cognitive expectations, warranting the use of Bloom’s taxonomy to inform further coding (Bloom 1956). Project descriptions on CitSci.org and text of hyperlinked websites were analyzed through qualitative content analysis (Elo and Kyngäs 2008).

Volunteer tasks, as described by project managers, were coded by cognitive level as low-, medium-, and high-order tasks (Figure 1). High-order tasks were those that were intended to benefit or enact change on the environment (e.g., to take actions to reduce erosion or to help something). Medium-order tasks were those that intended to help volunteers understand the environment (e.g., understand or question). Finally, low-order tasks were intended to ask volunteers to describe or otherwise measure aspects of the environment (e.g., upload a photo or measure the pH of water). Some projects asked for public opinion only, rather than for any other task to be performed. We restricted our coding of each project description to a single code consisting of either low, medium, or high. If project descriptions described multiple levels of tasks, we coded them to be at their highest level of Bloom’s taxonomy. Therefore, a project description that described both low and medium tasks was coded only as medium.

Coding also distinguished between an intended task and a performed task. Often, project descriptions included related actions for which the intended task had a larger scope with fewer explicit actions. The performed task was the means by which the intended task could be accomplished; this usually included specific action steps. In other words, the volunteer was expected to accomplish the intended task by doing the more specific performed task. For example, a volunteer might help by collecting water-quality data. In this case, it is intended that the volunteer helps. The specific task that they are expected to perform to accomplish this is collecting water quality data. For our analysis, the intended task was labeled as helping, and the performed task was labeled as collecting. Although the distinctions between intended and performed tasks may at first appear to be subtle, they are important because some tasks are related to others. This coding process was necessary to distinguish between what project managers expect citizen science volunteers to accomplish (the intended task) and how they are expected to accomplish it (the performed task). This distinction is rooted in educational psychology research that considers both intended learning outcomes and performed tasks (Anderson and Krathwohl 2000).

Volunteer tasks are meant to help the project, but all projects differ in their intended outcomes. We therefore considered who or what was benefitting from the indicated volunteer tasks. Each time a new beneficiary code was identified, all projects and website materials were re-coded. Benefits identified included: no described benefit, the environment, social ecological systems, volunteer awareness or knowledge, volunteer empowerment, the citizen science project itself, the community, and the scientific community. The environment and social ecological systems were considered separately because in the citizen science literature, the focus is often on management and policy outcomes rather than on direct environmental benefits (e.g., Shirk et al. 2012; McKinley et al. 2017). Furthermore, social ecological benefits related to management (Shirk et al. 2012) differ from community benefits that can increase local social capital (Conrad and Hilchey 2011) or promote community health (Den Broeder et al. 2017). By separating the benefits to volunteer awareness/knowledge from the benefits to volunteer empowerment, we sought to tease apart the degree of participation described by volunteer tasks. This distinction is important in the environmental education literature in which empowerment through skills and meaningful participation are viewed as the highest level of environmental education, whereas increasing awareness and knowledge constitute the lowest levels (UNESCO 1978).

To ensure trustworthiness of our analysis, we engaged in iterative coding and peer and expert debriefing (Creswell and Miller 2000; Nowell et al. 2017; O’Connor and Joffe 2020). The first author developed the initial codes and codebook, and engaged in inter-rater coding with the third author, who was trained in how to use the codebook, and independently coded a randomly selected 20% of the extracted content (as suggested by O’Connor and Joffe [2020]). The initial inter-rater coding reliability was 89%, but after expert debriefing, the codebook was revised (Appendix 1), and 100% agreement was obtained between two coders. Three months after the initial round of coding took place, the first author re-coded the project descriptions and websites to ensure the reliability of the codes through intra-rater coding, and 93% agreement was found. Each discrepancy was double-checked and evaluated through peer debriefing until our team reached consensus.

Volunteer tasks

Descriptions of volunteer tasks were categorized as intended and performed and were calculated across tasks found in CitSci.org project descriptions (n = 152) and hyperlinked website materials (n = 23; Table 1; Figure 2). Low-, medium-, and high-order tasks were intended respectively in 74, 17, and 52 of the 152 projects on CitSci.org. Two projects described volunteer opinions. When performed tasks were considered, low-, medium-, and high-order tasks were found in 136, 1, and 5 projects of the 152 projects on CitSci.org. Two of the projects were coded as a solicitation of volunteers’ opinions. No intended tasks were found in seven of the projects, and no performed tasks were found in eight of the projects. One project suggested that volunteers could help the project, a high-order intended task, but did not explain how it should be done; hence the project was coded as having a high-order intended task but with no performed task.

Intended and performed tasks were also identified in websites that were hyperlinked on CitSci.org project pages. Low-, medium-, and high-order tasks were intended respectively in 5, 0, and 17 of the 23 project websites. No projects asked volunteers to record their opinions. However, 15, 0, and 7 of 23 project websites asked volunteers to perform low-, medium-, and high-order tasks, respectively. None of the websites asked for an opinion. Of the 23 hyperlinked website materials, only one had no discernable task described.

We analyzed the breakdown of intended tasks by performed tasks on CitSci.org and hyperlinked websites according to cognitive level, the solicitation of just an opinion, and no task expectation (Figure 3). On CitSci.org, all the project descriptions that were coded as intending low- or medium-order tasks were coded as performing low-order tasks. Of the 52 intended high-order tasks, 45 were coded as low-order performed tasks, 1 was coded a medium-order performed task, 5 were coded high-order performed tasks, and 1 had no performed task. The two projects that requested opinions from volunteers and the seven projects that had no intended task had alignment between intended and performed tasks.

We conducted a similar analysis on the text obtained from hyperlinked websites. All five of the project websites that intended low order tasks were also coded as having low-order performed tasks. Of the 17 websites that intended high-order tasks, 10 were coded as having volunteers perform low-order tasks and 7 were coded as having volunteers perform high-order tasks. The one project that had no intended task was also coded as having no performed task. There were no intended or performed tasks coded as medium order or asking for an opinion on the websites.

Beneficiary of the tasks

We recorded who or what benefitted from the tasks that volunteers performed for each project using one of eight codes: the citizen science project, the environment, the social ecological system, the scientific community, volunteer awareness or knowledge, volunteer empowerment, the local community, or no described benefit (Table 2; Figure 4). Of the 152 project descriptions on CitSci.org, 80 are described as benefitting the citizen science project, 25 as benefitting the environment, and 24 as benefitting social ecological systems. Benefits to the scientific community are described in 17 project descriptions. Volunteer awareness or knowledge and volunteer empowerment are described as benefits in 11 and 7 projects respectively. Benefits to communities are described in 7 projects, and 16 projects have no described benefit.

Project beneficiaries were identified and coded from hyperlinked websites. Eight of the 23 websites describe benefits to the citizen science project, eight describe benefits to the environment, and nine describe benefits to the social ecological systems. Seven websites describe the benefits of the project to the scientific community. Volunteer awareness or knowledge is described as a benefit in five projects, while three websites describe volunteer empowerment as a benefit. Two websites describe benefits to the community, and on one of the websites there are no described benefits.

Characteristics of projects with higher-order tasks

On CitSci.org, five of the 152 total projects (3%) expect volunteers to perform high-order tasks compared with seven out of 23 websites (30%). Only one project is coded as describing high-order tasks on both their CitSci.org project description and their website; this project asks volunteers to educate the public about what they learned. Our findings are corroborated by two recent studies. A survey of 77 citizen science project managers reported that volunteers were mostly data collectors who rarely engaged in stewardship or communication activities (Wiggins and Crowston 2015). Another study of volunteers from six different citizen science projects revealed that while general communication about participating in the project was common, communication about findings of the research was uncommon (Phillips et al. 2019).

There were two noteworthy characteristics across the 11 projects (4 from CitSci.org, 6 from websites, and 1 from both) with expectations that volunteers perform a higher-order task. First, projects that engage volunteers in performing high-order tasks are more likely to describe volunteers as having a more direct sphere of influence on the discussed outcome. Eight describe benefits to the environment directly, four of which also describe benefits to social ecological systems. Citizen science is often described as benefitting social ecological systems like management and policy as opposed to benefitting the environment directly (Shirk et al. 2012; McKinley et al. 2017). When social ecological benefits are described, however, volunteers are described as one step removed from their actions benefitting the environment. Their actions might benefit (support) management or policy, which then goes on to benefit the environment, but are not described as engaging in actions that directly benefit the environment. That said, benefitting the environment is a common motive for volunteers in environmental citizen science (Alender 2016; Geoghegan et al. 2016).

Furthermore, none of the 11 projects suggest benefits to the citizen science project itself, the most commonly described benefit. Benefits to the citizen science project may have an eventual impact on other outcomes, but the language used to describe volunteer tasks does not directly suggest these outcomes. When projects describe volunteers as indirectly imparting outcomes, they maintain the primary discourse related to public participation in environmental issues: green governmentality (Lassen et al. 2011). Green governmentality maintains top-down structures in that the public provides help to experts, who then go on to enact change. However, civic environmentalism is a less common discourse in which members of the public drive decision-making and enact change on their own (Bäckstrand and Lövbrand 2006). Although project managers assigning tasks for volunteers will naturally maintain top-down green governmentality, when direct benefits are described, discourse shifts toward civic environmentalism because volunteers more directly impart project outcomes.

Second, these projects describe not only higher-order tasks, but also higher-level benefits to volunteers. Four of the eleven projects describe benefits to volunteer empowerment, while none of them describe increasing volunteer awareness or knowledge. This distinction is rooted in environmental education literature. The Tbilisi Declaration was drafted at the first Intergovernmental Conference on Environmental Education to outline the role and importance of environmental education for sustainability. Increasing awareness and gaining knowledge were viewed as the lowest levels of environmental literacy. However, learning the skills to solve environmental problems and thorough participation in environmental problem solving have been, and still are, viewed as the highest level of environmental literacy (UNESCO 1978; McBride et al. 2013). Therefore, several of the projects that engage volunteers in higher-order tasks also, whether purposefully or inadvertently, describe activities that are in alignment with more holistic and comprehensive forms of environmental education.

The 11 projects that ask volunteers to perform higher-order tasks may intend to empower volunteers, yet this cannot be claimed without follow-up interviews with the project leaders. However, a cross-case analysis of five volunteer biological monitoring projects revealed that level of participation and empowerment were not connected (Lawrence 2006). Volunteers participating in contributory projects were empowered to collect data more independently and more rigorously, and participants in a contractual project were empowered to protect local economically and culturally important resources. While projects on CitSci.org that intend higher-order tasks may involve volunteers in diverse participation and use language that more directly describes benefits, we cannot speculate about whether this is purposeful or if empowerment of volunteers actually occurs. Further research is needed to understand how language may affect volunteer empowerment.

Low-order tasks in citizen science

Low-order tasks are those that ask volunteers to describe or otherwise measure the environment, often through activities such as monitoring the presence or absence of a species, collecting samples, or uploading a picture. Because each of these tasks involves collection, most of the projects on CitSci.org are considered contributory (Shirk et al. 2012). These findings are supported by other studies that indicate that citizen science is primarily contributory (e.g., Bela et al. 2016; Pocock et al. 2017) and focuses on data collection (Phillips et al. 2019). It is important to note the distinction between task complexity and task order as per Bloom’s taxonomy (Wiggins and Crowston 2015; NASEM 2018). Performed tasks may be complex (e.g., identifying hard-to-identify species or taxa, measuring transects or nested plots, and measuring and collecting verification samples for subsequent laboratory analysis) yet still be coded as low order, given that they require volunteers to describe or measure the environment rather than analyze data or otherwise critically reflect on the meanings or interpretations of results.

Previous research on contributory citizen science suggests that these projects may not increase public scientific literacy compared with those projects that engage volunteers in deeper degrees of participation (Bonney et al. 2016). In fact, a content analysis of 327 project websites revealed that if volunteer learning objectives were defined, they were usually low order (Phillips et al. 2018). This is important because environmental education research indicates that participation in low-order tasks alone is insufficient to motivate pro-environmental behaviors changes (Balgopal and Wallace 2009) that make up the highest level of scientific literacy (Bloom 1956; UNESCO 1978). Because many volunteers tend to be college educated, or even retired scientists (Geoghegan et al. 2016), they exhibit high levels of scientific literacy prior to participating (Martin 2017). There are different types of scientific literacy that can be improved upon; for example, one can have high ornithology literacy but low watershed literacy (Uno and Bybee 1994). Therefore, individuals who are generally scientifically literate can still benefit from engaging in citizen science activities. Consideration of how project design affects scientific literacy is also important because learning is another common motive for participation (Domroese and Johnson 2017).

Contributory citizen science and engagement in low-order tasks can result in positive outcomes for both volunteers and the scientific community (Shirk et al. 2012). For example, volunteers often report high satisfaction and learning as a result of participation (e.g., Trumbull et al. 2000; Wright et al. 2015). Volunteers sometimes do not want to engage in more than data collection, suggesting that they are satisfied with engaging in low-order tasks (Martin, Christidis, and Pecl 2016; Lewandowski et al. 2017; Phillips et al. 2018). Furthermore, contributory citizen science has benefited the scientific community through expanded temporal and geographic sampling scale (Cooper et al., 2007), cost-effective data collection and analysis (Dickinson et al. 2010), and peer-reviewed publications, whether explicitly mentioned (Kullenberg and Kasperowski 2016) or not (Cooper, Shirk, and Zuckerberg 2014). We, by no means, intend to undervalue the contributions of contributory citizen science or its dedicated volunteers.

We do, however, wish to push the field of citizen science to consider two critiques that can be learned from political theory and from participatory democracy. First, common discourse often serves to perpetuate current norms and values, meaning that how we communicate about a phenomenon today affects how the phenomenon will occur in the future (Young 2001). Therefore, it is possible that describing low-order tasks now might affect how future citizen science projects are planned and designed. We recognize that our analysis is limited in that we studied projects only on CitSci.org and therefore cannot generalize to the greater state of communication regarding citizen science. However, this serves as a first step toward interesting questions for future citizen science and communication researchers to consider. Second, programs—in this case, citizen science projects—should be designed with non-dominant groups in mind (Sanders 1997; Kadlec and Friedman 2007). Therefore, even if most citizen science volunteers tend to be generally scientifically literate, project design should focus on how it can benefit and be accessible to individuals with lower or different types of scientific literacy (NASEM 2018). Only then can we expand the stakeholders who benefit from engaging in citizen science.

Project managers and volunteers work together in citizen science communities of practice. There are different models to evaluate the degree to which they work together (Shirk et al. 2012). In more traditional scientific communities of practice, scientists ask and answer research questions, while technicians perform tasks that help answer the scientists’ research questions and accomplish the scientists’ goals (Shapin 1989; Doing 2004). Hence, when the project managers describe low-order tasks for volunteers to perform in contributory projects and define project benefits that may misalign with volunteer motives for participation, they are described as citizen technicians rather than citizen scientists.

Limitations

First, we did not confirm the findings of this research with project managers through interviews; we relied on content analysis only. It is possible that the citizen science projects in our analysis have volunteers perform higher-order tasks than were described. Second, although the coding in this content analysis was based on initial agreement using over 20% of the data, rather than 100%, as is suggested by Krippendorff (2004), we followed the protocols advocated by O’Connor and Joffe (2020). Third, our analysis was designed to assume that the person who created a CitSci.org project page was the project manager; however, members of the public may have developed their own collegial projects on CitSci.org, indicating that they are engaging in more parts of the scientific process than just data collection, as our results may suggest. Fourth, CitSci.org does not vet projects, meaning that there are no minimum qualifications for people to create projects. While the lack of a vetting process is appealing to newcomers and is inclusive, especially to those involved in projects with few resources, it may also result in busy, time-constrained project managers paying less attention to the ways in which they describe their projects. Fifth, project managers may spend very little time describing their projects on a platform because they have already described their projects in detail elsewhere. Although we examined external website content descriptions to avoid this bias, detailed project descriptions external to CitSci.org descriptions may still exist that we were unable to obtain. Finally, we recognize that the citizen science projects used in our analysis may not be representative of all citizen science projects.