Educational context

We carried out this study in an introductory-level general science education course, FW221-Conservation of Natural Resources, with enrollment from across majors at North Carolina State University, a large, research-intensive, land-grant university in the southeastern United States. North Carolina State University has an initiative called the Citizen Science Campus program, that includes embedding citizen science experiences into campus life and providing opportunities for researchers to pilot test citizen science projects on campus.

The focal course is a 200-level, 3-credit, lecture-based class housed in the College of Natural Resources and taught every semester with a rotating instructor. FW221 is typically taught in-person, but in Fall 2020 the course was taught in a synchronous online environment because of the COVID-19 pandemic. Though topics and assignments vary each semester, the overall course learning objectives remain the same across instructors and formats, with a focus on students being able to explain, analyze, and evaluate historic and contemporary human uses, management, and stewardship of natural resources. All students are introduced to the concept of citizen science and how it contributes to the conservation of natural resources, and they are encouraged to explore opportunities for involvement.

In Spring 2019, students completed an assignment about indoor arthropod biodiversity. The assignment was required and comprised 10% of the final course grade. Part of the assignment involved creating profiles on SciStarter.org and iNaturalist and contributing at least one observation of arthropods indoors to the Never Home Alone project hosted by iNaturalist. In addition, students compared two online arthropod field guides: one prepared by a pest management company and one prepared by an ecology lab. Finally, students reflected on both parts of the assignment, pros and cons of indoor arthropod diversity, and both their own and the public’s attitudes toward indoor arthropods.

In Fall 2020, students were given three assignment options, two of which leveraged citizen science resources. The assignment was required and comprised 15% of the final course grade. The three assignment options were: (1) students conducted a guided outdoor investigation of biodiversity and invasive species topics while identifying invasive plants using Seek, an app that prepares youth for iNaturalist; (2) students investigated the topics of pollinators and ecosystem services at local, regional, and global scales by analyzing bumblebee observations submitted to the online iNaturalist database; and (3) students read and summarized an approved book related to a topic covered in the course. A final component of each option required students to share critical reflections on the assignment and how it related to course themes. Respondents among those who chose the book option were too few (n = 13) to include in our analysis.

The shift to three assignment options in 2020 was made to allow students greater flexibility and offer the ability to pursue a topic of personal interest, therefore providing more pathways for students to engage with science. The citizen science options offered were intentionally chosen to provide connections with course material and to highlight different aspects of citizen science, as defined by the Crowdsourcing and Citizen Science Act (2017), which includes activities such as making discoveries (Seek) and analyzing and interpreting data (iNaturalist).

Survey instrument

We created a 21-question Likert-scale survey (items listed in Supplemental File 1) using questions adapted primarily from DEVISE survey inventories developed by the Cornell Lab of Ornithology (Phillips et al. 2015; Porticella, Phillips, and Bonney 2017a,b). The survey included all 12 items from the DEVISE Interest in Science and Nature Scale (Phillips et al. 2015), all four items from the DEVISE Self-Efficacy for Learning and Doing Science Scale (Porticella, Phillips, and Bonney 2017b), and two items from the DEVISE Self-Efficacy for Environmental Action Scale (Porticella, Phillips, and Bonney 2017a). We added the item, “I like learning about the wildlife that lives in my community,” and adapted an item from the nature-relatedness scale (Nisbet and Zelenski 2013): “I think about how my actions impact the environment.” To assess students’ sense of obligation for taking environmental action, we created the item, “It is important for humans to protect species from extinction.” Students ranked all items on a scale of (1) Strongly Disagree to (5) Strongly Agree in 2019, and (1) Very Strongly Disagree to (7) Very Strongly Agree in 2020. 2020 responses were later recoded to match the 5-point scale from 2019 (see section entitled Data Analysis).

In Fall 2019, students were asked to provide open-ended feedback about their experience with the citizen science assignment on the post survey. In Fall 2020, when students had a choice of assignment, they were asked to indicate on the post-survey which assignment option they had chosen and why. All students in both semesters (2019 and 2020) were also asked on the pre-survey to indicate gender (female, male, non-binary, prefer not to say), college major (including a list of colleges at the university and an open-ended major response), and whether they had participated in citizen science before (yes, no, I don’t know), whereas the post-survey asked how likely they would be to participate in citizen science again (unlikely, unsure, likely).

Survey implementation

We distributed surveys using Qualtrics XM online software. In 2019, students were given time in class to complete the surveys and accessed the survey link through their SciStarter accounts. Students had three weeks to complete the assignment, with pre- and post-surveys given directly before and after. In 2020, owing to time constraints and the online course format, students were sent a direct link to the Qualtrics survey and asked to complete the survey on their own time. Because students were given more time to complete their chosen assignment to allow flexibility due to COVID-19, 2020 surveys were administered closer to two months apart (early/mid September, early/mid November), though still immediately before the assignment was introduced and after the assignment was due.

Total course enrollment in 2019 was 270 students, and there were 346 students in 2020. To assess changes over time, our analysis focused only on paired responses from students who completed both the pre- and the post-surveys. For Spring 2019, paired pre- and post-survey responses were recorded from 131 students (49% response rate); there were 115 unpaired surveys (106 pre-tests and 9 post-tests with no match) in 2019. For Fall 2020, paired pre-post responses were recorded from 78 students (23% response rate); there were 175 unpaired surveys (142 pre-tests and 33 post-tests) in 2020. After removing 13 responses from students who completed the book assignment, this resulted in a total sample of 196 paired pre-post surveys. Across both surveys and both years, all items (other than open-ended questions) had a 95% or higher completion rate for students who started the survey.

Survey population

A large proportion of respondents each year had majors in the College of Agriculture and Life Sciences (CALS) and the College of Natural Resources (CNR) (32% in 2019 and 47% in 2020), but a variety of other majors were also represented, most commonly from the College of Humanities and Social Sciences (21.5% in 2019 and 11.5% in 2020), the College of Sciences (19.2% in 2019 and 11.5% in 2020), and the College of Engineering (10.8% in 2019 and 12.8% in 2020). We classified students with majors in CALS and CNR, whose major courses of study aligned most closely with the course curriculum, as “majors,” and students in all other colleges as “non-majors.” Distribution of respondents’ majors mirrored distribution of majors among all students in the course (29% majors in 2019 and 40% in 2020). In both years, respondents were majority female (60.8% in 2019 and 53.8% in 2020). Because student genders are not recorded on course rosters, we were unable to compare respondents’ genders to overall course gender breakdown. Students’ previous exposure to citizen science was minimal. In 2019, only 14.6% of students had previously participated in citizen science and 5.4% were not sure; in 2020, 16.7% of students had done citizen science before and 17.9% were not sure.

Students who completed the Seek plant identification assignment in 2020 had the highest pre-post response rate (36%), whereas students completing the iNaturalist data analysis assignment or book assignments had lower completion rates (16% and 20%, respectively) compared with the overall course response rate (23%). In comparing student assignment choices in 2020, non-majors were evenly split between the Seek and iNaturalist options, whereas 30% more majors chose the plant identification option of Seek. To check for potential non-response, we used Welch’s Two-Sample t-tests to compare baseline interest and efficacy scores between students who completed only the pre-test and those who completed both the pre-and post-test. For all constructs in 2019, and most constructs in 2020, there were no significant baseline differences observed between those who completed only the pre-test versus those who completed both the pre- and post-tests (p > 0.1 in all cases). The only exception was interest in nature, which was lower in 2020 among students who completed only the pre-test and not the post-test [t(148.35) = 1.789, p = 0.076]. We therefore concluded that potential non-response bias in our sample was minimal.

Data analysis

Statistical analyses were performed using R Version 4.0.2 (R Core Team, 2020) in RStudio Version 1.2.1093 (RStudio Team 2020). All data were anonymized prior to analysis. Although 7-point Likert scales measuring key outcome variables were used on the 2020 survey to provide greater sensitivity than the 5-point scale in 2019, owing to smaller-than-expected sample sizes we ultimately pooled data from both years for concurrent analyses. We therefore recoded the 1:7-point Likert scales from 2020 to match the 1:5-point scales from 2019, maintaining the end-points of the bipolar scale as the most extreme ends of the spectrum (1 = 1, 2–3 = 2, 4 = 3, 5–6 = 4, 7 = 5) because extreme responses tend to be more common on bipolar scales (Moors, Kieruj, and Vermunt 2014). We reverse coded one item as noted in Supplemental File 1.

To assess factor structure of the 21 Likert-scale items, we performed principal axis factor analysis on pre-surveys and post-surveys. Factor analysis results were consistent, supporting the four hypothesized constructs of interest in nature, self-efficacy for environmental action, interest in science, and self-efficacy for learning and doing science (see Supplemental File 1 for detailed methodology and results). After confirming factor structure, we combined items and calculated a mean score for each respondent for all four constructs on both the pre- and post-tests. Because of small sample sizes, we set the alpha level to 0.10 for all analyses to avoid missing significant effects (i.e., false negatives or Type-2 errors) (Fiedler, Kutzner, and Krueger 2012).

We compared students’ baseline scores using separate analyses of variance (ANOVAs) for each construct with pre-scores as the outcome variable and year (2019 versus 2020), major category (major versus non-major), and an interaction term (year*major) as predictors (Supplemental File 2). To assess whether there were significant changes in these constructs after completing a citizen science assignment, we compared mean pre- and post-scores for each construct using paired t-tests. To investigate whether these changes varied by year and by major, we performed ANOVAs for each construct with change scores (post-test score minus pre-test score) as the outcome variable and year, major, and an interaction term as predictors. Finally, to investigate differences between assignment choices in 2020, we performed separate Welch’s Two-Sample t-tests comparing pre-scores and change scores by assignment (Seek versus iNaturalist) for each construct.

Following a sequential explanatory design (Ivankova, Creswell, and Stick 2006), we turned to qualitative analysis of open-ended survey questions to gain further insights about our quantitative findings, with the primary goal of illuminating differences between 2019 and 2020. We used inductive open coding followed by axial coding in line with grounded theory (Walker and Myrick 2006) to identify themes among the responses received on the 2019 post-survey question asking for assignment feedback and the 2020 post-survey question, “Why did you choose this assignment option?” to investigate students’ reasons for and reactions to participating in different types of citizen science activities (see Supplemental File 3: Qualitative Coding for details and code examples). Initial coding was performed independently by a member of the research team, but responses were reviewed and themes confirmed by additional members of the team prior to analysis.

Baseline interest and efficacy

Students’ average baseline scores were above the middle of the scale on all constructs but trended higher for items related to the environment (e.g., interest in nature, self-efficacy for environmental action) when compared with items related to science (interest in science, self-efficacy for learning and doing science; Table 1). For example, on the pre-test, the percentage of students scoring 4 or higher (“Agree” or “Strongly Agree”) was 54% for interest in nature, 74% for self-efficacy for environmental action, 20% for interest in science, and 36% for self-efficacy for learning and doing science. Responses also trended higher on post-scores as compared with pre-scores for all constructs. Additionally, within each of these themes, students scored higher for self-efficacy than they did for interest. Results of ANOVAs revealed that baseline scores varied significantly by major and year for all four constructs (Supplemental File 4). Mean baseline scores on all constructs were significantly higher for majors than non-majors and higher across all constructs in 2020 versus 2019.

Changes in student interest and efficacy

Paired t-tests revealed that, following citizen science experiences, students’ interest in science increased significantly [t(193) = –2.445, p = 0.015]. Interest in nature, self-efficacy for environmental action, and self-efficacy for learning and doing science revealed modest, but not significant, gains (Table 1).

Results of factorial ANOVAs exploring differences in change scores by major showed that, in general, non-majors tended to increase more than majors on all constructs, significantly so for self-efficacy for environmental action [F(1, 187) = 4.645, p = 0.032; Supplemental File 4], (Figure 1). Similarly, ANOVAs exploring differences by year revealed that students in 2020 tended to increase more than students in 2019, with significant differences in change scores for self-efficacy for environmental action [F(1, 187) = 3.078, p = 0.081] and interest in science [F(1, 190) = 4.618, p = 0.033] (Figure 2). Despite similar percentages of students with previous citizen science experience across the two years, fewer students in 2019 (38.2%) indicated they were likely to participate again in the future as compared with students in 2020 (68.8%). The interaction between major and year was significant for self-efficacy for learning and doing science [F(1, 187) = 3.710, p = 0.056], with minimal increases for both majors and non-majors in 2019, but non-majors increasing more than majors in 2020 (M_majors = –0.10, M_non-majors = +0.22). There were no significant differences in change scores by major or year for interest in nature.

Differing outcomes by assignment

In 2020, students who chose the skill-building assignment with Seek (n = 36) had higher baseline scores on all four constructs than students (n = 29) who chose the iNaturalist data analysis assignment (Table 2). In particular, students who selected Seek started with a significantly higher baseline interest in nature score [t(39.79) = 3.025, p = 0.004] than those who chose the iNaturalist assignment. However, students who chose the iNaturalist assignment saw overall larger increases in all four constructs, with a significantly higher change score for interest in nature [t(50.6) = –2.053, p = 0.045] (Table 2). This aligned with our earlier analyses of differences by major, as a higher percentage of non-majors selected the iNaturalist assignment as opposed to the Seek assignment (65% non-majors versus 50% for Seek).

Student experiences and motivations for assignment choices

Across the open-ended assignment feedback, four major themes emerged for student experiences and motivation for assignment choices (see Supplemental File 1 for code descriptions and examples). The identified themes were: (1) ease of assignment; (2) interest in the project topic; (3) assignment setting preferences; and (4) interest in the process of participation. A fifth theme that arose less often was (5) novelty of assignment. Responses related to ease of assignment included ideas of convenience, time and materials required, and technology. Interest in the project topic encompassed students with an existing affinity (or dislike) for a topic, interest in learning more about the topic, or general interest in the assignment. Most students who mentioned an assignment setting preference as their motivation for assignment selection indicated interest in getting outside. Interest in the process of participation was demonstrated through a desire to become more familiar with the app or project on which the assignment was based, as well as interest in learning more about the process of citizen science. Motivations related to assignment novelty arose with some students indicating they had done a similar assignment in other classes and preferred trying something different.

Broader outcomes of citizen science in the classroom

Our results demonstrated that students’ interest and efficacy in nature and science themes grew after participation in assignments incorporating citizen science experiences, with the largest increases observed for interest in science. Each of these variables is key to fostering the next generation’s connection to science and nature and promoting a more scientifically and ecologically literate society (Falk and Storksdieck 2010; Rosa, Profice and Collado 2018; Saribas, Kucuk, and Ertepinar 2017). Although citizen science’s potential influence on interest and efficacy has been acknowledged (Crall et al. 2013), few studies have empirically investigated the impacts of participation on these outcomes. By validating DEVISE scales developed by Phillips et al. (2015) and Porticella, Phillips, and Bonney (2017a,b) to accomplish these goals, we illustrate how these variables might be measured in future studies in both formal and informal education settings. The interest- and efficacy-related outcomes students reported on the surveys were supported by open-ended responses, which also revealed evidence of enhanced learning and engagement via citizen science experiences.

Although open-ended results provided evidence of learning about the assignment topic (Supplemental File 3), we did not formally assess student content knowledge. Incorporating citizen science into formal learning environments can generate synergies that enhance learning, but these activities might also interfere with standard course content and impede learning objectives. These interactions are not well understood, warranting further investigation (NASEM 2018). To avoid the problem of interference, researchers have emphasized the importance of making clear connections between the assignment and course objectives (Vance-Chalcraft et al. in review). In the present course, instructional scaffolding was used to intentionally link the concept of citizen science and individual assignment topics to overall course themes and learning objectives. Additionally, by contrasting participation in different types of assignments, we demonstrate the potential for different outcomes depending on the nature of the citizen science experience. This should be considered when incorporating such assignments into course curricula. For example, students who performed iNaturalist data analysis in 2020 demonstrated greater increases in their interest in science than students who simply identified plants with Seek without contributing to a citizen science project. Thus, an explicit explanation of how assignments contribute to the scientific process may be critical for instructors hoping to build science interest and efficacy.

Citizen science improves learning by course non-majors

Compared with agriculture or natural resource majors, non-majors enrolled in the course we studied started with lower baseline scores for interest and efficacy in nature and science. Although the higher baseline scores of majors may have limited their capacity to improve, the assignments seemed to generate greater benefits for non-majors, particularly increasing their self-efficacy for environmental action. Other studies have found similar results (Vitone et al. 2016; Caruso et al. 2016) and underscore the capacity of citizen science to enhance public engagement with science, even for individuals who are not predisposed toward science (Bonney et al. 2016).

Given a choice of assignments in 2020, majors were more likely to choose the hands-on Seek plant identification assignment, which aligns with majors’ pre-existing affinity for nature. That emphasis on interaction with nature might have functioned as a deterrent for non-majors seeking other assignment alternatives. However, the iNaturalist data-analysis option provided a gateway for non-majors to build their interest and efficacy for nature and science, independent of hands-on, field-based exercises. Open-ended responses reflected these patterns as well, demonstrating that while many students sought opportunities to get outside, others preferred to be indoors or simply found the data analysis topic more appealing.

Value of assignment choice

Comparisons between our assessments in 2019 and 2020 were striking and included differences in both baseline and change scores as well as students’ likelihood of participating in citizen science again in the future. These discrepancies might be explained by the difference in autonomy of choice in the assignment. Open-ended feedback from both years revealed the value of choice and the importance of providing multiple pathways for engagement with topics related to science and nature. For example, the highly varied feedback in 2019 about both the assignment topic and technological infrastructure of the project shows that different students experienced the same (required) assignment in very different ways, illustrating the value of providing choices and alternative pathways for students to engage with similar content. In 2020, different assignment options accommodated student choice and agency in topics as well as assignment format and setting. Across years and majors, students typically scored higher on nature and environment questions than general science questions. In addition, within each of these themes, students scored higher for self-efficacy versus interest in the topic, which makes sense given that some students may feel able to perform certain actions or learn certain concepts but would prefer to be doing or learning about something else. Variability of interests in nature and science underscores the importance of assignment choice when attempting to integrate citizen science into the college classroom.

These findings align with the psychological need for a sense of autonomy as a precursor to intrinsic motivation (Ryan and Deci 2000). Additionally, these findings corroborate existing literature in the education field related to the motivational benefits of student assignment choices (Dabrowski and Marshall 2018; Brooks and Young 2011). Our case study shows that, if citizen science is to be integrated into course curricula, instructors should provide multiple ways for students to engage. The motivations for engaging in citizen science are diverse (Tiago et al. 2017; Larson et al. 2020), as are students’ motivations for learning. Assignments that inspire multiple motivations are likely to be more effective pedagogical tools, as long as they are structured to align with course learning outcomes.

Limitations

Although we attribute a portion of the observed differences between scores in 2019 and 2020 to the role of student choice, other factors may also have been at play. For example, the course instructors were different and the mode of teaching differed (2019 instruction was in person and 2020 was virtual because of the COVID-19 pandemic). The nature of the citizen science experience also varied between years, with students contributing observations to a specific project in 2019, whereas in 2020, students engaged by making discoveries and analyzing and interpreting data (CCSA 2017). There remains a need for research incorporating more robust experimental design to better understand the effects of different types of citizen science experiences in formal learning environments, as well as to separate these from the overall impacts of course content. We also acknowledge that such an approach can pose ethical implications in the context of formal educational settings (Kember 2003). However, in our view, the potential benefits to student learning and engagement resulting from a better understanding of citizen science in the university classroom far outweigh the prospective challenges associated with implementing this experimental design into a course.

The smaller sample size in 2020 compared with 2019 could have influenced observed differences in scores, particularly if more engaged students were the ones completing the surveys. However, similar numbers of students completed the pre-tests in both years (237 in 2019 and 220 in 2020), and only one construct (interest in nature) demonstrated any significant post-test non-response bias in 2020. Additionally, students completing only the pre-test in 2020 still had significantly higher scores for interest in nature than students who completed both the pre- and post-tests in 2019.

Another factor that could have impacted students’ low scores in 2019 was the topic of the citizen science assignment: arthropods. Students had to search for “bugs” in their living spaces and get close enough to take pictures to submit to the Never Home Alone project. Open-ended feedback from 2019 illustrated that many students had negative feelings toward bugs and felt uncomfortable thinking about bugs living in their homes. However, many students also expressed enjoyment of or learning from the topic. These polarized perspectives underscore the importance of topics and choice when designing citizen science assignments.

Finally, similar to prior studies, we acknowledge the difficulty of separating the impacts of a citizen science assignment from overall course impacts (Vitone et al. 2016). Our hope was to investigate this in 2020 through comparisons between students completing the Seek and iNaturalist assignments versus those reading a book (i.e., a control group). Unfortunately, our low response rate, particularly for students reading a book, made this comparison impossible. Despite higher course enrollment in Fall 2020, we believe the COVID-19 pandemic and virtual nature of the course impacted students’ response rates to the survey. Although the survey was optional during both semesters, in 2019 students were asked to complete the surveys during class time, and in 2020, students were sent a link and encouraged to complete the survey online on their own time. Considering these limitations, future studies could expand the scope of our case study to consider the educational impacts of various types of citizen science experiences with larger samples of students across multiple disciplines and institutions.

Conclusion

Ample evidence points to the ability of citizen science to affect learning outcomes in informal education settings (NASEM 2018). With this case study, we have added to the small but growing literature demonstrating the influences of citizen science on student learning outcomes in formal education settings. Previous work has focused on subject area knowledge and knowledge of the scientific process (Caruso et al. 2016; Voss and Cooper 2010; Straub 2020), but our results reveal impacts on broader affective and behavioral outcomes that are key antecedents to sustained engagement and learning (Peter, Diekötter, and Kramer 2019; Phillips et al. 2018). Researchers and educators hoping to integrate citizen science into course curricula should think critically about how to design their assignments, and how the topics and structures they choose might impact different types of students (e.g., majors versus non-majors). Our results suggest that, for non-majors in particular, providing multiple pathways for engaging with environmental and science topics can enhance student learning outcomes stemming from citizen science assignments. We encourage instructors to incorporate choice into citizen science curricula in intentionally structured ways to accommodate diverse learners with different interests, ultimately fostering multiple pathways to engagement with science. Given the paucity of research on citizen science in higher education settings, further research is needed to advance understanding of the instructional contexts and attributes that maximize the potential value of citizen science as a teaching and learning tool in university classrooms.