Citizen science can engage members of the public in science through a variety of tasks including forming research questions, collecting and sharing observations, processing data, and analyzing data. Furthermore, these varied citizen science activities can occur in numerous settings and may result in a variety of outcomes for research, policy, and participants (Jordan, Ballard, and Phillips 2012; Phillips et al. 2018; Shirk et al. 2012). Participation in environmental citizen science has been shown to improve participants’ subject knowledge and understanding of the scientific process (Forrester et al. 2017; Jordan, Ballard, and Phillips 2012; Caruso et al. 2016), but can also provide additional benefits to participants and society by increasing skills, self-efficacy, and environmental science agency (Shirk et al. 2012; Overdevest, Orr, and Stepenuck 2004; Ballard, Dixon, and Harris 2017), cultivating a sense of place (Haywood, Parrish, and Dolliver 2016), promoting pro-environmental behavior changes (Lewandowski and Oberhauser 2017; Toomey and Domroese 2013), and building trust between the public and scientists (Bonney et al. 2014).
Whether facilitated by environmental educators or by science centers, or taking place at home, citizen science often occurs in informal learning environments. However, given its potential to enhance participants’ content knowledge and connections with science and the environment, citizen science also has the potential to be a valuable pedagogical tool in formal learning environments. Formal learning environments provide opportunities to scaffold learning experiences around citizen science, potentially enhancing the learning outcomes observed in more informal settings (NASEM 2018), as well as improving classroom learning outcomes. Furthermore, the diverse array of citizen science tasks, apps, and tools allow educators to leverage citizen science resources in creative ways to fit the curricular needs of a course and to meet learning objectives. For example, instructors could assign students to make direct contributions to an ongoing project via online or field-based observations, conduct novel analysis of existing citizen science data, complete training modules, and/or use training apps to prepare for future contributions to citizen science. In addition, assignments leveraging citizen science resources provide unique opportunities for formal educators to integrate authentic research experiences and hands-on learning opportunities (Oberhauser and LeBuhn 2012; Mitchell et al. 2017; Cardamone and Lobel 2016) that might bolster affective connections to course material and advance learning goals. Citizen science in formal educational settings offers a cost-effective, experiential departure from didactic-style courses, allowing students to engage in scientific reasoning while contributing to real research (Oberhauser and LeBuhn 2012).
Most research on citizen science in formal educational settings has focused on content-specific learning outcomes, including impacts on course grades, content knowledge, scientific reasoning, and understanding of the scientific process (Caruso et al. 2016; Voss and Cooper 2010; Straub 2020). However, other possible outcomes of citizen science, such as increasing connections to science and nature, are equally critical for preparing college students to be members of a scientifically literate, environmentally engaged public (Falk and Storksdieck 2010; Rosa, Profice, and Collado 2018; Saribas, Kucuk and Ertepinar 2017). Furthermore, a recent study of German secondary-school students participating in a biodiversity-themed citizen science project demonstrated that students with higher subject fascination also showed higher long-term knowledge retention (Schneiderhan-Opel and Bogner 2020), suggesting a critical connection between subject knowledge and affective factors such as interest that stimulate other learning outcomes.
Given its general public appeal, citizen science may be particularly well positioned to influence learning outcomes among students who demonstrate low pre-existing interest in the environment or who are not science majors. For example, in a study of college students in an entomology course designed for non-science majors, Vitone et al. (2016) found that participation in citizen science improved student attitudes toward science. Caruso et al. (2016) showed that non-science majors participating in citizen science as part of an introductory biology course demonstrated significantly higher course grades and critical thinking skills, as well as higher engagement than students who did not participate in the citizen science experience.
Considering all the potential learning outcomes that citizen science can generate, our study sought to understand how classroom assignments structured around nature-based citizen science resources might influence these outcomes. We therefore chose to examine the impacts of different citizen science experiences on interest and efficacy and both science and environmental literacy: students’ interest in nature and self-efficacy for environmental action, as well as interest in science and self-efficacy for learning and doing science (Falk and Storksdieck 2010; Saribas, Kucuk and Ertepinar 2017). These variables were measured in part with the DEVISE evaluation surveys, which were designed specifically to investigate broader outcomes of citizen science participation (Phillips et al. 2015; Porticella, Phillips, and Bonney 2017a,b). These validated scales were developed for and are widely used with citizen science in informal learning environments but have rarely been used in the university context.
Interest, or the perceived personal relevance of a subject, action, or cause, serves as a precursor to knowledge gain, as increased interest can promote learning and engagement (Phillips et al. 2015). Efficacy is defined as a person’s beliefs about his/her capabilities to learn specific content and to perform particular behaviors (Porticella, Phillips, and Bonney 2017a,b); it can therefore influence both learning and skill acquisition, providing a bridge between acquired knowledge and its application. For example, research on environmental literacy suggests interest and efficacy may be key precursors to pro-environmental action (Szczytko et al. 2019). Together, interest and efficacy for science and the environment are powerful predictors of learning outcomes to consider in assessments of educational interventions, especially those that contain a citizen science component (Peter, Diekötter, and Kramer 2019; Phillips et al. 2018).
Individual motivations for engaging in citizen science can also influence learning and participation outcomes (Larson et al. 2020). Citizen science assignments in the formal education sector may not achieve desired goals if participation is mandated and based solely on extrinsic motivators, such as grades. Conversely, assignments that allow for student choice promote autonomy in decision making, a critical element of Self-Determination Theory (Ryan and Deci 2000). Autonomy can fuel intrinsic motivation, which positively affects the perceived quality of engagement and learning outcomes, especially for activities related to science and the environment (Darner 2009). Therefore, we also aimed to understand how assignment structure – specifically providing students with a choice when selecting a citizen science task – might influence participation outcomes.
Here we describe a case study of learning associated with citizen science assignments in an undergraduate, natural resource-focused, general education science course. After accounting for baseline differences by student major and course year, our objectives were to: (1) determine whether citizen science experiences influenced interest in nature, self-efficacy for environmental action, interest in science, and self-efficacy for learning and doing science across student major, year, and assignment types, and (2) characterize students’ reasons for choosing different citizen science assignments and reactions to their experience.
We applied principles of the Scholarship of Teaching and Learning (SoTL) to study the integration of citizen science resources into a college classroom with the goal of enhancing students’ learning outcomes. Our study sought to balance validity, practicality, and ethical standards of teaching as promoted by SoTL principles (Gurung and Wilson 2013). Due to these considerations, our evaluation team (led by Smith and Allf) worked with the course instructors (Cooper and Pacifici) to use the existing course structure to explore outcomes of student experiences with citizen science–related assignments without placing any unnecessary additional burden on the students (Campbell and Stanley 1963). Although participation in the study was strongly encouraged, it was voluntary for all students in both years. The study was approved by the NC State University IRB, and survey participants were asked to provide informed consent prior to participation in both surveys.
We carried out this study in an introductory-level general science education course, FW221-Conservation of Natural Resources, with enrollment from across majors at North Carolina State University, a large, research-intensive, land-grant university in the southeastern United States. North Carolina State University has an initiative called the Citizen Science Campus program, that includes embedding citizen science experiences into campus life and providing opportunities for researchers to pilot test citizen science projects on campus.
The focal course is a 200-level, 3-credit, lecture-based class housed in the College of Natural Resources and taught every semester with a rotating instructor. FW221 is typically taught in-person, but in Fall 2020 the course was taught in a synchronous online environment because of the COVID-19 pandemic. Though topics and assignments vary each semester, the overall course learning objectives remain the same across instructors and formats, with a focus on students being able to explain, analyze, and evaluate historic and contemporary human uses, management, and stewardship of natural resources. All students are introduced to the concept of citizen science and how it contributes to the conservation of natural resources, and they are encouraged to explore opportunities for involvement.
In Spring 2019, students completed an assignment about indoor arthropod biodiversity. The assignment was required and comprised 10% of the final course grade. Part of the assignment involved creating profiles on SciStarter.org and iNaturalist and contributing at least one observation of arthropods indoors to the Never Home Alone project hosted by iNaturalist. In addition, students compared two online arthropod field guides: one prepared by a pest management company and one prepared by an ecology lab. Finally, students reflected on both parts of the assignment, pros and cons of indoor arthropod diversity, and both their own and the public’s attitudes toward indoor arthropods.
In Fall 2020, students were given three assignment options, two of which leveraged citizen science resources. The assignment was required and comprised 15% of the final course grade. The three assignment options were: (1) students conducted a guided outdoor investigation of biodiversity and invasive species topics while identifying invasive plants using Seek, an app that prepares youth for iNaturalist; (2) students investigated the topics of pollinators and ecosystem services at local, regional, and global scales by analyzing bumblebee observations submitted to the online iNaturalist database; and (3) students read and summarized an approved book related to a topic covered in the course. A final component of each option required students to share critical reflections on the assignment and how it related to course themes. Respondents among those who chose the book option were too few (n = 13) to include in our analysis.
The shift to three assignment options in 2020 was made to allow students greater flexibility and offer the ability to pursue a topic of personal interest, therefore providing more pathways for students to engage with science. The citizen science options offered were intentionally chosen to provide connections with course material and to highlight different aspects of citizen science, as defined by the Crowdsourcing and Citizen Science Act (2017), which includes activities such as making discoveries (Seek) and analyzing and interpreting data (iNaturalist).
We created a 21-question Likert-scale survey (items listed in Supplemental File 1) using questions adapted primarily from DEVISE survey inventories developed by the Cornell Lab of Ornithology (Phillips et al. 2015; Porticella, Phillips, and Bonney 2017a,b). The survey included all 12 items from the DEVISE Interest in Science and Nature Scale (Phillips et al. 2015), all four items from the DEVISE Self-Efficacy for Learning and Doing Science Scale (Porticella, Phillips, and Bonney 2017b), and two items from the DEVISE Self-Efficacy for Environmental Action Scale (Porticella, Phillips, and Bonney 2017a). We added the item, “I like learning about the wildlife that lives in my community,” and adapted an item from the nature-relatedness scale (Nisbet and Zelenski 2013): “I think about how my actions impact the environment.” To assess students’ sense of obligation for taking environmental action, we created the item, “It is important for humans to protect species from extinction.” Students ranked all items on a scale of (1) Strongly Disagree to (5) Strongly Agree in 2019, and (1) Very Strongly Disagree to (7) Very Strongly Agree in 2020. 2020 responses were later recoded to match the 5-point scale from 2019 (see section entitled Data Analysis).
In Fall 2019, students were asked to provide open-ended feedback about their experience with the citizen science assignment on the post survey. In Fall 2020, when students had a choice of assignment, they were asked to indicate on the post-survey which assignment option they had chosen and why. All students in both semesters (2019 and 2020) were also asked on the pre-survey to indicate gender (female, male, non-binary, prefer not to say), college major (including a list of colleges at the university and an open-ended major response), and whether they had participated in citizen science before (yes, no, I don’t know), whereas the post-survey asked how likely they would be to participate in citizen science again (unlikely, unsure, likely).
We distributed surveys using Qualtrics XM online software. In 2019, students were given time in class to complete the surveys and accessed the survey link through their SciStarter accounts. Students had three weeks to complete the assignment, with pre- and post-surveys given directly before and after. In 2020, owing to time constraints and the online course format, students were sent a direct link to the Qualtrics survey and asked to complete the survey on their own time. Because students were given more time to complete their chosen assignment to allow flexibility due to COVID-19, 2020 surveys were administered closer to two months apart (early/mid September, early/mid November), though still immediately before the assignment was introduced and after the assignment was due.
Total course enrollment in 2019 was 270 students, and there were 346 students in 2020. To assess changes over time, our analysis focused only on paired responses from students who completed both the pre- and the post-surveys. For Spring 2019, paired pre- and post-survey responses were recorded from 131 students (49% response rate); there were 115 unpaired surveys (106 pre-tests and 9 post-tests with no match) in 2019. For Fall 2020, paired pre-post responses were recorded from 78 students (23% response rate); there were 175 unpaired surveys (142 pre-tests and 33 post-tests) in 2020. After removing 13 responses from students who completed the book assignment, this resulted in a total sample of 196 paired pre-post surveys. Across both surveys and both years, all items (other than open-ended questions) had a 95% or higher completion rate for students who started the survey.
A large proportion of respondents each year had majors in the College of Agriculture and Life Sciences (CALS) and the College of Natural Resources (CNR) (32% in 2019 and 47% in 2020), but a variety of other majors were also represented, most commonly from the College of Humanities and Social Sciences (21.5% in 2019 and 11.5% in 2020), the College of Sciences (19.2% in 2019 and 11.5% in 2020), and the College of Engineering (10.8% in 2019 and 12.8% in 2020). We classified students with majors in CALS and CNR, whose major courses of study aligned most closely with the course curriculum, as “majors,” and students in all other colleges as “non-majors.” Distribution of respondents’ majors mirrored distribution of majors among all students in the course (29% majors in 2019 and 40% in 2020). In both years, respondents were majority female (60.8% in 2019 and 53.8% in 2020). Because student genders are not recorded on course rosters, we were unable to compare respondents’ genders to overall course gender breakdown. Students’ previous exposure to citizen science was minimal. In 2019, only 14.6% of students had previously participated in citizen science and 5.4% were not sure; in 2020, 16.7% of students had done citizen science before and 17.9% were not sure.
Students who completed the Seek plant identification assignment in 2020 had the highest pre-post response rate (36%), whereas students completing the iNaturalist data analysis assignment or book assignments had lower completion rates (16% and 20%, respectively) compared with the overall course response rate (23%). In comparing student assignment choices in 2020, non-majors were evenly split between the Seek and iNaturalist options, whereas 30% more majors chose the plant identification option of Seek. To check for potential non-response, we used Welch’s Two-Sample t-tests to compare baseline interest and efficacy scores between students who completed only the pre-test and those who completed both the pre-and post-test. For all constructs in 2019, and most constructs in 2020, there were no significant baseline differences observed between those who completed only the pre-test versus those who completed both the pre- and post-tests (p > 0.1 in all cases). The only exception was interest in nature, which was lower in 2020 among students who completed only the pre-test and not the post-test [t(148.35) = 1.789, p = 0.076]. We therefore concluded that potential non-response bias in our sample was minimal.
Statistical analyses were performed using R Version 4.0.2 (R Core Team, 2020) in RStudio Version 1.2.1093 (RStudio Team 2020). All data were anonymized prior to analysis. Although 7-point Likert scales measuring key outcome variables were used on the 2020 survey to provide greater sensitivity than the 5-point scale in 2019, owing to smaller-than-expected sample sizes we ultimately pooled data from both years for concurrent analyses. We therefore recoded the 1:7-point Likert scales from 2020 to match the 1:5-point scales from 2019, maintaining the end-points of the bipolar scale as the most extreme ends of the spectrum (1 = 1, 2–3 = 2, 4 = 3, 5–6 = 4, 7 = 5) because extreme responses tend to be more common on bipolar scales (Moors, Kieruj, and Vermunt 2014). We reverse coded one item as noted in Supplemental File 1.
To assess factor structure of the 21 Likert-scale items, we performed principal axis factor analysis on pre-surveys and post-surveys. Factor analysis results were consistent, supporting the four hypothesized constructs of interest in nature, self-efficacy for environmental action, interest in science, and self-efficacy for learning and doing science (see Supplemental File 1 for detailed methodology and results). After confirming factor structure, we combined items and calculated a mean score for each respondent for all four constructs on both the pre- and post-tests. Because of small sample sizes, we set the alpha level to 0.10 for all analyses to avoid missing significant effects (i.e., false negatives or Type-2 errors) (Fiedler, Kutzner, and Krueger 2012).
We compared students’ baseline scores using separate analyses of variance (ANOVAs) for each construct with pre-scores as the outcome variable and year (2019 versus 2020), major category (major versus non-major), and an interaction term (year*major) as predictors (Supplemental File 2). To assess whether there were significant changes in these constructs after completing a citizen science assignment, we compared mean pre- and post-scores for each construct using paired t-tests. To investigate whether these changes varied by year and by major, we performed ANOVAs for each construct with change scores (post-test score minus pre-test score) as the outcome variable and year, major, and an interaction term as predictors. Finally, to investigate differences between assignment choices in 2020, we performed separate Welch’s Two-Sample t-tests comparing pre-scores and change scores by assignment (Seek versus iNaturalist) for each construct.
Following a sequential explanatory design (Ivankova, Creswell, and Stick 2006), we turned to qualitative analysis of open-ended survey questions to gain further insights about our quantitative findings, with the primary goal of illuminating differences between 2019 and 2020. We used inductive open coding followed by axial coding in line with grounded theory (Walker and Myrick 2006) to identify themes among the responses received on the 2019 post-survey question asking for assignment feedback and the 2020 post-survey question, “Why did you choose this assignment option?” to investigate students’ reasons for and reactions to participating in different types of citizen science activities (see Supplemental File 3: Qualitative Coding for details and code examples). Initial coding was performed independently by a member of the research team, but responses were reviewed and themes confirmed by additional members of the team prior to analysis.
Students’ average baseline scores were above the middle of the scale on all constructs but trended higher for items related to the environment (e.g., interest in nature, self-efficacy for environmental action) when compared with items related to science (interest in science, self-efficacy for learning and doing science; Table 1). For example, on the pre-test, the percentage of students scoring 4 or higher (“Agree” or “Strongly Agree”) was 54% for interest in nature, 74% for self-efficacy for environmental action, 20% for interest in science, and 36% for self-efficacy for learning and doing science. Responses also trended higher on post-scores as compared with pre-scores for all constructs. Additionally, within each of these themes, students scored higher for self-efficacy than they did for interest. Results of ANOVAs revealed that baseline scores varied significantly by major and year for all four constructs (Supplemental File 4). Mean baseline scores on all constructs were significantly higher for majors than non-majors and higher across all constructs in 2020 versus 2019.
|CONSTRUCT||PRE-SURVEY MEAN SCORE||PRE-SURVEY SD||MEAN CHANGE SCORE (POST – PRE)||CHANGE SCORE SD||P-VALUE (PRE-POST PAIRED T-TEST)|
|Interest in nature||3.88||0.86||+0.014||0.512||0.695|
|Self-efficacy for environmental action||4.17||0.62||+0.027||0.466||0.431|
|Interest in science*||3.23||0.83||+0.094||0.536||0.015*|
|Self-efficacy for learning and doing science||3.63||0.68||+0.035||0.530||0.358|
Paired t-tests revealed that, following citizen science experiences, students’ interest in science increased significantly [t(193) = –2.445, p = 0.015]. Interest in nature, self-efficacy for environmental action, and self-efficacy for learning and doing science revealed modest, but not significant, gains (Table 1).
Results of factorial ANOVAs exploring differences in change scores by major showed that, in general, non-majors tended to increase more than majors on all constructs, significantly so for self-efficacy for environmental action [F(1, 187) = 4.645, p = 0.032; Supplemental File 4], (Figure 1). Similarly, ANOVAs exploring differences by year revealed that students in 2020 tended to increase more than students in 2019, with significant differences in change scores for self-efficacy for environmental action [F(1, 187) = 3.078, p = 0.081] and interest in science [F(1, 190) = 4.618, p = 0.033] (Figure 2). Despite similar percentages of students with previous citizen science experience across the two years, fewer students in 2019 (38.2%) indicated they were likely to participate again in the future as compared with students in 2020 (68.8%). The interaction between major and year was significant for self-efficacy for learning and doing science [F(1, 187) = 3.710, p = 0.056], with minimal increases for both majors and non-majors in 2019, but non-majors increasing more than majors in 2020 (Mmajors = –0.10, Mnon-majors = +0.22). There were no significant differences in change scores by major or year for interest in nature.
In 2020, students who chose the skill-building assignment with Seek (n = 36) had higher baseline scores on all four constructs than students (n = 29) who chose the iNaturalist data analysis assignment (Table 2). In particular, students who selected Seek started with a significantly higher baseline interest in nature score [t(39.79) = 3.025, p = 0.004] than those who chose the iNaturalist assignment. However, students who chose the iNaturalist assignment saw overall larger increases in all four constructs, with a significantly higher change score for interest in nature [t(50.6) = –2.053, p = 0.045] (Table 2). This aligned with our earlier analyses of differences by major, as a higher percentage of non-majors selected the iNaturalist assignment as opposed to the Seek assignment (65% non-majors versus 50% for Seek).
|CONSTRUCT||SEEK MEAN PRE-SCORE||INAT MEAN PRE-SCORE||P-VALUE, BASELINE DIFFERENCE||SEEK MEAN CHANGE SCORE||INAT MEAN CHANGE SCORE||P-VALUE, CHANGE SCORE DIFFERENCE|
|Interest in nature||4.60||3.96||0.004**||–0.12||+0.17||0.045*|
|Interest in science||3.59||3.25||0.168||+0.19||+0.23||0.802|
Across the open-ended assignment feedback, four major themes emerged for student experiences and motivation for assignment choices (see Supplemental File 1 for code descriptions and examples). The identified themes were: (1) ease of assignment; (2) interest in the project topic; (3) assignment setting preferences; and (4) interest in the process of participation. A fifth theme that arose less often was (5) novelty of assignment. Responses related to ease of assignment included ideas of convenience, time and materials required, and technology. Interest in the project topic encompassed students with an existing affinity (or dislike) for a topic, interest in learning more about the topic, or general interest in the assignment. Most students who mentioned an assignment setting preference as their motivation for assignment selection indicated interest in getting outside. Interest in the process of participation was demonstrated through a desire to become more familiar with the app or project on which the assignment was based, as well as interest in learning more about the process of citizen science. Motivations related to assignment novelty arose with some students indicating they had done a similar assignment in other classes and preferred trying something different.
Prior research has established links between citizen science experiences and learning outcomes in both formal educational settings and informal settings (NASEM 2018; Jordan et al. 2012; Phillips et al. 2018). Given the wide range of volunteer experiences and learning opportunities that exist, our case study highlights the potential value of citizen science in a higher education setting and reveals components of citizen science experiences that might help to propel learning outcomes such as interest and efficacy. In particular, we found that non-majors, students given choice in assignment, and students who chose to analyze data demonstrated the greatest gains in our observed constructs. These findings highlight the value of integrating choice into assignments leveraging citizen science, as well as the importance of providing different pathways for students to engage with science and environmental content, thus accommodating a diverse range of student interests and motivations.
Our results demonstrated that students’ interest and efficacy in nature and science themes grew after participation in assignments incorporating citizen science experiences, with the largest increases observed for interest in science. Each of these variables is key to fostering the next generation’s connection to science and nature and promoting a more scientifically and ecologically literate society (Falk and Storksdieck 2010; Rosa, Profice and Collado 2018; Saribas, Kucuk, and Ertepinar 2017). Although citizen science’s potential influence on interest and efficacy has been acknowledged (Crall et al. 2013), few studies have empirically investigated the impacts of participation on these outcomes. By validating DEVISE scales developed by Phillips et al. (2015) and Porticella, Phillips, and Bonney (2017a,b) to accomplish these goals, we illustrate how these variables might be measured in future studies in both formal and informal education settings. The interest- and efficacy-related outcomes students reported on the surveys were supported by open-ended responses, which also revealed evidence of enhanced learning and engagement via citizen science experiences.
Although open-ended results provided evidence of learning about the assignment topic (Supplemental File 3), we did not formally assess student content knowledge. Incorporating citizen science into formal learning environments can generate synergies that enhance learning, but these activities might also interfere with standard course content and impede learning objectives. These interactions are not well understood, warranting further investigation (NASEM 2018). To avoid the problem of interference, researchers have emphasized the importance of making clear connections between the assignment and course objectives (Vance-Chalcraft et al. in review). In the present course, instructional scaffolding was used to intentionally link the concept of citizen science and individual assignment topics to overall course themes and learning objectives. Additionally, by contrasting participation in different types of assignments, we demonstrate the potential for different outcomes depending on the nature of the citizen science experience. This should be considered when incorporating such assignments into course curricula. For example, students who performed iNaturalist data analysis in 2020 demonstrated greater increases in their interest in science than students who simply identified plants with Seek without contributing to a citizen science project. Thus, an explicit explanation of how assignments contribute to the scientific process may be critical for instructors hoping to build science interest and efficacy.
Compared with agriculture or natural resource majors, non-majors enrolled in the course we studied started with lower baseline scores for interest and efficacy in nature and science. Although the higher baseline scores of majors may have limited their capacity to improve, the assignments seemed to generate greater benefits for non-majors, particularly increasing their self-efficacy for environmental action. Other studies have found similar results (Vitone et al. 2016; Caruso et al. 2016) and underscore the capacity of citizen science to enhance public engagement with science, even for individuals who are not predisposed toward science (Bonney et al. 2016).
Given a choice of assignments in 2020, majors were more likely to choose the hands-on Seek plant identification assignment, which aligns with majors’ pre-existing affinity for nature. That emphasis on interaction with nature might have functioned as a deterrent for non-majors seeking other assignment alternatives. However, the iNaturalist data-analysis option provided a gateway for non-majors to build their interest and efficacy for nature and science, independent of hands-on, field-based exercises. Open-ended responses reflected these patterns as well, demonstrating that while many students sought opportunities to get outside, others preferred to be indoors or simply found the data analysis topic more appealing.
Comparisons between our assessments in 2019 and 2020 were striking and included differences in both baseline and change scores as well as students’ likelihood of participating in citizen science again in the future. These discrepancies might be explained by the difference in autonomy of choice in the assignment. Open-ended feedback from both years revealed the value of choice and the importance of providing multiple pathways for engagement with topics related to science and nature. For example, the highly varied feedback in 2019 about both the assignment topic and technological infrastructure of the project shows that different students experienced the same (required) assignment in very different ways, illustrating the value of providing choices and alternative pathways for students to engage with similar content. In 2020, different assignment options accommodated student choice and agency in topics as well as assignment format and setting. Across years and majors, students typically scored higher on nature and environment questions than general science questions. In addition, within each of these themes, students scored higher for self-efficacy versus interest in the topic, which makes sense given that some students may feel able to perform certain actions or learn certain concepts but would prefer to be doing or learning about something else. Variability of interests in nature and science underscores the importance of assignment choice when attempting to integrate citizen science into the college classroom.
These findings align with the psychological need for a sense of autonomy as a precursor to intrinsic motivation (Ryan and Deci 2000). Additionally, these findings corroborate existing literature in the education field related to the motivational benefits of student assignment choices (Dabrowski and Marshall 2018; Brooks and Young 2011). Our case study shows that, if citizen science is to be integrated into course curricula, instructors should provide multiple ways for students to engage. The motivations for engaging in citizen science are diverse (Tiago et al. 2017; Larson et al. 2020), as are students’ motivations for learning. Assignments that inspire multiple motivations are likely to be more effective pedagogical tools, as long as they are structured to align with course learning outcomes.
Although we attribute a portion of the observed differences between scores in 2019 and 2020 to the role of student choice, other factors may also have been at play. For example, the course instructors were different and the mode of teaching differed (2019 instruction was in person and 2020 was virtual because of the COVID-19 pandemic). The nature of the citizen science experience also varied between years, with students contributing observations to a specific project in 2019, whereas in 2020, students engaged by making discoveries and analyzing and interpreting data (CCSA 2017). There remains a need for research incorporating more robust experimental design to better understand the effects of different types of citizen science experiences in formal learning environments, as well as to separate these from the overall impacts of course content. We also acknowledge that such an approach can pose ethical implications in the context of formal educational settings (Kember 2003). However, in our view, the potential benefits to student learning and engagement resulting from a better understanding of citizen science in the university classroom far outweigh the prospective challenges associated with implementing this experimental design into a course.
The smaller sample size in 2020 compared with 2019 could have influenced observed differences in scores, particularly if more engaged students were the ones completing the surveys. However, similar numbers of students completed the pre-tests in both years (237 in 2019 and 220 in 2020), and only one construct (interest in nature) demonstrated any significant post-test non-response bias in 2020. Additionally, students completing only the pre-test in 2020 still had significantly higher scores for interest in nature than students who completed both the pre- and post-tests in 2019.
Another factor that could have impacted students’ low scores in 2019 was the topic of the citizen science assignment: arthropods. Students had to search for “bugs” in their living spaces and get close enough to take pictures to submit to the Never Home Alone project. Open-ended feedback from 2019 illustrated that many students had negative feelings toward bugs and felt uncomfortable thinking about bugs living in their homes. However, many students also expressed enjoyment of or learning from the topic. These polarized perspectives underscore the importance of topics and choice when designing citizen science assignments.
Finally, similar to prior studies, we acknowledge the difficulty of separating the impacts of a citizen science assignment from overall course impacts (Vitone et al. 2016). Our hope was to investigate this in 2020 through comparisons between students completing the Seek and iNaturalist assignments versus those reading a book (i.e., a control group). Unfortunately, our low response rate, particularly for students reading a book, made this comparison impossible. Despite higher course enrollment in Fall 2020, we believe the COVID-19 pandemic and virtual nature of the course impacted students’ response rates to the survey. Although the survey was optional during both semesters, in 2019 students were asked to complete the surveys during class time, and in 2020, students were sent a link and encouraged to complete the survey online on their own time. Considering these limitations, future studies could expand the scope of our case study to consider the educational impacts of various types of citizen science experiences with larger samples of students across multiple disciplines and institutions.
Ample evidence points to the ability of citizen science to affect learning outcomes in informal education settings (NASEM 2018). With this case study, we have added to the small but growing literature demonstrating the influences of citizen science on student learning outcomes in formal education settings. Previous work has focused on subject area knowledge and knowledge of the scientific process (Caruso et al. 2016; Voss and Cooper 2010; Straub 2020), but our results reveal impacts on broader affective and behavioral outcomes that are key antecedents to sustained engagement and learning (Peter, Diekötter, and Kramer 2019; Phillips et al. 2018). Researchers and educators hoping to integrate citizen science into course curricula should think critically about how to design their assignments, and how the topics and structures they choose might impact different types of students (e.g., majors versus non-majors). Our results suggest that, for non-majors in particular, providing multiple pathways for engaging with environmental and science topics can enhance student learning outcomes stemming from citizen science assignments. We encourage instructors to incorporate choice into citizen science curricula in intentionally structured ways to accommodate diverse learners with different interests, ultimately fostering multiple pathways to engagement with science. Given the paucity of research on citizen science in higher education settings, further research is needed to advance understanding of the instructional contexts and attributes that maximize the potential value of citizen science as a teaching and learning tool in university classrooms.
The data sets used in this study are available upon request to the corresponding author.
The supplementary files for this article can be found as follows:Supplemental File 1
Survey Design and Factor Analysis. DOI: https://doi.org/10.5334/cstp.434.s1Supplemental File 2
Mean Baseline Scores by Year and Major. DOI: https://doi.org/10.5334/cstp.434.s2Supplemental File 3
Qualitative Coding. DOI: https://doi.org/10.5334/cstp.434.s3Supplemental File 4
Sample ANOVA Output. DOI: https://doi.org/10.5334/cstp.434.s4
This research was approved and carried out under NCSU IRB Protocol #19074.
The authors would like to extend special thanks to Maria Gallardo-Williams and the NC State Office of Faculty Development’s SoTL Institute for support on this project. We would also like to acknowledge the support of National Science Foundation AISL award #1713562 and National Science Foundation RCN-UBE award #1919928. In addition, we would like to thank our anonymous reviewers for their insightful feedback in revising this manuscript.
CBC, LRL, LML, HES, BCA, and SF were partly supported in this work by National Science Foundation grant #1713562. Support for CBC also came from an NC State Office of Faculty Development 2020 SoTL Institute mini-grant.
The authors have no competing interests to declare.
Haley E. Smith: Conceptualization, Methodology, Formal analysis, Investigation, Writing—original draft, Writing—review & editing, Project administration. Bradley C. Allf: Conceptualization, Methodology, Data curation, Writing—review and editing. Lincoln R. Larson: Conceptualization, Methodology, Formal analysis, Investigation, Writing—original draft, Writing—review and editing, Supervision. Sara Futch: Formal analysis, Data curation, Writing—review and editing. Lisa Lundgren: Formal analysis, Data curation, Writing—original draft. Lara B. Pacifici: Conceptualization, Methodology. Caren B. Cooper: Conceptualization, Methodology, Formal analysis, Investigation, Writing—original draft, Writing—review and editing, Supervision, Project administration.
Ballard, HL, Dixon, CGH and Harris, EM. 2017. Youth-focused citizen science: Examining the role of environmental science learning and agency for conservation. Biological Conservation, 208: 65–75. DOI: https://doi.org/10.1016/j.biocon.2016.05.024
Bonney, R, Phillips, TB, Ballard, HL and Enck, JW. 2016. Can citizen science enhance public understanding of science? Public Understanding of Science, 25(1): 2–16. DOI: https://doi.org/10.1177/0963662515607406
Bonney, R, Shirk, JL, Phillips, TB, Wiggins, A, Ballard, HL, Miller-Rushing, AJ and Parrish, JK. 2014. Next steps for citizen science. Science, 343(6178): 1436–1437. DOI: https://doi.org/10.1126/science.1251554
Brooks, CF and Young, SL. 2011. Are choice-making opportunities needed in the classroom? Using self-determination theory to consider student motivation and learner empowerment. International Journal of Teaching and Learning in Higher.Education, 23(1): 48–59.
Campbell, DT and Stanley, J. 1963. Experimental and quasi-experimental designs for research. In: Gage, NL (ed.), Handbook of Research on Teaching, 1–84. Boston: Houghton Mifflin Company.
Cardamone, C and Lobel, L. 2016. Using citizen science to engage introductory students: From streams to the solar system. Journal of Microbiology and Biology Education, 17(1): 117–119. DOI: https://doi.org/10.1128/jmbe.v17i1.1082
Caruso, JP, Israel, N, Rowland, K, Lovelace, MJ and Saunders, MJ. 2016. Citizen science: The small world initiative improved lecture grades and California critical thinking skills test scores of nonscience major students at Florida Atlantic University. Journal of Microbiology and Biology Education, 17(1): 156–162. DOI: https://doi.org/10.1128/jmbe.v17i1.1011
Crall, AW, Jordan, R, Holfelder, K, Newman, GJ, Graham, J and Waller, DM. 2013. The impacts of an invasive species citizen science training program on participant attitudes, behavior, and science literacy. Public Understanding of Science, 22(6): 745–764. DOI: https://doi.org/10.1177/0963662511434894
Crowdsourcing and Citizen Science Act (CCSA). 2017. Public Law 114–329, Title IV, § 402.
Dabrowski, J and Marshall, TR. 2018. Motivation and engagement in student assignments: The role of choice and relevancy. Washington, DC: The Education Trust.
Darner, R. 2009. Self-determination theory as a guide to fostering environmental motivation. The Journal of Environmental Education, 40(2): 39–49. DOI: https://doi.org/10.3200/JOEE.40.2.39-49
Falk, JH and Storksdieck, M. 2010. Science learning in a leisure setting. Journal of Research in Science Teaching, 47(2): 194–212. DOI: https://doi-org.prox.lib.ncsu.edu/10.1002/tea.20319.
Fiedler, K, Kutzner, F and Krueger, JI. 2012. The long way from α-error control to validity proper: Problems with a short-sighted false-positive debate. Perspectives on Psychological Science, 7(6): 661–669. DOI: https://doi.org/10.1177/1745691612462587
Forrester, TD, Baker, M, Costello, R, Kays, R, Parsons, AW and McShea, WJ. 2017. Creating advocates for mammal conservation through citizen science. Biological Conservation, 208: 98–105. DOI: https://doi.org/10.1016/j.biocon.2016.06.025
Gurung, RAR and Wilson, JH. (eds.) 2013. Doing the scholarship of teaching and learning: Measuring systematic changes to teaching and improvements in learning. San Francisco, CA: Jossey-Bass.
Haywood, BK, Parrish, JK and Dolliver, J. 2016. Place-based and data-rich citizen science as a precursor for conservation action. Conservation Biology, 30(3): 476–486. DOI: https://doi.org/10.1111/cobi.12702
Ivankova, NV, Creswell, JW and Stick, SL. 2006. Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1): 3–20. DOI: https://doi.org/10.1177/1525822X05282260
Jordan, RC, Ballard, HL and Phillips, TB. 2012. Key issues and new approaches for evaluating citizen-science learning outcomes. Frontiers in Ecology and the Environment, 10(6): 307–309. DOI: https://doi.org/10.1890/110280
Kember, D. 2003. To control or not to control: The question of whether experimental designs are appropriate for evaluating teaching innovations in higher education. Assessment and Evaluation in Higher Education, 28(1): 89–101. DOI: https://doi.org/10.1080/02602930301684
Larson, LR, Cooper, CB, Futch, S, Singh, D, Shipley, NJ, Dale, K, LeBaron, GS and Takekawa, JY. 2020. The diverse motivations of citizen scientists: Does conservation emphasis grow as volunteer participation progresses? Biological Conservation, 242: 108428. DOI: https://doi.org/10.1016/j.biocon.2020.108428
Lewandowski, EJ and Oberhauser, KS. 2017. Butterfly citizen scientists in the United States increase their engagement in conservation. Biological Conservation, 208: 106–112. DOI: https://doi.org/10.1016/j.biocon.2015.07.029
Mitchell, N, Triska, M, Liberatore, A, Ashcroft, L, Weatherill, R and Longnecker, N. 2017. Benefits and challenges of incorporating citizen science into university education. PLoS ONE, 12(11). DOI: https://doi.org/10.1371/journal.pone.0186285
Moors, G, Kieruj, ND and Vermunt, JK. 2014. The effect of labeling and numbering of response scales on the likelihood of response bias. Sociological Methodology, 44(1): 369–399. DOI: https://doi.org/10.1177/0081175013516114
National Academies of Sciences, Engineering, and Medicine (NASEM). 2018. Learning Through Citizen Science. Washington, D.C: National Academies Press. DOI: https://doi.org/10.17226/25183
Nisbet, EK and Zelenski, JM. 2013. The NR-6: A new brief measure of nature relatedness. Frontiers in Psychology, 4: 813. DOI: https://doi.org/10.3389/fpsyg.2013.00813
Oberhauser, K and LeBuhn, G. 2012. Insects and plants: Engaging undergraduates in authentic research through citizen science. Frontiers in Ecology and The Environment, 10(6): 318–320. DOI: https://doi.org/10.1890/110274
Overdevest, C, Orr, CH and Stepenuck, K. 2004. Volunteer stream monitoring and local participation in natural resource issues. Human Ecology Review, 11(2): 177–185.
Phillips, T, Porticella, N, Bonney, R and Grack-Nelson, A. 2015. Interest in Science and Nature Scale (Adult Version). Technical Brief Series. Ithaca, NY: Cornell Lab of Ornithology.
Phillips, T, Porticella, N, Constas, M and Bonney, R. 2018. A framework for articulating and measuring individual learning outcomes from participation in citizen science. Citizen Science: Theory and Practice, 3(2): 1–19. DOI: https://doi.org/10.5334/cstp.126
Porticella, N, Phillips, T and Bonney, R. 2017a. Self-efficacy for environmental action scale (SEEA, generic). Technical Brief Series. Ithaca, NY: Cornell Lab of Ornithology.
Porticella, N, Phillips, T and Bonney, R. 2017b. Self-efficacy for learning and doing science scale (SELDS, generic). Technical Brief Series. Ithaca, NY: Cornell Lab of Ornithology.
R Core Team. 2020. R: A language and environment for statistical computing. Available at https://www.R-project.org/ (Last accessed 18 June 2021).
Rosa, CD, Profice, CC and Collado, S. 2018. Nature experiences and adults’ self-reported pro-environmental behaviors: The role of connectedness to nature and childhood nature experiences. Frontiers in Psychology, 9(1055). DOI: https://doi.org/10.3389/fpsyg.2018.01055
RStudio Team. 2020. RStudio: Integrated Development Environment for R. Available at http://www.rstudio.com/ (Last accessed 18 June 2021).
Ryan, RM and Deci, EL. 2000. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. The American Psychologist, 55(1): 68–78. https://doi-org.prox.lib.ncsu.edu/10.1037/0003-066X.55.1.68. DOI: https://doi.org/10.1037/0003-066X.55.1.68
Saribas, D, Kucuk, ZD and Ertepinar, H. 2017. Implementation of an environmental education course to improve pre-service elementary teachers’ environmental literacy and self-efficacy beliefs. International Research in Geographical and Environmental Education, 26(4): 311–326. DOI: https://doi.org/10.1080/10382046.2016.1262512
Schneiderhan-Opel, J and Bogner, FX. 2020. How fascination for biology is associated with students’ learning in a biodiversity citizen science project. Studies in Educational Evaluation, 66: 1–8. DOI: https://doi.org/10.1016/j.stueduc.2020.100892
Shirk, JL, Ballard, HL, Wilderman, CC, Phillips, T, Wiggins, A, Jordan, R, McCallie, E, Minarchek, M, Lewenstein, BV, Krasny, ME and Bonney, R. 2012. Public participation in scientific research. Ecology and Society, 17(2): 29. DOI: https://doi.org/10.5751/ES-04705-170229
Straub, MCP. 2020. A study of student responses to participation in online citizen science projects. International Journal of Science and Mathematics Education, 18(5): 869–886. DOI: https://doi.org/10.1007/s10763-019-10001-8
Szczytko, R, Stevenson, K, Peterson, MN, Nietfeld, J and Strnad, RL. 2019. Development and validation of the environmental literacy instrument for adolescents. Environmental Education Research, 25(2): 193–210. DOI: https://doi.org/10.1080/13504622.2018.1487035
Vance-Chalcraft, HD, Hurlbert, AH, Nesbitt Styrsky, J, Gates, TA, Bowser, G, Hitchcock, C, Reyes, MA and Cooper, CB. 2021. Citizen science in undergraduate education: Current practices and knowledge gaps. BioScience, In review.
Vitone, T, Stofer, K, Steininger, MS, Hulcr, J, Dunn, R and Lucky, A. 2016. School of ants goes to college: Integrating citizen science into the general education classroom increases engagement with science. Journal of Science Communication, 15(1): 1–24. DOI: https://doi.org/10.22323/2.15010203
Voss, MA and Cooper, CB. 2010. Using a free online citizen-science project to teach observation and quantification of animal behavior. The American Biology Teacher, 72(7): 437–443. DOI: https://doi.org/10.1525/abt.2010.72.7.9
Walker, D and Myrick, F. 2006. Grounded theory: An exploration of process and procedure. Qualitative Health Research, 16(4): 547–559. DOI: https://doi.org/10.1177/1049732305285972