Introduction
Citizen science participants can gain critical skills in scientific inquiry and a greater understanding of the natural world (). Data collection from citizen science, in turn, has the reciprocal effect of expanding research and monitoring efforts of under-studied biodiversity (; ). Participation by those outside of career research fields (citizen scientists) has been shown to expand subject knowledge and enhance learning in scientific inquiry () and may lead to a shift in motivations and heightened awareness of conservation practices and advocacy (; ). Generally, citizen science is the incorporation of non-experts in projects related to scientific research to expand the scope of observations available to research professionals. Given the possibilities of improving connections between science inquiry and real-life application, learning in the format of citizen science projects could be a powerful pedagogical tool in higher education settings. While people of diverse ages can participate in citizen scientist projects, traditional college students compose the newest possible generation of “citizen” scientists in their given field of study. The recruitment of college-age students into citizen science has the possibility of changing current demographic trends within aggregated citizen science projects if their participation is expanded (; ; ). The independence of choice in a secondary education setting provides a good introduction for citizen science to this new generation. College students begin to own their knowledge; expand their personal, academic, and professional horizons; and focus their interests and energies on topics they are passionate about within their field of study (). Higher education is the landing site of the next generation of scientists and citizen scientists. Students are often asked to make observations of the world around them and synthesize these data into informed opinions. Harnessing this new autonomy and integrating citizen science topics and activities into relevant college coursework increases student awareness and gives students the power to be involved in scientific inquiry on their own terms ().
Matching prospective motivations of students with perceived benefits during their interactions with voluntary citizen science participation could create a higher likelihood of future participation (). Characteristics of traditional citizen science projects, such as volunteering, self-directed learning, and a decentralized engagement structure may be particularly appealing to students who have just left behind more rigid learning environments in grade school (). As grade school (K–12) curricula are increasingly constrained to standards-based education, it may be more important than ever to provide opportunities for higher education to reframe science as a democratic process to promote science literacy (). Paradoxically, possibly due to standardization in the classroom, measures for the efficacy of applied citizen science at the grade school level are well developed but are less so for adult programs in higher education ().
Interpreting science is important to the decisions an individual makes for themselves and their community (; ). Providing students the training to inform themselves through citizen science projects may give them tools to address environmental issues in their current and future communities (). Additionally, college-age citizen scientists may bridge a gap in the age of average citizen scientist participants as participation skews older (; ). In a university classroom setting, diverse and historically marginalized groups may be introduced to citizen science projects, potentially creating a demographic shift from traditional citizen science participants in community-based projects that integrate the public at large ().
Recent reviews of citizen science literature have shown a deficit in published papers on implementation of citizen science curricula in higher education and even less on the study of students’ motivations in citizen science participation (Figure 1) (; ). Developing successful projects and understanding the motivations of perennial cycles of next-generation citizen scientists could benefit the application of these projects in the university setting broadly as well as the field as a whole.
Aligning evaluations, motivations, and learning outcomes. Here we compare motivations, learning outcomes, and evaluations between community and higher education–based citizen science. The criteria are based on those defined by Vance-Chalcraft et al. () in their assessment of college educators, on the definitions of Phillips et al. () and community, and on the three broad learning goals defined in our project. Both assessments and evaluations were broken into two broad categories that could be defined under social awareness and self-perception, and scientific inquiry.
Community-level citizen science projects have been well-studied in the past and a standardized approach to evaluating those project learning outcomes has been defined (). These community project outcomes focus on the skills, mindset, and behaviors project developers can assess as they attract and train participants. However, development of learning outcomes and strategies for implementation of effective higher education citizen science programs have been defined only recently (; ) as greater attention has been turned to evaluating efficacy of citizen science in the college classroom. Learning outcomes defined by Vance-Chalcraft and colleagues () were created from a census of university professors who have instituted citizen science in the classroom and include: excitement about science, authentic research, relevance of science, lifelong science learning, exposure to scientific methods, illustration of course content, process of science, and science justice/role models (Figure 2). From a hands-on or practical perspective, Abourashed et al. () defined many learning outcomes, one of which stresses the use of technology in a way that mirrors daily interaction for students. Apps like iNaturalist and Zooniverse, which provide free project landing pages, could benefit this learning outcome directly. In particular, iNaturalist is a type of online social network aimed at allowing learners and experts to curate information about observable biodiversity. Citizen science projects provide flexible alternatives to in-person classroom learning and the opportunity for students to interact with both laymen and experts on platforms such as iNaturalist, where they can view activities such as natural observation as part of their lifetime learning. Additionally, projects on these platforms allow students to utilize their smartphone, as they would outside a classroom, while participating in citizen science projects. Citizen science in curricula provides a mutual benefit, as contributions of citizen scientists through platforms such as eBird, iNaturalist, etc., are enabling scientists to gather robust data over greater spatial scales than would otherwise be possible (; )
Workflow of college students’ interaction with a voluntary project. Four classes, Plant Diversity (PLB300), Plant Biology (PLB200), Ethnobotany (PLB117), and Pollination Biology (PLB435), were first presented with the workflow in their classrooms, and students interacted with the material on the open-source platform iNaturalist. The material consisted of a pre- survey, hosted by SurveyMonkey, then a pollinator training module, and a follow-up post-survey.
Within this study, three broad goals were identified as being relevant to both pedagogy and further research on the application of citizen science in secondary education. For this project, we defined self-identity, self-efficacy, and skill growth in scientific inquiry as three learning outcomes to be determined or examined by using surveys and a learning module. Our project workflow was self-guided; autonomous learning occurred within the module based around ecological concepts and pollinator biology. Also included was training on scientific observations of pollinators and ways to contribute observations through citizen science platforms (iNaturalist). While the workflow for this project was voluntary and specific to pollinators, we consider it adaptable to graded curricula in many life science disciplines using classroom-based citizen science. For the purposes of this study, pollinator exposure and education were good vehicles for a citizen science project within the course curricula of the classes involved. Within the cultural zeitgeist, it is generally understood that pollinators (bees) are in decline, though grasping the importance of this event may be challenging to students because implications of anthropogenic pressures on pollinators is still poorly understood (). Additionally, previous studies of student knowledge of pollinators have found that students struggle with mechanisms of pollination as well as conservation practices (). Complex systems such as pollination may be oversimplified in primary education, and exposure through hands-on activities (such as citizen science projects) can improve ecological literacy in non-STEM majors ().
Here we describe the application of a self-guided citizen science training module presented to four undergraduate courses based in life sciences. We expected student participation in the training module would meet our broad science education learning goals of self-identity, self-efficacy, and skill growth. To monitor possible growth in these learning outcomes or goals, we developed a pre- and post-survey with questions focused on concepts of familiarity, likelihood of repeatability, importance ranking, and knowledge growth. By bookending our application of a self-guided and voluntary learning module with surveys in which students evaluate and rank their familiarity with skills in scientific inquiry and complex scientific concepts, we hope to understand the possible impact of citizen science projects in higher education.
Methods
We applied a voluntary and self-guided workflow bookended by pre- and post- assessments and centered around a training module, which focused on pollinator interactions, scientific observation, and the citizen science platform iNaturalist (). Team members (Tillotson and Weber) worked with course instructors to integrate this workflow into the most relevant parts of course curricula, as feasible. Both survey assessments could not be completed by students without providing their informed consent (Supplemental File 1: Consent Form). It was repeatedly stated through written and verbal announcements that all participation was voluntary, autonomous, and students could opt out at any time without penalty. However, students were encouraged to complete each step “for the efficacy of the project” in the classroom announcement. This project received full compliance with the institutional review board (IRB) of Southern Illinois University, Carbondale (SIUC).
Educational context
This study was completed in person in four courses with a mixed enrollment of majors at Southern Illinois University, Carbondale. All courses were housed in the College of Agricultural, Life, and Physical Sciences (CALPS). Classes were chosen where observations of the natural world are typically part of the core curricula: General Plant Biology (PLB200); Introduction to Ethnobotany and Economic Botany (PLB117); Diversity of Plants, Fungi, and Algae (PLB300); and Pollination Ecology (PLB435). Course levels differed, from upper level to the most basic introductory plant science courses. This activity was presented after a brief introduction to citizen science in the lab portion of these classes by the teaching assistant or by an unaffiliated third party. The project steps and links to surveys and modules were hosted on iNaturalist in an open journal post. As this was an autonomous and voluntary project, students who participated did not use scheduled class time to complete the project workflow. Workflow included a pre-survey, a training module, an encouragement to undertake observations on their own, and a post-survey (Figure 2). Participants (students) were presented with the survey and module through an in-person introduction in the lab portion of their classes, and a follow-up announcement was posted online on the university learning management system that contained the project workflow; this learning management system is used for most class communication and content-sharing for these four courses and is checked frequently by students. Surveys were open to participants for one month after this introduction. Within the workflow, students were prompted in the class announcement to subsequently interact with the pollinator training module. The pollinator training module, hosted on Google Slides, consisted of 36 total slides (Figure 3; Supplemental File 2: Module Slides). Of these, eight slides included the identification of floral visitors and mechanisms of pollination; eight slides detailed actual pollinator identification tips; ten slides addressed pollination syndromes, plant communities, pollinator assemblies and functional groups; five slides covered methodology of independent, repeatable observations of pollinators; and four slides addressed how iNaturalist functioned and the scientific benefits of using it for identification and observations. The last slide of the module concluded with the ecological services that pollinators provide humans. Pollinator categories were broken up into general functional groups (large bees, small bees, moths and butterflies, flies, and birds) as well as how to distinguish between types of bees (honey bee, bumblebee, small bees, carpenter bees), similar to other citizen science pollinator identification tools (e.g., ; ). Within the class announcement, students were encouraged (but not required) to undertake brief pollinator observations in the field, like those described in the module, before interacting with the post-survey.
Pollinator training module. Sections were broken down into identification of floral visitors and mechanisms of pollination; pollinator identification; pollination syndromes; plant communities, pollinator assemblies and functional groups, methodology of independent; repeatable observations of pollinators; the function of iNaturalist and scientific benefits behind using it for identification and observations; ecological services that pollinators provide humans.
Survey Population
A convenience sample of voluntary study subjects were taken from four life science courses in the plant biology department in the Fall Semester of 2022. Subjects varied between first to fourth year undergraduates, and between STEM and non-STEM majors, though upper-level courses (Diversity of Plants, Fungi, and Algae and Pollination Ecology) were mostly composed of upper-level students that had declared a major in a STEM field, housed in CALPS. All identities of participants were kept confidential; however, students were asked to voluntarily complete certain demographic information (race/ethnicity, gender, age range). All student answers were included, with the exception of those who did not answer all survey questions or those that we defined as nontraditional (ages 25–35, 35–55) as we aimed to understand learning goals in students primarily entering their college career from high school. Nontraditional students were retained for demographic analysis but excluded for analysis of participation and learning goals. Amongst demographic information, students were also asked to mark themselves as STEM or non-STEM. STEAM classification was not included to avoid confusion and to create a distinction between students who will graduate with a Bachelor of Arts instead of a Bachelor of Science. This distinction (STEM versus non-STEM) was used to understand how students with declared majors involving scientific inquiry or interacting with scientific experimentation perceive themselves. With these guidelines, students that marked their major as “Undecided” were excluded; only one student of our survey population marked themselves this way. An open-answer question then asked students to enter their declared major to verify status as STEM or non-STEM using the university’s course catalog ().
Survey Development
Survey design addressed the main learning goals of self-identity, self-efficacy, and skill growth. Self-identity was an important measure of how a student defined themselves in the sciences at a classroom scale and within their collegiate career. In testing self-efficacy, we wanted to understand to what degree students believed in their abilities to learn and perform science skills and what choices or effort they make in pursuit of these abilities. Skill growth tested confidence in repeating skills in scientific inquiry beyond the training. Survey questions were developed specifically for use in this study. Questions were sorted roughly into the three broad learning outcomes (Table 1). Our impetus for doing so stemmed from the uniqueness of the project design with its short duration and voluntary nature of participation. In designing our survey, we felt learning outcomes could not be addressed in broader, standardized surveys. However, DEVISE surveys () and the CSSES () were helpful examples in creating question themes that investigated overall constructs or learning outcomes. We recognize there has been a recent call for standardization of evaluations in the current citizen science literature (), especially within understudied areas (e.g., higher education). And the advantages of utilizing DEVISE and other standardized surveys in college citizen science projects are clear; they have been used previously (), have a strong research design, and are advantageous in understanding motivations and other criteria across a wide range of projects. However, we decided that our evaluation of learning goals may be limited without specific questions related to content within our training module. While not standardized, our design addresses similar learning goals as those outlined for both community and classroom citizen science projects (; ; ; ).
Table 1
Survey questions. Polar (11–20) and Likert scale (21–43) questions with answer scale, variations (skills, likelihood, importance), and broad learning outcomes designation (in order of appearance within the pre- and post-surveys).
| QUESTION NUMBER | QUESTION | VARIATIONS | LEARNING OUTCOMES |
|---|---|---|---|
| Polar questions, answer: Yes or No | |||
| 11 | Have you used iNaturalist or Seek (by iNaturalist)? | Self-efficacy | |
| 12 | Have you entered observations on iNaturalist? | Self-efficacy | |
| 13 | Have you used iNaturalist or Seek to identify pollinators? | Self-efficacy | |
| 14 | Have you participated in an observational study? | Skill growth in scientific inquiry | |
| 15 | Have you reported data for an experiment? | Skill growth in scientific inquiry | |
| 16 | Have you observed organisms in a field setting? | Skill growth in scientific inquiry | |
| 17 | Have you participated in a pollinator study? | Skill growth in scientific inquiry | |
| 18 | Have you participated in plant identification? | Skill growth in scientific inquiry | |
| 20 | Have you identified insect pollinators to a functional group? | Skill growth in scientific inquiry | |
| Likert questions, answer: Strongly disagree, Disagree, Agree, Strongly agree | |||
| Please begin each statement below with: “I currently have the skills to…” | |||
| 21 | Define a floral visitor | Skills | Skill growth in scientific Inquiry |
| 22 | Identify an insect pollinator to a functional group | Skills | Skill growth in scientific inquiry |
| 23 | Observe pollinators and their behavior | Skills | Self-efficacy, Skill growth in scientific inquiry |
| 24 | Predict likely plant-insect interactions (e.g., herbivory, nectarivore) just by observing the general type of insect (e.g., bee, fly) | Skills | Skill growth in scientific inquiry |
| 25 | Define a specialist or generalist | Skills | Skill growth in scientific inquiry |
| 26 | Define an ecological community | Skills | Skill growth in scientific inquiry |
| 27 | Measure abundance and density in plant communities | Skills | Skill growth in scientific inquiry |
| 28 | Link floral traits to possible functional groups of pollinators | Skills | Skill growth in scientific inquiry |
| 29 | Measure data relating to the local habitat and climate | Skills | Skill growth in scientific inquiry |
| 30 | Record data in a way that could be repeatable | Skills | Skill growth in scientific inquiry, self-efficacy |
| Likert questions, answer: Very unlikely, Unlikely, Likely, Very likely | |||
| Please begin each statement below with: “How likely are you to…” | |||
| 31 | Upload pollinator observations to a platform such as iNaturalist? | Likelihood | Self-efficacy |
| 32 | Use iNaturalist to identify pollinators or other plant-associated insects? | Likelihood | Self-efficacy, Self-identity |
| 33 | Look up additional information about a pollinator if you cannot identify it (for example: using iNaturalist)? | Likelihood | Self-efficacy |
| 34 | If you did upload your pollinator observations to a citizen science platform, how likely are you to record both the plant and insect records, so that researchers could use either? | Likelihood | Self-efficacy |
| 35 | Recommend or teach others to observe pollinators | Likelihood | Self-efficacy, Self-identity |
| 36 | Recommend the use of iNaturalist to someone else | Likelihood | Self-efficacy |
| Likert questions, answer: Not important, Somewhat important, Important, Very important | |||
| Please begin each statement with “How important…” | |||
| 37 | Are studies that examine insect functional groups? | Importance | Self-identity, Skill growth in scientific inquiry |
| 38 | Are observations that note how insects contact different parts of the flower? | Importance | Self-identity, Skill growth in scientific inquiry |
| 39 | Are the ecological services of pollinators? | Importance | Self-identity, Skill growth in scientific inquiry |
| 40 | Is citizen science (e.g., community and volunteer science) data that identifies pollinators? | Importance | Self-identity, Skill growth in scientific inquiry |
| 41 | Is it to be able to accurately identify an insect to the species level? | Importance | Self-identity, Skill growth in scientific inquiry |
| 42 | Is it to observe the behavior as well as the identification of a pollinator? | Importance | Self-identity, Skill growth inscientific inquiry |
| 43 | Is it to make pollinator observations publicly available? | Importance | Self-identity, Skill growth in scientific inquiry |
Survey Instrument
Students completed a pre-evaluation survey using SurveyMonkey. The pre- and post-surveys contain the same questions: five demographic, two check-all-that-apply (CATA), nine polar, and 23 Likert scale questions. Likert scales are broken into three separate groups that start with “I currently have the skills to…,” “How likely are you to…,” and “How important…,” and were given a forced-number Likert scale (4) with negative to positive ranking and arrangement (i.e., strongly disagree, disagree, agree, strongly agree). No negative questions, in which attitudes would be flipped, were included within the survey. Students differentiated between interactions with the survey by marking their attempts (first, second). Polar and Likert questions were delineated into measures of the broad learning outcomes of self-efficacy, self-identity, and skill growth in scientific inquiry as defined by the researchers. CATA questions were used to assess students’ current interaction with citizen science, whether just as a student or beyond (naturalist group, citizen scientist, no group affiliation, student, none, other). Motivations for interaction were represented by the question “reasons for participating in citizen science experiments” with baseline interaction as “part of a student curriculum.” Options were presented in the following order: part of a student curriculum, personal interest in biological science and natural history, volunteering with a purpose, reasons linked to climate change and biodiversity, enjoying and preserving nature, socializing with peers, and other.
Data Analysis
Statistical analyses were undertaken using R Version 4.2.2 () in R Studio Version RStudio 2022.12.0+353 (). Survey results were aggregated from SurveyMonkey after survey close, and all data remained anonymous throughout the project in accordance with IRB guidelines. Due to the voluntary nature of workflow and anonymity of the surveys, answers between pre- and post-surveys were inherently unpaired. To assess significant changes following module implementation in pre- and post-survey participants, we compared mean pre- and post-survey Likert question scores with self-reported delineation of STEM and non-STEM using both a Welch’s one- and two-tailed t-test to examine both difference and improvement in scale ranking between the two surveys. Polar questions were used as a proxy for self-efficacy, (ɑ = 0.05), evaluating shifts from negative (No) to positive (Yes) response between the pre- and post-survey for all respondents and were evaluated using one-sided McNemar’s Test (stat package; see Supplemental File 4: Data Analysis). To test for self-efficacy, we used analogs such as familiarity, repeatability, and/or likelihood (Table 1). Many self-efficacy questions occurred within the polar or yes/no section, and the Likert questions focused on likelihood (“How likely are you too…?”). Participation and demographic data were assessed qualitatively. Demographic data was compared between surveys for retention, and within the survey population for a description of the general make-up of participants. CATA questions were analyzed qualitatively to understand the different facets of how students described themselves in in participation and beyond.
Results
Participation and Demographics
In total, 109 undergraduate students among four separate classes (number of students: Ethnobotany PLB117 = 25, Plant Biology PLB200 = 62, Plant Diversity PLB300 = 12, Pollination Ecology PLB435 = 10) were presented with the option of voluntarily participating in the workflow composed of the pre- and post-survey, module, and independent pollinator observations (Figure 2). For the first survey, 47 of the 109 total students participated completely (43% completed the survey). Between the first and second survey, there was a 53% retention rate (post-survey = 25). Of the students that interacted with the first survey, 16 partially filled the demographic portion. These partial interactions were excluded from all further analysis. Survey interactions in which the respondent answered 50% or less of the survey were also excluded. Of the total complete student interactions possible (within the sample group), there was 43% with the first survey and 23% with the second survey. In evaluating the first survey demographic data, 36.2% students ranked themselves as STEM and 63.8% as non-STEM. Of those that ranked themselves non-STEM, only 3 of 27 would be graduating with a B.A. Degree. Students at the university surveyed (SIUC) are allowed to declare a major within their freshman year. This means that most students that ranked themselves as non-STEM have the possibility of graduating with a B.S. degree. Almost all these students were from CALPS, under which all degrees are a B.S. Many of these students were also Forestry majors—a degree in which students can learn and specialize in topics related to forest hydrology, forest conservation, and species management. Of the 24 students that fell under CALPS, only 5 marked themselves as STEM. The overwhelming majority of students and respondents were 18–25 years old, although some classes included a small percentage of nontraditional students ranging from 35–55 that responded to the survey (< 10%). While assessing self-efficacy in citizen science of nontraditional students is important, the focus of this study was to understand the self-efficacy of students entering a time of autonomy and individual choice (18–25), so nontraditional respondents were excluded from the analysis.
Retention of participants that identified as male or female was virtually equal between surveys (female = 34.9%, male = 36.0%). The majority of participants for both surveys identified as white (first = 85.1%, second = 96%) (Table 2), and while we did not have a demographic breakdown for classes due to the anonymous nature of the surveys, these results follow race and ethnicity trends for the undergraduate student body overall (). We addressed possible current participation in citizen science projects; these categories included: naturalist group, citizen scientist (no group affiliation), student, or none. For self-reporting of current affiliation in citizen science, participants overwhelmingly assigned themselves “student” (pre- = 87.2%, post- = 92%) with a minority also ranking themselves as “citizen scientist (with no group affiliation).” Main motivations for students interacting with citizen science for overall participants for the pre-survey were (in descending order) “part of a student curriculum” (48.9%), “enjoying and preserving nature” (42.6%), and “personal interest in biological science and natural history” (29.8%) (check all that apply). Self-identified STEM students often selected “part of a student curriculum” and “enjoying and preserving nature” together. For participants overall, motivations did not change notably between surveys. Students who self-identified as in a STEM field had a greater diversity in answers across the options for participation.
Table 2
Demographics of students who interacted with the pre- and post-survey. Total number of participants is provided, as well as percentage of participants completing the pre-survey, post-survey, and total retention between surveys out of a total of 109 possible participants.
| RESULTS, OVERALL PARTICIPANTS | |||||
|---|---|---|---|---|---|
| DEMOGRAPHICS | PRE-SURVEY (n) = 47 | POST-SURVEY (n) = 25 | PRE-SURVEY | POST-SURVEY | RETENTION BETWEEN SURVEYS (WHEN APPLICABLE) |
| Overall respondents | 47 | 25 | 43.12% | 23% | 53.19% |
| Self-ranked STEM | 17 | 7 | 36.17% | 28% | 29.17% |
| Self-ranked non-STEM | 30 | 18 | 63.83% | 72% | 37.50% |
| STEM (actual) | 39 | 22 | 82.98% | 88% | 36.07% |
| Non-STEM (actual) | 8 | 3 | 17.02% | 12% | 27.27% |
| Race and ethnicity | |||||
| Hispanic | 4 | 0 | 8.51% | ||
| White | 40 | 24 | 85.11% | 96% | 37.50% |
| Black or African American | 1 | 0 | 2.13% | – | – |
| Asian | 0 | 0 | – | – | – |
| American Indian or Alaska Native | 1 | 1 | 2.13% | 4% | – |
| Native Hawaiian or Pacific Islander | 0 | 0 | – | – | – |
| Other race, not listed | 2 | 0 | 4.26% | – | – |
| Gender | |||||
| Man | 28 | 15 | 59.57% | 60% | 34.88% |
| Woman | 16 | 9 | 34.04% | 36% | 36% |
| Transgender | 0 | 0 | – | – | – |
| Non-binary/non-conforming | 3 | 1 | 6.38% | 4% | – |
| Prefer not to respond | 0 | 0 | – | – | – |
| Age group | |||||
| 18–25 | 43 | 22 | 91.49% | 88% | \ |
| 25–35 | 1 | 1 | 2.13% | 4% | \ |
| 35–55 | 3 | 2 | 6.38% | 8% | \ |
| Current citizen science participation (CATA) | |||||
| naturalist group | 1 | 0 | 2.13% | – | |
| Citizen scientist (no group affiliation) | 5 | 4 | 10.64% | 16% | |
| Student | 41 | 23 | 87.23% | 92% | |
| None | 3 | 0 | 6.38% | – | |
| Other | 0 | 0 | – | – | |
| What is your main reason for participating in citizen science experiments (CATA; Motivations) | |||||
| Part of a student curriculum | 23 | 12 | 48.94% | 48% | |
| Personal interest in biological science and natural history | 14 | 3 | 29.79% | 12% | |
| Volunteering with a purpose | 9 | 5 | 19.15% | 20% | |
| Reasons linked to climate Change and biodiversity | 5 | 1 | 10.64% | 4% | |
| Enjoying and preserving nature | 20 | 10 | 42.55% | 40% | |
| Socializing with peers | 6 | 1 | 12.77% | 4% | |
| Other | 9 | 3 | 19.15% | 12% | |
Learning Outcomes
In analyzing results between pre- and post-surveys after module interaction for overall participants, we found a significant increase in Likert scale ranking (ɑ = 0.05) for all questions across all learning outcome assignments (Welch’s one-tailed t-test; self-efficacy, self-identity, skill growth) (Tables 4a, bc; Supplemental File: Survey Data). Amongst self-reported non-STEM students, all Likert scale questions, barring two related to self-efficacy learning outcomes, exhibited positive and significant rank increase between surveys (Table 3). For pre- and post-surveys, we found an overall significant increase in Likert scale ranking for questions across all learning outcome assignments when pooling all respondents (ɑ = 0.05; self-efficacy, self-identity, skill growth) (Tables 4a, bc).
Table 3
Comparison of pre- and post-survey polar questions for overall participants. McNemar’s one-sided test for polar (yes, no) questions.
| POLAR (YES, NO) QUESTIONS EVALUATED WITH MCNEMAR’S ONE-SIDED TEST (α = 0.05) | |||
|---|---|---|---|
| QUESTION NUMBER | QUESTION | OVERALL PARTICIPANTS (39, 22) | |
| χ2 | p-VALUE | ||
| Question 10 | Are you familiar with iNaturalist? | 11.13 | 0.0008492 |
| Question 11 | Have you used iNaturalist or Seek (by iNaturalist)? | 1.63 | 0.2012 |
| Question 12 | Have you entered observations on iNaturalist? | 0 | 1 |
| Question 13 | Have you used iNaturalist or Seek to identify pollinators? | 2.04 | 0.15 |
| Question 14 | Have you participated in an observational study? | 1.44 | 0.23 |
| Question 15 | Have you reported data for an experiment? | 0.66 | 0.4173 |
| Question 16 | Have you observed organisms in a field setting? | 8.76 | 0.003085 |
| Question 17 | Have you participated in a pollinator study? | 11.13 | 0.0008492 |
| Question 18 | Have you participated in plant identification? | 16.49 | 0.00004896 |
| Question 19 | Have you identified insect pollinators to a functional group? | 7.68 | 0.005578 |
Table 4a
Comparison of pre- and post-survey Likert scale questions for overall participants. Welch’s one-sided and two-sided t-tests for Likert scale questions (all), values highlighted are those of non-significance (4a–4c).
| WELCH’S ONE-SIDED AND TWO-SIDED T-TESTS FOR LIKERT SCALE QUESTIONS (ALPHA = 0.05) | |||||||
|---|---|---|---|---|---|---|---|
| ALL PARTICIPANTS | |||||||
| QUESTION # | PRE n, POST n | df | t STAT | t CRITICAL ONE-TAIL | P(T <= t) ONE-TAIL | t CRITICAL TWO-TAIL | P(T <= t) TWO-TAIL |
| Q20 | (39, 22) | 59 | –3.12935 | 1.671093 | 0.001361 | 2.000995 | 0.002723 |
| Q21 | (39, 22) | 57 | –4.52087 | 1.672029 | 1.58E–05 | 2.002465 | 3.16E–05 |
| Q22 | (39, 21) | 41 | –4.84794 | 1.682878 | 9.13E–07 | 2.019541 | 1.83E–05 |
| Q23 | (39, 22) | 49 | –2.06367 | 1.676551 | 0.022183 | 2.009575 | 0.044367 |
| Q24 | (37, 22) | 52 | –4.72528 | 1.674689 | 7.151E–05 | 2.006647 | 7.15E–05 |
| Q25 | (39, 22) | 55 | –4.35587 | 1.676551 | 8.99E–05 | 2.009575 | 0.00018 |
| Q26 | (39, 22) | 55 | –2.63393 | 1.673034 | 0.00547 | 2.004045 | 0.010939 |
| Q27 | (39, 22) | 56 | –2.49019 | 1.672522 | 0.007881 | 2.003241 | 0.015761 |
| Q28 | (39, 22) | 54 | –4.41641 | 1.673565 | 2.43E–05 | 1.673565 | 4.86E–05 |
| Q29 | (39, 22) | 51 | –1.79833 | 1.675285 | 0.039022 | 2.007584 | 0.078045 |
| Q30 | (39, 22) | 57 | –2.52963 | 1.672029 | 0.007104 | 2.002465 | 0.014208 |
| Q31 | (39, 22) | 46 | –2.97025 | 1.67866 | 0.004716 | 2.012896 | 0.004716 |
| Q32 | (39, 22) | 51 | –1.65067 | 1.675285 | 0.052476 | 2.007584 | 0.104952 |
| Q33 | (39, 22) | 51 | –1.87428 | 1.675285 | 0.033312 | 2.007584 | 0.066625 |
| Q34 | (39, 22) | 51 | –1.8475 | 1.675285 | 0.035239 | 2.007584 | 0.070479 |
| Q35 | (39, 22) | 45 | –1.4601 | 1.679427 | 0.075604 | 2.014103 | 0.151208 |
| Q36 | (39, 22) | 53 | –2.10989 | –1.674116 | 0.019803 | 2.005746 | 0.039606 |
| Q37 | (39, 22) | 55 | –1.64184 | 1.673034 | 0.053165 | 2.004045 | 0.10633 |
| Q38 | (38, 22) | 47 | –2.16026 | 1.677927 | 0.017942 | 2.011741 | 0.035884 |
| Q39 | (39, 22) | 55 | –1.01581 | 1.673034 | 0.157084 | 2.004045 | 0.314167 |
| Q40 | (39, 22) | 52 | –1.6686 | 1.674689 | 0.050604 | 2.006647 | 0.101208 |
| Q41 | (39, 22) | 48 | –2.35313 | 1.677224 | 0.01138 | 2.010635 | 0.02737 |
| Q42 | (39, 22) | 56 | –2.26547 | 1.672522 | 0.013685 | 2.003241 | 0.02737 |
| Q43 | (39, 22) | 42 | –1.24948 | 1.681952 | 0.109203 | 2.018082 | 0.218407 |
Table 4b
c Comparison of pre- and post-survey Likert scale questions for self-identifying STEM participants. Welch’s one-sided and two-sided t-tests for Likert scale questions (all) values highlighted are those of non-significance.
| WELCH’S ONE-SIDED AND TWO-SIDED T-TESTS FOR LIKERT SCALE QUESTIONS (ALPHA = 0.05) | |||||||
|---|---|---|---|---|---|---|---|
| SELF-IDENTIFYING NON-STEM | |||||||
| QUESTION # | n 1st,n 2nd | df | t STAT | t CRITICAL ONE-TAIL | P(T <= t) ONE-TAIL | t CRITICAL TWO-TAIL | P(T <= t) TWO-TAIL |
| Q20 | (24, 16) | 38 | –4.10133 | 1.685954 | 0.000104 | 2.024394 | 0.000209 |
| Q21 | (24, 16) | 37 | –5.52589 | 1.687094 | 1.38E-06 | 2.026192 | 2.75E-06 |
| Q22 | (24, 16) | 34 | –4.49963 | 1.690924 | 3.79E-05 | 2.032245 | 7.58E-05 |
| Q23 | (24, 16) | 38 | –1.8484 | 1.685954 | 0.036169 | 2.024394 | 0.072337 |
| Q24 | (22, 16) | 36 | –3.71374 | 1.688298 | 0.000344 | 2.028094 | 0.000689 |
| Q25 | (24, 16) | 34 | –4.07787 | 1.690924 | 0.000129 | 2.032245 | 0.000259 |
| Q26 | (24, 16) | 38 | –2.62784 | 1.685954 | 0.006163 | 2.024394 | 0.012327 |
| Q27 | (24, 16) | 38 | –1.93992 | 1.685954 | 0.029919 | 2.024394 | 0.059838 |
| Q28 | (24, 16) | 38 | –3.50557 | 1.685954 | 0.000593 | 2.024394 | 0.001186 |
| Q29 | (24, 16) | 36 | –1.472 | 1.688298 | 0.074855 | 2.028094 | 0.14971 |
| Q30 | (24, 16) | 38 | –2.6891 | 1.68594 | 0.00529 | 2.024394 | 0.010581 |
| Q31 | (24, 16) | 33 | –2.77836 | 1.69236 | 0.021781 | 2.030108 | 0.043562 |
| Q32 | (24, 16) | 35 | –2.09409 | 1.689572 | 0.021781 | 2.030108 | 0.043562 |
| Q33 | (24, 16) | 36 | –1.92248 | 1.688298 | 0.031243 | 2.028094 | 0.062485 |
| Q34 | (24, 16) | 36 | –1.63323 | 1.688298 | 0.055253 | 2.028094 | 0.110506 |
| Q35 | (24, 16) | 34 | –1.25032 | 1.690924 | 0.125032 | 2.032245 | 0.250064 |
| Q36 | (24, 16) | 36 | –2.47364 | 1.688298 | 0.009113 | 2.028094 | 0.018226 |
| Q37 | (24, 16) | 38 | –2.10479 | 1.685954 | 0.020989 | 2.024394 | 0.041978 |
| Q38 | (23, 16) | 33 | –2.507 | 1.69236 | 0.008639 | 2.034515 | 0.017278 |
| Q39 | (24, 16) | 37 | –1.54335 | 1.687094 | 0.065628 | 2.026192 | 0.131256 |
| Q40 | (24, 16) | 38 | –2.18064 | 1.685954 | 0.017735 | 2.024394 | 0.03547 |
| Q41 | (24, 16) | 37 | –2.37026 | 1.687094 | 0.01155 | 2.026192 | 0.023099 |
| Q42 | (24, 16) | 38 | –2.1586 | 1.685954 | 0.018631 | 2.024394 | 0.037262 |
| Q43 | (24, 16) | 30 | –1.50163 | 1.697261 | 0.071823 | 2.042272 | 0.143645 |
Table 4c
Comparison of pre- and post-survey Likert scale questions for self-identifying STEM participants. Welch’s one-sided and twosided t-tests for Likert scale questions (all) values highlighted are those of non-significance.
| WELCH’S ONE-SIDED AND TWO-SIDED T-TESTS FOR LIKERT SCALE QUESTIONS (ALPHA = 0.05) | |||||||
|---|---|---|---|---|---|---|---|
| SELF-IDENTIFYING STEM | |||||||
| QUESTION # | n 1st,n 2nd | df | t STAT | t CRITICAL ONE-TAIL | P(T <= t) ONE-TAIL | t CRITICAL TWO-TAIL | P(T <= t) TWO-TAIL |
| Q20 | (16, 7) | 15 | –3.41565 | 1.75305 | 0.001916 | 2.13145 | 0.003833 |
| Q21 | (16, 7) | 19 | –3.08127 | 1.729133 | 0.003073 | 2.093024 | 0.006146 |
| Q22 | (16, 7) | 19 | –6.36881 | 1.729133 | 2.07E-06 | 2.093024 | 4.14E-06 |
| Q23 | (16, 7) | 16 | –1.745884 | 1.745884 | 0.001848 | 2.119905 | 0.003696 |
| Q24 | (16, 7) | 19 | –4.808678 | 1.729133 | 6.13E-05 | 2.093024 | 0.000123 |
| Q25 | (16, 7) | 15 | –5.69431 | 1.75305 | 2.13E-05 | 2.13145 | 4.25E-05 |
| Q26 | (16, 7) | 19 | –2.72684 | 1.729133 | 0.006695 | 2.093024 | 0.01339 |
| Q27 | (16, 7) | 20 | –2.9137 | 1.724718 | 0.004294 | 2.085963 | 0.008587 |
| Q28 | (16, 7) | 20 | –4.18934 | 1.724718 | 0.000226 | 2.085963 | 0.000452 |
| Q29 | (16, 7) | 17 | –1.59502 | 1.739607 | 0.064565 | 2.109816 | 0.129129 |
| Q30 | (16, 7) | 12 | –0.71919 | 1.782288 | 0.242896 | 2.178813 | 0.485791 |
| Q31 | (16, 7) | 16 | –3.09676 | 1.745884 | 0.003463 | 2.119905 | 0.006927 |
| Q32 | (16, 7) | 13 | –1.02775 | 1.770933 | 0.322812 | 2.160369 | 0.322812 |
| Q33 | (16, 7) | 16 | –1.84712 | 1.745884 | 0.041653 | 2.119905 | 0.083307 |
| Q34 | (16, 7) | 18 | –1.67076 | 1.734064 | 0.056032 | 2.100922 | 0.112064 |
| Q35 | (16, 7) | 16 | –2.59843 | 1.745884 | 0.009701 | 2.119905 | 0.019403 |
| Q36 | (16, 7) | 13 | –1.02775 | 1.770933 | 0.161406 | 2.160369 | 0.322812 |
| Q37 | (16, 7) | 8 | –0.41362 | 1.859548 | 0.345007 | 2.306004 | 0.690015 |
| Q38 | (16, 7) | 12 | –1.23266 | 1.782288 | 0.120654 | 2.178813 | 0.241307 |
| Q39 | (16, 7) | 15 | 0.094072 | 1.75305 | 0.463148 | 2.13145 | 0.926297 |
| Q40 | (16, 7) | 10 | –0.21208 | 1.812461 | 0.418153 | 2.228139 | 0.836306 |
| Q41 | (16, 7) | 9 | –1.33162 | 1.833113 | 0.107862 | 2.262157 | 0.215724 |
| Q42 | (16, 7) | 12 | –1.52285 | 1.782288 | 0.153706 | 2.178813 | 0.153706 |
| Q43 | (16, 7) | 15 | –0.91028 | 1.75305 | 0.188535 | 2.13145 | 0.377069 |
Self-efficacy
Questions 11, 12, 13, and 15 exhibited no significant shift when evaluated with overall participants. These questions generally addressed familiarity with iNaturalist and observations or reporting (Table 3). In overall participants, we found self-efficacy improved between the pre- and post-survey for Likert questions (31–36; see Table 4a).
Self-Identity
Likert scale questions developed for self-identity initially represented the importance of ecological studies and the personal role someone could have in the scientific process. However, because students delineated themselves into self-identified STEM and non-STEM, we could evaluate the results of improvement across surveys in a different light. Self-identified STEM students showed little significant increase across surveys; however, scale ranking was very high across both. Areas of improvement for students that identified themselves as in STEM fields included those related to skills in scientific inquiry and familiarity with specific ecological interactions. This may have been due to the novel nature of the content, even for upper-level undergraduate STEM students. Among self-identified non-STEM students, most Likert scale questions (except two related to self-efficacy learning outcomes) exhibited positive and significant rank increase between surveys (Table 4b).
Skill Growth
Skill growth questions generally evaluated specific synthesis of knowledge related to identification (Questions 21–30; see Table 1). Overall, students shifted towards a more positive rank for these questions, showing improvement after introduction through the module. Self-ranked STEM and non-STEM also exhibited positive ranking across surveys for skill growth questions.
Discussion
Our results suggest that implementation of citizen science–based learning modules, even those that are brief and self-guided, can influence learning outcomes within a college classroom. The general increase in ranking (No to Yes; 1–4) across scales for those prone to seeing themselves outside of the sciences (self-ranked non-STEM) is an important indication that these supplements to more traditional curricula are effective. We understand some drawbacks to the study design, such as the voluntary nature of participation, the short duration, the low retention of survey respondents, and the possibility of students who participated in the pre-survey being more likely to complete the post-survey. Instituting this study within a classroom setting may have caused involuntary social desirability bias in participants, and how this may have influenced the data is unknown. Low improvement in ranking of self-ranked non-STEM majors of questions related to using or recommending iNaturalist (Table 4b) may have been improved with a longer-term project; without which, students may fail to see the possibilities for these tools in their own life. While the dataset is small in relation to the possible participants, it provides meaningful insights for the impact of brief training and student familiarity and efficacy of intended learning outcomes.
Motivations and Current Participation in Citizen Science
In multiple plant biology courses at SIUC, we surveyed students’ motivations for several reasons: to understand what may motivate student’s past classroom participation, to detect possible increases in diversity of motivations after interaction with the module, and to compare the motivations between self-identified STEM and non-STEM students (Table 2). The addition of a climate anxiety option is in response to a growing determination that having an outlet to make changes related to the environment is a healthy way of managing eco-based anxiety (), and our results suggest that for a small percentage of students (10.6%) this was true. Motivations of students varied more for those that identified themselves as STEM versus non-STEM. Non-STEM cited “part of a student curriculum” most, possibly because they viewed interaction with the module as perfunctory. For self-identifying STEM majors, variability could be due to a greater understanding or personal interest in the topic; studies report mixed findings when evaluating the relationship between personal interest and participation in citizen science in college students ().
Self-Efficacy
For overall participants, improvement in self-efficacy across the pre- and post-surveys for questions related to familiarity within polar (yes/no) question testing aligns with documentation of the positive impact of courses meant to develop self-efficacy employed in the college classrooms (). In measuring possible growth in competence, we hoped to understand to what degree the training module impacted students’ self-efficacy. However, previous experience as well as the uniqueness of individual experience may make understanding self-efficacy with scale metrics difficult (). Additionally, the voluntary nature of interaction and contribution by students may have limited development of skills related to self-efficacy. Students sense of self-efficacy may be better developed in co-created versus contributory citizen science projects ().
Self-Identity
In evaluating student self-identity in the sciences, we defined a clear mismatch in students declared major and their self-perception in those STEM fields. Finding that forestry and agricultural majors, both resource management focused, were misidentifying themselves as non-STEM was an unexpected take-away from the demographics part of our survey. Understanding how students self-identify in the sciences is an open area of research (); it is possible that self-identities change across college careers. We were unable to evaluate the potential for changes in self-identity in our study due to low and uneven sample sizes across college years and majors. However, it is possible that evaluations that are paired with citizen science modules conducted by instructors could help address systematic failures in self-identity and misunderstanding in science education in a college setting.
Addressing the failure of Agriculture and Forestry majors to perceive themselves as STEM majors should be taken into consideration when designing future college classroom citizen science projects that involve introductory science courses and first- and second-year students. General Plant Biology, a 200-level course, is often a prerequisite or recommended course for students entering into majors housed in the College of Agricultural and Life Sciences at SIUC. This course is meant to ensure a fundamental understanding of the basic properties of plant development and reproduction, develop a grasp of the scientific process, and introduce foundational topics in biology such as ecology, evolution, and conservation (as described in the SIUC course catalog; ). Agricultural students may seek positions as future land managers; perception and self-efficacy in the sciences could have direct benefits to future land management and commercial or private agriculture (i.e., ; ). Our findings stress the need for citizen science projects to be designed to evaluate and examine student self-perception.
Skill Growth
Outcomes related to skill growth through science inquiry should be foremost on the minds of educators looking to assess effectiveness when leveraging citizen science projects in the college classroom (). Even within a short project, overall survey participants exhibited positive rank increase for skill growth questions across surveys. This aligns with findings across other short-term community- and college-level citizen science projects (; ; ). Due to the voluntary nature of the project, we could not assess student interactions with the personal observation as part of the project workflow or evaluate the efficacy of hands-on interaction on skill growth. However, even without this verification, students in all groups recorded an increase in skills related to scientific inquiry. While teachers in higher education have outlined many social learning outcomes, most evaluations have focused on evaluating skill growth related to the topics addressed (; Figure 1). Skill growth in scientific inquiry has clear ties to an increase in self-efficacy but both should be assessed separately to understand if students will engage with citizen science projects or observation-based research independently.
Broad Outcomes of Citizen Science in the Classroom
In instituting citizen science in the classroom, it is important to understand on what level students will interact with a project and how integrated it must be within the curricula (). With the relatively high level of student interaction in this voluntary project (43%), as well as the increased familiarity with skills in scientific inquiry (Table 3), even limited integration into curricula can have a meaningful impact. Leveraging this type of citizen science project in the college classroom may provide the opportunity to balance pedagogy and digitization of teaching in a nontraditional setting (). Creating active and flexible learning experiences that result in meaningful data should be a key goal of classroom citizen science projects; the Covid-19 pandemic ushered in an era of robust virtual learning, which educators will continue to evaluate for its merits (; ; ). Our results demonstrated that within introductory or general education science requirements, citizen science can be an important way for non-STEM students to gain science literacy skills. For example, citizen science activities may highlight the importance of conducting a systematic study versus anecdotal evidence (), and collecting verifiable information versus values-based information (). Traditional lecture and testing methods may reinforce barriers that science is only for the “best and brightest” (). Epistemological approaches should stress that scientific knowledge is constructed by people through a process that can be democratic (). In this era of teaching, where an inundation of information is an everyday fact of life, a personal understanding of scientific processes will be necessary for citizens to make informed decisions.
Conclusions and Future Study
We found that employment of citizen science projects within a curriculum, even at a small scale, can have a positive impact on skill growth, self-identity and self-efficacy, and other learning outcomes that are the focus of science educators. Without evaluating demographics and designing a survey to include more comprehensive questions on self-identity we would have overlooked a significant mismatch in students’ perception of themselves in the sciences; students that misidentified themselves as non-STEM participants in our survey exhibited skill growth after module interaction. From a pedagogical perspective, project employment can identify groups of students that may not see themselves within the sciences and who may be better reached by means beyond that of standard science curricula. Understanding the demographics of engagement will continue to be important in the field of citizen science as we engage the next generation of citizen scientists, especially those from underrepresented groups. Standardized evaluations such as DEVISE may not be the most effective in measuring learning outcomes, but testing along themes defined by the growing body of research such as building interest and self-efficacy () and leveraging technology (; ) will continue to define the impact of project integration in the classroom. Even defining broad learning goals creates a framework for future evaluation in the field.
Data Accessibility Statement
All data used for this study are available in Supplemental Files 1–4; data are formatted and provided within the IRB guidelines.
Supplementary Files
The supplementary files for this article can be found as follows:
Supplemental File 1:Consent Form. DOI: https://doi.org/10.5334/cstp.641.s1
Supplemental File 2:Module Slides. DOI: https://doi.org/10.5334/cstp.641.s2
Supplemental File 3:Survey Data. DOI: https://doi.org/10.5334/cstp.641.s3
Supplemental File 4:Data Analysis. DOI: https://doi.org/10.5334/cstp.641.s4