Introduction
Citizen science involves the public in scientific processes to address fundamental or applied research questions. Investigations requiring repeated data collection at many locations over a broad geographic area, in particular, can benefit from citizen science, as long as data collection methods do not require sophisticated equipment or a highly skilled workforce (; ). The prevalence of smartphones with internet capabilities has facilitated the ease of data collection from numerous users through project-specific apps. Thus, citizen scientists can greatly enhance the quantity of data being collected to address large-scale questions (; ).
In addition to contributing significantly to scientific knowledge, citizen science has been found to confer broad cognitive benefits to participants. For example, studies have documented a greater understanding of the scientific method (; ; ; ), development of a scientific mindset (), retention of content knowledge (; ; ; ; ), and students’ beliefs in their own ability to do science (i.e., self-efficacy; ; ). As citizen science began in informal learning environments and remains most prevalent in these settings, these benefits have been studied most commonly in such environments, in which participants are self-selecting and motivated ().
In the past decade, citizen science has burgeoned as a popular tool for educating students in colleges and universities. According to a recent survey of undergraduate instructors, two of the primary reasons that instructors choose to incorporate citizen science in a course are to engage students and to expose them to authentic scientific research (). Participation in the pursuit of novel research findings is a powerful way to facilitate student conceptualization of science and how it is conducted (; ; ; ). Involvement in research also has been shown to be an efficient way to build critical thinking and communication skills, as well as to tap into students’ creativity and ability to collaborate effectively with others (; ; ).
Although it seems reasonable to hypothesize that many of the cognitive benefits found in participants in informal learning settings would translate to student participants in higher education, multiple differences exist among these populations. For example, students in a college or university course may be motivated to participate, not by intrinsic interest, but by extrinsic factors such as grades. Thus, students may be less likely to benefit cognitively, as autonomy and motivation are known to be important facilitators of learning (; ; ; ). In contrast, students could achieve higher-level cognitive gains since the structured nature of undergraduate courses may provide more support for and sequencing of scientific content and practices related to citizen science ().
Little research has been published, however, on how involvement in citizen science impacts student learning in an undergraduate course (; ). Most of the literature on the use of citizen science in higher education has documented perceived benefits of participation in citizen science through student reflections, instructor perceptions, or anecdotal evidence (; ; ; ). Vitone et al. () used an assessment instrument to examine changes in student content knowledge after participation in citizen science but found no significant gains and cited the need for more sensitive assessment instruments. More research is needed to determine the true impacts of citizen science participation on undergraduate students (). Until standardized tools are available to examine the impact of citizen science participation at multiple levels, individual case studies are an important means of understanding the influence of citizen science participation on students (). Case studies are valuable for providing insight into participants’ experiences ().
We present a case study describing the incorporation of the same citizen science project, Caterpillars Count!, into biology courses at three universities. Caterpillars Count! is a project focused on understanding how the abundance of caterpillars and other arthropods found on trees and shrubs varies in time, in space, and with host plant species (). Foliage arthropods are an important component of forest ecosystems, and recent studies highlighting widespread insect declines (, , ) point to the importance of large-scale monitoring that can shed light on where and how arthropod communities are changing. In addition to concerns over long-term trends, some arthropod groups may also be shifting toward earlier seasonal activity (or phenology) due to increasingly warmer springs with earlier leaf-out (, ). If species who consume these arthropods, like migratory songbirds, are not shifting the timing of migration and reproduction to similar degrees, they risk becoming phenological mismatches with the resources they depend on to successfully raise their young (, ).
This case study examines whether the modest integration of Caterpillars Count! into undergraduate courses increases biology content knowledge, understanding of citizen science, and scientific literacy. The institutional characteristics, duration of student participation in the project, and scientific content being emphasized through Caterpillars Count! varied for the three institutions. Thus, these three case studies represent unique but complementary course experiences. In this paper, we use a mixed methods approach () that includes both survey and open-ended student responses to chronicle the outcomes associated with the use of this citizen science project in three courses, varying in course and student characteristics.
Methods
We incorporated Caterpillars Count! into introductory biology courses at three large University of North Carolina system schools: East Carolina University (ECU), North Carolina State University (NCSU), and the University of North Carolina Chapel Hill (UNC). We selected Caterpillars Count! (https://caterpillarscount.unc.edu/; ) as the citizen science project because of how the project’s focus and logistics fit within our course contexts. Caterpillars Count! also had the advantage of being developed by an ecologist at UNC so students could contribute to research that began locally and, in some cases, could interact directly with the research team. Details about the project are available in Hurlbert et al. () or on the project website, but information most relevant to the use of this project with undergraduate students is described below.
The Caterpillars Count! website (https://caterpillarscount.unc.edu/resources) provides a variety of resources for training, as well as data access, exploration, and visualization. Students can learn to distinguish the most common arthropod orders found on woody vegetation from identification guides, and test their ability to find, identify, and estimate the length of arthropods in a virtual survey game (https://caterpillarscount.unc.edu/virtualSurvey). Participants may explore maps and graphs of any data submitted to the project, and not just the data they have collected themselves. Visualizations include maps of arthropod density or biomass, graphs of density or biomass over time, and graphs of arthropod community composition by order (https://caterpillarscount.unc.edu/mapsAndGraphs). Data underlying these graphical summaries, as well as the raw data themselves, are freely available for download and can serve as the basis for various inquiry-based learning activities.
The basic survey unit in Caterpillars Count! is a branch survey, in which a participant records the total number of arthropods seen on a branch, their identity to taxonomic Order (e.g., beetles versus spiders versus caterpillars), and their body lengths to the nearest millimeter. The protocol is designed primarily for broad-leaved trees and shrubs with an average leaf length of at least 5 cm and with foliage at or below eye height. A participating “site” typically consists of 10–125 marked branches arranged in “survey circles” of five branches each (). Each site (e.g., university campus) generally samples branches within their designated survey circles multiple times throughout the spring and summer using one of two survey methods that they select: 1) visual inspection of each branch over an area of 50 leaves and associated twigs and petioles, or 2) a beat sheet, which is held under a branch while the branch is beat with a stick ten times to dislodge any arthropods. Ideally, all five branches in each survey circle are monitored on the same date for each sampling session, which allows the estimation of overall arthropod density at the site on a given date. In some cases, especially when assigning branch surveys as part of an undergraduate class, students are provided a more flexible four-to-seven-day window over which they must conduct their surveys. Data may be submitted in the field using a free mobile app or entered through the project website. Users may optionally submit photos of the arthropods they observe. These photos are automatically shared with the citizen science platform iNaturalist (https://inaturalist.org), where identification can be crowd-sourced by amateur and expert naturalists.
At all three institutions, students in an introductory biology course used a standardized sampling protocol to collect arthropod abundance data and add it to the project-wide Caterpillars Count! database. The emphasis on data visualization and data analysis in these focal courses was limited and differed by institution. Each institution chose to emphasize slightly different content regarding the project and spent different amounts of time on the project in class during 2018 and 2019 (see below for additional details). It took approximately ten minutes for students to sample a single branch, but each institution sampled differing numbers of survey circles (made up of 5 branches each) with differing amounts of repeated sampling of the same survey circles through time (see below). Students used training resources from the Caterpillars Count! website to learn arthropod identification to Order and took training quizzes to test their ability to identify arthropods before proceeding with sampling. Student identifications (again to Order) entered into the Caterpillars Count! database were not verified by an expert. Students had the option to contribute photos to iNaturalist, but few did. Thus, we are unable to provide a robust assessment of the quality of the scientific data the students provided to the project. Previously, a rigorous assessment of the quality of data uploaded to Caterpillars Count! by volunteers (not just students) versus experts showed that data contributed by volunteers was of generally high quality ().
Integration of citizen science at each institution
East Carolina University
At ECU, this project was included in an environmental biology lab course for non-science majors. This one-credit introductory lab course is associated with a three-credit lecture course, but Caterpillars Count! was implemented solely in the lab sections. These lab sections were taught by graduate teaching assistants who were overseen by a faculty lab coordinator. Caterpillars Count! was included in six sections of the course during 2018 (three in Fall Semester, three in Spring Semester) and seven sections during 2019 (three in Fall Semester, three in Spring Semester, and one in Summer Session). Each lab section has a maximum of 24 students, and the total enrollment for all lab sections from 2018 through 2019 was 276 students. Lab content includes topics such as sampling and identification of various taxonomic groups, water quality, wastewater treatment, tree identification and population density, macroinvertebrate diversity, toxicity testing, and soil properties.
Ideas related to Caterpillars Count! were integrated throughout the curriculum, even if a particular lab week was not specifically focused on working on the project (Table 1). For example, an activity to review the metric system used leaves and arthropods for measurements to get students comfortable with these taxa for several weeks prior to data collection for Caterpillars Count!. To introduce the citizen science project, the sampling protocol was reviewed in class, and the students watched a video from the Caterpillars Count! website. Students worked in groups of four, in which each group had one branch assigned per survey circle. When sampling during the lab time, the instructor had the class move from circle to circle together, with students separating within the circle to sample their assigned branch. Generally, students in one lab section could sample three survey circles (15 branches total) during a three-hour lab period. Students in each of the three lab sections sampled the same survey circles. Only two survey circles were sampled during each summer session. Students repeated their sampling at the same survey circles once or twice per semester, depending on scheduling. During the spring semesters, sampling was done late in the semester to allow time for leaves and arthropods to emerge; sampling was done early in the fall semesters before deciduous trees drop their leaves.
Table 1
Example timeline of activities related to the Caterpillars Count! project at ECU.
| ACTIVITY | WEEK IN SEMESTER |
|---|---|
| Arthropod and leaf measurements in metric measurement exercise | 1 |
| Practice estimating lengths in mm in scientific method lab | 2 |
| Arthropod identification using samples of arthropods in lucite Arthropod ID quiz on Caterpillars Count! website | 2 and 3 |
| Leaf/tree identification along a transect | 3 |
| Caterpillars Count! data collection and phenology activity | 4 |
| Caterpillars Count! data visualization exercise | 7 |
North Carolina State University
At NCSU, Caterpillars Count! was added to a three-credit general education introductory biology lecture course (BIO 105, Biology in the Modern World) with student majors coming from outside the College of Sciences. Groups of four to five students were delegated to a single tree within a single survey circle for the duration of the project. With the large class sizes per semester, this allowed 13 survey circles (65 survey branches) to be sampled each semester. Initially (Spring Semester 2018), students performed all data collection in a single class period, meaning that there was only one hour that students spent observing leaves and recording data. Beginning in Fall Semester 2018, data collection was divided into four lecture days, in which students collected attribute data of individual trees such as leaf size and tree size on the first day and then collected arthropod data at their designated tree for the three remaining days. Lecture class time was used for the first day of data collection, but the remaining data collection sessions were conducted at the students’ choice of times on days specified by the instructor. In total, data collection on individual days came to approximately 20 to 30 minutes.
Caterpillars Count! was used in conjunction with the ecology unit of the course wherein students gathered data on arthropod biodiversity, identified down to Order. Prior to arthropod biodiversity data collection, the students took the online identification quizzes, available through the project website until they achieved 90% accuracy, after which they were permitted to collect the data. These arthropod data were then used to calculate biodiversity indices for different sampling sites around campus. Additionally, the experience of seeing the arthropods provided context for discussing the chemical basis of color across arthropods, birds, and humans. Two semesters (Fall 2018 and Spring 2019) had a final project where students compiled all biodiversity data and analyses into a poster that was presented during a class-wide poster session.
University of North Carolina Chapel Hill
At UNC, the project was added to a mixed-majors introductory biology lecture course called Principles of Biology. The course is a three-credit-hour general education course with an optional one-credit lab. The study included two large sections of the lecture component of Biol 101 in Spring Semester 2018 (557 students) and Spring Semester 2019 (555 students), with each section taught by a different instructor, but the same instructors during each of these semesters.
Caterpillars Count! was incorporated into the broad context of the course in the Vision and Change () theme of “interactions within and between systems.” Students were introduced to phenology at the beginning of the semester as they learned about the process of science. Near the end of the semester, in a unit around biodiversity (and when trees began to bud), students were introduced to the concept of phenology again with a lesson by Allen Hurlbert (co-author and project founder of Caterpillars Count!). During the lesson, students learned why the project was significant, practiced answering questions using the data sets, and learned how to collect data. Students completed a homework assignment that included basic arthropod identification, which was necessary for data collection. Students worked in groups of about five to sample all five survey branches within an assigned survey circle, a step that took each group approximately 30 minutes total for the semester. A total of 24 circles were created, with each group sampling only once. Because there were so many students each semester, several groups were assigned the same circle over a two-week period. As such, each circle was sampled, on average, once per day, each time by a new group. Approximately 240 surveys were conducted per semester over the two-week period. Besides completing a pre- and post-assessment of scientific literacy and entering data collected, students also wrote short responses reflecting on the value of citizen science.
Study design and instruments
Student surveys
We created a pre- and post-assessment for students that included some original questions related to project content and drew some questions from a published test of science literacy from Gormally, Brickman, and Lutz (). The short assessment tested ideas such as phenological mismatches, confounding factors, and how to interpret data. The questions were vetted by at least three biology faculty before being included in the survey. Students completed the assessment immediately before the citizen science project data collection began and again immediately after their participation in the project ended to try to minimize the effects of the course overall, separate from the citizen science implementation. In total, more than 1,400 undergraduates participated in the citizen science project during 2018 and 2019, with 1,201 students completing the pre- and post-assessment (Table 2). Wilcoxon Rank Sums Tests were used to examine differences between pre- and post-survey scores for each survey item, as well as the total on all items on a survey. Separate analyses were conducted for each university, as the project was implemented differently in each setting.
Table 2
Although more than 1,400 undergraduates participated in Caterpillars Count! during 2018 and 2019 at these universities, only 1,201 students completed both the pre- and post-assessment (shown here by university). Zeros indicate semesters or summer sessions in which the citizen science project was not included in the course.
| NUMBER OF STUDENTS | |||||
|---|---|---|---|---|---|
| SPRING 2018 | FALL 2018 | SPRING 2019 | SUMMER 2019 | FALL 2019 | |
| ECU | 63 | 23 | 36 | 9 | 28 |
| NCSU | 71 | 270 | 195 | 0 | 0 |
| UNC | 275 | 0 | 231 | 0 | 0 |
Reflections
During the Spring Semesters of 2018 and 2019, students at UNC were asked to respond to a writing prompt, in addition to completing the pre- and post-assessment. This prompt was “What do you think is the value of citizen science projects?” A total of 893 students submitted their responses individually in the course learning management system in two course sections during Spring 2018 and two sections during Spring 2019. We created codes on the basis of the citizen science literature and added emergent codes on the basis of the student reflections. We then labeled each student response as matching one or more codes using NVivo (QSR International). We went through the student responses multiple times until the coding scheme seemed to best reflect the data. Finally, we synthesized the codes and data to formulate broad themes in the responses.
Results
Scientific data collection
Incorporating Caterpillars Count! into undergraduate courses at these three institutions has added a large quantity of data to the national project database of arthropod abundance. In total, 2,801 visual surveys and 2,516 beat sheet surveys were conducted by groups of students at these three universities. They documented 13,759 total arthropods on their campuses.
Student surveys
All three institutions showed gains in the percent of students correctly identifying the answers after completing the citizen science project (W = 2107598.5, Z = 9.2913, p < 0.0001; Figure 1). The questions showing the largest gains at all three institutions were most related to the citizen science project, including concepts related to arthropod identification and a proper understanding of what citizen science is (Supplemental Table 1). Other question topics showing gains differed by institution, probably due to differences in emphasis during teaching.
Box plots depicting the pre- and post-scores of student content knowledge and science literacy for each institution separately (a = East Carolina University [ECU], b = North Carolina State University [NCSU], c = University of North Carolina Chapel Hill [UNC]). Medians are shown with a solid line and the means are shown with a dotted line.
East Carolina University surveys
A significantly higher percentage of students at ECU correctly identified the answers to content and citizen science questions after completing the citizen science project (W = 38554, Z = 5.7599, p < 0.0001; Figure 1), for an average 11.7% increase at ECU (Figure 1). The largest gains were made on questions about citizen science, phenological mismatches, using evidence to test hypotheses, and arthropod identification (Supplemental Table 1).
North Carolina State University surveys
A significantly higher percentage of students at NCSU correctly identified the answers to content and citizen science questions after completing the citizen science project (W = 430343.5, Z = -5.3014, p < 0.0001; Figure 1), for an average 6.3% increase at NCSU (Figure 1). The largest gains were made on questions about citizen science, arthropod identification and function, phenological mismatches, and graph interpretation (Supplemental Table 1).
University of North Carolina Chapel Hill surveys
Students showed an average 7.9% increase at UNC (Figure 1), with a significantly higher percentage of students correctly identifying the answers to content and citizen science questions after completing the citizen science project (W = 275576, Z = 6.0484, p < 0.0001; Figure 1). The largest gains were made on questions about citizen science, phenological mismatches, arthropod identification and function in an ecosystem, and using evidence to test hypotheses (Supplemental Table 1).
Student reflections
Common themes emerging from student responses included: benefits to the student, benefits to the environment, benefits to the community, and data benefits. Benefits to the student included the fact that participation in the citizen science project reinforced class concepts, was enjoyable, provided hands-on opportunities, or changed student perceptions (e.g., not all research is done by a PhD with a lab coat, increased desire to participate in more citizen science, biology includes more than observing microbes under a microscope). Benefits to the environment included topics related to appreciation or learning (e.g., “It’s good for people to go out every once in a while to look at nature”), environmental awareness (e.g., “… see the impact of climate change on a local level”), or environmental action (e.g., “could lead to some sort of public policy to better allow humans and nature to coexist”). The theme of benefits to the community describes the idea of the public contributing to science, making a difference in the community, or teamwork (e.g., bringing together individuals with similar interests). The final theme, data benefits, included the ideas of more data being collected over larger geographic areas and increasing the speed of scientific discovery.
Some student reflections fit multiple code categories and represent broader and deeper thinking (hereafter called “deep thinking”), whereas other responses fit only one or two code categories, representing more surface-level thinking (hereafter called “surface thinking”). Responses associated with deeper thinking generally describe positive experiences for the individual, such as gaining hands-on experience and finding the project engaging, but also include aspects of contributing to the community. For example, one student wrote,
“I think citizen science projects help people outside the STEM field realize the importance of science and recognize some current environmental issues. If everyone did this project, they would realize — like I did — that climate change is affecting more right now than we think about on a daily basis. Citizen science is also a great way to get lots of data collected. I think if more people were involved with citizen science projects, there would be a larger base of funds for R&D, and there would be a greater appreciation for STEM research.”
Whereas reflections associated with surface thinking were generally short and focused generally on logistical benefits (e.g., citizen science allows the collection of data while saving money and resources) or impacts on the individual.
“I benefitted by learning the different types of bugs there are.”
“More people to aid in collecting more data….”
Although the vast majority of responses contained positive feelings about their participation in citizen science, some negative perceptions were described. These negative expressions could generally be grouped into one of two categories: personal lack of interest or frustrations with their student group (often in relation to coordinating times to get together or not trusting their fellow students to collect accurate data).
“….i learned [spelling corrected] citizen science is not for me.”
“The only downside to citizen science projects is that there is no oversight or screening process on who collects data, so some data may be recorded improperly.”
In addition, a small number of students made conclusions that may reflect misunderstandings of the scientific process. For example, one student wrote,
“Since all data is collected using the same method, the scientists do not have to worry about data reliability.”
Discussion
Impacts on participants
Pre- and post-assessment data and student reflections provide evidence that participation in citizen science benefited students. When asked, instructors perceived involvement in Caterpillars Count! to be an effective way to get students interested in science, to teach students new concepts, and to introduce them to the idea of citizen science. The assessment results support these instructor perceptions. Students at all three institutions showed content gains and increased familiarity with citizen science. As expected, student understanding of the topics most closely associated with the project (e.g., arthropod identification, phenology) showed the largest gains. Longer -term involvement, participation in multiple projects, or additional educational support may be needed to produce more substantial increases in overall scientific literacy. Student reflection data also expressed that their participation in citizen science increased their engagement in the course, reinforced course content, and sometimes changed their perceptions of science.
These results agree with student reflections and instructor perceptions on citizen science in higher education from existing literature, that undergraduates learn and increase motivation from their course involvement in citizen science. Importantly, our results are one of the very few that include assessment results. Vitone et al. () used both quantitative and qualitative assessments to conclude that involvement in citizen science can positively alter students’ attitudes toward science, but not scientific content knowledge, and suggested that their quantitative approach may not have been powerful enough to detect content-based learning gains.
All three institutions integrated the Caterpillars Count! project into a modest portion of their courses. Each university emphasized different aspects of the project, however, and the course settings (e.g., lecture versus lab) and student populations (proportion of non-science majors, class sizes, student demographics) varied. Thus, basic content gains may be less dependent upon the exact method of citizen science implementation within a course and based more on the general inclusion of the project. This may ease some of the pressure on faculty to find the “right way” to incorporate citizen science into a course and may provide support for the inclusion of these types of projects with undergraduates.
Other impacts
Undergraduate students from these three courses documented more than 13,000 arthropods in this two-year period, with modest amounts of time spent on arthropod surveys. This wealth of data is invaluable to scientists eager to understand patterns of arthropod biodiversity and how arthropod phenology may be shifting in response to climate change. For example, these data have led to the conclusion that caterpillars are much less common on these university campuses than in the surrounding areas. In fact, only 118 caterpillars have been found in total on all three campuses out of the 13,759 arthropods counted. This paucity of caterpillars is biologically significant because caterpillars are a prized food source for consumers such as birds. This result is compelling, but future research will be needed to determine if it is due to ecological differences (e.g., land and pest management, tree species composition, etc.) or to sampling artifacts.
These data are available for students (at any institution) to use. There are multitudes of ecological questions that could be addressed with this data set by current and future students in these, and other, courses. Thus, citizen science data sets, such as this one, provide diverse options for instructors wishing to have their students manage large data sets, ask independent questions, or practice creating data visualizations. An additional positive outcome of this shared participation in citizen science is that the amount of dialogue between instructors at different universities about evidence-based teaching practices has increased dramatically.
Challenges and limitations
The instructors in this study described two logistical challenges to using Caterpillars Count! with their undergraduates. One is that students are asked to estimate the amount of leaf herbivory during their surveys, but the ability to estimate with confidence requires practice. Thus, instructors may need to include time for students to practice estimating herbivory on real or artificial leaves before beginning data collection for the project. In addition, the timing of data collection can be awkward with an academic calendar as peak arthropod abundance generally is in the summer when classes may not meet. Moreover, leaves may not be out in the spring early enough for data collection depending on the latitude of a particular institution. Thus, instructors must carefully consider the project parameters and timing to determine suitability for their course.
The educational data collected here have some limitations. As is common in education research studies implementing new techniques, we did not include a comparison group. By including multiple courses at multiple universities that differ in student population, instructor method, and course content, though, we have increased the generality of our findings. In addition, our evaluation tool was not thoroughly validated, though we did receive input from multiple faculty members and included some questions from a published, validated assessment of science literacy ().
Conclusions
In conclusion, undergraduate course participation in citizen science led to: 1) greater biology content knowledge and scientific process knowledge among undergraduates, 2) the collection of many novel ecological data that are available to answer a wide range of questions, and 3) increased collaboration and dialogue about high-impact teaching practices among course instructors at multiple institutions. Thus, the inclusion of Caterpillars Count! in undergraduate courses was beneficial to the undergraduate participants, scientists using project data, and future students accessing the data.
Data Accessibilty statement
Student-generated scientific data is available through the Caterpillars Count! project website (https://caterpillarscount.unc.edu/). Requests for human subject educational research data can be made by email (vancechalcrafth@ecu.edu).
Supplementary File
The supplementary file for this article can be found as follows:
Supplemental Table 1Wilcoxon Rank Sums Test results on differences between pre- and post-survey scores for each survey item. DOI: https://doi.org/10.5334/cstp.424.s1