Emergency remote learning and laboratories

Once the World Health Organization (WHO) declared COVID-19 a pandemic, higher education institutions worked quickly to move from in-person courses to emergency remote learning (temporary shift of instructional delivery to an alternative mode due to a crisis; Hodges et al. 2020). Once the COVID-19 pandemic started, instructors that were not already teaching online struggled to make laboratories accessible remotely and to continue to give students authentic, hands-on experiences (Race et al. 2021; Barton 2020; Bacon and Peacock 2021). Laboratory course instructors struggled to find ways to continue inquiry-based, hands-on, and field experiences when the students had to move off campus, and everyone had restrictions in place with stay-at-home orders (Race et al. 2021; Barton 2020; Bacon and Peacock 2021). One way we overcame these limitations of remote learning was to incorporate citizen science, specifically Budburst (Budburst 2020), into our remote teaching version of our introductory biology laboratory courses.

Budburst was founded in 2007 by climate science researchers who wanted to incorporate volunteers to collect data on plant phenology (Budburst 2020). The program was sponsored by the National Science Foundation and run by the National Ecological Observatory Network with the Chicago Botanic Garden (Budburst 2020). One overarching goal of Budburst is to collect data on plant phenology throughout all the seasons to determine how climate change affects these plants (Budburst 2020). The program has allowed participants to not only collect data but also engage with a long-term data set that is free to download from their website.

The purposes of this paper are to describe how we integrated citizen science into a remote teaching laboratory module and to identify the scientific skills students gained as demonstrated by an individual write-up of the project. First, we describe how we converted the citizen science project Budburst into a remote learning experience for an introductory laboratory. Then we assess the skills that the students developed by using long-term data sets from Budburst and students’ data to communicate their findings on phenology. We investigated these skills by scoring an anonymous subset of individual research papers using a validated empirical and representational skills rubric. We predicted that students would develop scientific skills such as experimental design, quantitative methods, acknowledgment of method limitations, and data interpretation, despite the emergency remote nature of the course in spring 2020.

Study location and demographics

This study consisted of students in the integrated introductory biology and chemistry course at an R1 institution in the mid-Atlantic region. The course is the second semester of a two-part introductory course that covers plants, evolution, ecology, and physiology. The students in the course are a majority freshmen (>90%), and are made up of biology majors (40%) and other life science majors including medical diagnostics, exercise science, biomedical engineers, and wildlife and ecology majors.

Remote Learning Budburst Project

This remote-learning version of the Budburst project was adapted from the in-person version (Lansverk et al. 2020) and implemented during spring 2020 when the university moved from in-person teaching to remote learning because of the COVID-19 pandemic on March 12, 2020. This modified version allowed students to complete it while studying at home and following all the safety guidelines put forth by individual states and communities. Students were instructed to follow all local safety precautions. If the students could not collect their own data due to safety restrictions or were in areas that did not have trees, then they used a data set that was collected near the university by the course teaching team. The project continued to be aligned with AAAS Vision and Change competencies (Table 1) (Clemmons et al. 2020). This project was incorporated into the remote learning curriculum to allow students a hands-on experience with collecting field data, using large data sets, and analyzing and interpreting results during a time when all other hands-on activities were limited (Table 1).

The project was completed over a five-week period during remote learning (Table 1) and included the introduction of the project, four weeks of data collection, and one additional week for the students to complete their individual paper assignment (Table 1; see also Supplemental Table 1). The students were provided the research question, “How does temperature affect the phenology of trees?” at the beginning of the project and were introduced to the Budburst website to learn more about phenology and the different stages that occurred during the spring season. Notable changes from the in-person project were the elimination of the physical collaborations of teamwork; instead, students completed the data collection individually by measuring the trees that were found at or near their homes.

The first week in remote learning, students were introduced to Budburst and learned about phenology and what stages could be seen during the spring for deciduous trees. The methods on Budburst instruct participants to identify a phenophase and determine whether a tree is in the early, middle, or late stages by percent. Students recorded percentages weekly to allow them to have a continuous data set and to visualize the changes in phenophase and stage for each of their trees. We worked with the students to develop an Excel data sheet to collect their individual and long-term data. Budburst now has a data sheet available on their website, which did not exist during this course. During the first week, the students took time to investigate the trees in their yard or local neighborhood, and pick three trees of identical species or three trees of different species to study. Students used Seek (www.inaturalist.org/pages/seek_app) from iNaturalist to help identify the trees, contacted a senior laboratory technician in the program that is trained as a botanist (titled: Botanist on Call) for assistance, or spoke with family members who planted the trees in their yard and knew the identity of the trees to determine the focal trees for their study. Students collected data on those specific trees over four weeks. Students were required to take GPS coordinates and photos of the focal trees throughout the four weeks to have visuals and to be able to ask any questions if they needed guidance on phenophase. GPS coordinates were used only during data collection to allow students authentic field research methods, and those coordinates were not shared with other students or included in the paper. Students also could document any issues that arose with the tree or phenophase as a result of weather; for example, there were major storm events in the region that resulted in all the flowers being blown away or even loss of the tree. All of these observations and data collections allowed the students to participate in field sampling, which prompted the students to observe the differences between field and laboratory data. These experiences allowed the students to reflect on the limitations of the data used for this project. Students could submit their data to Budburst but were not required to because of constraints in the species found in their surrounding areas and their collection location (e.g., students outside the USA). Students who could not collect their own tree data included those who lived in areas with especially strict local lockdown regulations.

During the third week of the project, the students were introduced to long-term data sets, and were shown how to search, download, and work with the data in Budburst and Weather Underground (www.wunderground.com/). The students then used the Budburst website to find the closest geographic location and tree species to their own and then downloaded the data. The students had to work through the large data sets to determine what data was needed and if they could find the appropriate phenophase. For temperature data, students used Weather Underground’s long-term data sets to determine how temperatures differed over time. During synchronous online laboratory time, students were taught how to graph the different data types using Excel and to interpret their results (Table 1). Each student wrote up their findings in an individual paper.

The individual paper allowed the students to not just participate in a citizen science project, but also discover how scientists might use the data sets from these projects in their own research, and it allowed students to refine the skill of communicating scientific findings (Supplemental Table 2). The students were able to describe phenophases for their trees and describe their findings for both their individual data set and the data set collected from the Budburst website. The students discussed the limitations of their own field data and the data set from Budburst in the methods and discussion sections. In the discussion, students identified what factors might be affecting phenology besides temperature, and any other limitations that were not directly related to the methods section. Finally, students were supposed to connect their findings to the bigger picture and reflect on how, in general, collecting these data could help study climate change.

Quantitative Coding

The students completed a consent form at the beginning of the semester that encompassed all assignments and surveys completed during their time in the integrated biology and chemistry course, which fulfilled the university’s IRB requirements. The students’ final individual writing assignment was used as the summative assessment for the ecology module in the biology course and for qualitative research assessment. All personal identification was removed and replaced with a random identification number. The researchers selected a random subset (n = 60) of the individual papers to code using an adapted and validated rubric based on the Assessment of Scientific Argumentation in the Classroom (ASAC) to evaluate laboratory argumentation skills (Walker et al. 2018; Walker et al. 2019). The original rubric consisted of 23 skills that we believed second semester biology majors would be able to obtain in the introductory biology laboratory course (Supplemental Table 3). These skills were determined prior to the start of the project.

The researchers were two of the four instructors in the laboratory component who oversaw the graduate teaching assistants but were not involved in the original grading of the individual papers during the course for course credit. The rubric was placed in Qualtrics and set up as a survey to gather all coding responses. We used the rubric that was validated for written argumentation or skills (Lansverk et al. 2020). We revalidated the rubric based on the remote learning paper assignment. We had an outside instructor help us validate the survey for the paper assignment to make sure all coders interpreted the description in the same manner and were not biased based on our knowledge of the project. The two coders completed two more rounds of validation by individually scoring three papers using the descriptions for each rubric item. The inter-rater reliability was then calculated, and if any item did not result in an 85% or greater score, those areas were reviewed and the description for that rubric item was updated until we came to a consensus for all items on the rubric. Once we were confident in the rubric and in the rubric descriptions, and the inter-rater reliability was at or greater than 85%, we each scored 60 random papers. We had a mean of 93% (range: 89% to 100%) for inter-rater reliability for all the papers. The coding was completed by presence or absence of the skill and if students completed those skills to a specific standard. For example, the figures had to have correct axis labels and be the accurate graph type for the data set. Once the scoring was complete, the data were totaled and percentages were calculated to determine how well the students were able to complete each skill.

Data and limitations

When we scored for data collection and visual representation skills, we found most students were able to work both on their own data and on the long-term data set from Budburst. The students were able to determine the limitations of the experiment and the data sets used (100%) and locate relevant information (95%) from Weather Underground and Budburst to help expand the information used to answer their research question and support or reject their hypothesis/prediction (Figure 1). Students created appropriate figures (88.3%) and tables (83.3%) based on the data that needed to be included to discuss their findings (Figure 1). We found that students had a grasp on developing effective figure labels (71.7%) but struggled with the table labels (38.3%) (Figure 1). Finally, we defined troubleshooting technical issues to include weather events that resulted in either the students not being able to collect their data or the students describing how these events affected their trees and their study results. Not many students discussed troubleshooting technical issues (18.3%), because not many students experienced the severe weather events because they were located through the mid-Atlantic region, USA and a few were international.

Experimental Development and Paper Conclusion

Students were able to gather and use the literature in their introductions including in-text citations (93.3%; Figure 2). The experimental development skills included the creation of the hypothesis or prediction statement, the experimental design, interpretation of results, and conclusions (Supplemental Table 4). All the students included in the coding were able to design and incorporate key elements to their experimental design (100%; Figure 2). Even though all students were given a research question at the beginning of the project, only 63.3% of them included the actual question in the paper (Figure 2). Students were able to generate a hypothesis/prediction (95%), create a claim supporting or rejecting the statement (96.7%), and then use the evidence to support their claim (90%) (Figure 2). Some students did not include the interpretations of the figures (76.7%) and tables (66.7%) (Figure 2). In their conclusions, 73.3% of the students were able to discuss possible alternative explanations for the data and possible differences seen between trees of similar species (Figure 2). Students struggled to connect their findings about trees with changes in temperature and to identify additional information needed to support their results from individual and long-term data collection (23.3%; Figure 2).