Self-determination theory

Despite this promise, it is unlikely that science literacy benefits will accrue if higher education students are not motivated to participate in CS. Even though studies have examined motivations of volunteers in the general public (e.g., Bowser et al. 2013; Crowston and Prestopnick 2013; Hiller and Kitsantas 2016; Silva et al. 2016), these studies are of free-choice volunteers and, therefore, cannot be transferred to students fulfilling graduation requirements. This paper uses self-determination theory (SDT) (Deci and Ryan 2012; Ryan and Deci 2020) to address this gap in the literature by exploring factors promoting motivation among undergraduates to continue doing CS in the future following an initial educational experience.

SDT directly addresses intrinsic (internal) motivation through multilayered theories that attempt to identify individual components of motivation. Within the SDT framework, three psychological needs—relatedness (how much the student connects the educational experience to their lives), autonomy (how much freedom of choice the student has within the educational experience), and competence (how much the student feels they understand the educational experience)—must be met to promote intrinsic motivation. Common external motivators, such as grades, performance evaluations, or requirements to present to the class, can also impact internal motivation in classrooms (Ryan and Deci 2020). Fully understanding the motivations of students to pursue CS in their futures based on the educational experience of doing CS in the higher education classroom requires an attempt to understand how the structure of the CS educational experience aligns with the psychological needs of SDT.

For undergraduates in introductory biology courses aimed at non-majors, competence and relatedness may be particularly important considerations. One challenge facing students in introductory biology courses is the sheer breadth of content to master: topics typically range from the microscopic scale of biochemistry, DNA, and cells to the macroscopic scale of anatomy, predator-prey relationships, and biogeography. These topics require different kinds of content knowledge, data collection techniques, and even technology to investigate. Although these diverse topics can be matched to CS projects (e.g., Wiggins and Crowston 2015; Vance-Chalcraft et al. ‘in press’), the literature has not yet explored whether students relate to CS differently depending on the topic at hand. Given that studies on course-based undergraduate research projects have suggested that computer-based molecular labs and bench-based ecological labs can differentially affect undergraduates’ attitudes (Kirkpatrick et al. 2019), it is reasonable to wonder whether the same can be said for molecular biology or ecological CS projects. Thus, we wanted to know:

1. Does the scale of CS subjects, either molecular or ecological, relative to the “human scale” size of students, differentially impact undergraduates’ willingness to participate in future CS activities?
2. To what extent do salient student-level characteristics (e.g., STEM or non-STEM focus, major, course grade) influence any effect related to molecular/ecological scale?

To address these questions, we present the results of a multilevel mixed model using undergraduate survey responses following CS experiences in an introductory biology class for non-majors. In response to the first question, we hypothesized that we would detect a significant difference in motivation for future participation in ecological versus molecular biology CS projects owing to greater familiarity with the larger-sized research subjects featured in ecological projects. Students interact with macro-scale ecological subjects, like squirrels and birds, in their everyday lives, whereas molecular biology subjects, such as protein sequences and cells, are not visible to the unaided eye and thus are more abstract. We anticipated that this difference would address the SDT requirements of relatedness and competence for motivation in that students would better relate to familiar subjects and would feel more competent in their knowledge about these same subjects. Regarding the second question, on the basis of prior research about student characteristics in introductory biology courses (e.g., Cotner, Thompson, and Wright 2017), we anticipated that student major tracks and individual student course grades would also contribute to any observed effects. In particular, we expected STEM-focused majors to positively contribute to student motivation because the coursework potentially related to a domain of interest and students in these majors already had competence in science.

Data collection

Data collection about the attitudes of higher education students using ecological versus molecular biological CS projects was accomplished through surveys of students enrolled in the introductory biology class for non-majors at a large research university in the southeastern United States between the years 2016 and 2019. Class sizes averaged 156 students during the study period (minimum enrollment = 34, maximum enrollment = 234).

Twice per semester, students were assigned to: 1) choose one of the instructor-selected CS projects available on the SciStarter platform (Hoffman et al. 2017; Supplemental Table 1); 2) perform the CS project for as long as they wished; and then 3) complete a survey about their experience. One assignment focused on ecological projects and the other on projects that had molecular biological research objectives. The same projects were included each semester unless one was cancelled by the CS project managers. In those cases, another project took its place (see Table 1). In both the Spring and Fall semesters of 2016, students completed the molecular biological projects first and ecology projects second. For years 2017–2019, the ecology CS projects were done first during the Fall semesters, whereas during the Spring semesters, students typically performed the molecular biological projects first. This was because of shifts in the curricular order between Fall and Spring semesters. In line with the SDT principle of autonomy (Deci and Ryan 2012; Ryan and Deci 2020), students could select a project that most closely matched their interests.

Students received complete credit for submitting the survey irrespective of the time spent doing CS or the quality of their survey responses. Knowing that the assignment was credit-only might have discouraged some students from putting forth their full effort, potentially biasing the data in unknown ways. For example, one known issue with having CS tied to a required course assignment is the potential for erroneous data collected by unenthusiastic students (Mitchell et al. 2017). SDT also suggests that having a task-dependent reward, such as a grade, attached to an assignment will generally have negative effects on intrinsic motivation for the assignment (Ryan and Deci 2020). The instructor attempted to mitigate this hurdle by eliminating the judgment of quality of their survey responses. Freedom from a grading rubric might have allowed students to pursue the CS project on their own terms, adding to the autonomous principle of rising intrinsic motivation within SDT.

Although negativity could arise from a graded CS assignment, binding this assignment to a course requirement likely increased the number of students contributing data compared with simply offering extra credit. As such, responses may better reflect the cross-sectional views of the class in general.

This paper is a secondary analysis of de-identified data from the required survey assignments matched to student course grade and major. Per IRB, identifying information such as name, age, year in school, and primary language were not made available, nor were known student-level factors such as gender (e.g., Eddy, Brownell, and Wenderoth 2014; Cooper and Brownell 2016; Matz et al 2017), race (e.g., Carlone and Johnson 2007; Chen et al 2021), or first-generation-student status (e.g., Harackiewicz et al 2014). The original survey consisted of three Likert-type items with five response options each and eight open-ended responses. Surveys also collected information about the course section and the name of the selected project. This analysis uses the three Likert-type items, course section identifier, project name, student grade, and student major.

Multilevel modeling of complex data structures in education

Classic linear regression assumes that residuals are independent; in other words, that each observation represents a unique sample. However, social science data, particularly in education, frequently violate this assumption (Rabe-Heskreth and Skrondal 2012). For instance, imagine a scenario in which all the children in a neighborhood are assigned to the same school. In this scenario, students are nested within the neighborhood which is nested within the school. A multilevel approach attempts to tease apart the effects of student-level factors (e.g., prior knowledge, motivation, gender) versus the effects of neighborhood-level factors (e.g., unemployment rates, number of grocery stores) versus the effects of school-level factors (e.g., teacher turnover, relative poverty, school-wide programming) by partitioning out variance among the structural levels (Bryk and Raudenbush 1988).

In some instances, data are multilevel but not hierarchical: some schools may enroll students from across multiple neighborhoods and some of those neighborhoods may send students to multiple schools. Students can also move to new schools or neighborhoods (Garner and Raudenbush 1991). In this case, students are cross-classified within neighborhoods and also within schools, neither of which is subordinate to the other. Approaching such cases without addressing the cross-classification can lead to severely biased results (Ye and Daniel 2017), so analyses should proceed using multi-level multiple-membership techniques designed to address cross-classification.

Recognizing the statistical complexity of our data, we predicted that student endorsement of future CS participation is influenced by major, individual achievement, and the size of CS project subjects; mediated by class section and project attempted; and represented as a cross-classification multiple membership model:

DOAGAIN_ijk = β₀ + β₁MAJORijk + β₂GRADE_i + β₃MICROBIOi + u⁽³⁾_CLASS(i) +

                    u⁽²⁾_PROJECT(i) + е_i

                u⁽³⁾_CLASS(i)~ N(0, σ²_u(3))

                u⁽²⁾_PROJECT(i)~ N(0, σ²_u(2))

                        e_i ~ N(0, σ²_e)

Table 2 displays key student- and project-level variables used in the analysis.

More than 40 majors were represented, ranging from 1 case (several) to 177 cases (Psychology), with half of the reported majors having ten cases or fewer. We aggregated majors to the college level and encoded them as a series of dummy variables: PCM (College of Management, n = 490), COS (College of Sciences, n = 71), CHASS (College of Humanities, Arts, and Social Sciences, n = 1,022), CNR (College of Natural Resources, n = 127), CALS (College of Agriculture and Life Sciences, n = 119), TEXTILES (College of Textiles, n = 12), ENGCOMPSCI (College of Engineering, n = 145), ARTDES (College of Art and Design, n = 81), ED (College of Education, n = 333), TRANSFER (students indicated that they were transferring between colleges, n = 62), and OTHER (no major, undeclared, not seeking a degree, n = 598). Dual-major students were counted in the data for each college in which they were enrolled.

One case in the data set lacked a course grade. Thirteen other cases lacked the response variable. These cases were distributed across class sections and projects without a discernable pattern, so they were removed from analysis. Analyses were performed using STATA 16 (StataCorp 2019).

After cleaning, the data consisted of 2,962 individual observations from 1,654 unique students across 14 class sections (µ = 211.517 observations) and 12 projects (µ = 246.833 observations). More than 100 more observations were made for the molecular projects (n = 1,547) than for the ecology projects (n = 1,415). Table 3 presents the unclustered means and standard deviations for the three Likert-type items DOAGAIN, EASY, and PROJRATE. Although EASY measured perceptions of SciStarter rather than CS, we included it because all CS assignments in the course were exclusively coupled with SciStarter. As research on search engines and digital learning systems suggests ease of use can significantly impact students’ perceptions of their subject-matter learning (e.g., Cheng, 2019) and future use (e.g., Lavidas et al. 2019; Tseng, 2020), SciStarter might have affected the critical SDT principle of competence and thus might have impacted students’ motivation to participate in future CS projects.

Analysis and Results

The overall mean for CS projects was 2,930 on a 1–5 scale (median score = 3, interquartile range = 2). We used a nonparametric Friedman test to compare unclustered Likert-type data from molecular biology projects (µDOAGAIN = 2.858) to ecology projects (µDOAGAIN = 3.009). Results suggested a small statistically significant difference between the two groups (Q = 10.756, p < 0.010).

Kruskal-Wallis tests revealed significant differences in the ratings for future participation (DOAGAIN) among classes (𝝌² = 73.290, 13 d.f., p < 0.001) and among projects (𝝌² = 45.490, 11 d.f., p < 0.001), suggesting possible clustering effects related to each. Therefore, we retained both clustering variables in our model. Cross-tabulation revealed that each project received observations from 2 to 14 classes, and each class contributed to between 2 and 10 projects

Following Leckie (2013), the unconditional model was conceived as a two-level cross-classified model with students nested in classes and projects (Figure 1). A likelihood ratio (L-R) test (𝝌² = 47.540, 1 d.f.; p < 0.0001) indicated that a multi-level model was preferred to a simpler linear regression. L-R tests comparing the unconditional model to separate students-within-classes (𝝌² = 4.260, 1 d.f.; p < 0.05) and students-within-projects (𝝌² = 26.180, 1 d.f.; p < 0.001) models indicated that class- and project-level clustering were significant and should be retained. Clustering to account for possible class-project interactions did not significantly improve model fit (𝝌² = 0.140, 1 d.f.; p = 0.706) and therefore was dropped to avoid overparameterization. This resulted in a fully unconditional model (M1 in Table 4) with approximately 2.5% of the variance ascribed to class effects, 0.5% of the variance ascribed to project effects (ICC = 0.031), and µDOAGAIN = 2.920.

Next, student factors were inputted into the model (M2 in Table 4). How user-friendly students rated the SciStarter interface (EASY) and how a student rated an individual project (PROJRATE) were identified as significantly contributing to variance in the ratings for future CS participation (DOAGAIN; all p’s < 0.0001). A L-R test suggested that adding student-level covariates significantly improved the model fit compared to the unconditional model (𝝌² = 1602.240, 15 d.f.; p < 0.0001). Given that we would be using the same data set and large sample size (n/p > 40) for all models, we opted to use the AIC score (Akaike 1998) rather than the AICc metric to guide model selection (Burnham and Anderson 2004). AIC scores for each model (M1 = 9207.242, M2 = 7637.005) also indicated improved fit over the fully unconditional model.

To understand project effects separate from student-level effects, we ran the full model, including the project-level designation of ecological or microbiological subjects (MICROBIO) as well as the project dummy variables to tease out any differences among specific projects (M3). The full model returned a similar pattern to the student-level model, with only ease of use (EASY) and project rating (PROJRATE) identified as significant (all p’s < 0.0001). The project-level factor MICROBIO was no longer shown to be significant once we accounted for clustering. A L-R test comparing M2 and M3 did not indicate a significant difference between the two (𝝌² = 15.950, 12 d.f.; p = 0.193), and including all covariates actually increased the M3 AIC score.

We then employed backward stepwise deletion to fit the model using AIC scores (M4). Ease of use (EASY; p < 0.0001), project rating (PROJRATE; p < 0.0001) and enrollment in a teacher-certification program (ED; p < 0.050) were identified as significant student-level factors. Final course grade (GRADE) and enrollment in either the College of Sciences (COS) or Engineering (ENGCOMPSCI) were not significant, but were retained owing to AIC. Although MICROBIO dropped out of the model, three projects (GENES, NOVARNA, and MARKBIRD) were identified as significant project-level factors (all p’s < 0.050). A fourth project, NANOCRAFT, though not significant, was also retained owing to AIC.

Results should be interpreted cautiously, particularly when considering the between-group effects, owing to the small numbers of class sections and projects (Luo 2013; Chung et al. 2018). Furthermore, doing multi-level analysis with Likert-type responses tends to underestimate variances, thus increasing chances of a Type II error (e.g., Beal and Dawson 2007; Grilli and Rampichini 2011).

Content focus in citizen science may be less important than project structure

Though content focus (molecular biological versus ecological) was not significant on its own, three specific projects in molecular biology and ecology significantly affected how students viewed their future participation: Mark My Bird, NOVA RNA Lab, and Genes in Space. Assessment of the qualitative responses by students is yet to be performed, but we provide preliminary suggestions that may partially explain commonalities between these projects. It seems that for the latter two projects, the video game interfaces positively influenced student perceptions. Students also appreciated how easy Genes in Space was to understand. On the other hand, many students who participated in Mark My Bird found the instructions confusing even as they commented positively on the interesting way to observe bird diversity.

These responses highlight the importance of relatedness and competence, though in ways we had not anticipated. Even though we had hypothesized that subject matter would be key, students responded to design features that felt familiar (e.g., video game interfaces), sparked their interest (e.g., novel observation techniques), and helped them complete a task (e.g., instructions). Since students had some autonomy to select projects, we cannot rule out our original hypothesis about subject matter, but this finding suggests we need to expand our thinking around the sources of relatedness in CS project design.

One question that our results raise is whether and how virtual versus fieldwork tasks impact motivation. Unfortunately, we were not able to compare fieldwork to virtual projects because BirdEEZ was the only project that did not have a virtual option available for students. However, future research can leverage projects like Squirrel Mapper that offer both fieldwork and virtual task options. Even just within the virtual realm, there are a number of task design approaches that are worth exploring, including extreme gamification to the point of being video games (e.g., Genes in Space), more directly foregrounding the science in reconstructing the study subject (e.g., Phylo), classifying photographs (Squirrel Mapper), and using digital mapping tools (Mark My Bird). Utilizing planned contrasts and random assignments to explore how students relate to such specific design features, how different choices affect student competence, and how these work in tandem to promote internal motivation for CS can yield critical insights for educators and project designers alike.

Major had mixed influences on students’ perceptions of future citizen science participation

On the basis of the work done by Cotner and colleagues (2017) to explore differences between students majoring in biology versus those who are non-STEM majors, we had expected that enrollment in the College of Sciences (COS) and College of Engineering (ENGCOMPSCI) would be associated with student endorsement of future CS participation, even if not significantly. However, it was surprising that enrollment in the College of Life and Agricultural Sciences (CALS) or the College of Natural Resources (CNR) were not retained in the final model. Even more surprising was that enrollment in a teacher-certification program (ED) was the only major-related factor that returned as significant.

Considered from an SDT perspective, however, these results align with the principles of relatedness and competence. To begin, prior research with pre-service teachers has shown they approach CS as a tool to use with their own students (Scott 2016). The education students in our study might have approached this assignment similarly, endorsing future participation in CS as a function of their intended career. This result suggests that relatedness, combined with a desire for competence, may motivate CS participation among pre-service teachers.

However, when we reviewed the pool of students from CALS and CNR, we realized the available CS choices were less clearly tied to outcomes emphasized in programs such as Agribusiness Management (CALS) and Sports Management (CNR). Students enrolled in these programs may not have perceived the project choices available to them in this particular course as related or relevant to their own goals. Students in these STEM-adjacent, management-focused programs may also have felt less competent in science than we anticipated on the basis of college enrollment alone.

These results suggest improvements for future surveys. First, adding a pre-survey can identify changes in motivation over time due to CS experiences in class. Second, adding items about expectations can help provide insight into relatedness. Third, adding items about prior science coursework and major-required coursework in science can provide insight into competence and inform researchers looking for ways to aggregate majors based on STEM intensity rather than administrative units.

Individual experiences may not align to attitudes about future participation

Lastly, we were surprised that the mean ratings for individual projects (PROJRATE) and for perceived ease of use (EASY) were higher than those for future CS participation (DOAGAIN) across categories (see Table 3) because we supposed that a student would rate the likelihood of repeating a positive experience at least as high as the experience itself and because of research highlighting the importance of ease of use for digital learning tools (Lavidas et al. 2019). In light of the results, examining the disconnect between individual educational experiences with CS and future CS participation through SDT is instructive. A single encounter with a CS project, as designated in the assignment studied here, likely did not invoke feelings of mastery or competence. Moreover, undergraduates might not have perceived the CS projects offered to them as sufficiently related to their own lives, interests, goals, or even the course content to warrant future participation. As Ryan and Deci (2020) show, if basic psychological needs for relatedness and competence are not met in the classroom, internal motivation suffers.

In light of these experimental results and theoretical considerations, we propose that instructors wishing to use CS in their classrooms take careful stock of the relationships among the projects chosen, the class activities, and the lives of the students to intentionally ground assignments in the principles of relatedness and competence. Some possible ways to more fully integrate CS projects into the educational experience of students, and thereby increase relatedness and competence around CS, include reading project-related media and journal articles, learning about the research team, mapping contributions to the project, using collected CS data in class, etc. These recommendations align with the National Academies’ recommendations that educators can support learning through CS by intentionally leveraging the built-in structures and supports that already exist in classrooms (2018).

Implications for future research

The CS survey was planned to be a non-random instrument used in class rather than a data collection instrument for research. Although the instructor tried to provide autonomy by way of having students choose the projects and the amount of time they attempted each project, attaching an assignment grade (albeit credit/no credit) might have made some students feel as though they were not in control, which could lead to negative feelings about the CS experience (Vance-Chalcraft et al. ‘in press’). Conversely, connecting the assignment to a grade might have brought in participants who would not have otherwise tried CS. Additional research is necessary to understand the pros and cons of increasing students’ exposure to CS through external motivators such as grades.

The issue of non-random assignment raises another obvious extension of this study. To better understand the kinds of CS experiences that contribute toward increased science literacy in non-STEM majors, future research should methodically test different content orientations and project features to explore how those combinations affect intrinsic motivation. Though prior research about CS in higher education has focused largely on general principles, our research shows how studies can provide specific design feedback. Thus, we add our voices to literature calling for research into identifying project design features that best encourage future CS participation (e.g., Bonney et al. 2009; Tinati et al. 2015; National Academies 2018; Golumbic, Baram-Tsabari, and Koichu 2020).

Such research can put students in positions of responsibility for improving CS and could lead to the kinds of cooperation among students, course instructors, and CS project management described by Vance-Chalcraft et al. (‘in press’) and the National Academies (2018). However, for such collaborations to occur, it will be necessary for project managers to have open channels of communication with users of their project (e.g., Golumbic, Baram-Tsabari, and Koichu 2020). In fact, opening communication between project users and the sponsoring research team could alleviate some power differentials between these parties, thereby making CS even more equitable and ethical (e.g., Riesch and Potter 2014).