Citizen Science, Learning, and Territories

According to the Organisation for Economic Co-operation and Development or OECD (2020) and the European Centre for the Development of Vocational Training or Cedefop (2014), learning can be broadly divided into formal, informal, and nonformal learning. Formal learning refers to learning in an environment that is organized and structured such as a school. It includes learning objectives, learning materials (e.g., textbooks, iPads), and learning resources (e.g., teachers, a tutoring service) that have been allocated to support learning. Learning is intentional from the perspective of the learner, who usually receives certification upon completion such as a diploma. In contrast, informal learning is situational, or experience-based, such as talking with co-workers at the workplace. Learners are not proactively thinking of their own learning in the same way as in formal learning; hence, it is usually unintentional. It does not include learning objectives, and learning materials are not allocated to learning. However, resources such as counseling (if provided) may contribute to learning. Informal learning may sometimes be certified, such as in the completion of a series of online quizzes on, for example, plant species (Cedefop 2014; OECD 2020).

In between formal and informal learning is nonformal learning. Although its definition is debated, nonformal learning generally refers to a semi-structured or semi-organized environment that supports learning, such as a museum. While learning is embedded in the environment, learning objectives are more implicit (i.e., available but not necessarily promoted), learning materials are present (e.g., booklets, diagrams), and learning resources (e.g., workshops) are available but more limited compared with formal learning. Learning is intentional, but it does not usually result in certification (Cedefop 2014; OECD 2020). Overall, much of our learning involves a mixture of informal and nonformal learning in addition to formal learning (National Research Council 2009).

As we can see, each type of learning involves a varied use of materials and resources in conjunction with opportunities for interaction (i.e., communication between people), which is known as sociomateriality (Orlikowski 2007). The types of learning and their sociomaterial counterpart in CS can be understood through the discussion of territories. According to Kloetzer et al. (2021), territories “indicate different sociomaterial contexts and resources, cultural and institutional values, and, sometimes, the various groups who may take part in citizen science projects (p. 286).” In other words, the territories for learning in CS are constructed through the varying combinations of project characteristics (e.g., values or aims, goals, the research area), individuals (e.g., citizen scientists, policymakers, project stakeholders), resources (e.g., training and support), and materials (e.g., smartphones, sound recorders). Kloetzer et al. (2021) list six different territories about which we explain next in relation to the three types of learning.

Formal learning primarily includes the formal education territory (i.e., schools). CS can occur in schools through its integration with curricula so that learning objectives are defined and learning materials as well as resources are provided. As students (who also act as citizen scientists) are already enrolled in the school, learning is intentional; however, it is not clear whether certification from the school includes a direct connection to CS because competing interests such as ensuring resources (e.g., lesson plans) are applicable to school curricula and CS activities. Informal learning primarily takes place within the family territory. Here, CS projects aim to encourage exploratory learning through interaction (e.g., discussion with others) in everyday situations without the need for acquiring or utilizing certain materials (e.g., measuring devices). Depending on the project, learning resources such as tutoring may be available. Learning itself is not proactively pursued nor established in this territory; hence, learning is meagerly perceived (Kloetzer et al. 2021).

Nonformal learning takes place mainly in the territories of out-of-school education, museums, and local and global communities. Learning remains intentional for out-of-school education programs and museums as both typically support learning through, for example, workshops or courses, which include hands-on materials such as a water sampling kit, even though specific learning objectives are not always explicit. Interestingly, some out-of-school education programs may allow citizen scientists to operate independently from professional scientists during participation while others include apprenticeship, which may pave the way for certification.

For the local and global communities territory, CS projects are flexibly structured, and learning is part self-directed and part collaborative. A community or out-of-school program may provide the necessary materials (e.g., iPads) and resources (e.g., training) to learn, and some may directly collaborate with citizen scientists; however, sometimes the learning materials provided are not used for the learning that is intended. While certification for skills and knowledge obtained are not necessarily commonplace, dialogue between citizen scientists, the community, and project stakeholders may be enhanced and thus lay the foundation for future CS projects (Kloetzer et al. 2021). Overall, CS project initiators seem to find a stronger launchpad in this territory since learning is not compelled, and scientific literacy is not explicitly emphasized as the primary goal or purpose (National Research Council 2009).

The online territory is more difficult to categorize as it includes characteristics from formal, informal, and nonformal learning. Online CS projects such as Grass Gazers may serve as a substitute for when in-class, formal learning is not possible because of, for example, the outbreak of a disease (see Van Haeften et al. 2021). Online CS projects may also provide online materials (e.g., e-books, videos) for self-study as well as resources (e.g., forums, webinars) that allow citizen scientists to connect with other citizen scientists, researchers, policymakers, teachers, and formal education providers such as universities (Aristeidou and Herodotou 2020; Bruckermann et al. 2020; Kloetzer et al. 2021; Roche et al. 2020). Learning can be intentional when enrolling in substitute study programs such as Grass Gazers or unintentional when troubleshooting technical issues during, e.g., data collection (Kloetzer et al. 2021).

Citizen scientists in the online territory are typically self-directed, whether learning objectives are explicitly present or not (Aristeidou and Herodotou 2020), reflecting a more constructivist approach to learning, in which learners autonomously manage their own learning (Packer and Goicoechea 2000). Like the family territory, when learning is not perceived, participation is difficult to sustain (Kloetzer et al. 2021). Learning in online CS is not typically certified, but there have been efforts to address this such as the European Guidelines for Validating Non-Formal and Informal Learning (Cedefop 2016) or the Informal Learning in Citizen Science (ILICS) model by Kloetzer et al. (2013) as mentioned by Aristeidou and Herodotou (2020, p. 2).

A Model for Interpreting the Process of Learning in Citizen Science

Recently, researchers have developed different models to better capture and understand learning in CS. For instance, Phillips et al. (2018) developed a framework to capture and evaluate perceived learning through six categories: “interest, self-efficacy, motivation content, process and nature of science knowledge, skills of science inquiry, and behavior and stewardship (p. 7).” Kloetzer et al. (2021) took a different approach and created a comprehensive “thematic map of volunteers’ learning” in CS by combining prior research from Citizen Cyberlab (Jennett et al. 2016) to focus on how and what is learned as well as the barriers for learning (p. 300). Bruckermann et al. (2020) note that project initiators typically do not focus on the design and evaluation of learning and thus developed the Model for the Design and Evaluation of Learning in CS projects or MODEL-CS. The model functions as a heuristic tool for analyzing CS projects and learning through identifying and connecting the stages of a project: inputs, CS activities, outputs, and outcomes.

At the input stage of a project, professional scientists form learning opportunities based on their goals and motivation as well as those of citizen scientists. Next, citizen scientists utilize the learning opportunities by engaging in one or more CS activities. Sufficient use of CS activities via participation generates measurable and observable outputs for professional scientists to examine. Finally, outputs are then transformed into scientific outcomes or individual learning outcomes (which we refer to as perceived learning), which may (not) correspond with one another. However, it is important to note that the outcomes at the end of a project feedback to professional scientists and citizen scientists and thus shape their current and even future goals and motivations for other projects (Bruckermann et al. 2020, p. 890).

Research Questions

In this study, we investigate the following research questions:

1. What kind of project-related and individual factors can be identified?
2. How are the project-related and individual factors associated with the perceived learning of citizen scientists engaged in CS activities?

Overview

Our study is based on a survey (see Supplemental File 1: Survey instrument, for relevant survey questions) that addressed adult citizen scientists (over the age of 16) in Europe. An adaptation of Lohman’s (2005) informal learning survey and research conducted by Jennett et al. (2016) about learning in CS were used as the basis for creating answer choices. We define citizen scientist as any person who has participated in scientific projects in collaboration with professional scientists by voluntarily, contributing to, for example, data collection, analysis, and/or dissemination of a scientific project or observational campaign, even if the term CS was not explicitly used (Haklay 2013). During its development, we piloted the survey with 11 citizen scientists who provided feedback. Overall, our survey focused on Europe because it was created within the context of the EU Horizon 2020 CS Track project.

Dissemination and approach

The survey (available in 10 languages) was disseminated systematically in Europe via universities and other research institutions, environmental organizations, civil societies, policy makers, volunteering organizations, etc. from January 2021 until July 2021. We also sent direct emails to projects and institutes conducting CS, inviting them to respond to the survey and/or distribute it throughout their networks; hence, principles of snowball sampling were used (see Goodman 1961). The survey was promoted on social media channels and CS-related forums. We also asked the platforms of national and international CS associations for support and promotion of the survey. Throughout the survey dissemination, we were actively searching for best practices of similar survey distribution strategies (e.g., Ganzevoort and Van den Born 2020). We also sent follow-up emails and reminders to encourage more citizen scientists to answer.

Altogether, we focused on 11 CS activities (e.g., asking questions, searching information from the internet, trial and error) and their relationship with one another. We then examined the following project-related factors: initiators, research areas, and goals. For individual factors, we examined participation in CS (in years), regularity of participation, number of projects in which respondents participated, and CS activities in which respondents were engaged in learning. Regarding the project-related factors, it was possible to choose more than one option (i.e., one CS project could relate to many research areas).

Sample characteristics of survey respondents

A total of 1,083 respondents from 38 countries and 5 continents responded to the survey. To detect possible attrition bias, a logistic regression was implemented using the following demographic characteristics obtained from our survey: gender, age, education level, yearly income level, and participation in CS (in years) (see Supplemental File 2: Survey data, Tables A1 and A2). We found that respondents who had been engaged in CS from 6 to 10 years (p < 0.04) or for more than 10 years (p < 0.001) as well as those who identified themselves as male (p < 0.03) were more likely to respond. Those who had a post-secondary non-tertiary education (p < 0.04) were less likely to respond. However, according to the Wald Chi-Squared test, the only demographic characteristic that was significant was how long the respondent had engaged in CS (X² = 26.0 df = 3, p < 0.001). The data analysis for the subsample was based on data with no missing values in considered CS activities (N = 596).

Data analysis

Regarding relevant CS activities, we referred to those from the survey question: “In your experience, to what extent do you feel you have learned something while doing the following activities?” (see Survey instrument, item 21). This Likert-based question listed values from one (not at all) to five (a great deal). The value of six meant that the CS activity was not relevant or possible in the project and was thus considered as missing information. Exploratory factor analysis (EFA) was conducted to determine the relationship and correlation between CS activities as well as to reduce the dimension of response variables. The answer choice of “Other, please specify” covers multiple different CS activities and was also excluded from EFA.

Our subsample size of N = 596 is suitable for factor analysis (Guadagnoli and Velicer 1988). The univariate and multivariate normality within the data set was examined before initiating EFA. Since the data set cannot be considered normally distributed (see Survey data, Table A3), EFA was initiated using Spearman’s rank correlation (see Survey data, Table A4). The suitability of the data set was examined using Kaiser-Meyer-Olkin criterion (Kaiser and Rice 1974). A value close to one indicates high suitability for factor analysis and our value is 0.91.

Figure 1 represents a scree plot of eigenvalues that can be used to determine the extraction of factors (Cattell 1966). The curve drops after the first factor, with another, smaller drop after the fifth factor.

Another criterion for factor extraction is discovering the cumulative percentage of the variance explained. Depending on the discipline, the factors with cumulative percentage from 60 to 80 of the total variances explained are considered usable (Hair et al. 2010, Streiner 1994). In this study, 75% of the total variances were explained after determining five factors. Table 1 shows all the factor components that are extracted.

Factors were not statistically significantly correlated (see Survey data, Table A5), so the loadings of factors were rotated using the Varimax rotation (Kaiser 1958). In this study, the component loadings of 0.35 or higher are included when interpreting the factors. The communality (h²) provides the percent of variance in a variable explained by all common factors. The original CS activities from the survey, rotated factor loadings, and the communalities are presented in Table 2 below. Based on the factor loadings and the earlier discussion on territories (Kloetzer et al. 2021), which is based on sociomateriality (Orlikowski 2007), factor 1 was named sociomaterial learning, factor 2 was named social learning, factor 3 was named reflective learning, factor 4 was named situational learning, and factor 5 was named material learning.

Factor scores were then determined by calculating the matrix production of the factor loading presented in Table 2 and observations were centered. The factor scores were not normally distributed (Shapiro-Wilk normality test: Factor 1 scores W = 0.96, p < 0.001; Factor 2 scores W = 0.96, p < 0.001; Factor 3 scores W = 0.95, p < 0.001; Factor 4 scores W = 0.97, p < 0.001; Factor 5 scores W = 0.95, p < 0.001). We used the non-parametric Kruskal-Wallis test to compare the factor scores between the different groups. We calculated the eta-squared measure (Tomczak and Tomczak 2014) as an effect size for the Kruskal-Wallis test and interpreted it with the thresholds 0.01, 0.06, and 0.14 for small, medium, and large effects respectively (Morse 1999). Initial to the comparisons, we grouped the data based on the project-related factors (e.g., research area of the project) and individual factors (e.g., number of participated projects). The Dunn’s A test (1961) was used for pairwise comparisons (see Survey data, Table A6). A p-value of less than 0.05 was considered as the limit for a statistically significant result.

Since it was possible to select more than one initiator, research area, or goal for a project in the survey (see Survey instrument, items 12, 13, and 14), respondents were first manually grouped based on the project’s research area so that each project, along with their initiator and goal, belonged to one of four project groups: biology, history, other natural sciences, and other research areas.

Next, respondents were clustered using K-modes clustering (Chaturvedi, Green, and Caroll 2001) based on individual factors related to experience in CS (i.e., participation in years, regularity of participation, number of projects in which respondents participated, and CS activities in which respondents were engaged in learning). The number of clusters was selected using the elbow method, in which you identify the point where eigenvalues level off. Since the procedure can yield results in locally optimal solutions (Chaturvedi, Green, and Caroll 2001), we repeated it 200 times to obtain stable clustering.

Project-related and individual factors: groups and clusters

Concerning the first research question (What kind of project-related and individual factors can be identified?), we first manually grouped the respondents based on project-related factors and then clustered respondents based on individual factors. Table 3 presents the project groups and project initiators, in which the project groups are listed vertically (e.g., biology, history, etc.) and the project initiators are listed horizontally (e.g., an association/a non-governmental agency, scientific institution, etc.).

Next, Table 4 presents the project groups and project goals. Like Table 3, the project groups are listed vertically (e.g., biology, history, etc.) and the project goals are listed horizontally (e.g., data/research, education/outreach, etc.).

Regarding participation clusters, we found a four-cluster K-modes solution (Table 5). The descriptive statistics of participation clusters is presented in Table A7 in the Survey data. The four clusters are listed vertically and their corresponding participation characteristics (e.g., participation in CS (years), regularity of participation, etc.) are listed horizontally.

Project-related and individual factors associated with perceived learning

For our second research question (How are the project-related and individual factors associated with the perceived learning of citizen scientists engaged in CS activities?), we addressed how the project groups and participation clusters were associated with perceived learning in CS activities. First, we examined the learning factor scores between the project groups (Table 6).

The highest median in sociomaterial learning was in the other research areas group (median = 1.24). The history (0.76) and other natural sciences groups (0.43) had positive median scores, indicating an above-average perceived learning. The sociomaterial learning score median was negative within the biology group (–0.54), indicating a below average perceived learning. The differences between the groups were statistically significant (Kruskal-Wallis test, Table 6), yet the effect size was small (eta squared = 0.05). A Dunn’s test (1961) for paired comparisons revealed that the most significant differences were between biology and other research areas groups (Z = –4.79, Holm-adjusted p < 0.001), and between the biology and history groups (Z = –3.97, Holm-adjusted p < 0.001).

Social learning score medians were very similar within project groups (Biology, median = 0.41, history, median = 0.41, other research areas, median = 0.41, other natural sciences, median = 0.36). The Kruskal-Wallis test did not indicate differences between project groups (Table 6) and the effect size was small (eta squared = 0.008).

Reflective learning scores were similar within the other natural sciences group (median = 0.45), the other research areas group (median = 0.45), and the history group (median = 0.39). Like the sociomaterial learning score, the reflective learning score median was below zero in the biology group (–0.23). This indicates that perceived reflective learning is below average in this group. According to the Kruskal-Wallis test, the differences between project groups were statistically significant (Table 6), but the effect size was small (eta squared = 0.01). Pairwise comparisons with Dunn’s tests (1961) revealed that the statistically significant difference was between the biology and other natural sciences groups (Z = –2.77, Holm-adjusted p = 0.03).

The situational learning medians were quite similar within project groups (History, median = 0.53, other research areas, median = 0.12, biology, median = 0.05, other natural sciences, median = 0.04). According to the Kruskal-Wallis test, the differences in situational learning scores were not statistically significantly different within project groups (Table 6) and the effect size was small (eta squared = 0.007).

Looking at the perceived learning for material learning, the groups other research areas (median = 0.38), history (0.33) and other natural sciences (0.24) had similar medians. The biology group is the only group presented with a negative median (–0.27), indicating a below average perceived learning for material learning. The differences were statistically significant (Table 6), but the effect size was small (eta squared = 0.03). According to Dunn’s test (1961), the statistically significant differences were between the biology and history groups (Z = –3.35, Holm-adjusted P = 0.004), and between the biology and other research areas groups (Z = –3.69, Holm-adjusted P = 0.001).

Next, we examined the perceived learning score medians in participation clusters (Table 7).

The results in Table 7 indicate that diverse participation in CS activities was associated with higher scores in sociomaterial learning, since clusters two and three have the highest medians (1.24, 2.03). In these clusters, citizen scientists participated in CS often and diversely (see Survey data, Table A7). According to the Kruskal-Wallis test, the differences between clusters were statistically significant (Table 7) and the effect was large (eta squared = 0.21). Further examinations reveal that differences were statistically significant between all the clusters except between clusters one and four, which had the lowest score medians in sociomaterial learning (see Survey data, Table A6).

Similar observations can be seen in other learning factors. For all factors, the third cluster of citizen scientists with long experience in CS, and who participated frequently in multiple CS projects, had the highest perceived learning (social learning median = 1.08, reflective learning median = 1.01, situational learning median = 1.10, material learning median = 0.70). Correspondingly, perceived learning was lower in the clusters that had fewer participated CS projects and a few engaged CS activities. For all factors, the effect size was either medium or large (social learning, eta squared = 0.14; reflective learning, eta squared = 0.10; situational learning, eta squared = 0.14; material learning, eta squared = 0.13). The pairwise differences between factors are statistically significant in all comparisons except between clusters two and three (see Survey data, Table A6). When comparing scores for material learning, the difference between the first and the fourth participation clusters is not statistically significant.