Scientific knowledge in the field of ecology is increasingly enriched by data acquired by the general public participating in citizen science (CS) programs. Yet, doubts remain about the reliability of such data, in particular when acquired by schoolchildren. We built upon an ongoing CS program, Oak Bodyguards, to assess the ability of schoolchildren to accurately estimate the strength of biotic interactions in terrestrial ecosystems. We used standardized protocols to estimate attack rates on artificial caterpillars and insect herbivory on oak leaves. We compared estimates made by schoolchildren with estimates made by professional scientists who had been trained in predation and herbivory assessments (henceforth, trained scientists), and trained scientists’ estimates with those made by professional scientists with or without expertise (untrained) in predation or herbivory assessment. Compared with trained scientists, both schoolchildren and untrained professional scientists overestimated attack rates, but assessments made by the latter were more consistent. Schoolchildren tended to overestimate insect herbivory, as did untrained professional scientists. Raw data acquired by schoolchildren participating in CS programs therefore require several quality checks by trained professional scientists before being used. However, such data are of no less value than data collected by untrained professional scientists. CS with schoolchildren can be a valuable tool for carrying out ecological research, provided that the data itself is acquired by professional scientists from material collected by citizens.
Scientific knowledge is more accessible than ever before, particularly owing to an increase in open access publications and the outreach activities of scientists worldwide. Still, many topics in life and environmental sciences that are considered settled by scientists are misunderstood by the general public, even among individuals with substantial science literacy and education (Drummond and Fischhoff 2017; Fiske and Dupree 2014; Kahan et al. 2012). Citizen science (CS) programs rely on participation of the general public in scientific research in collaboration with or under the direction of professional scientists (European Commission 2013; Haklay 2015). The rapid development of these programs, in addition to vastly increasing available data, offers an unprecedented opportunity to bridge gaps between science and society by engaging the general public with the process of science and increasing motivation for inquiry and interest in scientific topics.
CS programs in the field of ecology can benefit both science and society (Wals et al. 2014). For professional scientists, involving the general public enables the collection of data on broader spatial and temporal scales than would otherwise be possible (i.e., crowdsourcing). This practice has been recognized as a highly effective way to track various biological phenomena (Dickinson et al. 2012; Schwartz, Betancourt, and Weltzin 2012). Typical CS studies in ecology address the effect of environmental factors on biodiversity (e.g., Lucky et al. 2014; Miczajka, Klein and Pufal 2015; Saunders et al. 2018) or climate change impact on plant or animal phenology (Ekholm et al. 2019; Hurlbert et al. 2019; Schwartz, Betancourt, and Weltzin 2012). In turn, volunteers engaged in CS programs can gain recognition for their skills and develop a deeper understanding of scientific concepts and the scientific process (Trumbull et al. 2000). This may positively contribute to both science and environmental education (Wals et al. 2014) and raise awareness of environmental issues. As a result, CS programs are now promoted by major funding agencies in Europe and North America (e.g., European Commission 2013; McLaughlin, Benforado, and Liu 2019).
Engaging schoolchildren and their teachers can enhance the long-term educational and social goals of CS programs for several reasons (Makuch and Aczel 2018). First, school pupils are guided by their instructors when learning about the scientific question raised by the CS program, and about the nature of science and its social aspects (Jenkins 2011; Koomen et al. 2018). Second, exposure to outdoor nature during childhood provides a long-lasting positive relationship with the environment while increasing people’s interest and knowledge about nature (Ganzevoort and van den Born 2019; Wells and Lekies 2012). Third, CS programs that involve self-selecting volunteers may underrepresent many social groups—although strategies exist to increase engagement (Pandya 2012)—whereas CS programs that target schoolchildren for CS projects have the potential to engage a wider cross-section of society in science (Jordan et al. 2011).
Nonetheless, the enthusiastic views of win-win interactions through CS programs have been questioned by social scientists and ecologists (Jordan et al. 2011). The former point out that the educational and social impact may be overstated (Brossard, Lewenstein, and Bonney 2005; Kelemen-Finan, Scheuch, and Winter 2018; Riesch and Potter 2014; Scheuch et al. 2018; Trumbull et al. 2000), while the latter are concerned about the accuracy of data collected by the general public (Burgess et al. 2016), especially when schoolchildren are involved. The main reason for these concerns is that CS data are arguably of lower quality than those collected by professional scientists (Burgess et al. 2016; Makuch and Aczel 2018; Riesch and Potter 2014). In response, it has been proposed that data collected by schoolchildren involved in CS programs can contribute to environmental research, provided that research methods are kept simple and require skills that the children already have or are able to gain when mentored by adults (Makuch and Aczel 2018; Miczajka, Klein, and Pufal 2015; Saunders et al. 2018), and the participants receive training, even remotely (Ratnieks et al. 2016). However, only a few studies have directly compared the quality of data acquired by professional scientists versus schoolchildren (Miczajka, Klein, and Pufal 2015; Pocock and Evans 2014; Saunders et al. 2018; Steinke et al. 2017). Evidence that CS programs can generate reliable scientific productions are needed to engage scientists with CS.
Here, we report on the preliminary results of the Oak Bodyguards CS program which has so far involved schoolchildren and professional scientists from 16 European countries. The project aims to assess the effects of climate on two key biotic interactions occurring widely in natural and anthropogenic ecosystems, i.e., the top-down and bottom-up forces controlling insect herbivory on leaves of the pedunculate oak, Quercus robur. This species is one of the most common and emblematic forest trees in Europe (Leroy, Plomion, and Kremer 2019), with a geographic range spanning more than 19 degrees of latitude. Furthermore, it is also widespread in natural, rural, suburban, and urban environments. In this project, schoolchildren and professional scientists placed dummy plasticine caterpillars in oak trees to estimate attack rates (Lövei and Ferrante 2017; Mäntylä et al. 2008; Roslin et al. 2017). We assessed the accuracy of CS data by comparing attack rate and insect herbivory estimates by three types of observers: professional scientists with previous experience in the project methodology (henceforth called trained professional scientists), professional scientists with no previous experience in the project methodology (untrained professional scientists), and schoolchildren. We first compared caterpillar attack rate estimates by schoolchildren or untrained professional scientists with those of a single professional scientist (Elena Valdés Correcher, henceforth known as EVC) trained to identify predation marks on artificial larvae. Second, in a separate experiment, schoolchildren and trained and untrained professional scientists estimated leaf insect herbivory from the percentage of leaf area removed or damaged by insect herbivores (Johnson, Bertrand, and Turcotte 2016), and we compared their herbivory estimatesto determine whether schoolchildren were able to conduct an ecological experiment and acquire scientific data of a quality comparable to that acquired by professional scientists. We use the results to discuss risks and opportunities for the future of CS programs with schoolchildren.
We designed a simple protocol that was applied by both schoolchildren and trained and untrained professional scientists. The protocol was written by scientists in collaboration with science instructors and communication officers. It was available in French, English, German, Spanish, and Portuguese (Castagneyrol et al. 2019).
In early 2018, 58 teachers with their students and 27 scientists from 16 European countries participated in the project. Each school and scientist selected a minimum of 1 and maximum of 18 mature pedunculate oak trees with lower branches accessible from the ground (schoolchildren: 1 to 8 oak trees, median = 2; scientists: 1 to 18 oak trees, median = 6). We imposed no restrictions on oak tree location, age, or size, but professional scientists were asked to choose oaks in woods larger than 1 ha. All partners measured oak tree circumference at 1.30 m from the ground and recorded oak coordinates with the GPS function of their smartphones.
All partners installed dummy caterpillars on lower branches of their selected oak trees to estimate attack rate, and haphazardly collected fresh leaves from the same trees to estimate insect herbivory. Although most of the schools estimated attack rates, none assessed herbivory. We also set up a complementary experiment to evaluate precision and accuracy of estimating insect herbivory by schoolchildren and professional scientists (see section entitled Insect herbivory below).
To control for latitudinal variation in environmental conditions, we matched the start of the experiment to the local phenology of the oak trees. Six weeks after oak budburst, partners installed 20 dummy caterpillars per tree, i.e., five caterpillars on each of four branches (facing north, south, east, and west) with a minimum distance of 15 cm between caterpillars. Caterpillars were made of the same green plasticine (Staedler, Noris Club 8421, green) provided to all partners by the project coordinators (B. Castagneyrol, EVC). To standardize caterpillar size among partners, caterpillars were made from a ball of plasticine of 1 cm diameter, and gently pressed/rolled onto the middle of a 12 cm-long metallic wire until a 3 cm-long caterpillar was obtained. Partners were instructed to attach the caterpillars to branches using wire, and leave the caterpillars on trees for 15 days prior to recording predation marks. Schoolchildren counted predation marks and attributed them to birds, mammals, arthropods, or reptiles. In 2018, they tagged and photographed every caterpillar with the suspected predation marks from any potential predator taxa. To minimise the probability of false negative results, we also advised the schoolchildren to send photographs of marks that were not clearly recognized as predation marks. Photos were taken from three different angles to show the observed damage and were labeled in such a way that the file name indicated both tree and caterpillar ID. Professional scientists were asked to gently remove all caterpillars from the trees and send them back to the project coordinators. One school also returned caterpillars, although this was not requested. A second survey using the same procedure immediately followed the first one. In 2019, both schoolchildren and professional scientists were instructed to send caterpillars back to the project coordinators. Photos and actual caterpillars were used by EVC to double-check and to standardize the predation assessment made by individual partners.
Every partner received a field bite guide containing a collection of photos illustrating predation marks left by different types of predators as well as false positive marks on plasticine surfaces that were made by leaves, buds, or finger nails. The different predator guilds that can be easily identified from their typical marks left on plasticine include passerine birds, rodents, snakes, lizards, and insects—mainly beetles and bush-crickets (Lövei and Ferrante 2017). The bite guide was available online and accessible to all partners through a hyperlink from the protocol (Castagneyrol et al. 2019), and teachers were invited to contact the scientific coordinator or local scientific partners in cases of uncertainty regarding the marks.
All partners were required to record their observations in the same standardized recording form. Partners indicated (a) the total number of caterpillars installed; (b) the number of caterpillars with any type of predation marks, (c) the number of caterpillars without predation marks; and (d) the number of caterpillars with predation marks left by birds (typically V-shaped beak marks and holes), arthropods (mandible marks), mammals (parallel teeth marks), or lizards (ellipse-shaped line of small teeth marks). Therefore, the same attacked caterpillar made a minimum of two entries in the recording form. We intentionally asked for redundant information to limit the risk of error in data reporting.
Data and biological material were collected by both schoolchildren and professional scientists during the same time period (from May through July). Project partners filled in the recording form and sent it to the project coordinators with the photos or the caterpillars. A single observer (EVC) with expertise in identifying predation marks on model caterpillars (Valdés-Correcher et al. 2019) screened every photo or caterpillar to verify observations reported by partners. It must be noted that false positives were more likely to be identified from the photos than false negatives. False positives are caterpillars classified by project partners as having been attacked when they were not. Because of previous reports (Low et al. 2014) and our own experience with undergraduate students trained to identify predation marks on artificial prey, we anticipated that schoolchildren and their teachers would be overly enthusiastic, making false positives more likely than false negatives. Schoolchildren were instructed to take photos of caterpillars with suspected predation marks, even marks they could not attribute to any predator type. It is therefore possible that they did not notice real predation marks on caterpillars that were photographed because they had marks left by buds, leaves, or finger nails. Such cases would represent false negatives. The probability of detecting false negative was not an issue when project partners returned caterpillars to the project coordinators.
For each oak tree and survey period, we assessed attack rate as the proportion of dummy caterpillars with at least one predation mark. Although we asked partners to record predation marks left by different types of predators (in particular birds and arthropods), this level of precision could not be reached on photos because of low resolution. Therefore, we quantified overall attack rate, regardless of predator type.
We estimated the precision and accuracy of attack-rate assessments by schoolchildren and untrained professional scientists by running two separate linear mixed-effect models with attack rate estimated by schoolchildren or professional scientists as a dependent variable, attack rate estimated by a single trained professional scientist and year (as factor) as independent variables, and Partner ID and Tree ID nested within Partner ID as random factors (Johnson, Bertrand, and Turcotte 2016). From each regression, we quantified the bias (a deviation between attack rate estimated by partners and a single trained observer) as the intercept (β0). Positive deviation from β0 = 0 indicates an overestimation of attack rate by partners. We quantified accuracy as the regression slope (β1), where β1 = 1 indicates high accuracy and β1 ≠ 1 indicates that accuracy in attack-rate assessment varied with actual attack rate. We used parametric bootstrapping with 1,000 simulations to compute 95% confidence interval (CI) around β0 and β1 and estimate how they deviated from 0 and 1, respectively. The null hypotheses were that β0 = 0 and β1 = 1. We considered that the null hypothesis was rejected if the 95% CI did not bracket zero or one. The significance of the fixed effect of year was tested based on the F-distribution and estimating degrees of freedom with Kenward-Roger methods (Kuznetsova, Brockhoff, and Christensen 2017).
To compare insect herbivory estimated by schoolchildren versus trained and untrained professional scientists, we set up a complementary survey (administered by AB). In April 2019, we prepared 12 sets of 5 oak leaves randomly drawn from a large sample of oak leaves collected in September 2018 on 162 oak trees around Bordeaux city (SW France) and stored in paper bags at –18°C. For each set of leaves, five trained professional scientists with previous experience in scoring insect herbivory on oak leaves (BC, EVC, AB, TD, and YK [see acknowledgements]) estimated insect herbivory as the percentage of leaf area removed or impacted by insect herbivores by giving each individual leaf a damage score: (0: 0%, A: 1–5%, B: 6–15%, C: 16–25%, D: 26–0%, E: 51–75%, F: > 75%; Castagneyrol et al. 2013). To reduce variability in estimates of herbivory due to observers, we created digital model leaves with given amounts of simulated herbivore damage that were used as examples for the seven damage classes (Castagneyrol et al. 2019). Leaf chewers were the main source of insect herbivory on oak leaves, but because leaves were drawn at random from a large pool of leaves, some were attacked by leaf miners, although none had galls. We asked participants to score total insect herbivory, regardless of damaging agents. As a result, the damage score incorporated leaf area removed by chewers as well as covered by leaf mines.
We invited schoolchildren 11 to16 years old (and their teachers) from six local secondary schools (equivalent US grades 6–10) to visit the first author’s research facilities (INRA research station of Pierroton, Bordeaux, France). Five groups of 10 to 12 students were introduced to the study of insect herbivory by the survey administrator, who challenged them to score insect herbivory as accurately as professional scientists would do. Students worked in groups of 2 or 3, with a total of 24 student groups. Each group was given 3 sets of 5 leaves, selected at random from the pool of 12 leaf sets. All students scored damage using the same digital model leaves as a template. In total, each of the 12 leaf sets was processed by six independent groups of students.
The same day (or the day after), we invited INRA permanent and non-permanent staff members to participate in the survey. The volunteers were researchers, engineers, technicians, and Master of Science students. They were considered untrained professional scientists). They received the same information from the survey administrator as secondary school students and used the same templates to score herbivory. Each of the nine volunteers processed every set of five leaves.
We did not keep records of individual leaves and we therefore averaged herbivory estimates across leaves for each set. We first tested whether individuals with a different background differed in their estimation of insect herbivory by running Linear Mixed-effects Models (LMM) with (log-transformed) insect herbivory as a response variable, observer type (Observer) as a fixed-effect factor, and leaf-set identity and observer identity as random effect factors. Because repeated handling of the same leaves may have caused some breakage, leading to a progressively increased estimation of herbivory, we added Time (number of hours since the first assessment) and Time × Observer interactions as additional fixed effects in the model. The model equation was
where β0 was the model intercept (i.e., Observerschoolchildren), β1 and β2 were the coefficients of the fixed effects of the treatment for trained (ObserverTrained) and untrained professional scientists (ObserverUntrained), β3 was the effect of Time, β4 and β5 were the effects of the Time × Observer interaction, γj and δk were the random intercepts for the observer and leaf-set identities, and εijk were the residuals. For γj, δk and εijk, we assumed a normal distribution with zero mean and variance σ2γ, σ2δ, and σ2ε, respectively. σ2ε contained variation among observers in scoring different leaf sets, i.e., the Observer × Leaf set interaction, but also all other noise. The significance of fixed effects was tested based on the F-distribution and estimating degrees of freedom with Kenward-Roger methods (Kuznetsova, Brockhoff, and Christensen 2017).
Second, we used σ2δ to quantify consistency among observers in rating herbivory. To do so, we ran an intercept only LMM for each group separately (i.e., for students and for trained and untrained professional scientists) and calculated intraclass correlation (ICC) for the Leaf set random factor (σ2δ/(σ2δ + σ2γ + σ2ε)). ICC represents the proportion of the total variance that is explained by Leaf set identity. It is a metric commonly used to estimate repeatability (Nakagawa and Schielzeth 2010). The greater the ICC, the greater rating consistency among observers scoring the same leaf set. We used parametric bootstrap with 1,000 random draws to estimate ICC 95% CI.
In total, 7,338 dummy caterpillars were installed on 195 oak trees by 58 schools and 27 scientists. Schools and scientists’ data came from from 8 and 14 countries throughout Europe, respectively (Figure 1). Schoolchildren installed and returned 3,289 dummy caterpillars. They counted 1,802 of them as attacked by predators (i.e., 55%), whereas EVC counted only 868 caterpillars with predation marks (26%). Professional scientists installed 4,045 caterpillars, 1,629 of which they identified as attacked by predators (40%); EVC counted 1,338 of these caterpillars as attacked by predators (33%).
Attack-rate estimates by schoolchildren were more biased (intercept estimate ± 95% bootstrap CI: β0 = 40.63 ± [22.45, 59.27]) than those by professional scientists (β0 = 23.41 ± [13.11, 33.17]). Detailed examination of pairwise comparisons at the tree level reveals that 81.5% of assessments made by schoolchildren were above the 1:1 line (Figure 2), thus indicating overestimation of attack rate as compared with assessments made by a single trained observer.
There was no relationship between attack rates estimated by schoolchildren versus a single trained observer (slope estimate ± 95% bootstrap 95% CI: β1 = 0.43 ± [–0.02, 0.90]), whereas professional scientists made more accurate assessments (β1 = 0.66 ± [0.54, 0.77], Figure 2). Attack rates estimated by schoolchildren and professional scientists did not differ between years (F1, 53.9 < 0.01, P = 0.952 and F1, 23.5 = 0.22, P = 0.644, respectively).
Insect herbivory estimates by trained professional scientists were the lowest (mean ± SE = 9.00% ± 0.51%, range 2.20% to 19.6%) (Figure 3; Figure S1 in the Supplemental File), whereas insect herbivory estimates by untrained professional scientists were the highest (14.65% ± 1.01%, range from 3.80% to 62.00%) (Figure 3; Figure S1 in the Supplemental File). Schoolchildren estimates of insect herbivory were intermediate (11.55% ± 0.64%, range from 2.20% to 27.40%) (Figure 3; Figure S1 in the Supplemental File). Both untrained professional scientists and schoolchildren consistently overestimated insect herbivory compared wiht trained professional scientists (Figure S1 in the Supplemental File), but this effect was not statistically significant at α = 0.05 (F2,31.9 = 2.79, P = 0.076) (Figure 3). Herbivory did not vary significantly with time (Time: F1, 28.5 < 0.01, P = 0.954; Time × Observer: F2, 33.0 = 0.62, P = 0.544).
Interestingly, ICC revealed that the consistency of herbivory estimates was comparable between trained professional scientists (ICC ± 95% CI: 0.58 ± [0.31, 0.84]) and schoolchildren (0.54 ± [0.22, 0.76]), whereas estimates made by untrained professional scientists were less consistent (0.44 ± [0.13, 0.67]).
Our comparison of data collected by different audiences (schoolchildren, untrained scientists, and trained scientists) allowed us to examine the quality of ecological data collected by schoolchildren, and to suggest improvements for future CS programs.
The main strength of CS programs, from a research perspective, is the collection power achieved by volunteers (especially if the data are independently verified). However, our findings proved ambiguous with respect to whether the resulting data are of sufficient quality to yield scientifically robust results. On the one hand, we clearly show that schoolchildren overestimated attack rate compared with trained professional scientists (Figure 2). They also tended to overestimate insect herbivory, but this effect was not significant at the common α = 0.05 threshold (Figure 4). On the other hand, professional scientists with mixed expertise in these fields also tended to overestimate attack rate and insect herbivory (Figures 2 and 3).
Importantly for the interpretability of the data, overestimation of attack rates was consistent across schools, as overestimation occurred in 81% of observations. Attack rates as assessed by professional scientists were, on average, slightly higher than attack rates re-estimated by a single trained observer. However, pairwise comparisons revealed that over- and underestimation of attack rates were more balanced in this group. In sharp contrast, we also found schoolchildren assessed insect herbivory in a more consistent way than untrained professional scientists did. Collectively, our results indicate that data provided by schoolchildren should be considered with caution, but the same holds true for data provided by untrained professional scientists.
Overestimation principally arose from partners scoring scratch marks left by contact with buds or leaves as signs of predation (Figure 4). Other sources of overestimation of predation cannot be ignored. Although no teachers mentioned vandalism of experiments, researchers should be aware of this possibility, particularly when caterpillars are placed on trees in urban environments. This may lead to missing caterpillars falsely scored as attacked. In addition, schoolchildren were told by teachers that the aim of the study was to determine “who protects oaks” against herbivores. It is possible that schoolchildren (and their teachers too) felt they had to see predation marks because this is what they perceived as the aim of the experiment. However, although confirmation bias is more likely to occur in schoolchildren and their teachers, it is important to stress that this type of cognitive bias is also common among trained professional scientists who may have interpreted small cracks (for example) on the caterpillar surface as predation marks (Forstmeier, Wagenmakers, and Parker 2017; Zvereva and Kozlov 2019).
Although the protocol clearly specified how to standardize caterpillar size and shape, and emphasized the importance of standardization, we noticed that the dimensions of dummy caterpillars varied widely, both within and among schools. In other studies, the probability of detecting predation marks left by avian or arthropod predators was found to be influenced by the length and width of artificial caterpillars (Lövei and Ferrante 2017). It is unlikely that variability in the dimension of artificial caterpillars has affected the comparison of attack rate as estimated by schoolchildren versus trained observers. However, the variation found should be regarded as a potential source of bias in large-scale multi-partner studies. As a potential mitigation procedure, researchers can provide pre-made caterpillars to project partners (Roslin et al. 2017). That said, making caterpillars according to a standard protocol is also an important dimension of student training. Despite potential biases in data collection, the pedagogical aspects of citizen science programs at schools must not be neglected, and scientists must recognize trade-offs between scientific and pedagogic objectives when planning mitigation procedures. As a compromise, scientists could provide partners with a reference caterpillar made of hardened undeformable clay. 3D-printed models of caterpillars attacked by different predator types may also be included as examples. In any case, we advise that project partners be instructed to carefully pack caterpillars when sending these to lead scientists for calibration of predation assessment. We also recommend that data collected by schoolchildren are not directly used in the project—their value lays in the pedagogical outcomes—but that trained professional scientists use their own scoring on the material provided by schoolchildren.
Johnson et al. (2016) found that bias in herbivory assessment decreased with the number of years of experience in herbivory assessment. Assuming that being trained as a scientist increases accuracy and the sense of rigor, we expected that herbivory would have been scored more accurately by untrained professional scientists than by schoolchildren. Our findings do not support this prediction. Although both schoolchildren and untrained professional scientists ranked the different leaf sets in the same order, for a given leaf set, schoolchildren always overestimated herbivory compared with trained professional scientists, with only one exception (Figure S1 in the Supplemental File), and untrained professional scientists always overestimated herbivory compared with schoolchildren, with only one exception (Figure S1 in the Supplemental File). However, this tendency was not statistically clear (Figure 3).
Interestingly, both schoolchildren and trained professional scientists assessed herbivory in a more consistent way than untrained professional scientists did, as revealed by the greater ICC in estimates. Individuals may vary in their observational skills, but training likely reduces this variability. Schoolchildren formed groups of 2 to 3 participants, while untrained professional scientists were alone when estimating herbivory. It is possible that within-group discussion leveled out intrinsic variability in observational skills and therefore variability of estimates made by schoolchildren. An alternative explanation for this unexpected finding is that schoolchildren took the activity more seriously than untrained professional scientists did. Regardless of the cause, these results stress that schoolchildren are no less reliable than untrained professional scientists when it comes to estimating insect herbivory (on oak leaves).
CS programs can help to generate a large amount of data, but the quality has been questioned, especially when these big data are not based on standard protocols (Bayraktarov et al. 2019; Burgess et al. 2016). Few studies have evaluated the quality of data collected by schoolchildren participating in CS programs (Miczajka, Klein and Pufal 2015; Saunders et al. 2018; Steinke et al. 2017). It emerges from these studies that schoolchildren can actually provide data accurate enough to support ecological research, provided that the tasks they are requested to undertake are adapted to their skills and that they receive proper training (Miczajka, Klein and Pufal 2015; Ratnieks et al. 2016; Saunders et al. 2018). Although we could not provide face-to-face training sessions for every school partner involved in the Oak Bodyguards project, the project methodology was simple and based on a detailed protocol. Nonetheless, this simplicity did not suffice to guarantee unbiased data, as illustrated by the fact that schoolchildren consistently overestimated attack rates. We therefore emphasize that CS programs relying on data collected by schoolchildren should include several checks of data quality and appropriate mitigation procedures. In particular, training sessions undertaken face-to-face or at least remotely must be planned before data collection (Ratnieks et al. 2016). Finally, whenever possible, the researcher analyzing the data should recover the raw material collected by children, or at the very least access pictures that allow for the re-assessment of measurements (Ekholm et al. 2019; Steinke et al. 2017). Importantly, these recommendations also hold true for large multi-partner research programs, as we also detected bias in data collected by professional scientists (Zvereva and Kozlov 2019). Whether variability in observations made by schoolchildren is random or can be modelled using appropriate covariates is an important question deserving further attention.
We found that schoolchildren involved in CS programs can support ecological research, but only if their contributions are considered with caution. The acquisition of reliable data requires experimental procedures that are easy to implement, but even so, a measurement of interpretation bias seems essential. Several quality checks and curation procedures are needed prior to using data collected by schoolchildren for ecological research. Unexpectedly, we found that such checks are necessary even for data acquired by professional scientists. It must be kept in mind that thrill, motivation, and self-confidence are keys to schoolchildren engagement with science and with practical scientific activities (Ganzevoort and van den Born 2019; Ruiz-Mallen et al. 2016). Our findings that schoolchildren did no worse than untrained professional scientists in collecting ecological data (here, in estimating insect herbivory) can strengthen their confidence and help them gain motivation and a positive attitude toward science in general. Despite legitimate concerns about the quality of data acquired by schoolchildren, following a protocol, collecting and formatting data, and sharing the process with scientists are valuable parts of training schoolchildren in scientific literacy. The trade-off between positive learning outcomes and the quality of raw data cannot be ignored, but with appropriate data quality checks and curation procedures, it actually favors the implementation of CS programs at school.
Raw data used in the present article are available in the Supplemental File.
The authors warmly thank all young European citizens and their teachers who have made this study possible. They also thank professional scientists who have kindly participated in this study: Xoaquin Moreira (Misión Biológica de Galicia, MBG-CSIC), Giada Centenaro, Stefan K. Müller (Freie evangelische Schule Lörrach), Olga Mijón Pedreira (teacher IES Rosais 2, Vigo-Spain), Mikhail Kozlov and Elena Zvereva (Section of Ecology, Department of Biology, University of Turku, FI-20014 Turku, Finland), Andreas Schuldt (University of Göttingen, Forest Nature Conservation, Göttingen), Aurélien Sallé (Laboratoire de Biologie des Ligneux et des Grandes Cultures, INRA, Université d’Orléans, 45067 Orléans, France) Mickael Pihain and Andreas Prinzing (Research Unit “Ecosystèmes, Biodiversité, Evolution”, University of Rennes 1/CNRS, 35042 Rennes, France), Chloe Mendiondo and Claire Colliaux (Department of Agroecology, Aarhus University, Flakkebjerg Research Centre, DK-4200 Slagelse, Denmark), Louise Montgomery and Felix Hall (Department of Biological Sciences, Royal Holloway University of London, Egham, UK). We thank Thomas Damestoy and Yasmine Kadiri (trained professional scientists) and Laure Dubois, Yannick Mellerin, Isabelle Lesur, Yves Ritter, Corinne Vacher, Geoffrey Haristoy, Thomas Folituu and Alex Stemmelen (untrained professional scientists) for having scored insect herbivory with BC, EVC, and AB, as well as Christelle Guilloux and Ingvild Marchand (Lycée Grand Air, Arcachon), Emilie Goyran (Collège Gaston Flament, Marcheprime), Danouckka Vignal-Tudal, and Erwan Thépaut (collège Leroi Gourhan, Le Bugue), and all their students. We also thank Michael Pocock and two anonymous reviewers for their helpful comments on an earlier version of this article.
This study has been carried out with financial support from the French National Research Agency (ANR) in the frame of the Investments for the future Programme, within the Cluster of Excellence COTE (ANR-10-LABX-45).
The authors have no competing interests to declare.
BC conceived the study, analyzed the data, and led the writing. BC and EVC coordinated the research, with help from DH, MB, MKD, MMG, MSL and RT. EVC acquired and formatted the data. All authors contributed data, critically commented, and edited the manuscript.
Bayraktarov, E, Ehmke, G, O’Connor, J, Burns, EL, Nguyen, HA, McRae, L, Possingham, HP and Lindenmayer, DB. 2019. Do Big Unstructured Biodiversity Data Mean More Knowledge? Frontiers in Ecology and Evolution, 6: 239. DOI: https://doi.org/10.3389/fevo.2018.00239
Brossard, D, Lewenstein, B and Bonney, R. 2005. Scientific knowledge and attitude change: The impact of a citizen science project. International Journal of Science Education, 27(9): 1099–1121. DOI: https://doi.org/10.1080/09500690500069483
Burgess, H, DeBey, L, Froehlich, H, Schmidt, N, Hille Ris Lambers, J, Tewksbury, J and K. Parrish, J. 2016. The science of citizen science: Exploring barriers to use as a primary research tool. Biological Conservation, 208. DOI: https://doi.org/10.1016/j.biocon.2016.05.014
Castagneyrol, B, Giffard, B, Péré, C and Jactel, H. 2013. Plant apparency, an overlooked driver of associational resistance to insect herbivory. Journal of Ecology, 101(2): 418–429. DOI: https://doi.org/10.1111/1365-2745.12055
Castagneyrol, B, Valdés-Correcher, E, Kaennel Dobbertin, M and Gossner, M. 2019. Predation assessment on fake caterpillars and leaf sampling: Protocol for schools. protocols.io. DOI: https://doi.org/10.17504/protocols.io.42pgydn
Dickinson, JL, Shirk, J, Bonter, D, Bonney, R, Crain, RL, Martin, J, Phillips, T and Purcell, K. 2012. The current state of citizen science as a tool for ecological research and public engagement. Frontiers in Ecology and the Environment, 10(6): 291–297. DOI: https://doi.org/10.1890/110236
Drummond, C and Fischhoff, B. 2017. Individuals with greater science literacy and education have more polarized beliefs on controversial science topics. Proceedings of the National Academy of Sciences, 201704882. DOI: https://doi.org/10.1073/pnas.1704882114
Ekholm, A, Tack, AJM, Bolmgren, K and Roslin, T. 2019. The forgotten season: the impact of autumn phenology on a specialist insect herbivore community on oak. Ecological Entomology, 44(3): 425–435. DOI: https://doi.org/10.1111/een.12719
Fiske, ST and Dupree, C. 2014. Gaining trust as well as respect in communicating to motivated audiences about science topics. Proceedings of the National Academy of Sciences, 111(Supplement 4): 13593–13597. DOI: https://doi.org/10.1073/pnas.1317505111
Forstmeier, W, Wagenmakers, E-J and Parker, TH. 2017. Detecting and avoiding likely false-positive findings – a practical guide. Biological Reviews, 92(4): 1941–1968. DOI: https://doi.org/10.1111/brv.12315
Fox, J, Weisberg, S, Adler, D, Bates, D, Baud-Bovy, G, Ellison, S, Firth, D, Friendly, M, Gorjanc, G, Graves, S, Heiberger, R, Laboissiere, R, Monette, G, Murdoch, D, Nilsson, H, Ogle, D, Ripley, B, Venables, W, Winsemius, D, Zeileis, A and R-Core. 2016. car: Companion to Applied Regression.
Ganzevoort, W and van den Born, R. 2019. The Thrill of Discovery: Significant Nature Experiences Among Biodiversity Citizen Scientists. Ecopsychology, 11(1): 22–32. DOI: https://doi.org/10.1089/eco.2018.0062
Hurlbert, A, Hayes, T, McKinnon, T and Goforth, C. 2019. Caterpillars Count! A Citizen Science Project for Monitoring Foliage Arthropod Abundance and Phenology. Citizen Science: Theory and Practice, 4(1): 1. DOI: https://doi.org/10.5334/cstp.148
Jenkins, LL. 2011. Using citizen science beyond teaching science content: a strategy for making science relevant to students’ lives. Cultural Studies of Science Education, 6(2): 501–508. DOI: https://doi.org/10.1007/s11422-010-9304-4
Johnson, MTJ, Bertrand, JA and Turcotte, MM. 2016. Precision and accuracy in quantifying herbivory. Ecological Entomology, 41(1): 112–121. DOI: https://doi.org/10.1111/een.12280
Jordan, RC, Gray, SA, Howe, DV, Brooks, WR and Ehrenfeld, JG. 2011. Knowledge Gain and Behavioral Change in Citizen-Science Programs. Conservation Biology, 25(6): 1148–1154. DOI: https://doi.org/10.1111/j.1523-1739.2011.01745.x
Kahan, DM, Peters, E, Wittlin, M, Slovic, P, Ouellette, LL, Braman, D and Mandel, G. 2012. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change, 2(10): 732–735. DOI: https://doi.org/10.1038/nclimate1547
Kelemen-Finan, J, Scheuch, M and Winter, S. 2018. Contributions from citizen science to science education: an examination of a biodiversity citizen science project with schools in Central Europe. International Journal of Science Education, 40(17): 2078–2098. DOI: https://doi.org/10.1080/09500693.2018.1520405
Koomen, MH, Rodriguez, E, Hoffman, A, Petersen, C and Oberhauser, K. 2018. Authentic science with citizen science and student-driven science fair projects. Science Education, 102(3): 593–644. DOI: https://doi.org/10.1002/sce.21335
Lövei, GL and Ferrante, M. 2017. A review of the sentinel prey method as a way of quantifying invertebrate predation under field conditions. Insect Science, 24(4): 528–542. DOI: https://doi.org/10.1111/1744-7917.12405
Lucky, A, Savage, AM, Nichols, LM, Castracani, C, Shell, L, Grasso, DA, Mori, A and Dunn, RR. 2014. Ecologists, educators, and writers collaborate with the public to assess backyard diversity in The School of Ants Project. Ecosphere, 5(7): art78. DOI: https://doi.org/10.1890/ES13-00364.1
Makuch, K and Aczel, M. 2018. Children and citizen science. In: UCL Press. 391–409. DOI: https://doi.org/10.14324/111.9781787352339
Mäntylä, E, Alessio, GA, Blande, JD, Heijari, J, Holopainen, JK, Laaksonen, T, Piirtola, P and Klemola, T. 2008. From Plants to Birds: Higher Avian Predation Rates in Trees Responding to Insect Herbivory. Rands, S (ed.), PLoS ONE, 3(7): e2832. DOI: https://doi.org/10.1371/journal.pone.0002832
McLaughlin, J, Benforado, J and Liu, SB. 2019. Report to Congress describes the breadth and scope of Federal crowdsourcing and citizen science. citizenscience.gov. Available at https://www.citizenscience.gov/2019/06/18/report-to-congress-2019/# [Last accessed 24 June 2019].
Miczajka, VL, Klein, A-M and Pufal, G. 2015. Elementary school children Contribute to Environmental Research as Citizen Scientists. PLoS ONE, 10(11). DOI: https://doi.org/10.1371/journal.pone.0143229
Pandya, RE. 2012. A framework for engaging diverse communities in citizen science in the US. Frontiers in Ecology and the Environment, 10(6): 314–317. DOI: https://doi.org/10.1890/120007
Pocock, MJO and Evans, DM. 2014. The Success of the Horse-Chestnut Leaf-Miner, Cameraria ohridella, in the UK Revealed with Hypothesis-Led Citizen Science. PloS ONE, 9(1): e86226. DOI: https://doi.org/10.1371/journal.pone.0086226
Ratnieks, FLW, Schrell, F, Sheppard, RC, Brown, E, Bristow, OE and Garbuzov, M. 2016. Data reliability in citizen science: learning curve and the effects of training method, volunteer background and experience on identification accuracy of insects visiting ivy flowers. Methods in Ecology and Evolution, 7(10): 1226–1235. DOI: https://doi.org/10.1111/2041-210X.12581
Riesch, H and Potter, C. 2014. Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions. Public Understanding of Science (Bristol, England), 23(1): 107–120. DOI: https://doi.org/10.1177/0963662513497324
Roslin, T, Hardwick, B, Novotny, V, Petry, WK, Andrew, NR, Asmus, A, Barrio, IC, Basset, Y, Boesing, AL, Bonebrake, TC, Cameron, EK, Dáttilo, W, Donoso, DA, Drozd, P, Gray, CL, Hik, DS, Hill, SJ, Hopkins, T, Huang, S, Koane, B, Laird-Hopkins, B, Laukkanen, L, Lewis, OT, Milne, S, Mwesige, I, Nakamura, A, Nell, CS, Nichols, E, Prokurat, A, Sam, K, Schmidt, NM, Slade, A, Slade, V, Suchanková, A, Teder, T, van Nouhuys, S, Vandvik, V, Weissflog, A, Zhukovich, V and Slade, EM. 2017. Higher predation risk for insect prey at low latitudes and elevations. Science, 356(6339): 742–744. DOI: https://doi.org/10.1126/science.aaj1631
Ruiz-Mallen, I, Riboli-Sasco, L, Ribrault, C, Heras, M, Laguna, D and Perie, L. 2016. Citizen Science: Toward Transformative Learning. Science Communication, 38(4): 523–534. DOI: https://doi.org/10.1177/1075547016642241
Saunders, ME, Roger, E, Geary, WL, Meredith, F, Welbourne, DJ, Bako, A, Canavan, E, Herro, F, Herron, C, Hung, O, Kunstler, M, Lin, J, Ludlow, N, Paton, M, Salt, S, Simpson, T, Wang, A, Zimmerman, N, Drews, KB, Dawson, HF, Martin, LWJ, Sutton, JB, Webber, CC, Ritchie, AL, Berns, LD, Winch, BA, Reeves, HR, McLennan, EC, Gardner, JM, Butler, CG, Sutton, EI, Couttie, MM, Hildebrand, JB, Blackney, IA, Forsyth, JA, Keating, DM and Moles, AT. 2018. Citizen science in schools: Engaging students in research on urban habitat for pollinators. Austral Ecology, 43(6): 635–642. DOI: https://doi.org/10.1111/aec.12608
Scheuch, M, Panhuber, T, Winter, S, Kelemen-Finan, J, Bardy-Durchhalter, M and Kapelari, S. 2018. Butterflies & wild bees: biology teachers’ PCK development through citizen science. Journal of Biological Education, 52(1): 79–88. DOI: https://doi.org/10.1080/00219266.2017.1405530
Schwartz, MD, Betancourt, JL and Weltzin, JF. 2012. From Caprio’s lilacs to the USA National Phenology Network. Frontiers in Ecology and the Environment, 10(6): 324–327. DOI: https://doi.org/10.1890/110281
Steinke, D, Breton, V, Berzitis, E and Hebert, PDN. 2017. The School Malaise Trap Program: Coupling educational outreach with scientific discovery. PLOS Biology, 15(4): e2001829. DOI: https://doi.org/10.1371/journal.pbio.2001829
Trumbull, D, Bonney, R, Bascom, D and Cabral, A. 2000. Thinking Scientifically during Participation in a Citizen-Science Project. Science Education, 84: 265–275. DOI: https://doi.org/10.1002/(SICI)1098-237X(200003)84:2<265::AID-SCE7>3.0.CO;2-5
Valdés-Correcher, E, van Halder, I, Barbaro, L, Castagneyrol, B and Hampe, A. 2019. Insect herbivory and avian insectivory in novel native oak forests: Divergent effects of stand size and connectivity. Forest Ecology and Management, 445: 146–153. DOI: https://doi.org/10.1016/j.foreco.2019.05.018
Wals, AEJ, Brody, M, Dillon, J and Stevenson, RB. 2014. Convergence Between Science and Environmental Education. Science, 344(6184): 583–584. DOI: https://doi.org/10.1126/science.1250515
Wells, NM and Lekies, K. 2012. Children and nature: following the trail to environmental attitudes and behaviour. In: Citizen Science: public collaboration in environmental research, 201–213. Cornell University Press. Ithaca, NY. DOI: https://doi.org/10.7591/cornell/9780801449116.003.0015
Zvereva, EL and Kozlov, MV. 2019. Biases in studies of spatial patterns in insect herbivory. Ecological Monographs. in press(0): e01361. DOI: https://doi.org/10.1002/ecm.1361