Introduction

Animal pollination directly affects the quality and yield of 75% of the world’s leading food crops (Klein et al. 2007), and 87.5% of flowering plants benefit from pollination by animals (Ollerton, Winfree, and Tarrant 2011). As pollinators provide an essential ecosystem service, monitoring their status and trends on a local and global level is of economic and cultural importance, and is necessary for effective conservation policy. Through monitoring, recent global pollinator assessments have recorded large-scale declines in Europe and North America (IPBES 2016).

Citizen science is defined as “scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional scientists and scientific institutions” (OED 2014). A network of volunteer participants of variable backgrounds and experience are engaged to gather and contribute to large data sets on broad temporal or spatial scales, using methodology developed by (or alongside) trained experts and researchers. Citizen science contributes knowledge towards indicators in the UN Sustainable Development Goals, and with increased partnerships and investment has the potential to contribute towards many more indicators on nature and the environment (Fraisl et al. 2020). Citizen science projects most commonly fall in the areas of conservation, ecology, and biology (Kullenberg and Kasperowski 2016), and ecological projects cover a broad range of taxa from local to global levels (Dickinson et al. 2012). Such data collected by citizen scientists form the basis of many successful international monitoring schemes. For example, multi-taxa recordings through iNaturalist (inaturalist.org) and global bird sightings through eBird (The Cornell Lab of Ornithology, ebird.org) have collected data that have contributed to conservation action. Currently, bias exists in the geographic location and taxonomic groups, with many citizen science programmes based in North America and Europe, and with a general focus on animals (Chandler et al. 2017), reflecting similar biases with conventional scientific projects (Theobald et al. 2015).

Citizen science has proved valuable in monitoring wild bee populations, and determining the effectiveness of interventions to conserve pollinators. Such projects have contributed knowledge on pollination services (e.g., Bees ‘n Beans: Birkin and Goulson 2015), management practices (e.g., Squash Bee Survey: Appenfeller, Lloyd and Szendrei 2020), abundance in urban landscapes (e.g., Native Bee Watch: Mason and Arathi 2019), effects of urban wildflower patches (e.g., Sow Wild! Griffiths-Lee, Nicholls, and Goulson 2022), nesting ecology (e.g., The Solitary Bee Project: Maher, Manco, and Ings 2019; BeeWatch: Lye et al. 2012) and pollinator populations (e.g., UK Pollinator Monitoring Scheme: ukpoms.org.uk). Citizen scientists are also collecting valuable data on other flying foraging insects, such as social wasps (bigwaspsurvey.org) and hoverflies (hoverflylagoons.co.uk).

There are many benefits of utilising citizen science in ecological monitoring, including the potential increased spatial and temporal scale of data collection and the possibility of accessing private locations to conduct sampling (Bonney et al. 2009). Engagement in a citizen science project provides benefits not only for the experts collecting the data; such projects can facilitate behavioural changes in conservation and environmental issues, and create educational opportunities (Bonney et al. 2016; Merenlender et al. 2016). The desire to learn about pollinators and contribute to science drives participation in pollinator-focused projects (Domroese and Johnson 2016), and this active engagement can create an emotional connection and lifelong commitment to nature (Schuttler et al. 2018).

Perceived limitations of data collected by citizen scientists typically focus on accuracy and inconsistencies in data collected by non-experts (Aceves-Bueno et al. 2017; Gardiner et al. 2012; Burgess et al. 2017; Law et al. 2017). For example, smaller and less conspicuous bees may be misclassified or go unnoticed, with bias toward species or groups that can be identified (Maher, Manco, and Ings 2019; Kremen, Ullman, and Thorp 2011). Even the identification of more conspicuous, larger taxa such as bumblebees can be prone to errors at the species level (Austen et al. 2016; Falk et al. 2019; Roy et al. 2016). However, several insect-based studies comparing observations of citizen scientists with those of experts have found the results to be similar (Kremen, Ullman, and Thorp 2011; Mason and Arathi 2019; Griffiths-lee, Nicholls, and Goulson 2020, Maher, Manco, and Ings 2019; Dennis et al. 2017). Effective training of citizen scientists is important for data accuracy (Roy et al. 2016; Kremen, Ullman, and Thorp 2011) and success may depend on methods and taxa collected. As unverified data records submitted by citizen scientists risk incorrect conclusions (Falk et al. 2019), and biases and potential errors are poorly understood, it has been argued that citizen science data should be seen as complementary to researcher-led data, rather than as an alternative to it (Dickinson, Zuckerberg, and Bonter 2010).

There are a variety of sampling methods available to monitor insect groups, each with its advantages, limitations, and potential bias towards certain taxa (McCravy 2018). Pan traps are bowls of soapy water, painted in different colours to attract a range of foraging insects. They are considered the “most efficient, unbiased, and cost-effective method for sampling bee diversity” (Westphal et al. 2008). Although smaller species of bees are more commonly captured by pan traps, they do typically capture a broad range of genera (Droege et al. 2010). Different colour pan traps are considered attractive to different bee taxa (Geroff, Gibbs, and McCravy 2014; Leong and Thorp 1999) and a recent meta-analysis found that yellow pan traps most efficiently sampled smaller solitary bees, while blue was best for bumblebees (Hutchinson et al. 2021). Pan traps are also effective at monitoring aculeate and parasitoid wasps (Bąkowski, Piekarska-Boniecka, Dolańska-Niedbała 2013; Heneberg and Bogusch 2014). A set of coloured traps is better for overall monitoring of bees to capture common and uncommon species (Buffington et al. 2020; Toler, Evans and Tepedino 2005), and the addition of nectar guides increases the number of specimens collected (Wilson et al. 2016). Yellow sticky traps are elevated, bright yellow, flat traps covered in non-drying sticky glue, often with a black grid to aid insect counts. Yellow sticky traps have been effectively used to sample parasitoid wasps (Hall et al. 2019, Griffiths-Lee, Nicholls, and Goulson 2022; Wallis and Shaw 2008) and hoverflies (Burgio and Sommaggio 2007).

Larger insects such as bumblebees are most effectively monitored by visual identification on transect walks (Hutchinson et al. 2021). Visual observation and counts of pollinating insects can be successfully conducted in citizen science projects, such as Polli:Nation (Cruickshanks et al. 2018) and the UK Pollinator Monitoring Scheme. However, visual identification of insects by untrained professionals can be prone to errors, although this is mostly documented when trying to identify them to a finer taxonomic level (Maher, Manco, and Ings 2019; Kremen, Ullman, and Thorp 2011).

Multiple sampling methods are recommended for a more complete data set, which is particularly important when studying species richness (McCravy 2018). As pan traps and yellow sticky traps can be set and collected without entomological training, and as a timed insect watch is an enjoyable and accessible approach to obtaining abundance information, these sampling methods were deemed suitable for use in the Sow Wild! project. Sow Wild! was a hypothesis-driven citizen science project which investigated the effectiveness of creating a 4 m² (2 × 2 m) wildflower mini-meadow in attracting beneficial insects in private gardens (Griffiths-Lee, Nicholls, and Goulson 2022). Citizen scientists sowed and maintained the mini-meadow, and then successfully collected data over two years. Using these data we aimed to determine: 1) the accuracy of identification of pan trap insect samples by citizen scientists, by comparing insects recorded by citizen scientists and by researchers; 2) if non-destructive observation-only sampling techniques (insect watches) were representative of the data collected with specimen-based methods (pan trap and yellow sticky traps); 3) which sampling methods were more or less effective at collecting data on specific taxa, and which were suitable for citizen science projects.

Methods

Results

Insect identification by citizen scientists

Results of pan trap sample identification data collected by citizen scientist participants and professional researchers were compared, to determine whether citizen scientists tended to overestimate or underestimate the abundance of certain insect groups. Counts of bumblebees and honeybees were comparable (X² = 0.47, p = 0.49 and X² = 0.05, p = 0.82 respectively; Figure 1). However, numbers of solitary bees were underestimated (X² = 6.26, p = 0.01; Figure 1) and social wasps were overestimated by citizen scientists (X² = 19.17, p = 0.00001; Figure 1). Hoverfly counts did not significantly differ between citizen scientists and researchers, although counts of hoverflies were notably higher for citizen scientists (X² = 1.09, p = 0.3; Figure 1).

Figure 1 

Mean abundance of broad insect groups, as identified by citizen scientists from pan trap samples, compared with professional researchers’ identification of the same samples.

Sampling methods and Sow Wild! project results

We compared the mean abundance of broad insect groups considering the Sow Wild! project treatments (mini-meadow, 10 m away, control sites) and the three different sampling methods (the set of four coloured pan traps and yellow sticky traps using researcher data, and insect watch using citizen science data) (Figure 2). Patterns between pan trap data and yellow sticky trap data are similar, with control gardens having lower insect abundance than those with mini-meadows (Figure 2). However, patterns of insect abundance recorded during the insect watch conducted by citizen scientists differ from pan trap and yellow sticky trap methods, with control sites having the highest insect abundance and mini-meadows the least (Figure 2). Insect watches also recorded a higher abundance of the more conspicuous groups; bumblebees, honeybees, and hoverflies, compared with researcher-verified pan traps and yellow sticky traps.

Figure 2 

Mean abundance of bumblebees, honeybees, hoverflies, social wasps, and solitary bees recorded over the three sampling methods used in the Sow Wild! project (insect watch, pan trap set, and yellow sticky traps) and each of the project treatments (sampling mini-meadow, 10 m away from mini-meadow, and control sites). Pan trap and yellow sticky trap data collected by researchers, insect watch data collected by citizen scientists.

Sampling methods for insect groups

The method of sampling had significant effects on the capture rate (abundance) of all broad insect groups considered (bumblebees, honeybees, hoverflies, solitary bees, social wasps, solitary wasps) (Table 1). Insect watches conducted by citizen scientists produced the most observations for all groups except for social wasps (to note, citizen scientists were not asked to record solitary wasps). Yellow sticky traps were the most effective at collecting social and solitary wasps (Table 1). Of the pan traps, white pan traps were the most effective pan traps at capturing pollinators overall, especially bumblebees and solitary bees (Table 1). Blue and pink pan traps consistently collected similar data for each of the insect groups, and this was far less than the white and yellow pan traps.

Table 1

GLMM ANOVA results for effects of sampling method on the abundance of insect group and bee species richness. Abundance of broad insect groups (bumblebee, honeybee, hoverfly, solitary bee, solitary wasp) recorded in each of the sampling methods used (insect watch, blue pan traps, pink pan traps, white pan traps, yellow pan traps, yellow sticky traps) and richness of bee species (including solitary bee, bumblebee and honeybee) collected in pan traps only. Presented with mean ± standard error, median (IQR), chi-square X², degrees freedom df, significance *** p < 0.001 and Tukey’s Honest Significant Difference test for comparisons (designated by letters in bold).

ABUNDANCE	X2	DF	P=	AVERAGE	BEE WATCH	PAN TRAP (BLUE)	PAN TRAP (PINK)	PAN TRAP (WHITE)	PAN TRAP (YELLOW)	YELLOW STICKY
Bumblebee	324.31	5	<2.2e–16***	Mean ± SE	3.65 ± 0.23	0.1 ± 0.09	0.06 ± 0.08	0.45 ± 0.1	0.18 ± 0.09	0.6 ± 0.13
				Median (IQR)	2 (5) (c)	0 (0) (a)	0 (0) (a)	0 (1) (b)	0 (0) (a)	0 (1) (b)
Honeybee	161.62	5	<2.2e–16***	Mean ± SE	1.49 ± 0.23	0.05 ± 0.08	0.07 ± 0.1	0.2 ± 0.1	0.09 ± 0.1	0.35 ± 0.13
				Median (IQR)	0 (2) (d)	0 (0) (a)	0 (0) (a)	0 (0) (bc)	0 (0) (ab)	0 (0) (c)
Hoverfly	194.6	5	<2.2e–16***	Mean ± SE	2.23 ± 0.23	0.03 ± 0.13	0.04 ± 0.08	0.1 ± 0.1	0.16 ± 0.09	0.52 ± 0.22
				Median (IQR)	0 (3) (d)	0 (0) (a)	0 (0) (ab)	0 (0) (ab)	0 (0) (b)	0 (0) (c)
Solitary bee	195.78	5	<2.2e–16***	Mean ± SE	1.22 ± 0.31	0.08 ± 0.1	0.14 ± 0.1	0.66 ± 0.14	0.53 ± 0.12	0.34 ± 0.15
				Median (IQR)	0 (1) (c)	0 (0) (a)	0 (0) (a)	0 (1) (b)	0 (1) (b)	1 (2) (c)
Social wasp	217.58	5	<2.2e–16***	Mean ± SE	0.42 ± 0.18	0.01 ± 0.07	0.02 ± 0.07	0.02 ± 0.07	0.04 ± 0.07	0.99 ± 0.13
				Median (IQR)	0 (0) (b)	0 (0) (a)	0 (0) (a)	0 (0) (a)	0 (0) (a)	0 (1) (c)
Solitary wasp	677.82	4	<2.2e–16***	Mean ± SE	NA	0.88 ± 0.21	0.85 ± 0.15	1.04 ± 0.14	1.76 ± 0.17	15.2 ± 0.26
				Median (IQR)	NA	0 (1) (a)	0 (1) (a)	0 (2) (a)	1 (2) (b)	12 (15) (c)
Richness
All bee	132.77	3	<2.2e–16***	Mean ± SE	NA	0.11 ± 0.06	0.13 ± 0.07	0.61 ± 0.07	0.38 ± 0.07	NA
				Median (IQR)	NA	0 (0) (a)	0 (0) (a)	0 (1) (c)	0 (1) (b)	NA

Researcher-counted yellow sticky traps had the highest proportion of social wasps and the four colour pan traps were relatively equal in the proportion of insect groups collected (Figure 3). Insect watches conducted by citizen scientists collected the highest proportion of bumblebees, and also collected the lowest proportion of solitary bees, noticeably less than the other sampling methods (Figure 3).

Figure 3 

Proportion of insect groups (bumblebees, honeybees, hoverflies, social wasps, and solitary bees) collected by sampling method (blue, pink, white and yellow pan trap, insect watch, yellow sticky traps). Pan trap and yellow sticky trap data collected by researchers, insect watch data collected by citizen scientists.

Of the researcher-verified pan traps, white and yellow pan traps were equally effective at capturing the most common bee species despite white pan traps capturing more of these insects overall. Pink and blue pan traps were also equally effective at capturing common bee species (Figure 4).

Figure 4 

Abundance heatmap of twenty most abundant wild bee species. Based on count of bees sampled by pan trap colour (blue, pink, white, yellow). Square root transformed for visualisation purposes.

The colour of pan trap also had a significant effect on the species richness of bees collected, with white pan traps collecting the highest richness, followed by yellow pan traps. Blue and pink traps caught the lowest species richness of bees (Table 1).

Discussion

Using researcher-verified data collected by citizen scientists in the Sow Wild! project, we were able to compare insect identification by citizen scientists and researchers from samples collected using pan traps in private gardens and allotments. We found that counts of the more conspicuous (that is, larger and more well-known) bumblebees and honeybees were similar between citizen scientists and researchers. However, numbers of solitary bees in pan trap samples were underestimated by citizen scientists, and social wasps were overestimated by citizen scientists. Although hoverfly counts did not significantly differ between researchers and citizen scientists, the overall counts of hoverflies were much higher from citizen scientists. Therefore, although we were not able to verify exactly which specimens were misidentified, it is highly probable that solitary bees may have been mistaken for social wasps and hoverflies.

In Year 1 of the study, when faced with numerous small flies in the pan trap samples, we found that citizen scientists were discouraged by sorting through and recording the groups present. Similarly, Kleinke et al. (2018) discussed how volunteers found the task of counting numerous seeds too onerous. In Year 2, we told citizen scientists that exact fly counts were unnecessary, and we also did not ask citizen scientists to count solitary wasps as this would have been too difficult. However, we found that when high numbers of flies were recorded by citizen scientists, parasitoid wasps were numerous and commonly misidentified as small flies (personal observation). Considering proportions of insects recorded during the observational insect watch conducted by citizen scientists, compared with the specimen-based sampling methods (pan traps and yellow traps), the proportions of solitary bees were again lower than expected and probably also under-recorded during the insect watches. Previous studies have also concluded that less conspicuous groups tend to be under-recorded by citizen scientists (Kremen, Ullman, and Thorp 2011) especially the smaller solitary bees (Maher, Manco, and Ings 2019).

For the Sow Wild! project (Griffiths-Lee et al. 2022), we looked at the abundance of insect groups recorded in the different treatments (that is, amongst mini-meadows, 10 m away, and control sites) to measure the effectiveness of mini-meadows in recruiting beneficial insects in gardens and allotments, using researcher-verified specimen-based data (pan traps and yellow sticky traps). However, insect watches conducted by citizen scientists recorded higher abundance of insects in control gardens, whereas pan traps and yellow sticky traps recorded higher abundance of insects in those gardens with a mini-meadow. Therefore, incorrect conclusions may have been drawn if we had relied solely on the observations made by citizen scientists. However, we did not conduct site visits nor did we conduct researcher-led insect watches, so we cannot directly compare data collected by insect watches compared with verified sample-based methods. Instead, the unexpected project results between sample-based and observational techniques may have been due to participant bias. Although advised to measure out the 2 × 2 m area, few participants reported doing so. Indeed, those participants in the control group may not be able to visualise a 4 m² area in 3D space without measuring out this transect, compared with the groups actively working with a mini-meadow of clearly defined size. Without pre-measured transects, it would be easy to report insects outside the transect as a subconscious act, or perhaps participants underestimated the importance of reporting zeros. It could also simply be that the vegetation in the wildflower patch may have made it more challenging to spot insects compared with a control garden without many plants. Indeed, previous studies concluded that data collected by citizen scientists and researchers yield similar results (Griffiths-Lee, Nicholls, and Goulson 2020; Kremen, Ullman, and Thorp 2011; Mason and Arathi 2019; Maher, Manco, and Ings 2019; Dennis et al. 2017). The difference in results reported in this paper could instead be due to the level of training and complexity of the protocol. For example, Kremen, Ullman and Thorp (2011) were able to offer in-person training with fewer participants (n = 13, compared with our project with an initial larger pool of citizen scientists, n = 150), and Griffiths-Lee, Nicholls, and Goulson (2020) conducted targeted insect watches on a smaller scale. With the growth of citizen science as a cost-effective tool for producing high-quality data (Breeze et al. 2021) there has been increasing focus on the use of statistical analysis to deal with potential inaccuracies in count data (e.g., Clare et al 2019), which could be included in the analysis of data collected by citizen scientists.

We also found that the different sampling methods (insect watches, pan traps in four colours, yellow sticky traps) vary in their effectiveness at collecting different insect groups, all of which can be easily and successfully incorporated into citizen science projects. Such knowledge is useful when selecting the method to monitor specific taxa and could avoid excessive lethal sampling. The insect watch collected the most insect observations out of all the methods. However, higher counts of insects in the insect watch could have been in part due to repeated counting of a re-visiting insect, which would have only been counted once in the specimen-based sampling. We found white traps were the most effective pan trap colour overall, collecting the highest bee species richness, and collecting around a third more beneficial insects than yellow traps, which were the second most effective colour. The efficiency of pan trap colours differs according to bee species (Toler, Evans, and Tepedino 2005), body size (Wilson, Griswold, and Messinger 2008; Hutchinson et al. 2021; Krahner et al. 2021), sex (Leong and Thorp 1999), surrounding landscape and habitat (Saunders and Luck 2013; McCravy 2018), and neighbouring crops (Hutchinson et al. 2021). We found white pan traps collected more bumblebees and solitary bees, and yellow pan traps collected more social wasps and solitary wasps, highlighting effective pan trap colours in sampling beneficial insects in UK urban environments. Furthermore, we found that yellow sticky traps were more effective than pan traps at sampling solitary wasps, solitary bees, social wasps, hoverflies, and honeybees. These results agree with previous studies that conclude yellow sticky traps are useful in sampling parasitoid wasps (Hall et al. 2019, Wallis and Shaw 2008) and hoverflies (Burgio and Sommaggio 2007). However, the smaller species were difficult to remove from yellow sticky traps, and hence identification to species would be difficult for some taxa.

Generally, it is assumed that citizen science projects with a simple protocol will retain more volunteers (Birkin and Goulson 2015). However, we found volunteers were not discouraged despite a slightly complicated experimental protocol, and the commitment required to set up the initial meadow plots. Forty-eight of the initial participants continued through to Year 2 of the project, which produced high-quality data and significant findings on effective pollinator habitat management in gardens (Griffiths-Lee, Nicholls, and Goulson 2022). The majority of those who provided feedback on reasons for leaving the project reported changes in personal circumstances, health reasons, or that the mini-meadow did not establish sufficiently to continue with the project, and many participants remained engaged and interested even after dropping out. Participant dropout may have been non-random, with poorly established mini-meadows or those that caught fewer insects perhaps leaving the more productive pollinator-friendly gardens continuing with the project. Yet for this project, dropout rates were unlikely due to group allocation, as similar proportions of participants remained in each group (control, mix 1, and mix 2) throughout the project. Future citizen science projects could be designed to include analysis on drop-out reasons, for example, due to participants’ interpretation of success. It is an interesting next step to understand the potential limitations, as such elective dropout may create bias in results.

As a standardised hypothesis-driven project, Sow Wild! relied on fewer dedicated participants than unstructured, opportunistic projects. To retain participants, we set up social media accounts to create a sense of community, interacted with participants regularly, and provided feedback during and at the end of the project. Such communication and interaction are acknowledged to enhance engagement rates (Birkin and Goulson 2015; Mason and Arathi 2019). For future projects, we would further recommend a survey at the beginning and end of the project to give a better understanding of motivations and how these align with the protocol to ultimately retain more participants (Domroese and Johnson 2016).

We asked participants to take photographs of the mini-meadow/site to aid floral identification. However, photographs were non-standardised, and therefore abundance of individual flower species could not be discerned. More detailed protocol for taking photographs in such projects would be valuable. We also asked the participants to identify insects to broad group with the aid of an ID guide and practice through an online insect identification quiz, when in reality direct training is more desirable to aid identification. A few sessions of remote training can be as effective as one session of direct training, and even a slide show can increase identification accuracy (Ratnieks et al. 2016), and hence could be used to improve accuracy in similar future projects with limited resources.

Conclusion

We recommend that verification of specimen identity by researchers is a valuable component of a hypothesis-led citizen science project such as Sow Wild!, owing to different patterns in data collected in verified specimen-based versus observation-only data sets, and the under-recording of the less conspicuous taxa. We conclude that citizen scientists can follow experimental protocols, collect valuable data on the abundance of the more conspicuous insect groups, and successfully contribute to ecological datasets. Dickinson et al. (2010) state that citizen science should complement traditional researcher-led studies, and Kremen, Ullman, and Thorp (2011) argue that invertebrate monitoring should include citizen scientists and professional experts. This could be achieved with expert verification of all data as we have done, or through a random sub-sample for larger projects. Submission of photographs for verification is also useful for observation projects (Falk et al. 2019), and some studies suggest the collection of reference data will highlight inaccuracies (Aceves-Bueno et al. 2017). Indeed, using unverified data risks drawing incorrect conclusions about rare or declining species (e.g., Gardiner et al. 2012). We conclude that different sampling methods need to be considered when designing a citizen science project, depending on taxa and hypothesis. To monitor a range of beneficial insects, a combination of the methods discussed in this study could be deployed as they are all attractive to different insect groups. To limit by-catch, sampling methods can be used selectively if there is a particular taxon of interest.

Data accessibility Statement

The data that support the findings of this study are openly available in Figshare at http://doi.org/10.25377/sussex.22118291.

Supplemental Files

The Supplementary files for this article can be found as follows: