Accepted on 07 Aug 2017
Submitted on 31 Jan 2017
Worldwide, 35% of the global food supply is highly reliant on animal-mediated pollination (Klein et al. 2007; Nicholls and Altieri 2013). Global estimates of the value of this ecosystem service range from $112 to $200 billion USD annually (Costanza et al. 1997; Kremen et al. 2007; Winfree et al. 2007). Unfortunately, across the United States and Europe, severe declines in the supply of European honey bees (Apis mellifera L.) and wild pollinators have been detected (Aizen and Harder 2009; Gordon et al. 1998; Goulson et al. 2008; Potts et al. 2010). Understanding how changes in pollinator abundance, diversity, and community composition influence the resilience and reliability of pollination services in both natural and managed habitats is a key conservation goal (Dicks et al. 2013). Citizen science is one approach that researchers can take to increase the sampling intensity needed to address these questions, which often require studies over a large geographic area and long time-scale (DEFRA 2014).
Within recent years, there has been tremendous growth in the use of citizen science approaches to study pollinators including managed and wild bees and butterflies. Citizen science has proven useful in examinations of pollinator distributions and declines (Matteson et al. 2012; Moskowitz and Haramaty 2013; Stafford et al. 2010), responses to climate change (Breed et al. 2012), population genetics (Harpur et al. 2015), detection of exotic species (Ashcroft et al. 2012; Wal et al. 2015), nesting and hive properties (Graham et al. 2014; Lye et al. 2012; Sponsler and Johnson 2015), pesticide impacts (Muratet and Fontaine 2015), responses to habitat and landscape features (Bates et al. 2014; Everaars et al. 2011; Kremen et al. 2011), migrations (Davis et al. 2012; Howard and Davis 2015; 2009), overwintering (Howard et al. 2010), disease dynamics (Satterfield et al. 2015), and larval survivorship (Nail et al. 2015).
Researchers also have become engaged in designing and testing citizen science protocols to measure pollination services (Birkin and Goulson 2015, Potter and Lebuhn 2015). Measuring the function of the pollinator community can be done without documenting its taxonomic composition (Birkin and Goulson 2015), which removes the potential for identification errors that can be common in invertebrate citizen science programs (Gardiner et al. 2012). Comparison of variables such as fruit weight and seed set among pollinator-accessible, pollinator-excluded, and hand-pollinated (self-pollinated and/or cross-pollinated) plants is a common method used by researchers to quantify pollination services (Garibaldi et al. 2011, Blaauw and Isaacs 2014). However, engaging citizen scientists in such a study, which necessitates following a relatively complex protocol across multiple months, can present challenges to volunteer retention and data accuracy.
In 2014, we established the citizen science program Pollination Investigators to examine the potential of citizen scientists to collect pollination services data within their home gardens. In the initial year of the program our goals were to 1) Evaluate the suitability of four commonly grown crops as sentinel indicators of pollination services, 2) Determine what step(s) in the experimental process proved the most challenging for citizen scientists using a follow-up survey, and (3) Use our study and survey results to improve upon our citizen science protocol.
To initiate the Pollination Investigators program, we held three 6 hour workshops in Cincinnati, Cleveland, and Wooster, OH, in May 2014. A total of 84 people from across Ohio attended the workshops, which focused on recruiting volunteers for Pollination Investigators and The Buckeye Lady Beetle Blitz (Gardiner et al. 2012) citizen science programs. Two hours of each workshop were devoted to Pollination Investigators program training. Attendees were members of The Ohio State University Master Gardener Program, whose active community volunteers complete an intensive horticultural training program. The mission of the program is to empower trained volunteers to educate others with research-based gardening information. The program focuses on several initiatives including integrated pest management, invasive species detection and eradication/management, backyard and local foods, and environmental horticulture (http://mastergardener.osu.edu).
At our workshop, we explained why we established the Pollination Investigators program, outlined all protocols, and included a hands-on demonstration that showcased how to execute the pollination services experiment. After listening to the program, interested volunteers were given a toolkit that included data collection protocols, data sheets, a plant care handout and pest identification guide, mesh pollinator exclusion bags, and eight vegetable starts. Volunteers were given two seedlings each of cucumber (Cucumis sativus, var Marketmore 76), tomato (Solanum lycopersicum, var Celebrity F1), sweet pepper (Capsicum annuum, var Sweet Bananarama), and sunflower (Helianthus annuus, var Dwarf Sunspot). We selected these crops because they are commonly grown in home gardens and vary in their dependence on insect-mediated pollination services. Tomato, sweet pepper, and sunflower will produce fruit in the absence of insect visitation, but fruit size and seed set increase with exposure to insect pollinators (Greenleaf and Kremen 2006; Hogendoorn et al. 2006; Shipp et al. 1994). Both sweet pepper and tomato are “buzz pollinated” crops and are visited by wild solitary and social pollinators more frequently than managed honey bees (Raw 2000; Winfree et al. 2008). Sunflower is pollinated by both honeybees and wild bees, although wild bees are more effective pollinators (Greenleaf and Kremen 2006; Parker 1981). Cucumber is highly dependent on both honeybees and wild bee pollinators to produce fruit; it has separate male and female flowers and large sticky pollen grains that travel poorly by wind (Ghazoul 2005; Lowenstein et al. 2012; Stanghellini et al. 1997).
Volunteer participants followed a multi-step protocol to measure pollination services. First, each volunteer planted the eight seedlings within their garden. When buds appeared, volunteers selected six flowers to monitor for sweet pepper, tomato, and cucumber plants (female flowers only; photos of male and female flowers were provided in the protocol), and two flowers to monitor for sunflower plants. The flowers could all be on one plant or located on both of the test plants. Three flowers per crop were assigned to an “Open” treatment and the other three to an “Exclusion” treatment, except in the case of the sunflowers, in which the volunteers selected one flower head per treatment. We asked volunteers to set up three sets of the treatments for species that produced multiple flowers per plant because we expected some flowers or developing fruits to be lost due to arthropod feeding, disease infection, or other causes prior to harvest. Pollinators were allowed to visit the Open treatment flowers, which volunteers identified by marking with yellow plastic coated wire ties secured around the plant stem at the base of each flower. Volunteers secured the mesh pollinator exclusion bags over each Exclusion treatment flower bud and marked the plant stem at the base of these flowers with a red plastic coated wire tie. Following flowering, the exclusion bags were removed. For the sunflower plants, the Exclusion treatment remained bagged after flowering until harvest, and a second exclusion bag was secured over the Open treatment following flowering to prevent animals from consuming developing seeds.
Although cucumber flowers are dependent on insect pollination we instructed our volunteers to bag female flowers in this initial investigation to 1) Demonstrate the importance of pollinators in the production of this crop and 2) Determine if any volunteers reported seed set from bagged flowers (an indication that tracking flowers to harvest proved difficult). A plant care handout provided to volunteers indicted a target harvest date of 70–80 days post-transplant for the four crops. Volunteers were asked to harvest their fruit within this timeframe. At harvest volunteers were asked to weigh each Open and Exclusion treatment fruit (seed head in the case of sunflower) and dissect and count all seeds present. They could then enter their data into a fillable online form or mail completed data sheets included in their toolkit to a provided address.
During the experiment, volunteers were also asked to measure sunlight, pest, and disease incidence, and surrounding bloom abundance and bloom area. The proportion of the volunteer’s garden receiving full sunlight was measured on one clear day at 800 h, 1200 h, and 1600 h. Volunteers were asked to identify and count any insect pests present on their test plants at the day of harvest using a provided identification guide. Disease symptoms were categorized using an online photo guide as most closely resembling a mosaic virus, bacterial wilt, bacterial leaf spot, anthracnose fungus, mildew fungus, or rust fungus. Volunteers were asked to report the percentage of the plant exhibiting the symptoms. Finally, to measure nectar plant availability, volunteers divided all garden area in their yard into three relatively equally sized plots. Within each plot they selected a sampling location by tossing a pen from the corner of the plot. Using the pen’s location as the center, they created a three foot by three foot (0.91 m2) quadrat within which all flowers were counted, identified, and measured (length and width).
In the fall of 2015, we constructed a 15-question survey using the program SurveyMonkey® (www.surveymonkey.com) to evaluate Pollination Investigators. The aims of this evaluation were to determine the 1) Effectiveness of our workshop training in explaining the sampling procedures; 2) Ability of the citizen scientists to establish their experiment and complete sampling procedures, and 3) Factors that limited or prevented participation in the program. Before distribution of the survey a research protocol was submitted to the Institutional Review Board (IRB) through the Office of Responsible Research Practices (ORRP) at The Ohio State University, which was determined exempt (Study ID: 2015E0521).
Following ORRP approval we convened a focus group to review the survey. To form this group, we emailed all individuals who received a Pollination Investigators toolkit at one of our three 2014 workshops and asked for volunteers. Five individuals responded who were willing to participate. The goal of the focus group was to uncover any points of confusion within the survey tool, such as unclear questions or incomplete or unclear answer choices for a given question. Edits suggested by focus group participants were noted and changes applied to the survey prior to its release. The final draft of the 15-question survey was distributed to everyone who attended our training workshops approximately one year after completing the program, on September 21, 2015 via email (see Supplementary materials). Recipients were given 14 days to respond to the online survey, with a reminder email sent after 7 days. Survey data were evaluated using Chi-Square Goodness of Fit tests (X2) using XLSTAT version 19.02 software (XLSTAT, 2017).
Following our training workshop, 64 (76.2%) attendees elected to participate in Pollination Investigators and 14.1% (n = 9) of these volunteers submitted at least partial data at the end of the project (Figure 1A). The number of volunteers submitting at least one fruit weight and seed count for an Open and Exclusion treatment ranged from a low of six submissions for sunflower to nine submissions for tomato.
A) The number of individuals who attended Pollination Investigators training workshops, elected to participate, and submitted data. B) Workshop attendees were asked if any protocol steps remained unclear following our training as part of a follow-up survey. Respondents could check all tasks that remained confusing. The answer choices were: Collecting harvested fruit seed counts from test plants, collecting harvested fruit weight from test plants, measuring bloom abundance and area in your garden, measuring percentage sunlight in your garden, and attaching pollinator exclusion bags on test plants. Measuring sunlight and collecting seed counts remained points of confusion to the greatest percentage of respondents.
Of the 84 volunteers who attended one of the three Pollination Investigator programs, 29 (35%) responded to the survey, which began by asking participants if, after listening to our training, they decided to pick up the sampling toolkit for the Pollination Investigators program. Twenty (69.0%) of the respondents selected “yes” and 9 (31.0%) answered “no.” The participants who answered no did not proceed further in the survey and instead were asked “Please explain why you decided against taking a pollination toolkit.” Reasons for opting out were time constraints (3 individuals), protocol complexity/amount of labor involved (2 individuals), previous issues with wildlife consuming vegetable plants (1 individual), a lack of space (1 individual), and not interested/not sure (2 individuals).
For the 20 survey respondents who participated in Pollination Investigators, we next addressed their understanding of the data-collection protocol. This included a survey question that listed five major tasks and asked respondents to check all that they found confusing following our training. We found a significant difference in the number of volunteers selecting each task (X2 = 9.49, df = 4, p = 0.05); collecting sunlight data (50.0%) and counting seeds (38.8%) were confusing to the largest percentages of respondents (Figure 1B).
The majority of respondents established each vegetable crop; 17 (94.4%) respondents indicated that they planted tomato, sunflower, and sweet pepper, and 16 established cucumber seedlings (88.9%). However, from 23.5 to 41.2% of respondents reported that their seedlings did not survive to the flowering stage. Survivorship did not vary significantly among test plant species (X2 = 1.51, df = 3, p = 0.68), but ranged from a low of 52.9% survivorship for cucumber and sunflower seedlings to a high of 70.6% for sweet pepper. The majority of participants who had at least one seedling survive to flowering did initiate the experiment by selecting flowers to monitor, and attaching pollinator exclusion bags to the Exclusion treatment. The percentage of respondents who attached the exclusion bags did not differ among crops (X2 = 1.03, df = 3, p = 0.79). When asked why respondents with plants that survived to flowering were unable to establish their experiment, we found a significant difference in the number of individuals selecting five answer choices (X2 = 20.36, df = 4, p < 0.001). Wildlife damage to flowers (40.0%) and damage to flowers during bag attachment (25.0%) were chosen most frequently (Figure 2).
We asked volunteers who had plants that survived to the flowering stage, yet failed to establish their pollination treatments what went wrong. Volunteers cited wildlife damage to the flowers and damage caused to the flowers during exclusion bag attachment most frequently. Those who selected “other” reported that a lack of time prevented continued participation. No volunteers reported that unclear instructions regarding how to attach exclusion bags or missing the flowering period were obstacles to their continued participation.
We received fruit weight data for cucumber (7 submissions), tomato (9 submissions), and sweet pepper (7 submissions), but no volunteers submitted sunflower head weights. We found a significant difference in the number of times that survey respondents selected among six potential reasons for a lack of fruit weight submission (X2 = 21.22, df = 5, p < 0.001); no fruit set was the most frequently selected answer choice (Figure 3A).
A) We asked volunteers what prevented them from submitting fruit weight data. The answer choices were: The task was too difficult, did not have an adequate scale, no fruit set, didn’t have time, lost track of what fruit I was monitoring, and plants died. Volunteers could check all answers that applied. No fruit set was the most commonly selected answer choice. B) Volunteers were also asked what prevented their successful collection and submission of seed counts. A lack of fruit set followed by plants dying were the most frequently selected answer choices.
The final step in the experiment protocol was to dissect any fruit produced in the Open and Exclusion treatments and count the seeds present. The number of seed counts submitted ranged from five for sunflower to eight for tomato. There was a significant difference in the number of times that volunteers selected among five potential explanations for not submitting seed counts (X2 = 12.38, df = 4, p = 0.02). A lack of fruit set, plants dying prior to harvest, and the task being too difficult were the most frequently cited reasons for not submitting these data (Figure 3B).
Our final survey question asked volunteers if they would be more likely to participate in Pollination Investigators in the future if we removed one or more of the tasks included in the 2014 protocol. We found a significant difference in the number of people electing to remove five potential tasks (X2 = 15.68, df = 4, p = 0.03). The most frequently selected task was counting seeds, which represented 75% of the responses (Figure 4).
The final question in our online survey asked if volunteers would be more likely to participate in Pollination Investigators again if we removed one or more tasks. Respondents could check all tasks that they would like to see removed from the protocol. The answer choices were: Counting seeds, weighing fruit, taking bloom count and area measurements, recording pest and disease incidence, and measuring sunlight. Counting seeds received 75% of the total responses.
Public interest and concern for pollinator conservation has grown dramatically in the last decade given documented declines in managed and wild bee fauna (Domroese and Johnson 2017). Collection of long-term monitoring data is a major contribution that citizen scientists can make toward effective conservation programs (Dickinson et al. 2010). With any invertebrate sampling there is concern that using a citizen science approach can lead to errors (Gardiner et al. 2012; Roy et al. 2016). For example, Gardiner et al. (2012) found that relying on citizen-submitted lady beetle data without expert verification would have resulted in researchers overestimating rare, declining species and underestimating common exotic fauna. However, with training, citizen scientists can collect useful and accurate pollinator information (Kremen et al. 2011; Deguines et al. 2012). Methods such as expert verification of volunteer-collected arthropod specimens (Gardiner et al. 2012), hands-on training (Kremen et al. 2011; Ratnieks et al. 2016), and/or assigning target species to groups based on color or pattern (Roy et al. 2016) all reduce, although do not always eliminate, volunteer errors.
Citizen scientists can also contribute to our scientific knowledge of pollinator ecology by measuring their ecological function (Oberhauser and LeBuhn 2012; Birkin and Goulson 2015). This type of investigation can be done without actually identifying the contributors to pollination service, thus removing the potential for taxonomic errors. However, pollination services studies typically require that citizen scientists commit to an experiment extending across several weeks of the growing season. For example, the Bees ‘n Beans citizen science program quantifies bee activity based on the number and weight of broad bean (Vica faba) pods and seeds produced in pollinator excluded, open-pollinated, and hand cross-pollinated treatments (Birkin and Goulson 2015). Similarly, Pollination Investigators was designed to collect pollination services data using a multi-step experiment that culminates in citizen scientists reporting fruit weights and seed counts from insect-accessible and caged vegetable plant flowers.
In the first year of the Pollination Investigators program, we distributed 63 toolkits to volunteers who attended training workshops and had a data return rate of 14.1% (n = 9). Given this low rate of data submission, we initiated a survey to identify the steps in our protocol that resulted in citizen scientists dropping out of the project. Our survey results highlight plant survivorship rates and protocol complexity as two factors that must be addressed to increase citizen scientist retention throughout the measurement of pollination services.
The majority of citizen scientists who responded to our survey established the seedlings that they were given at our training workshops, however, we lost a substantial number of these volunteers due to plant mortality. Our four sentinel plant species (cucumber, sunflower, sweet pepper, and tomato) exhibited similar mortality rates prior to flowering, ranging from 29.4% for sweet pepper up to 47.1% for cucumber and sunflower. This demonstrates the need for a substantial programmatic focus on tools and training aimed at supporting plant survivorship, growth, and reproduction. Several factors likely contributed to test plant mortality; survey respondents frequently cited damage from wildlife. Options to protect plants from wildlife injury, such as using fencing or deterrent sprays, should be provided to volunteers along with basic plant production protocols and arthropod and disease management guidelines. Importantly, the majority of volunteers with live plants began their experiment by selecting open and exclusion flowers to monitor and attaching pollination exclusion bags (72.7–100% across plant species). Thus, citizen scientists were willing to track plant development for several weeks and establish the necessary experimental treatments to measure pollination. However, a lack of fruit set on surviving plants was the most common reason given for a lack of fruit weight and seed count data submissions by these volunteers. This suggests that plant survivorship and fruit production, rather than the complexity of establishing the experimental treatments, were bottlenecks that prevented willing volunteers from continuing with the study.
Even with increased investment in training and tools, plant mortality and/or a lack of fruit set will likely continue to limit data submission rates. These biological issues will probably necessitate a larger initial sample size than was accomplished in our initial program year (n = 64 participants in 2014). We thought that providing volunteers with seedlings rather than seeds would increase plant survivorship and increase data submission rates, offsetting the higher investment cost per toolkit and reduced enrollment due to volunteers having to attend in-person workshops to obtain program materials. Data from the Bees ‘n Beans program demonstrates that this is not necessarily the case, as a similar data submission rate of 14.5% (versus 14.1% for Pollination Investigators) was obtained from volunteers who were provided with seeds (Birkin and Goulson 2015). Distributing toolkits with seeds allows a larger number of citizen scientists to be recruited, eliminates the need to require in-person training, and allows researcher investment to be redirected from plant production to other program aspects.
Another key goal of our survey was to identify ways to simplify pollination services protocols to increase volunteer participation and retention. Our results illustrate two protocol changes that are likely to aid in this goal. First, when volunteers attempted to establish their pollination experiment by attaching pollinator exclusion bags to flowers, 31.3% said they damaged the flowers and thus stopped participating in the study. Bees ‘n Beans program volunteers placed entire potted plants in a mesh bag instead of bagging individual flowers (Birkin and Goulson 2015), thus 1) reducing likelihood of plant damage when establishing the treatment, 2) eliminating the need to monitor plants frequently for bud formation, 3) allowing fruit weight and seed counts to be generated from a number of fruit per plant without having to keep track of individual flowers/fruit, and 4) reducing the likelihood that the mesh comes in direct contact with floral structures and possibly promoting mechanical pollen transfer. Second, 75% of our participants felt that counting seeds was too onerous a task. Seed data are important in quantifying pollination services; however, this task could be transferred to researchers by providing envelopes for citizen scientists to collect and submit seeds, or by asking participants to submit a photograph of the seeds. Thus, when selecting a sentinel plant, researchers should consider how difficult the seeds will be for volunteers to extract, and the amount of time needed in the laboratory to process seeds or photographs.
Developing a citizen science program often involves a compromise between an ideal statistical design and ensuring sufficient levels of participation to meet the program goals (Pocock et al. 2015). We conclude that citizen scientists are willing and able to follow a relatively complex protocol to establish and collect data from multiple pollination services treatments. However, plant mortality hindered the majority of those unable to complete the study. Importantly, we found that our investment in the production of seedlings versus seeds did not necessarily result in greater rates of plant survivorship or volunteer retention. Therefore, regardless of the plant material distributed, providing as much information as possible regarding proper plant care, pest and disease management, and wildlife deterrence is critical to the success of any citizen science program studying pollination services.
Further, scientists must be mindful that a season-long study involves a greater time commitment than some biomonitoring activities with which volunteers might be familiar. In our initial program year, we included four commonly grown indicator plants that vary in their dependence on insect-mediated pollination. Although we felt that including multiple indicators to quantify pollination within a site was ideal, selecting one plant species will reduce the time commitment placed on willing volunteers and perhaps encourage greater participation. Further, shifting the task of seed counting from citizen scientists to program staff and providing volunteers with a simplified method to exclude pollinators from no-pollinator control treatments would simplify the tasks assigned to citizen scientists.
The additional file for this article can be found as follows:
Supplementary MaterialsSurvey sent to all Pollination Investigators participants. DOI: https://doi.org/10.5334/cstp.99.s1
We thank all the citizen scientists who participated in Pollination Investigators and those who responded to our survey. Thank you to Jean Haley of the North Central Integrated Pest Management Center for her guidance on developing our survey. We also thank two anonymous reviewers whose helpful feedback greatly improved our manuscript. This project was funded by the USDA NIFA Crop Protection and Pest Management Competitive Grants Program (Grant number: 2014-70006-22507).
The authors have no competing interests to declare.
Aizen, M. and Harder, L., 2009. The truth about honeybees. New Scientist 204(2731): 26–27. DOI: https://doi.org/10.1016/S0262-4079(09)62810-4
Ashcroft, M.B., Gollan, J.R. and Batley, M., 2012. Combining citizen science, bioclimatic envelope models and observed habitat preferences to determine the distribution of an inconspicuous, recently detected introduced bee (Halictus smaragdulus Vachal Hymenoptera: Halictidae) in Australia. Biological Invasions 14(3): 515–527. DOI: https://doi.org/10.1007/s10530-011-0092-x
Bates, A.J., Sadler, J.P., Grundy, D., Lowe, N., Davis, G., Baker, D., Bridge, M., Freestone, R., Gardner, D., Gibson, C., Hemming, R., Howarth, S., Orridge, S., Shaw, M., Tams, T. and Young, H., 2014. Garden and landscape-scale correlates of moths of differing conservation status: significant effects of urbanization and habitat diversity. PLoS One 9: e86925. DOI: https://doi.org/10.1371/journal.pone.0086925
Birkin, L. and Goulson, D., 2015. Using citizen science to monitor pollination services. Ecological Entomology 40(Suppl S1): 3–11. DOI: https://doi.org/10.1111/een.12227
Blaauw, B.R. and Isaacs, R., 2014. Flower plantings increase wild bee abundance and the pollinations services provided to a pollination-dependent crop. Journal of Applied Ecology 51(4): 890–898. DOI: https://doi.org/10.1111/1365-2664.12257
Breed, G.A., Stichter, S. and Crone, E.E., 2012. Climate-driven changes in northeastern US butterfly communities. Nature Climate Change 3: 142–145. DOI: https://doi.org/10.1038/nclimate1663
Costanza, R., D’Arge, R., de Groot, R., Farber, S., Grasso, M., Hannon, B., Limburg, K., Naeem, S., O’Neill, R.V., Paruelo, J., Raskin, R.G., Sutton, P. and van den Belt, M., 1997. The value of the world’s ecosystem services and natural capital. Nature 387: 253–260. DOI: https://doi.org/10.1038/387253a0
Davis, A.K., Nibbelink, N.P. and Howard, E., 2012. Identifying large- and small-scale habitat characteristics of monarch butterfly migratory roost sites with citizen science observations. International Journal of Zoology. DOI: https://doi.org/10.1155/2012/149026
Deguines, N., Julliard, R., de Flores, M. and Fontaine, C., 2012. The whereabouts of floral visitors: contrasting land-use preferences revealed by a country-wide survey based on citizen science. PLoS ONE 7: e45822. DOI: https://doi.org/10.1371/journal.pone.0045822
Department for Environment Food and Rural Affairs (DEFRA), 2014. The National Pollinator Stragegy: for bees and other pollinators in England. www.gov.uk/government/publications.
Dickinson, J.L., Zuckerberg, B. and Bonter, D.N., 2010. Citizen science as an ecological research tool: challenges and benefits. Annual Review of Ecology, Evolution, and Systematics 41: 149–172. DOI: https://doi.org/10.1146/annurev-ecolsys-102209-144636
Dicks, L.V., Abrahams, A., Atkinson, J., Biesmeijer, J., Bourn, N., Brown, C., Brown, M.J.F., Carvell, C., Connolly, C., Cresswell, J.E., Croft, P., Darvill, B., De Zylva, P., Effingham, P., Fountain, M., Goggin, A., Harding, D., Harding, T., Hartfield, C., Heard, M.S., Heathcote, R., Heaver, D., Holland, J., Howe, M., Hughes, B., Huxley, T., Kunin, W.E., Little, J., Mason, C., Memmott, J., Osborne, J., Pankhurst, T., Paxton, R.J., Pocock, M.J.O., Potts, S.G., Power, E.F., Raine, N.E., Ranelagh, E., Roberts, S., Saunders, R., Smith, K., Smith, R.M., Sutton, P., Tilley, L.A.N., Tinsley, A., Tonhasca, A., Vanbergen, A.J., Webster, S., Wilson, A. and Sutherland, W.J., 2013. Identifying key knowledge needs for evidence-based conservation of wild insect pollinators: A collaborative cross-sectoral exercise. Insect Conservation and Diversity 6(3): 435–446. DOI: https://doi.org/10.1111/j.1752-4598.2012.00221.x
Domroese, M.C. and Johnson, E.A., 2017. Why watch bees? Motivations of citizen science volunteers in the Great Pollinator Project. Biological Conservation 208: 40–47. DOI: https://doi.org/10.1016/j.biocon.2016.08.020
Everaars, J., Strohbach, M.W., Gruber, B. and Dormann, C.F., 2011. Microsite conditions dominate habitat selection of the red mason bee (Osmia bicornis, Hymenoptera: Megachilidae) in an urban environment: A case study from Leipzig, Germany. Landscape and Urban Planning 103(1): 15–23. DOI: https://doi.org/10.1016/j.landurbplan.2011.05.008
Gardiner, M.M., Allee, L.L., Brown, P.M., Losey, J.E., Roy, H.E. and Smyth, R.R., 2012. Lessons from lady beetles: accuracy of monitoring data from US and UK citizen-science programs. Frontiers in Ecology and the Environment 10(9): 471–476. DOI: https://doi.org/10.1890/110185
Garibaldi, L.A., Steffan-Dewenter, I., Kremen, C., Morales, J.M., Bommarco, R., Cunningham, S.A., Carvalheiro, L.G., Chacoff, N.P., Dunenhoffer, J.H., Greenleaf, S.S., Holzschuh , Issacs, R., Krewenka, K., Mndelik, Y., Mayfield, M.M., Morandin, LA., Potts, S.G., Ricketts, T.H., Szengyorgi, H., Viana, B.G., Westphal, C., Winfree, R. and Klien, A.M., 2011. Stability of pollination services decreases with isolation from natural areas despite honey bee visits. Ecology Letters 14(10): 1062–1072. DOI: https://doi.org/10.1111/j.1461-0248.2011.01669.x
Ghazoul, J., 2005. Buzziness as usual? Questioning the global pollination crisis. Trends in Ecology and Evolution 20(7): 367–373. DOI: https://doi.org/10.1016/j.tree.2005.04.026
Gordon, A.W., Bernhardt, P., Ron, B., Burquez, A., Buchmann, S., Cane, J., Cox, P.A., Dalton, V., Feinsinger, P., Ingram, M., Inouye, D., Jones, C.E., Kennedy, K., Kevan, P., Koopowitz, H., Medellin, R., Medellin-Morales, S., Nabhan, G.P., Pavlik, B., Tepedino, V., Torchio, P. and Walker, S., 1998. The Potential Consequences of Pollinator Declines on the Conservation of Biodiversity and Stability of Food Crop Yields. Conservation Biology 12(1): 8–17. URL: http://www.jstor.org/stable/2387457. DOI: https://doi.org/10.1046/j.1523-1739.1998.97154.x
Goulson, D., Lye, G.C. and Darvill, B., 2008. Decline and conservation of bumble bees. Annual Review of Entomology 53: 191–208. DOI: https://doi.org/10.1146/annurev.ento.53.103106.093454
Graham, J., Tan, Q., Jones, L. and Ellis, J., 2014. Native Buzz: Citizen scientists creating nesting habitat for solitary bees and wasps. Florida Scientist 77(4): 204–218. URL: http://www.jstor.org/stable/24321925
Greenleaf, S.S. and Kremen, C., 2006. Wild bees enhance honey bees’ pollination of hybrid sunflower. Proceedings of the National Academy of Science U. S. A. 103(37): 13890–13895. DOI: https://doi.org/10.1073/pnas.0600929103
Harpur, B., Chapman, N., Krimus, L., Maciukiewicz, P., Sandhu, V., Sood, K., Lim, K., Rinderer, T.E., Allsopp, M.H. and Oldroyd, B.P., 2015. Assessing patterns of admixture and ancestry in Canadian honey bees. Insectes Sociaux 62(4): 479–489. DOI: https://doi.org/10.1007/s00040-015-0427-1
Hogendoorn, K., Gross, C.L., Sedgley, M. and Keller, M., 2006. Increased tomato yield through pollination by native Australian Amegilla chlorocyanea (Hymenoptera: Anthophoridae). Journal of Economic Entomology 99(3): 828–833. DOI: https://doi.org/10.1603/0022-0493-99.3.828
Howard, E., Aschen, H. and Davis, A.K., 2010. Citizen science observations of monarch butterfly overwintering in the southern United States. Psyche: A Journal of Entomology 2010. DOI: https://doi.org/10.1155/2010/689301
Howard, E. and Davis, A.K., 2015. Investigating long-term changes in the spring migration of monarch butterflies (Lepidoptera: Nymphalidae) using 18 years of data from Journey North, a citizen science program. Annals of the Entomological Society of America 108(5): 664–669. DOI: https://doi.org/10.1093/aesa/sav061
Howard, E. and Davis, A.K., 2009. The fall migration flyways of monarch butterflies in eastern North America revealed by citizen scientists. Journal of Insect Conservation 13(3): 279–286. DOI: https://doi.org/10.1007/s10841-008-9169-y
Klein, A.-M., Vaissière, B.E., Cane, J.H., Steffan-Dewenter, I., Cunningham, S.A., Kremen, C. and Tscharntke, T., 2007. Importance of pollinators in changing landscapes for world crops. Procedings of Biological Science 274(1608): 303–313. DOI: https://doi.org/10.1098/rspb.2006.3721
Kremen, C., Ullman, K.S. and Thorp, R.W., 2011. Evaluating the quality of citizen-scientist data on pollinator communities. Conservation Biology 25(3): 607–617. DOI: https://doi.org/10.1111/j.1523-1739.2011.01657.x
Kremen, C., Williams, N.M., Aizen, M.A., Gemmill-Herren, B., LeBuhn, G., Minckley, R., Packer, L., Potts, S.G., Roulston, T., Steffan-Dewenter, I., Vázquez, D.P., Winfree, R., Adams, L., Crone, E.E., Greenleaf, S.S., Keitt, T.H., Klein, A.M., Regetz, J. and Ricketts, T.H., 2007. Pollination and other ecosystem services produced by mobile organisms: A conceptual framework for the effects of land-use change. Ecology Letters 10(4): 299–314. DOI: https://doi.org/10.1111/j.1461-0248.2007.01018.x
Lowenstein, D.M., Huseth, A.S. and Groves, R.L., 2012. Response of wild bees (Hymenoptera: Apoidea: Anthophila) to surrounding land cover in Wisconsin pickling cucumber. Environmental Entomology 41(3): 532–540. DOI: https://doi.org/10.1603/EN11241
Lye, G.C., Osborne, J.L., Park, K.J. and Goulson, D., 2012. Using citizen science to monitor Bombus populations in the UK: Nesting ecology and relative abundance in the urban environment. Journal of Insect Conservation 16(5): 697–707. DOI: https://doi.org/10.1007/s10841-011-9450-3
Matteson, K.C., Taron, D.J. and Minor, E.S., 2012. Assessing citizen contributions to butterfly monitoring in two large cities. Conservation Biology 26(3): 557–564. DOI: https://doi.org/10.1111/j.1523-1739.2012.01825.x
Moskowitz, D. and Haramaty, L., 2013. National Moth Week – a new global citizen science project focused on moths. Terrestrial Arthropod Review 6: 185–200. DOI: https://doi.org/10.1163/18749836-06021065
Muratet, A. and Fontaine, B., 2015. Contrasting impacts of pesticides on butterflies and bumblebees in private gardens in France. Biological Conservation 182: 148–154. DOI: https://doi.org/10.1016/j.biocon.2014.11.045
Nail, K., Stenoien, C. and Oberhauser, K.S., 2015. Immature monarch survival: Effects of site characteristics, density, and time. Annals of the Entomological Society of America 108(5): 680–690. DOI: https://doi.org/10.1093/aesa/sav047
Nicholls, C.I. and Altieri, M.A., 2013. Plant biodiversity enhances bees and other insect pollinators in agroecosystems. A review. Agronomy for Sustainable Development 33(2): 257–274. DOI: https://doi.org/10.1007/s13593-012-0092-y
Oberhauser, K. and LeBuhn, G., 2012. Insects and plants: engaging undergraduates in authentic research through citizen science. Frontiers in Ecology and the Environment 10(6): 318–320. DOI: https://doi.org/10.1890/110274
Parker, F.D., 1981. How efficient are bees in pollinating sunflowers? Journal of the Kansas Entomological Society 54(1): 61–67. URL: http://www.jstor.org/stable/25084133.
Pocock, M.J.O., Newson, S.E., Henderson, I.G., Peyton, J., Sutherland, W.J., Noble, D.G., Ball, S.G., Beckmann, B.C., Biggs, J., Brereton, T., Bullock, D.J., Buckland, S.T., Edwards, M., Eaton, M.A., Harvey, M.C., Hill, M.O., Horlock, M., Hubble, D.S., Julian, A.M., Mackey, E.C., Mann, D.J., Marshall, M.J., Medlock, J.M., O’Mahony, E.M., Pacheco, M., Porter, K., Prentice, S., Procter, D.A., Roy, H.E., Southway, S.E., Shortall, C.R., Stewart, A.J.A., Wembridge, D.E., Wright, M.A. and Roy, D.B., 2015. Developing and enhancing biodiversity monitoring programmes: a collaborative assessment of priorities. Journal of Applied Ecology 52(3): 686–695. DOI: https://doi.org/10.1111/1365-2664.12423
Potter, A. and Lebuhn, G., 2015. Pollination service to urban agriculture in San Francisco, CA. Urban Ecosystems 18(3): 885–893. DOI: https://doi.org/10.1007/s11252-015-0435-y
Potts, S.G., Biesmeijer, J.C., Kremen, C., Neumann, P., Schweiger, O. and Kunin, W.E., 2010. Global pollinator declines: Trends, impacts and drivers. Trends in Ecology and Evolution 25(6): 345–353. DOI: https://doi.org/10.1016/j.tree.2010.01.007
Ratnieks, F.L.W., Schrell, F., Sheppard, R.C., Brown, E., Bristow, O.E. and Garbuzov, M., 2016. Data reliability in citizen science: learning curve and the effects of training method, volunteer background and expereince on identification accuracy of insects visiting ivy flowers. Methods in Ecology and Evolution 7(10): 1226–1235. DOI: https://doi.org/10.1111/2041-210X.12581
Raw, A., 2000. Foraging behaviour of wild bees at hot pepper flowers (Capsicum annuum) and its possible influence on cross pollination. Annals of Botany 85(4): 487–492. DOI: https://doi.org/10.1006/anbo.1999.1090
Roy, H.E., Baxter, E., Saunders, A. and Pocock, M.J.O., 2016. Focal plant observations as a standardised method for pollinator monitoring: opportunitites and limitations for mass participation citizen science. PLoS ONE 11(3): e0150794. DOI: https://doi.org/10.1371/journal.pone.0150794
Satterfield, D.A., Maerz, J.C. and Altizer, S., 2015. Loss of migratory behaviour increases infection risk for a butterfly host. Proceedings of the Royal Society B: Biological Sciences 282(1801): 20141734. DOI: https://doi.org/10.1098/rspb.2014.1734
Shipp, J.L., Whitfield, G.H. and Papadopoulos, A.P., 1994. Effectiveness of the bumble bee, Bombus impatiens Cr. (Hymenoptera: Apidae), as a pollinator of greenhouse sweet pepper. Scientia Horticulturae 57: 29–39. DOI: https://doi.org/10.1016/0304-4238(94)90032-9
Sponsler, D.B. and Johnson, R.M., 2015. Honey bee success predicted by landscape composition in Ohio, USA. PeerJ 3: e838. DOI: https://doi.org/10.7717/peerj.838
Stafford, R., Hart, A.G., Collins, L., Kirkhope, C.L., Williams, R.L., Rees, S.G., Lloyd, J.R. and Goodenough, A.E., 2010. Eu-social science: the role of internet social networks in the collection of bee biodiversity data. PLoS ONE 5(12): e14381. DOI: https://doi.org/10.1371/journal.pone.0014381
Stanghellini, M.S., Ambrose, J.T. and Schultheis, J.R., 1997. The effects of honey bee and bumble bee pollination on fruit set and abortion of cucumber and watermelon. American Bee Journal 137: 386–391.
van der Wal, R., Anderson, H., Robinson, A., Sharma, N., Mellish, C., Roberts, S., Darvill, B. and Siddharthan, A., 2015. Mapping species distributions: A comparison of skilled naturalist and lay citizen science recording. Ambio 44(Suppl 4): 584–600. DOI: https://doi.org/10.1007/s13280-015-0709-x
Winfree, R., Griswold, T. and Kremen, C., 2007. Effect of human disturbance on bee communities in a forested ecosystem. Conservation Biology 21(1): 213–223. DOI: https://doi.org/10.1111/j.1523-1739.2006.00574.x
Winfree, R., Williams, N.M., Gaines, H., Ascher, J.S. and Kremen, C., 2008. Wild bee pollinators provide the majority of crop visitation across land-use gradients in New Jersey and Pennsylvania, USA. Journal of Applied Ecology 45(3): 793–802. DOI: https://doi.org/10.1111/j.1365-2664.2007.01418.x
XLSTAT, 2017. Data Analysis and Statistics Software for Microsoft Excel. Version 19.02. Addinsoft. New York, NY.