Participation in citizen science projects has been on the rise in recent decades (Shirk et al. 2012). Members of the public can participate in projects spanning many different contexts, spatial scales, and areas of interest. With increases in web-based citizen science projects and tools alongside traditional efforts, the possibilities for participation may seem limitless (Geoghegan et al. 2016; Tulloch et al. 2013). Individuals’ participation in citizen science efforts can vary dramatically, from collecting data for someone else’s project to working alongside researchers to develop and execute nearly every part of a project. However, there is a general dearth of knowledge about what motivates individuals to participate in different types of citizen science efforts, particularly co-created projects, and why some individuals persist when others drop out. We sought to examine the motivations and barriers to participation in a pilot project designed to blend online cyberlearning resources and online collaborative tools with traditional in-person efforts to support volunteers’ development and delivery of co-created citizen science projects.
Citizen science projects have been defined as ranging along a spectrum of contributory, collaborative, and co-created projects. Contributory projects are generally designed by scientists, and members of the public contribute to such projects primarily through data collection. While collaborative projects are also designed by scientists and involve data contributions by the public, the public may be involved in project refinement, design, data analysis, or the sharing of results. Co-created projects are designed by scientists and members of the public working together, where at least some of the participants are actively involved in most, if not all, steps of the research process (Bonney et al. 2009a). Co-created citizen science projects represent the highest level of volunteer engagement (Shirk et al. 2012), and alongside collaborative projects, may provide a platform to reach new audiences with topics and approaches that are relevant to individual and community needs and interests (Bonney et al. 2009a). However, the move of currently engaged volunteers from projects that are largely contributory toward more co-created projects is slow (Geoghegan et al. 2016).
Previous research on citizen science projects shows numerous possible outcomes for individuals (e.g., increased knowledge, skills), the scientific field at-large (e.g., increased capacity for accurate data collection, advancement of scientific understanding) and for socio-ecological systems (e.g., potential increased trust among stakeholders, faster decision-making responses; see Shirk et al. 2012). Participation in co-created projects also may provide individuals with a greater understanding of the scientific process, their community structure, social context, environmental regulation, and opportunities to communicate their findings to the public (Bonney et al. 2016). Some drawbacks to co-created projects include a higher investment of time required by those managing the project and serving in supporting roles (e.g., scientists, subject-matter experts) and the potential for slower project completion (Bonney et al. 2009a; Wilderman et al. 2004). Increased social interactions are often necessary in co-created projects to deepen understanding and sustain participation, and these may be viewed as either a motivation or barrier to participation for those working on the project (Shirk et al. 2012).
Self-determination theory offers a theoretical lens with which to examine motivations and barriers to participate and persist in such projects (Ryan and Deci 2000a; Ryan and Deci 2000b). According to this theory, intrinsic motivation is enhanced in situations that heighten feelings of competence, autonomy, and relatedness. Thus, individuals may be more likely to persist in co-created projects if they are able to overcome challenges (competence), have opportunities for self-direction (autonomy), and feel a sense of belonging (relatedness). Intrinsic motivation may serve as an initial driver toward participation, but it also may develop in the course of working on a project, which may encourage participant persistence even in the face of project challenges.
Research on the factors that motivate participation and retention in citizen science projects is still in its infancy, but existing data show some distinctions between motivations for participating in field-based versus online, crowdsourced projects. Previous studies on local, community-based citizen science projects have shown that opportunities for in-person social interaction among participants, conservation of natural resources, and fulfillment of a general interest in conservation are common motivators for participation (Bradford and Israel 2004; King and Lynch 1998). In contrast, research on online, crowdsourced citizen science projects has shown that contributing to scientific research and bolstering participants’ online reputations are important motivators (Nov et al. 2014; Raddick et al. 2010; Raddick et al. 2014). Moreover, participant motivations have been shown to fluctuate over the course of an online project (Rotman et al. 2012) and to differ based on the quality or quantity of participation in online environments (Nov et al. 2014). Many of these purely online, crowdsourced citizen science projects are broadly focused on science, technology, engineering, and mathematics (STEM) topics, while community-based projects are often focused on local environments and their natural history. Motivations and barriers to participation may differ depending on the project focus, regardless of whether the project is primarily in-person or online (Geoghegan et al. 2016).
Recruitment and retention are major challenges in any citizen science effort, but research on online, crowdsourced projects demonstrate exceptionally high turnover rates (Butler 2001; Chiu et al. 2006; Crimmins et al. 2014; Nov et al. Anderson 2011). Matching project activities with volunteer motivations may be the best method for achieving volunteer satisfaction and retention and also may enhance recruitment for individuals with diverse interests and backgrounds (Chu et al. 2012; Stukas et al. 2009). Two studies on engaging new audiences in citizen science projects revealed that the highest level of interest arose from projects that were highly relevant to the pressing needs and issues of the community (Bonney et al. 2009a; Garibay 2004; Garibay 2009). Citizen science projects that align with participants’ existing hobbies, previously existing interests, or previous engagement in other activities also may increase the salience and motivation to participate (Bonney et al. 2009a; Rotman et al. 2012).
Once individuals are recruited, effective training and collaborative learning may be strong contributors to enhanced citizen science participation, particularly in co-created projects. Online tools and resources provide a cost-effective option for that training (Ellul et al. 2001; Lye et al. 2012; Silvertown et al. 2013; Sunde and Jessen 2013). Augmenting cyberlearning with social media tools in citizen science projects may provide opportunities to bridge seemingly divergent motivations across diverse participants.
One tool designed to allow individuals or mediated groups to engage in collaborative learning that may be particularly useful in co-created projects is mental modeling (Gray et al. 2017). This technique uses fuzzy-logic cognitive mapping (Kosko 1986) in which participants develop concept maps of components of their system of interest, the positive and negative relationships of components within that system, and the relative strengths of those relationships (e.g., high, medium, low) (Gray et al. 2017). Collaborative learning through mental modeling has been used in group settings to model the structure and function of a range of systems related to complex environmental issues, which can assist in planning efforts such as coastal disaster planning (see Gray et al. 2015; Gray et al. 2017; Henly-Shepard et al. 2015).
Collaborative learning also can foster communities of practice for those working on the project (Lave and Wenger 1991). In this setting, a community of practice refers to the social interactions that take place during the process of developing and executing a co-created citizen science project that moves people from novice at the start to more knowledgeable and skillful over time. Communities of practice may emerge in both field-based and online settings, and developing a sense of community between scientists and participants in online environments may be particularly important (Bell et al. 2008; Clary and Snyder 1999; Ryan et al. 2001; Newman et al. 2017; Mankowski et al. 2011). Participating in such a community may lead to shifts in participant identities that become more aligned with the project (Lave and Wenger 1991). These positive social interactions can bolster an individuals’ sense of relatedness, which can further enhance their motivation for the project (Ryan and Deci 2000a; Ryan and Deci 2000b).
In this study, we investigate the motivations and barriers for volunteers’ participation in a pilot project that combines cyberlearning and traditional in-person resources to support volunteers in the development and execution of co-created citizen science projects. The cyberlearning components include self-guided online training; collaboration through social networking; and mental modeling. These web-based resources are used in tandem with in-person training and meetings to facilitate the work of citizen scientists in developing, executing, and evaluating their co-created projects. We are particularly interested in how this unique blend of online and in-person resources may sustain or enhance volunteers’ motivation to persist in the project by addressing individual’s needs for competence, autonomy, and relatedness (Ryan and Deci 2000a; Ryan & Deci 200b).
The Virginia Master Naturalist (VMN) program is a community-based volunteer program that engages volunteers in environmental education, stewardship, and citizen science projects across the state of Virginia, USA. Through local chapters, VMN volunteers train and serve alongside a myriad of partnering groups and organizations to address locally relevant conservation needs. In 2015, volunteers contributed more than 35,500 hours toward a range of citizen science projects supporting local, state, and national initiatives (Prysby 2015). Volunteers in the VMN program display demographic characteristics that are representative of participants of many citizen science projects: Predominantly white, female, highly educated, and over 50 years of age (Merenlender et al. 2016).
Our pilot project augmented traditional teaching methods and project resources currently used in VMN citizen science projects with an online, collaborative learning approach. We aimed to provide a new suite of online resources to be used alongside traditional methods to motivate volunteers to develop co-created citizen science projects to benefit their local communities. All online project components were facilitated on a web-based platform with each project having its own site (see Gray et al. 2017). To recruit participants for this project, VMN staff sent emails to all VMN volunteers and chapter leaders to encourage participation. They also delivered presentations at local VMN chapters and statewide events. Recruitment began in January 2013 and concluded in January 2015.
Approximately eight hours of self-guided online training videos and in-person meetings provided information about the scientific method, ecosystem functions, adaptive management techniques, using mental models, and project assessment. Each online training session was followed by a short test for participants to evaluate their knowledge of the content. The online training was self-paced and completed before interested volunteers came together to collaborate and ultimately develop their local citizen science project. The process that each group followed during project development mirrored five steps in adaptive management: (1) discuss; (2) plan; (3) implement; (4) evaluate; and (5) share (Gray et al. 2017; Foundations of Success 2009).
During the first and second steps, volunteers were encouraged to exchange ideas with each other, key stakeholders, and subject-matter experts using online discussion forums and/or in-person meetings to define a conservation issue of concern for their local community. Some participants entered the project with a clear issue in mind whereas others did not. They articulated their ideas and understanding of the issue through individual and collaborative modeling (Gray et al. 2013). Modeling was facilitated by the online Mental Modeler software tool embedded in our project website (Gray et al. 2013).
Mental Modeler allowed individual volunteers an opportunity to represent the dynamics of a targeted environmental problem, share ideas collaboratively, discuss models with other project stakeholders (e.g., participants, site managers, subject-matter experts), and revise these models to inform project goals and project design. Individuals first began using the software to develop their own individual mental models representing their understanding of the key components and relationships associated with their issue. Next, the entire group came together to develop a single collaborative group model. To facilitate group engagement with the modeling software and encourage meaningful discussion, the group mental modeling sessions were synchronous and led by members of the research team either in person or using online video conference software (e.g., WebEx). Specific subject matter experts were sought out and engaged by the volunteers and contributed expertise and guidance through email exchanges, phone conversations, in-person meetings, and mental modeling sessions.
We gave project participants multiple opportunities to revise and discuss their problem definitions and models as they worked alongside other stakeholders to finalize the detailed plan for their co-created citizen science projects. Participants created their own individual mental models to represent their understanding of the complexities of this issue and then came together to collaboratively develop a single group model. An example of a collaborative mental model developed by a volunteer group interested in investigating potential solutions for Johnson grass (Sorghum halepense) removal and native grass restoration at a national monument site is provided in 1. Conversations by project volunteers and park staff and an expert on grassland plant management prior to the modeling sessions revealed key considerations in research design, possible removal treatments, and candidates for native species to use in restoration. Two collaborative group modeling sessions allowed the volunteers to formalize their understanding of key components and drivers of this issue before they formulated their project plan (e.g., experimental design, research questions) in concert with those key stakeholders.
At step three (implement), each group executed its citizen science project, which may have consisted of field work, applying different treatments through experimental design, collecting data, and analyzing data with respect to research questions. Step four (evaluate) was an iterative process in which participants reflected on the successes and failures of the project design, how best to address findings from data analysis moving forward, and elements requiring adjustment and re-design. Participants also self-evaluated the extent to which the environmental and social goals of the project were reached. Finally, participants were encouraged to share lessons learned from working on their project with other citizen scientists via the website.
To examine volunteer participation and persistence, two of the authors (serving as volunteer co-created project coordinators) developed a survey consisting of 27 questions focused on participant motivations, scientific knowledge, self-efficacy, previous experiences related to the project, and demographic information. Because the survey was distributed at a very early stage in the project, we used questions adapted from previous research and from prior experience in similar projects (Bradford and Israel 2004; Brossard et al. 2005; Crall et al. 2013; King and Lynch 1998). The prospective volunteers were given a choice of a paper or online survey. Sixteen volunteers took the paper survey and 47 took the online survey. Because 11 volunteers did not complete the majority of the survey, we removed them prior to analysis, resulting in a final sample of 52 volunteers. The survey questions are available as a supplementary file.
Participants responded to survey questions about their level of prior experience in activities relevant to the project using a four-point Likert-type scale, ranging from “none” to “lots” (Table 1) and the frequency in which they participated in these activities, using a seven-point Likert-type scale ranging from “almost never” to “daily” (Table 2). These survey questions provide an indirect measurement of competency as defined by self-determination theory. To assess motivations, survey participants ranked their primary, secondary, and tertiary motivations to participate from a list of seven potential motivators taken from previous research (Table 3; Bradford and Israel 2004; Crall et al. 2013; Cox et al. 2015; Curtis 2015; Land-Zandstra et al. 2016; Raddick et al. 2010; Wright et al. 2015). We included one “other” open-ended category so that individuals could provide responses not on the list. Demographic survey items (i.e., gender, age, race, education, income, and employment status) also were included at the end of the survey. Survey data were analyzed with descriptive statistics, and an independent-samples t-test analysis was used to compare the responses of drop-outs to volunteers who persisted.
|Activity||Valid % (N = 51)|
|Searching the Internet for information||0||2.0||31.4||66.7|
|Using a computer||0||2.0||33.3||64.7|
|Using social networking sites (e.g., Facebook)||27.5||23.5||25.5||23.5|
|Participating in conservation projects||11.8||25.5||45.1||17.6|
|Planning or leading conservation projects||54.9||21.6||19.6||3.9|
|Sharing the results of a conservation project||54.9||27.5||11.8||5.9|
|Evaluating conservation projects||58.8||29.4||9.8||2.0|
|Activity||Valid % (N = 49)|
|Almost never||Less than once a month||Once a month||2–3 times a month||Once a week||2–3 times a week||Daily|
|Searching for information online||0||0||0||4.0||4.0||28.0||64.0|
|Spending time outdoors in natural settings||0||0||2.0||8.2||10.2||24.5||55.1|
|Educating others about environmental issues||12.0||32.0||20.0||18.0||4.0||10.0||4.0|
|Writing or calling politicians to express opinions on environmental issues||48.0||36.0||10.0||2.0||0||0||4.0|
|Participating in citizen science projects||18.0||18.0||24.0||16.0||10.0||12.0||2.0|
|Attending community events related to environmental issues||12.0||28.0||34.0||18.0||6.0||2.0||0|
|Creating a volunteer project focused on the environment||48.0||28.0||18.0||4.0||2.0||0||0|
|Motivator||% (N = 52)|
|Interest in the environment||36.5||17.3||19.2|
|Interest in protecting a local natural area or resource||21.2||21.2||15.4|
|General interest in science||15.4||7.7||5.8|
|Interest in natural resource management||11.5||15.4||19.2|
|Interest in citizen science||7.7||23.1||26.9|
Out of the 63 volunteers who began the project, 36 persisted through training and project participation, whereas 27 dropped out at some stage of the training or project implementation. As such, the dropout rate for this project was 43%. Of the 52 volunteers who completed the survey, 34 remained in the project and 18 did not. Participants who completed the survey were representative of the demographics of those who started in the project. To better understand the dropout rate, two members of the research team conducted semi-structured telephone interviews with individuals who dropped out to learn more about their motivations and barriers to participation. We contacted all of the volunteers who completed the training but dropped out (N = 21) in addition to other volunteers who dropped out prior to completing the training. Fifteen interviews were conducted (56% response rate of all who dropped out). Of those interviewed, seven volunteers completed the training and started a project (47% of total interview respondents).
Interview questions gathered data on the overall participant experience, experience with the website and associated tools, what participants learned through working with a team, and suggestions for project improvement to facilitate higher retention rates. Participants were asked about their goals for participating, and if they did participate, whether their goals were met. Participants ranked seven pre-defined motivations for participating in the project derived from a study by Bruyere and Rappe (2007) for comparison. The question read: “Please rank each of the following motivators as a high, medium, or low motivator to join the project: (1) allow me to work in or improve an area I care about; (2) help the environment; (3) work on an interesting project; (4) learn about nature; (5) makes me feel happy or good; (6) meet people/work with friends; (7) experience that may be helpful to my career.” Volunteers who started the self-guided online training were then asked why they didn’t use the online tools and discussion as much as anticipated. Following this question, all participants were asked “Please rank each of the following barriers as a high, medium, or low barrier to continuing in the project: (1) I didn’t have enough time; (2) There was not enough social interaction; (3) I was unfamiliar with the online tools; (4) I was not motivated to continue; (5) I did not have good Internet access; (6) there were technical problems with the online tools.” For all of the open-ended questions, three researchers coded all responses for construct validity and finished when 100% agreement was attained. Interview responses were then coded using a thematic analysis by one of the authors. Self-determination theory also was applied after these analyses had been completed to provide a theoretical framework with which to examine elements of competence, autonomy, and relatedness that emerged from the initial interview analysis.
Results from the survey (N = 52) showed a demographic profile of participants similar to other related citizen science programs (Merenlender et al. 2016): Predominantly white (95.8%); highly educated, achieved a degree beyond high school (95.9%); and $50,000 (USD) or more in annual gross total income (66.7%). Forty-nine percent were female (three respondents chose not to disclose gender), 90% were 50 years or older, and many were retired from their primary career (75.5%). Only two respondents were under the age of 40. Fifteen respondents had obtained masters degrees and seven respondents had obtained terminal degrees (e.g., PhD, MD, JD).
Survey responses for participants’ prior experience with activities associated with this pilot project are summarized in Table 1 and Table 2. These data provide some insights into the potential level of expertise of participants prior to beginning the project and which aspects of the project aligned most closely with their prior activities. More than 64% of respondents reported having lots of experience using computers and searching the Internet (Table 1). Volunteers’ experience using social networking sites was more evenly distributed, with 22.6% of the respondents having lots of experience using social networking sites and 26.4% having none. Participants had less experience evaluating, planning or leading, and sharing the results of a conservation project, although many volunteers had some level of experience participating in conservation projects and conducting fieldwork. Most volunteers spent a lot of time searching for information online and a lot of time outdoors in natural settings (Table 2). Although most did not spend a lot of time in political activism, attending community events related to environmental issues, or creating volunteer projects focused on the environment, more than half of respondents did some level of these activities. Furthermore, prior participation in citizen science projects and educating others about environmental issues was extremely diverse, with over 75% of respondents reporting some level of participation associated with these two activities. As such, the volunteers in this project represent an already active group of citizens, potentially through their involvement with the VMN program.
Respondents ranked “interest in the environment” and “interest in protecting a local natural area or resource” as their primary motivators to participate in the project (Table 3). Of the two, interest in the environment was the most frequently reported. Few respondents were primarily motivated by an “interest in citizen science” or “curiosity.” Interest in citizen science was, however, a commonly cited secondary and tertiary motivator. Only five respondents wrote in a motivator not included in the list. However, these responses were recoded into closed-ended answers because of their obvious alignment with the closed-ended response options.
Comparisons of volunteers who dropped out versus persisted. An independent samples t-test analysis was performed on pre-survey data to compare participants who dropped out of the project (N = 18) to those who persisted (N = 34). These analyses yielded no statistically significant differences between motivations for participants who dropped out versus those who persisted. However, two statistically significant differences were found between the prior activities of the two groups. First, volunteers who persisted in the project had a higher frequency of past participation in citizen science projects (M = 3.69, SD = 1.69) prior to starting the project compared with those who dropped out (M = 2.50, SD = 1.43, t(48) = 2.51, p = .015, two-tailed). Volunteers who remained in the project had previously participated in citizen science projects, on average, nearly 2–3 times a month. Those who did not remain in the project, on average, did so less than once a month. Second, participants who remained in the project also had a higher gross income (M = 3.96, SD = 1.53) than those who dropped out (M = 2.86, SD = 1.41, t(40) = 2.27, p = .029, two-tailed). Gross income was measured on a six-point Likert-type scale that ranged from “less than $20,000” to “more than $150,000.” Volunteers who remained in the project had an average reported gross income of $80,000 to $99,999, whereas volunteers who dropped out had an average reported gross income of $50,000 to $79,999.
The drop-out interviews revealed four major themes related to volunteer retention in this project: Time commitment; struggles with the online components; lack of tangible, real-world project impact; and limited social interactions. Self-determination theory (SDT), with its emphasis on intrinsic motivation arising from competence, autonomy, and relatedness, provides a theoretical lens with which to examine these themes (Ryan and Deci 2000a; Ryan and Deci 2000b). Some participants questioned their ability to complete various activities associated with this project (competence), felt somewhat forced into creating a project from scratch or to follow the structure of this pilot project (autonomy), and were unable to find support through social interaction or to identify roles and responsibilities that were salient (relatedness). Direct quotations from participants are included to provide evidence for these key themes, and nuances within each (called sub-themes). Additional excerpts from participant interviews providing further evidence are referenced with each associated theme, or sub-theme, and can be found in Table 4.
|Theme I: Time Commitment|
|1) Struggles with time commitment||“We were a little bit strained for time …”|
|“For my role, I didn’t have time to participate in an actual project, actually go in the field, collect the data, enter the data, be immersed in it …”|
|“It seemed like a really complex process, for me, and more than what I wanted to do and that I have time to do.”|
|“If I was retired and not working, it would rock!”|
|Theme II: Struggles with the online components|
|2) Struggles with Online Training||“ … very long and there were so many questions involved.”|
|3) Excitement with online training||“ …excited because I felt I wanted more scientific rigor in master naturalists, about ecosystems and things.”|
|“The online lectures were really interesting.”|
|4) Struggles with Mental Modeling and Website||“ … the ‘clunkiness’ of some of the software.”|
|“I reached out [to VMN staff] to say I’m stuck … and felt this was way over my head”|
|“For me I think the process needed to be a little simpler … the online site was difficult to get into and navigate …”|
|5) Excited about Mental Modeling||“I really enjoyed it, and felt that as a tool it was very cool, in terms of putting your components in and making your connections … and maybe thinking about scenarios …”|
|“It was intriguing … I can really see that it could be very useful.”|
|“I liked that it was very visual, because it helps you conceptualize all these parameters that you would ordinarily maybe not think of …”|
|6) Uncertain of the Value of Mental Modeling||“The more complicated you got [with the model] … very quickly, the visual end of it, you can’t make any sense out of it.”|
|Theme III: Lack of tangible, real-world projects|
|7) Lack of project relevance||“In order for me to learn best, show me a real example… and walk me through the whole thing.”|
|“I was having trouble figuring out how to make this technology relevant …”|
|8) Uncertain of practical value of the project||“Hearing about some successful projects so that I have a clearer idea of what sorts of things people have done and how successful they have been …”|
|9) Challenges of not having a pre-defined project||“Maybe the project would work best with a person who already had a problem that they wanted to solve.”|
|“I knew I would never see the people again… I knew that this was a project that wasn’t real …”|
|10) Lack of project salience||“[the] average volunteer for nature… they want something that is more tuned in with what they are seeing every day in their area around them in their own area… this is what’s in your backyard, let’s learn more about it.”|
|Theme IV: Lack of social interactions|
|11) Desire for more social interaction||“If we had a buddy to work with, I think that would have been fun.”|
|“I really thought that there would be more people.”|
Time commitment. A lack of time was the most commonly cited barrier during interviews with volunteers who dropped out of this project. None of the participants interviewed completed the suite of in-person meetings, self-guided online trainings, and modeling sessions associated with the project. During volunteer recruitment, it was estimated that these activities would take eight hours to complete. The self-guided online trainings appeared to take some volunteers longer than anticipated. As two volunteers stated, “… feeling like oh my goodness, what have I gotten myself into because I’m not sure I have time for this,” and “it’s geared towards those individuals that have the time to really spend on a citizen science project, that’s a long-term project, versus the weekend scientist.” The interviews revealed that the time commitment was a major barrier for 80% of those who dropped out (Table 4, sub-theme 1).
Struggles with the online components. Some volunteers acknowledged the challenging nature of the self-guided online training sessions (Table 4, sub-theme 2). One volunteer stated, “I had trouble following what [was] being presented.” However, some volunteers seemed up for the challenge and were excited about these sessions with statements like “the classes, the videos, and the tests were great, I thought they were so informative and interesting” (Table 4, sub-theme 3). Despite some positive comments about the online training sessions, these opportunities for learning were not a strong enough motivator to prevent respondents from dropping out of the project.
Eight volunteers mentioned unfamiliarity with the online tools (e.g., Mental Modeling) and technical problems as moderate to high barriers to participation. Statements like “the computer modeling, that sort of got way over my head quickly,” and “with our current group, there are not enough people that have the kind of background, I guess, that understand enough to take advantage of the [modeling] tool” represent clear barriers for some volunteers (Table 4, Sub-theme 4). While some volunteers saw the value in the modeling process, “it made people start thinking outside of the box, what are the impacts to this thing, what are the impacts to that thing, and that was my favorite part,” others did not, “it was a little chaotic, and so it became, like, pretty quickly hard to process, it became a little bit cloudy or muddled” and “it was kind of boring … I think all of us looked at that [modeling] and thought I have other things to do” (Table 4, sub-themes 5 and 6). Ultimately, some volunteers were left with the feeling that they didn’t have the support needed to fully engage with the online resources. Two volunteers summarized by stating “I felt like the way we did it, we were sort of grasping at things” and “us older people, we are good students, we take it all in, but we’re not used to getting involved and starting an online forum, I’ve never done that….”
Lack of tangible, real-world impact. Seven out of the 15 participants interviewed wanted more relevant hands-on work during the project, particularly during the online training and modeling sessions (Table 4, sub-theme 7). Many of these participants felt disconnected from the content, even uncertain of how the tools in this pilot project would help them develop a co-created project of their own. Two volunteers clearly didn’t see the relevance of the tools by saying “… I started spending a great deal of time, and struggling, and stressing, and just couldn’t see where this would help with our work doing citizen science as a chapter …” and “[we needed] a clear idea on where it [the training course] was going on and what it was for.” Respondents also seemed to struggle with the potential roles they could fill in their respective projects. This was a barrier for one volunteer, who said “… I would love to be doing something or contributing, I just haven’t figured out how … make it easy to transition [from training] to [other things] you can sign up for.” Still others desired a better understanding of the practical value of the online tools, as one volunteer stated “… I would like to see a presentation …[showing] some actual projects that were done through that tool and what the outcome was for actual on-the-ground conservation, if anything changed or happened as a result of that work” (Table 4, sub-theme 8).
Not having an issue or citizen science project previously identified was a barrier for five out of the 15 respondents interviewed (Table 4, sub-theme 9). These volunteers struggled with the relevance of developing a project from scratch without having all of the pieces already in place. As one respondent put it, “I think a lot of volunteers are more goal oriented, okay, give them a specific project and say this is your job, and they’ll happily go do it, assuming it’s in an area that interests them, but it’s a bit much to ask most people to invent the wheel.” Another volunteer stated “I’m really into plant ID, and a lot of the work I do is clearing brush from areas … but nothing has occurred to me for research that should be done around that … so I didn’t see any point in spending my time going through a course teaching me how to do research when I don’t have any idea of a project to do.” Rather than theorizing or hypothesizing about how a project might work using the online collaborative resources, some volunteers preferred to skip ahead to taking hands-on practical action that they felt might lead to a meaningful impact (Table 4, sub-theme 10). As two volunteers said, “I spend enough time on the computer as it is, and the thought of doing a full online course, is like, you must be kidding” and “myself, and most of the master naturalists that are in our group, just want to do hands-on kinds of things, I’m not so sure we want to do all that kind of research piece.”
Limited social interactions. The limited social interaction during the training sessions was a disappointment for four respondents (Table 4, sub-theme 11). As one volunteer stated, “when I was taking the training, I really felt like the Lone Ranger.” Building a relationship with others, particularly knowledgeable individuals, was a primary motivator for one of these volunteers to join the project who said, “honestly, one of the big reasons why I got involved in the training in the first place is relationships, relationships are important to me.” While only two respondents mentioned the limited social interaction as a high barrier to continuing in the project, it appeared to be at least a contributing factor leading to dropping out for some volunteers.
This project developed a series of online cyberlearning tools to be used by volunteers in the development, execution, and evaluation of their own co-created citizen science projects. The most commonly reported primary motivations for volunteer participation in this project included helping the environment and protecting a local natural area or resource. These findings reflect similar motivations to participation from other studies (e.g., Bruyere and Rappe 2007; Clary et al.1996; Jordan et al. 2011, Phillips et al. 2006; Ryan et al. 2001; West et al. 2015). However, the dropout rates in this project suggest a mismatch between some participants’ motivations and the project’s design, the motivations of some participants and project managers, and project managers’ assumptions of the use of new technologies and procedures by some participants (Geoghegan et al. 2016).
A comparison of motivations to participate in the project between participants who dropped out and those who persisted did not reveal any statistically significant differences. Those with more prior experience in citizen science projects, however, tended to persist. More experienced participants may have been more strongly motivated to expand their role and go deeper into the project, whereas less experienced participants may have found the intensity of the project undesirable. This is in line with prior research that suggests that participants with higher familiarity in a particular setting tend to be better oriented and can focus more attention on tasks at hand (Moscardo 1999). More prior experience also may suggest greater competence associated with project activities motivating more seasoned volunteers to persist in the project (Ryan and Deci 2000a; Ryan and Deci 2000b). Project managers should thus recognize the potential implications of participants’ prior relevant experience on the learning curve associated with using new technology and the rigors for developing co-created citizen science projects. Additional resources and assistance may be required for volunteers with less prior experience to ensure adequate support for their efforts.
From a technology perspective, previous experience, such as regular use of a computer and searching the Internet for information, may not necessarily transfer to competency with other online applications. Interviews with participants who dropped out of this project commonly reported frustration with the amount of time spent in the online environment, even though online trainings and tools were designed with the intent of supporting collaboration and the development of on-the-ground projects. A lack of time and volunteer fatigue are common barriers reported in online learning (Higgins and Shackleton 2015; Park and Choi 2009; Muilenburg and Berge 2005), and project managers must ensure that new technologies do not alienate citizen scientists more accustomed to traditional approaches (Geoghegan et al. 2016). Online resources that are efficient, in addition to being relevant, user-friendly, and enjoyable, are likely important for retention. Volunteers should be made aware of expected time commitments upfront and be provided with clear instructions and sufficient support to enhance competency should complications arise.
Many volunteers had some experience conducting fieldwork and working on conservation projects, but few had substantial experience planning or leading conservation projects, sharing results, or evaluating project success—elements that were key aspects to developing co-created projects within this effort. Interviews with participants who dropped out of the project revealed that many felt overwhelmed, overworked, or had difficulty finding a comfortable role within the project. This may have been particularly acute for those who had less prior citizen science experience. Future project managers could consider different options for involvement (e.g., data collection only) to suit individuals who may want to make a contribution to the project but have little time available to lead or help develop one (Amichai-Hamburger 2008; Eveleigh et al. 2014; Farquhar and Wing 2008). Such options may enhance participants’ sense of autonomy over their role(s) within the project and ensure that critical competencies are met prior to assuming different roles within the project (Ryan and Deci 2000a; Ryan and Deci 2000b).
The motivations for citizen science participants within the same project can vary dramatically (Clary and Snyder 1999) and may change over time (Geoghegan et al. 2016). Offering varying levels and types of engagement for diverse participants may provide comfortable entry points from which they can begin to try new things, foster new interests, and deepen their engagement (Bonney et al. 2009a; Everett and Geoghegan 2016). A study by Geoghegan et al. (2016) suggests that motivations change over time for some citizen science participants and that these changes can lead to movement from more passive roles to active roles in a project. Project managers should consider the ways in which they move people from collecting data to developing citizen science projects and at each level meet individual needs for competency (e.g., they feel like they can meet the challenge at the next level), autonomy (e.g., they have a choice to advance to the next level), and relatedness (e.g., they feel a sense of belonging at the next level) (Ryan and Deci 2000a; Ryan and Deci 2000b).
Research from the broader informal learning field, in particular environmental education and interpretation, suggests that project-based, experiential work that is relevant to participants can lead to high levels of engagement and generally more positive outcomes (Ardoin et al. 2013; Athman and Monroe 2002; Ballantyne and Packer 2009; Beck and Cable 2002; Brochu and Merriman 2002; Ham 1992; Heimlich and Norland 1994; Stern and Powell 2013; Stern et al. 2014). Research in the field of citizen science reveals much the same (Garibay 2004; Garibay 2009; Rotman et al. 2012), and co-created citizen projects commonly arise in response to a relevant environmental or health crisis at the community level (Bonney et al. 2009a). Working on a pressing issue may have increased the salience of the online resources and motivated some participants to persevere through challenges encountered during their use.
Experiential learning and social interactions form the basis for the development of a community of practice, in which social interactions can increase motivations for involvement as people begin to identify themselves as members of a community and ultimately become highly skilled (Lave and Wenger 1991). In the development of such communities, novice participants move from peripheral participation in the learning community to more central participation as they become more active and engaged within that community. In this view, learning is strongly enhanced by social interaction and collaboration, potentially leading to a stronger sense of relatedness (Ryan and Deci 2000a; Ryan and Deci 2000b). As individuals become more central, the need to invest greater commitments of time, effort, and responsibility may be required to develop a strong identity as a member of the community (Wenger 1998).
It was our hope that participants would use the resources and tools in this pilot project to build upon their previous knowledge and through these collaborative efforts to move into more central roles developing a co-created citizen science project of their own. Two main barriers inhibited this development for some participants. First, the online environment proved a less natural setting for meaningful social interaction for some participants. Second, some volunteers struggled to see the value of the online resources if they didn’t have an immediate, personally relevant, tangible project already in mind to which to apply the key concepts and tools. This lack of relevance may have prohibited some volunteers from seeing the benefits of moving deeper into this community such that they dropped out of the project.
Based on these findings, we restructured our project to minimize these potential barriers to participation and to increase the relevance of the project to participants (see Gray et al. 2017; Newman et al. 2017). It is important for project managers to consider how a project impacts the participants themselves and to make adjustments to its procedural and communication approaches to sustain participation and increase satisfaction (Geoghegan et al. 2016; West and Pateman 2016). First, we eliminated the online training sessions, which were both time consuming and, while enjoyed by some participants, clearly were not meaningful or relevant to others. Many volunteers reported finding the information they needed from each other, subject-matter experts, site managers, and local and online resources instead of through our training videos. As such, participants were able to increase their autonomy and have some control over identifying the most relevant competencies they needed to increase so they could develop and execute their co-created projects. Second, we allowed volunteers to use alternative ways to engage with the online cyberlearning tools, thereby enhancing their sense of autonomy over their experience. For example, some participants developed their individual mental models on paper prior to re-engaging with the software at a later time during collaborative group modeling sessions. Some studies suggest that breaking down more complex tasks into smaller building blocks (Eveleigh et al., 2014; Rotman et al. 2012) may reduce anxiety by providing a scaffolding approach (Nov et al. 2011; Derrick 2003) to build up to more intricate tasks. We also provided additional assistance through the development of how-to-guides and training sessions to ensure volunteers had the necessary information to build their competencies regarding the use of online tools. We received notifications from volunteers who had issues accessing or using the project website; we worked on troubleshooting those issues and on promoting a sense of relatedness and community between project researchers and the volunteers working within it. We are currently in the process of making final revisions to the website and resources for future widespread use.
Web-based cyberlearning tools are innovative and exciting resources that have the potential to support volunteers’ development and execution of co-created citizen science projects. This study has revealed some insights into the motivations for initial participation and potential reasons for participant dropout. Volunteers were primarily motivated to participate because of an interest in the environment and an interest in protection of an area they care about. Co-created projects that are salient with individuals appear to be critically important in motivating participation. Dropouts cited the high time commitment, struggles using the online trainings and collaborative tools, and an inability to find a role that fit their prior knowledge and interests within the project. The findings suggest the importance of streamlining online trainings and collaborative resources as much as possible, ensuring that they are easy and fun to use, and linking them directly to a tangible project of interest to participants. Volunteers should have access to, and an understanding of, the diverse roles that are available within a co-created citizen science project. Project managers should provide a clear pathway for further advancement within a project and be adaptable with the implementation of a suite of tools and resources intended to support individuals’ needs for competence, autonomy, and relatedness to motivate sustained and increased participation. Many of the volunteers who started this project persisted through the online trainings and used the online tools (e.g., Mental Modeling) to develop and implement co-created citizen science projects. As such, we believe that using online and on-the-ground citizen science approaches have promise for transitioning volunteers from the role of data collectors toward co-creators of meaningful citizen science projects.
All participants in this project gave their signed consent prior to participation and their confidentiality was maintained throughout data collection. This study was approved by the Virginia Tech Institution Review Board (#13-560).
The authors thank the volunteers with the Virginia Master Naturalists who participated in this project. This manuscript greatly benefitted from the thoughtful comments from reviewers. Funding for the study was provided by the National Science Foundation Award 1227550. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
The authors have no competing interests to declare.
Amichai-Hamburger, Y. (2008). Potential and promise of online volunteering. Computers in Human Behavior 24: 544–562, DOI: https://doi.org/10.1016/j.chb.2007.02.004
Ardoin, N.M., Clark, C. and Kelsey, E. (2013). An exploration of future trends in environmental education research. Environmental Education Research 19(4): 499–520, DOI: https://doi.org/10.1080/13504622.2012.709823
Athman, J. and Monroe, M.C. (2002). Enhancing natural resource programs with field trips. University of Florida: IFAS Extension. Available at: http://edis.ifas.ufl.edu.
Ballantyne, R. and Packer, J. (2009). Introducing a fifth pedagogy: experience based strategies for facilitating learning in natural environments. Environmental Education Research 15(2): 243–262, DOI: https://doi.org/10.1080/13504620802711282
Bell, S., Marzano, M., Cent, J., Kobierska, H., Podjed, D., Vandzinskaite, D., Reinert, H., Armaitience, A., Grodzińska-Jurczak, M. and Muršič, R. (2008). What counts? Volunteers and their organisations in the recording and monitoring of biodiversity. Biodiversity and Conservation 17(14): 3443–3454, DOI: https://doi.org/10.1007/s10531-008-9357-9
Bonney, R., Ballard, H., Jordan, R., McCallie, E., Phillips, T., Shirk, J. and Wilderman, C.C. (2009a). Public participation in scientific research: defining the field and assessing its potential for informal science education. A CAISE Inquiry Group Report. Washington, D.C.: Center for Advancement of Informal Education (CAISE).
Bonney, R., Phillips, T.B., Ballard, H.L. and Enck, J.W. (2016). Can citizen science enhance public understanding of science?. Public Understanding of Science 25(1): 2–16, DOI: https://doi.org/10.1177/0963662515607406
Bradford, B.M. and Israel, G.D. (2004). Evaluating volunteer motivation for sea turtle conservation in Florida. Gainesville, FL: University of Florida, Agriculture Education and Communication Department, Institute of Agriculture and Food Sciences, AEC. 372.
Bruyere, B. and Rappe, S. (2007). Identifying the motivations of environmental volunteers. Journal of Environmental Planning and Management 50(4): 503–516, DOI: https://doi.org/10.1080/09640560701402034
Butler, B. (2001). Membership size, communication activity, and sustainability: A resource-based model of online social structures. Information Systems Research 12: 346–362, DOI: https://doi.org/10.1287/isre.12.4.346.9703
Chiu, C.M., Hsu, M.H. and Wang, E.T.G. (2006). Understanding knowledge sharing in virtual communities: An integration of social capital and social cognitive theories. Decision Support Systems 42(3): 1872–1888, DOI: https://doi.org/10.1016/j.dss.2006.04.001
Chu, M., Leonard, P. and Stevenson, F. (2012). Growing the base for citizen science: Recruiting and engaging people In: Dickinson, J. and Bonney, R. eds. Citizen science: Public participation in environmental research. Ithaca and London: Cornell University Press, DOI: https://doi.org/10.7591/cornell/9780801449116.003.0005pp. 69–81.
Clary, E.G. and Snyder, M. (1999). The motivations to volunteer: Theoretical and practical considerations. Current Directions in Psychological Science 8(5): 156–159, DOI: https://doi.org/10.1111/1467-8721.00037
Clary, E.G., Snyder, M. and Stukas, A.A. (1996). Volunteers’ motivations: findings from a national survey. Nonprofit and Voluntary Sector Quarterly 25(4): 485–505, DOI: https://doi.org/10.1177/0899764096254006
Cox, J., Young Oh, E., Simmons, B., Graham, G., Greenhill, A., Lintott, C., Masters, K. and Woodcock, J. (2015). Doing good online: An investigation into the characteristics and motivations of digital volunteers. SSRN. Available at: http://ssrn.com/abstract=2717402.
Crall, A.W., Jordan, R., Holfelder, K.A., Newman, G., Graham, J. and Waller, D.M. (2013). The impacts of an invasive species citizen science training program on participant attitudes, behavior, and science literacy. Public Understanding of Science 22: 745–776, DOI: https://doi.org/10.1177/0963662511434894
Crimmins, T.M., Weltzin, J.F., Rosemartin, A.H., Surina, E.M., Marsh, L. and Denny, E.G. (2014). Focused campaign increases activity among participants in nature’s notebook, a citizen science project. Natural Science Education 43: 64–72, DOI: https://doi.org/10.4195/nse2013.06.0019
Curtis, V. (2015). Motivation to Participate in an Online Citizen Science Game: A Study of Foldit. Science Communication 37(6): 723–746, DOI: https://doi.org/10.1177/1075547015609322
Derrick, M.G. (2003). Creating environments conducive for lifelong learning. New Directions for Adult and Continuing Education 100: 5–18, DOI: https://doi.org/10.1002/ace.115
Ellul, C., Francis, L. and Haklay, M. (2001). A flexible database-centric platform for citizen science data capture. 2011 IEEE Seventh International Conference on e-Science Workshops (eScienceW). : 39–44.
Everett, G. and Geoghegan, H. (2016). Initiating and continuing participation in citizen science for natural history. BMC Ecology 16(Suppl 1): 15–22, DOI: https://doi.org/10.1186/s12898-016-0062-3 S13.
Farquhar, S.A. and Wing, S. (2008). Methodological and ethical considerations in community-driven environmental justice research: two case studies from rural North Carolina In: Minkler, M. and Wallerstein, N. eds. Community-based participatory research for health. San Francisco, CA: Jossey-Bass Publishers, pp. 221–241.
Foundations of Success (2009). Conceptualizing and planning conservation projects and programs: A training manual. USA: Bethesda. Foundations of Success. Available at: http://cmp-openstandards.org/guidance/conceptualizing-and-planning-conservation-projects-and-programs/.
Geoghegan, H., Dyke, A., Pateman, R., West, S. and Everett, G. (2016). Understanding motivations for citizen science In: Final report on behalf of UKEOF. University of Reading, Stockholm Environment Institute (University of York) and University of the West of England.
Gray, S., Gray, S., Cox, L. and Henly-Shepard, S. (2013). Mental modeler: A fuzzy-logic cognitive mapping modeling tool for adaptive environmental management. Proceedings of the 46th International Conference on Complex Systems. : 963–973, DOI: https://doi.org/10.1109/HICSS.2013.399
Gray, S.A., Gray, S., De Kok, J.L., Helfgott, A.E.R., O’Dwyer, B., Jordan, R. and Nyaki, A. (2015). Using fuzzy cognitive mapping as a participatory approach to analyze change, preferred states, and perceived resilience of social-ecological systems. Ecology and Society 20(2): 11.DOI: https://doi.org/10.5751/ES-07396-200211
Gray, S., Jordan, R.C., Crall, A., Newman, G., Hmelo-Silver, C., Huang, J., Novak, W., Mellor, D., Frensley, T., Prysby, M. and Singer, A. (2017). Combining participatory modelling and citizen science to support volunteer conservation action. Biological Conservation 208: 76–86, DOI: https://doi.org/10.1016/j.biocon.2016.07.037
Henly-Shepard, S., Gray, S. and Cox, L. (2015). Facilitating community adaptation through partcipatory modeling and social learning. Environmental Science and Policy 45: 109–122, DOI: https://doi.org/10.1016/j.envsci.2014.10.004
Higgins, O. and Shackleton, C.M. (2015). The benefits from and barriers to participation in civic environmental organisations in South Africa. Biodiversity Conservation 24: 2031–2046, DOI: https://doi.org/10.1007/s10531-015-0924-6
Jordan, R.C., Gray, S.A., Howe, D.V., Brooks, W.R. and Ehrenfeld, J.G. (2011). Knowledge gain and behavior change in citizen-science programs. Conservation Biology 25: 1148–1154, DOI: https://doi.org/10.1111/j.1523-1739.2011.01745.x
Kosko, B. (1986). Fuzzy knowledge combination. International Journal of Intelligent Systems 1(4): 293–320, DOI: https://doi.org/10.1002/int.4550010405
Land-Zandstra, A.M., Devilee, J.L.A., Snik, F., Buurmeijer, F. and van den Broek, J.M. (2016). Citizen science on a smartphone: Participants’ motivations and learning. Public Understanding of Science 25(1): 45–60, DOI: https://doi.org/10.1177/0963662515602406
Lave, J. and Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press, DOI: https://doi.org/10.1017/CBO9780511815355
Lye, G.C., Osborne, J.L., Park, K.J. and Goulson, D. (2012). Using citizen science to monitor Bombus populations in the UK: Nesting ecology and relative abundance in the urban environment. Journal of Insect Conservation 16: 697–707, DOI: https://doi.org/10.1007/s10841-011-9450-3
Mankowski, T.S., Slater, S.J. and Slater, T.F. (2011). An Interpretive Study of Meanings Citizen Scientists Make When Participating in Galaxy Zoo In: Jensen, J.B., Manning, J.G. and Gibbs, M.G. eds. Earth and Space Science: Making Connections in Education and Public Outreach. Astronomical Society of the Pacific, 443pp. 150–155, DOI: https://doi.org/10.19030/cier.v4i4.4165
Merenlender, A., Crall, A.W., Drill, S., Prysby, M. and Ballard, H.L. (2016). Evaluating environmental education, citizen science, and stewardship through naturalist programs. Conservation Biology 30(6): 1255–165, DOI: https://doi.org/10.1111/cobi.12737
Muilenburg, L.Y. and Berge, Z.L. (2005). Student barriers to online learning: A factor analytic study. Distance Education 26(1): 29–48, DOI: https://doi.org/10.1080/01587910500081269
Newman, G., McGracy, B., Clyde, M., Chandler, M., Haklay, M., Ballard, H., Gray, S., Mellor, D. and Gallo, J. (2017). Leveraging the power of place in citizen science for effective conservation decision making. Biological Conservation 208: 55–64, DOI: https://doi.org/10.1016/j.biocon.2016.07.019
Nov, O., Arazy, O. and Anderson, D. (2011). Technology-mediated citizen science participation: A motivational model, Paper presented at the AAAI International Conference on Weblogs and Social Media. Barcelona, Spain
Nov, O., Arazy, O. and Anderson, D. (2014). Scientists@Home: What drives the quantity and quality of online citizen science participation?. PloS ONE 9(4)DOI: https://doi.org/10.1371/journal.pone.0090375
Phillips, S.J., Anderson, R.P. and Schapire, R.E. (2006). Maximun entropy modeling of species geographic distributions. Ecological Modelling 190: 231–259, DOI: https://doi.org/10.1016/j.ecolmodel.2005.03.026
Prysby, M. (2015). Program reports, Available at: http://www.virginiamasternaturalist.org/program-reports.html.
Raddick, M.J., Bracey, G., Gay, P.L., Lintott, C.J., Cardamone, C., Murray, P., Schawinski, K. and Vandenberg, J. (2014). Galaxy zoo: Motivations of citizen scientists. Astronomy Education Review 12(1)DOI: https://doi.org/10.3847/AER2011021
Raddick, M.J., Bracey, G., Gay, P.L., Lintott, C.J., Murray, P., Schawinski, K., Szalay, A.S. and Vandenburg, J. (2010). Galaxy zoo: Exploring the motivations of citizen science volunteers. Astronomy Education Review 9(1)DOI: https://doi.org/10.3847/AER2009036
Rotman, D., Preece, J., Hammock, J., Procita, K., Handsen, D., Parr, C., Lewis, D. and Jacobs, D. (2012). Dynamic Changes in Motivation in Collaborative Citizen-Science Projects. Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work (CSCW ‘12). New YorkACM: 217–226, DOI: https://doi.org/10.1145/2145204.2145238
Ryan, R.L., Kaplan, R. and Grese, R.E. (2001). Predicting volunteer commitment in environmental stewardship programmes. Journal of Environmental Planning and Management 44(5): 629–648, DOI: https://doi.org/10.1080/09640560120079948
Ryan, R.M. and Deci, E.L. (2000a). Self-determination theory and the facilitation of intrinsic motivation, social development and well-being. American Psychologist 55(1): 68–78, DOI: https://doi.org/10.1037/0003-066X.55.1.68
Ryan, R.M. and Deci, E.L. (2000b). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology 25: 54–67, DOI: https://doi.org/10.1006/ceps.1999.1020
Shirk, J.L., Ballard, H.L., Wilderman, C.C., Phillips, T., Wiggins, A., Jordan, R., McCallie, E., Minarchek, M., Lewenstein, B.V., Krasny, M. and Bonney, R. (2012). Public participation in scientific research: A framework for deliberate design. Ecology & Society 17(2): 29.DOI: https://doi.org/10.5751/ES-04705-170229
Silvertown, J., Buesching, C.D., Jacobson, S.K. and Rebelo, T. (2013). Citizen science and nature conservation. Key Topics in Conservation Biology 2: 127–142, DOI: https://doi.org/10.1002/9781118520178.ch8
Stern, M.J., Powell, R.B. and Hill, D. (2014). Environmental education program evaluation in the new millennium: What do we measure and what have we learned?. Environmental Education Research 20(5): 581–611, DOI: https://doi.org/10.1080/13504622.2013.838749
Stukas, A., Worth, K., Clary, E. and Snyder, M. (2009). The matching of motivations to affordance in the volunteer environment. Nonprofit and Voluntary Sector Quarterly 38: 5–28, DOI: https://doi.org/10.1177/0899764008314810
Sunde, P. and Jessen, L. (2013). It counts who counts: An experimental evaluation of the importance of observer effects on spotlight count estimate. European Journal of Wildlife Research 59: 645–653, DOI: https://doi.org/10.1007/s10344-013-0717-8
Tulloch, A.I.T., Possingham, H.P., Joseph, L.N., Szabo, J. and Martin, T.G. (2013). Realising the full potential of citizen science monitoring programs. Biological Conservation 165: 128–138, DOI: https://doi.org/10.1016/j.biocon.2013.05.025
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge, MA: Cambridge University Press, DOI: https://doi.org/10.1017/CBO9780511803932
West, S. and Pateman, R. (2016). Recruiting and retaining participants in citizen science: What can be learned from the volunteering literature?. Citizen Science: Theory and Practice 1(2): 1–10, DOI: https://doi.org/10.5334/cstp.8
Wilderman, C.C., Barron, A. and Imgrund, L. (2004). Top down or bottom up? ALLARM’s experience with two operational models for community science. Article 235 in Proceedings of the 4th National Water Quality Monitoring Council Conference. 17–20 May 2004, Chattanooga, Tennessee, USA Available at: http://acwi.gov/monitoring/conference/2004/poster_presentations.html.
Wright, D.R., Underhill, L.G., Keene, M. and Knight, A.T. (2015). Understanding the motivations and satisfactions of volunteers to improve the effectiveness of citizen science programs. Society of Natural Resources: An International Journal, : 1–17, DOI: https://doi.org/10.1080/08941920.2015.1054976