Citizen Science and Informal Science Learning

The educational underpinnings of citizen science—particularly when involving adults—draw heavily from Informal Science Education (ISE), what Falk and Dierking (2003) refer to as “free-choice learning”—lifelong, self-directed learning that occurs outside K-16 classrooms. Two influential documents from the ISE field provided a starting point for our study. The Framework for Evaluating Impacts of Informal Science Education Projects (Friedman et al. 2008), supported by the National Science Foundation (NSF), was the first publication produced by the ISE field that described a “standard” set of learning outcomes (referred to as impact categories) that could be used to systematically measure project-level outcomes. (We will refer to this publication as the “ISE Framework” for the remainder of this paper.) A major goal of the framework was to facilitate cross-project and cross-technique comparisons of the impacts of ISE projects on public audiences. The five impact categories are:

• Knowledge, awareness, or understanding of Science, Technology, Engineering, and Math (STEM) concepts, processes, or careers
• Engagement or interest in STEM concepts or careers
• Attitude toward STEM concepts, processes, or careers
• Skills based on STEM concepts or processes
• Behavior related to STEM concepts, processes, or careers

A second document, Learning Science in Informal Environments: People, Places, and Pursuits (National Research Council 2009), focuses on characterizing the cognitive, affective, social, and developmental aspects of science learners. Termed the “LSIE strands,” these aspects of science participation include:

• Interest and motivation to learn about the natural world
• Application and understanding of science concepts
• Acquisition of skills related to the practice of science
• Reflecting on science as a way of knowing, participating in, and communicating about science
• Identifying oneself as someone capable of knowing, using, and contributing to science

The authors of the LSIE strands noted that while the concepts originated in research, at the time of writing they had not yet been applied or analyzed in any systematic venue. The significant overlap between the LSIE strands and the ISE Framework’s impact categories is shown in Table 1.

A third ISE document also contributed to framing this study. In 2009, an inquiry group sponsored by the Center for Advancement of Informal Science Education (CAISE) produced “Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education” (Bonney et al. 2009a), which was created as a “first step toward developing an organized methodology for comparing outcomes across a variety of Public Participation in Scientific Research (PPSR) projects” (p.20). This paper included a rubric of potential citizen science learning outcomes, based on the ISE Framework, and examined ten NSF-funded citizen science projects to assess whether they reported outcomes similar to those described in the ISE Framework.

One result of the CAISE report was a realization that citizen science practitioners were measuring project outcomes in varied ways, making it difficult for cross-programmatic research to study the collective impact of the field. Another result was the development of a project typology based on the level of participant involvement in the scientific process (Bonney et al. 2009a). This typology described “Contributory” projects that are researcher-driven, where participants primarily focus on data collection; “Collaborative” projects that are typically led by researchers, but may include input from participants in phases of the scientific process such as designing methods, analyzing data, and disseminating results; and “Co-created” projects that involve participants in all aspects of the scientific process, from defining a question to interpreting data to disseminating results (Figure 1). This typology allowed projects designed for different reasons and in different ways to be grouped to help researchers understand common outcomes.

The three documents described above served as foundations for articulating learning outcomes from citizen science participation, however, they lacked systematic empirical support. This study provides such support by ground truthing and applying the concepts within the ISE Framework, the LSIE strands, and the Bonney et al. (2009a) rubric to the field of citizen science.

Intended Learning Outcomes

To describe and understand the learning outcomes that are intended or desired by citizen science practitioners as they develop projects, we first identified individual projects by conducting a semi-structured search of the following citizen science portals: Citizen Science Central (citizenscience.org); InformalScience (informalscience.org); SciStarter (scistarter.com); Citizen Science Alliance (citizensciencealliance.org); and National Directory of Volunteer Monitoring Programs (yosemite.epa.gov/water/volmon.nsf/). The last portal included 800 projects, from which we sampled every fifth one. If a project was listed on multiple portals, we included it only once. In total, 327 citizen science projects met our criteria for study inclusion: Being open to participation in the U.S. or Canada, having an online presence, and being operational at the time of the search (2011). The complete list of databases and search terms used to locate citizen science projects is available in the supplemental material for this paper (Appendix A).

From each of the 327 project websites, we gathered the following information: Project name, URL, contact information, general goal statements, learning objectives or desired outcomes (if any), and potential indicators of learning (if any). Nine percent of project websites did not describe intended learning outcomes (e.g., some projects stated their goals to be purely scientific in nature), but the remaining 92% of projects described at least one. We coded each goal statement and learning objective into one of the major categories outlined in the ISE Framework (knowledge, engagement, skills, attitude, behavior, other) and into sub-codes outlined in the assessment rubric by Bonney et al. (2009a).

Several projects described multiple learning outcomes. In these cases, each distinct outcome was coded separately. For example, the Great Lakes Worm Watch states that its goal is “increasing scientific literacy and public understanding of the role of exotic species in ecosystems change.” Objectives are to “provide the tools and resources for citizens to actively contribute to the development of a database documenting the distributions of exotic earthworms and their impacts across the region as well as training and resources for educators to help build understanding of the methods and results of scientific research about exotic earthworms and forest ecosystems ecology.” The text from the goal statement and learning objectives (left) were coded into the outcomes categories on the right:

Results from our coding of project goals and objectives are presented in Table 2. They reveal that the number of aspirational learning outcomes for projects ranged from zero to as many as seven, with about 40% of projects including at least two. The majority of projects (59%) focus on influencing skills related to data collection and monitoring. Intended outcomes for these projects are often stated as “Volunteers gain data collection and reporting skills.” The second most frequently stated intended learning outcome (28% of projects) was understanding of content knowledge (e.g., “volunteers learn about macroinvertebrates and stream health”). The third most-common intended outcome, increased environmental stewardship—which typically includes some type of behavior change (e.g., “engage watershed residents in protecting water quality”)—was specified by about 26% of projects. Other intended learning outcomes were mentioned much less frequently, including increases in knowledge of the nature of science, data analysis skills, interest in the environment, civic action, data submission, communication skills, use of technology, science careers, study design, and also shifts in attitude/awareness. Considering all projects for which intended learning outcomes were stated, each of the ISE Framework impact categories was represented, suggesting a strong alignment between learning outcomes desired for citizen science participants and those for participants in the informal science learning community more generally.

Status of Citizen Science Project Evaluation

To uncover the status of evaluation of citizen science learning outcomes across the field, we conducted an online survey of citizen science practitioners in March 2011. Delivered via Survey Monkey™, the survey contained 25 questions, including 20 closed-ended questions with predetermined options including “other.” The remaining five questions were open-ended, providing text boxes for answers. Only one question, which asked respondents to classify their project according to the three-model typology of citizen science developed by Bonney et al. (2009a), required a response. Additional questions focused on the duration of the project, the approximate number of participants, and the type of training that participants received. Respondents also were asked if any type of evaluation had ever been conducted for their project; details about evaluations that were conducted; what learning outcomes described in the ISE Framework had been measured; and what other types of outcomes had been measured. The complete set of survey questions is available in the supplemental material (Appendix B).

Following approval by the Cornell University Institutional Review Board (#1102002014), we sent an email invitation to potential respondents describing the goal of the survey and explaining that participation was voluntary and confidential. Two reminder emails were sent approximately two and four weeks following the initial invitation. An informed consent statement was included at the start of the survey. Potential respondents were recruited via the citizenscience.org listserv (citsci-discussion-l), which anyone could join, and which at the time of recruitment had approximately 1,100 members. Not all members of the listserv were project leaders, and multiple list members likely represented a single project, making it difficult to know the actual number of projects represented by listserv members. After the survey was closed, we made sure that all responding projects were included in the previously described website review, to obtain as much overlap between the two datasets as possible.

The survey was completed by 199 respondents representing 157 unique projects (some projects had multiple entries, in which case only the first entry was included; other respondents failed to include information about their project name, which was optional). All but ten of the 157 unique projects also were represented in the project website data. The remaining ten projects that responded to the online survey but were not in the website review were either no longer operational, not in the US or Canada, or did not have a web presence. The majority of projects (72 or 37%) had been operating from 1–5 years, and nearly half (49%) had fewer than 100 participants. Because most questions were optional, response rates varied for different survey items.

Results revealed that of the 199 respondents, 114 or 57% had undertaken some type of project evaluation. More than half of the evaluations were administered by internal project staff to measure project outcomes or impacts, mostly using data collected through surveys. About one third of respondents reported conducting post-only or pre-and posttest evaluation designs. Reasons for conducting project evaluations included: Gauging participant learning; identifying project strengths and weaknesses; obtaining additional funding or support; promoting a project more broadly; and providing recommendations for project improvement. In addition to asking about project learning outcomes (described in the next section), the survey also asked what other aspects of the project had been evaluated. Two thirds of participants reported measuring satisfaction or enjoyment with the project, followed by motivation to participate (53%) and evaluation of project outputs such as numbers of participants, web hits, journal articles, and amount of data collected (44%). Other measured outcomes included scientific/conservation (39%); effectiveness of workshops and trainings (38%); data quality (37%); community capacity building (23%); and social policy change (3%).

Another open-ended question asked respondents “Please do your best to provide the name or description of any instrument (e.g., Views on Science and Technology Survey) used to collect evaluation data, even if you developed the instrument.” Of the 72 respondents to this question, only three had used a pre-existing, validated instrument. The majority of respondents had developed their own instruments in-house or had an external evaluator develop original instruments. A handful of respondents replied with “Survey Monkey” or some other data collection platform as opposed to describing an evaluation instrument. Some mentioned tools such as GPS units or calipers as instruments used by the project, while others stated that they did not understand the question. When asked about their overall satisfaction with their evaluations, more than half of respondents expressed agreement or strong agreement that evaluations were of high quality, that evaluation findings were informative to the project developers, that recommendations from the evaluation were implemented, that the project had improved as a result of evaluation, that they learned a lot about evaluation, and that they felt confident they could personally conduct an evaluation in the future.

Survey respondents also were asked about aspects of the evaluation process for which they would like assistance. The highest priority was help with developing goals, objectives, and indicators, followed by creating or finding appropriate survey instruments, help with analyzing or interpreting data, and help with data collection. Participants also were asked what specific resources would be most helpful for conducting evaluations. The most common replies were a database of potential surveys and data collection instruments; sample evaluation reports from citizen science; examples of evaluation designs; and an entry-level guide for conducting evaluations.

Finally, respondents were presented with a list of eight different online organizations that support or provide resources for evaluation and were asked how often they access them. Surprisingly, the majority of respondents had never heard of any of the resources or organizations. The only exception was citizenscience.org, which was used by 46% of respondents, but rarely (as opposed to frequently, sometimes, or never). These results show a range of evaluation efforts and a positive attitude toward evaluation and findings among citizen science practitioners, but also a need for more knowledge of and accessibility to evaluation tools and resources.

Measurement of Learning Outcomes

Respondents who reported having conducted evaluations (114 or 57%) were asked “For the most recent evaluation of your project, which broad categories of learning outcomes, if any, were evaluated?” Responses to this question were based on the ISE Framework broad impact categories. Aggregated results across all projects revealed that interest or engagement in science was the most commonly measured outcome (46%), followed by knowledge of science content (43%). Behavior change resulting from participation and attitudes toward science process, content, careers, and the environment accounted for 36% and 33%, respectively, of measured learning outcomes. Science inquiry skills (e.g., asking questions, designing studies, data collection, data analysis, and using technology) were the least commonly measured outcomes across all projects (28%). In an open-ended question about other types of learning outcomes, about 10% of respondents also described measuring motivation and self-efficacy or confidence to participate in science and environmental activities.

Considering differences in categories of learning outcomes measured within project types, contributory projects (for which there were 69 respondents that had conducted evaluations) reported measuring interest in science most frequently (43%) and skills of science inquiry least frequently (18%). Two-thirds of all collaborative projects (N = 21) measured content knowledge, followed by interest (57%), behavior change (52%), and attitudes and skills (both 43%). Only nine survey respondents represented co-created projects that had conducted evaluations, and of these, skills of science inquiry were measured most often. Responses combined across projects and separated among project types are summarized in Figure 2.

Earlier in this paper we showed that a majority of citizen science project websites described intended learning outcomes very similar to those in the ISE framework, although not always using the same language. Results from the online practitioner survey added to our “ground truthing” of the ISE Framework, as respondents described attempts to measure these same outcomes, albeit to varying degrees. Open-ended responses highlighted the need to emphasize efficacy as an important learning outcome in citizen science. Survey respondents also made it clear that additional resources were needed to help formulate and measure learning outcomes.

Interest in Science and the Environment

We define interest as the degree to which an individual assigns personal relevance to a science or environmental topic or endeavor. Within ISE, Hidi and Renninger (2006) treat interest as a multi-faceted construct encompassing cognitive (thinking), affective (feeling), and behavioral (doing) domains across four phases of adoption: triggered situational interest typically stimulated by a particular event and requiring support by others; maintained situational interest, which is sustained through personally meaningful activities and experiences; emerging individual interest characterized by positive feelings and self-directed pursuit of re-engaging with certain activities; and well-developed individual interest leading to enduring participation and application of knowledge. Our definition of interest is compatible with Hidi and Renninger’s (2006) later phases of interest development, which are characterized by positive feelings and an increasing investment in learning more about a particular topic. Interest in science is considered a key driver to pursuing science careers in youth (Maltese and Tai 2010; Tai et al. 2006) and sustained lifelong learning and engagement in adults (Falk et al. 2007; Hidi and Renninger 2006). Over time, this type of interest can lead to sustained engagement and motivation and can support identity development as a science learner (Fenichel and Schweingruber 2010; National Research Council 2009). Further, interest is noted as an important precursor to deeper engagement in democratic decision-making processes regarding science and technology (Mejlgaard and Stares 2010).

Although interest is considered to be an attitudinal structure (see Bauer et al. 2000; Fenichel and Schweingruber 2010; Sturgis and Allum 2004), equating interest with attitudes should be avoided because attitude is a very broad construct, encompassing related but distinct sub-constructs such as efficacy, interest, curiosity, appreciation, enjoyment, beliefs, values, perseverance, motivation, engagement, and identity (Osborne et al. 2003). Interest also has been used synonymously with engagement (Friedman et al. 2008), but as McCallie et al. (2009) point out, engagement has yet to be well defined and has multiple meanings within the literature, particularly in ISE.

Citizen science projects, especially those for which repeated visits or experiences are the norm, can lend themselves to deeper and sustained interest in science and the environment, yet few studies have looked at interest as an outcome, and those that have find mixed results. Price and Lee (2013) reported increased interest in science among Citizen Sky observers, especially among participants who engaged in online social activities. Crall et al. (2012) examined general interest in science as a reason for participation in citizen science and suggested that interest was not a driving force for joining a project. Interest in specific nature-based topics, i.e., butterflies, was seen as a driver for engagement and also as a motivator for adding increasingly more complex data protocols to the French Garden Butterflies Watch project (Cosquer et al. 2012). Other research has shown that interest in use of natural resources can be a very strong determinant for future and sustained involvement in the decision-making process about management of natural resources (Danielsen et al. 2009). From these studies, it appears that examining interest in science more broadly may be less effective than measuring specific science topics. However, an audience’s pre-existing interests in specific topics may not change significantly through participation.

Self-efficacy for Science and the Environment

Another important outcome for studying learning is self-efficacy, i.e., a person’s beliefs about his/her capabilities to learn specific content and to perform particular behaviors (Bandura 1997). Research has found that self-efficacy affects an individual’s choice, effort, and persistence in activities (Bandura 1982, 2000; Schunk 1991). Individuals who feel efficacious put more effort into their activities and persist at them longer than those who doubt their abilities. Self-efficacy is sometimes referred to as “perceived competence” (in Self Determination Theory) and “perceived behavioral control” (in Ajzen’s Theory of Planned Behavior, Ajzen 1991). Berkowitz et al. (2005) treat self-efficacy as an essential component in environmental citizenship (along with motivation and awareness), which is dependent on an individual’s belief that they have sufficient skills, knowledge, and opportunity to bring about positive change in their personal lives or community.

In the context of citizen science, self-efficacy is the extent to which a learner has confidence in his or her ability to participate in a science or environmental activity. In a study involving classrooms, middle school students participating in a horseshoe crab citizen science project showed greater gains in self-efficacy than a control group (Hiller 2012). In an online astronomy project, however, researchers found a significant decrease in efficacy toward science, possibly owing to a heightened awareness of how much participants did not know about the topic (Price and Lee 2013). Crall et al. (2011) determined that self-efficacy is not only important in carrying out the principal activities of a project but also in the potential for individuals to carry out future activities related to environmental stewardship. Working in a participatory action project with Salal harvesters, Ballard and Belsky (2010) found that the process of co-developing and implementing different experiments increased workers’ self efficacy regarding their skills in scientific research. Although efficacy was not called out directly in the ISE Framework, it can be considered part of the LSIE Strand 6, “identity as a learner” (National Research Council 2009). Self-efficacy also was mentioned by project leaders in our online survey and thus appears to be an important potential outcome from citizen science participation.

Motivation for Science and the Environment

Motivation is a multi-faceted and complex attitudinal construct that describes some form of goal setting to achieve a behavior or end result. The LSIE strands (National Research Council 2009) include motivation to sustain science learning over an individual’s lifetime as an important aspect of learning in informal environments. The literature on volunteerism frames motivation as an important factor in effective recruitment, accurate placement, and volunteer satisfaction and retention (Clary and Snyder 1999, Esmond et al. 2004). Of the dozens of theories on motivation, two perspectives seem especially relevant to volunteerism and citizen science. First, the Volunteer Functions Inventory (VFI), developed by Clary et al. (1998), examines how behaviors help individuals achieve personal and social goals. Clary et al.’s (1998) categories of motivation include values (importance of helping others); understanding (activities that fulfill a desire to learn); social (influence by significant others); career (exploring job opportunities or work advancement); esteem (improving personal self-esteem); and protective (escaping from negative feelings). Wright et al. (2015) studied the motivations of birders in South Africa using a modified version of the VFI and found five categories of motivation to be most important: Recreation/nature; values; personal growth; social interactions; and project organization.

The second perspective comes from Self-Determination Theory (SDT), which treats motivation as an explanatory variable for meeting basic psychological needs (i.e., competency, relatedness, and autonomy) and describes different types of motivations as falling on a continuum from intrinsic to extrinsic (Ryan and Deci 2000a, 2000b). According to SDT, individuals are likely to continue pursuing a goal to the extent that they perceive intrinsic value in the pursuit of that goal (i.e., the extent to which they experience satisfaction in performing associated behaviors themselves versus performing behaviors to comply with extrinsic goals such as conforming to social pressures, fear, or receiving rewards). Although SDT can help practitioners better understand the psychological needs behind participation, few published studies have used SDT in the context of citizen science. One exception is a paper by Nov et al. (2014), which used SDT with social movement participation models in an examination of three digital citizen science projects. These researchers found that intrinsic motivation was one of four drivers that influenced quantity of participation, but that it did not affect quality of participation.

In the context of citizen science, motivation can serve as both an input and outcome, i.e., to understand the basis of motivation for ISE/citizen science experiences (input) and to sustain motivation to continue participating over long time periods (outcome). However, most studies have examined reasons for participation such as the desire to contribute (see Bell et al. 2008; Hobbs and White 2012; McCaffrey 2005; Raddick et al. 2010; Reed et al. 2013), rather than motivations, which describe the psychological underpinnings of behavior (e.g., “because it makes me feel good”). In an examination of motivation in online projects, Rotman et al. (2012) described a complex and changing framework for motivation that was influenced by participant interest, recognition, and attribution. Although several studies have purported to examine motivation, it has not been defined nor studied uniformly throughout the field of citizen science. Nevertheless, the major consensus appears to be that motivation for citizen science, like other volunteer activities, is dynamic and complex.

Content, Process, and Nature of Science Knowledge

Included within the ISE Framework’s impact category of “awareness, knowledge, and understanding” are several subcategories such as knowledge and understanding of science content; knowledge and understanding of science processes; and knowledge of the Nature of Science. Knowledge of science content refers to understanding of subject matter, i.e., facts or concepts. Knowledge of the process of science refers to understanding the methodologies that scientists use to conduct research (for example, the hypothetico-deductive model or “scientific method”). Knowledge of the Nature of Science (NOS) refers to understanding the epistemological underpinnings of scientific knowledge and how it is generated, sometimes presented from a post-positivist perspective (Lederman 1992). NOS addresses tenets of science such as tentativeness; empiricism; subjectivity; creativity; social/cultural influence; observations and inferences; and theories and laws (see Lederman 1992, 1999, Lederman et al. 2001, 2002). For improving scientific literacy, understanding of NOS and the process of science are generally considered more important than understanding basic content or subject matter (American Association for the Advancement of Sciences 1993; National Research Council 1996; NGSS 2013), and knowledge of the process of science is a regular component of well-established assessments of science knowledge (National Science Board 2014). Despite this recognition, most attempts to measure science literacy within the ISE field fall back on content knowledge, i.e., rote memorization of facts, rather than knowledge of the nature or process of science (Bauer et al. 2000; Shamos 1995).

Indeed, citizen science evaluations have typically emphasized measuring gains in topical content knowledge as opposed to science process knowledge, with mixed results (Ballard and Huntsinger 2006; Bonney 2004; Braschler et al. 2010; Brewer 2002; Devictor et al. 2010; Evans et al. 2005; Fernandez-Gimenez et al. 2008; Jordan et al. 2011; Kountoupes and Oberhauser 2008; Krasny and Bonney 2005; Phillips et al. 2006; Sickler et al. 2014; Trumbull et al. 2000; Trumbull et al. 2005). Overdevest et al. (2004) did not find a significant increase in project participant knowledge about streams and water quality, probably because new volunteers were already highly knowledgeable about the subject matter. Price and Lee (2013) actually found a decrease in science content knowledge among project participants, likely owing to exaggerated notions of participants’ self-perceived content knowledge before starting the project and the realization of how much they did not know after participating in the project.

However, a few studies have used measures of the process of science to assess impacts of citizen science project participation. Jordan et al. (2011) and Brossard et al. (2005) used adaptations of the science and engineering indicators and showed no gains in understanding of the process of science as a result of citizen science participation. In contrast, Ballard et al. (2008) used interview data to show evidence that the Salal harvesting project “… increased local people’s understanding of the scientific process and of the ecosystem on which they were a part (p. 14)”. And significant increases in understanding of the process of science before and after participation in a stream water quality-monitoring project were reported by Cronin and Messemer (2013). However, this study had a very small sample size, which may limit generalizability of the results.

Likewise, few citizen science projects have attempted to study understanding of the NOS. Jordan et al. (2011) found no evidence for change in knowledge of the NOS using pre-post scenario-based questions in an invasive species project. Price and Lee (2013) found little evidence that project participation influenced epistemological beliefs about NOS, owing to the fact that “epistemological beliefs are personal beliefs and thus harder to change after participating in only one citizen science project” (p. 793). These findings suggest that while citizen science can effectively demonstrate gains in content knowledge, it has a long way to go before it can positively establish increases in understanding of science process and the NOS.

Skills of Science Inquiry

Skills of science inquiry are observable practices that can be transferred to daily life, such as asking and answering questions; collecting data; developing and using models; planning and carrying out investigations; reasoning about, analyzing, and interpreting data; constructing explanations; communicating information; and using evidence in argumentation (National Academies of Science, Engineering, and Medicine 2016; NGSS Lead States 2013).

The hands-on nature of many environmentally based citizen science projects makes them particularly well suited to influence the development and/or reinforcement of certain science-inquiry skills including asking questions; designing studies; collecting, analyzing, and interpreting data; and discussing and disseminating results (Bonney et al. 2009a; Jordan et al. 2012; Phillips et al. 2012; Trautmann et al. 2012). Top priorities for many practitioners are helping participants learn to follow protocols and exercise accurate data collection skills, because these practices directly influence data quality. The field-wide emphasis on data quality likely comes from the large percentage of contributory, scientist-driven projects, for which a key goal is gathering data of sufficient quality to add to the existing knowledge base through publication in peer-reviewed journals. Consequently, many citizen science projects most effectively influence skills that are related to data and sample/specimen collection, identification of organisms, instrument use, and sampling techniques. Many projects also engage participants in the use of various technological tools such as GPS units, digital thermometers, water conductivity instruments, rain gauges, nets, and smartphones, to name just a few (Figure 4).

A few researchers have begun to study skill acquisition in citizen science. Becker et al. (2013) showed an increase in the ability to estimate noise levels with increasing participation in WideNoise, a soundscape project operated through mobile devices. Increases in youths’ self-reported science inquiry skills, such as their perceived ability to identify pond organisms and to develop testable hypotheses before and after participation in Driven to Discover, also have been reported (Meyer et al. 2014). Sullivan et al. (2009) describe the use of communication prompts and strategies to “steer birders toward providing more useful data” and essentially change the birding habits of eBird participants to increase data quality. Using the theory of legitimate peripheral participation, Mugar et al. (2014) used practice proxies, a form of virtual and trace ethnography, to increase accuracy of data annotation among new members. Additionally, some projects have successfully conducted small-scale studies that compare volunteer-collected data to those collected by experts, thereby creating a baseline metric for assessing their participants’ skills (see Crall et al. 2011; Jordan et al. 2011; Schmeller et al. 2009).

Another hallmark of citizen science is the collection of large, publicly available data sets and rich, interactive data visualizations. Many projects that provide data visualizations may seek to enhance skills related to data interpretation, i.e., the ability to effectively comprehend information and meaning, often presented in graphical form (Devictor et al. 2010). In one of the few studies examining data interpretation in citizen science, Thompson and Bonney (2007) showed that even the majority of “active users” of eBird did not properly use the extensive array of data-analysis tools. Numerous studies in educational research have shown that assessing the type of reasoning skills needed for data interpretation requires asking a series of reflective questions to determine one’s justification underlying the reasoning (e.g., Ayala et al. 2002; Roth and Roychoudhury 1993).

Other inquiry skills such as study design, communication, critical thinking, decision making skills, and critically evaluating results are less studied within the citizen science literature. Crall et al. (2012) used open-ended questions to determine whether engaging in an invasive species project improved the abilities of participants to explain a scientific study, write a valid research question, and provide a valid sampling design. These researchers noted positive gains in all but the ability to explain a scientific study. Char et al. (2014) found an increase from pre-post training in the ability of COASST volunteers to correctly weigh evidence to determine whether it contained sufficient information for accurately identifying species. These few studies show the potential for studying citizen science participants to evaluate the development of complex science inquiry skills, but such studies are in their infancy.

Behavior and Stewardship

Behavior change and development of environmental stewardship are among the most sought-after outcomes in science and environmental education programs, both in and out of schools (Bodzin 2008; Heimlich et al. 2008; Kollmuss and Agyeman 2002; Stern 2000; Stern et al. 2008; Vining et al. 2002). Theories examining various determinants of environmental behavior include those espousing the links between knowledge, attitude, and behavior (Hungerford and Volk 1990; Kollmuss and Agyeman 2002; Osbaldiston and Schott 2012; Schultz 2011); attitudes and values (Ajzen 1985; Fishbein and Ajzen 1975); behavior modification and intervention (De Young 1993); and nature exposure (Kaplan 1995; Kellert and Wilson 1993; Ulrich 1993; Wilson 1984).

We define behavior and stewardship as measurable actions resulting from engagement in citizen science, but external to the protocol activities and the specific project-based skills of the citizen science project. For example, collecting water quality data may be a new behavior for a project participant, but if the data collection is part of the project protocol it should be measured as a new skill rather than a new behavior. However, somebody decreasing their water usage as a result of participating in a water quality monitoring project would be an example of behavior change. Our literature review identified five categories of behavior and stewardship that are of interest to the citizen science field and for which we provide definitions below: Global stewardship behaviors; place-based behaviors; new participation; community or civic action; and transformative lifestyle changes.

Global stewardship refers to deliberate changes in behavior that minimize someone’s individual ecological footprint and which collectively can have global influence (e.g., installing low-flow shower heads, recycling, purchasing energy-efficient appliances). Place-based behaviors refer to observable actions to directly maintain, restore, improve, or educate about the health of an ecosystem beyond the activities of a citizen science project (e.g., removing invasive species; cleaning up trash; eliminating pesticide use; purchasing locally grown food; engaging in outreach to youth groups). New participation is defined as engagement in science or environmental activities, organizations, or projects spurred on by participation in a citizen science project. Community or civic action refers to participation in civic, governmental, or cultural affairs to solve problems at the local, regional, or national level. Actions could include donating to environmental organizations, signing petitions, speaking out against harmful environmental practices, or recruiting others to participate in environmental causes. Finally, transformative lifestyle changes are efforts that require a strong up-front cost or long-term commitment to maintain, such as investing in a hybrid vehicle, becoming a vegetarian, or pledging to use mass transit whenever possible.

Citizen science projects, especially those dealing with environmental topics, are typically hands-on, occur in local environments, and require repeated monitoring and data gathering, making them natural conduits for affecting behavior change (Wells and Lekies 2012). However, research has been limited and results have been mixed regarding the actual influence of citizen science on behavior change. For example, in a study examining two different projects, one on pollinators and one on coyotes, Toomey and Domroese (2013) show that participants engage in new activities and change their gardening practices, but otherwise did not take part in advocacy or change their environmental stewardship practices. Crall et al. (2012) found significant differences between current and planned behavior as a result of participating in an invasive species project using self-reported measures, but the actual behavior change was not well described. Using a case-study approach, Oberhauser and Prysby (2008) claim that participants of the Monarch Larva Monitoring Project “work to preserve habitat at many levels, from advocating a more environmentally friendly mowing regimen and insect-friendly pest control, to challenging parking lot, building, and road development projects that threaten monarch habitat (p. 104).” However, the source of these data or accompanying methodologies are not clearly described. Cornwell and Campbell (2012) also used a case study approach and were able to document advocacy and political action by volunteers which directly benefited sea turtle conservation. Evans et al. (2005) documented locally, place-based stewardship in a bird breeding program, while other projects showed no change in place-based stewardship practices (Jordan et al. 2011). In a study of human health effects of industrial hog operations, Wing et al. (2008) describe actions being taken by community groups to engage in decision-making that addresses local environmental injustices. Taken together, these examples provide some evidence that citizen science may influence behavior and stewardship, but more robust methodologies are needed to establish causation. Plenty of anecdotal data also highlight other examples of behavior change that have not been published or exist only in the gray literature.