Start Submission

Reading: Scaling Up: Citizen Science Engagement and Impacts Beyond the Individual


A- A+
Alt. Display

Research Papers

Scaling Up: Citizen Science Engagement and Impacts Beyond the Individual


Mary Clare Hano ,

Office of Research and Development, United States Environmental Protection Agency, Durham, NC, US
X close

Linda Wei,

Office of Research and Development, United States Environmental Protection Agency, Durham, NC, US
X close

Bryan Hubbell,

Office of Research and Development, United States Environmental Protection Agency, Durham, NC, US
X close

Ana G. Rappold

Office of Research and Development, United States Environmental Protection Agency, Durham, NC, US
X close


As the application of citizen science expands to address increasingly complex social problems (e.g., community health), there is opportunity to consider higher-order engagement beyond that of individual members of a community.

The US Environmental Protection Agency (EPA) is working to foster public engagement in science through Smoke Sense, which is a citizen science research project that aims to reduce the public health burden of wildland fire smoke. Smoke Sense is facilitated by a mobile app that shares information on air quality, smoke, and health, and collects individual reports of smoke exposure and concurrent health symptoms. However, Smoke Sense is also generating interest among organizations that are working in this problem domain at the local, state, national, and tribal levels. The interest in the citizen science project at the organizational level led us to ask: What would motivate members of organizations to engage in a citizen science project like Smoke Sense, and how do they envision that engagement could improve public health?

To explore these questions, we collected data through interviews with individuals who were engaged in Smoke Sense through their work at local, state, and tribal environmental and health agencies in the western US. An inductive, thematic analysis suggests that individuals’ motivations stem from their experiences and challenges with smoke in their different roles (e.g., personal, professional, and community member). Respondents envisioned leveraging both the app itself and the data generated by individuals in their region to enhance and support their existing outreach and communication efforts related to smoke. Citizen science projects that aim to address issues in complex problem domains should be designed to complement the work of partner organizations.

How to Cite: Hano, M.C., Wei, L., Hubbell, B. and Rappold, A.G., 2020. Scaling Up: Citizen Science Engagement and Impacts Beyond the Individual. Citizen Science: Theory and Practice, 5(1), p.1. DOI:
  Published on 07 Jan 2020
 Accepted on 26 Nov 2019            Submitted on 21 Mar 2019

Introduction and Background

Public engagement in science

Citizen science is shaping the way that we build knowledge and influence change. By citizen science, we mean the voluntary participation of the public – not just citizens of any particular government – in any stage of the scientific process beyond the role of research subject (US Environmental Protection Agency 2016; Office of Science and Technology Policy 2015). This approach to research relies on the involvement of individuals who may have little to no formal training in scientific research processes. In recent years new interdisciplinary membership associations (e.g., European Citizen Science Association 2018; Citizen Science Association 2018), academic conferences, and peer-reviewed journals have grown in response to the demand for this inclusionary approach to research. In addition, new governmental policies and guidance in the US and European Union to address social and scientific challenges have been developed and leverage citizen science (Office of Science and Technology Policy 2015; Horizon2020 2018; Brabham 2008; US Environmental Protection Agency 2016).

Many citizen science research projects explore environmental questions, typically structuring public engagement as data collection and analysis. However, citizen science is evolving, and there is a growing desire to more effectively incorporate stakeholder perspectives in various stages of research, for example, in problem formulation, data collection and analysis, and dissemination (Irwin 2018; Wiggins and Crowston 2011). The interest and growth in the application of citizen science inspires questions about how the interdisciplinary theories, methodologies, processes, and outcomes associated with this research design can expand scientific knowledge and inform strategies to address complex social issues.

Complex social problems, community systems, citizen science, and motivations to engage

Complex social problems are issues that concern our broader society, are not the responsibility of any single organization or entity, and which may be improved upon but rarely “solved” (Rittel and Webber 1973; Head and Alford 2013; Loseke 2017). Such issues arise in domains including education, public health, national security, disaster management, environmental protection, and environmental health. Efforts to address complex social problems often involve organizations that work in these domains at local, state, regional, national, and international levels. For example, efforts to improve public health occur at the local level among organizations such as local public health departments, medical providers, social services, and community collaboratives. At the state level, these public health efforts occur among state health agencies, state medical associations, and regional medical providers. At a national level, efforts to improve public health occur among federal health agencies like the US Environmental Protection Agency (EPA) and US National Institutes for Health (NIH), as well as national advocacy and membership organizations. Internationally, public health work occurs through multilateral organizations like the World Health Organization, UNICEF, Doctors Without Borders, and US AID. Within and across levels, the collection of organizations forms a system.

Strategies designed to improve complex public problems may be targeted at a system level (Provan and Milward 1995; Foster-Fishman, Nowell, and Yang 2007; Durlak et al. 2007; Altman 1995). System-level change focuses on understanding the stakeholders associated with a complex problem, examining how they are working toward addressing the solution individually and collectively, and identifying and implementing strategies to improve the collective effort. Foster-Fishman, Nowell, and Yang (2007) define systems change as an “intentional process designed to alter the status quo by shifting and realigning the form and function of a targeted system.” There is an overlap in the concepts of system change and citizen science, as well as other forms of participatory research like community-based research and action research, as all of them depend upon broad stakeholder engagement (Braun et al. 2012; Cacari-Stone et al. 2014; Israel et al. 2000; Israel et al. 2008).

When it comes to efforts to bring about system-level change, understanding how individuals, organizations, and systems are connected and interact is critical to producing outcomes. Integrating citizen science concepts with system-change projects has the potential to increase engagement in the projects by providing structured opportunities through which individuals and groups can participate in collective change efforts. To capitalize on the potential for increasing engagement in broad system-change efforts through incorporating citizen science concepts, understanding motivations for engaging in citizen science is important.

Literature on the topic of motivations for engaging in citizen science has a strong focus on specific factors that prompt or sustain individual engagement in citizen science data collection or analysis projects. It reveals both intrinsic and extrinsic motivations for initial and continued engagement with a project (Rotman et al. 2012; Reed et al. 2013; Raddick et al. 2010; Nov, Arazy, and Anderson 2011; Iacovides et al. 2013; Crowston and Prestopnik 2013). Intrinsic factors include gaining a sense of helping, a sense of collectivism, enjoyment from interacting with others, and the positive reactions about participating received from others (Reed et al. 2013; Nov, Arazy, and Anderson 2011). With respect to extrinsic factors, Nov, Arazy, and Anderson (2011) found that motivators such as game-based rewards were less relevant for intentions to engage. However, Iacovides et al. (2013) found that interactive games may prompt continued participation.

Despite these and other papers focused on individual motivations to participate in citizen science, the literature includes little focus on understanding citizen science in terms of efforts to bring about system-level change. Accordingly, little is known about the motivations to engage in citizen science from the perspective of members of system-level organizations. The objectives of this study are to understand what factors motivate individuals who are members of public-serving organizations to engage in citizen science and how they foresee citizen science projects influencing their community. These objectives are reflected in the following research questions:

RQ1: What motivates members of organizations to engage in citizen science projects related to their professional roles?

RQ2: In what ways do members of organizations foresee citizen science projects being used to impact their stakeholders and communities?


The research questions for this study call for an inductive approach to inquiry, as there is limited attention in the literature about organizational motivations for engagement in citizen science projects in complex social problems, thus little evidence-base for testing hypotheses (Nowell and Albrecht 2018; Morse and Niehaus 2009). When using an inductive approach to inquiry the researcher does not begin with a proposition and explore the extent to which it holds true within a specific context, as is the case with a deductive research approach. Rather, the researcher begins with a case or phenomenon of interest and seeks to richly describe it for the purpose of drawing meaningful insights (Nowell and Albrecht 2018; Eisenhardt and Graebner 2007).


Smoke Sense is a citizen science program led by the US EPA (U.S. Environmental Protection Agency 2018). Smoke Sense aims to reduce adverse health effects related to wildfire smoke by educating users about wildfire smoke and health, and encouraging action related to protective health behaviors. Smoke Sense also aims to investigate health outcomes associated with smoke exposure, individual behavioral responses to smoke, and effective health risk communication. Participation in Smoke Sense is facilitated through a mobile app; the main dashboard is shown in Figure 1. Since its launch in August 2017, the Smoke Sense app has been downloaded by more than 30,000 users.

Figure 1 

Smoke Sense App dashboard, which is the main user interface within the app.

Smoke Sense intends to impact individual and system levels, shown in the diagram in Figure 2, which displays the project’s multi-level two-way interaction framework. The connections shown are among the Smoke Sense research team, individual participants, and networks of organizational partners across local, state, federal, and tribal agencies. Through these connections, the Smoke Sense team has intentional interactions and communications with users across levels. Smoke Sense engages with individuals across the US via the mobile app, through which users can access contemporaneous information on local air quality, fires, and smoke plumes, as well as information on health effects of poor air quality and actions that can be taken to reduce exposures. On a weekly basis, users are asked to report any smoke exposures, health symptoms, and actions taken in response to smoke, and to respond to questions about their perceptions on health messaging and social norms. All data that users provide through the Smoke Sense app are anonymous.

Figure 2 

Smoke Sense Research Initiative Interaction Framework.

Moving from an individual level to community, state, regional, and national levels, the Smoke Sense research team interacts with members of organizations and groups of organizations that are also working in the same problem domain of air quality and health. The Smoke Sense research team aims to connect with organizations that provide services or supports to community members and other stakeholders, recognizing the shared objectives among these partner organizations. Examples of organizations that the Smoke Sense team works with include public schools, universities, local and state departments of public health, local and state environmental agencies, and consortia and multiorganizational partnerships. Each of these organizations is doing something to address the issue of smoke at the local, state, national, or tribal level. Smoke Sense leverages the insights and connections of these organizations to advance individual-level engagement with the project, which will hopefully lead to reducing the public health burden of exposure to smoke.

Data collection interview process

To explore the research questions, we collected interview data from individuals in communities affected by wildland fire smoke and who are members of local-, state-, and tribal-level organizations working in the environmental and health fields. Semi-structured interviews were conducted using an interview protocol developed by the research team and based on established qualitative data collection guidelines and principles (Patton 2015; Miles, Huberman, and Saldana 2013; Creswell 2014). Prior to data collection, the instrument was reviewed for structure and clarification by reviewers external to the research team. The first section focused on the respondent’s experiences with smoke, the second on factors associated with their motivations for engagement in Smoke Sense as part of their professional responsibilities, and the third on the ways they envision the individuals and organizations in their community might benefit or be influenced by Smoke Sense and similar citizen science projects. A copy of the interview protocol is included in the supplemental file.

The semi-structured interview method allowed participants to respond to each question however they chose, without being limited to a set of predefined response options. Participating in the interviews was completely voluntary, as was responding to any of the questions posed during the interview. Participants were free to decline an interview request and were advised during the informed consent process that they were welcome to decline to answer any of the questions simply by stating they do not wish to respond and would like to move to the next item. Each interview was conducted by phone. Interviews lasted approximately 45–60 minutes each and were recorded and transcribed. The transcripts were checked for accuracy as part of the data organization and cleaning process by cross-walking the text of each transcript document while listening to a recording of the interview. The interview transcripts used in analysis are confidential and are stored on secure EPA servers. The study was approved by the University of North Carolina – Chapel Hill Institutional Review Board.


We used a purposeful sampling strategy for sample identification (Goetz and LeCompte 1984; Patton 2002; Palinkas et al. 2015; Morse and Niehaus 2009). We chose this strategy because exploring the research questions in this context requires interviewees who can provide information-rich responses. In other words, we needed to interview individuals who could speak about their motivations to engage in Smoke Sense and the impacts they expected that would result from that engagement. To identify potential respondents, we drew upon the list of organizations and individuals who had interacted with the Smoke Sense team as part of the citizen science project. Individuals who met the following criteria were invited to participate: (1) members of organizations in the health and environmental fields and thus part of the system that responds to smoke events; (2) individuals who engaged with Smoke Sense during its pilot year; and (3) indiiduals who worked in areas affected by smoke during the 2017 wildfire season. As an exploratory study conducted by the US EPA, the research was designed while taking into account the requirements of the Paperwork Reduction Act of 1995, which limits to nine the number of individuals who can be recruited for participation in research without further information collection request approvals. To recruit participants, the research team emailed a group of nine individuals to share introductory information about the study. In the following week, the team followed up with each person by phone to explain more about the study, invite the person’s participation, and answer any questions. Nine individuals agreed to participate; however, only eight were available for an interview during the data collection timeframe (December 2017 through January 2018). These individuals included three employees of public organizations at the local level, four at the state level, and one at the tribal level in the western United States, all of whose work intersects wildland fire and associated smoke, air quality, and public health.


A within-case, inductive phenomenological thematic analysis strategy was used to analyze the transcripts (Moran 2002; Miles, Huberman, and Saldana 2013; Patton 2002). A two-person team of coders engaged in a multi-step process to identify and characterize insights shared among participants. In this process, selections of text salient to the research questions were coded using short descriptive characterizations of the content, abstracting from the verbatim response to higher level thematic ideas that could be combined across respondents. Prior to coding, the transcriptions were assessed for quality, checking for omissions and errors by cross-walking each transcript against the audio recordings. Next, each coder read the set of cleaned transcripts to develop a general understanding of the data prior to analysis. Then the coding team engaged in an iterative coding process to reduce the data and identify themes by research question (Figure 3). In this process, each person independently coded the transcripts by identifying selections of text and applying descriptive codes to them, using the direct quote in some cases. Next, the coders met to reconcile and reach consensus about which selections of text were identified and how those selections were characterized and coded. This coding strategy is a simple, commonly used approach to identifying meanings within quotes. For second-order coding, the first-order codes and accompanying quotations were reanalyzed according to how each fit the responses to the research questions and their relationship to each other. The last cycle was third-order coding, which further refined the categories and is presented in the following section. Throughout the analysis process, the team regularly recorded and discussed interim impressions and the relationship between those impressions and our understanding of the data. That process allowed the team to take into account how our perceptions and deepening understanding of the data were influencing the coding and interpretation processes (Patton 2002; Miles, Huberman, and Saldana 2013).

Figure 3 

Iterative coding process used in analysis.

As with all scientific research, rigor, reliability, and validity are critical in qualitative analysis (Lincoln and Guba 1985; Morse 2015; Morse et al. 2002). In qualitative data analysis, these aspects are often intertwined; however, reliability is about consistency and dependability (Morse 2015; Given 2008), and validity is about credibility and appropriateness of the process and findings in the context of the research objectives and questions (Patton 2002; Miles, Huberman and Saldana 2013). Consistency in the approach and analysis process is demonstrated in qualitative research with careful attention to documentation and transparency and the extent to which the coding and findings can be replicated by others who are familiar with the data. In the present study, the team recorded and detailed all aspects of the data collection and analysis processes shown in Figure 3 and worked to reach consensus by discussing and reconciling codes to support consistency and reproducibility of the coding schema. Using source, stakeholder, analyst, and theory/perspective triangulation is a common approach to establishing validity in qualitative research (Patton 2002; Miles, Huberman, and Saldana 2013). We shared interim findings with a selection of respondents to assess the extent to which the analysis and findings reflect their understanding of the issue. Further, the interim and final codes were reviewed and assessed by members of the research group who were not part of the coding team.


We asked respondents what motivated their engagement in the Smoke Sense citizen science project, and what they anticipated or perceived that engagement might mean for themselves and their communities. In discussing their motivations to engage and anticipated outcomes from their engagement, respondents shared their personal and professional experiences with wildfire smoke, along with the challenges that their organizations and communities faced during those past experiences (Figure 4). The themes presented here were identified across respondents and are shared with select quotes.

Figure 4 

Motivations and anticipated implications for engaging in Smoke Sense as a member of an organization.

What motivates organizational members to engage in Smoke Sense?

Respondents’ experiences translated into different motivations depending on whether they were discussing their motivations as individuals, as members of their organization, or as members of their broader community. Partly driving respondents’ interest in Smoke Sense from a personal perspective was concern about how smoke affects their loved ones. The potential for the app to be a new resource, coupled with their expectation that wildfire smoke will continue to occur in the future, heightened their interest.

“As a parent of two young children, smoke is something that I was more acutely aware of than I may have been otherwise in terms of my personal experience.” State Health Agency Member

Another way that respondents talked about their motivations was in the context of their professional roles as members of organizations that respond to smoke. A variety of public and private organizations at the local, state, and national levels work to share information with individual and organizational stakeholders during wildfire smoke events. Respondents emphasized how members of these organizations work beyond the scope of their own discipline or area of expertise to access information to better understand the situation. Frequently this information is used as part of their own organization’s communication messaging that is shared with community stakeholders and partner agencies. The content of this outreach communication typically focuses on the status of the smoke situation, potential health impacts, the actions that individuals and organizations should take, impacts on scheduled events (including school and other formal or informal events), and the expected duration. One popular way that this information is communicated is through smoke blogs.

“People want to know what the forecast looks like, how the smoke is going to impact their health, whether to cancel outdoor events like soccer or football practices, and basically how to protect themselves and how long they’re going to be in that protection mode.” State Environmental Agency Member

“We try to have coordinated communication effort for the public [that includes the recommendation] to go to the blog for general smoke information including current levels of smoke, forecasts, and all kinds of other information about the wildfire, the health effects from smoke, and things people can do to protect themselves. … I’d say the smoke blog is our main source of multiple agencies contributing to the communication effort.” State Environmental Agency Member

Interviewees reported that their interest and motivation to participate in Smoke Sense grew out of their responsibility to be aware of what resources are available in the area of wildfire smoke and health. As professionals who work to collect, assess, interpret, and share information, there was a sense of responsibility for being aware of available resources and tools.

“I do a lot of the statewide coordination work and the Smoke Sense app is in my workhouse. So I have an obligation to know about what’s going on, what’s available out there in terms of apps and information about smoke, and what other agencies are working on.” State Environmental Agency Member

Respondents were members of organizations that have a responsibility for communicating to the public and partner agencies about air quality and health. Smoke Sense seemed like a tool that could assist with their existing professional communication responsibilities. Respondents perceived the app as a new resource that could potentially support or complement their own efforts to protect public health.

“[The project] was a really natural fit with my job. Wildfire smoke is a big air quality concern. I’m the only air quality person at the department of health, so it’s a big part of what I do. Even if there hadn’t been a statewide event, I was already looking for ways to improve our health communication.” State Health Agency Member

The Smoke Sense project being facilitated through a mobile application was another factor that garnered interest and engagement among respondents. Interviewees were particularly interested in learning more about the app because it delivers information directly to stakeholders’ phones, which was a communication pathway that they recognized as important but had not yet leveraged in their own organizations.

“For years I have been expressing the need for our communication to be usable on smartphone interface for the public.” State Environmental Agency Member

In addition to discussing motivations from personal and professional roles, interviewees also discussed their motivations in the context of their roles as a community member. Respondents noted the citizen science project was an opportunity to increase community engagement and incorporate their community’s experiences into a broader conversation about smoke and public health. Others were motivated by a curiosity about what the app could possibly do for community members.

“[I wanted to] make sure tribal voices are heard.” Tribal Environmental Agency Member

“I was curious about the app, and what it could do for citizens.” State Environmental Agency Member

What Smoke Sense could mean for individuals, organizations, and communities

The second section of the interview focused on respondents’ perceptions about what citizen science projects such as Smoke Sense could mean for their communities. At the bottom of Figure 4 are the themes that we identified across respondents’ responses with respect to what they anticipated would be an outcome of engaging with Smoke Sense. These themes were discussed in terms of the impact that engagement in the project could have for individual users of the app, the organizations that respondents were members of, and their community at large.

Respondents anticipated that the app will help individual users make a connection between wildfire smoke, air quality, and personal health, which are topics perceived by respondents as separate in the minds of many community members. Interviewees expect this to be achieved by users’ interactions with the app and by answering questions about smoke reports and concurrent health symptoms. By viewing questions on both topics in close proximity in the same app environment, respondents suggested that app users may begin making stronger mental associations between the concepts of air quality and health.

“Asking questions about peoples’ experiences with smoke and their health effects, puts [the relationship between those] on their mind, [they may start] thinking ‘oh these things are related; there’s a reason they’re asking.’ The questions become a little bit of a tool for indicating something we think is important.” -State Health Agency Member

“Linking air quality and health on the app helps people make that connection in their head. … I think the app has a lot of potential to connect those topics for them. They might have attributed the health effects to something else and not really paid attention to their smoke exposure, so that’s part of the opportunity here.” Local Government Representative

“We put out quite a bit of information about air quality all year round but especially during wildfire season. And we’ve been building up that level of awareness [in our community] for a long time. I think one of the reasons I like the app is because it directly ties the air quality to the effects on the people. A lot of people wouldn’t even tie those together unless they were prompted to, so I think the app is a really good tool to get that done.” Tribal Environmental Agency Member

On the organizational level, the app was thought of as a tool that would support organizations’ communication efforts. One way that the app can help organizations is by serving as a trusted source of information to which they could direct their stakeholders as part of their organization’s communication work. Respondents also emphasized the inherently complex nature of air quality data. Numerous sources of air quality information and a wide range of indices are available to understand current and forecast air quality conditions and associated potential health risks. The complexity of available information coupled with emergent air quality events, such as inundations with wildfire smoke, can make communicating about the issue a challenge. Help with communication is particularly important during intense smoke events because organizational bandwidth is at a premium.

“Smoke was a topic of people’s concern and frustration, especially from parents, because they would be downloading a bunch of different apps and they all use a slightly different algorithm. So they’d have to ask themselves, ‘Which apps do I believe?’ There is confusion as to what air quality numbers are out there, and what they really mean.” State Environmental Agency Member

“I think for people it can be a lot [to process] when you go to DEQ site and they have two numbers, then you go to AirNow and you get a third number. And you can get the NowCast number which can be done for any point in the day and it’s like ‘Gosh what should I do?’” Local Government Representative

“I think Smoke Sense is another great tool to communicate air quality. That’s really one of the major overriding factors of my interest in the app, is that besides the data gathering, it’s an easy way to get people aware of and to know about air quality.” Tribal Environmental Agency Member

“Making the information available to the public in real time and at their fingertips will help with the amount of pressure and requests that come in to all the different agencies. We don’t have capacity to be on the phone all day long with everybody, answering all their specific questions. Almost everybody has access to a smartphone these days and it is pretty consistent information they’re looking for.” State Environmental Agency Member

Respondents also discussed the need for their organization to effectively serve sensitive and at-risk populations during large-scale smoke events, and the potential for Smoke Sense to help those efforts. For example, people who are young, low-income, and even visitors to the area may not be prepared for smoke intrusion or may have limited awareness about and access to resources that can help them reduce exposure. Particularly challenging is the situation where information is needed to make a decision to act on behalf of others. Respondents frequently receive requests for assistance from administrators in schools and other youth-serving organizations. These requests are for help interpreting information about risks in order to make a decision about canceling or hosting events, including school. Children are identified as a sensitive population with respect to increased risk of adverse health outcomes associated with air pollution, so the administrators who are making the decision that affects the students’ exposures may especially benefit from enhanced and targeted outreach.

“In September when school started, the schools were really struggling with having smoke in the community and having kids in classrooms and at athletic events. I spend a lot of time talking to athletic directors trying to coach them on understanding air quality data, and they’re also getting direction from the state athletic association. And all the numbers are very confusing, even to me and I do a lot of research and have been watching it for a while with our prescribed burn program. I still get confused as to the differing numbers. So that part with schools is still a big challenge.” Local Government Representative

“I see the need for our health district and our agency to meet with the school district and talk about what their policy is for smoke. Hopefully the health district can provide a little bit more input in terms of the orange versus red decision.” Local Environmental Agency Member

“There is some individual who has to make a decision whether to keep the school open during a wildfire.… You have individuals trying to do the best they can with what information they have … and to me that’s a ripe audience [for you] to say, ‘Hey, can we help you out here?’” -State University Member

In addition to the benefits that individual users and organizations responding to smoke may realize, respondents also discussed the potential for their overall community to benefit from a citizen science project like Smoke Sense. Benefits to the community arise in part from the project’s potential for generating novel scientific knowledge in a broad sense, by collecting data that are not currently available for research in the area of wildfire smoke, air quality, and public health.

“We’re a data driven agency and we’ve got people whose job [is] looking for the most updated peer-reviewed literature on this topic specifically, and there’s not a lot out there.” State Environmental Agency Member

“[Through this project] we can actually get that tie between concentration levels and what people’s health effects were.” Tribal Environmental Agency Member

“I think it is a great tool that we can use to understand some of the clinical effects that we don’t really have great research about in terms of the health literature.” State Health Agency Member

“We are doing a community survey to assess the community’s health and trauma in the aftermath of our wildfire event, but that survey is a one-time survey. We’ll gather that data and then report on it.… If you could have this continuous feed of data, it would be a lot more informative and robust, wouldn’t it be great to be ready and to be collecting data using Smoke Sense as soon as the event starts.” Local Health Agency Member

A second community benefit is the possibility for organizations and communities to access geographically relevant data, submitted from users in the area that respondents’ organizations served. The plan for these data could include use in local response, decision-making, planning, research, and outreach initiatives. Respondents saw the potential to combine community-specific data from the Smoke Sense app with other local or state air quality information to support community health. Another example was using the Smoke Sense data to understand baselines and trends in levels of exposures, awareness, and engagement in protective health behaviors of community members. Having that locally relevant data would support their communication and outreach efforts as well as potentially be useful in advancing programs and other initiatives.

“We can use that information as leverage for grants and expanding the programs over here.… I also like the idea of using local results in combination with other surveys and information I already have about air quality.” Tribal Environmental Agency Member

“Having that data would help us to know the basic level of awareness and where we need to go in terms of increasing awareness. We need to have some baseline information so we can see where we started and how are we doing as far as disseminating that information. Having the data would also help us understand if people actually adopt these protective behaviors, which protective behaviors are they adopting, and how long are they persisting in those protective behaviors in at times when the air quality is bad.” State University Member

Findings are summarized in Table 1.

Table 1

Motivations to engage in and anticipated benefits of citizen science transcend individual, organizational, and community levels.

Motivations Benefits

  • Concern for health of family members
  • Protect health and increase awareness
  • Interest in leveraging the app in organizational efforts associated with smoke
  • Professional responsibility for staying abreast of new tools and resources regarding smoke
  • Advance organizational efforts in the problem area
  • Interested in amplifying their community’s voice in the national conversation about smoke
  • Increase individual awareness of connections between wildland fire, smoke, air quality, and health


Implications for citizen science

Citizen science research has clear potential to advance progress in addressing complex social problems. Citizen science projects that address complex social issues may have increased impact by including in the research design scientific objectives as well as educational and action objectives. Wiggins and Crowston’s (2011) typology of citizen science project objectives is a useful starting point for considering citizen science design such that the project might effectively address a complex social problem. The Smoke Sense app combines educational, investigative, and virtual engagement objectives. This combination was an important part of these respondents’ perceptions for broader engagement with Smoke Sense.

Citizen science projects that aim to address issues in complex problem domains should be designed to complement collective efforts among organizations working with a system. A design that is complementary to broader efforts is especially important for federally led projects in order to build increased cooperation among federal, state, local, and tribal partners. Further, there are often a range of organizations working within any given problem domain, and compared to federal agencies, these state, local, and tribal partners may have closer connections with affected populations. For example, in addition to federal environmental and health agencies there also are state, local, and tribal environmental and health agencies working to reduce the public health burden of wildfire smoke. By considering these broader efforts, federal citizen science projects can support the collective objectives across organizations and potentially attract the interest and participation in partners’ work. Smoke Sense provided additional information to the intended audiences of state, local, and tribal agencies that were working on the issue of wildfire smoke and public health, while also collecting information that could help shape future outreach by these agencies. We learned there is interest among partners to obtain and analyze data specific to their area and to potentially integrate Smoke Sense findings into their efforts. Thus, it is important to include structured pathways for continued interaction with partner agencies within a system, including sharing findings and data with individuals and organizations that are also working in the same area. Forethought should be given to how outcomes of federal citizen science projects might impact local, state, and tribal communities, and safeguards should be implemented to ensure that the projects do not inadvertently harm or make the issue more challenging for community organizations that are responding to an issue.

Implications for community response to wildfire smoke and health

There is growing evidence of the adverse health outcomes associated with exposure to particles found in wildfire smoke (Reid et al. 2016a; Reid et al. 2016b; Rappold et al. 2017; Rappold et al. 2011; Wettstein et al. 2018; Cascio 2018; Mirabelli et al. 2009). Increases in individual awareness of both the issue and recommended health behaviors influence the potential for increased engagement in recommended health protective behaviors. Citizen science projects like Smoke Sense can amplify health risk messaging about wildfire smoke by providing an additional platform for reaching individuals affected by smoke. Organizational representatives can use Smoke Sense as a tool to talk about complex social issues with the communities they serve. Additionally, citizen science projects may help staff in partner agencies, such as school administrators or community leaders, by collecting and disseminating data and frameworks that facilitate decision making with respect to mitigating community exposure to smoke from wildland fire.

Implications for Smoke Sense

Understanding the motivations of organizational participants can potentially increase the impact of Smoke Sense by informing plans for how the citizen science project can evolve to better meet collective objectives. One example of how an increased understanding of organizational participants’ motivations and needs is influencing the evolution of the Smoke Sense project is the development of an online data visualization tool. Through these interviews we learned of the importance of making geographically relevant data available to partner organizations working to address the health risks of exposure to wildland fire smoke. The Smoke Sense Data Visualization Tool allows for near real-time access and exploration of the anonymous data submitted through the Smoke Sense app. Through the Smoke Sense Data Visualization Tool, data can be stratified and downloaded, and visualizations of the data can be saved for use by others. Another way that an increased understanding of organizational participants’ motivations is impacting the Smoke Sense project is through new release versions of the app. For example, the smoke forecasting feature was initially not part of the app during the pilot season, but forecasting was identified by partners as a primary topic of information requests. Forecasting was incorporated to provide information that can help individuals better plan their activities during smoky conditions. Understanding partner needs, motivations, and expectations is critical to fostering change in complex problem domains. The Smoke Sense citizen science project team is working to better understand organizational partners and subsequently translate increased understanding into the citizen science project design such that the app can more effectively support collective change in complex problems.


We selected participants who represent public organizations at the intersection of environment and health and who engaged with Smoke Sense in the pilot year of the project. Thus, findings are drawn from a limited set of semi-structured interviews and are not generalizable. Additionally, Smoke Sense is led by a federal agency, the US EPA, which could have affected respondents’ answers to interview questions because of a real or perceived power differential at the organizational level. However, this was not raised as a concern during any of the interviews.


Citizen science methodologies are evolving rapidly. This study provides insights into organizational involvement in citizen science projects that explore issues in complex problem domains. We study this in the context of wildfire smoke and public health. The incidence of large-scale wildfires is increasing (National Interagency Fire Center 2018). Large-scale fires produce smoke plumes that can potentially extend hundreds of miles and affect millions of people living far from the actual fire. In the environmental public health literature, there is increasing interest in the public health impacts of exposure to this smoke and mounting evidence of adverse clinical outcomes associated with exposure. However, less is known about subclinical outcomes and behavioral responses to wildland fire smoke exposure. Smoke Sense is a citizen science project conducted by US EPA. Smoke Sense helps to fill this gap by collecting information about subclinical health outcomes while simultaneously translating current research into health risk messaging to encourage exposure-reducing behaviors. This study is an initial investigation into the motivations and expectations of organizational leaders across public-serving organizations who engage in citizen science projects in the context of wildfire smoke and health.

Using an inductive approach to inquiry, we interviewed a group of individuals who participated in US EPA’s Smoke Sense project as part of their professional responsibilities. We learned that organizational members may be particularly interested in citizen science projects that have the potential to support and advance their own organization’s efforts. Citizen science projects that aim to address issues in complex problem domains should be designed taking into account the broader social and organizational systems in which they are working. In the present study, the Smoke Sense project included educational objectives as well as investigative objectives which supported the organizations that engaged with the project in two ways. First, Smoke Sense supported the communication and outreach efforts of partner organizations. Second, Smoke Sense was viewed as a potential resource to address the need among those partners for data that reflected their stakeholders. Additionally, technologically innovative projects, such as mobile apps, can serve as an appealing tool for partner organizations to use when discussing the issue with stakeholders. However, much remains to be understood about engagement in citizen science at the organization and system levels – for example, understanding how this type of engagement impacts collective goals, which types of engagement are most important, and what processes are most productive for all partners.

By scaling up the opportunities for engagement within citizen science projects from individuals to organizations, we can increase the impact of our citizen science research by supporting partners that are working toward a common goal. Leveraging engagement at the organizational level to bring about change in complex social problems requires a citizen science design that strategically considers partner organizations’ efforts and needs. Understanding the challenges that partner organizations face in dealing with complex social issues, and designing citizen science projects that complement their efforts to the greatest extent possible, can advance the collective goal of addressing overarching complex social problems.

Supplementary File

The supplementary file for this article can be found as follows:

Supplementary File 1

Appendix Scaling Up Interview Guide. DOI:


We are grateful to the individuals who participated in this project for their time and insights. We also want to thank our internal and peer reviewers for their detailed and thoughtful feedback.


This manuscript has been reviewed by the Center for Public Health and Environmental Assessment, United States Environmental Protection Agency, and approved for publication. Approval does not signify that the contents necessarily reflect the views and policies of the Agency, nor does mention of trade names or commercial products constitute endorsement or recommendations for use.

Competing Interests

The authors have no competing interests to declare.

Author Contributions

All authors contributed substantively to the study design and development of the manuscript.


  1. Altman, DG. 1995. Sustaining interventions in community systems: On the relationship between researchers and communities. Health Psychology, 14: 526. DOI: 

  2. Brabham, DC. 2008. Crowdsourcing as a model for problem solving: An introduction and cases. Convergence, 14: 75–90. DOI: 

  3. Braun, KL, Nguyen, TT, Tanjasiri, SP, Campbell, J, Heiney, SP, Brandt, HM, Smith, SA, Blumenthal, DS, Hargreaves, M, Coe, K, Ma, GX, Kenerson, D, Patel, K, Tsark, J and Hébert, JR. 2012. Operationalization of Community-Based Participatory Research Principles: Assessment of the National Cancer Institute’s Community Network Programs. American Journal of Public Health, 102: 1195–1203. DOI: 

  4. Cacari-Stone, L, Wallerstein, N, Garcia, AP and Minkler, M. 2014. The Promise of Community-Based Participatory Research for Health Equity: A Conceptual Model for Bridging Evidence With Policy. American Journal of Public Health, 104: 1615–1623. DOI: 

  5. Cascio, WE. 2018. Wildland fire smoke and human health. Sci Total Environ, 624: 586–595. DOI: 

  6. Citizen Science Association. 2018. Available at [Last accessed 8 November 2019]. 

  7. Creswell, JW. 2014. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks: SAGE Publications. 

  8. Crowston, K and Prestopnik, NR. 2013. Motivation and Data Quality in a Citizen Science Game: A Design Science Evaluation. 450–459. DOI: 

  9. Durlak, JA, Taylor, RD, Kawashima, K, Pachan, MK, Dupre, EP, Celio, CI, Berger, SR, Dymnicki, AB and Weissberg, RP. 2007. Effects of positive youth development programs on school, family, and community systems. American Journal of Community Psychology, 39: 269–286. DOI: 

  10. Eisenhardt, KM and Graebner, ME. 2007. Theory building from cases: Opportunities and challenges. Academy of management journal, 50: 25–32. DOI: 

  11. European Citizen Science Association. 2018. Available at [Last accessed 8 November 2019]. 

  12. Foster-Fishman, PG, Nowell, B and Yang, H. 2007. Putting the system back into systems change: a framework for understanding and changing organizational and community systems. Am J Community Psychol, 39: 197–215. DOI: 

  13. Given, LM. 2008. The Sage encyclopedia of qualitative research methods. Thousand Oaks, CA: Sage Publications, Inc. DOI: 

  14. Goetz, JP and Lecompte, MD. 1984. Ethnography and Qualitative Design in Educational Research. Academic Press. 

  15. Head, BW and Alford, J. 2013. Wicked Problems. Administration & Society, 47: 711–739. DOI: 

  16. Horizon2020. 2018. Horizon 2020. Available at [Last accessed 25 October 2018]. 

  17. Iacovides, I, Jennett, C, Cornish-Trestrail, C and Cox, AL. 2013. Do Games Attract or Sustain Engagement in Citizen Science? A Study of Volunteer Motivations. CHI 2013 Extended Abstracts. DOI: 

  18. Irwin, A. 2018. Citizen Science Comes of Age. Nature, 480. 25 October 2018. DOI: 

  19. Israel, BA, Schulz, A, Parker, E, Becker, A, Allen, A and Guzman, J. 2000. Community-Based Participatory Research Principles. In: O’fallon, LR, Tyson, FL and Dearry, A (eds.), Rationale and Policy Recommendations, A Keynote Address at the Preliminary Meeting of Successful Models of Community–Based Participatory Research, Final Report, 29–31. Washington, DC: NIEHS. 

  20. Israel, BA, Schulz, AJ, Parker, EA and Becker, AB. 2008. Critical issues in developing and following community-based participatory research principles. Community-based participatory research for health. Jossey-Bass. 

  21. Lincoln, YS and Guba, EG. 1985. Naturalistic inquiry. Sage. DOI: 

  22. Loseke, D. 2017. Thinking about social problems: An introduction to constructionist perspectives. New York: Routledge. DOI: 

  23. Miles, MB, Huberman, AM and Saldana, J. 2013. Qualitative Data Analysis. Sage. 

  24. Mirabelli, MC, Kunzli, N, Avol, E, Gilliland, FD, Gauderman, WJ, Mcconnell, R and Peters, JM. 2009. Respiratory symptoms following wildfire smoke exposure: airway size as a susceptibility factor. Epidemiology, 20: 451–9. DOI: 

  25. Moran, D. 2002. Introduction to phenomenology. Routledge. DOI: 

  26. Morse, JM. 2015. Critical analysis of strategies for determining rigor in qualitative inquiry. DOI: 

  27. Morse, JM, Barrett, M, Mayan, M, Olson, K and Spiers, J. 2002. Verification strategies for establishing reliability and validity in qualitative research. International journal of qualitative methods, 1: 13–22. DOI: 

  28. Morse, JM and Niehaus, L. 2009. Principles and procedures of mixed methods design. Walnut Creek, CA: Left. 

  29. National Interagency Fire Center. 2018. Fire Statistics. Available at [Last accessed 14 November 2019]. 

  30. Nov, O, Arazy, O and Anderson, D. 2011. Dusting for science: motivation and participation of digital citizen science volunteers. Proceedings of the 2011 iConference, ACM, 68–74. DOI: 

  31. Nowell, B and Albrecht, K. 2018. A reviewer’s guide to qualitative rigor. Journal of Public Administration Research and Theory. DOI: 

  32. Office of Science and Technology Policy. 2015. Executive Office of the President. Memorandum to the Heads of Executive Departments and Agencies. 

  33. Palinkas, LA, Horwitz, SM, Green, CA, Wisdom, JP, Duan, N and Hoagwood, K. 2015. Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. Administration and policy in mental health, 42: 533–544. DOI: 

  34. Paperwork Reduction Act. 1995. Paperwork Reduction Act. United States. 

  35. Patton, MQ. 2002. Qualitative Research & Evaluation Methods. SAGE Publications. 

  36. Patton, MQ. 2015. Qualitative research & evaluation tools. 4th. Los Angeles, CA: Sage. 

  37. Provan, KG and Milward, HB. 1995. A preliminary theory of interorganizational network effectiveness: A comparative study of four community mental health systems. Administrative science quarterly, 1–33. DOI: 

  38. Raddick, MJ, Bracey, G, Gay, PL, Lintott, CJ, Murray, P, Schawinski, K, Szalay, AS and Vandenberg, J. 2010. Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. Astronomy Education Review, 9. DOI: 

  39. Rappold, AG, Reyes, J, Pouliot, G, Cascio, WE and Diaz-Sanchez, D. 2017. Community Vulnerability to Health Impacts of Wildland Fire Smoke Exposure. Environ Sci Technol, 51: 6674–6682. DOI: 

  40. Rappold, AG, Stone, S, Cascio, WE, Neas, LM, Kilaru, VJ, Carraway, MS, Szykman, J, Ising, A, Cleve, WE, Meredith, JT, Vaughan-Batten, H, Deyneka, L and Devlin, RB. 2011. Peat Bog Wildfire Smoke Exposure in Rural North Carolina Is Associated with Cardiopulmonary Emergency Department Visits Assessed through Syndromic Surveillance. Environmental Health Perspectives, 119: 1415–1510. DOI: 

  41. Reed, J, Raddick, MJ, Lardner, A and Carney, K. 2013. An Exploratory Factor Analysis of Motivations for Participating in Zooniverse, a Collection of Virtual Citizen Science Projects. 610–619. DOI: 

  42. Reid, CE, Brauer, M, Johnston, FH, Jerrett, M, Balmes, JR and Elliott, CT. 2016a. Critical Review of Health Impacts of Wildfire Smoke Exposure. Environ Health Perspect, 124: 1334–43. DOI: 

  43. Reid, CE, Jerrett, M, Tager, IB, Petersen, ML, Mann, JK and Balmes, JR. 2016b. Differential respiratory health effects from the 2008 northern California wildfires: A spatiotemporal approach. Environ Res, 150: 227–35. DOI: 

  44. Rittel, HWJ and Webber, MM. 1973. Dilemmas in a General Theory of Planning. Policy Sciences, 4: 155–169. DOI: 

  45. Rotman, D, Preece, J, Hammock, J, Procita, K, Hansen, D, Parr, C, Lewis, D and Jacobs, D. 2012. Dynamic changes in motivation in collaborative citizen-science projects. Proceedings of the ACM 2012 conference on computer supported cooperative work, ACM, 217–226. DOI: 

  46. U.S. Environmental Protection Agency. 2016. Environmental Protection Belongs to the Public, A Vision for Citizen Science at EPA. Available at [Last accessed 2019]. 

  47. U.S. Environmental Protection Agency. 2018. Smoke Sense Study: A Citizen Science Project Using a Mobile App. Available at [Last accessed 2019]. 

  48. Wettstein, ZS, Hoshiko, S, Fahimi, J, Harrison, RJ, Cascio, W and Rappold, AG. 2018. Cardiovascular and cerebrovascular emergency department visits associated with wildfire smoke exposure in California in 2015. Journal of the American Heart Association, 7. DOI: 

  49. Wiggins, A and Crowston, K. 2011. From Conservation to Crowdsourcing: A Typology of Citizen Science. 44th Hawaii International Conference on System Sciences, IEEE, 1–10. DOI: