Public engagement in science

Citizen science is shaping the way that we build knowledge and influence change. By citizen science, we mean the voluntary participation of the public – not just citizens of any particular government – in any stage of the scientific process beyond the role of research subject (US Environmental Protection Agency 2016; Office of Science and Technology Policy 2015). This approach to research relies on the involvement of individuals who may have little to no formal training in scientific research processes. In recent years new interdisciplinary membership associations (e.g., European Citizen Science Association 2018; Citizen Science Association 2018), academic conferences, and peer-reviewed journals have grown in response to the demand for this inclusionary approach to research. In addition, new governmental policies and guidance in the US and European Union to address social and scientific challenges have been developed and leverage citizen science (Office of Science and Technology Policy 2015; Horizon2020 2018; Brabham 2008; US Environmental Protection Agency 2016).

Many citizen science research projects explore environmental questions, typically structuring public engagement as data collection and analysis. However, citizen science is evolving, and there is a growing desire to more effectively incorporate stakeholder perspectives in various stages of research, for example, in problem formulation, data collection and analysis, and dissemination (Irwin 2018; Wiggins and Crowston 2011). The interest and growth in the application of citizen science inspires questions about how the interdisciplinary theories, methodologies, processes, and outcomes associated with this research design can expand scientific knowledge and inform strategies to address complex social issues.

Complex social problems, community systems, citizen science, and motivations to engage

Complex social problems are issues that concern our broader society, are not the responsibility of any single organization or entity, and which may be improved upon but rarely “solved” (Rittel and Webber 1973; Head and Alford 2013; Loseke 2017). Such issues arise in domains including education, public health, national security, disaster management, environmental protection, and environmental health. Efforts to address complex social problems often involve organizations that work in these domains at local, state, regional, national, and international levels. For example, efforts to improve public health occur at the local level among organizations such as local public health departments, medical providers, social services, and community collaboratives. At the state level, these public health efforts occur among state health agencies, state medical associations, and regional medical providers. At a national level, efforts to improve public health occur among federal health agencies like the US Environmental Protection Agency (EPA) and US National Institutes for Health (NIH), as well as national advocacy and membership organizations. Internationally, public health work occurs through multilateral organizations like the World Health Organization, UNICEF, Doctors Without Borders, and US AID. Within and across levels, the collection of organizations forms a system.

Strategies designed to improve complex public problems may be targeted at a system level (Provan and Milward 1995; Foster-Fishman, Nowell, and Yang 2007; Durlak et al. 2007; Altman 1995). System-level change focuses on understanding the stakeholders associated with a complex problem, examining how they are working toward addressing the solution individually and collectively, and identifying and implementing strategies to improve the collective effort. Foster-Fishman, Nowell, and Yang (2007) define systems change as an “intentional process designed to alter the status quo by shifting and realigning the form and function of a targeted system.” There is an overlap in the concepts of system change and citizen science, as well as other forms of participatory research like community-based research and action research, as all of them depend upon broad stakeholder engagement (Braun et al. 2012; Cacari-Stone et al. 2014; Israel et al. 2000; Israel et al. 2008).

When it comes to efforts to bring about system-level change, understanding how individuals, organizations, and systems are connected and interact is critical to producing outcomes. Integrating citizen science concepts with system-change projects has the potential to increase engagement in the projects by providing structured opportunities through which individuals and groups can participate in collective change efforts. To capitalize on the potential for increasing engagement in broad system-change efforts through incorporating citizen science concepts, understanding motivations for engaging in citizen science is important.

Literature on the topic of motivations for engaging in citizen science has a strong focus on specific factors that prompt or sustain individual engagement in citizen science data collection or analysis projects. It reveals both intrinsic and extrinsic motivations for initial and continued engagement with a project (Rotman et al. 2012; Reed et al. 2013; Raddick et al. 2010; Nov, Arazy, and Anderson 2011; Iacovides et al. 2013; Crowston and Prestopnik 2013). Intrinsic factors include gaining a sense of helping, a sense of collectivism, enjoyment from interacting with others, and the positive reactions about participating received from others (Reed et al. 2013; Nov, Arazy, and Anderson 2011). With respect to extrinsic factors, Nov, Arazy, and Anderson (2011) found that motivators such as game-based rewards were less relevant for intentions to engage. However, Iacovides et al. (2013) found that interactive games may prompt continued participation.

Despite these and other papers focused on individual motivations to participate in citizen science, the literature includes little focus on understanding citizen science in terms of efforts to bring about system-level change. Accordingly, little is known about the motivations to engage in citizen science from the perspective of members of system-level organizations. The objectives of this study are to understand what factors motivate individuals who are members of public-serving organizations to engage in citizen science and how they foresee citizen science projects influencing their community. These objectives are reflected in the following research questions:

RQ1: What motivates members of organizations to engage in citizen science projects related to their professional roles?

RQ2: In what ways do members of organizations foresee citizen science projects being used to impact their stakeholders and communities?

Context

Smoke Sense is a citizen science program led by the US EPA (U.S. Environmental Protection Agency 2018). Smoke Sense aims to reduce adverse health effects related to wildfire smoke by educating users about wildfire smoke and health, and encouraging action related to protective health behaviors. Smoke Sense also aims to investigate health outcomes associated with smoke exposure, individual behavioral responses to smoke, and effective health risk communication. Participation in Smoke Sense is facilitated through a mobile app; the main dashboard is shown in Figure 1. Since its launch in August 2017, the Smoke Sense app has been downloaded by more than 30,000 users.

Smoke Sense intends to impact individual and system levels, shown in the diagram in Figure 2, which displays the project’s multi-level two-way interaction framework. The connections shown are among the Smoke Sense research team, individual participants, and networks of organizational partners across local, state, federal, and tribal agencies. Through these connections, the Smoke Sense team has intentional interactions and communications with users across levels. Smoke Sense engages with individuals across the US via the mobile app, through which users can access contemporaneous information on local air quality, fires, and smoke plumes, as well as information on health effects of poor air quality and actions that can be taken to reduce exposures. On a weekly basis, users are asked to report any smoke exposures, health symptoms, and actions taken in response to smoke, and to respond to questions about their perceptions on health messaging and social norms. All data that users provide through the Smoke Sense app are anonymous.

Moving from an individual level to community, state, regional, and national levels, the Smoke Sense research team interacts with members of organizations and groups of organizations that are also working in the same problem domain of air quality and health. The Smoke Sense research team aims to connect with organizations that provide services or supports to community members and other stakeholders, recognizing the shared objectives among these partner organizations. Examples of organizations that the Smoke Sense team works with include public schools, universities, local and state departments of public health, local and state environmental agencies, and consortia and multiorganizational partnerships. Each of these organizations is doing something to address the issue of smoke at the local, state, national, or tribal level. Smoke Sense leverages the insights and connections of these organizations to advance individual-level engagement with the project, which will hopefully lead to reducing the public health burden of exposure to smoke.

Data collection interview process

To explore the research questions, we collected interview data from individuals in communities affected by wildland fire smoke and who are members of local-, state-, and tribal-level organizations working in the environmental and health fields. Semi-structured interviews were conducted using an interview protocol developed by the research team and based on established qualitative data collection guidelines and principles (Patton 2015; Miles, Huberman, and Saldana 2013; Creswell 2014). Prior to data collection, the instrument was reviewed for structure and clarification by reviewers external to the research team. The first section focused on the respondent’s experiences with smoke, the second on factors associated with their motivations for engagement in Smoke Sense as part of their professional responsibilities, and the third on the ways they envision the individuals and organizations in their community might benefit or be influenced by Smoke Sense and similar citizen science projects. A copy of the interview protocol is included in the supplemental file.

The semi-structured interview method allowed participants to respond to each question however they chose, without being limited to a set of predefined response options. Participating in the interviews was completely voluntary, as was responding to any of the questions posed during the interview. Participants were free to decline an interview request and were advised during the informed consent process that they were welcome to decline to answer any of the questions simply by stating they do not wish to respond and would like to move to the next item. Each interview was conducted by phone. Interviews lasted approximately 45–60 minutes each and were recorded and transcribed. The transcripts were checked for accuracy as part of the data organization and cleaning process by cross-walking the text of each transcript document while listening to a recording of the interview. The interview transcripts used in analysis are confidential and are stored on secure EPA servers. The study was approved by the University of North Carolina – Chapel Hill Institutional Review Board.

Sample

We used a purposeful sampling strategy for sample identification (Goetz and LeCompte 1984; Patton 2002; Palinkas et al. 2015; Morse and Niehaus 2009). We chose this strategy because exploring the research questions in this context requires interviewees who can provide information-rich responses. In other words, we needed to interview individuals who could speak about their motivations to engage in Smoke Sense and the impacts they expected that would result from that engagement. To identify potential respondents, we drew upon the list of organizations and individuals who had interacted with the Smoke Sense team as part of the citizen science project. Individuals who met the following criteria were invited to participate: (1) members of organizations in the health and environmental fields and thus part of the system that responds to smoke events; (2) individuals who engaged with Smoke Sense during its pilot year; and (3) indiiduals who worked in areas affected by smoke during the 2017 wildfire season. As an exploratory study conducted by the US EPA, the research was designed while taking into account the requirements of the Paperwork Reduction Act of 1995, which limits to nine the number of individuals who can be recruited for participation in research without further information collection request approvals. To recruit participants, the research team emailed a group of nine individuals to share introductory information about the study. In the following week, the team followed up with each person by phone to explain more about the study, invite the person’s participation, and answer any questions. Nine individuals agreed to participate; however, only eight were available for an interview during the data collection timeframe (December 2017 through January 2018). These individuals included three employees of public organizations at the local level, four at the state level, and one at the tribal level in the western United States, all of whose work intersects wildland fire and associated smoke, air quality, and public health.

Analysis

A within-case, inductive phenomenological thematic analysis strategy was used to analyze the transcripts (Moran 2002; Miles, Huberman, and Saldana 2013; Patton 2002). A two-person team of coders engaged in a multi-step process to identify and characterize insights shared among participants. In this process, selections of text salient to the research questions were coded using short descriptive characterizations of the content, abstracting from the verbatim response to higher level thematic ideas that could be combined across respondents. Prior to coding, the transcriptions were assessed for quality, checking for omissions and errors by cross-walking each transcript against the audio recordings. Next, each coder read the set of cleaned transcripts to develop a general understanding of the data prior to analysis. Then the coding team engaged in an iterative coding process to reduce the data and identify themes by research question (Figure 3). In this process, each person independently coded the transcripts by identifying selections of text and applying descriptive codes to them, using the direct quote in some cases. Next, the coders met to reconcile and reach consensus about which selections of text were identified and how those selections were characterized and coded. This coding strategy is a simple, commonly used approach to identifying meanings within quotes. For second-order coding, the first-order codes and accompanying quotations were reanalyzed according to how each fit the responses to the research questions and their relationship to each other. The last cycle was third-order coding, which further refined the categories and is presented in the following section. Throughout the analysis process, the team regularly recorded and discussed interim impressions and the relationship between those impressions and our understanding of the data. That process allowed the team to take into account how our perceptions and deepening understanding of the data were influencing the coding and interpretation processes (Patton 2002; Miles, Huberman, and Saldana 2013).

As with all scientific research, rigor, reliability, and validity are critical in qualitative analysis (Lincoln and Guba 1985; Morse 2015; Morse et al. 2002). In qualitative data analysis, these aspects are often intertwined; however, reliability is about consistency and dependability (Morse 2015; Given 2008), and validity is about credibility and appropriateness of the process and findings in the context of the research objectives and questions (Patton 2002; Miles, Huberman and Saldana 2013). Consistency in the approach and analysis process is demonstrated in qualitative research with careful attention to documentation and transparency and the extent to which the coding and findings can be replicated by others who are familiar with the data. In the present study, the team recorded and detailed all aspects of the data collection and analysis processes shown in Figure 3 and worked to reach consensus by discussing and reconciling codes to support consistency and reproducibility of the coding schema. Using source, stakeholder, analyst, and theory/perspective triangulation is a common approach to establishing validity in qualitative research (Patton 2002; Miles, Huberman, and Saldana 2013). We shared interim findings with a selection of respondents to assess the extent to which the analysis and findings reflect their understanding of the issue. Further, the interim and final codes were reviewed and assessed by members of the research group who were not part of the coding team.

What motivates organizational members to engage in Smoke Sense?

Respondents’ experiences translated into different motivations depending on whether they were discussing their motivations as individuals, as members of their organization, or as members of their broader community. Partly driving respondents’ interest in Smoke Sense from a personal perspective was concern about how smoke affects their loved ones. The potential for the app to be a new resource, coupled with their expectation that wildfire smoke will continue to occur in the future, heightened their interest.

“As a parent of two young children, smoke is something that I was more acutely aware of than I may have been otherwise in terms of my personal experience.” State Health Agency Member

Another way that respondents talked about their motivations was in the context of their professional roles as members of organizations that respond to smoke. A variety of public and private organizations at the local, state, and national levels work to share information with individual and organizational stakeholders during wildfire smoke events. Respondents emphasized how members of these organizations work beyond the scope of their own discipline or area of expertise to access information to better understand the situation. Frequently this information is used as part of their own organization’s communication messaging that is shared with community stakeholders and partner agencies. The content of this outreach communication typically focuses on the status of the smoke situation, potential health impacts, the actions that individuals and organizations should take, impacts on scheduled events (including school and other formal or informal events), and the expected duration. One popular way that this information is communicated is through smoke blogs.

“People want to know what the forecast looks like, how the smoke is going to impact their health, whether to cancel outdoor events like soccer or football practices, and basically how to protect themselves and how long they’re going to be in that protection mode.” State Environmental Agency Member

“We try to have coordinated communication effort for the public [that includes the recommendation] to go to the blog for general smoke information including current levels of smoke, forecasts, and all kinds of other information about the wildfire, the health effects from smoke, and things people can do to protect themselves. … I’d say the smoke blog is our main source of multiple agencies contributing to the communication effort.” State Environmental Agency Member

Interviewees reported that their interest and motivation to participate in Smoke Sense grew out of their responsibility to be aware of what resources are available in the area of wildfire smoke and health. As professionals who work to collect, assess, interpret, and share information, there was a sense of responsibility for being aware of available resources and tools.

“I do a lot of the statewide coordination work and the Smoke Sense app is in my workhouse. So I have an obligation to know about what’s going on, what’s available out there in terms of apps and information about smoke, and what other agencies are working on.” State Environmental Agency Member

Respondents were members of organizations that have a responsibility for communicating to the public and partner agencies about air quality and health. Smoke Sense seemed like a tool that could assist with their existing professional communication responsibilities. Respondents perceived the app as a new resource that could potentially support or complement their own efforts to protect public health.

“[The project] was a really natural fit with my job. Wildfire smoke is a big air quality concern. I’m the only air quality person at the department of health, so it’s a big part of what I do. Even if there hadn’t been a statewide event, I was already looking for ways to improve our health communication.” State Health Agency Member

The Smoke Sense project being facilitated through a mobile application was another factor that garnered interest and engagement among respondents. Interviewees were particularly interested in learning more about the app because it delivers information directly to stakeholders’ phones, which was a communication pathway that they recognized as important but had not yet leveraged in their own organizations.

“For years I have been expressing the need for our communication to be usable on smartphone interface for the public.” State Environmental Agency Member

In addition to discussing motivations from personal and professional roles, interviewees also discussed their motivations in the context of their roles as a community member. Respondents noted the citizen science project was an opportunity to increase community engagement and incorporate their community’s experiences into a broader conversation about smoke and public health. Others were motivated by a curiosity about what the app could possibly do for community members.

“[I wanted to] make sure tribal voices are heard.” Tribal Environmental Agency Member

“I was curious about the app, and what it could do for citizens.” State Environmental Agency Member

What Smoke Sense could mean for individuals, organizations, and communities

The second section of the interview focused on respondents’ perceptions about what citizen science projects such as Smoke Sense could mean for their communities. At the bottom of Figure 4 are the themes that we identified across respondents’ responses with respect to what they anticipated would be an outcome of engaging with Smoke Sense. These themes were discussed in terms of the impact that engagement in the project could have for individual users of the app, the organizations that respondents were members of, and their community at large.

Respondents anticipated that the app will help individual users make a connection between wildfire smoke, air quality, and personal health, which are topics perceived by respondents as separate in the minds of many community members. Interviewees expect this to be achieved by users’ interactions with the app and by answering questions about smoke reports and concurrent health symptoms. By viewing questions on both topics in close proximity in the same app environment, respondents suggested that app users may begin making stronger mental associations between the concepts of air quality and health.

“Asking questions about peoples’ experiences with smoke and their health effects, puts [the relationship between those] on their mind, [they may start] thinking ‘oh these things are related; there’s a reason they’re asking.’ The questions become a little bit of a tool for indicating something we think is important.” -State Health Agency Member

“Linking air quality and health on the app helps people make that connection in their head. … I think the app has a lot of potential to connect those topics for them. They might have attributed the health effects to something else and not really paid attention to their smoke exposure, so that’s part of the opportunity here.” Local Government Representative

“We put out quite a bit of information about air quality all year round but especially during wildfire season. And we’ve been building up that level of awareness [in our community] for a long time. I think one of the reasons I like the app is because it directly ties the air quality to the effects on the people. A lot of people wouldn’t even tie those together unless they were prompted to, so I think the app is a really good tool to get that done.” Tribal Environmental Agency Member

On the organizational level, the app was thought of as a tool that would support organizations’ communication efforts. One way that the app can help organizations is by serving as a trusted source of information to which they could direct their stakeholders as part of their organization’s communication work. Respondents also emphasized the inherently complex nature of air quality data. Numerous sources of air quality information and a wide range of indices are available to understand current and forecast air quality conditions and associated potential health risks. The complexity of available information coupled with emergent air quality events, such as inundations with wildfire smoke, can make communicating about the issue a challenge. Help with communication is particularly important during intense smoke events because organizational bandwidth is at a premium.

“Smoke was a topic of people’s concern and frustration, especially from parents, because they would be downloading a bunch of different apps and they all use a slightly different algorithm. So they’d have to ask themselves, ‘Which apps do I believe?’ There is confusion as to what air quality numbers are out there, and what they really mean.” State Environmental Agency Member

“I think for people it can be a lot [to process] when you go to DEQ site and they have two numbers, then you go to AirNow and you get a third number. And you can get the NowCast number which can be done for any point in the day and it’s like ‘Gosh what should I do?’” Local Government Representative

“I think Smoke Sense is another great tool to communicate air quality. That’s really one of the major overriding factors of my interest in the app, is that besides the data gathering, it’s an easy way to get people aware of and to know about air quality.” Tribal Environmental Agency Member

“Making the information available to the public in real time and at their fingertips will help with the amount of pressure and requests that come in to all the different agencies. We don’t have capacity to be on the phone all day long with everybody, answering all their specific questions. Almost everybody has access to a smartphone these days and it is pretty consistent information they’re looking for.” State Environmental Agency Member

Respondents also discussed the need for their organization to effectively serve sensitive and at-risk populations during large-scale smoke events, and the potential for Smoke Sense to help those efforts. For example, people who are young, low-income, and even visitors to the area may not be prepared for smoke intrusion or may have limited awareness about and access to resources that can help them reduce exposure. Particularly challenging is the situation where information is needed to make a decision to act on behalf of others. Respondents frequently receive requests for assistance from administrators in schools and other youth-serving organizations. These requests are for help interpreting information about risks in order to make a decision about canceling or hosting events, including school. Children are identified as a sensitive population with respect to increased risk of adverse health outcomes associated with air pollution, so the administrators who are making the decision that affects the students’ exposures may especially benefit from enhanced and targeted outreach.

“In September when school started, the schools were really struggling with having smoke in the community and having kids in classrooms and at athletic events. I spend a lot of time talking to athletic directors trying to coach them on understanding air quality data, and they’re also getting direction from the state athletic association. And all the numbers are very confusing, even to me and I do a lot of research and have been watching it for a while with our prescribed burn program. I still get confused as to the differing numbers. So that part with schools is still a big challenge.” Local Government Representative

“I see the need for our health district and our agency to meet with the school district and talk about what their policy is for smoke. Hopefully the health district can provide a little bit more input in terms of the orange versus red decision.” Local Environmental Agency Member

“There is some individual who has to make a decision whether to keep the school open during a wildfire.… You have individuals trying to do the best they can with what information they have … and to me that’s a ripe audience [for you] to say, ‘Hey, can we help you out here?’” -State University Member

In addition to the benefits that individual users and organizations responding to smoke may realize, respondents also discussed the potential for their overall community to benefit from a citizen science project like Smoke Sense. Benefits to the community arise in part from the project’s potential for generating novel scientific knowledge in a broad sense, by collecting data that are not currently available for research in the area of wildfire smoke, air quality, and public health.

“We’re a data driven agency and we’ve got people whose job [is] looking for the most updated peer-reviewed literature on this topic specifically, and there’s not a lot out there.” State Environmental Agency Member

“[Through this project] we can actually get that tie between concentration levels and what people’s health effects were.” Tribal Environmental Agency Member

“I think it is a great tool that we can use to understand some of the clinical effects that we don’t really have great research about in terms of the health literature.” State Health Agency Member

“We are doing a community survey to assess the community’s health and trauma in the aftermath of our wildfire event, but that survey is a one-time survey. We’ll gather that data and then report on it.… If you could have this continuous feed of data, it would be a lot more informative and robust, wouldn’t it be great to be ready and to be collecting data using Smoke Sense as soon as the event starts.” Local Health Agency Member

A second community benefit is the possibility for organizations and communities to access geographically relevant data, submitted from users in the area that respondents’ organizations served. The plan for these data could include use in local response, decision-making, planning, research, and outreach initiatives. Respondents saw the potential to combine community-specific data from the Smoke Sense app with other local or state air quality information to support community health. Another example was using the Smoke Sense data to understand baselines and trends in levels of exposures, awareness, and engagement in protective health behaviors of community members. Having that locally relevant data would support their communication and outreach efforts as well as potentially be useful in advancing programs and other initiatives.

“We can use that information as leverage for grants and expanding the programs over here.… I also like the idea of using local results in combination with other surveys and information I already have about air quality.” Tribal Environmental Agency Member

“Having that data would help us to know the basic level of awareness and where we need to go in terms of increasing awareness. We need to have some baseline information so we can see where we started and how are we doing as far as disseminating that information. Having the data would also help us understand if people actually adopt these protective behaviors, which protective behaviors are they adopting, and how long are they persisting in those protective behaviors in at times when the air quality is bad.” State University Member

Findings are summarized in Table 1.

Implications for citizen science

Citizen science research has clear potential to advance progress in addressing complex social problems. Citizen science projects that address complex social issues may have increased impact by including in the research design scientific objectives as well as educational and action objectives. Wiggins and Crowston’s (2011) typology of citizen science project objectives is a useful starting point for considering citizen science design such that the project might effectively address a complex social problem. The Smoke Sense app combines educational, investigative, and virtual engagement objectives. This combination was an important part of these respondents’ perceptions for broader engagement with Smoke Sense.

Citizen science projects that aim to address issues in complex problem domains should be designed to complement collective efforts among organizations working with a system. A design that is complementary to broader efforts is especially important for federally led projects in order to build increased cooperation among federal, state, local, and tribal partners. Further, there are often a range of organizations working within any given problem domain, and compared to federal agencies, these state, local, and tribal partners may have closer connections with affected populations. For example, in addition to federal environmental and health agencies there also are state, local, and tribal environmental and health agencies working to reduce the public health burden of wildfire smoke. By considering these broader efforts, federal citizen science projects can support the collective objectives across organizations and potentially attract the interest and participation in partners’ work. Smoke Sense provided additional information to the intended audiences of state, local, and tribal agencies that were working on the issue of wildfire smoke and public health, while also collecting information that could help shape future outreach by these agencies. We learned there is interest among partners to obtain and analyze data specific to their area and to potentially integrate Smoke Sense findings into their efforts. Thus, it is important to include structured pathways for continued interaction with partner agencies within a system, including sharing findings and data with individuals and organizations that are also working in the same area. Forethought should be given to how outcomes of federal citizen science projects might impact local, state, and tribal communities, and safeguards should be implemented to ensure that the projects do not inadvertently harm or make the issue more challenging for community organizations that are responding to an issue.

Implications for community response to wildfire smoke and health

There is growing evidence of the adverse health outcomes associated with exposure to particles found in wildfire smoke (Reid et al. 2016a; Reid et al. 2016b; Rappold et al. 2017; Rappold et al. 2011; Wettstein et al. 2018; Cascio 2018; Mirabelli et al. 2009). Increases in individual awareness of both the issue and recommended health behaviors influence the potential for increased engagement in recommended health protective behaviors. Citizen science projects like Smoke Sense can amplify health risk messaging about wildfire smoke by providing an additional platform for reaching individuals affected by smoke. Organizational representatives can use Smoke Sense as a tool to talk about complex social issues with the communities they serve. Additionally, citizen science projects may help staff in partner agencies, such as school administrators or community leaders, by collecting and disseminating data and frameworks that facilitate decision making with respect to mitigating community exposure to smoke from wildland fire.

Implications for Smoke Sense

Understanding the motivations of organizational participants can potentially increase the impact of Smoke Sense by informing plans for how the citizen science project can evolve to better meet collective objectives. One example of how an increased understanding of organizational participants’ motivations and needs is influencing the evolution of the Smoke Sense project is the development of an online data visualization tool. Through these interviews we learned of the importance of making geographically relevant data available to partner organizations working to address the health risks of exposure to wildland fire smoke. The Smoke Sense Data Visualization Tool allows for near real-time access and exploration of the anonymous data submitted through the Smoke Sense app. Through the Smoke Sense Data Visualization Tool, data can be stratified and downloaded, and visualizations of the data can be saved for use by others. Another way that an increased understanding of organizational participants’ motivations is impacting the Smoke Sense project is through new release versions of the app. For example, the smoke forecasting feature was initially not part of the app during the pilot season, but forecasting was identified by partners as a primary topic of information requests. Forecasting was incorporated to provide information that can help individuals better plan their activities during smoky conditions. Understanding partner needs, motivations, and expectations is critical to fostering change in complex problem domains. The Smoke Sense citizen science project team is working to better understand organizational partners and subsequently translate increased understanding into the citizen science project design such that the app can more effectively support collective change in complex problems.

Limitations

We selected participants who represent public organizations at the intersection of environment and health and who engaged with Smoke Sense in the pilot year of the project. Thus, findings are drawn from a limited set of semi-structured interviews and are not generalizable. Additionally, Smoke Sense is led by a federal agency, the US EPA, which could have affected respondents’ answers to interview questions because of a real or perceived power differential at the organizational level. However, this was not raised as a concern during any of the interviews.