Start Submission

Reading: Designing Equitable, Transparent, Community-engaged Disaster Research

Download

A- A+
Alt. Display

Case Studies

Designing Equitable, Transparent, Community-engaged Disaster Research

Authors:

Diana Rohlman ,

Oregon State University, US
X close

Samantha Samon,

Oregon State University, US
X close

Sarah Allan,

Oregon State University, US
X close

Michael Barton,

Oregon State University, US
X close

Holly Dixon,

Oregon State University, US
X close

Christine Ghetu,

Oregon State University, US
X close

Lane Tidwell,

Oregon State University, US
X close

Peter Hoffman,

Oregon State University, US
X close

Abiodun Oluyomi,

Baylor College of Medicine, US
X close

Elaine Symanski,

Baylor College of Medicine, US
X close

Melissa Bondy,

Stanford University, US
X close

Kim Anderson

Oregon State University, US
X close

Abstract

Disaster research faces significant infrastructure challenges: regional and federal coordination, access to resources, and community collaboration. Disasters can lead to chemical exposures that potentially impact human health and cause concern in affected communities. Community-engaged research, which incorporates local knowledge and voices, is well suited for work with communities that experience impacts of environmental exposures following disasters. We present three examples of community-engaged disaster research (CEnDR) following oil spills, hurricanes, and wildfires, and their impact on long-term social, physical, and technical community infrastructure. We highlight the following CEnDR structures: researcher/community networks; convenient research tools; adaptable data collection modalities for equitable access; and return of data.

How to Cite: Rohlman, D., Samon, S., Allan, S., Barton, M., Dixon, H., Ghetu, C., Tidwell, L., Hoffman, P., Oluyomi, A., Symanski, E., Bondy, M. and Anderson, K., 2022. Designing Equitable, Transparent, Community-engaged Disaster Research. Citizen Science: Theory and Practice, 7(1), p.22. DOI: http://doi.org/10.5334/cstp.443
197
Views
62
Downloads
1
Citations
1
Twitter
  Published on 19 May 2022
 Accepted on 26 Jan 2022            Submitted on 14 May 2021

Introduction

Environmental disasters in the United States are increasing in frequency and severity (Smith 2020). Disasters encompass geographic, structural, and social events contributing to or causing adverse human health effects, economic loss and environmental impacts (Chmutina and Von Meding 2019). Research is needed to address community concerns regarding potential impacts to human and environmental health, following the life-saving efforts often necessitated by a disaster (Miller et al. 2016). Characterizing chemical exposures is one such concern, as highlighted by the Deepwater Horizon oil spill (Gulf of Mexico), Hurricane Harvey (Texas), and recent wildfires in the Northwest (NW) United States. During Hurricane Harvey, there were 89 reported releases of air toxics, yet it was unclear which chemical contaminants were released, or in what quantities (Miller and Craft 2018). Additionally, there continue to be significant gaps in knowledge regarding the composition and subsequent health impacts of wildfire smoke in the Northwest (Aguilera et al. 2021). Disaster research aims to generate knowledge that will help populations support real-time response decisions during disasters, understand outcomes and recovery, and prepare for future disasters.

Disaster research has many challenges (Miller et al., 2016). Traditional research methodologies, including recruitment, retention, data collection, and data analysis, may be ill suited for disaster research (Bongers et al. 2008; Knack et al. 2006; Henderson et al. 2009). Disasters that involve potential chemical exposure require characterization of the type and quantity of chemicals in environmental media, yet a lack of baseline data complicates analysis (Knack et al. 2006; Lurie et al. 2013). Most exposure assessment tools rely on electricity, the internet, or trained data collection personnel. Biomonitoring technologies capable of detecting chemical analytes in urine or blood rely on physical infrastructure commonly impacted by disasters, such as controlled locations to collect and store samples. The digital collection of questionnaire or medical history data is similarly hampered (Henderson et al. 2009), especially when reliant on participant tracking and retention (Knack et al. 2006). There is a shared similarity amongst many of these challenges; they are dependent on pre-existing critical infrastructure. The challenges faced by disaster research are those faced on a much larger scale by communities dependent on these socio-technical systems (Choi et al. 2019). As such, disaster research must be carefully situated in this space to prevent additional demands on stressed or failing critical infrastructure (Choi et al. 2019).

Passive sampling is well-suited to disaster research in which characterization of chemical exposure is desired, as the sampler mimics biological membranes, thereby allowing analysis of bioavailable contaminants from air, water, soil, or sediment (Dixon et al. 2020; Anderson and Hillwalker 2008). The samplers do not require electricity, are low cost and low maintenance, are capable of assessing a breadth of chemicals (>1,530) (Bergmann et al. 2018), and can be configured as environmental or personal samplers (O’Connell et al. 2014). The samplers can be integrated with existing technological infrastructure, such as the NOAA Hazard Mapping System (Schroeder et al. 2008), questionnaire and biological data (Rohlman et al. 2019a), and US Census data for describing neighborhood socioeconomic status (Oluyomi et al. 2021) to provide a comprehensive sampling strategy following a disaster.

Beyond collecting chemical exposure data, obtaining consensus among researchers, communities, and regulatory agencies on which contaminants to evaluate is difficult (Reams et al. 2017). Identifying the research issues and subsequent prioritization requires coordination between multiple stakeholders, and should include community members (Miller et al. 2016). Following the Gulf of Mexico oil spill, the National Institute of Environmental Health Sciences (NIEHS) developed the Disaster Research Response (DR2) platform in 2012. The platform collates disaster-related materials and protocols, resources, and research tools; addresses issues of coordination between researchers, agencies, and public health; and collects timely health data following a disaster (Miller et al. 2016). This program also conducts trainings with first responders, regional and federal agencies, and researchers in environmental health (Miller et al. 2016). Efforts to increase coordination amongst researchers and their community partners have resulted in substantial changes to the ways in which disaster research is conducted, such as the implementation of a single IRB and standardized participant registries (Packenham et al. 2017; Collogan et al. 2004; Symanski et al. 2021).

Given the inherent unpredictability of disasters, it is a challenge for researchers to collect baseline data, access knowledge regarding the extent of the disaster, and select study sites and participants (Bongers et al. 2008; Henderson et al. 2009; Knack et al. 2006). To address these compounded uncertainties, the inclusion of community voice enhances DR2 efforts (Miller et al. 2016; Errett et al. 2019). Research methodologies that engage communities for this purpose include citizen science (CS), community-engaged research (CEnR), and community-based participatory research (CBPR). CS has been defined as the participation of non-scientists in projects with public interest (engagement), yet where the main question or study design originated from academics (Woolley et al. 2016; O’Fallon and Finn 2015). CEnR is defined as research driven by academic interest and questions, yet engages the community in aspects of this endeavor, such as providing input on the study design or selection of the data collection tool (O’Fallon and Finn 2015). CBPR is situated within community knowledge and strengths, and uses a collaborative partnership model to co-develop research that addresses community concerns (Israel et al. 1998). Wooley et al. (2016) developed a framework for differentiating between these forms of engaged research (Figure 1) that parsed elements of engagement, involvement, and participation, with participation defined as that beyond a study subject. Whereas CBPR encompasses aspects of engagement, participation, and involvement (Israel et al. 1998), CS efforts rely on engagement around the research topic to increase participation in data collection. CEnR includes at least two elements of this framework (Figure 1). These research methodologies are designed to promote equitable research practices by prioritizing community priorities, needs, and strengths (Wallerstein et al. 2019). Indeed, CPBR focuses explicitly on the ways in which partnerships are formed and power is distributed (Wallerstein et al. 2019). For example, researchers and community members participate equally in the process, whereas CS and CEnR may be more heavily weighted toward the academic or community partner on aspects of research priorities, governance, data sharing, and communication (Cooper et al. 2021; Guerrini et al. 2021; Andress et al. 2020).

Venn diagram of intersections between citizen science (CS), community-engaged research
Figure 1 

Venn diagram of intersections between citizen science (CS), community-engaged research (CEnR), community-engaged disaster research (CEnDR), and community-based participatory research (CBPR) aligned with elements of engagement, involvement, and participation, as described by Wooley et al (2016).

DR2 incorporates CS, CEnR, and CBPR to various extents. The benefits of CEnR are well catalogued (Israel et al. 1998). This framework has been applied to disaster planning (Wells et al. 2013) and disaster research (Miller et al. 2016). CBPR is an important, yet time-consuming undertaking (Israel et al. 1998), and thus difficult to implement following an unexpected disaster, wherein community partnerships may not yet exist. CS has been successfully used following disasters (Hicks et al. 2019). Studies that utilize forms of CS suggest that datasets developed by CS can inform disaster response and preparedness (Chari et al. 2019; Hicks et al. 2019). However, CEnR approaches appear to be well suited for conducting research during and after disasters, herein termed community-engaged disaster research (CEnDR). In this approach, community concerns drive research efforts (engagement), community context can account for existing or potential inequities and inform data methodology and collection tools (involvement), and community members may aid in the collection of data (participation). Relative to CEnR, CEnDR is subject to additional guidelines for working with impacted populations. These guidelines recognize the impact a disaster may have on infrastructure, to include reduced access to resources (medical facilities, clean water, food, transportation, school, and housing, among others), impacts on local researchers, and potential for research overload (Collogan et al. 2004; Ausbrooks et al. 2009).

Here, we discuss three case studies to describe the challenges of disaster research and lessons learned utilizing the CEnDR framework following these discrete disaster events.

Description of Case Studies

The following describes the subsequent research efforts to evaluate chemical exposures following the Deepwater Horizon oil spill (Gulf of Mexico), Hurricane Harvey (Texas), and wildfires (NW United States). Each study highlights various aspects of CEnDR, from informing study design to training community members to collect data (Table 1), along with the associated infrastructural challenges (Table 2). In all studies we prioritized the safety and well-being of participants and clarified that research participation was voluntary.

Table 1

Description of case studies, community concerns following the disaster, level of community-engaged disaster research (CEnDR), and description of the research methodology and tools used.


DEEPWATER HORIZON GULF OF MEXICO OIL SPILL (MC252). HURRICANE HARVEY NORTHWEST WILDFIRES

Disaster type Chemical disaster – oil spill Hurricane-caused flooding Wildfire

Date of disaster April 20 – September 19, 2010 August 17 – September 2, 2017 June – October, annually

Description of disaster An industrial disaster on the Deepwater Horizon resulted in an estimated 4.9 million barrels of oil spilled from the Mississippi Canyon 252 (MC252) well over 87 days. Researchers collected pre-oiling samples as it took weeks for oil to reach the beaches. Rainfall from Hurricane Harvey resulted in up to 60 inches of rainfall in 4 days, leading to catastrophic flooding (Oluyomi et al. 2021). Thirteen Superfund sites flooded in Houston, TX (EPA 2017). Wildfires are increasing in size, and subsequent smoke impact is leading to acute health impacts (Aguilera et al. 2021). Wildfire smoke composition varies between fires (type of wood burned, type of structures burned).

Geographic extent (US) Gulf of Mexico and the coasts of Louisiana, Alabama, Mississippi, and Florida Texas, Louisiana Fires located in Washington, Idaho, Oregon, California. Smoke has travelled across the US and into Canada.

Community concerns (research issue) Impacts to human health, marine life, economy.
Concern regarding exposure to oil and chemical dispersants via contaminated air and water, local seafood, use of public and private beaches, residual oil, and potential for re-exposure due to hurricanes unearthing buried oil near the shore (Allan et al. 2012).
Exposure to air toxics following emergency shut-down and start-up procedures at petrochemical facilities and the subsequent release of benzene and other chemicals (Miller and Craft 2018).
Concern regarding exposure to chemicals from the 13 flooded Superfund sites
Concern regarding exposure to wildfire smoke from air or deposition on vegetables
Chemical composition of wildfire smoke and impacts on human health
Behaviors that would reduce exposure to wildfire smoke.

Level of CEnDR Involvement
community provided input on study design and sampling sites and helped maintain security of samplers
Provided local knowledge about relevant activities near sampling sites
Engagement
community remained engaged through research dissemination and scientific outreach at a workshop
Involvement
community involved in recruitment efforts and input regarding report-back efforts (focus groups)
Engagement
Ongoing research dissemination and forums
Involvement
Involved in the research issue identification
Participation
Community members set up samplers and collected samples before, during, and after wildfires.

Participant retention rate Not applicable Retention across two time points = 57%. Retention across three time points (3 years) = 62%.

Research methodology Environmental sampling conducted before, during, and after shoreline oiling.
Chemical analytes: Polycyclic aromatic hydrocarbons.
Personal sampling conducted during and one year after flooding.
Chemical analytes: Polycyclic aromatic hydrocarbons, flame retardants, polychlorinated biphenyls, pesticides, dioxins and furans, endocrine disruptors, personal care products, industrial, and pharmaceutical chemicals.
Secondary data: Area deprivation index; proximity to toxic waste sites.
Environmental sampling and questionnaires conducted before, during, and after wildfires.
Chemical analytes: Polycyclic aromatic hydrocarbons.
Secondary data: PM2.5, NOAA Hazard Mapping data.

Research tool(s) Environmental passive samplers. Passive sampling wristbands; questionnaires. Environmental passive samplers; questionnaires.

Duration of study May 2010 – May 2011 August 2017 – present 2018 – 2020

Table 2

Challenges and mitigations associated with community-engaged disaster research (CEnDR).


INFRASTRUCTURE CHALLENGE MITIGATION

ALL DISASTERS

Physical Geographic constraints due to distance from disaster or physical damage caused by disaster; researchers impacted by disaster. Use of NIH-funded centers and networks to identify collaborators and appropriate research tools.

Technological Coordination amongst scientists, regional and federal agencies, community organizations. Development of a research dashboard to reduce participant burden.

Social Funding for timely research responses that includes CEnDR. Include disaster research as a theme in currently funded research programs and Centers

Timely ethical review for research studies; coordination between institutions Pre-positioned disaster IRB

Context. Historical context; culture; existing disparities Research designed with and by communities

Lack of central information repository for all studies relating to the disaster Increased collaboration to share tools and reduce burden on communities

Multiple Methods of mitigating exposures specific to the unique disaster that are relevant, economic, feasible and appropriate. Present multiple mitigation activities, cognizant of regional and cultural context, financial burden, availability, and housing status (rent/own/other).

GULF OF MEXICO OIL SPILL

Physical Access to study sites; location of study sites; site security Work with Federal and State agencies to obtain appropriate permitting.

Social Study site selection Work with existing community groups to identify appropriate sampling locations.

Community context. Given the nature of the disaster, tensions were high. Researchers set clear expectations with the study (source of funding, goal of research, communication of results).

Inability to collect personal samples. Developed a pre-positioned disaster IRB.

HURRICANE HARVEY

Physical Access to enrollment sites. Worked with community partners to select accessible, convenient recruitment and enrollment locations.

Long lines to complete enrollment. Tiered sign-up times; self-sign up.

Technological Data collection reliant on Wi-Fi. Utilize mobile hotspots or survey software capable of off-line performance.

Social Participant retention and long-term communication. Conduct widespread recruitment across web, newspapers, radio, and community partners to communicate upcoming study opportunities.
Provide routine updates via email, telephone, and community forums.

Community context (literacy, distrust, historically underrepresented communities). Conduct focus groups to determine community concerns and methods of report-back.

NORTHWEST WILDFIRES

Physical Shipping delays due to wildfires and evacuations. Mail kits prior to wildfire season

Sample preparation. Severe wildfire smoke contaminated the laboratory, preventing new kit assembly. Prepare samplers prior to a disaster.

Technological Real-time identification and characterization of wildfires across multiple states Utilized publicly available databases and participant feedback on wildfire incidence and severity

Internet and Telephone capabilities; rural areas impacted by wildfires had decreased internet and cell phone capability, decreasing communication especially during wildfires. Verbal completion of questionnaires during limited Internet
Written and photographic instructions provided with kits

Training materials Video, written and photographic instructions provided with kits

Social Influence of personal and community behaviors on air quality. Post-study questionnaire captures co-variate data.

Participant retention Rolling recruitment for new participants; flexible deployment dates to accommodate participant schedules.

Deepwater Horizon Oil Spill, Gulf of Mexico

On April 20, 2010, an industrial disaster on the Deepwater Horizon drilling rig in the Gulf of Mexico led to approximately 4.9 million barrels of oil spilled over nearly five months (Table 1). The oil spill impacted multiple communities and the economy, including recreational and commercial fishing, shrimping, and tourism (Lichtveld et al. 2016; Sumaila et al.; 2012). Following the disaster, we deployed to four states in the Gulf of Mexico (Louisiana, Alabama, Mississippi, and Florida). Here, community members were involved in study design, helped identify sites to deploy samplers, provided valuable logistical support, and remained engaged in monitoring and protecting the samplers over the course of multiple deployments. They also relayed important contextual information for interpreting environmental data, such as shoreline clean-up activities that occurred when researchers were not present. They remained engaged with the study via regular interactions with the researchers, study updates, and a workshop (Table 1, Figure 2).

General timeline of community-engaged disaster research (CEnDR) case studies
Figure 2 

General timeline of community-engaged disaster research (CEnDR) case studies. The disaster duration refers to the initial event and does not encompass disaster recovery. Icons indicate areas of community engagement (forums, outreach via research dissemination), participation (focus groups, data collection), and involvement (community input on study design/methodology).

We set up environmental passive samplers (air, water) at shoreline sites within each state before oil from the spill reached the coastline, ensuring baseline data. Coordination with local and federal agencies was necessary to obtain permits for placing samplers in coastal waters. Sampling continued for a year, for a total of nine sampling events (Allan et al. 2012).

Here, we collected environmental baseline data, yet no personal samples were collected because of the lack of a pre-positioned institutional review board (IRB) protocol. However, by responding immediately using environmental samplers, the project documented pre-, during-, and post-spill oil chemical concentrations in nearshore environments in the Gulf of Mexico. These data have been widely utilized to help understand the impacts of the Deepwater Horizon oil spill.

The disaster response leveraged existing infrastructure such as organized community groups that categorized concerns, as well as a sampling technology that did not rely on disaster-impacted infrastructure. Based on lessons learned from this disaster research, the team initiated a pre-positioned disaster IRB for use in future disasters to allow collection of personal samples, and to improve opportunities for community involvement and participation (Rohlman et al. 2020).

Hurricane Harvey (Houston, TX)

Hurricane Harvey, a category 4 hurricane, made landfall in Texas on August 25, 2017 (Table 1). The subsequent rainfall caused catastrophic flooding across the greater Houston area. Within three weeks, we initiated the collection of personal chemical exposure data by leveraging a pre-positioned IRB and collaborations between three academic institutions: Oregon State University (OSU), The University of Texas Health Science Center at Houston (UTHealth), and Baylor College of Medicine (BCM). UTHealth and BCM had existing community and governmental partnerships, enabling rapid community engagement and involvement (Symanski et al. 2021).

We integrated recruitment efforts with existing relief efforts by enrolling participants at community centers and churches identified by community partners. Houston-area neighborhoods were prioritized, based on the preponderance of flood-damaged properties (Oluyomi et al. 2021). Community partners were involved in recruitment efforts and engaged with the study via study updates and the study website (Figure 2). Participants wore a silicone wristband for seven days and filled out a questionnaire. A second data collection occurred 12–14 months later to obtain post-disaster data.

All participants had the option to receive their wristband data, and all requested this option, indicating high engagement. Focus groups provided an opportunity for community members to be involved in report development and to ensure that data was placed within the social and historical context of the community. This increased level of involvement ensured reports were specific to community questions and concerns. For example, rather than displaying aggregate data, the focus group participants preferred viewing data presented by neighborhood in recognition of existing disparities in socioeconomic status.

The study struggled to maintain long-term engagement with study participants, potentially owing to the permanent displacement of residents from their homes following the flood (Table 2). Disaster-impacted infrastructure posed an additional challenge. Initially, a portion of data collection required Wi-Fi, which was frequently unavailable at recruitment sites (Table 2). Inter-university partnerships, existing partnerships with community groups, and willingness to share resources and tools substantially strengthened the study.

Through the focus groups, we learned that additional information around exposure mitigation and risk reduction was needed and should be communicated alongside the return of data. We have followed the recommendation to include mitigation and reduction information in this and subsequent studies.

Northwest Wildfires

Wildfires in the NW have increased in size over the past decade (Halofsky et al. 2020). Starting in 2017, OSU researchers held informal conversations with community members and the OSU Extension service to characterize environmental health concerns (Table 1). To evaluate chemical exposure before, during, and after wildfires in ambient outdoor and indoor air at residential locations over three years (Table 1), we used environmental passive samplers and questionnaires. The goal was to generate data to inform public health advisories related to smoke exposure and improve individual recommendations for smoke-impacted individuals.

Using a convenience sampling approach, we identified community members across four states (Washington, Oregon, California, and Idaho) based on their proximity to historically wildfire-impacted areas. Volunteers completed self-paced training with videos and written instructions and worked with researchers to identify sampling locations and deploy samplers. Briefly, we asked participants to set up two passive air sampling monitors (indoor and outdoor) for up to three weeks, and complete a questionnaire (description of study location and characterization of individual and community behaviors as related to smoke exposure). All participants accepted the offer to receive the data from their location.

After each sampling period, results were returned to study participants using a template previously developed with communities (Rohlman et al. 2019b). The results highlighted contaminants associated with wildfires, and demonstrated differences between indoor and outdoor air before, during, and after wildfires. Following lessons learned from Hurricane Harvey to include additional resources, we included information about chemicals found in wildfire smoke and recommendations for improving indoor air quality.

This study significantly benefited from community participation given the geographic extent of the fires. Additionally, the study was designed from a disaster preparedness standpoint with samples collected before the disaster occurred, and thus benefited from the time allowed to plan the study, obtain an IRB, train participants, and collect baseline data. Despite applying lessons learned from previous CEnDR approaches, we identified new issues. Independent of the fires, rural participants had limited internet access, and therefore completed questionnaires over the telephone, although this service was also unreliable (Table 2). During fires, underlying infrastructure became heavily impacted. The postal service was unable to deliver samplers in areas under certain evacuation levels, and smoke contaminated the research laboratory itself (Table 2). The use of CEnDR alleviated some of these challenges, for example by sending samplers to community members prior to a disaster. Furthermore, many participants called or texted to report wildfire conditions within their area, often before official databases captured these conditions (Table 2).

Summary of disaster impacts on infrastructure and lessons learned

Disaster research is complicated by the unpredictable timing, the locations, and the severity inherent to disasters. As outlined in Table 2, there are additional challenges to disaster research. Here, we share lessons learned from conducting CEnDR following three disasters within the United States. As shown in Figure 2, community engagement, involvement, and participation were unique to the study design. While the three studies differed in disaster type, location, and approach (Table 1), they can inform a general sequence for CEnDR approaches (Figure 3). Researchers can prepare by developing training, IRB protocols, and tools that are well suited to disasters. While community partnerships may not exist prior to a disaster, principles of CEnDR may facilitate new partnerships and guide development of study designs, sampling approaches, and return of data. Finally, the return of data to individuals and communities can increase transparency and apply study results to community concerns.

General sequence for community-engaged disaster research (CEnDR). Community input
Figure 3 

General sequence for community-engaged disaster research (CEnDR). Community input and knowledge is integrated at study initiation, with transparent data sharing options. While depicted as linear steps, CEnDR exists on a continuum, with responses addressing the immediate disaster through to disaster recovery and preparedness for future disasters.

Following identification of the challenges and opportunities posed by infrastructure before and during disasters, we developed a series of comprehensive lessons learned (Table 3). These lessons extend beyond the individual case studies, which focused on chemical disasters. Where possible, we have included specific examples, taken from researchers’ own accounts, to demonstrate the value of these recommendations for future CEnDR.

Table 3

Lessons learned following review of challenges and mitigations specific to the three case studies. Community-engaged disaster research: CEnDR.


LESSONS LEARNED

  • Prepare for the unpredictable
  • Cross-coordination across agencies, institutions, non-profits, and communities is essential
  • Ensure relevance, transparency, and equitable participation
  • Study design and data collection methodologies must be adaptable to the physical constraints of the location
  • Identify or plan for baseline data collection
  • Anticipate and mitigate research fatigue
  • Integrate return of data into CEnDR studies
  • Provide exposure reduction resources within the context of the disaster
  • Identify timely funding for disaster research

Prepare for the unpredictable

Preparation for disasters and CEnDR is critical for a timely, informed response. Such preparation includes developing a pre-positioned IRB to enable collection of personal chemical exposure measures and other data, training staff, preparing materials for training community members to self-collect data, and preparing collection tools in advance of potential disasters (Figure 3).

Prior to initiating CEnDR, ethical review of study activities may be necessary. Unfortunately, the timeline of IRB review is not often compatible with disaster research and community expectations (Lurie et al. 2013). This was a challenge faced by researchers during the Deepwater Horizon oil spill, with many studies delayed 10 months (Miller et al. 2016). There are ongoing efforts to increase the prevalence of pre-positioned IRBs that will mitigate this difficulty (Packenham et al. 2017). This leads to additional considerations, such as how to set up such protocols across multiple institutions. Following Hurricane Harvey, collaborating institutions leveraged existing protocols, tools, and methodologies to respond to the disaster within three weeks (Oluyomi et al. 2021).

The occurrence of seasonal wildfires in the Northwest are largely predictable. However, this allows researchers to recruit and train participants prior to fire outbreaks, using historical fire data to inform study site selection. Storing a repository of prepared samplers, and sending samplers before a disaster whenever possible, ensures that researchers are ready to respond quickly, irrespective of disaster impacts on researchers or researcher facilities. For example, smoke from the 2020 wildfire season contaminated the research laboratory itself, preventing sampler preparation. Previously stockpiled samplers were available to conduct the necessary collection of data at all study sites.

Cross-coordination across agencies, institutions, non-profits, and communities is essential

The benefit of cross-coordination during DR2, and associated challenges, has been previously discussed (Reams et al. 2017; Errett et al. 2019). An additional consideration around cross-coordination relates to the difficulties of researchers who are themselves impacted by the disaster. Sharing resources and leveraging existing community and governmental relationships with academic institutions can provide a strong internal infrastructure. As an example, during Hurricane Harvey, several of the researchers were impacted and dealing with flooded homes as they secured funding and designed the study. Relying on collaboration from other institutions enabled a robust research response despite these complications. Additionally, the DR2 program has built a repository of protocols, resources, and data collection tools (Miller et al. 2016), and research amongst NIEHS-funded centers has assessed the capacity of researchers to respond to disasters (Errett et al. 2019). However, cross-coordination between researchers and community members remains reliant on local, established relationships. A single, disaster-specific dashboard may enable such coordination with communities and with federal funding agencies.

Ensure relevance, transparency, and equitable participation

The strength of CEnDR is the integration of scientific knowledge and community-based knowledge. This intersection provides science-based data that is responsive to community concerns. Following the Deepwater Horizon oil spill, researchers selected sampling sites in consultation with community members. These consultations allowed for the inclusion of nuanced information regarding the way sites were used, their proximity to vehicle exhaust (boat and car), and ways individuals may be exposed to oil-contaminated water.

Furthermore, the inclusion of community voices and knowledge provides a strong platform for enhancing equitable participation via flexible study design and alternative methods of participation (i.e., verbal questionnaires in cases of limited internet). We found that community input on how to return data incorporates community knowledge and context, identifies equitable dissemination methods, and can ensure that reports represent multiple languages, as needed.

Study design and data collection methodologies must be adaptable to the physical constraints of the location

The physical impacts of disasters are wide-ranging and often unique to each disaster. However, common issues include difficulty accessing impacted sites and communities, and integration with existing relief efforts. In the case of Deepwater Horizon, researchers traveled across the country to collect and deploy samplers. The locations themselves were often challenging to access given beach closures due to oiling and finding appropriate structures upon which to attach the samplers (Table 2). Many physical challenges were mitigated as the sampling team grew familiar with the environment. Across all case studies, many challenges attributed to disaster research were mitigated given the selection of data collection tools and the ability to distribute and recover passive samplers by mail at ambient temperatures (Anderson et al. 2017).

Identify or plan for baseline data collection

Occasionally the nature of the disaster allows for collection of baseline data, as seen with the Deepwater Horizon oil spill and NW wildfires. In studies wherein researchers are unable to collect baseline data, coordination between researchers and communities may identify alternative datasets or build post-disaster sampling into the study design. This was the approach taken following Hurricane Harvey, which collected samples one year later for comparison purposes (Figure 2).

Anticipate and mitigate research fatigue

The Hurricane Harvey study relied upon repeated enrollment over one year. Enrollment suffered from low retention (57%; Table 1), potentially due to residential displacement from flooding, changes in contact information, or research fatigue. The NW wildfire study covered three years, with 62% of participants remaining engaged over that duration (Table 1). However, many participants tried to complete the annual questionnaire or sampler setup from memory, leading to errors, or a general feeling that the provided training was insufficient. Thus, future longitudinal studies should consider tiered training materials, recognizing that long-term participants may need different training or visually different training materials to re-engage.

Integrate return of data into CEnDR studies

Communities should be involved when returning individual- or community-level reports. One method is to hold focus groups to learn how best to return data to participants. For Hurricane Harvey, we learned more about concerns related to chemical exposure, preferences regarding data visualization modalities (text, bar charts, tables), and the way in which data people wanted to receive their data (email, mail, online). This helped us align the design of the report with community needs.

Returning data necessitated data sharing and collaboration between collaborating institutions, and integration of all activities under a single IRB protocol (Table 2). The return of data identified challenges within the administrative aspect of CEnDR. Specifically, the IRB process needs to be rapid and flexible enough to accommodate multiple institutions and projects, with the ability to share information quickly and confidentially between institutions to allow for timely reporting to study participants.

Provide exposure reduction resources within the context of the disaster

In the early stages of Hurricane Harvey, we shared resources specific to community concerns, such as cleaning up flood water and removing mold. We presented exposure reduction resources along a continuum, taking into consideration economic, cultural, and physical constraints. As the chemical exposure study progressed, we shifted to provide methods related to reducing individual chemical exposure, per community requests.

Identify timely funding for disaster research

We initiated each case study prior to securing funding, given the sudden onset of disasters. Opportunities for time-sensitive research funding exist yet are still on the order of months from grant submission to receipt of funds. Here, multiple NIEHS-funded Centers collaborated within and across institutions to respond to disasters. As we collected data, we identified and leveraged funding opportunities to analyze and return collected data.

Conclusions

Increased collaboration and coordination, such as that facilitated by the DR2 program, has begun to address the need for shared resources, data collection tools, and protocols. However, identifying the research issue and prioritizing health concerns remain challenges unique to each disaster. Miller et al (2016) noted the importance of community involvement in this process yet acknowledged there is no organized infrastructure to support this. Additionally, the unpredictable nature of disasters often means that responding researchers may not have existing partnerships with the impacted community. Collaboration between researchers can leverage community partnerships; however, in a survey of Environmental Health Sciences Core Center directors, while collaboration was listed as necessary to increase community engagement, relevance, and equitable participation, it raised concerns of outside researchers potentially damaging trusted community partnerships (Errett et al. 2019).

Wallerstein et al. (2019) identified multiple strategies underpinning successful, robust academic-community partnerships. These strategies included recognizing and including community strengths and knowledge, facilitating communication, and representing and addressing community-defined outcomes for the research (Wallerstein et al. 2019). These are strategies that mesh well with the CEnDR framework of prioritizing engagement, involvement, and participation, and can be adaptive to disaster scenarios. For example, CEnDR must be responsive to impacts from a disaster on critical infrastructure (Choi et al. 2019), as well as competing community priorities. As discussed above, the nature of a disaster may preclude the deliberate, careful development of a community partnership. Given the impacts of a disaster, communities may view the need for active engagement, involvement, or participation as a barrier. However, in our experience with CEnDR, having independent academic research groups engaging with local communities provided clear benefits. Smaller organizations were more nimble and able to deploy rapidly and scale projects appropriately to be responsive to community needs. The use of data-sharing agreements between researchers and communities, as well as the development of community advisory boards, may help mitigate concerns related to jeopardizing relationships with partners.

There is the related concern that without comprehensive collaboration between researchers, communities may receive multiple research requests. Increased collaboration under a CEnDR framework can reduce multiple requests and lower burdens to participation by aligning research goals and study designs. In our case studies, we identified a need to coordinate with other researchers that sought to characterize exposures following a disaster. Following the Gulf of Mexico oil spill, there was a call for increased collaboration to reduce such fatigue (Finucane et al. 2020). To enhance collaboration and reduce multiple, related community requests, platforms like the DR2 program may help connect disaster-impacted communities with researchers. While it is critical to have local participation in disaster research, it is not feasible to have a multitude of hyper-local groups that maintain capacity for research on a wide range of potential impacts from unpredictable disaster events. Rather, a national framework that connects communities and researchers within and outside of their region, and concurrently allows research groups to coordinate their efforts, would be mutually beneficial.

A CEnDR approach provides for the return of individual- and community-level data. The return of data has multiple benefits, including increased transparency and giving participants knowledge and agency to make informed decisions regarding their environment and their health (National Academies of Sciences and Medicine 2018). In the aforementioned case studies, all communities received final data, with Hurricane Harvey and NW wildfire participants receiving individual-level data. The return of data is embedded in the pre-positioned IRB and communicated as a data-sharing policy with all study participants. The ability to host focus groups, as was done with Hurricane Harvey, allowed us to better understand new and existing problems that may have been exacerbated by the hurricane.

A final challenge in disaster research includes the collection of baseline data (Knack et al. 2006). While CEnDR may be useful in identifying areas in which to collect such information, prioritized based on community knowledge of an area, data collection is reliant upon funding. The lack of funding was identified as the predominant barrier to conducting disaster research among EHS CC directors (Errett et al. 2019). Expanding existing rapid response funding mechanisms to accommodate and prioritize CEnDR can increase the relevance of research and increase coordination between researchers.

In sum, these case studies highlighted important issues in terms of the existing physical, technological, and social infrastructure for disaster research. Simultaneously, the studies also identified opportunities for CEnDR to leverage underlying strengths and to build an infrastructure for improving disaster response research. The infrastructure proposed here relates to existing technology, to include passive samplers, questionnaires, and the DR2 program that enhances collaboration. The case studies additionally recognize that while CEnDR positively engages with infrastructure, the nature of doing research during disasters also identifies the limitations of such an approach. CEnDR is further subject to limitations of the tools used (requirements for underlying technological and physical infrastructure), and the methods for engaging communities, recognizing differing levels of community involvement, engagement, and participation. CEnDR may not be appropriate for all disaster-response research given the caveats discussed above. The decision to move forward with a CEnDR study should be done in consultation with community groups and liaisons, with discussions around time commitments, compensation for community time and knowledge, and clear expectations regarding the use and dissemination of study data.

These case studies help inform a structure for long-term CEnDR that recognizes the disaster research continuum, from preparing for disasters through disaster response and recovery. The use of CEnDR, which includes community voice and knowledge, and transparent data-sharing policies can mitigate many challenges associated with disaster research and ensure that research is responsive to and informed by community needs.

Ethics and Consent

The Gulf of Mexico research collected environmental samples and did not collect personal identifiable information. Hurricane Harvey study protocols were reviewed and approved by the Institutional Review Boards at the Baylor College of Medicine (IRB #: HÓ 42111), Oregon State University (IRB#: 8146) and The University of Texas Health Science Center at Houston (IRB#: HSC-SPH-17-0846). NW wildfire study protocols have been reviewed and approved by the Human Research Protection Program at Oregon State University (IRB-2019-0312).

Acknowledgements

The authors gratefully acknowledge the community groups, liaisons, and study participants engaged in these studies.

Funding Information

Research reported in this publication was supported by the National Institute of Environmental Health Sciences of the National Institutes of Health (P42ES016465, P30 ES000210, P30 ES030287, R21 ES020120, R21 ES029460, R21 ES029616, T32ES0007060). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. HMD was supported in part by the ARCS Foundation®.

Competing Interests

KAA and DR disclose a financial interest in MyExposome, Inc., which is marketing products related to the research being reported. The terms of this arrangement have been reviewed and approved by Oregon State University in accordance with its policy on research conflicts of interest. The authors have no other competing interests to disclose. The other authors declare they have no actual or potential competing financial interests.

Authors Contributions

DR led conceptualization of the work, with all authors contributing to the theoretical concepts. The following authors contributed to the investigation of the individual case studies: Gulf of Mexico – SEA, LGT, KAA; Hurricane Harvey – KAA, DR, SS, MB, HD, LGT, PDH, AO, ES, MLB; NW Wildfires – KAA, DR, MB, CCG, PDH. The passive samplers were provided by KAA. DR wrote the original draft, and all authors reviewed and edited. All authors have approved of this manuscript.

References

  1. Aguilera, R, Corringham, T, Gershunov, A, Leibel, S and Benmarhnia, T. 2021. Fine particles in wildfire smoke and pediatric respiratory health in California. Pediatrics, 147. DOI: https://doi.org/10.1542/peds.2020-027128 

  2. Allan, SE, Smith, BW and Anderson, KA. 2012. Impact of the Deepwater Horizon oil spill on bioavailable polycyclic aromatic hydrocarbons in Gulf of Mexico coastal waters. Environmental science & technology, 46: 2033–2039. DOI: https://doi.org/10.1021/es202942q 

  3. Anderson, K and Hillwalker, W. 2008. Bioavailability. In: Encylcopedia of Ecology, 1: 348–357. Editors-in-Chief: Sven Erik Jørgensen and Brian D. Fath. DOI: https://doi.org/10.1016/B978-008045405-4.00375-X 

  4. Anderson, KA, Points, GL, Donald, CE, Dixon, HM, Scott, RP, Wilson, G, Tidwell, LG, Hoffman, PD, Herbstman, JB and O’Connell, SG. 2017. Preparation and performance features of wristband samplers and considerations for chemical exposure assessment. Journal of exposure science & environmental epidemiology, 27: 551–559. DOI: https://doi.org/10.1038/jes.2017.9 

  5. Andress, L, Hall, T, Davis, S, Levine, J, Cripps, K and Guinn, D. 2020. Addressing power dynamics in community-engaged research partnerships. Journal of Patient-Reported Outcomes, 4: 24. DOI: https://doi.org/10.1186/s41687-020-00191-z 

  6. Ausbrooks, CYB, Barrett, EJ and Martinez-Cosio, M. 2009. Ethical issues in disaster research: Lessons from Hurricane Katrina. Population Research and Policy Review, 28: 93–106. DOI: https://doi.org/10.1007/s11113-008-9112-7 

  7. Bergmann, AJ, Scott, RP, Wilson, G and Anderson, KA. 2018. Development of quantitative screen for 1550 chemicals with GC-MS. Analytical and bioanalytical chemistry, 410: 3101–3110. DOI: https://doi.org/10.1007/s00216-018-0997-7 

  8. Bongers, S, Janssen, NA, Reiss, B, Grievink, L, Lebret, E and Kromhout, H. 2008. Challenges of exposure assessment for health studies in the aftermath of chemical incidents and disasters. Journal of exposure science & environmental epidemiology, 18: 341–359. DOI: https://doi.org/10.1038/jes.2008.23 

  9. Chari, R., Sayers, ELP, Amiri, S, Leinhos, M, Kotzias, V, Madrigano, J, Thomas, EV, Carbone, EG and Uscher-pines, L. 2019. Enhancing community preparedness: an inventory and analysis of disaster citizen science activities. BMC public health, 19: 1–15. DOI: https://doi.org/10.1186/s12889-019-7689-x 

  10. Chmutina, K and Von meding, J. 2019. A Dilemma of language: “Natural disasters” in academic literature. International Journal of Disaster Risk Science, 10: 283–292. DOI: https://doi.org/10.1007/s13753-019-00232-2 

  11. Choi, J, Naderpajouh, N, Yu, DJ and Hastak, M. 2019. Capacity building for an infrastructure system in case of disaster using the system’s associated social and technical components. Journal of Management in Engineering, 35: 04019013. DOI: https://doi.org/10.1061/(ASCE)ME.1943-5479.0000697 

  12. Collogan, LK, Tuma, F, Dolan-Sewell, R, Borja, S and Fleischman, AR. 2004. Ethical issues pertaining to research in the aftermath of disaster. Journal of traumatic stress, 17: 363–372. DOI: https://doi.org/10.1023/B:JOTS.0000048949.43570.6a 

  13. Cooper, CB, Rasmussen, LM and Jones, ED. 2021. Perspective: The power (dynamics) of open data in citizen science. Frontiers in Climate, 3. DOI: https://doi.org/10.3389/fclim.2021.637037 

  14. Dixon, HM, Poutasse, CM and Anderson, KA. 2020. Silicone wristbands and wearables to assess chemical exposures. In: Phillips, KA, Yamamoto, DP and Racz, L (eds.), Total Exposure Health: An Introduction, 1 edn. a: CRC Press. DOI: https://doi.org/10.1201/9780429263286-11 

  15. EPA. 2017. EPA: Status of superfund sites in areas affected by Harvey. Washington, DC: Environmental Protection Agency. 

  16. Errett, NA, Haynes, EN, Wyland, N, Everhart, A, Pendergrast, C and Parker, EA. 2019. Assessing the national capacity for disaster research response (DR2) within the NIEHS Environmental Health Sciences Core Centers. Environmental Health, 18: 1–9. DOI: https://doi.org/10.1186/s12940-019-0498-y 

  17. Finucane, ML, Clark-Ginsberg, A, Parker, AM, Becerra-Ornelas, AU, Clancy, N, Ramchand, R, Slack, T, Parks, V, Ayer, L, Edelman, AF, Petrun Sayers, EL, Nataraj, S, Bond, CA, Lesen, AE, Ferreira, RJ, Drakeford, L, Fiore, J, Weden, MM, Venable, B and Black, AB. 2020. Building community resilience to large oil spills: Findings and recommendations from a synthesis of research on the mental health, economic, and community distress associated with the Deepwater Horizon Oil Spill. Rand Corporation. DOI: https://doi.org/10.7249/RRA409-1 

  18. Guerrini, CJ, Crossnohere, NL, Rasmussen, L and Bridges, JFP. 2021. A best–worst scaling experiment to prioritize concern about ethical issues in citizen science reveals heterogeneity on people-level v. data-level issues. Scientific Reports, 11: 19119. DOI: https://doi.org/10.1038/s41598-021-96743-4 

  19. Halofsky, JE, Peterson, DL and Harvey, BJ. 2020. Changing wildfire, changing forests: the effects of climate change on fire regimes and vegetation in the Pacific Northwest, USA. Fire Ecology, 16: 4. DOI: https://doi.org/10.1186/s42408-019-0062-8 

  20. Henderson, TL, Sirois, M, Chen, AC-C, Airriess, C, Swanson, DA and Banks, D. 2009. After a disaster: Lessons in survey methodology from Hurricane Katrina. Population Research and Policy Review, 28: 67–92. DOI: https://doi.org/10.1007/s11113-008-9114-5 

  21. Hicks, A, Barclay, J, Chilvers, J, Armijos, MT, Oven, K, Simmons, P and Haklay, M. 2019. Global mapping of citizen science projects for disaster risk reduction. Frontiers in Earth Science, 7: 226. DOI: https://doi.org/10.3389/feart.2019.00226 

  22. Israel, BA, Schulz, AJ, Parker, EA and Becker, AB. 1998. Review of community-based research: assessing partnership approaches to improve public health. Annual review of public health, 19: 173–202. DOI: https://doi.org/10.1146/annurev.publhealth.19.1.173 

  23. Knack, JM, Chen, Z, Williams, KD and Jensen-Campbell, LA. 2006. Opportunities and challenges for studying disaster survivors. Analyses of social issues and public policy, 6: 175–189. DOI: https://doi.org/10.1111/j.1530-2415.2006.00116.x 

  24. Lichtveld, M, Sherchan, S, Gam, KB, Kwok, RK, Mundorf, C, Shankar, A and Soares, L. 2016. The Deepwater Horizon oil spill through the lens of human health and the ecosystem. Current environmental health reports, 3: 370–378. DOI: https://doi.org/10.1007/s40572-016-0119-7 

  25. Lurie, N, Manolio, T, Patterson, AP, Collins, F and Frieden, T. 2013. Research as a part of public health emergency response. Mass Medical Soc. DOI: https://doi.org/10.1056/NEJMsb1209510 

  26. Miller, A, Yeskey, K, Garantziotis, S, Arnesen, S, Bennett, A, O’Fallon, L, Thompson, C, Reinlib, L, Masten, S and Remington, J. 2016. Integrating health research into disaster response: The new NIH disaster research response program. International journal of environmental research and public health, 13: 676. DOI: https://doi.org/10.3390/ijerph13070676 

  27. Miller, AE and Craft, E. 2018. Rapid-response local monitoring of toxic air pollution after Hurricane Harvey. ISEE Conference Abstracts. DOI: https://doi.org/10.1289/isesisee.2018.S01.03.18 

  28. National Academies OF Sciences, Engineering and Medicine. 2018. Returning individual research results to participants: guidance for a new research paradigm. National Academies Press. 

  29. O’Connell, SG, Kincl, LD and Anderson, KA. 2014. Silicone wristbands as personal passive samplers. Environmental science & technology, 48: 3327–3335. DOI: https://doi.org/10.1021/es405022f 

  30. O’Fallon, L and Finn, S. 2015. Citizen science and community-engaged research in environmental public health. Lab Matters, 4. 

  31. Oluyomi, AO, Panthagani, K, Sotelo, J, Gu, X, Armstrong, G, Luo, DN, Hoffman, KL, Rohlman, D, Tidwell, L and Hamilton, WJ. 2021. Houston hurricane Harvey health (Houston-3H) study: assessment of allergic symptoms and stress after hurricane Harvey flooding. Environmental Health, 20: 1–15. DOI: https://doi.org/10.1186/s12940-021-00694-2 

  32. Packenham, JP, Rosselli, RT, Ramsey, SK, TayloR, HA, Fothergill, A, Slutsman, J and Miller, A. 2017. Conducting science in disasters: Recommendations from the NIEHS working group for special IRB considerations in the review of disaster related research. Environmental health perspectives, 125: 094503. DOI: https://doi.org/10.1289/EHP2378 

  33. Reams, MA, Harding, AK, Subra, W, Lam, NS, O’Connell, SG, Tidwell, L and Anderson, KA. 2017. Response, recovery, and resilience to oil spills and environmental disasters: Exploration and use of novel approaches to enhance community resilience. Journal of Environmental Health, 80: 8–15. 

  34. Rohlman, D, Bethel, J, Hoffman, P, Tidwell, LG and Anderson, KA. 2020. Oregon State University Chemical exposure disaster study protocol. In: Superfund Research Program, Pacific Northwest Center for Translational Environmental Health Research., Oregon State University, Corvallis, OR (ed.). NIH Public Health Emergency and Disaster Research Response (DR2): Disaster Literature: Database for Disaster Medicine and Public Health. 

  35. Rohlman, D, Dixon, HM, Kincl, L, Larkin, A, Evoy, R, Barton, M, Phillips, A, Peterson, E, Scaffidi, C, Herbstman, JB, Waters, KM and Anderson, KA. 2019a. Development of an environmental health tool linking chemical exposures, physical location and lung function. BMC public health, 19: 854. DOI: https://doi.org/10.1186/s12889-019-7217-z 

  36. Rohlman, D, Donatuto, J, Heidt, M, Barton, M, Campbell, L, Anderson, KA and Kile, ML. 2019b. A case study describing a community-engaged approach for evaluating polycyclic aromatic hydrocarbon exposure in a Native American community. International journal of environmental research and public health, 16: 327. DOI: https://doi.org/10.3390/ijerph16030327 

  37. Schroeder, W, Ruminski, M, Csiszar, I, Giglio, L, Prins, E, Schmidt, C and Morisette, J. 2008. Validation analyses of an operational fire monitoring product: The Hazard Mapping System. International Journal of Remote Sensing, 29: 6059–6066. DOI: https://doi.org/10.1080/01431160802235845 

  38. Smith, AB. 2020. 2010–2019: A landmark decade of US. billion-dollar weather and climate disasters. National Oceanic and Atmospheric Administration. 

  39. Sumaila, UR, Cisneros-Montemayor, AM, Dyck, A, Huang, L, Cheung, W, Jacquet, J, Kleisner, K, Lam, V, Mccrea-Strub, A and Swartz, W. 2012. Impact of the Deepwater Horizon well blowout on the economics of US Gulf fisheries. Canadian Journal of Fisheries and Aquatic Sciences, 69: 499–510. DOI: https://doi.org/10.1139/f2011-171 

  40. Symanski, E, Han, HA, Han, I, Mcdaniel, M, Whitworth, KW, McCurdy, S, Perkison, WB, Rammah, A, Lewis, PGT and Delclos, GL. 2021. Responding to natural and industrial disasters: partnerships and lessons learned. Disaster medicine and public health preparedness, 1–4. DOI: https://doi.org/10.1017/dmp.2020.467 

  41. Wallerstein, N, Muhammad, M, Sanchez-Youngman, S, Rodriguez Espinosa, P, Avila, M, Baker, EA, Barnett, S, Belone, L, Golub, M, Lucero, J, Mahdi, I, Noyes, E, Nguyen, T, Roubideaux, Y, Sigo, R and Duran, B. 2019. Power dynamics in community-based participatory research: A multiple–case study analysis of partnering contexts, histories, and practices. Health Education & Behavior, 46: 19S–32S. DOI: https://doi.org/10.1177/1090198119852998 

  42. Wells, KB, Tang, J, Lizaola, E, Jones, F, Brown, A, Stayton, A, Williams, M, Chandra, A, Eisenman, D and Fogleman, S. 2013. Applying community engagement to disaster planning: developing the vision and design for the Los Angeles County Community Disaster Resilience initiative. American Journal of Public Health, 103: 1172–1180. DOI: https://doi.org/10.2105/AJPH.2013.301407 

  43. Woolley, JP, McGowan, ML, Teare, HJ, Coathup, V, Fishman, JR, Settersten, RA, Sterckx, S, Kaye, J and Juengst, ET. 2016. Citizen science or scientific citizenship? Disentangling the uses of public engagement rhetoric in national research initiatives. BMC medical ethics, 17: 1–17. DOI: https://doi.org/10.1186/s12910-016-0117-1