Citizen science (CS) has been rapidly gaining popularity over the past several decades (Pocock et al. 2017), as public participation can increase both the scale of environmental research and public engagement in environmental issues. Benefits also have been demonstrated for CS participants themselves, including increased understanding of science, awareness of the local environment, and confidence to participate in scientific activities (Bonney et al. 2009; Dickinson et al. 2012; Evans et al. 2005; Jordan et al. 2012; National Academies of Sciences 2018; Trumbull et al. 2000).
As CS participants also have demonstrated increased knowledge and self-efficacy to address local environmental problems, CS may serve as an advocacy tool for disenfranchised communities burdened by environmental health risks, henceforth “environmental justice (EJ) communities,” to strategically address local environmental health risks (Allen 2018; Averett 2017; Bonney et al. 2016; Brown 1997; Carr 2004; Den Broeder et al. 2017; Ottinger 2010; Sandhaus, Kaufmann, and Ramirez-Andreotta 2018). However, as members of EJ communities are more likely to have disproportionate demands placed on their time and resources, those most likely to be affected by environmental health stressors may be the least likely to participate in research to address it (Bullard 2008; Cole and Foster 2000; Morello-Frosch et al. 2011). Additionally, potential economic, cultural, and racial barriers to relationship building between academic researchers and EJ community members increase the possibility of unsuccessful community engagement, or even exacerbating harms to community members (Allen 1998; Foster and Dunham 2015; Saxton et al. 2015).
Parallel to the recent rise of CS, other community-driven approaches are being increasingly employed in environmental health research. These approaches, such as community-based participatory research (CBPR) and participatory action research (PAR), emphasize close partnership between the research institution and EJ community to address community-identified interests and issues (Baum, MacDougall, and Smith 2006; Israel et al. 2012; Minkler and Wallerstein 2011; O’Fallon and Dearry 2002). In some cases, EJ communities have self-organized to conduct research and take action against environmental health risks without the backing of a research institution (Dhillon 2017; Heaney et al. 2007; Hoover 2016; Scott 2016). These examples overlap with CS approaches that emphasize participant involvement, such as collaborative/co-created CS (Bonney et al. 2009; Shirk et al. 2012) and extreme CS (Haklay 2013), and offer important guidance on successful engagement between research institutions and EJ communities (Minkler et al. 2008; Wallerstein and Duran 2010; Wilson et al. 2014).
While CBPR or PAR for environmental health is often conducted with communities of color, volunteers CS projects initiated by research institutions are predominantly college-educated and white (Evans et al. 2005; Haklay 2015; National Academies of Sciences 2018; Pandya 2012a; Soleri et al. 2016). Prior studies on CS participant engagement and motivation have recognized this demographic homogeneity in their participant population (Domroese and Johnson 2017; Frensley et al. 2017; Hobbs and White 2012; Land-Zandstra et al. 2016; Raddick et al. 2013). This has limited understanding of how non-white or non-college educated individuals might be better engaged in CS projects. As practitioners across the CS spectrum aim for increased participant diversity and equity (Pandya 2012b; Soleri et al. 2016; Sorensen et al. 2019), considering the influence of “dispositional variables” such as education level and income (Penner 2002) and learning from the experiences of current diverse CS participants could facilitate this understanding.
Prior research provides recommendations for conducting environmental health CS projects, but these recommendations lack direct program participant input (Barzyk et al. 2018; Den Broeder et al. 2018; English, Richardson, and Garzón-Galvis 2018). In the study herein, 120 demographically diverse participants and 7 promotoras (community health workers) in Project Harvest, an ongoing co-created CS project, contribute their perspectives toward the following research questions: 1) What are participant motivations, support structures, and barriers to participation in an environmental health CS project? 2) Are there demographic or community-specific differences in how Project Harvest participants are motivated to participate, receive support, or experience barriers to participation?
For the purpose of this study, motivation is defined as what influences an individual’s decision to sign up and engage in the initial stages of the project (Clary and Snyder 1999; Deci and Ryan 2008; Deckers 2018). Support distinctly refers to influences that sustain participation in the project. Self-efficacy (SE), defined as one’s perception of their own capability, was considered a related measure of support, as SE beliefs “determine how people feel, think, motivate themselves and behave” (Bandura 1994). Findings of this research build on existing literature related to CS participant experience, motivations, and engagement (Bruyere and Rappe 2007; Carballo-Cárdenas and Tobi 2016; Carrera et al. 2018; Hobbs and White 2012; Land-Zandstra et al. 2016; Measham and Barnett 2008; Raddick et al. 2013; Rambonnet et al. 2019; Sandhaus 2017) to inform a “deliberate design” (Shirk et al. 2012) for engaging diverse CS participants.
Project Harvest (www.projectharvest.arizona.edu) was launched in 2017 as a collaborative effort by researchers at the University of Arizona (UA) and the community-based organization Sonora Environmental Research Institute, Inc. (SERI), to facilitate community-led environmental monitoring in four geographic areas of Arizona, USA, with known sources of environmental contamination. Primary project goals are to produce local data on contaminants present in harvested rainwater and in rainwater-irrigated soil and food plants; increase community involvement in environmental decision-making; and increase environmental health literacy in underserved rural and urban communities.
Project Harvest is a co-created CS project (Shirk et al. 2012), for the following reasons: 1) The central research question, to determine the degree of health risk or safety in harvested rainwater and rainwater-irrigated soil and food plants, originated from the voiced concerns of community members in Dewey-Humboldt, AZ, during participation in a previous CS project (Ramirez-Andreotta et al. 2014); 2) Research design and project proposal were collaboratively crafted by UA researchers and SERI; 3) Throughout the project, feedback from SERI staff, promotoras, and participants continues to inform responsive modifications in project methods. Project Harvest is in its 2nd year of participatory data collection at the time of this writing.
The promotora model of health promotion is a strategy that has been successfully used in disempowered communities to increase community-level knowledge of health impacts and health behavior (Deitrick et al. 2010; Hunter et al. 2004; Ingram et al. 2008). The Spanish term promotora, commonly defined in English as community health worker, refers to community members who share information with peers in culturally appropriate settings using culturally appropriate communication methods.
Applying experience from prior applications of this model in environmental health contexts (May et al. 2003; Ramírez et al. 2015), Project Harvest employs promotoras as the designated educators and support persons for participants. As SERI staff promotoras have previous in-depth experience with residents in the urban participant communities, four SERI promotoras were designated to support these participants through an inter-organizational agreement. For the three rural communities, promotoras were recruited through public community training events, described further below, and through local organizational partners.
Two Project Harvest promotoras are predominantly Spanish-speaking, one is fluent in English and Spanish, and four are predominantly English-speaking. With the exception of one promotora who has lived in her rural community for only three years, the promotoras have lived in their communities for at least a decade, with one being a 4th generation resident. Professional backgrounds are diverse, including teaching, government program administration, and community organizing. Level of education in the group spans from high school diploma to graduate degrees. Each promotora supports 15–25 participant households throughout the three years of the project.
The geographic locations of partnering communities in Arizona, USA are illustrated in Figure 1. Three locations–Dewey-Humboldt, Globe/Miami, and Hayden/Winkelman are considered rural, while Tucson is considered urban. Following established recommendations for community-based participatory research (CBPR) (Israel et al. 2012), communities were defined as a unit of identity, rather than by municipal boundaries. For this reason, in two of the rural geographic areas, neighboring municipalities that share histories, sources of environmental health risk, and cultural attributes were defined as one community. In the urban area of Tucson, however, two distinct communities were defined by how they were recruited to the project. As SERI’s organizational focus is low-income households, participants recruited directly by SERI are predominantly low-income and Spanish speaking. Other Tucson participants were recruited through the water utility (Tucson Water) as former recipients of a rainwater harvesting rebate program, and are predominantly non-low-income, college educated, and English speaking.
Initial community outreach and participant recruitment took place through public trainings facilitated by project staff in the four selected areas, which incorporated participants’ knowledge and lived experiences to co-create training content on rainwater harvesting and environmental contamination (Davis et al. 2018). Of the workshop participants who were interested to sign up with Project Harvest, however, many were ineligible due to the requirements that participants live in specific geographic areas and harvest rainwater at home. The majority of the Project Harvest participants were recruited through their promotora or a local community organization. Participants in Project Harvest overall are economically and racially diverse, with just over 50% 1) self-identifying as low-income or below based on HUD guidelines; 2) self-identifying as a non-white race/ethnicity (predominantly Latino/Hispanic); and 3) not having a college degree. Additionally, 25% speak Spanish as their primary language. As each partnering community is unique, however, characteristics and demographics of each defined community are illustrated in Table 1.
|Community; # Active Participants||City/Town, County, Population1||Participant Recruitment Methods||Participant Race/Ethnicity; Language4||Participant Low-Income (LI) Status4||Participant Education Level Range and Median4||Local Environ-mental Health Risk Sources|
|Tucson SERI (TS)
N = 55
|Tucson, Pima, 520,116||Recruited through community-based partner organization SERI, who works with low-income Tucsonans on environmental health issues.||80% Latino/Hispanic
4% Multiple Races
|Range: Elementary to bachelor’s degree
Median: High school
|Multiple Toxic Release Inventory (TRI) sites including Tucson Interna-tional Airport Area Superfund Site and an unlined landfill. Aircraft and electronics manufacturing have largely contributed to trichloroethylene (TCE) and other contaminants observed in soil, groundwater, and municipal water (Conforma Tech, Inc. 2014; Rodenbeck Sven E. and Maslia Morris L. 1998).|
|Tucson Water (TW)
N = 38
|Tucson Water recruited via email to past recipients of a rainwater harvesting system rebate. Interested respondents were screened via phone by SERI staff.||69% White
8% Latino/Hispanic 5% Asian
5% Multiple Races
|16% LI 68% Non-LI||Range: High school to graduate degree
Median: Graduate degree
N = 12
|Dewey-Humboldt, Yavapai, 3,894||Participants recruited at community training and by promotora.
UA researchers had prior community relationships through other participatory research projects (Ramirez-Andreotta et al. 2016; Ramirez-Andreotta, Brusseau, Artiola, Maier, and Gandolfi, 2015).
8% Multiple Races
|Range: Some college, degree not completed, to bachelor’s degree
Median: Trade or technical school
|Iron King Mine – Humboldt Smelter Superfund Site includes approximately four million cubic meters of mine tailings from legacy mine and smelter.
Documented dioxin spraying in Pinal Mountains in the 1960s (U.S. DHS, 1998).
Arsenic level in home drinking water observed above.
EPA drinking water standard in 2013 (Ramirez-Andreotta, Brusseau, Beamer, & Maier, 2013).
N = 25
|Globe and Miami2, Gila
|6 participants recruited through community training, others through outreach by promotora and Gila County Cooperative Extension.||72% White
|Range: High school to bachelor’s degree
Median: Trade or technical school
|Two active open pit mines, active smelter and rod mill, and legacy mine in Miami. Additional active open pit mine 15 mi outside of study area in Superior, AZ.
Mountain View Mobile Home Estates site in Globe was on the Superfund Program’s National Priorities List (NPL) for asbestos contamination of soil and groundwater until clean-up activities completed in 1988.
N = 21
|Hayden and Winkelman3, Gila, 1,015||Prior relationship between community and university through National Institute of Environmental Health Sciences Superfund Research Program partnership.
School superintendent promoted teacher involvement. Most participants recruited directly through promotora.
|Range: Middle school to bachelor’s degree
Median: Some college, degree not completed
|ASARCO Hayden Plant Alternative Superfund Site, includes the ASARCO smelter, concentrator, former Kennecott smelter and all associated tailings facilities.
In 2016, ASARCO was involved in a $150 million settlement with the US Department of Justice and US Environmental Protection Agency for violations of the Clean Air Act. (Anon 1970)
To understand motivation, support, and barriers to CS participation, data were collected directly from 120 Project Harvest participants via telephone interviews, open-ended surveys, rank scale questionnaires, focus groups, and email/phone communications from participants. Data from these four data sources were aggregated, controlling for repeated participants, and analyzed using mixed methods including thematic coding and descriptive statistics. Additionally, feedback was solicited from the seven promotoras via telephone or in-person interviews, to triangulate participant data. Participants and promotoras were consented under the University of Arizona IRB as an approved project. All interview, survey, and focus group questions were developed by a subgroup of the Project Harvest team, defined as a “complex team” (Davidson 2018) dedicated to participant learning research (learning research team), which includes the project PI, external project evaluator, lab manager, and three graduate student research assistants. Semi-structured interview questions were scripted based on established guidelines (Gall, Gall, and Borg 2007; Turner 2010), and qualitative coding performed based on established literature (Creswell and Poth 2017; Davidson, Thompson, and Harris 2017; Scammell 2010; Tracy 2010).
Participants also completed a rank scale item survey related to self-efficacy (SE) at their initial training in the project, and again after one year of participation, to measure self-efficacy as a related indicator to support in participation. Finally, cited reasons for participant resignation were documented and analyzed as a related indicator for barriers to participation. Statistical analyses were conducted to determine any significant differences between communities and demographic groups, related to SE, resignation from the project, or within participants expressing common themes identified from qualitative data.
A timeline of data collection activities is illustrated in Figure 2. Table 2 summarizes the sources of data by research questions they addressed and analysis methods used. Data collection and analysis methods are described by each data source in greater detail below.
|Research question||Participant phone interviews||Participant Year One survey||Participant focus groups||Other participant communication||Promotora interviews||Participant Self-efficacy (SE) survey||Reasons cited for participant resignation|
|What motivates Project Harvest participants to join the project?||x||x||x||x||x|
|What supports participants in their continued participation?||x||x||x||x||x||x|
|What participant barriers deter participation?||x||x||x||x||x||x|
|What are the differences in motivation, support, and barriers between community or demographic groups?||x||x||x||x||x||x||x|
|Analysis Methods||– Data aggregated across multiple sources, controlled for repeat participants
– Qualitative coding for themes
– Chi square tests conducted to determine significant different between community or demographic groups
– Cramér’s V calculated for every Chi square analysis to measure effect size
|– Qualitative coding for participant themes; compared to participant data
– Qualitative coding for promotora themes
|– Paired samples t-test used to measure change in SE over first year of participation
– Kruskal-Wallis test used to determine significant difference in SE (PRE, Y1, and PRE-Y1 change) between communities. – Mann-Whitney U tests used to determine significant differences in SE (PRE, Y1, and PRE-Y1 change) between demographic groups.
|– Data compiled by reason for resigning
– Chi square tests conducted to determine significant difference between community or demographic group representation within the group of resigned participants
Four members of the learning research team attempted to contact all active participants (N = 144) by phone in August 2018, after most participants had completed at least one sampling season and before data-sharing events were held. The semi-structured interview script (Appendix A, supplemental files) aimed to solicit participant feedback on training, project materials, sampling methods, data reporting methods, and overall experience. Fifty three English-speaking and 20 Spanish-speaking participants were reached and willing to participate in an interview. Interviews were conducted by phone in the participant’s dominant language and lasted between 8 and 45 minutes. To increase response rate and maintain an informal atmosphere, the interviewer took detailed notes rather than recording interviews.
Project Harvest participants complete written surveys annually, designed to measure an array of participant outcomes, including science literacy, environmental literacy, numeracy, environmental action, and community building, which relate to research questions of the project not addressed in this study. Promotoras were trained in survey administration during onboarding, and surveys were administered by promotoras as part of a home visit in the participant’s dominant language.
Responses to one open-ended question, included in the survey administered after one year of participation, were included as data for this study. This open-ended question reads: “Since starting with Project Harvest, have you learned anything new? Please write any new ideas, observations or questions you have about …,” followed by the headings 1) Rainwater Harvesting, 2) Rainwater Contamination, 3) Human Health and the Environment, 4) Doing Experiment or Taking Samples, and 5) Other. Although this question was designed to assess participant perceptions of their own learning, responses were included in the data set as many responses addressed elements of the study which supported or frustrated participants.
Two data sharing events have been conducted in each of the four communities at the time of this writing. These serve as social events for participants to meet, receive results from their own rainwater samples and community aggregate data, and discuss data with Project Harvest staff and each other. Following time to view personalized data reports and a staff presentation on data interpretation, participants met in focus groups of 2–5 with two staff facilitators per group. Facilitators followed a semi-structured interview script (Appendix B, supplemental files) to guide the group in interpreting results related to rainwater risk or safety, as well as solicit participant feedback on their experience of receiving and interpreting their data, and any changes they plan to make based on data received. In almost every group, participant discussion and questions also emerged related to project methods and rationale, as well as participant experience in other aspects of the project. Focus group discussions were recorded and professionally transcribed, and transcriptions were reviewed by the interviewer/facilitator and corrected for accuracy.
Unsolicited participant feedback about project experience via phone, email, or in person communication to the principal investigator or lab manager was documented by the staff member involved in the communication and compiled by the learning research team as part of the participant data set.
Data from sources 1–4 above were aggregated and analyzed using established mixed methods (Tashakkori and Teddlie 2020). Some deductive codes were created based on collaborative discussion and observations from learning research team members. From emergent themes in the data, a coding team of three members created a codebook to capture main concepts and coded all data for themes using NVivo qualitative analysis software (QSR International Pty Ltd. Version 11, 2017). Early in the process, the coding team compared individual coding on a subset of 25 responses to a survey question, and on two focus group transcripts, and observed 88% agreement, suggesting acceptable inter-rater reliability (Armstrong et al. 1997; Krippendorff 2004). The team met throughout the coding process to revisit codebooks, sometimes choosing to create or merge code categories based on inductive themes in the data. The first author supervised coding activities for consistency and met regularly with the learning research team to discuss coding themes, trends, and examples, to collaboratively discuss results and interpret findings. Under each of the three parent codes, “Participant motivation to sign up/start participating,” “Participant support in participating,” and “Participant barriers to participating,” a set of child codes was developed for each that reflected what was observed in the data. Quantity of participant communications, as well as non-emergence of new themes towards the end of the coding process, suggests data saturation to adequately address the research questions (Guest, Bunce, and Johnson 2006; Mason 2010; Saunders et al. 2018).
Following the coding process, all data were migrated into SPSS quantitative analysis software (IBM 2017) with thematic codes by data source coded to specific participants. From this, we were able to calculate frequencies of participants who expressed each theme, by data source and as a combined frequency over all four participant data sources.
To understand if participants from different demographic or geographic groups may be motivated, supported, or discouraged differently in the project, Chi square tests were performed to assess correlation within participant groups expressing each child code theme to different demographic groups. For each child code category, we tested for correlation to participant groups by defined community (TS, TW, DH, GM, HW), self-identified race/ethnicity (White/Caucasian, Latino/Hispanic, Asian/Asian-American, Multiple, and Other were represented), income level (“low-income” or “non-low-income” according to 2015 HUD guidelines), and education level (4-year college graduates or higher compared to non-college graduates). All demographic data were self-reported by participants and collected via a survey administered by the promotora at the first home visit. Participants who did not report a certain demographic characteristic were removed from the data pool prior to demographic analyses. Of 120 total contributing participants, 19 did not report race/ethnicity, 21 did not report household income, and 41 did not report highest level of education achieved. To determine correlation in binary demographic categories (income and education level), Chi square tests were sufficient to produce a p value to indicate if one of the two demographic groups were either under- or overrepresented in the theme group beyond what would be expected by the null hypothesis. For demographic categories with more than two groups (community and race/ethnicity), if the Chi square test produced a statistically significant result, post hoc testing was performed to determine which demographic group(s) were either under- or overrepresented in the theme group beyond what would be expected by the null hypothesis using adjusted residuals to reduce the chance of Type 1 error (Beasley and Schumacker 1995; García-Pérez and Núñez-Antón 2003). In an effort to not rely solely on statistical significance in making meaning of the data (Amrhein, Greenland, and McShane 2019), Cramér’s V also was calculated for every Chi square analysis to measure effect size (Kirk 2007). All statistical analyses were performed using SPSS quantitative analysis software (IBM 2017).
As each promotora is herself a member of her participant community and has frequent communication with participants, promotoras were interviewed to solicit motivations, support, and barriers experienced personally, as well as what they observed in their participants (Appendix C, supplemental files). These interviews were conducted in January 2019, two years into promotoras’ participation in the project, after rapport and trust had developed with research staff. Interviews lasted from 35 to 75 minutes and were conducted in the promotora’s preferred language by a research assistant. Promotora interviews were recorded and professionally transcribed, and transcriptions were reviewed by the interviewer/facilitator and corrected for accuracy prior to analysis.
As promotora data served to triangulate participant data, the same team coded these transcripts themes following the codebook developed from participant data, as well as independently coded for themes related to the promotora herself, using NVivo qualitative analysis software (QSR International Pty Ltd. Version 11, 2017). Frequency of participant themes expressed was compared to frequency of themes in participant data.
Participant surveys, administered annually by promotoras as part of a home visit in the participant’s dominant language, include rank scale items modified from a validated survey instrument developed by Cornell Lab of Ornithology (Phillips et al. 2014) to measure self-efficacy (SE) in adult citizen science participants. Survey items for learning science (four items) and SE for doing science (four items) were modified per instructions given by the instrument creators to relate directly to environmental quality monitoring and included in both pre-participation (PRE) and year one (Y1) surveys (Appendix D, supplemental files). These data allowed for assessment of participant baseline and mid-participation SE, as a related measure to participant support. Paired samples t-test was used to assess change in self-efficacy over the first year of participation.
A Kruskal-Wallis test was used to determine significant difference in SE (PRE, Y1, and PRE-Y1 change) between communities. Mann-Whitney U tests were used to determine significant differences in correlation of SE (PRE, Y1, and PRE-Y1 change) to race/ethnicity (White/Caucasian or Latino/Hispanic), income level (“low-income” or “non-low-income” according to 2015 HUD guidelines), and education level (4-year college graduates or higher compared to non-college graduates).
Reasons cited by participants choosing to resign from the project (N = 23), communicated directly by the participant to project staff or their promotora, were tracked as additional data to inform study design aspects that may create barriers to participation. Primary reasons cited for resigning were aggregated by number of participants citing. A Chi square test was used to assess any correlations between resigning from the project and any community or demographic groups.
Frequency of themes expressed by more than one promotora related to their own motivation, support, and barriers to success in their role is illustrated in Table 3. Project Harvest promotoras are paid staff, and 4 of the 7 promotoras were contracted through their employing organization, so different motivations and experiences from volunteer participants were expected. However, 5 of the 7 promotoras talked about the relationships with their community members as the most rewarding part of the job, and the connection to project personnel was the second most frequently described motivation. Similarly, the most frequently cited support to promotoras was activities where project team collaboration occurred, highlighting personal relationships and team membership as critically important. As an example, one rural promotora described having initially seen the job listing but not considering applying due to her non-science background. However, after personally meeting and talking with the project PI in Spanish during a public training event in her town, she applied.
|Parent Code: Promotora motivation|
|Child Code||Example of Child Code||# Promotoras Citing (N = 7)|
|Connection to community||“The rewarding part is all the connections that I have got with my participants, the relationship. …So all these connections with people in the community- I have been privileged to be let into their homes.” – Promotora 5||5|
|Connection to project purpose and personnel||“Being able to come onsite to Tucson and see the lab and see the people and, like, how I think our data-sharing was good for the participants to see us and you know what happened, was good for me to see you know what happens once I give you stuff or who are people, and names on the phone or faces, voices on the phone seeing names and people in person. Definitely make, I think, any project better. I liked the external board meeting, how that whole day went and again learning a lot about that here were people who felt our work was a value.” – Promotora 6||4|
|Facilitating participant learning||“For me the best part that I like is actually making the- Going over and watching them perform the experiment. For them to do the experiment for them to put on the gloves and for them to actually look at the results, because a lot of these people have lived in the community for a while. So for them to be interested in their environment and what’s going on has been really motivating to me. For them to be more become more aware of their surroundings.” – Promotora 7||3|
|Interest to learn about CS||“Interested in how you design something like this.” – Promotora 3||3|
|Project is interesting/want to learn||“It’s interesting for me …because I am also learning much more than just the course we took at the beginning of the training we took.” – Promotora 1 (Translated from Spanish)||2|
|Parent Code: Promotora support|
|Child Code||Example of Child Code||# Promotoras Citing (N = 7)|
|Group membership||“I’ve wanted to stay involved because of your team- [Project Harvest staff] is amazing and everyone in your team is great.” -Promotora 3 “Then I was surprised when they gave us the certificate, even I, who was a promotora, said, ‘Wow, so cool I’m a scientist!’ That also motivates us.” – Promotora 2||4|
|Adequate training and preparation||“I mean… l never felt like I was doing something and didn’t have the right tools. Because you gave them to us. Making sure we were very well trained and you know because I had participants that say, how come you [can] do [the experiments]? You know I said I well we had a lot of training and a lot of practice!” – Promotora 7||2|
|Parent Code: Promotora barriers|
|Child Code||Example of Child Code||# Promotoras Citing (N = 7)|
|Challenges recruiting/engaging local community||“I’ve put up flyers have gotten no response. Businesses- No response. One is the librarian because we hold our events there.” – Promotora 6||5|
|Computer use barrier||“And that’s the frustrating part for me because … computers didn’t come to [the community] til the maybe the last five years I want to say. We weren’t accountable to be using the computer all the time. We had to have a hard copy of everything and so I’m still kind of like old school I guess… On the computer I can spend 20–30 minutes trying to figure out what I did wrong and never get to do what I wanted to do.” – Promotora 7||3|
|Parent Code: Promotora barriers|
|Child Code||Example of Child Code||# Promotoras Citing (N = 7)|
|Project feedback not considered||“I was there because when I attended the training in Tucson, in two different opportunities, we were invited to make comments. But those comments that we made didn’t translate to the [participant instruction] books before they were printed. So it was… I think they were kind of behind in their deadlines. But that was, for me, was something frustrating.” – Promotora 5||3|
|Project methods changed after training||“The instructions- to me it’s not a problem- but maybe because of the, well that in the training it was all completely differently set up.” – Promotora 6||3|
|Misconceptions about roles within project team||“They stopped having one group meeting and they divided it. And so I am fine with the English speaking meetings which is nice. But I always wonder why … If we would say we [are] all treated equally, the promotoras. So that is something that has been hard for me to deal with internally.” – Promotora 5||2|
|Participant protocol more complicated than expected||“And I read the book I read all the protocols and I went to [the Project Harvest] half day training and I still…I know that you just fill the bottles but I still don’t feel comfortable.” – Promotora 4||2|
The majority of promotoras cited unforeseen difficulty in recruiting participants as their main frustration. Additionally, three of the seven promotoras experienced difficulty using project-provided ipads and web-based project survey and reporting tools, even after one-on-one training. As discussed further below, participants of these three promotoras were also more likely to experience difficulty with web-based project resources themselves. Despite the collaborative nature of the team and frequent communications, distance communication relied heavily on conference calls, often in English and Spanish. This presented challenges in maintaining consistent transparency between university-based staff and promotoras, which may have contributed to themes cited about frustration and confusion with internal processes.
Frequency of themes observed in participant motivation, with noted significant differences observed for any demographic group (community, race/ethnicity, household income level, education level), is illustrated in Table 4. The most frequently observed motivations were 1) a positive attitude about rainwater harvesting and/or home gardening (38%), and 2) the desire to know about contaminants in the local environment (24%), both of which relate to the specific study content. Motivation from interest to learn about the topic, environmental/health concerns, and wanting to contribute (to science, the environment, or the community) were other motivations cited by 9–25% of participants.
|Parent Code: Participant Motivation|
|Child Code||Examples of Child Code||Total Contributing Participants (N = 120)||Significant Difference by Group1||Omnibus Significance and Effect Size2||# Promotoras citing theme (N = 7)|
|Positive attitude about rainwater harvesting or gardening||“Rainwater is good for my plants, it has a lot of nutrients and it helps them grow more quickly.” -T104 (Translated from Spanish)
“People like the idea of saving water and they’re very enthusiastic about participating.” – Promotora 4
(N = 46)
|*TS (+) (50%)(N = 23)
*TW (+) (24%)(N = 11)
|*Community φc = .332||2|
|Want to know contaminants in local environment||“Looking forward to results- I expect most contaminants will be onsite rather than environmental/airborne. I look forward to getting my hypotheses tested.” -G413||24%
(N = 29)
|*Non-low-income (+) (59%)(N = 17)||*Income level φc = .215||–|
|Project is interesting/want to learn||“Well, it’s been very interesting for me and the program you present is what we’re going to be seeing in the lab, too. It’s something, well, we keep learning.” -T161 (Translated from Spanish)||21%
(N = 25)
|*College grads (+) (40%)(N = 10)||*College grad φc = .240||–|
|Health risk concerns||“I live … below the Pinal Mountains, where it was sprayed with Agent Orange and lots of people here have cancer. I am curious and concerned about what might be in our soil.” -G424||16%
(N = 19)
|*GM (+) (42%)(N = 8)||*Community φc = .363||1|
|Contribute to scientific research||“Collecting data for the UA and public to be informed about our environment is my main motivation.” -G412||13%
(N = 17)
|*Non-college grads (–) (6%)(N = 1)
*White (+) (82%)(N = 14)
(12%)(N = 2)
|*College grad φc = .426
*Race/ethnicity φc = .350
|Contribute to the environment||“It is important we better understand rainwater harvesting because we will be needing more of it when there is less water in the Colorado River” -T128 (Translated from Spanish)||12%
(N = 15)
|Connection to project purpose||“I think one of the most important things is the participants feel like they’re something- the need to feel like you’re something, you’re part of something bigger. You feel like you’re contributing.” – Promotora 4||9%
(N = 12)
|Connection to the community||“Air and water quality impacts everyone in our community. These studies are very important.” -H203||9%
(N = 11)
Significant differences in participant motivation were observed between communities. Participants motivated by rainwater harvesting or gardening enthusiasm were predominantly Tucson residents (50% from the TS community and 24% from the TW community), which relate to cultural popularity of these activities in Tucson. GM participants were more likely to cite health concerns as a primary motivation, and the GM promotora was the only one to express health concerns as a personal motivation to get involved. This motivation aligns with the local history of environmental contamination (see Table 1) and a recent local awareness of environmental health risk prompted by recent EPA actions (US EPA 2018).
Within the participant group motivated by contribution to science, White participants were overrepresented, while both Latino/Hispanic participants and participants without a college degree were underrepresented. Similarly, only a promotora working with the predominantly White and college-educated TW community cited contribution to scientific research as an observed participant motivation. Additionally, participants with college degrees were overrepresented within the group motivated by personal learning, and non-low-income participants were overrepresented within the group motivated to find out about contaminants in the environment. These results suggest that the motivation of gaining knowledge (unrelated to a perceived risk) may resonate more with higher socioeconomic status participants, while low-income individuals may be more motivated to address a potential risk.
TS community participants are also recipients of a low-cost rain barrel through community-based organization SERI’s low-income rainwater harvesting program. Both TS promotoras expressed their participants may have signed up for Project Harvest because of previous relationship with, and perceived sense of obligation to, the community-based organization, rather than any motivation related to the project itself. This may have affected demographic distribution in motivation theme groups, as TS community members (majority Latino/Hispanic) predominantly cited motivations related to rainwater harvesting rather than those related to environmental contamination or receiving data. Notably, social allegiance could also serve as a barrier to participation. Rumors that Project Harvest aimed to damage the local mining corporation, a major employer, was described by one rural promotora as a significant challenge to recruitment: “I wasn’t able to recruit people that speak Spanish in my area … because of this perception that we--the program--was going to find things that were damaging to the mine and then that will be the issue later. I think that had a lot to do with not getting more people into the program.”
Figure 3 illustrates mean scores for self-efficacy (SE) for science for participants pre-participation (PRE) and after one year of participation (Y1), by demographic and geographic groups. Pre-participation SE was high (average 4.0 on a 1–5 scale) and did not change after one year. There was no significant difference in pre-participation SE, or change in SE after one year, observed by community, race/ethnicity group, household income group, or education level group, respectively.
Frequency of themes observed in participant support, with noted significant differences observed for any community or demographic group, is illustrated in Table 5. Direct communication to interpret results was most frequently cited as important participant support, with no significant differences between communities or demographic groups. This direct communication, as well as Receiving/Understanding results, also commonly cited as an important participant support, predominantly occurred at data sharing events. Comments coded for these two themes describe in-person discussion of results as both increasing participants’ sense of connection to the project and serving as a powerful incentive to follow sampling procedures carefully and correctly.
|Parent Code: Participant Support|
|Child Code||Example of Child Code||Total Contributing Participants (N = 120)||Significant Difference by Group1||Omnibus Significance and Effect Size2||# Promotoras citing theme
(N = 7)
|Direct communication with project staff to interpret data||PH staff: You spray the house with what?
T158: With something for the bugs.
PH staff: …maybe you could also tell us the type of [pesticide] you use and in the lab we can check if that spray–
T158: If it’s contaminating… I will send you a picture.
(N = 49)
|Initial home visit training with promotora||“[Promotora] was very good at showing us what to do.” -H218||40%
(N = 48)
|*Non-college grad (+) (56%) (N = 27)||*College grad φc = .235||–|
|Promotora support||“Insecure while we’re doing it…but I call my promotora and she helps me.” -T169 (Translated from Spanish)||28%
(N = 33)
|*GM (+) (36%)(N = 12)
(6%)(N = 2)
|*Community φc = .348||4|
|Direct communication with project staff to clarify methods||“Asked questions about when to upload results, whether to wash experimental vials, and if there is anything to get back to [her promotora].” -Staff notes on conversation with D308||23%
(N = 27)
|*DH (+) (26%)(N = 7)
*TS (–) (7%)(N = 2)
|*Community φc = .457||–|
|Receiving/understanding results||“Every time I did a sample this year, I was so nervous that I wasn’t doing it right… Having attended tonight’s [data sharing event], and gotten the data, I realize that maybe I did okay. I think I have a lot more confidence to do it by myself next year.” -T194||19%
(N = 24)
|Instructional booklet||“Instruction book is useful.” -T137||15%
(N = 19)
|*TW (+) (58%)(N = 11)
*College grad (+) (42%)(N = 8)
|*Community φc = .316
*College grad φc = .238
|Gain confidence with practice||“It reminds me of chemistry lab but I am getting the hang of it.” -G415
“The beginning of the first year I would go to their homes. And usually when it would rain they would call me and I would be going through the experiment and I was like wow are these people ever going to learn to just do it on their own? …But as I was doing the last sampling they were actually putting their gloves on and they were ready to do the whole thing themselves, collecting the dirt, doing the experiment. They were motivated to do it themselves.” – Promotora 7
(N = 12)
|Household members working together||“nice to have Dad help.” -T004, Y1S||6%
(N = 8)
|Talks with other participants about project||“We just live two trailers down from each other… [the other participant lives] two trailers closer to the mine than I am… And I was remembering my [DIY experiment] results, and seeing her results, and then thinking, oh my goodness, there is a difference, even within the proximity there.” -H212, Y1FG||6%
(N = 8)
|Familiarity with past experiences||“I took [water] samples working for mines.” -G425, Y1S
“No problem- just like school!” -H214, Y1S
(N = 7)
Some differences were observed between community and demographic groups in what methods of support were most meaningful. Participants without a college degree were more likely to cite the initial home training with the promotora as an important support, which may suggest the promotora model as especially relevant for participants with less formal education. Within the group who specifically cited promotora support as helpful, GM community members were represented more than the expected distribution and TW community members were represented less. The latter may relate to a local issue with one of the initial promotoras in the TW community, who faced personal challenges early in the project and did not continue after the first round of home visits (another promotora, included in this study, was subsequently hired). This personnel turnover in TW limited relationship building in the early stages of the project.
Conversely, college graduates and TW community members were significantly more likely to cite the instruction booklet as especially helpful. Participant instruction booklets were produced in both Spanish and English and went through several rounds of revisions based on promotora feedback, in an effort to be as clear and accessible as possible. Despite these efforts, these results suggest that written materials may provide disproportionate support to college-educated and urban participants. Project Harvest staff also created short instructional videos as an alternate communication method, which two promotoras and a small subset (5%) of participants reported as helpful. However, we observed that the promotora’s own comfort level with technology, which is highly variable within the group, correlated with their participants’ exposure to online project resources and tools. The three promotoras who cited technology challenges for themselves personally in the project also cited their perception of this challenge for their participants.
Participants in the DH community, as well as non-low-income participants, were overrepresented within those who contacted project staff directly. As DH participants are predominantly White, English-speaking, and college educated, as well as many being transplants from urban areas, it is possible that their identities increase their comfort level in directly contacting professional scientists. Conversely, TS participants, who are predominantly Latino/Hispanic, Spanish speaking, and not college educated, were underrepresented in those who engaged directly with research staff. This may be due to the effectiveness of promotoras in the TS community. However, this may also relate to sociocultural barriers for those who are not represented in dominant representations of “scientist,” similar to sociocultural barriers for people of color for participation in STEM generally (Hurtado et al. 2010; National Center for Science and Engineering Statistics 2019). As one TS promotora said, “They waited for me to help … maybe because they were doing it themselves and they do not feel like chemists or biologists or doctors or whatever,” suggesting that participant identity may have acted as a barrier to participation in her community.
Frequency of themes observed in participant barriers to participation, with noted significant differences observed for any community or demographic group, is illustrated in Table 6. Participant reasons for resigning from the project, communicated directly by the participant or via their promotora, are illustrated in Table 7, and provide additional insight on what study design aspects may deter participation. There was no statistically significant correlations observed between resigning and any community, race/ethnicity, household income level, or education level group.
|Parent Code: Participant Barriers|
|Child Code||Examples of Child Code||Total Contributing Participants (N = 120)||Significant Difference by Group1||Omnibus Significance and Effect Size2||# Promotoras citing theme (N = 7)|
|Tasks more time-consuming or complicated than expected||“People not having a good understanding of the time commitment. For DIY, some people were overwhelmed by the amount of time it would take- sometimes people had questions, but generally felt supported by booklet, was more unwilling to take the time.” – Promotora 3
“The one problem I had was the arsenic test…Because we had this much liquid, this much space, and then a test strip that we couldn’t get wet and had to agitate it for 45 seconds…I mean, this is a lot of failure built into this process, I thought.” -D307
(N = 26)
(19%)(N = 5)
|*Race/ethnicity φc = .304||7|
|Not confident in sampling or doing experiments||“I had to constantly review steps of collection and testing as didn’t trust my memory and didn’t want to mess up results for that reason.” -T122||18%
(N = 21)
|No computer/internet access||“Does not have internet access” – Staff notes on interview with T116
“[Participant] doesn’t have a smartphone and she doesn’t have a computer.” – Promotora 5
(N = 21)
(52%)(N = 11)
*HW (+) (33%)(N = 7)
(5%)(N = 1)
(0%)(N = 0)
*Non college grad (+)(70%) (N = 13)
*Latino (+) (76%)(N = 16)
(10%)(N = 2)
|*Community φc = .376
*College grad φc = .257
*Race/ethnicity φc = .417
|Technical issues with rainwater harvesting||“Not going well because gutter system not collecting.” -T133||13%
(N = 15)
|Difficulty with instructions||“I’m pretty sure there’s instructional booklet, but I’d rather just have her explain it to me.” I’m doing an apprenticeship program [as part of employment at the mine] and have to do so much reading, it’s hard for me to read stuff, I would rather just do it with someone.” – H213||11%
(N = 13)
|Computer/tech use barriers||“Has a smart phone but does not know how to use internet on smart phone.” -Staff notes on interview with T153||10%
(N = 12)
|*College grad (+) (67%)(N = 8)||*College grad φc = .365||3|
|Logistical barriers to sample drop-off||“Only thing difficult is the limited window and location for sample drop-off. Makes it difficult for working people.” – T142||8%
(N = 10)
|Unexpected life events||“It’s been a tough month with losses in the family so we’ve been away.” -G425||8%
(N = 10)
|*Non low income (+) (80%)(N = 8)||*Income φc = .365||–|
|Unclear on how to submit samples or results||“Did both the winter and monsoon experiments for water, have the results and photos and pages of journal notes. Have been meaning to upload them, not sure if I remember how.” -T122||8%
(N = 9)
|Reason Cited*||# Participants (N = 23)||% Participants Resigned|
|No reason given/Not able to contact||16||70%|
|Did not have enough time to participate||5||22%|
|Unexpected life event||5||22%|
|Technical challenges with rainwater harvesting||5||22%|
|Confused or frustrated with procedures||4||17%|
|Computer access challenges||1||4%|
Of participants who dropped out of the project, 22% cited not having enough time. Similarly, the most frequently cited barrier to participation both by participants and promotoras was that participant protocols were too time-consuming or complicated. Latino/Hispanic participants were underrepresented in the group that reported this barrier. However, this underrepresentation may be related to previously discussed findings that TS participants were less likely to directly contact project staff, and may have felt a sense of obligation to the promotora organization, as well as to the documented “simpatía” cultural norm in Latino/Hispanic communities, which emphasizes conflict avoidance and harmony in interpersonal relations (Marin and Marín 1991).
The largest number of differences between community and demographic group representation were observed in the participant group reporting lack of computer or Internet access. Latino/Hispanic participants and non-college graduates were overrepresented in this group, whereas White participants were underrepresented. Similarly, two predominantly Latino/Hispanic communities (TS, HW) were overrepresented in this group, and two predominantly White communities (TW, DH) were underrepresented in this group, with only one community (GM) represented within an expected range. One participant cited “lack of computer access” as their reason for resigning from the project. This barrier was unanticipated, as Project Harvest participants were not required to have computer access, promotoras were equipped and trained to upload participant results, and paper versions of surveys and results submission forms are available. However, the extra step of contacting the promotora to help complete the results submission may have increased the burden of participation for some.
College graduates were statistically overrepresented in the group that has computer access but some barriers to use, which is likely related to the fact that more non-college graduates had no computer/internet access, and were thus ineligible for the “computer/tech use barriers” category. Technology use barriers cited were varied, for example, the participant website requires a login ID, which caused confusion for some. Another participant said, “I don’t know why the upload step seems like a burden, it’s just the part that’s not as exciting and you want to put off for later.”
Findings discussed inform suggested strategies for increasing diverse participants in CS, as outlined in Table 8, and discussed further below.
|Project Harvest Participant Findings||Responsive CS Project Design Strategies|
|Motivation||Motivation to participate varies by community.||
|Non-traditional participants1 are more likely to be motivated by existing relationships with individuals or an organization, or by addressing a perceived risk.||
|Non-traditional participants1 are less likely to be motivated by learning for the sake of learning or by contributing to science.||
|Support||Non-traditional participants1 are more likely to be supported by personal interactions than by written materials.||
|Barriers||Non-traditional participants1 are less likely to have reliable computer or internet access.||
|Lack of time was the most frequently reported barrier for participants generally.||
No significant differences between community or demographic groups were observed related to self-efficacy (SE) for science, resigning from the project, reporting lack of confidence, or reporting difficulty understanding instructions. These results importantly suggest that although socioeconomic factors affect countless life outcomes, from educational attainment to physical health and mortality (Anderson et al. 1997; Damian et al. 2015; Gee and Payne-Sturges 2004; Walpole 2003), they do not predict success of a citizen scientist. Consequently, the high proportion of White and college educated participants in CS may be due in part to culturally biased project design. As Max Liboiron of CLEAR Laboratories stated in the 2019 keynote address to the Citizen Science Association, “The inclusion model often is a model of equality, where it brings people into a space that’s already not designed for them. It treats everyone the same, bringing them into contact with accredited science. We already know this doesn’t work– you can bring many women and people of color into science and they still ‘fall out of the STEM pipeline’ because that pipeline is built for someone else” (Liboiron 2019). Conversely, the success of CBPR and other participant-driven approaches for environmental health research has been attributed to community members themselves generating the research question, designing the study, and leading the process, allowing for the entire project to evolve within the cultural context of the participant community (Cacari-Stone et al. 2014; Heaney et al. 2011; Lichtveld et al. 2016; Minkler et al. 2010).
Although motivation to contribute to scientific research has been widely cited in participants of other environmental CS projects (Domroese and Johnson 2017; Land-Zandstra et al. 2016; Phillips 2018; Raddick et al. 2013), findings here suggest contribution to science, as well as learning for sake of learning, may be motivations that apply disproportionately to White, college educated, and non-low-income participants. Participants from communities of color were more likely to be motivated by addressing a relevant problem, or prior relationship with a community organization and its staff. This suggests relevancy of the study content and personnel involved may be important motivation factors for engaging diverse CS participants.
Additionally, participants from predominately White communities were more likely to communicate with university-based staff, while Latino/Hispanic participants were more likely to communicate only with their promotora. As participants of color in other environmental health studies have expressed distrust of formal study authorities (Scammell et al. 2009), participant liaisons that share participant identity traits may be critical for increasing diverse CS participation. In the experience of this study and others (e.g., Lichtveld et al. 2016), hiring on-the-ground project staff from within partnering communities presents some working challenges (e.g., cultural/linguistic differences, distance communication) but are far outweighed by the benefits of local knowledge and relationships to improve participant engagement.
Several promotoras described the value to participants of study elements that recognized them as “part of the team” (e.g., receiving a newsletter, postcard, or certificate; community events; direct communication; data sharing events). The importance of participant recognition for sustained participation has been cited elsewhere (Rotman et al. 2014; West and Pateman 2016), and may hold even greater importance where participants question whether they “belong” in science based on identity. Informal community events where participants and community members can talk to scientists and university-based staff may similarly promote a sense of group membership and identity as a “scientist.”
Woven throughout the data is the theme of personal relationships as both the glue that holds Project Harvest together and the oil that allows it to run smoothly. Participants expressing appreciation for their promotora often used words more personal (“I just love her!”) than professional. The most widely cited motivation by promotoras was connection to their own community members. Participants described taking water samples with their kids, parents, grandparents, school classrooms, and each other. This aligns with related CS participant research findings: Environmental stewardship project participants in an urban community of color reported being more motivated by their commitment to community than to the natural environment (Sorensen et al. 2018), and both “connection to common humanity” and “connection to community organizations” were found to be significant motivations for participants in community-centered environmental health research (Carrera et al. 2018). One study of participants across six environmental CS projects found social motivation factors to be especially important for co-created CS projects (Phillips et al. 2019). Although these social connections remain difficult to track, they may be immensely important to the motivation and engagement of underrepresented citizen scientists.
In Project Harvest, social data sharing events served as the best venues for relationship building among participants. At these events, neighboring participants were observed comparing results and discussing potential causes for the difference in their contaminant levels (“She is maybe 40 ft closer [to the smelter]”), as well as exchanging phone numbers for reasons unrelated to the project. Although Project Harvest was carefully designed to protect participant privacy, participants at data sharing events were observed sharing openly about their community’s history, household practices, and personal health issues. This observed open data sharing act among participants served as an additional mechanism for relationship building and social participation in learning. Ample literature documents the importance of data report back in environmental health research (Brody et al. 2014; Perovich et al. 2018; Ramirez-Andreotta et al. 2016; Vousden et al. 2014). The benefits from data sharing events observed here suggest that designing for relationship building and social negotiation of knowledge can deepen participants’ sense of connection to the project, the study purpose, and each other.
Social participation learning theory (Lave and Wenger 1991) asserts there is no separation between participating and learning, as the act of participation is embedded with cultural and contextual knowledge. In a community of practice, a group of practitioners share an interest and continuously deepen their knowledge of this interest area through practice and sharing information (Wenger 2011). Through frequent participation, participants can move closer to the center of the community, where they immersed in the group knowledge and culture. Scholars have conceptualized online crowdsource CS groups as communities of practice (Jackson et al. 2014; Liberatore et al. 2018; Mugar et al. 2014; Newman et al. 2012), though this model of social learning and co-creation of culture is arguably even more relevant for an in-person CS community. Conceptualizing CS communities as communities of practice has been previously proposed as a strategy for maximizing learning and engagement (Phillips et al. 2019).
It may benefit CS projects to deliberately design, and adequately allocate resources, for relationship building at every level of the project, ensuring that research staff, community-based staff, and participants have regular opportunities for dialogue. In Project Harvest, this design includes: 1) In-person community environmental health trainings as participant recruitment events, 2) Weekly conference calls with project staff and promotoras, 3) Annual promotora professional development events with project staff, 4) Regular opportunities for open, friendly engagement between project staff, promotoras, and participants at community trainings, data sharing events, and informal “open house” events throughout each year.
Data sharing events previously discussed were held at familiar local venues, food was provided, and all participants received postcard invitations and text message reminders prior to the event. Despite these efforts, less than one third of total participants actually attended an event. Participants and promotoras both cited the time required to do project tasks as the greatest barrier to participation. Designing for relationship building, which inherently requires participant time, while also honoring participant time constraints may be a challenge in participatory research “that won’t go away” (Long et al. 2016). Strategies from other CS projects include using tiered levels of participation (e.g., Ablah et al. 2016) to simultaneously make participation accessible for those with limited time and resources, and provide those with more time the option to get more deeply involved. As Project Harvest staff and promotoras look forward to planning the coming year’s participant events, we are challenged with how to facilitate social participation and co-learning, while acknowledging the many competing priorities that participants have on their time.
The technology access divisions observed between communities, race/ethnicity, and education level groups were unanticipated as a barrier, as the Project Harvest participant website does not require technical skills beyond those used on Facebook, which many participants reported using, and promotoras were equipped with ipads and training to assist with web-based tools. Challenges have been addressed to date by creating paper versions of all participant forms, including a results worksheet, accompanied by self-addressed stamped envelopes in participant kits as an option to mail results. Still, these challenges provide important lessons for CS project design: 1) Web-based project tools may bias participation by White and/or college-educated participants, 2) Cultural relevance of technology varies widely by community and within communities, and 3) Access to an Internet device may not preclude other barriers to using project-related technology.
Multiple iterative changes have been made in Project Harvest to improve accessibility based on participant and promotora feedback, including: 1) Promotoras scheduling home training visits to fit participants’ schedules, including the option of scheduling multiple shorter visits, 2) Offering 3-5 rainwater sample drop off days per season for more flexibility, 3) Allowing participants to submit results forms online or via mail anytime, 4) Giving participants the choice of methodology (collect samples and drop off for lab testing, or complete their own experiments and report results only) in the third year of the project, and 5) Adding “Project Harvest Open House” social/educational events as additional opportunities for participants to meet staff and each other. Continuing to develop and share strategies to minimize CS participant burdens while maximizing relationship building emerges from this study as a priority as the environmental health CS field continues to evolve.
As the research questions relate to CS participant experience, the decision to solicit participant voice directly was a conscious one. However, the reliance on self-reported data introduces the possibility of bias. Multiple forms of participant data, as well as promotoras’ understanding of their participants, were used to triangulate data and lend validity to findings.
Although the study design differentiates between participant “motivation” to sign up for and participate in an environmental health CS project, and participant “support” to continue participating and staying engaged, some interaction exists between these two categories (Ryan, Kaplan, and Grese 2001; West and Pateman 2016). Motivation and engagement are malleable, and may shift throughout participation in the project based on participant experience and contextual factors (Carballo-Cárdenas and Tobi 2016; Hibbert, Piacentini, and Dajani 2003; Lawson and Lawson 2013). Project Harvest participant data will continue to be collected over the next two to three years in an effort to better understand these dynamics.
Environmental health CS offers an opportunity for historically disenfranchised community members to participate in the creation and interpretation of knowledge related to contaminants in their local environments. CS studies can make participation more accessible and meaningful for demographically underrepresented participants by designing study methods that align with the partnering community’s local political dynamics, culture, context, communication style, and relevant technology. Findings of this study importantly reflect a lack of association between participant self-efficacy and race, income, or education level. Feedback from participants and promotoras in Project Harvest emphasize the importance of the following in environmental health CS project design for diverse participants: 1) Considering existing relationships and community-identified problems as participant motivation, 2) Designing participant methods to include personal support structures and relationship-building, and 3) Designing for participant time and technology access as significant limitations to participation. These findings serve to inform best practices in environmental health CS research design as well as equitable CS project design generally.
The supplementary files for this article can be found as follows:Appendix A
Participant Interview Script August 2018. DOI: https://doi.org/10.5334/cstp.253.s1Appendix B
Focus Group Script December 2018. DOI: https://doi.org/10.5334/cstp.253.s2Appendix C
Promotora Interview Script January 2018. DOI: https://doi.org/10.5334/cstp.253.s3Appendix D
Participant Self-Efficacy Survey. DOI: https://doi.org/10.5334/cstp.253.s4
We would like to thank all Project Harvest participants who contributed to this study, as well as promotoras Imelda Cortez, Margaret Dewey, Theresa Foley, Palmira Henriquez, Miriam Jones, Lisa Ochoa, and Aviva O’Neil, and for their candid feedback and commitment to the project and their participants. We would like to thank research assistants Alma Anides Morales and Iliana Manjón for conducting interviews and focus groups that contributed to this dataset, Alma Anides Morales and Nikki Lippert for their assistance in qualitative coding, Dorsey Kaufmann who created the images in Figures 1 and 2, and lab manager Shana Sandhaus for support of multiple project activities which enabled this research. We would like to thank Dr. Bruce Johnson and Dr. Joe Heimlich for their reviews and suggestions on the manuscript.
This research was funded by the National Science Foundation’s Division of Research and Learning – Advancing Informal STEM Learning Program, grant award number 1612554.
The authors have no competing interests to declare.
LFD, MDR-A, and SRB collaboratively conceptualized and designed the study. MDR-A is principle investigator on the project and supervises all project activities. LFD oversaw data collection activities. LFD and SRB conducted data analysis. LFD drafted the original manuscript and supervised editing. MDR-A and SRB reviewed manuscript and contributed to edits.
Allen, BL. 1998. Women Scientists and Feminist Methodologies in Louisiana’s Chemical Corridor. Michigan Feminist Studies, 13(1). DOI: https://doi.org/10.1080/08164649.1998.9994909
Allen, BL. 2018. Strongly Participatory Science and Knowledge Justice in an Environmentally Contested Region. Science, Technology, & Human Values, 43(6): 947–971. DOI: https://doi.org/10.1177/0162243918758380
Amrhein, V, Greenland, S and McShane, B. 2019. Scientists rise up against statistical significance. Nature, 567(7748): 305. DOI: https://doi.org/10.1038/d41586-019-00857-9
Anderson, RT, Sorlie, P, Backlund, E, Johnson, N and Kaplan, GA. 1997. Mortality Effects of Community Socioeconomic Status. Epidemiology, 8(1): 42–47. DOI: https://doi.org/10.1097/00001648-199701000-00007
Armstrong, D, Gosling, A, Weinman, J and Marteau, T. 1997. The Place of Inter-Rater Reliability in Qualitative Research: An Empirical Study. Sociology, 31(3): 597–606. DOI: https://doi.org/10.1177/0038038597031003015
Averett, N. 2017. New Blood: The Promise of Environmental Health Citizen Science Projects. Environmental Health Perspectives, 125(11). DOI: https://doi.org/10.1289/EHP2484
Barzyk, TM, Huang, H, Williams, R, Kaufman, A and Essoka, J. 2018. Advice and Frequently Asked Questions (FAQs) for Citizen-Science Environmental Health Assessments. International Journal of Environmental Research and Public Health, 15(5). DOI: https://doi.org/10.3390/ijerph15050960
Baum, F, MacDougall, C and Smith, D. 2006. Participatory action research. Journal of Epidemiology & Community Health, 60(10): 854–857. DOI: https://doi.org/10.1136/jech.2004.028662
Beasley, TM and Schumacker, RE. 1995. Multiple Regression Approach to Analyzing Contingency Tables: Post Hoc and Planned Comparison Procedures. The Journal of Experimental Education, 64(1): 79–93. DOI: https://doi.org/10.1080/00220973.1995.9943797
Bonney, R, Phillips, TB, Ballard, HL and Enck, JW. 2016. Can citizen science enhance public understanding of science? Public Understanding of Science, 25(1): 2–16. DOI: https://doi.org/10.1177/0963662515607406
Brody, JG, Dunagan, SC, Morello-Frosch, R, Brown, P, Patton, S and Rudel, RA. 2014. Reporting individual results for biomonitoring and environmental exposures: lessons learned from environmental communication case studies. Environmental Health: A Global Access Science Source, 13(40). DOI: https://doi.org/10.1186/1476-069X-13-40
Brown, P. 1997. Popular Epidemiology Revisited. Current Sociology, 45(3): 137–156. DOI: https://doi.org/10.1177/001139297045003008
Bruyere, B and Rappe, S. 2007. Identifying the motivations of environmental volunteers. Journal of Environmental Planning and Management, 50(4): 503–516. DOI: https://doi.org/10.1080/09640560701402034
Cacari-Stone, L, Wallerstein, N, Garcia, AP and Minkler, M. 2014. The Promise of Community-Based Participatory Research for Health Equity: A Conceptual Model for Bridging Evidence With Policy. American Journal of Public Health, 104(9): 1615–1623. DOI: https://doi.org/10.2105/AJPH.2014.301961
Carballo-Cárdenas, EC and Tobi, H. 2016. Citizen science regarding invasive lionfish in Dutch Caribbean MPAs: Drivers and barriers to participation. Ocean & Coastal Management, 133: 114–127. DOI: https://doi.org/10.1016/j.ocecoaman.2016.09.014
Carr, AJL. 2004. Policy Reviews and Essays. Society & Natural Resources, 17(9): 841–849. DOI: https://doi.org/10.1080/08941920490493846
Carrera, JS, Brown, P, Brody, JG and Morello-Frosch, R. 2018. Research altruism as motivation for participation in community-centered environmental health research. Social Science & Medicine, 196: 175–181. DOI: https://doi.org/10.1016/j.socscimed.2017.11.028
Clary, EG and Snyder, M. 1999. The Motivations to Volunteer: Theoretical and Practical Considerations. Current Directions in Psychological Science, 8(5): 156–159. DOI: https://doi.org/10.1111/1467-8721.00037
Damian, RI, Su, R, Shanahan, M, Trautwein, U and Roberts, BW. 2015. Can personality traits and intelligence compensate for background disadvantage? Predicting status attainment in adulthood. Journal of Personality and Social Psychology, 109(3): 473–489. DOI: https://doi.org/10.1037/pspp0000024
Davidson, J. 2018. Qualitative Research and Complex Teams. Understanding Qualitative Research. Oxford, New York: Oxford University Press. DOI: https://doi.org/10.1093/oso/9780190648138.001.0001
Davidson, J, Thompson, S and Harris, A. 2017. Qualitative Data Analysis Software Practices in Complex Research Teams: Troubling the Assumptions About Transparency and Portability. Qualitative Inquiry. DOI: https://doi.org/10.1177/1077800417731082
Davis, LF, Ramirez-Andreotta, MD, McLain, JET, Kilungo, A, Abrell, L and Buxner, S. 2018. Increasing Environmental Health Literacy through Contextual Learning in Communities at Risk. International Journal of Environmental Research and Public Health, 15(10): 2203. DOI: https://doi.org/10.3390/ijerph15102203
Deci, EL and Ryan, RM. 2008. Self-determination theory: A macrotheory of human motivation, development, and health. Canadian Psychology/Psychologie canadienne, 49(3): 182–185. DOI: https://doi.org/10.1037/a0012801
Deckers, L. 2018. Motivation: Biological, psychological, and environmental. Routledge. DOI: https://doi.org/10.4324/9781315178615
Deitrick, LM, Paxton, HD, Rivera, A, Gertner, EJ, Biery, N, Letcher, AS, Lahoz, LM, Maldonado, E and Salas-Lopez, D. 2010. Understanding the Role of the Promotora in a Latino Diabetes Education Program. Qualitative Health Research, 20(3): 386–399. DOI: https://doi.org/10.1177/1049732309354281
Den Broeder, L, Devilee, J, Van Oers, H, Schuit, AJ and Wagemakers, A. 2018. Citizen Science for public health. Health Promotion International, 33(3): 505–514. DOI: https://doi.org/10.1093/heapro/daw086
Den Broeder, L, Lemmens, L, Uysal, S, Kauw, K, Weekenborg, J, Schönenberger, M, Klooster-Kwakkelstein, S, Schoenmakers, M, Scharwächter, W, Weerd, A, van de Baouchi, SE, Schuit, AJ and Wagemakers, A. 2017. Public Health Citizen Science; Perceived Impacts on Citizen Scientists: A Case Study in a Low-Income Neighbourhood in the Netherlands. Citizen Science: Theory and Practice, 2(1). DOI: https://doi.org/10.5334/cstp.89
Dhillon, CM. 2017. Using citizen science in environmental justice: participation and decision-making in a Southern California waste facility siting conflict. Local Environment, 22(12): 1479–1496. DOI: https://doi.org/10.1080/13549839.2017.1360263
Dickinson, JL, Shirk, J, Bonter, D, Bonney, R, Crain, RL, Martin, J, Phillips, T and Purcell, K. 2012. The current state of citizen science as a tool for ecological research and public engagement. Frontiers in Ecology and the Environment, 10(6): 291–297. DOI: https://doi.org/10.1890/110236
Domroese, MC and Johnson, EA. 2017. Why watch bees? Motivations of citizen science volunteers in the Great Pollinator Project. Biological Conservation, 208: 40–47. DOI: https://doi.org/10.1016/j.biocon.2016.08.020
English, PB, Richardson, MJ and Garzón-Galvis, C. 2018. From Crowdsourcing to Extreme Citizen Science: Participatory Research for Environmental Health. Annual Review of Public Health, 39(1): 335–350. DOI: https://doi.org/10.1146/annurev-publhealth-040617-013702
Evans, C, Abrams, E, Reitsma, R, Roux, K, Salmonsen, L and Marra, PP. 2005. The Neighborhood Nestwatch Program: Participant Outcomes of a Citizen-Science Ecological Research Project. Conservation Biology, 19(3): 589–594. DOI: https://doi.org/10.1111/j.1523-1739.2005.00s01.x
Foster, A and Dunham, IM. 2015. Volunteered geographic information, urban forests, & environmental justice. Computers, Environment and Urban Systems, 53: 65–75. DOI: https://doi.org/10.1016/j.compenvurbsys.2014.08.001
Frensley, T, Crall, A, Stern, M, Jordan, R, Gray, S, Prysby, M, Newman, G, Hmelo-Silver, C, Mellor, D and Huang, J. 2017. Bridging the Benefits of Online and Community Supported Citizen Science: A Case Study on Motivation and Retention with Conservation-Oriented Volunteers. Citizen Science: Theory and Practice, 2(1). DOI: https://doi.org/10.5334/cstp.84
García-Pérez, MA and Núñez-Antón, V. 2003. Cellwise Residual Analysis in Two-Way Contingency Tables. Educational and Psychological Measurement, 63(5): 825–839. DOI: https://doi.org/10.1177/0013164403251280
Gee, GC and Payne-Sturges, DC. 2004. Environmental Health Disparities: A Framework Integrating Psychosocial and Environmental Concepts. Environmental Health Perspectives, 112(17): 1645–1653. DOI: https://doi.org/10.1289/ehp.7074
Guest, G, Bunce, A and Johnson, L. 2006. How Many Interviews Are Enough? An Experiment with Data Saturation and Variability. Field Methods, 18(1): 59–82. DOI: https://doi.org/10.1177/1525822X05279903
Haklay, M. 2013. Citizen science and volunteered geographic information: Overview and typology of participation. In: Crowdsourcing Geographic Knowledge, 105–122. Dordrecht: Springer. DOI: https://doi.org/10.1007/978-94-007-4587-2_7
Heaney, CD, Wilson, S, Wilson, O, Cooper, J, Bumpass, N and Snipes, M. 2011. Use of community-owned and -managed research to assess the vulnerability of water and sewer services in marginalized and underserved environmental justice communities. Journal of Environmental Health, 74(1): 8. DOI: https://doi.org/10.1289/isee.2011.00095
Heaney, CD, Wilson, SM and Wilson, OR. 2007. The West End Revitalization Association’s Community-Owned and -Managed Research Model: Development, Implementation, and Action. Progress in Community Health Partnerships: Research, Education, and Action, 1(4): 339–349. DOI: https://doi.org/10.1353/cpr.2007.0037
Hibbert, S, Piacentini, M and Dajani, HA. 2003. Understanding volunteer motivation for participation in a community-based food cooperative. International Journal of Nonprofit and Voluntary Sector Marketing, 8(1): 30–42. DOI: https://doi.org/10.1002/nvsm.199
Hobbs, SJ and White, PCL. 2012. Motivations and barriers in relation to community participation in biodiversity recording. Journal for Nature Conservation, 20(6): 364–373. DOI: https://doi.org/10.1016/j.jnc.2012.08.002
Hoover, E. 2016. ‘We’re not going to be guinea pigs;’ Citizen Science and Environmental Health in a Native American Community. Journal of Science Communication, 15(01). DOI: https://doi.org/10.22323/2.15010205
Hunter, JB, de Zapien, JG, Papenfuss, M, Fernandez, ML, Meister, J and Giuliano, AR. 2004. The Impact of a Promotora on Increasing Routine Chronic Disease Prevention among Women Aged 40 and Older at the U.S.-Mexico Border. Health Education & Behavior, 31(4_suppl): 18S–28S. DOI: https://doi.org/10.1177/1090198104266004
Hurtado, S, Newman, CB, Tran, MC and Chang, MJ. 2010. Improving the rate of success for underrepresented racial minorities in STEM fields: Insights from a national project. New Directions for Institutional Research, 2010(148): 5–15. DOI: https://doi.org/10.1002/ir.357
Ingram, M, Sabo, S, Rothers, J, Wennerstrom, A and de Zapien, JG. 2008. Community Health Workers and Community Advocacy: Addressing Health Disparities. Journal of Community Health, 33(6): 417–424. DOI: https://doi.org/10.1007/s10900-008-9111-y
Jackson, C, Østerlund, C, Crowston, K, Mugar, G and Hassman, KD. 2014. Motivations for sustained participation in citizen science: Case studies on the role of talk. In: 17th ACM Conference on Computer Supported Cooperative Work & Social Computing. 2014 Citeseer. DOI: https://doi.org/10.1109/HICSS.2015.196
Jordan, RC, Ballard, HL and Phillips, TB. 2012. Key issues and new approaches for evaluating citizen-science learning outcomes. Frontiers in Ecology and the Environment, 10(6): 307–309. DOI: https://doi.org/10.1890/110280
Kirk, R. 2007. Statistics: an introduction. Nelson Education. DOI: https://doi.org/10.1007/978-0-230-21274-9_1
Krippendorff, K. 2004. Reliability in Content Analysis: Some Common Misconceptions and Recommendations. Human Communication Research, 30(3): 411–433. DOI: https://doi.org/10.1093/hcr/30.3.411
Land-Zandstra, AM, Devilee, JLA, Snik, F, Buurmeijer, F and van den Broek, JM. 2016. Citizen science on a smartphone: Participants’ motivations and learning. Public Understanding of Science, 25(1): 45–60. DOI: https://doi.org/10.1177/0963662515602406
Lave, J and Wenger, E. 1991. Google-Books-ID: CAVIOrW3vYAC. Situated Learning: Legitimate Peripheral Participation. Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511815355
Lawson, MA and Lawson, HA. 2013. New Conceptual Frameworks for Student Engagement Research, Policy, and Practice. Review of Educational Research, 83(3): 432–479. DOI: https://doi.org/10.3102/0034654313480891
Liberatore, A, Bowkett, E, MacLeod, CJ, Spurr, E and Longnecker, N. 2018. Social Media as a Platform for a Citizen Science Community of Practice. Citizen Science: Theory and Practice, 3(1): 3. DOI: https://doi.org/10.5334/cstp.108
Lichtveld, M, Kennedy, S, Krouse, RZ, Grimsley, F, El-Dahr, J, Bordelon, K, Sterling, Y, White, L, Barlow, N, DeGruy, S, Paul, D, Denham, S, Hayes, C, Sanders, M, Mvula, MM, Thornton, E, Chulada, P, Mitchell, H, Martin, WJ, Stephens, KU and Cohn, RD. 2016. From Design to Dissemination: Implementing Community-Based Participatory Research in Postdisaster Communities. American Journal of Public Health, 106(7): 1235–1242. DOI: https://doi.org/10.2105/AJPH.2016.303169
Long, JW, Ballard, HL, Fisher, LA and Belsky, JM. 2016. Questions That Won’t Go Away in Participatory Research. Society & Natural Resources, 29(2): 250–263. DOI: https://doi.org/10.1080/08941920.2015.1024368
Marin, G and Marín, BV. 1991. Hispanics: Who are they? In: Research with Hispanic Populations. Research with Hispanic populations. Newbury Park: United States of America, California, Newbury Park: SAGE Publications, Inc. p. DOI: https://doi.org/10.4135/9781412985734
Mason, M. 2010. Sample Size and Saturation in PhD Studies Using Qualitative Interviews. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 11(3). DOI: https://doi.org/10.17169/fqs-11.3.1428
May, ML, Bowman, GJ, Ramos, KS, Rincones, L, Rebollar, MG, Rosa, ML, Saldana, J, Sanchez, AP, Serna, T, Viega, N, Villegas, GS, Zamorano, MG and Ramos, IN. 2003. Embracing the local: enriching scientific research, education, and outreach on the Texas-Mexico border through a participatory action research partnership. Environmental Health Perspectives, 111(13): 1571–1576. DOI: https://doi.org/10.1289/ehp.5771
Measham, TG and Barnett, GB. 2008. Environmental Volunteering: motivations, modes and outcomes. Australian Geographer, 39(4): 537–552. DOI: https://doi.org/10.1080/00049180802419237
Minkler, M, Garcia, AP, Williams, J, LoPresti, T and Lilly, J. 2010. Sí Se Puede: Using Participatory Research to Promote Environmental Justice in a Latino Community in San Diego, California. Journal of Urban Health, 87(5): 796–812. DOI: https://doi.org/10.1007/s11524-010-9490-0
Minkler, M, Vásquez, VB, Tajik, M and Petersen, D. 2008. Promoting Environmental Justice Through Community-Based Participatory Research: The Role of Community and Partnership Capacity. Health Education & Behavior, 35(1): 119–137. DOI: https://doi.org/10.1177/1090198106287692
Morello-Frosch, R, Zuk, M, Jerrett, M, Shamasunder, B and Kyle, AD. 2011. Understanding The Cumulative Impacts Of Inequalities In Environmental Health: Implications For Policy. Health Affairs; Chevy Chase, 30(5): 879–87. DOI: https://doi.org/10.1377/hlthaff.2011.0153
Mugar, G, Østerlund, C, Hassman, KD, Crowston, K and Jackson, CB. 2014. Planet hunters and seafloor explorers: legitimate peripheral participation through practice proxies in online citizen science. In: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing – CSCW ’14, 109–119. 2014 Baltimore, Maryland, USA: ACM Press. DOI: https://doi.org/10.1145/2531602.2531721
National Academies of Sciences, E. 2018. Learning Through Citizen Science: Enhancing Opportunities by Design. DOI: https://doi.org/10.17226/25183
Newman, G, Newman, S, Wiggins, A, Crall, A, Graham, E and Crowston, K. 2012. The future of citizen science: emerging technologies and shifting paradigms. Frontiers in Ecology and the Environment, 10(6): 298–304. DOI: https://doi.org/10.1890/110294
O’Fallon, LR and Dearry, A. 2002. Community-based participatory research as a tool to advance environmental health sciences. Environmental Health Perspectives, 110(Suppl 2): 155–159. DOI: https://doi.org/10.1289/ehp.02110s2155
Ottinger, G. 2010. Buckets of Resistance: Standards and the Effectiveness of Citizen Science. Science, Technology, & Human Values, 35(2): 244–270. DOI: https://doi.org/10.1177/0162243909337121
Penner, LA. 2002. Dispositional and Organizational Influences on Sustained Volunteerism: An Interactionist Perspective. Journal of Social Issues, 58(3): 447–467. DOI: https://doi.org/10.1111/1540-4560.00270
Perovich, LJ, Ohayon, JL, Cousins, EM, Morello-Frosch, R, Brown, P, Adamkiewicz, G and Brody, JG. 2018. Reporting to parents on children’s exposures to asthma triggers in low-income and public housing, an interview-based case study of ethics, environmental literacy, individual action, and public health benefits. Environmental Health, 17(1): 48. DOI: https://doi.org/10.1186/s12940-018-0395-9
Phillips, T, Minarchek, M, Porticella, N, Shirk, J, Wilderman, C, Ellenbogen, K and Bonney, R. 2014. DEVISE: Building evaluation capacity in PPSR and Informal Science Education. Cornell Lab of Ornithology: Ithaca, NY, USA.
Phillips, TB, Ballard, HL, Lewenstein, BV and Bonney, R. 2019. Engagement in science through citizen science: Moving beyond data collection. Science Education, 103(3): 665–690. DOI: https://doi.org/10.1002/sce.21501
Pocock, MJO, Tweddle, JC, Savage, J, Robinson, LD and Roy, HE. 2017. The diversity and evolution of ecological and environmental citizen science. PloS One, 12(4): e0172579. DOI: https://doi.org/10.1371/journal.pone.0172579
Raddick, MJ, Bracey, G, Gay, PL, Lintott, CJ, Cardamone, C, Murray, P, Schawinski, K, Szalay, AS and Vandenberg, J. 2013. Galaxy Zoo: Motivations of Citizen Scientists. arXiv:1303.6886 [astro-ph, physics:physics].
Rambonnet, L, Vink, SC, Land-Zandstra, AM and Bosker, T. 2019. Making citizen science count: Best practices and challenges of citizen science projects on plastics in aquatic environments. Marine Pollution Bulletin, 145: 271–277. DOI: https://doi.org/10.1016/j.marpolbul.2019.05.056
Ramírez, DM, Ramírez-Andreotta, MD, Vea, L, Estrella-Sánchez, R, Wolf, AMA, Kilungo, A, Spitz, AH and Betterton, EA. 2015. Pollution Prevention through Peer Education: A Community Health Worker and Small and Home-Based Business Initiative on the Arizona-Sonora Border. International Journal of Environmental Research and Public Health, 12(9): 11209–11226. DOI: https://doi.org/10.3390/ijerph120911209
Ramirez-Andreotta, MD, Brody, J, Lothrop, N, Loh, M, Beamer, P and Brown, P. 2016. Reporting back environmental exposure data and free choice learning. Environmental Health, 15(1): 2. DOI: https://doi.org/10.1186/s12940-015-0080-1
Ramirez-Andreotta, MD, Brusseau, ML, Artiola, J, Maier, RM and Gandolfi, AJ. 2015. Building a co-created citizen science program with gardeners neighboring a superfund site: The Gardenroots case study. International Public Health Journal, 7(1).
Ramirez-Andreotta, MD, Brusseau, ML, Artiola, JF, Maier, RM, Gandolfi, AJ, Caron, RM and Merrick, J. 2014. Building a co-created citizen science program with gardeners neighboring a superfund site: The Gardenroots case study.
Ramirez-Andreotta, MD, Brusseau, ML, Beamer, P and Maier, RM. 2013. Home gardening near a mining site in an arsenic-endemic region of Arizona: Assessing arsenic exposure dose and risk via ingestion of home garden vegetables, soils, and water. Science of the Total Environment, 454–455: 373–383. DOI: https://doi.org/10.1016/j.scitotenv.2013.02.063
Rodenbeck Sven, E and Maslia Morris, L. 1998. Groundwater Modeling and GIS to Determine Exposure to TCE at Tucson. Practice Periodical of Hazardous, Toxic, and Radioactive Waste Management, 2(2): 53–61. DOI: https://doi.org/10.1061/(ASCE)1090-025X(1998)2:2(53)
Rotman, D, Hammock, J, Preece, J, Hansen, DL, Boston, CL, Bowser, A and He, Y. 2014. Motivations Affecting Initial and Long-Term Participation in Citizen Science Projects in Three Countries. In: 2014. DOI: https://doi.org/10.9776/14054
Ryan, RL, Kaplan, R and Grese, RE. 2001. Predicting Volunteer Commitment in Environmental Stewardship Programmes. Journal of Environmental Planning and Management, 44(5): 629–648. DOI: https://doi.org/10.1080/09640560120079948
Sandhaus, S, Kaufmann, D and Ramirez-Andreotta, M. 2018. Public participation, trust and data sharing: gardens as hubs for citizen science and environmental health literacy efforts. International Journal of Science Education, Part B, 9(1): 54–71. DOI: https://doi.org/10.1080/21548455.2018.1542752
Saunders, B, Sim, J, Kingstone, T, Baker, S, Waterfield, J, Bartlam, B, Burroughs, H and Jinks, C. 2018. Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & Quantity, 52(4): 1893–1907. DOI: https://doi.org/10.1007/s11135-017-0574-8
Saxton, DI, Brown, P, Seguinot-Medina, S, Eckstein, L, Carpenter, DO, Miller, P and Waghiyi, V. 2015. Environmental health and justice and the right to research: institutional review board denials of community-based chemical biomonitoring of breast milk. Environmental Health: A Global Access Science Source, 14: 90. DOI: https://doi.org/10.1186/s12940-015-0076-x
Scammell, MK. 2010. Qualitative Environmental Health Research: An Analysis of the Literature, 1991–2008. Environmental Health Perspectives, 118(8): 1146–1154. DOI: https://doi.org/10.1289/ehp.0901762
Scammell, MK, Senier, L, Darrah-Okike, J, Brown, P and Santos, S. 2009. Tangible evidence, trust and power: Public perceptions of community environmental health studies. Social Science & Medicine, 68(1): 143–153. DOI: https://doi.org/10.1016/j.socscimed.2008.10.002
Scott, DN. 2016. ‘We Are the Monitors Now’: Experiential Knowledge, Transcorporeality and Environmental Justice. Social & Legal Studies, 25(3): 261–287. DOI: https://doi.org/10.1177/0964663915601166
Shirk, JL, Ballard, HL, Wilderman, CC, Phillips, T, Wiggins, A, Jordan, R, McCallie, E, Minarchek, M, Lewenstein, BV, Krasny, ME and Bonney, R. 2012. Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society, 17(2). DOI: https://doi.org/10.5751/ES-04705-170229
Soleri, D, Long, J, Ramirez-Andreotta, M, Eitemiller, R and Pandya, R. 2016. Finding Pathways to More Equitable and Meaningful Public-Scientist Partnerships. Citizen Science: Theory and Practice, 1(1). DOI: https://doi.org/10.5334/cstp.46
Sorensen, AE, Jordan, R, Blaise, G, Brown, J, Campbell, L, Aronson, M and Johnson, M. 2018. Drivers of Public Participation in Urban Restoration Stewardship Programs: Linkages Between Environmental Identity and Knowledge, and Motivations. Arboriculture & Urban Forestry, 44: 266–282.
Sorensen, AE, Jordan, RC, LaDeau, SL, Biehler, D, Wilson, S, Pitas, J-H and Leisnham, PT. 2019. Reflecting on Efforts to Design an Inclusive Citizen Science Project in West Baltimore. Citizen Science: Theory and Practice, 4(1): 13. DOI: https://doi.org/10.5334/cstp.170
Tashakkori, A and Teddlie, C. 2020. The SAGE Handbook of Applied Social Research Methods. In: 2nd ed. Thousand Oaks, California: SAGE Publications, Inc. p. DOI: https://doi.org/10.4135/9781483348858
Tracy, SJ. 2010. Qualitative Quality: Eight “Big-Tent” Criteria for Excellent Qualitative Research. Qualitative Inquiry, 16(10): 837–851. DOI: https://doi.org/10.1177/1077800410383121
Trumbull, DJ, Bonney, R, Bascom, D and Cabral, A. 2000. Thinking scientifically during participation in a citizen-science project. Science Education, 84(2): 265–275. DOI: https://doi.org/10.1002/(SICI)1098-237X(200003)84:2<265::AID-SCE7>3.0.CO;2-5
U.S. Department of Health and Human Services, Agency for Toxic Substances and Disease Registry. 1998. Health Consultation: Dioxin contaminated Aerial Spraying Landing Locations Kellner, Icehouse, and Sixshooter Canyons, 9.
Vousden, CL, Sapru, S and Johnson, JE. 2014. Communicating About Biomonitoring and the Results of a Community -Based Project: A Case Study on One State’s Experience. Journal of Environmental Health; Denver, 77(5): 20–26.
Wallerstein, N and Duran, B. 2010. Community-Based Participatory Research Contributions to Intervention Research: The Intersection of Science and Practice to Improve Health Equity. American Journal of Public Health, 100(S1): S40–S46. DOI: https://doi.org/10.2105/AJPH.2009.184036
Walpole, M. 2003. Socioeconomic Status and College: How SES Affects College Experiences and Outcomes. The Review of Higher Education, 27(1): 45–73. DOI: https://doi.org/10.1353/rhe.2003.0044
West, S and Pateman, R. 2016. Recruiting and Retaining Participants in Citizen Science: What Can Be Learned from the Volunteering Literature? Citizen Science: Theory and Practice, 1(2). DOI: https://doi.org/10.5334/cstp.8
Wilson, S, Campbell, D, Dalemarre, L, Fraser-Rahim, H and Williams, E. 2014. A Critical Review of an Authentic and Transformative Environmental Justice and Health Community — University Partnership. International Journal of Environmental Research and Public Health, 11(12): 12817–12834. DOI: https://doi.org/10.3390/ijerph111212817