Start Submission

Reading: Reflecting on Efforts to Design an Inclusive Citizen Science Project in West Baltimore


A- A+
Alt. Display

Case Studies

Reflecting on Efforts to Design an Inclusive Citizen Science Project in West Baltimore


Amanda E. Sorensen ,

University of Nebraska-Lincoln, 3310 Holdrege St, Hardin Hall Room 504, Lincoln, NJ, US
X close

Rebecca C. Jordan,

Rutgers University, 59 Lipman Drive, Waller Hall 104, New Brunswick, NJ, US
X close

Shannon L. LaDeau,

The Cary Institute of Ecosystem Studies, 2801 Sharon Turnpike, Millbrook, NY, US
X close

Dawn Biehler,

University of Maryland-Baltimore County, 1000 Hilltop Circle, 211-L Sondheim Hall, Baltimore, MD, US
X close

Sacoby Wilson,

University of Maryland, Department of Environmental Science and Technology, College Park, Maryland, US
X close

John-Henry Pitas,

University of Maryland-Baltimore County, 1000 Hilltop Circle, 211-L Sondheim Hall, Baltimore, MD, US
X close

Paul T. Leisnham

University of Maryland, Department of Environmental Science and Technology, College Park, Maryland, US
X close


Citizen science (CS) has been an increasingly utilized means by which scientists leverage members of the public to increase the amount of data collected and analyzed. However, the underrepresentation of individuals from certain socio-cultural groups can have consequences that can manifest in the scientific outcomes of those CS projects such as biases in the data. Additionally, this underrepresentation can potentially affect long-term viability and support of CS as a community of practice. CS programs that promote greater inclusivity would likely provide opportunities for communities to define, investigate, and address pressing issues in collaboration with professional scientists. In this paper we discuss a CS project that sought to include underrepresented communities in Baltimore, Maryland using Pandya’s framework for inclusive CS. While the project met all of its scientific research goals, translating the CS for broader social outcomes in the community proved challenging. Here we highlight perspectives from local community members and research personnel about the barriers to CS engagement, challenges in translating scientific outcomes to social justice efforts, and opportunities to address these barriers in CS program development and design.

How to Cite: Sorensen, A.E., Jordan, R.C., LaDeau, S.L., Biehler, D., Wilson, S., Pitas, J.-H. and Leisnham, P.T., 2019. Reflecting on Efforts to Design an Inclusive Citizen Science Project in West Baltimore. Citizen Science: Theory and Practice, 4(1), p.13. DOI:
  Published on 08 Mar 2019
 Accepted on 28 Dec 2018            Submitted on 01 May 2018


The number of scientific research projects that use citizen science (hereafter CS) in some capacity has increased significantly over the past few decades (Conrad and Hilchey 2011; Follett and Strezov 2015). CS has the potential to democratize science (USEPA 2016) by making the scientific process participatory and inclusive of many stakeholders in knowledge generation, dissemination, and decision-making (Jordan et al. 2017; McCormick 2007). Furthermore, CS may confer broader benefits to communities (environmental democracy, scientific literacy, citizen inclusion in local issues) and local ecosystems (conservation and biodiversity efforts) (Ballard et al. 2018; Cooper et al. 2007; Conrad and Hilchey 2011; Larese-Casanova and Prysby 2018; Mueller et al. 2012). However, while studies have shown positive outcomes for CS participants in terms of scientific learning (Evans et al. 2005; Jordan et al. 2011; Phillips et al. 2018) and civic engagement (Dunlap 1992; Marcinkowski 1993; Nerbonne and Nelson 2004), participation in CS does not reflect the broader demographics of the US (Pandya 2012). While there has been no formal meta-analysis of representation in citizen science, a recent report on diversity in CS has found that participants tend to include white and more well-educated individuals (NASEM 2018). Other work has found that younger individuals (post K-12 aged adults) engage less in CS than older individuals (typically of retirement age) (Merenlender et al. 2016).

Meanwhile, CS as a field has grown up in parallel with but often disconnected from the scholarly field (and activist practice) of environmental justice (EJ), which inherently stresses the role of communities of color in defining environmental science questions for the purpose of improving health and environmental self-determination. Corburn (2005) explores the potential for CS to help engage traditionally marginalized communities in data collection and advocacy in the context of environmental and social justice. Instead of being viewed largely as recipients of science, with focused outreach, marginalized people also can be stakeholders, participants, or even equal partners (Dhillon 2017; Heaney et al. 2007) in the generation and use of scientific evidence. The field of EJ has long been attentive to questions of who participates in science, and CS has begun to move in this direction as well (Pandya 2012).

In this manuscript, we explore issues of representation in and community benefit from CS through the lens of a recent CS project, the Baltimore Mosquito Study. This project aimed to engage the diverse communities of Baltimore, Maryland in addressing local environmental and social issues. The project was created with the explicit intention of working with citizen scientists to collect data about an invasive mosquito population, and then translating those scientific outcomes to actions with local community groups to help address environmental justice issues within West Baltimore. This paper is a reflection on the experience of developing and implementing this CS project, highlighting discussions with project personnel and local community members about the barriers to CS engagement, challenges in translating scientific outcomes to broader community benefits, opportunities to address social inequities in science, and recommendations to help address representation issues in CS.


Often intertwined in the goal of democratizing science through CS is an intent to conduct meaningful scientific research that can inform and address environmental injustice (Corburn 2005) through advocacy and policy efforts (Robertson and Hull 2001). Environmental injustice can take many forms; at a basic level, it includes unequal burdens of environmental hazards (such as landfills, incinerators, Toxic Release Inventory sites, industrial livestock production) and unequal access to environmental amenities (such as parks) across geographies, communities, and populations, most strongly affecting communities of color (Abara et al. 2012; Lerner 2005; Wilson 2009). The scholarly field of environmental justice—and EJ activists—insist that it is not only the geographic and demographic distribution of hazards and amenities that matters; also important are the political and social processes by which communities are allowed to control their environmental fate or are deprived of control (Holifield 2001). In many cities across the globe, government and private agencies aim to transform overburdened and underserved neighborhoods through urban “greening” efforts, i.e., increasing green spaces, including tree canopy, to ameliorate heat island effects and storm water impacts on water quality (Pataki et al. 2011). Urban greening may be expected to have positive effects on local property values, crime deterrence (Brown and Bentley 1993), and ecosystem functions, but outcomes are inherently more complex. For example, Troy and Grove (2008) demonstrated that residents valued shared “green spaces,” including public parks, when crime rates were low, but perceived them as a disamenity when crime rates were high because of their potential to attract and host negative activity.

To transform these communities, scholars and activists argue that informing and addressing environmental injustice cannot be done solely by institutions, but necessarily requires community participation (e.g., Carr 2004). Inclusive CS, building on the experiences and knowledge of the EJ field, could provide opportunities for communities to define, investigate, and address pressing environmental justice problems in collaboration with professional scientists. To do so would require scientists to actively seek voices from individuals in the places and institutions that are not often invited to the decision-making process but who often have a high stake in the outcomes. In an effort to address these issues of inclusion and representation in CS, Pandya (2012) proposed a general framework for designing CS projects that align with community priorities to increase inclusion. This framework involves five actions for CS project development and implementation: (1) aligning research and education with community priorities, (2) planning for co-management of the project, (3) engaging the community at every step, (4) incorporating multiple kinds of knowledge, and (5) disseminating results from the work widely (outside of scientific publication). In this manuscript, we discuss and reflect on the successes and challenges in implementing Pandya’s inclusive CS framework from the perspective of both project personnel and the community in the process of designing and implementing a CS project.

Baltimore Mosquito Study Project Description

The Baltimore Mosquito Study (BMS) began with the aim to investigate the feedbacks between resident use and management of local green spaces, mosquito ecology, and human exposure to mosquito bites. The research further examined variation in these coupled natural-human dynamics across neighborhoods with varying socio-economic status and experiences of urban disinvestment in Baltimore, Maryland (Biehler et al. 2018). Disinvestment – specifically as part of racist redlining and urban renewal policies – has led to population decline, property abandonment, neglect by city government, and exploitation of the neighborhood as a site for illegal dumping. Amid these historical and ongoing dynamics, the BMS was designed to engage focal communities in discussions and activities aimed at better understanding the factors that influence resident use, management, and valuation of neighborhood green spaces, with specific attention to the role of perceived and measured mosquito abundances. This project was particularly timely due to the relatively recent establishment of the daytime biting mosquito, Ae. albopictus (tiger mosquito) (LaDeau et al. 2013) and rising interest in green infrastructure solutions for urban environmental issues in the city of Baltimore (Bodnaruk et al. 2017; Cole et al. 2016; Grove et al. 2018). Urban mosquito ecology in Baltimore, MD is predominantly driven by the abundance and persistence of human-made container habitats (LaDeau et al. 2013; Little et al. 2017), and the most common mosquito species detected is Ae. albopictus. Since its introduction to the US in the mid-1980s, Ae. albopictus has emerged as the predominant human biting mosquito in eastern US cities (Goodman et al. 2018; Gratz 2004; Leisnham et al. 2011). Like other peridomestic species, juvenile Ae. albopictus use a range of artificial water-holding container habitats (Hawley 1988). Its establishment in urban areas has resulted in increased reported complaints, owing to its aggressive day-time biting behavior and the ineffectiveness of conventional abatement methods (Bodner et al. 2016; Hawley 1988; LaDeau et al. 2013). Container habitat is distributed heterogeneously across neighborhoods, correlating strongly with patterns of property abandonment (LaDeau 2013; Little et al. 2017). Control efforts for Ae. Albopictus focus primarily on juvenile source removal (removing water-retaining elements within the landscape such as trash and abandoned buildings) (Bartlett-Healy et al. 2012).

The five focal neighborhoods of BMS were located in West Baltimore. These neighborhoods were chosen for this project because of their close geographic proximity (within 2km) to one another, similarity in proximity to major landscape features (i.e., the harbor and other major naturalistic areas), and representation of the gradient in socio-economic status (SES) (low, median, high household income) across Baltimore (see Figure 1). The neighborhoods in this research were identified using online data from the US Census Bureau and Baltimore City ( and The median household income of Baltimore at the inception of this project was $41,385. Focal blocks within these neighborhoods were randomly selected from all the blocks within each neighborhood that were classified primarily as residential (as opposed to industrial, school zoning, or high volume residential). Some of the blocks within the low SES neighborhoods were characterized by high rates of housing abandonment, multi-family dwellings, illegal garbage dumping (often by individuals outside of the community), and general neglect. The high SES neighborhoods are characterized by predominately single-family homes with individual yards, maintained public outdoor spaces, and generally gentrified communities.

The BMS collected data on mosquito ecology, human responses, and environmental conditions. We found that unmanaged infrastructure and associated trash in lower income neighborhoods, where disinvestment had limited both trash collection services and redevelopment, contributed to greater adult mosquito production because it provided favorable habitat for all life-stages (LaDeau et al. 2013; Little et al. 2017). Many trash containers provide receptacles for rain water that are routinely utilized by Aedes for their developmental stages (e.g., eggs, larvae) and can produce large abundances of biting adults that can then, presumably, rest in shaded habitats around the abandoned infrastructure and overgrown vegetation. The study further evaluated specific intervention strategies for managing the mosquito infestation and engaging local residents in managing and enjoying their green spaces. As Troy and Grove (2008) might predict, many Baltimore residents expressed suspicion and irritation toward efforts to bring green infrastructure into their neighborhoods, citing concerns about crime, pest production, and garbage (Biehler et al. unpublished).

Figure 1 

Neighborhoods that participated in the BMS project are denoted with a black outline. These neighborhoods were composed of primarily residential areas and represented the socio-economic gradient of West Baltimore. L = Low socio-economic status, M = Median socio-economic status, H = High socio-economic status. See Little et al. (2017) Supplementary Material for further information on methods used for categorizing demographic and economic data in neighborhood identities.

As part of the broader BMS, citizen scientists collected mosquito nuisance and presence data and identified potential breeding habitats. The protocol involved participants designating a space in their neighborhood to monitor each month during active mosquito months (June, July, August) for potential larval mosquito habitat, describing individual mosquito nuisance rates and perceived mosquito levels in their community (see Jordan et al. 2017 for full project protocol and description). Citizen scientists for this project were recruited from the focal neighborhoods at neighborhood events, at local neighborhood organization meetings, and through word of mouth during 2014 and 2015 (Figure 2). Over these two years, 70 citizen scientists were recruited and 37 individuals completed the data collection. Below we highlight Pandya’s five actions and how the BMS sought to engage a diverse and underserved community in inclusive CS.

Figure 2 

One of the Citizen Science program leaders, Rebecca Jordan (center), enrolling a new participant at a local community block party in West Baltimore. Photo Credit: Amanda Sorensen.

Implementing Inclusive CS

Aligning research and education with community priorities

We worked with community-based organizations to both market and advise on the scope of the project to ensure that the research goals aligned with community interests. For example, we worked with Parks & People Foundation, which has been active in implementation of green infrastructure and community engagement in West Baltimore for several decades. Prior to the first year of ecological research (the mosquito-based sampling), project leaders met with local community groups within each of the focal neighborhoods to help align project research efforts with community interests. In these meetings, project leaders sought to establish community priorities in the context of invasive mosquitoes and how this project could address their priorities. Through these conversations with community groups and individuals, four main priorities emerged: (1) making accessible outdoor open spaces healthy and appealing; (2) alleviating the burden of mosquito exposure in disinvested communities; (3) targeted reinvestment in disinvested communities with substantial participation by residents (i.e., revitalization of vacant properties according to residents’ planning priorities including affordable housing along with safe and high-quality green space); and (4) improvement of city sanitation services. Considering these priorities, the project began with a direct focus on priorities 1 and 2, with the expectation that addressing mosquito problems within the community would necessarily involve priorities 3 and 4 as we believed these to be related to mosquito presence.

Planning for co-management of the project and engaging the community at every step

To promote co-management of the project, CS block leaders were recruited from within the community as liaisons to the project to work in partnership with project personnel on the CS efforts. In this role, block leaders recruited others to participate, collected CS data, gave feedback on project progress, and communicated project results at community meetings. Block leaders were compensated (US $100/month during the active research months) for their contributions to the project. Two of the co-authors who managed the CS program met quarterly in-person with block leaders to discuss their reflections on the project. In addition to block leaders sharing on project progress, another team member who served as the main community point of contact attended monthly community organization meetings and met with project participants to discuss and solicit feedback on the project itself.

As a result of feedback from block leaders and community members, project leaders regularly refined CS methods and developed new project elements that were in line with community interests. For example, the CS data collection protocol was developed during the first year through an iterative feedback process with participants and the mosquito scientists, negotiated by the CS managers. In response to participant feedback, a new data submission process was implemented to ensure greater accessibility to all (i.e., online and paper data submission with pre-stamped return envelopes for those who had minimal Internet access). In addition, a tire drive program was developed and neighborhood clean-up events were held by the project staff, initiated by the interest of the community. A few dedicated citizen scientists co-generated with project personnel a civic ecology project (see Jordan et al. 2018 for full civic ecology description). This civic ecology effort was possible because the CS engagement was flexible in that project personnel could adjust activities and goals based on the interests and motivations of the residents. The civic ecology project operated in parallel to the main CS effort, with some individuals participating in both. Finally, the project also developed a small grants program in partnership with Parks & People Foundation, the local non-governmental organization (NGO) in West Baltimore. Through this program, participants and community members applied for one of four small grants (US $1,500 each) to enact community improvement and beautification efforts related to reducing habitat for mosquitoes.

Incorporating multiple kinds of knowledge

There were two primary modes of citizen participation in the BMS, contributory CS and Photovoice. To address the question of mosquito abundance requires spatially explicit knowledge about mosquito distribution throughout the city and potential breeding habitats (Jordan 2017). Because these data are difficult to collect across a large heterogeneous landscape, this research lent itself to a contributory citizen science model (see Shirk et al. 2012 for levels of CS contributions) and required participation of residents across the neighborhoods to inform understanding of mosquito persistence across the landscape. This distributed data collection network was critical in understanding mosquito distribution across the urban landscape for the mosquito biologists.

To address the question of resident use, management, and valuation of neighborhood greenspace and how that interplays with mosquitoes, an additional aspect of the project engaged community members in Photovoice. Photovoice is a qualitative community-based participatory research method to document and reflect an individuals’ experience within a community (Wang et al. 1997). Two co-authors brought their expertise using the Photovoice methodology to this project from other environmental justice projects (Aber et al. 2017; Brandt et al. 2017; Chanse et al. 2017). Outcomes of the Photovoice research effort are discussed elsewhere (Biehler et al. in prep). Further, the photos that were taken were shown at a community event described below.

Disseminating results from the work widely (outside of scientific publication)

Numerous beneficial professional scientific outcomes have resulted from engaging citizen scientists in this research. In this work, we found that CS mosquito nuisance data reported during peak mosquito months was a reliable measure of relative abundance of mosquitoes found by professional scientists (Jordan et al. 2017). This finding has implications for how scientists and professional managers (e.g., Department of Environmental Protection, Department of Health, or state/local mosquito commissions) monitor and manage mosquitoes. If, as our work suggests, CS nuisance data are reliable measures of mosquito abundance, more scientists and professionals may turn to CS as a supplement, or even alternative, in their mosquito monitoring efforts.

Within the community, the project followed multiple avenues to disseminate results. The results and discussions from the Photovoice aspect of the project were presented by project personnel and participants at focal neighborhood meetings. These presentations generated numerous fruitful discussions about perceptions of the neighborhood and resulting actions that the neighborhood associations could take in response. Additionally, the Photovoice results were presented at a community environmental art exhibit in the Fall of 2015. Photovoice participants were invited to present their work as a part of this exhibit, which brought in residents across Baltimore. Finally, in the civic ecology effort, the citizen scientists worked with project personnel to understand how the CS research fit into the broader picture of environmental justice issues facing the community. Out of this work, members of the civic ecology effort developed plans for a beautification project to encourage people to dispose of waste properly and monitor whether these types of interventions can influence behavior to meet both goals (reducing waste in the neighborhood and reducing mosquito populations) (see Jordan et al. 2018 for further information).

Project Reflections: Community Perceptions

Throughout the project, reflections from CS participants and community members were collected by the two co-authors who led the CS effort, and a third co-author who was the main community point of contact, using an interpretivist qualitative approach (Lapan et al. 2011). The two co-authors took detailed field notes during all meetings and communications with participants and noted direct quotes from participants. The third co-author took detailed field notes during all of the monthly meetings that he attended and one-on-one meetings with community members, noting direct quotes from participants and characterizing participant attitudes and thoughts toward the project and the mosquitoes. From the summary of all of these reflections and identifying emergent themes, it is clear that despite numerous meaningful scientific outcomes, perceptions of the project were mixed.

One theme that emerged from these meetings was that the topic of the research, mosquitoes, was generally low on the priority list for communities, particularly in lower SES neighborhoods. During initial community meetings at project onset, community members helped to set the four priorities discussed above, one of which specifically focused on alleviating mosquito burdens. However, project onset overlapped in time with the death of Freddie Gray – a young man from an adjacent neighborhood – from injuries sustained while in police custody, and the subsequent uprising of Baltimore residents to protest police brutality and surveillance. As one participant noted, “mosquitoes ain’t a big issue for people when there have been 30 homicides this month. There is a gang war out there!” This particular participant noted his frustration about the focus of the project numerous times, highlighting the greater needs and issues that the community was facing. Because the grant and research capacity focused on the socio-ecological drivers of mosquitoes, some participants felt that the project was abstract and trivial given the current social climate. This lack of alignment with perceived community needs led some to suggest that the project was another example of them being “over studied” by scientists, or being treated like an “object” in research rather than collaborators in research. The community point of contact co-author found that when he was given the opportunity to report about the progress of the project, community members often saw these findings as irrelevant to their community development goals. The community point of contact found that if he discussed the mosquito research findings in the context of broader issues like vacancy, dumping, or health risks associated with mosquitoes, community members saw greater value in the research. Interestingly, toward the end of this phase of the project in late 2015, the Zika virus pandemic became international news. During this time, the community point of contact noticed that individuals in the community were significantly more receptive to the project overall and expressed greater interest in the implications of the mosquito research on their community. The CS project leaders also saw a substantial increase in participation interest from residents during this same time. The shifts in community priorities, given the projects’ research commitments, posed a challenge for project personnel to address the priorities set during initial meetings while being responsive to changes in community interests and needs.

Although there was a significant inverse relationship between mosquito abundance and SES (mosquito abundance was higher in lower SES areas) (Little et al. 2017), residents in higher SES neighborhoods reported being more bothered by mosquitoes (LaDeau et al. 2013). Door-to-door surveys showed that increasing median income was associated with increased desire to use yards and other shared green spaces for recreation (LaDeau et al. unpublished data). Residents who reported the greatest mosquito nuisance were those who wanted green spaces, while residents in the lowest SES neighborhoods were more likely to avoid lots with high vegetation for reasons other than mosquitoes. In median and high SES neighborhoods, there were more manicured outdoor greenspace that retained water (e.g., bird baths, potted plants, decorative landscape features) that allow mosquitoes to breed and the population to persist in particularly dry periods. Indeed, individuals in the high SES neighborhoods frequently expressed frustration to team members that they could not enjoy their manicured outdoor space in the dry summer due to mosquito presence. Therefore, because of this dynamic of resident greenspace use, the project may have inadvertently been seen as addressing the interests of higher SES communities, though we aimed to approach all aspect of the project with the intention of focusing on disinvested (lower SES) communities.

Additionally, participants noted that one of the major barriers to ongoing community participation in the CS research was the effects of data collection itself. Part of the data collection protocol asked participants to report monthly on potential mosquito habitats, which, in this community, take the form of abandoned buildings, litter, and illegal dumping from outside entities. Using data on trash issues throughout the neighborhoods, community members were encouraged to contact city services (calling 311 to report trash and request the city to clean it up) as a part of translating data to on-the-ground outcomes. The community point of contact noted that participants began to express fatigue over the month-to-month meetings, as the individuals felt increasingly frustrated that they kept noticing, and calling about, the same trash piles and abandoned buildings but nothing changed despite their efforts. One participant said during a monthly meeting “I stopped 311-ing a while ago … [it’s just the] same issues as before, Mount Claire St and Boyd St has a lot of trash, dumping and overgrowth near the community garden. It is mostly construction trash, it has been called in and it was removed, to be replaced with more construction waste.” Another participant noted how the trash problem was worsening and reported that “trash [was] being set on fire [in the] 1200 block Harlem Avenue.” In this instance, despite the good intentions to translate data to tangible outcomes, the lack of response from the city to the citizens’ efforts became a negative reinforcement and reduced their sense of agency in this context. The community point of contact noted, “It’s almost like rehashing it all with me once a month made them realize how the same issues kept cropping up over and over again and it was frustrating for them (the participants) to come to that realization.” Meanwhile, when two of the project personnel met with city agency staff, the staff insisted that the 311-complaint line was the best and only way for residents to report environmental problems such as illegal dumping and mosquito infestations. The city’s emphasis on resident reporting of individual complaints runs counter to the civic aims of engaged CS and environmental justice approaches, which instead emphasize collective action against systematic inequities.

Project Reflections: Personnel

In reflections from project leaders and personnel, it was clear that the project faced many obstacles to becoming truly inclusive. There were many successes in terms of professional scientific outcomes (i.e., publications, novel research findings, increased scale and scope of mosquito data) as well as inclusive CS outcomes (i.e., community-led beautification efforts, civic ecology project, photovoice demonstrations), which we attribute to following and implementing Pandya’s inclusive CS framework. However, there were some challenges in translating the scientific research for social outcomes. A major challenge identified by most project leaders was the research funding timeline. In national competitive grants, there is minimal capacity to generate a research project in partnership with community members that truly aligns with community and scientific research goals. One project leader noted, “The funding and publication demands of inter/transdisciplinary research do not match those of federal granting or university level expectations both in terms of time, resources, and with respect to project ownership.” This sentiment was echoed by another project leader, who remarked “The mechanism of getting and retaining funding (short-term competitive grants with really low success rates) is not really amenable to engaging multiple stakeholders in project planning and sustainability.” A third project leader noted that the struggle to truly integrate community members into grant-funded research is because “in this kind of work, we had to define clear scientific/academic goals in the proposal – which is what we were funded to accomplish. However, community participation is such an integral part of defining and translating science for social justice efforts and we weren’t able to fully accomplish that level of engagement.” As most research institutions expect their faculty to be actively pursuing and securing grants, co-generating CS research programs that successfully meet the scientific research and community needs—given the constraints highlighted above—is particularly challenging. One project leader noted, “This speaks to the pressure and culture of academia to overemphasize and over-incentivize scientific inquiry and knowledge production and underemphasize community engagement and translation of research to action to actually solve problems.” Researchers such as Heaney et al. (2007) have identified such challenges and advanced a model of Community-Owned and -Managed Research (COMR) as an alternative, although this model also fits awkwardly with many institutional demands and has been slow to gain traction among major scholarly funders.

Another major challenge noted by leaders was building rapport with the community. As with many CS projects, project leaders were coming to the community as outsiders. While the project did have consistent and engaged project personnel, building trust with individuals takes time. Issues of trust-building are not unique to this project, but project personnel felt that the lack of a strong foundation of trust made this research in particular very difficult. One project leader, in discussing integrative CS for transdisciplinary research said “… adaptive change can be really tough especially when we [project leaders and personnel] are not members of these social groups.” This lack of foundational trust made disappointing developments within the project difficult to translate to the community. For instance, our research confirmed that vacant buildings and sanitation problems were related to mosquito problems, which could be factors in feedback loops influencing disinvestment from the city and accessibility to outdoor greenspaces. However, these findings were not as effective in addressing the priorities of the community as was hoped at project onset. This was summed up in one project leaders’ reflections saying,

“The trouble is that it seems we can’t solve the mosquito problem as long as vacant buildings are prevalent, and this is an extremely costly problem that the City of Baltimore is doing too little to address. It would be great if we could translate our findings into strong recommendations to the City for more building removal/rehabilitation, but this is still a long shot; if the City is unwilling/unable to invest in solving this problem for all the other reasons that vacant buildings are bad, would adding mosquito data to the evidence against vacant buildings really help? Still, we can try, and communicating this information to policy makers and communities can at least add that evidence to their argument for doing something about vacant buildings.”

It was clear that even with our efforts, there were bigger social and political barriers to change within this community. One project leader noted that the use of the community-based participatory research (CBPR) framework to develop an authentic community-university partnership would have been a better approach in this community. (Because CBPR emerged as a theme from our work and in project leader discussions, we discuss CBPR in the context of CS in further detail in the next section.) In their reflections this individual said, “This project could have been improved by having a strong community advisory board to make sure we were getting bidirectional communication on research goals and objectives which could have led to earlier project adjustments. Mosquitoes should have been discussed in the larger context of ecological disamenity following the environmental justice framework. Other efforts related to resiliency, greening, fair development, and revitalization probably would have resonated better with local residents.”

A true CBPR effort would be challenging, however, due to some project limitations. First, we had a few logistical challenges. None of the lead project personnel lived in Baltimore, and pre-existing social networks in West Baltimore were largely unknown to the team during project conception. Given these constraints in fully implementing the CBPR framework, contributory CS was seen as the option that met the needs of the project, but in a community like West Baltimore, it meant that CS engagement and therefore the ability to translate CS outcomes to broader justice efforts was low. One project personnel speculated that, given the nature of the research (mosquitoes), CS engagement would have been low in most communities; however, in more affluent communities, pre-existing networks can make engagement and easier. This ease of engagement may be one of the drivers behind the unequal representation in CS identified in the literature.

Implications and Recommendations

While using Pandya’s framework likely made our project more successful and inclusive than it would have been otherwise, we recognize and acknowledge that efforts to translate intention to actualization are imperfect. It is clear that even though we intentionally sought to engage a community that is underrepresented in CS and to implement an inclusive CS paradigm, our efforts were not sufficient. The challenges we grappled with here and project personnel reflections emphasize that other bodies of knowledge and literature (i.e., CBPR, COMR) can be an invaluable resource in thinking about how to implement an inclusive framework. Here, we discuss insights from other relevant research areas, describe project leaders’ suggestions and recommendations, and highlight resources for future projects.

Most obvious to us as researchers was the ultimate mismatch between project demands and the requirements to create a fully inclusive CS framework such that it translates to broader environmental and social justice needs. Pandaya’s (2012) framework focuses on including community stakeholders in defining research questions that advance the communities’ priorities. In the context of the BMS, project leaders sought to align the project’s goals and implementation with this inclusive framework through community goal setting sessions, but the project was beholden to a set of research questions as a part of the grant funding. There is often little opportunity within the broader scientific funding context to include stakeholders during the project conception and development phases; therefore, this is a core challenge to developing truly inclusive CS projects, as many researchers using CS are faced with similar constraints. From our experiences, project leaders recommended, for traditional research efforts implementing a CS element into the project, building in greater flexibility into the direction of research project such that it can be responsive to community needs. This moved project personnel discussions in the direction of community-based participatory research (CBPR), advocated by many environmental justice scholars and activists. However, to enact this recommendation requires broader systemic changes and flexibility from the various institutions (governmental, academic, non-profit) that support CS research. This speaks to the necessity of (1) maintaining or bolstering Capacity funding (or federal allotments for research institutions to support research and extension programs) for land-grant colleges, as sustained long-term funding is key when you have multiple diverse stakeholders that need to work together to tackle complex problems and (2) more funding to community-based organizations, particularly groups that focus on environmental justice and health issues, (i.e., Balazs and Morello-Frosch 2013; Wilson et al. 2008) These two steps in conjunction can help build organizational and programmatic capacity to provide energy to partnerships with academic institutions for CS and CBPR efforts.

A second recommendation agreed upon by all project leaders was that projects of this nature be planned on longer timelines, which would include substantial proposal development in partnership with the community following the CBPR framework and principles. To do this would require time and resources to identify relevant community stakeholders, convene meetings, and create broader opportunities for individuals to discuss community priorities, guide the development of research questions, co-create study questions, and develop a plan for dissemination that will translate research to action through a community-driven process. However, the typical three- or five-year timeline for most conventional grant funding opportunities for ecological research and the lack of readily available support for proposal development means that unconventional sources and relationships must be developed and relied upon. One project leader recommended, “Ideally, we would all get around the table for a series of workshops and meetings and plan the project, but that takes a lot of work, time, and money. So, the current framework was that we’d have a broad outline in the proposal and amend it as the project gets going.” One recommendation to overcome this barrier was leveraging funding opportunities (e.g., National Science Foundation planning grants) that support research staff in developing connections with communities. Given the competitive funding rates for government sponsored research, in the US in particular, this cycle of working with a community to build up a grant, only to have not it funded, may reduce trust and desire to engage from local communities if researchers do not follow through with at least some of the plans developed with the community even without research funding. If the grant does not get funded, it does not mean that effort should not be continued to achieve the goals and objectives of the community-university partnership. An authentic partnership focuses on real outcomes for communities of concern.

A third recommendation was to provide more opportunities for community members to be employed in the project. In our project, CS block leaders were compensated as part of their participation, and many expressed their appreciation of our acknowledgement that we valued their contributions and time. This type of direct benefit may be more welcome by communities and should be built into CS project design. In CBPR, community-based organizations typically join the team as Co-Investigators or Co-Principal Investigators and receive their own subcontract. This helps to address inequities in power between community-based organizations and academic institutions (Wallerstein and Duran 2010; Wilson et al. 2007). A project leader discussing CBPR noted, “If the community-owned and managed research (COMR) framework (Heaney et al. 2007) was employed, we would strive for equity in funding for academic partners and community partners. In the future, we should include local community-based organizations as research partners who can act as Co-Investigators.”

Fourth, projects should find ways to build in efforts to work not just with community members, but also with the broader social structure of the community. One of the facets of inclusive CS— and CBPR— highlights community action and policy changes as an outcome (Bell-Elkins 2002; Eden 2006; Pandaya 2012). The citizen scientists regularly took action by notifying the city of dumping areas that were potential mosquito breeding grounds, but these actions were rarely successful as the city infrequently followed up in addressing them. In this case, there was a lack of capacity for the CS project to truly address community needs. The illegal dumping, abandoned houses, and general civic disinvestment in the community are not issues that individual action can solve. These types of issues would be best addressed systemically through policy, instead of piecemeal efforts such as a clean-up here and a call to 3-1-1 there. Most project leaders suggested the critical need for more work in translating the findings from this research into recommendations to the city. In our case, to really see our project succeed, we needed cooperation and engagement from the City of Baltimore and perhaps even state and federal agencies that also bear responsibility for the state of housing and disinvestment in West Baltimore. A major challenge to such cooperation, however, is the mismatch between the timelines of policy development and implementation and grant-funded research. Policies to address ecological issues can take many years to create and would benefit from persistent long-term advocacy, which scientists in their research capacity may not be able to provide. Therefore, in inclusive CS, collaboration between research projects and existing organizations (NGOs, neighborhood groups, advocacy organizations) who can continue to push for these policy efforts after the research has concluded should be planned at the outset. There are examples from CBPR literature in which scientists have participated heavily in advocacy in relation to their research (Israel et al. 2010), working in partnership with local community groups and policy makers. Integrating a Collaborative Problem Solving Model (Lasker and Weiss 2003; Wilson et al. 2007) into the research project at the outset and engaging government agencies as a stakeholder group at the different stages of the research process can further these goals.

Finally, communicating limitations of the research and our capacity as researchers is challenging. Project personnel often had difficulty communicating that even with our research on the mosquitoes, the research process (i.e., trapping mosquitoes) itself is not sufficient to get rid of mosquitoes entirely from West Baltimore. Community members often expressed frustration that the project could not implement more mosquito control measures, which may have perpetuated mistrust of the project team. Given the complexity of mosquito ecology and the limitations of pest management techniques for Ae. albopictus, it is incredibly difficult to completely eradicate and is further exacerbated by the prevalence of disamenties (i.e., abandoned buildings, illegal dumping sites) in these neighborhoods. However, some community members expressed frustration that project personnel were not doing enough and would not because their community was already neglected. With this broader political and societal dysfunction in the system that we were working in, it was difficult to find the appropriate role of CS. Where do CS projects draw the line in their efforts outside of the scientific research and priorities set at project onset (in the context of BMS, do we fulfill garbage disposal services)? We can look again to work in the CBPR literature which has developed frameworks and strategies on good report-back procedures (Morello-Frosch et al. 2015), and which highlights the need to truly invest in foundational relationships between researchers at the community. Israel et al. (1998, 2005) explain that good partnerships must be equitable and open. To accomplish a good partnership there should be significant devotion of resources, devotion to CBPR values, jointly developed operating norms, flexibility, common goals, cultural sensitivity, identification of key community leaders, participation of researchers who understand the community and academic institutions, data quality assurance, and research that is translated to action, including interventions and policy change. Similarly, Minkler et al. (2008) used a theoretical framework to analyze the partnerships present in four CBPR case studies. The study found that good partnerships have strong community leadership and active community participation; draw upon diverse and specialized skill sets; include good networking; have shared values and power; have an ability and willingness to fight entrenched powers and focus on larger contexts, characterized by dialogue and critical reflection to promote respect; and have broader institutional, financial, and network support.

The lack of foundational understanding between the project personnel and the community members in our project created sometimes disappointing outcomes for both. Literature from CBPR has demonstrated a successful history in enacting inclusive project planning and implementation (Balazs and Morello-Frosch 2013; Wallerstein and Duran 2010) and can be a model for CS projects looking to do the same. A number of community-engaged research projects have included CS as a major element to help community members address various environmental justice and health issues. Heaney et al. (2007) describe a CS framework designed in 1999 by the West End Revitalization Association (WERA), a community-based environmental justice and health organization, based in Mebane, North Carolina. WERA designed this framework in response to poor relationships that it had with local researchers who were performing scientific racism in their community. Community-owned and -managed research (COMR) builds on CBPR principles promoting collaboration with community-based organizations (CBOs) with demonstrated organizational capacity around defined EJ issues and further shifts to a process where CBOs prioritize research goals. However, COMR differs from CBPR by requiring that the CBO be funded directly as the sole PI and project manager of research activities. This leads to more effective promotion of (1) the CBO’s authority to select university “experts” whom they identify as amenable to their prioritized EJ or health issues, (2) community management of the research collaboration process to prioritize, maximize, and leverage available funding, and (3) community ownership of databases to ensure implementation of solutions for evolving community issues and corrective actions (after initial research and data generation activities are completed). COMR prioritizes the community’s goal for initiation of compliance and enforcement efforts by local, state, and federal government officials to address EJ, public health, planning, and civil rights violations of existing statutes and laws.

Wilson et al. (2014) is another example of strategies and outcomes when merging CS with other participatory approaches. Here, researchers applied Community Campus Partners for Health’s Principles of Partnership to evaluate the impact of the community-university environmental justice partnership between the Low Country Alliance for Model Communities, the University of Maryland, the University of South Carolina, and the Medical University of South Carolina. In this evaluation, the team describes various challenges and lessons learned including: 1) developing a shared vision would have led to earlier buy-in from the community; 2) constant communication is needed for authentic and meaningful engagement; 3) memorandum of understandings and strong personal relationships can help create respect and trust; and 4) addressing inequities in power and resources has to be a priority of community-university partnerships. Even with its challenges, this partnership has been quite successful in training residents as citizen scientists, building community capacity to address local environmental health issues, providing tools that can be used for better community-driven decision-making such as EJRADAR, helping to eliminate hazards including shutting down a local incinerator, stopping a recycling plant from being built on the old incinerator site, and empowering residents to be more involved in land use and zoning decisions.

CBPR might be a worthy goal in some CS endeavors – especially when broadening socio-economic and ethno-racial inclusion – but CBPR and CS literatures and intentions are often advancing in parallel without necessarily learning from one another. Likewise, while CBPR processes might have funding and have advanced in some disciplines, such as public health, the opportunities and benefits have not been as evident in ecology. Additionally, CS projects need to consider how to balance the scientific research needs of projects along with the time, costs, and effort associated with CBPR. Green (1995) defined CBPR as “inquiry with the participation of those affected by an issue for the purpose of education and action for effecting change,” whereas CS has been broadly defined as meaningful engagement of the public in the scientific process to ask and answer scientific questions (Bonney et al. 2009; Shirk et al. 2012). As these definitions suggest, both CBPR and CS may involve inclusion of members of the public in scientific research and have overlapping goals (i.e., increased knowledge, social change, policy change), but CS does not require the elements justice and social change whereas CBPR does. However, as CS often includes goals of understanding and addressing environmental injustice (Corburn 2005) through advocacy and policy efforts (Robertson and Hull 2001), more work understanding how to balance these scientific and social goals in the CS paradigm is important. In particular, CBPR principles recognize and try to address issues related to inclusion, equity, and power, which is important for CS projects to explicitly consider when creating CS programs as an opportunity to inform and shape broader social efforts. Developing more CS to include CBPR, civic ecology, and other participatory approaches (e.g., participatory modeling, collaborative conservation) may help to address underrepresentation in CS and help to create a culture of equity and inclusion.


We would like to thank the following organizations: No Boundaries, Parks & People Foundation, Neighborhood Design Center, SOWEBO partnerships, Franklin Square Community Association, Harlem Park Community Association, Union Square Community Association, and Hollins Market Community Association, all of which worked with us throughout this project. This research was supported through a National Science Foundation CNH: #1211797. Field and data logistics were further supported by the Baltimore Ecosystem Study (NSF-LTER DEB 1027188). All work was done with IRB approval from the Cary Institute of Ecosystem Studies (#112-2011) and Rutgers University (#E13-573).

Competing Interests

The authors have no competing interests to declare.


  1. Abara, W, Wilson, SM and Burwell, K. 2012. Environmental justice and infectious disease: Gaps, issues, and research needs. Environmental Justice, 5(1): 8–20. DOI: 

  2. Aber, A, Waxman, N, Khatib, A, Bamfo, A, Simon-Ogan, D and Wilson, S. 2017. Use of Photovoice to Highlight Environmental Justice Issues: The Power of Photography in Buzzard Point, Washington, DC. Environmental Justice, 10(2): 36–42. DOI: 

  3. Balazs, CL and Morello-Frosch, R. 2013. The three Rs: How community-based participatory research strengthens the rigor, relevance, and reach of science. Environmental Justice, 6(1): 9–16. DOI: 

  4. Ballard, HL, Harris, EM and Dixon, CGH. 2018. Science Identity and Agency in Community and Citizen Science: Evidence & Potential. Commissioned for the Committee on Designing Citizen Science to Support Science Learning. Retrieved from: <>. 

  5. Bartlett-Healy, K, Unlu, I, Obenauer, P, Hughes, T, Healy, S, Crepeau, T, Farajollahi, A, Kesavaraju, B, Fonseca, D, Schoeler, G, Gaugler, R and Strickman, D. 2012. Larval mosquito habitat utilization and community dynamics of Aedes albopictus and Aedes japonicus (Diptera: Culicidae). Journal of medical entomology, 49(4): 813–824. DOI: 

  6. Bell-Elkins, J. 2002. Assessing the CCPH principles of partnership in a community–campus partnership [Internet]. Seattle (WA): Community–Campus Partnerships for Health. 

  7. Biehler, D, Baker, J, Pitas, JH, Bode-George, Y, Jordan, R, Sorensen, AE, Wilson, S, Goodman, H, Saunders, M, Bodner, D, Leisnham, PT and LaDeau, SL. 2018. Beyond “the Mosquito People”: The Challenges of Engaging Community for Environmental Justice in Infested Urban Spaces. In: The Palgrave Handbook of Critical Physical Geography, 295–318. Palgrave Macmillan, Cham. DOI: 

  8. Bodnaruk, EW, Kroll, CN, Yang, Y, Hirabayashi, S, Nowak, DJ and Endreny, TA. 2017. Where to plant urban trees? A spatially explicit methodology to explore ecosystem service tradeoffs. Landscape and Urban Planning, 157: 457–467. DOI: 

  9. Bodner, D, LaDeau, SL, Biehler, D, Kirchoff, N and Leisnham, PT. 2016. Effectiveness of print education at reducing urban mosquito infestation through improved resident-based management. Plos One, 11. DOI: 

  10. Bonney, R, Cooper, CB, Dickinson, J, Kelling, S, Phillips, T, Rosenberg, KV and Shirk, J. 2009. Citizen science: A developing tool for expanding science knowledge and scientific literacy. BioScience, 59(11): 977–984. DOI: 

  11. Brandt, HM, Haynes, VE, Rice, LJ, Campbell, D, Williams, E and Glover, SH. 2017. Using photovoice as a tool for community engagement to assess the environment and explore environmental health disparities. Journal of Health Disparities Research and Practice, 10(3): 6. 

  12. Carr, AJL. 2004. Why do we all need community science. Society and Natural Resources, 17: 841–849. DOI: 

  13. Chanse, V, Mohamed, A, Wilson, SM, Dalemarre, L, Leisnham, PT, Rockler, A, Shirmohammadi, A and Montas, H. 2017. New approaches to facilitate learning from youth: Exploring the use of Photovoice in identifying local watershed issues. The Journal of Environmental Education, 48(2): 109–120. DOI: 

  14. Cole, JN, Miller, AJ, Stapleton, E and Welty, C. 2016. Quantifying Spatial Patterns of Channel Geometry and Stream Incision in Urban Drainage Network. Journal of Hydrologic Engineering, 22(2). DOI: 

  15. Conrad, CC and Hilchey, KG. 2011. A review of citizen science and community-based environmental monitoring: issues and opportunities. Environmental monitoring and assessment, 176(1–4): 273–291. DOI: 

  16. Cooper, C, Dickinson, J, Phillips, T and Bonney, R. 2007. Citizen science as a tool for conservation in residential ecosystems. Ecology and Society, 12(2). DOI: 

  17. Corburn, J. 2005. Street science: community knowledge and environmental health justice (urban and industrial environments). Cambridge, MA: The MIT Press. DOI: 

  18. Dhillon, CM. 2017. Using citizen science in environmental justice: Participation and decision making in a Southern California waste facility siting conflict. Local Environment, 22(12): 1479–1496. DOI: 

  19. Eden, S. 2006. Public participation in environmental policy: considering scientific, counter-scientific and non-scientific contributions. Public Understanding of Science, 5: 183–204. DOI: 

  20. Evans, C, Abrams, E, Reitsma, R, et al. 2005. The Neighborhood Nestwatch Program: participant outcomes of a citizen-science ecological research project. Conservation Biology, 19: 589–94. DOI: 

  21. Goodman, H, Egizi, A, Fonseca, DM, Leisnham, PT and LaDeau, SL. 2018. Primary blood-hosts of mosquitoes are influenced by social and ecological conditions in a complex urban landscape. Parasites & Vectors, 11. DOI: 

  22. Green, LW. 1995. Study of Participatory Research in Health Promotion: Review and Recommendations for the Development of Participatory Research in Health Promotion in Canada. Vancouver, British Columbia: Royal Society of Canada. 

  23. Grove, M, Ogden, L, Pickett, S, Boone, C, Buckley, G, Locke, DH, Lord, C and Hall, B. 2018. The legacy effect: understanding how segregation and environmental injustice unfold over time in Baltimore. Annals of the American Association of Geographers, 108(2): 524–537. DOI: 

  24. Hawley, WA. 1988. The biology of Aedes albopictus. Journal of the American Mosquito Control Association, 434: 1–40. 

  25. Heaney, CD, Wilson, SM and Wilson, OR. 2007. The West End Revitalization Association’s community-owned and-managed research model: Development, implementation, and action. Progress in Community Health Partnerships: Research, Education, and Action, 1(4): 339–349. DOI: 

  26. Israel, BA, Coombe, CM, Cheezum, RR, Schulz, AJ, McGranaghan, RJ, Lichtenstein, R, Reyes, AG, Clement, J and Burris, A. 2010. Community-based participatory research: A capacity-building approach for policy advocacy aimed at eliminating health disparities. American journal of public health, 100(11): 2094–2102. DOI: 

  27. Israel, BA, Parker, EA, Rowe, Z, Salvatore, A, Minkler, M, López, J, Butz, A, Mosley, A, Coates, L, Lambert, G, Potito, PA, Brenner, B, Rivera, M, Romero, H, Thompson, B, Coronado, G and Halstead, S. 2005. Community-based participatory research: Lessons learned from the Centers for Children’s Environmental Health and Disease Prevention Research. Environmental health perspectives, 113(10): 1463. DOI: 

  28. Israel, BA, Schulz, AJ, Parker, EA and Becker, AB. 1998. Review of community-based research: Assessing partnership approaches to improve public health. Annual review of public health, 19(1): 173–202. DOI: 

  29. Jordan, RC, Gray, SA, Howe, DV, Brooks, WR and Ehrenfeld, JG. 2011. Knowledge gain and behavioral change in citizen-science programs. Conservation Biology, 25(6): 1148–1154. DOI: 

  30. Jordan, RC, Sorensen, AE, Biehler, D, Wilson, S and LaDeau, S. 2018. Citizen science and civic ecology: Merging paths to stewardship. Journal of Environmental Studies and Sciences, 1–11. 

  31. Jordan, RC, Sorensen, AE and Ladeau, S. 2017. Citizen Science as a Tool for Mosquito Control. Journal of the American Mosquito Control Association, 33(3): 241–245. DOI: 

  32. LaDeau, SL, Leisnham, PT, Biehler, D and Bodner, D. 2013. Higher mosquito production in low-income neighborhoods of Baltimore and Washington, DC: Understanding ecological drivers and mosquito-borne disease risk in temperate cities. International Journal of Environmental Research and Public Health, 10: 1505–1526. DOI: 

  33. Lapan, SD, Quartaroli, MT and Riemer, FJ. (eds.) 2011. Qualitative research: An introduction to methods and designs. 37. John Wiley & Sons. 

  34. Larese-Casanova, M and Prysby, M. 2018. Engaging people in nature stewardship through Master Naturalist programs. Human–Wildlife Interactions, 12(2): 11. 

  35. Lasker, RD and Weiss, ES. 2003. Broadening participation in community problem solving: A multidisciplinary model to support collaborative practice and research. Journal of Urban Health, 80(1): 14–47. DOI: 

  36. Leisnham, PT, Towler, L and Juliano, SA. 2011. Geographic variation of photoperiodic diapause but not adult survival or reproduction of the invasive mosquito Aedes albopictus (Diptera: Culicidae) in North America. Annals of the Entomological Society of America, 104(6): 1309–1318. DOI: 

  37. Little, E, Biehler, D, Leisnham, PT, Jordan, R, Wilson, S and LaDeau, SL. 2017. Socio-Ecological Mechanisms Supporting High Densities of Aedes albopictus (Diptera: Culicidae) in Baltimore, MD. Journal of Medical Entomology, 54: 1183–1192. DOI: 

  38. McCormick, S. 2007. Democratizing science movements: A new framework for mobilization and contestation. Social Studies of Science, 37(4): 609–623. DOI: 

  39. Minkler, M, Vásquez, VB, Tajik, M and Petersen, D. 2008. Promoting environmental justice through community-based participatory research: The role of community and partnership capacity. Health Education & Behavior, 35(1): 119–137. DOI: 

  40. Morello-Frosch, R, Varshavsky, J, Liboiron, M, Brown, P and Brody, JG. 2015. Communicating results in post-Belmont era biomonitoring studies: Lessons from genetics and neuroimaging research. Environmental research, 136: 363–372. DOI: 

  41. Mueller, M, Tippins, D and Bryan, L. 2012. The future of citizen science. Democracy & Education, 20(1). 

  42. National Academies of Sciences, Engineering, and Medicine (NASEM). 2018. Learning Through Citizen Science: Enhancing Opportunities by Design. Washington, DC: The National Academies Press. DOI: 

  43. Pandya, RE. 2012. A framework for engaging diverse communities in citizen science in the US. Frontiers in Ecology and the Environment, 10: 314–317. DOI: 

  44. Phillips, T, Porticella, N, Constas, M and Bonney, R. 2018. A Framework for Articulating and Measuring Individual Learning Outcomes from Participation in Citizen Science. Citizen Science: Theory and Practice, 3(2). 

  45. Robertson, DP and Hull, RB. 2001. Beyond Biology: Toward a more public ecology for conservation. Conservation Biology, 15: 970–979. DOI: 

  46. Shirk, JL, Ballard, HL, Wilderman, CC, Phillips, T, Wiggins, A, Jordan, R, McCallie, E, Minarchek, M, Lewenstein, BV, Krasny, ME and Bonney, R. 2012. Public participation in scientific research: A framework for deliberate design. Ecology and Society, 17(2): 29. DOI: 

  47. Troy, A and Grove, JM. 2008. Property values, parks, and crime: A hedonic analysis in Baltimore, MD. Landscape and urban planning, 87(3): 233–245. DOI: 

  48. Wallerstein, N and Duran, B. 2010. Community-based participatory research contributions to intervention research: The intersection of science and practice to improve health equity. American journal of public health, 100(S1): S40–S46. DOI: 

  49. Wang, C and Burris, MA. 1997. Photovoice: Concept, methodology, and use for participatory needs assessment. Health education & behavior, 24(3): 369–387. DOI: 

  50. Wilson, S, Campbell, D, Dalemarre, L, Fraser-Rahim, H and Williams, E. (2014). A critical review of an authentic and transformative environmental justice and health community University partnership. International journal of environmental research and public health, 11(12): 12817–12834. DOI: 

  51. Wilson, SM. 2009. An ecologic framework to study and address environmental justice and community health issues. Environmental Justice, 2(1): 15–24. DOI: 

  52. Wilson, SM, Heaney, CD, Cooper, J and Wilson, O. 2008. Built environment issues in unserved and underserved African-American neighborhoods in North Carolina. Environmental Justice, 1(2): 63–72. DOI: 

  53. Wilson, SM, Wilson, OR, Heaney, CD and Cooper, J. 2007. Use of EPA collaborative problem-solving model to obtain environmental justice in North Carolina. Progress in Community Health Partnerships: Research, Education, and Action, 1(4): 327–337. DOI: