An Editor’s Note relating to this publication can be found here: http://doi.org/10.5334/cstp.264
A citizen science collaboration between Flint residents, the Virginia Tech “Flint Water Study” team, and others helped to uncover the Flint Lead-in-Drinking Water Crisis and a community-wide outbreak of Legionella. The resulting Federal Emergency declaration in January 2016 resulted in more than $600 million in relief funding, an acknowledged case of environmental injustice, and resignations/indictments of some public officials. But after responsible government entities apologized and attempted to make amends and help with the recovery, some “citizen scientists” began making public statements that were in direct conflict with public health messaging of scientific authorities. A general state of science anarchy resulted, which created further distrust and confusion. Some practices employed were consistent with a concept of “citizen engineering,” which aims to “undermine engineering [and science] expertise” in the name of “democratizing” science. “Citizen Engineers” view concepts of scientific rigor and objectivity as justification for abuse of power by authorities and scientists, and they embrace biases and conflicts of interest that scientists aspire to guard against. While there are ethical guidelines for professional scientists on research misconduct, no such framework exists for policing instances of unethical behavior by citizen scientists. Possible abuses of citizen science documented in Flint explored in this case study include: 1) collection of non-representative data that created unjustified fear among residents about the safety of water used for bathing and showering, 2) perceived financial conflicts of interest, and 3) falsification of data to obtain relief resources, support lawsuits, gain media attention, or support erroneous scientific conclusions. We also report the journey of an aspiring citizen scientist who openly acknowledged mistakes, made the “right” decision in relation to handling an ethical dilemma, and who was then publicly attacked for doing so. This experience highlights challenges to the practice of citizen science, especially during high profile emergency interventions and disasters involving environmental injustice.
An Editor’s Note relating to this publication can be found here: http://doi.org/10.5334/cstp.264
“We’re in a post-truth world with eroding trust and accountability…. In our new normal, experts are dismissed, alternative facts are (sometimes flagrantly) offered, and public figures can offer opinions on pretty much anything. And thanks to social media, pretty much anyone can be a public figure.” Nick Enfield (Enfield 2017)
This case study calls attention to lessons learned about the practice of citizen science during high profile emergency interventions and disasters involving environmental injustice. A citizen science collaboration among Flint residents, the Virginia Tech “Flint Water Study” team, and others first exposed contamination in Flint’s water supply in 2015. The resulting Federal Emergency declaration in January 2016 resulted in more than $600 million in relief funding, an acknowledged case of environmental injustice, and resignations/indictments of some public officials. But after the Federal Emergency was declared, ethical dilemmas associated with abuse of citizen science were encountered, conclusions of which were used to support unscientific public health messages that were in direct conflict with those of the relief agencies. A general state of science anarchy resulted which created further distrust and confusion, highlighting the lack of frameworks to police instances of unethical behavior by those claiming to be citizen scientists. While it is relatively easy for scientific experts to identify good vs. poor citizen science practices, Flint demonstrates that social justice advocates are sometimes willing to openly embrace faulty methods or even misconduct to achieve a populist objective, thereby creating a high likelihood of confrontation in future high profile cases like the Flint Water Crisis.
“It’s regular, good, pure drinking water, and it’s right in our backyard.” Dayne Walling, Former Mayor of Flint (City of Flint 2014)
To save about 5 million US dollars while constructing a new water pipeline, Michigan state-appointed emergency managers switched drinking water sources for the City of Flint from Lake Huron (purchased from the Detroit Water and Sewerage Department [DWSD]) to the local Flint River on April 25, 2014 (Fonger 2015a; Schwake et al. 2016). A failure to implement federally mandated corrosion control treatment unleashed a “perfect storm” within Flint’s antiquated water distribution system of unlined iron water mains, lead/galvanized steel service pipes, and lead-bearing internal plumbing. The uncontrolled corrosion depleted the chlorine disinfectant residual in the system, elevating levels of microbes including Escherichia coli and Legionella pneumophila. An in-depth scientific analysis of deficiencies that caused Flint’s water quality problems can be found elsewhere (Schwake et al. 2016; Masten, Davies, and McElmurry 2016; Allen et al. 2017; Olson et al. 2017; Pieper, Tang, and Edwards 2017; Rhoads et al. 2017; Pieper et al. 2018; Roy and Edwards 2018a).
In parallel with the scientific and engineering problems, public trust was lost due to failures of government agencies at all levels (Bernstein and Dennis 2016; Canepari and Leduff 2016; Edwards 2016a; Edwards 2016b; Gray et al. 2016; Shapiro and Gringlas 2018). Analysis of Google web searches illustrates that Flint residents began unusual online searches for information about their tap water soon after the switch in summer 2014 (Matsa, Mitchell, and Stocking 2017). A series of red flags indicating growing problems were reported by consumers to appropriate local and state government officials during the time that Flint River water was being used (Apr 2014–Oct 2015), and these warnings were either ignored or in some cases covered up (Edwards 2016). Exemplary problems included reports of red water, hair loss, skin rashes, three boil water advisories, rusting of engine parts at a General Motors factory, and violations of disinfection byproduct (DBP) regulations. But the crisis entered a new phase when high lead levels (greater than 100 ppb) were detected by a City of Flint official in the home of resident LeeAnne Walters (Pieper, Tang, and Edwards 2017).
“We proved that citizens and scientists working together could form a great alliance, and that grassroots science can have a sky-high impact.” LeeAnne Walters (Flint Water Study 2016)
After Walters (a.k.a. “Resident Zero”) discovered high water lead in her home (a.k.a. “Ground Zero”) through a sample collected by the City of Flint, she became a practicing citizen scientist who systematically educated herself about water lead issues and diagnosed the cause for elevated blood lead in her children. She also began collaborating with Miguel Del Toral, one of the foremost national experts of the federal Lead and Copper Rule (LCR) in the US Environmental Protection Agency (EPA) working in nearby Chicago. Walters and Del Toral unequivocally determined that there was no federally required lead corrosion control in the city’s water (Lurie 2016; Smith and Thomson 2017). In April 2015, Walters sampled her tap water with the assistance of authors of this paper, demonstrating hazardous waste levels of water lead even after prolonged flushing (Pieper et al. 2017). Those data contributed to a detailed technical memo by Del Toral in late June 2015 which highlighted the imminent and substantial endangerment to Flint residents (Del Toral 2015). In early August 2015, Walters and other Flint residents learned that the Michigan Department of Environmental Quality (MDEQ)–which had the primary responsibility for safety of Flint water–intended to ignore the memo and cover up water lead problems without interference from senior management at the EPA (Roy and Edwards 2015; EPA OIG 2016). Del Toral was effectively silenced by EPA (Edwards 2016a).
Given that local and federal agencies were not going to carry out their duty to protect Flint residents, a citizen science collaboration was launched by Virginia Tech in early August 2015. It involved the American Civil Liberties Union-Michigan, Walters, activist groups, and many others. Virginia Tech provided the technical plan, analytical support, and funding, while Flint residents donated their local knowledge, homes as test sites, and hard work. The collaboration started with a team from Virginia Tech driving to Flint and sampling the water distribution system, water heaters, and home plumbing for a wide array of potential chemical and microbial contaminants, which also laid a groundwork for the citizen coordinated citywide sampling event focused on water lead, which ultimately involved 269 geographically distributed Flint homes. Of the hundreds of analytes examined in the initial phase of testing in summer 2015 by the Virginia Tech team, only lead exceeded federal standards (Edwards 2015; Cooper 2016; Paynter 2016; Averett 2017; Rhoads et al. 2017; Pieper et al. 2018). That result, coupled with a later independent analysis by researchers at Hurley Medical Center demonstrating rising blood lead in some Flint children, led local medical authorities and, eventually, the State of Michigan to declare a water health advisory on October 1, 2015 and to switch back to Lake Huron water supplied by DWSD on October 15, 2015 (Hanna-Attisha et al. 2016). These events are covered in the 2017 PBS NOVA documentary “Poisoned Water” (Smith and Thomson 2018).
In late 2015, high levels of Legionella pneumophila were detected inside large buildings in Flint by Virginia Tech (Rhoads et al. 2017), and provision of these data to authorities helped to prompt revelations of a two year Legionnaires’ Disease outbreak in 2014 and 2015, with about a dozen deaths that occurred after the switch to Flint River water. Shortly thereafter (January 13, 2016), President Barack Obama declared a Federal Emergency (Wakefield 2016). The elevated lead in water, elevated lead in children’s blood, the Legionella outbreak, and the Federal Emergency created a media sensation that eventually mobilized hundreds of millions of US dollars of federal, state, and private relief funding for Flint (French 2016).
In the aftermath of the Federal Emergency declaration, the responsible and largely discredited government entities publicly accepted some blame and re-dedicated themselves to the relief effort, along with assistance mobilized by the Federal Emergency Management Agency (FEMA) and the National Guard, which distributed filters and bottled water (Wakefield 2016). The governor publicly apologized and set up a task force that included members of Virginia Tech, Hurley Medical Center, and other former vocal government critics to advise the recovery efforts (Bosman and Smith 2016; Johnson 2016a). Fifteen civil servants within the State of Michigan (including MDEQ) and City of Flint were charged for their role in the crisis (Egan 2017a).
EPA, MDEQ, and the City of Flint then took additional measures to enhance disinfection and corrosion control beyond that normally present in treated water purchased from DWSD, to hasten recovery of the distribution system for both microbial and chemical contaminants. Subsequent sampling tracked the water system’s recovery in four citizen-led sampling campaigns in March 2016, July 2016, November 2016, and August 2017. Because Del Toral and Virginia Tech had proven that all official state water lead data before 2016 were collected using so-called “cheats” that effectively minimized detection of lead hazards at the tap, the citizen science datasets collected without cheats represented the only internally consistent measures of water lead levels during the water crisis and its aftermath. As a result, our follow-up citizen science team sampling efforts were funded by the federal government (i.e., US EPA) in March, July, and November 2016 to track the recovery (Roy and Edwards 2015; Milman and Glenza 2016; Pieper et al. 2017; Tang et al. 2017; Pieper et al. 2018).
The citizen science sampling events and our other water quality research in Flint demonstrated expected improvements in tap water quality from the perspective of lead, Legionella, and metals that cause discolored water. Regulatory monitoring by MDEQ showed no problems with disinfection by-product regulations at any point during 2016, with 90th percentile lead levels below the EPA action level beginning in summer 2016. After 2015, the official MDEQ lead measurements were no longer in significant disagreement with our citizen science monitoring data (Allen et al. 2017; Lynch and Chambers 2017; Pieper et al. 2017).
From a scientific and regulatory standpoint, Flint’s water met all existing federal standards by summer 2017, which by some definitions could mark an end to the public health crisis. But the problem of lost trust was an enormous issue (Canepari and Leduff 2016; Goodnough and Atkinson 2016; Adewunmi 2017; Fonger 2017a; Chavez 2018; The Detroit News 2018; Shapiro and Gringlas 2018), creating a crisis of confidence (Roy and Edwards 2018a) and an environment ripe for exploitation by rumors and fearmongering (Roy 2017). While the mistrust in government agencies was self-inflicted and deserved through at least early 2016, we argue herein that the credible work of relief agencies after that time were systematically undermined by those operating under a banner of citizen science.
“You gotta understand you cannot be bathing in this water … a hot steamy shower–you’re simply asking for trouble.” Bob Bowcock (The Steve Harvey Show 2016)
In January 2015, environmental activist Erin Brockovich called attention to the ongoing problems with disinfection by-products (DBPs) in Flint, and sent her water expert Robert Bowcock to the city, who spoke to residents and recommended altering treatment (“fluoride, lime, and softener”) that could get “good water quality” in two months (Davis 2015, Fonger 2015b, Fonger 2015c, Ketchum 2015; Fonger 2015e; NBC News 2016). Bowcock did not recommend corrosion control but did suggest a switch back to Detroit water (Fonger 2015d). After President Obama’s Emergency Declaration, Brockovich continued to emphasize that residents should get filters “on their showerhead” (The Late Show with Stephen Colbert 2016) to reduce exposure to DBPs in showers, while Bowcock appeared on The Steve Harvey show alongside a local pediatrician and emphasized that “bathing in the water is not safe” (The Steve Harvey Show 2016). At all times after the switch back to Detroit water, however, Flint had been meeting federal DBP standards and there was no basis to declare the water unsafe for bathing.
“[We shouldn’t be having the debate of] whether or not my organization [Water Defense] is (sic) scientists, because we are admitting we are citizen scientists.… All we are doing is giving people information and there is nothing wrong with that. And the more information people have, the better off they are.” Mark Ruffalo (Water Defense 2016a)
In late January 2016, months after the switch back to Lake Huron water from DWSD and just two weeks after President Obama’s declaration of a Federal Emergency had mobilized FEMA and the National Guard (Delaney 2016a; Wakefield 2016; Water Defense 2016b; Roy 2017), Mr. Scott Smith of actor Mark Ruffalo’s environmental nonprofit Water Defense came to Flint at the request of residents. Mr. Smith and Water Defense had recently launched WaterBug™–a green tentacled sponge product designed to “empower citizens to take water quality testing into their own hands and protect themselves” (Ruffalo 2015).
Over the next five months, investigations by our team and others revealed that Mr. Smith had no formal scientific credentials to support a title of “Chief Scientist” and also had possible financial conflicts of interest. Of greatest concern for someone holding a high-profile public role with a 501(c)(3) nonprofit intervening in a Federal Emergency situation, he was: 1) inventor of foam materials used in the WaterBug™ and allegedly could receive royalty payments from the manufacturer for their purchase (Delaney 2016a); and 2) owned a for-profit company Aquaflex™ announced in May 2016, which was pitched to investors at the time as a maker of proprietary filtration systems to “satisfy unmet needs” as the Water Defense nonprofit “continue<d> to identify and diagnose water contamination sites.” While Mr. Smith publicly asserted at a Water Defense press conference in March 2016 that “The ultimate solution here is … to create green jobs, to create … solar powered [water] filtration [systems] … and help the economy in the process,” he did not prominently mention these possible financial conflicts or disclose the investor pitch to sell filters through his private Aquaflex™ venture or other parties (OPFLEXInventor 2013; Business Wire 2016; Delaney 2016a; Edwards 2016c; Justice League NYC 2016; Smith 2018).
Citizen science sampling efforts, like all scientific sampling, must be rigorous and defensible and, whenever possible, use established protocols (e.g., from Standard Methods for the Examination of Water and Wastewater, 23rd ed.; APHA, AWWA, and WEF 2017). They also must try to practice appropriate quality assurance/quality control, maintain data integrity (Riesch and Potter 2014; Resnik et al. 2015; Geoghegan et al. 2016), and provide proper controls when appropriate to compare results to other geographic locations (Ottinger 2010) or federal standards (e.g., from the US EPA and World Health Organization or WHO). For example, “proper” lead-in-water testing requires collecting 1 L water samples from the cold water kitchen or bathroom tap after a minimum 6 hour stagnation time according to EPA regulations (Pieper, Tang, and Edwards 2017). In contrast, the proprietary Water Defense WaterBug™ had never been compared or calibrated against any accepted water testing standard, subjected to any peer-review, or even product-tested by standards organizations such as the National Sanitation Foundation.
It eventually became obvious that the floating WaterBug™ sponges, partly submerged in water, could collect chemicals from both water and ambient air, creating confusion when the detected contaminants were assumed to come from the water (Virginia Tech 2016; Williams 2016). At one point, a Flint resident became concerned about bathing and showering in her home because the WaterBug™ detected high levels of acetone, which set off a search for sources of acetone contamination to the public water supply. However, the resident herself later determined that her bathroom contained multiple bottles of nail polish remover (100% acetone), which likely caused the false positive initially attributed to the water via the air (K. Webber, personal communication, Feb 4, 2018). In terms of microbial sample sterility, the resident claimed that Mr. Smith rubbed the WaterBug™ on his arm to demonstrate its relationship to skin testing, before placing it into the bathtub (S. Ganim, personal communication, Feb 13, 2018), consistent with other breaches of sterile microbial sampling protocols observable in Water Defense videos released online (Roy and Edwards 2017).
During January to May 2016, Water Defense also routinely collected water samples in Flint homes from non-regulatory and improper sources including hose bibs, water meters, and water heaters and their sediment clean-out valves. All of these sampling locations create well-known false positives for lead, due to infrequently used plumbing components that are not designed to dispense water fit for human consumption because of known high lead risks (Edwards 2016d; Roy and Edwards 2017). As such they are termed by EPA as “improper” in relation to collecting data relevant to compliance with Federal law. While such samples can be used for diagnostic purposes, the resulting high lead data from these improper sites were publicly broadcast on social media as indicating serious and ongoing water lead concerns. Those data were complemented by the WaterBug™ test results in “detailed lab reports” containing as much as 97 pages of raw data listing hundreds of chemicals allegedly contaminating the water, which were presented without context as to what concentrations of these chemicals would be normal or “unsafe” in tap water (TYT 2017a). Water Defense published press releases on their website with sensational titles including “Dangerous chemicals discovered in baths/showers of Flint, MI,” participated in town halls and press conferences, posted on social media with the hashtag #citizenscience, and were featured prominently in local media and Facebook groups by citizens (Emery 2016; Lynch and Carah 2016; Smith 2016a; Smith 2016b; Water Defense 2016c; Water Defense 2016d; WNEM 2016; Roy and Edwards 2017). From their very first video, taken before they had collected or analyzed a single Flint water sample, Water Defense appeared to have a bias toward finding data indicating that the water was “not safe for bathing and showering” (Water Defense 2016b).
These messages were amplified on social media–where “falsehoods diffuse significantly farther, faster, deeper and more broadly than the truth” (Vosoughi et al. 2018)–and reached Flint residents during early 2016. For example, a TV news report on March 21, 2016 noted that “[Mr. Smith] for weeks has been urging people in Flint not to bath (sic) in <the water>” (Parkinson 2016). At this time, the Michigan Department of Health and Human Services’ (MDHHS) Chief Medical Executive Dr. Eden Wells tried to communicate the stance of all major agencies and researchers engaged in Flint relief efforts to Water Defense:
“On the bathing issue–both Dr. Mona [Hanna-Attisha] and the MDHHS state that based on what is currently known about the water system (including all the water tests, etc.), it is safe to bathe and shower. CDC and CDC ATSDR are in concurrence. Dr. Marc Edwards … is also in concurrence.” (Edwards 2018a)
Wells also shared details about an ongoing investigation of skin rashes in Flint by CDC that eventually showed a normal rash incidence at that time. However, Water Defense was combative in response to Wells and others and would not accept a scientific consensus opinion about bathing and showering safety during the Federal Emergency (Edwards 2018).
The fact that Mr. Smith had no professional degrees in science or engineering (Edwards 2016c) did not impede broadcasting of his messages and, assisted by Mr. Ruffalo’s star power, it may even have helped, but at some point the Chief Scientist title was dropped (OPFLEXInventor 2013; Delaney 2016a; Smith 2016c; Smith 2017). Mr. Ruffalo personally appeared on CNN, falsely stating that disinfection by-products (DBPs) were coming from Flint’s pipes damaged by corrosion, and reiterated his organization’s opinion that “[no one] can tell the people of Flint that it’s safe to bathe in [Flint] water,” again directly contradicting a unified message from all relief agencies that the Flint water was not more dangerous for bathing than other cities (CNN 2016; MDHHS 2016; Williams 2016). Independent professors from Stanford, University of Massachusetts-Amherst, and University of South Carolina with expertise in DBPs were recruited by our team to conduct their own investigations in Flint. Their findings also flatly contradicted Ruffalo’s false statement about DBPs originating from Flint pipes, which supported the relief agencies’ message to some extent (Johnson 2016b).
“As an ethical matter, pseudoscience is not—contrary to popular belief—merely a harmless pastime of the gullible; it often threatens people’s welfare, sometimes fatally so.” Maasimo Pigliucci and Marten Boudry (Pigliucci and Boudry 2013)
The government agencies whose earlier claims about water safety for lead were proved wrong by our citizen science collaboration had little or no credibility in Flint after late 2015 (Bernstein and Dennis 2016). In fact, after our prior citizen science success, almost any group directly contradicting the agencies was given media coverage, with explicit reference to the prior vindication of whistleblowers Drs. Edwards and Hanna-Attisha, and Mr. Del Toral. Given their past failures, the agencies were also unwilling to directly contradict anyone openly undermining their expertise and authority. Distrust, fear, and anxiety toward the water by citizens was already justified, and residents continued to report skin rashes, hair loss, and other dermal ailments that they felt were due to water (Goodnough 2016; Roy 2017). While the US Centers for Disease Control and Prevention (CDC) conducted a study that indicated higher than normal rates of rashes during the water crisis in 2014–2015, confirming residents’ beliefs about health risks from that time, they also reported normal rates of rashes after the water source was switched back to DWSD (CDC 2016a). Nevertheless, our team personally received at least a dozen communications from worried residents who cited Water Defense data and associated press releases as justifications for stopping their normal bathing or showers.
An outbreak of Shigellosis began soon after Water Defense arrived in Flint (Acosta 2016; CDC 2016b) (Figure 1). Shigellosis is a gastrointestinal illness caused by Shigella bacteria, often resulting in severe diarrhea, stomach cramps, and fever. The bacteria is most commonly spread through the fecal-oral pathway (e.g., by coming in contact with infected stool and/or soiled diapers), and only rarely from contaminated water in the US. The disease is best controlled by normal bathing/showering and regular handwashing with soap in disinfected water (CDC 2018). Flint is in Genesee County, which had a Shigella incidence 1.05 cases/month (35% lower than the national average) when the city was being served by Flint River water during the crisis (April 2014–October 2015), up from a rate of 0.47 cases/month during a comparable control period (April 2012–October 2013) before the water switch from DSWD (one-tailed paired t-test; p < 0.05). But shigellosis rose to 13.8 cases/month (6.7 times the national incidence rate from February to October 2016) in the immediate aftermath of the Federal Emergency in January 2016 and Water Defense’s arrival. Overlaying the timelines of monthly shigellosis incidence in 2016 with key public statements from Water Defense, Virginia Tech, health agencies (Genesee Co. Health Dept. or GCHD, and CDC), and some Flint residents during the outbreak is enlightening (Figure 1).
What was the source of the outbreak? All water samples (150 total) collected by Virginia Tech from 30 Flint homes in June 2016 (a peak month of the shigellosis outbreak) were later found to be negative for Shigella, providing strong evidence that the outbreak was not from water (Strom et al. 2016). A CDC-led study relying on survey questions, clinical dermatologic evaluations, genetic mapping, and water quality testing in Flint homes (n = 390) also later concluded that Shigella did not likely originate from Flint drinking water. From early 2016 the relief agencies and Virginia Tech considered it plausible that the misinformation on the dangers of bathing and showering compounded the legitimate fears of Flint residents, resulting in less effective hygiene that would make a Shigella outbreak more likely and of longer duration. Indeed, by August 2016, CDC revealed alarming statistics that nearly 80% of Flint residents surveyed from January 21 to April 29, 2016 reported that they had changed their bathing and showering habits including “showering less frequently” (75.3%), “taking shorter showers” (70.6%), and bathing in bottled or filtered water (CDC 2016a; Hanrahan 2018). In a follow-up survey of Flint residents that had shigellosis (n = 24), 52% reported that they had changed their bathing and showering habits, a much higher percentage than changes in bathing reported in other affected households in Genesee and Saginaw Counties (J. Yoder, Personal Communication, Feb 1, 2018). Top motivating factors for these behavior changes were “media reports” and “health concerns” about bathing (CDC 2016a; CDC 2016b).
In light of Water Defense’s lack of expertise and the possible harm to public health arising from their problematic messaging, our team attempted to reason privately with the group in April 2016, well before studies confirmed that the Shigella source was likely conventional transmission and not from the water. Without any explanation, Water Defense did not call into a scheduled teleconference meeting to discuss the issue. We then asked them to produce scientific evidence backing up their erroneous public statements about bathing and showering dangers, and informed them of the public health harm that could result if residents’ bathing habits were altered by their statements. At one point Erin Brockovich’s water expert Mr. Bowcock wrote us that while Water Defense had “spent big money on samples,” their DBP testing was “ridiculous,” and he “brokered an opportunity to fix this mess” but Water Defense refused (R. Bowcock, personal communication, May 16–19, 2016). Having tried to resolve the issue twice and failed, we informed Water Defense that we felt ethically obligated to publicly undermine their scientific credibility on our website. We even shared a draft blog to encourage them to correct their messaging and avoid a public confrontation, but after they refused, we posted the blog because we felt that their actions posed a direct threat to the public health and welfare (Edwards 2016c; Edwards 2016d; Edwards and Roy 2016).
The resulting distasteful confrontation–between our citizen science team engaged in Flint and a non-profit backed by a high-profile Hollywood actor–did create negative publicity for Water Defense and Ruffalo, generating widespread national media coverage including The New York Times Magazine and other sources (Delaney 2016a; Delaney 2016b; Hohn 2016; Jerome 2016; Lynch 2016; Williams 2016). To further combat misinformation, we coordinated the 3rd and 4th party-independent EPA funded sampling events for DBPs in cold tap, hot tap, and shower water in Flint households, and eventually demonstrated that the DBP situation was completely normal (Allen et al 2017; Goodwill et al. 2018). We also sought written scientific opinions and analytical reports from third-party academic water experts and assisted journalistic investigations into Mr. Smith’s financial dealings (Allen et al. 2017; Delaney 2016a; Johnson 2016b; Lynch 2016; Williams 2016; Roy 2017; Goodwill et al. 2018). These efforts debunked many claims and compromised the reputation of Water Defense, but they also hardened loyalty to Water Defense among some Flint activists who would not acknowledge their unscientific approach. A few residents viewed our actions as betraying common-cause activist unity against the County, State, and Federal government health authorities.
While a direct cause and effect relationship cannot be proven, there is indeed a strong temporal link consistent with the conventional hypothesis regarding spread of Shigella and the altered bathing habits of Flint residents (Figure 1). And at a minimum, considering the unified statement of scientific authorities (Cole and Fellows 2008), Water Defense and others were sending the wrong message at the wrong time and in the wrong place about bathing and showering dangers, using an approach directly attacking the expertise of the relief agencies and scientists who had relevant experience and sound data. Following our confrontation, only 7 shigellosis cases were reported from January to June 2017, indicating that the risk had finally returned to levels considered normal. A few citizens, however, remain adamant that the Shigella outbreak came from the water (Table 1), demonstrating the remarkable staying power of bad science as illustrated by the persistent “vaccines cause autism” exemplar (Allan and Ivers 2010; Mikulak 2015).
|No.||Resident A Quotes||Reference|
|1||“[The State of Michigan is] willing to kill people [for land]”||CAN TV 2018|
|2||“[Discolored water photo from 2015 shared in July 2018 proves that there is an ongoing] legal genocide [in Flint]”||Mays 2017–18|
|3||“Our showers are like gas
“Anything is going to release steam or heat […] and make our very well-insulated homes gas chambers”
|4||“We have shigella because we wash our hands”||CAN TV 2018|
|5||“The [State-distributed lead] filters cause dysentery”||CAN TV 2018|
|6||“The State of Michigan has blocked [a study showing lead filters were harmful] that Wayne State [University] has been trying to get … out for a year”||CAN TV 2018|
|7||“I didn’t win an award for highest lead. Right now, [Resident B’s] got that … [Resident B] likes tiaras. You get a lead tiara, baby doll”||Mays 2017|
|8||“They also say you cannot get lead poisoning through showering. So, here’s [Resident B’s] blood lead level. 38.4 [μg/dL]”||Mays 2016–18|
|9||“This gorgeous #Flint baby is only 3 months old and was born w/high lead and e. coli b/c mom took baths while pregnant”||Mays 2017–18|
|10||“You get two times the exposure in a 10-minute shower than you do drinking two liters of the same water” <a Water Defense claim>||Moore and Mendes 2017|
|11||“And the next person who suggests
our kids had higher lead levels prior to the Water Crisis is going
to get a public shaming for shoddy and LAZY research and
<Responding to actual data showing exactly that>
|12||“Residents have THOUSANDS of reasons (AND ppb of lead in our water) to NOT trust the “it’s all better” lie forced upon us”||Mays 2017–18|
|13||“Transdermal absorption of lead [from shower water] is an issue” <shared alongside a Medscape citation that states the opposite>||Mays 2016–18; Edwards and Roy 2018|
|14||“Shower filters are [quickly] spent because of the [corrosion control chemicals]”||Mays 2016–18|
“Unfortunately, the media have trouble distinguishing between real science and propaganda cross-dressed as science.” Linda Bowles, columnist (Harmon 2000)
Water Defense initially seemed undeterred by either bad publicity or worrisome news of the unfolding Shigella outbreak, and issued a press release stating that they “never suggested or implied that [their] concerns outweigh the dangers of not bathing” (Water Defense n.d.1) even though media coverage (television and social media) interpreted their message otherwise (Figure 2). In December 2016, a political reporter named Jordan Chariton from The Young Turks Network (TYT Politics) began to work closely with Water Defense. In East Chicago, IN, Mr. Chariton carried out “water testing” under Mr. Smith’s guidance in the basement of one home, by scraping the rust buildup on the outside of a sewer pipe with a goal of finding “bacterial fungi” that the EPA could not find, while collecting water in unsanitary plastic bottles from a water sediment clean-out tap on a water heater (TYT 2016b).1 The Chariton-Smith duo followed this with testing in at least three other East Chicago homes using the WaterBug™ sponge and other improper sampling methods, while livestreaming and/or broadly disseminating video interviews with residents presented with worrisome test results on YouTube and Facebook (Roy and Edwards 2017; TYT 2017b; TYT 2017c).
As a TYT Politics reporter, Mr. Chariton engaged in Flint between Dec 2016–Oct 2017 by holding interviews, conducting a Town Hall meeting, and uploading videos questioning Flint’s improving water quality based on anecdotal evidence from residents and prominently citing Mr. Smith’s WaterBug™ data (TYT 2016a; TYT 2016c; TYT 2017a; TYT 2017d). Mr. Smith called out traditional Flint and national journalists as practicing “contrived reporting,” characterizing the multiple sources of evidence that Flint water quality was improving as “fake news,” and accusing normal media of being sellouts (Roy and Edwards 2017). Chariton and Smith also pushed a conspiracy theory that the Virginia Tech citizen science team was no longer being truthful and had “sold out” residents because we received funding from the EPA and State of Michigan. TYT even hosted a prime-time news report centered on making an edited innocuous private phone conversation between the leader of our Flint Water Study citizen science team (Dr. Edwards) and a Flint resident (recorded without consent) into a conspiracy about hiding dangers from residents (Smith 2016d; TYT 2017a). Chariton also attacked the EPA and State of Michigan repeatedly on air (Chariton 2017; TYT 2016d; TYT 2017e; TYT 2017f).
In collaboration with some Flint residents, we again made a difficult decision to directly undermine this false messaging, which we deemed dangerous to the public welfare. To achieve this objective, we compiled vignettes from Water Defense and TYT Politics’ published YouTube videos to highlight obvious improper water sampling methods in an online blog post (Roy and Edwards 2017). The videos documented invalid methods, improper sampling locations, unsterile protocols for microbial sampling, and misrepresentation of some test results–in essence, revealing the lack of rigor and poor scientific understanding by Water Defense and TYT. After the blog post went live, Mr. Chariton was visibly angry, and wrote a series of threatening emails demanding that we remove the blog post or face a “public dispute with the largest online news channel in the world,” which would include future feature stories that “prove [Dr. Edwards is] a hack” (Edwards and Roy 2017a). He also threatened lawsuits and at one point hired a lawyer. We did not delete the videos, and were indeed attacked continually online by Mr. Chariton, until he was finally fired from the network in November 2017 after acknowledging sexual relations with subordinates in Flint (Haze 2017; Lima 2017). This chance event, and not the unscientific testing, erroneous reporting, or conspiracy theories, was the only reason that Chariton’s Flint reporting was temporarily halted. Indeed, after just a few months out of the spotlight, Chariton returned in 2018 as an unaffiliated journalist, declaring that the State of Michigan was perpetrating “ethnic cleansing” of Flint residents and renewing his public attacks on the State of Michigan and our team (Edwards and Roy 2018).
“George Orwell first noted, the true genius in advertising is to sell you the solution and the problem.” Ben Goldacre, Bad Science (Goldacre 2011)
In December 2016, a Flint family began posting videos on Facebook, praising a whole-house filter “package deal” from a company called NLP Aqua Solutions that would cost US $11,300–$13,800 per Flint household to install (Murphy 2016; Edwards and Roy 2017b). Mr. Smith and NLP representatives appear in some of the videos (Murphy 2016; Murphy 2017), including one where the resident read the following aloud from a document allegedly written by Mr. Smith:
“[NLP] is the first filtering technology I have reviewed with test results that makes me comfortable in saying the water is safe for drinking, showering, and bathing…. I look forward to taking a shower at your house and drinking the water after the treated NLP Aqua Solutions technology” (Withstand With Wolves 2017; S. Smith, personal communication, Jun 28, 2018)
Given Mr. Smith’s affiliation with Ruffalo’s Water Defense, this statement seemed to imply a celebrity endorsement of a for-profit, proprietary product, amidst the Federal Emergency and widespread fear of bathing/showering that Water Defense itself helped to create. The Flint family also referenced Smith as stating that their particular filtered water was “safer than bottled water” (Withstand With Wolves 2017). After determining that these filters cost 5–7 times more than other off-the-shelf filters certified to remove more contaminants, we noted that the package deal also included water treatment devices called “conditioners” (Figure 3) that the company sales literature stated could soften water without using any chemicals and could protect home plumbing and appliances from damage. The NLP company website made claims that this specially conditioned water boosted plant root growth and even increased milk production when consumed by cows. We tested this device in the laboratory and found that it did not perform as claimed, consistent with assertions by other Internet sources that this conditioner was “junk science” (Lower 2013; NLP Aqua Solutions 2016; Murphy 2017; Edwards and Roy 2017b).
Our effort to expose what we then considered to be profit-minded, unscientific, and harmful activities by Mr. Smith while representing Water Defense resulted in numerous “Cease and Desist” letters from Mr. Ruffalo’s lawyers. We did not cease or desist, but eventually uncovered additional evidence of perceived financial conflicts, including social media postings on the filter package deal where US $2,100 was apparently earmarked for testing by Mr. Smith, and letterheads of lab reports given to Flint residents that had Mr. Smith’s for-profit “Aquaflex™” address on them (Edwards and Roy 2017b). These revelations finally resulted in Ruffalo’s nonprofit and Mr. Smith parting ways in June 2017, perhaps to protect Ruffalo’s celebrity or because their arrangement violated Water Defense’s commitment of not “hav<ing> any […] connection with [Aquaflex™]” (S. Ganim, Personal Communication, Feb 13, 2018; BJ Chisholm, Personal Communication, Feb 16, 2018; Water Defense n.d.2).
“Tell a lie once and all your truths become questionable.” Anonymous
In our initial sampling efforts in August 2015, the citizen science team of residents and scientists exercised extreme care to accurately collect samples, developing tamper-resistant sample kits, screening videos on proper sampling techniques to residents whenever possible, and collecting samples ourselves when we thought that residents might not follow the protocols. The citizens correctly anticipated that if our team found a citywide lead problem, both our motivation and methods would be questioned, and they wanted data to withstand scrutiny. After all, our team was directly challenging the power and conclusions of MDEQ and US EPA. When one reporter asked Virginia Tech “How do you know Flint residents are not just adding lead to the water to gain attention?”, we cited the care in sample collection shown by the residents and some high water lead samples collected exclusively by our Virginia Tech team.
After the initial triumphs following our citywide sampling event and unprecedented publicity, the tables were completely reversed. Nearly any sampling result put forth by residents, or any hypothesized link between ailments and tap water, could be broadcast in the national and international media frenzy. Water Defense, TYT Politics, and some citizens actively began to use improper sampling methods that obtained artificially “high” water lead results, which could be used to gain media attention or to push an agenda.
Resident A confirmed to us in writing in 2015 that her home had a copper service line and modern lead-free plumbing (i.e., no lead solder) inside her house. The maximum water lead level detected at her kitchen faucet from five samples collected during the crisis period with highly corrosive Flint River water was 8 ppb. This was below the 15 ppb EPA action level and consistent with her initial claims that her plumbing had no lead solder or lead pipe. Her results changed dramatically (150 ppb) after she began to openly collaborate with Mr. Smith and collected samples from a hose bib in her basement, which is an improper sampling location according to US EPA because it creates false positive (high lead) results (Edwards 2017). Despite willfully violating the written protocols in test kits provided by the State of Michigan or Virginia Tech, which clearly indicate to test only proper locations using the kits, Resident A’s high lead results were broadcast by Mr. Smith and Mr. Ruffalo on social media with no mention of the improper sampling approach. As time went on, her lead results rose to 160 ppb and then went “sky-high” to 1,740 ppb during our fourth round of citizen science testing (Figure 4). After getting these results, Resident A held press conferences, gave interviews, spoke at the 2017 Women’s March in Washington D.C. advertising her lead levels “in the 4-digits,” and also disseminated her results via gullible national media to assert that water quality in Flint was actually getting worse (Mays 2016b; Mays 2016–18; Smith 2016a; Ruffalo 2016; PRWeb 2016; Scipion 2016; Corkins 2017; Edwards 2017; Mendes and Moore 2017). Our team reported her improper sampling to the US EPA and told Resident A that she could not engage in further citizen science sampling with our team (Edwards 2017).
Resident A also attempted to use these high lead results in a manner that illustrated a possible financial conflict of interest. For example, in one publicly accessible social media posting, she cited her high lead result, and desire to be given a free service line replacement, which at the time was prioritized for residents who had dangerous lead pipe (Walters 2017). After other residents called her out for this opportunism, and produced our written documentation showing that Resident A believed her service line was copper, Resident A deleted the posting. Over 2015–18, her claims of illness changed from rashes and hair loss (Smith 2015, The Tom Sumner Show 2016) to being poisoned and “lead and copper [permanently] stored in [her] brain” from water (Mays 2017–18, Mendes and Moore 2017). She insisted, without support, that “the longer [the government is] waiting to pull the [lead] pipes out, the bacteria’s worse, all the different cancer-causing byproducts are worse” (Mcfarland 2017). She was also lead plaintiff on several lawsuits (Mays et al. v. City of Flint et al. Genesee County Circuit Court; Mays et al. v. Snyder et al. Michigan Court of Claims; and Mays et al. v. Snyder et al. US Federal Court for the Eastern District of Michigan) in which high lead data played a role (Flint Water Class Action 2016–17). At least one media report of her improper high lead test result from Virginia Tech was explicitly tied to her personal lawsuits (Mendes and Moore 2017).
Resident A publicly displayed questionable citizen science judgment even before she started working with Water Defense. For example, she posted a publicly accessible video on Facebook in January 2016, which was viewed 17,000 times, where a family member evaluated state-supplied bottled water using a “water tester.” The water tester was a Total Dissolved Solids meter that measured dissolved salts in water, not lead, but the family member claimed that it showed the bottled water was “horrible” for consumption (Water You Fighting For 2016, Roy 2017). During October 2015–January 2016, she claimed that “21,000 people” in Flint “lost their homes” because they couldn’t afford water bills, that Child Protective Services threatened to remove children from Flint households due to bad water, and that Flint residents were legally forbidden from selling their homes due to the water crisis–all of which were debunked by outside sources (GMO Free News 2015, RT America 2016, Snopes 2016a, Snopes 2016b). More recently, Resident A shared a 2015 Flint photo of a fire hydrant spewing discolored water on her public social media page, claiming it was actually from late-June 2018 (Figure 5) and alleging that it illustrated “legal genocide” of Flint residents (Mays 2017–18, Roy and Edwards 2018b). The photo was liked and shared tens of thousands of times before it was debunked by our team and others (Roy and Edwards 2018b). After this was exposed, Resident A stated “Here is yet ANOTHER example of [the Flint Water Study team] taking it upon themselves to attack poisoned Flint residents and call them liars” (Edwards and Roy 2018). It has been argued elsewhere that some activists do not believe in or respond to scientific evidence, especially if it challenges their personal ideologies and political goals (Dreger 2015).
Other illustrative comments from this resident (see Table 1 for a representative list) included claims that the Flint water was transforming bathroom showers into “gas chambers” and, therefore, “until every single piece of plumbing is replaced, [Flint residents] are in danger” (LIFETIME 2017; TATM 2017). She openly attacked all the agencies, Virginia Tech, and other residents who presented data indicating that the water quality was improving. Throughout, this resident’s disruptive activities and statements were rarely refuted, in fact they were openly embraced, encouraged, and publicized by two supportive professors who were engaged in Flint (Lambrinidou 2016–18; Mays 2016–18; Sullivan 2017; CAN TV 2018).
A few months after we had reported Resident A to EPA for collecting improper samples in our federally funded citizen science project, Resident B had a water lead level sampling trend even more startling than Resident A. During the recovery phase of the system prior to compliance with the federal Lead and Copper Rule in July 2016, when there was still a citywide problem, 12 analyzed water samples from her home were between 5–44 ppb. However, 10 months after Resident A first publicized an approach to collecting improper samples (attributed to Mr. Smith) that obtained very high water lead data, Resident B’s lead levels reached and surpassed hazardous waste levels (>5,000 ppb) in July 2017 (Figure 6), triggering both alarm and widespread media coverage (Pierret 2017). Resident B reportedly had a copper service line.
Perplexed as to how this could be, we submitted a Freedom of Information Act (FOIA) request to the US EPA which revealed that Mr. Smith and a colleague discovered two lead fishing sinkers inside her plumbing in early August 2017 (Edwards and Roy 2017c). Before we made the FOIA public, Resident A, Mr. Chariton, and other residents had broadcast Resident B’s high lead results on social media, and her data were also cited at the premier of the LIFETIME movie to support the assertion that the water lead crisis was continuing in Flint (Cher et al. 2017; Hammond 2017; Roy and Edwards 2017d). We also revealed that Resident B had possible financial conflicts of interest, as evidenced by launch of a personal GoFundMe online fundraiser that cited her high water lead results, and interviews in which she claimed that all “[water] mains, service lines, everybody’s […] hot water heaters, refrigerators, washers, everything” should be replaced (ECH 2017; MDEQ 2018a). Resident B is also part of a class action lawsuit along with Resident A (Edwards and Roy 2017d).
The timeline of her public social media postings also documented formation of her personal hypothesis that Flint residents could get lead poisoning simply by taking a shower (Smith 2016b; Marx 2017; Roy and Edwards 2017d). This hypothesis had been frequently cited publicly by Mr. Smith, despite the fact that it was contradicted by decades of sound research from reputable agencies, scientists, and engineers. Mr. Bowcock also called Smith’s hypothesis about lead poisoning from showers “ridiculous” (R. Bowcock, personal communication, May 16, 2016). Taking on a role of a biomedical research pioneer, Resident B put forth the lead poisoning by showering hypothesis in an interview to Mr. Chariton’s TATM channel on June 23, 2017, and then took a test proving her blood lead tested low at 2.7 μg/dL just two days later. Then, just six weeks later, she shared a blood test result indicating extreme lead poisoning (38.4 μg/dL) on social media by posting: “you most certainly can get lead poisoning by showering in toxic water!” (Marx 2017). A deluge of social media and local media coverage of this discovery followed. But importantly, at no point was any public mention made of the potentially important discovery of the lead fishing sinkers inside her plumbing by Resident A, Resident B, or Mr. Chariton.
After we published the FOIA with videos of the lead sinkers found in her plumbing and laid out the timeline of events online, including her GoFundMe launch (claiming that she was poisoned before obtaining her high blood lead test results), Chariton wrote that the sinkers had “NOTHING to do with [Resident B’s] high lead-levels” but her lawyer nonetheless barred her from talking to the media (Conat 2017; Fonger 2017b; Roy and Edwards 2017d). Resident B’s lawyer first claimed that no lead sinkers were found inside her bathroom tap, but then retracted the statement and, finally, made a claim that the lead sinkers would not matter anyway because they would not leach any lead. We disproved that statement with our own experiments using lead sinkers, and one of us (MAE) even took a shower in toxic levels of lead with no detectable biological impacts (Edwards and Roy 2017d). Her lawyer then labeled the lead sinker story a “distraction” from the class action lawsuit that Resident B was party to and suggested that “a big defamation lawsuit” could be filed against us. The resident’s GoFundMe page disappeared within approximately two weeks of our blog posting (Roy and Edwards 2017d). The state has declined to release other emails associated with this case under the Freedom of Information Act, stating that this “could jeopardize an investigation by [EPA’s] Criminal Investigation Division” (Fonger 2017c).
Resident A (and members of her social network) repeatedly used Resident B’s astronomically high lead results to manufacture outrage regarding Flint’s water quality (e.g., Mays 2017), even months after the lead sinker issue was exposed on our website (e.g., CAN TV 2018). At the launch event of the LIFETIME movie “FLINT” in October 2017 featuring Resident A as the heroine and a false portrayal of Ms. Walters as a quitter with an unsupportive husband (Cher et al. 2017), the high water lead data from Resident B were cited to assert in the media that “[Flint residents] are still being poisoned” (Mcfarland 2017). Again, there was no mention of the fact that Mr. Smith had reported a discovery of lead sinkers in her plumbing to EPA. Resident B also never mentioned the lead sinkers at an MDEQ public hearing as late as March 2018 when reporting her high water lead results (MDEQ 2018a).
“There is nothing to fear except the persistent refusal to find out the truth.” Dorothy Thompson (BrainyQuote 2018)
“When the facts change I change my mind, sir. What do you do?” Attributed to John Maynard Keynes
The aforementioned actions of a nonprofit, an online news network, and a few residents (and unequivocal support of such actions from a few academics) suggest abuse of the citizen science approach to research including: Data falsification, failing to disclose and manage conflicts of interest, making scientific claims without evidence, or co-opting the term “citizen science” as a tool for activism. The events also highlight legitimate concerns, ethical dilemmas, and perverse incentives surrounding citizen science, at least in this unusual situation. The repeated and irresponsible practice of bad science to attack and disrupt expertise of government science agencies and others during the Flint Water Crisis relief effort endangered the public welfare and undermined trust in science itself.
We have been outspoken about the danger of perverse incentives in modern academia, where promotion and tenure are increasingly tied to quantitative metrics including research papers and funding dollars and which, in turn, increase the likelihood of misconduct and threaten scientific integrity (Edwards and Roy 2017e). Flint demonstrates that perverse incentives also exist for citizen scientists, especially if they find answers that lead to financial rewards and media attention (i.e., a LIFETIME movie portrayal, class action lawsuits, and GoFundMe campaigns). It is possible that such incentives are not as “anomalous” as previously assumed (Ottinger 2016), especially in cases involving lawsuits and environmental injustice.
“The method of science, as stodgy and grumpy as it may seem, is far more important than the findings of science.” Carl Sagan
Freitag and colleagues (2016) list 12 strategies to enhance credibility of citizen science projects, including prior expertise of project leaders, substantial training of citizen scientists, science advising to strengthen data collection, peer-oversight between citizen scientists, quality assurance protocols, and cross-comparison of citizen science data with data from professionals. This approach emphasizes rigor and objectivity and views science as a truth-seeking exercise, albeit subject to the same limitations and biases that all science is subject to. We evaluated our initial citizen science collaboration that exposed the Flint Water Crisis against Freitag et al.’s criteria and believe that it satisfied all 11 applicable criteria. The Water Defense-TYT Politics alliance, in contrast, arguably satisfied only two (see Table 2).
|No.||Strategies for demonstrating credibility||Assessment of VT-Flint citizen science collaboration||Assessment of Water Defense-TYT Politics alliance|
|1||Prior expertise (is there a “formalized minimum standard that volunteers must meet to participate?”: Yes, No)||Not Applicable; water samples for lead are normally collected by residents under the federal LCR||Not Applicable; anyone can “take water quality testing into their own hands” using WaterBug™ (Ruffalo 2015)|
|2||Training (“time investment” to train volunteers: None, Low, Medium, High)||Low; required viewing of YouTube sampling video||None for WaterBug™|
|3||Science advising (“partnership with a university lab”, etc.: Yes, No)||Yes; with Virginia Tech scientists||Yes; but science advisors listed had no drinking water expertise (Water Defense n.d.3, Smith 2018)|
IN THE FIELD|
|4||Ranking system (Volunteers designated as “experts” after gathering experience or passing tests: Yes, No)||Yes; for select residents. Flint residents paid by EPA to execute citizen sampling||No; Any citizens can be experts and “take water quality testing into their own hands” using WaterBug™ (Ruffalo 2015)|
|5||In-person oversight (“expert” volunteers to “directly oversee data collection” to minimize data collection errors: Yes, No)||Yes; cross-checking survey data between citizens||No; Mr. Smith did not initially have expertise in EPA protocols and collected many improper samples (Roy and Edwards 2017)|
|6||Retraining (Advancement of volunteer skills through more trainings, readings, etc.: Yes, Optional, No)||Yes; select residents sampling for chlorine||No; WaterBug™ was hypothesized as useful for all contaminants, including heavy metals, pathogens, DBPs|
|7||Technological aids (Simplifying data collection using technology: Yes, No)||Yes; Mostly using Excel||No; “independent lab” provided data dump in “lab reports” of up to 97 pages (TYT 2017a)|
IN THE OFFICE|
|8||Validation of observations: (Checks for human error and “statistics-driven flagging of incorrect data,” etc.: Yes, No)||Yes; Attempt for rigorous evaluation of collection methods for each sampled home||No|
|9||Cross-comparison (Comparing volunteer data to those collected by scientific experts to demonstrate credibility of “methods and data”: Yes, No)||Yes; samples collected from same locations by citizen scientists and third-party experts||No|
|10||Publication (external peer-review and/or publishing of data and findings: Yes, No)||Yes; several papers published in respected journals||Yes; peer-review of data by scientists (but none with drinking water expertise), no journal articles|
|11||Management use (Decision-makers use citizen science data: Yes, No)||Yes; informed Federal Emergency declaration and EPA funded work||No; Attempt was made to undermine decision-makers|
|12||Quality assurance protocol (Standard QA practice to “calibrate methods, technology, and practice over time”: Yes, No)||Yes; as per Standard Methods and EPA guidelines||No; WaterBug™ not vetted against established EPA/WHO guidelines|
Above and beyond the framework of Freitag, the Flint experience and additional research (e.g., Resnik et al. 2015) has highlighted other (qualitative and ethical) characteristics of good citizen science which are lacking when citizen science can be hijacked by science anarchists and social justice advocates with agendas other than truth-seeking. These include embracing scientific rigor; willingly reporting evidence counter to one’s previous expectations or goals (i.e., intellectual honesty); working only in one’s area of competence; transparency; meeting obligations to correct misinformation; and openly addressing conflicts of interest.
“Democracy cannot function when every citizen is an expert. Yes, it is unbridled ego for experts to believe they can run a democracy while ignoring its voters; it is also, however, ignorant narcissism for laypeople to believe that they can maintain a large and advanced nation without listening to the voices of those more educated and experienced than themselves.” Tom Nichols (Nichols 2017)
It could be argued that citizen science attempts to reduce the worrisome power and expertise gap between experts and non-experts through collaboration, respect for an individual’s experiences and observations (“other ways of knowing”), and working together to create new knowledge (Wynne 1989; Barth 1995; Collins and Evans 2002; Allen 2003; Backstrand 2004; Yankelovich 2005; Brunner 2006; Alessa 2009; Dosemagen and Rolfes 2010; Ottinger 2010; Stange 2010; Hall 2013; Pigliucci 2014; Mikulak 2011; Eller 2016; UCS n.d.). These citizen science collaborations have often served as catalysts for scientific discovery and exposed abuses of power and scientific misconduct (Ottinger 2016; Ottinger 2017), as illustrated in Tonawanda NY, Washington D.C., and Flint where local knowledge, experience, and advocacy were essential (Shogren et al. 2011; DOJ 2014; Lambrinidou et al. 2014; Averett 2017; Cooper 2017). If scientists ignore individuals with local knowledge and expertise, the results can be disastrous (Wynne 1989; Evans and Collins 2007).
In contrast, the disruptive actions of Water Defense, TYT Politics, and Flint Residents A-B could be considered an unqualified extreme of “anything goes” in a “citizen engineering” model, which was recently defined as an exercise by which “community members (“non-experts”) identify scientific questions and proceed through a formal process … sometimes but not always with the cooperation of trained scientists,” where “… disruption of engineering expertise is a central goal….” (Riley et al. 2016). This model was best demonstrated when Resident A made several dubious scientific claims during a February 2018 public health panel (Table 1) at the University of Illinois Chicago, which aimed to “tell stories from community members […] to get scientific […] facts [about issues]” (CAN TV 2018). At this event, Resident A stated:
“[The State of Michigan is] willing to kill people [for their land]. They don’t care. […] They are going to clear that land out [and] knock the home down [for rich developers].” (CAN TV 2018)
without any contradiction from other panelists, moderators, or audience members. She also repeated the false narrative about bathing and showering dangers:
“We get told by a certain PhD that […] you poor, uneducated, dumb and dirty people in Flint. You have shigella because you didn’t wash your hands. Because you are too scared of the water. Excuse me. We have shigella because we wash our hands.” (CAN TV 2018)
Resident A used similar “dumb and dirty” quotes on her public Facebook page and in The New York Times Magazine (Hohn 2016; Mays 2016–18) to discredit the warnings from the county health department and other authorities about possible dangers of altered bathing habits. She attacked the Federal, State, and Country authorities in one post entitled “Blaming the Victims: The Lies our County, State and Federal Government Tell about Flint.” The post reiterated that government officials’ messaging about bathing was a “LIE” that “Flint is full of dumb and dirty people who get sick because they’re not bathing and washing their hands” (Mays 2016–18). She further stated “Shame on you people for saying garbage like that just to try to throw off the fact that the bacteria in our water is making us sick,” and that she was “ready to turn those shaming tables back around on you #ItsYourTurnNow” (Mays 2016–18).
Interestingly, Resident A was a contributing member of our initial Flint Citizen Science team until late 2015–early 2016 when she started to collaborate with Water Defense and TYT’s Jordan Chariton, whereas our team chose to work alongside the very agencies that we had previously attacked after they apologized. Resident A seemed to remain stuck in attack mode, even after the heroic EPA whistleblower Miguel Del Toral who originally worked with Walters was recruited to become the public face of the EPA response (Edwards 2016e; Fonger 2016).
We note that the best scientists, with decades of training, must always be on guard to minimize the likelihood of self-deception and politicization of their work (Curry 2004; Pellizzoni 2011; Nuzzo 2015; Mirowski 2017). A competent scientist has developed self-awareness, moral intuition, and a strong sense of humility, especially in their proven area of expertise (Margolis 1997; Kruger and Dunning 1999; Klein 2016). Scientists are looked at as “purveyors of truth” (Marmot 2017), which is indeed an ideal worth aspiring to while communicating both facts and uncertainties. This perception of scientists was also evident in a Spring 2016 Flint poll that showed “University Scientists” (i.e., Virginia Tech) as being “Helpful and Trusted” by Flint Residents (Gray et al. 2016) and the “whistle-blowers for Flint” (Maddow 2016). The general public does trust scientific expertise, with Pew and Gallup polls in 2016 showing that more than 75% of Americans trust scientists to act in the larger interest and 65% rate engineers’ honesty and integrity as “High” or “Very High” (up from 48% in 1976), respectively (Gallup 2016; Kennedy 2016). In 45 years, “science is the one [public] institution that has not suffered any erosion of public confidence” (Dastagir 2017). Similar trends can be seen in Europe (European Commission 2012).
On the other hand, citizen science abusers who openly embrace subjectivity and attack all expertise embody the Dunning-Kruger Effect (Figure 7), which is summarized: “Those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it” (Kruger and Dunning 1999). The chronicled examples illustrate a populist anti-elitist, “all opinions are equally valid” anarchist movement that aims to undermine expertise, scientific rigor, and organized knowledge. This is also embraced in the “Citizen Engineering” academic model that is occasionally taught to non-scientists on U.S. college campuses, which can be viewed as supporting anti-expertise social justice movements (Riley et al. 2016; Nichols 2017; Edwards and Roy 2018; McIntyre 2018). With no scientific background or legitimate experience in water quality or engineering, Mr. Smith (Water Defense), Mr. Chariton (TYT Politics), and Flint Residents A-B were truly “unskilled and unaware of [their incompetence]” (Kruger and Dunning 1999). When they took the spotlight and engaged in citizen science abuse in post-Federal Emergency Flint, the results certainly did achieve the goal of “disrupting expertise” in “communities,” but without any targeted desirable effect of making “crucial contributions that improve engineering practice” (Riley et al. 2016).
To be clear, we are not opposed to activism that stems from legitimate science practiced by or with citizen scientists, sometimes termed “scientific authority-driven based citizen science” (Ottinger 2016; Ottinger 2017). The citizen scientists of Flint, MI and St. Joseph’s, LA engaged in such activism with the expert assistance of Virginia Tech and Louisiana State University (Edwards et al. 2016; Oransky and Marcus 2017), which led the states of Michigan and Louisiana to declare public health emergencies and provide millions in relief funds (Edwards 2016f; Hersher 2016). We also believe that we were morally and ethically obligated to call out bad expertise of authorities (e.g., MDEQ and US EPA) and agenda-driven science done to “reflect prejudices and interests–economic, political, ideological–of powerful groups” (Horgan 2014), which led to grossly unethical and harmful actions (Krauss et al. 1992; Margolis 1997; Morua et al. 2011). In the aftermath of disease outbreaks and disasters that require multi-pronged trust-building efforts, the problem certainly cannot be fixed by simply bringing in more expertise (Sjoberg 1999; Yearly 2000). But it also cannot be fixed by unrelenting and unfair attacks on expertise as documented in this paper. Finally, there is indeed “space for negotiation” (Nowotny 2003) between traditional expertise and local knowledge to allow for co-creation of new knowledge with “credible” data quality that is “[fit] for use” (Holdren 2015), without succumbing to citizen science anarchy where merited scientific expertise is disrupted with a goal of advancing populist agendas.
As we were making final revisions to this case study for publication, Mr. Smith (formerly of Water Defense) reached out to US EPA and expressed an interest in initiating:
“A private discussion with [Flint Water Study] about how his data is being misrepresented by the less-than-scrupulous activists in Flint” and supporting the official position of experts and agencies “on the progress of the Flint distribution system” (M. Durno, personal communication, June 14, 2018).
We later learned that Mr. Smith had previously sought to file a defamation case against Dr. Edwards, but was told that his best course of action would be to correct his errors publicly. He had previously written to EPA and others, threatening defamation cases against numerous EPA employees including heroic whistleblower Miguel Del Toral (Edwards 2018). After considering the situation, we decided to work with Mr. Smith to correct the record, clear his name of several misunderstandings, and to obtain insights on his citizen science “lessons learned.”
Mr. Smith acknowledged mistakes with his testing, communication, and resident interactions, and admitted that some of his work could have been construed in a manner that changed residents’ bathing and showering habits. He noted that the many samples he took per EPA approved methods gave results consistent to those of scientists, but the WaterBug™ data were not comparable (S. Smith, personal communication, Nov 28, 2018). After writing a blog post on our website clarifying many issues (summarized in Exhibit 1), he then bravely owned up to his mistakes in the media including stating that “[Smith] says he made mistakes about dangers of bathing in Flint water” and “Ruffalo’s Former Associate Recants Flint Water Claims” (Delaney 2018, Fonger 2018). We openly applauded his candor and helped set the record straight on our website, and we amended the introduction of our prior adversarial postings accordingly (Edwards 2018b). We also acknowledged that there was no evidence that he profited from any business dealings in Flint, in spite of the many perceived conflicts of interest.
Exhibit 1: Lessons I Learned in Flint by Scott C. Smith (Abridged Version) (adapted from complete posting available at http://bit.do/scott-smith; edited and reorganized for clarity).
After doing what most would consider to be the “right thing” and apologizing, Mr. Smith was then viciously attacked by the same groups he was previously allied with. The Flint resident who claimed to have benefitted from the NLP filters wrote him: “How much money did it take for you to do this to us. People’s lives are more important than money. What happened to you. I’m letting the navio (sic) nation know you did this to us. No one is going to trust you again. So I hope you got a great pay check for this” (S. Smith, personal communication, Jul 11, 2018). Resident B, whose lead sinkers in plumbing were ethically and properly reported to EPA by Mr. Smith, promptly texted Mr. Smith “Et tu Brutus” (S. Smith, personal communication, Aug 14, 2018). Still another resident wondered if Smith was being “used” by Dr. Edwards (S. Smith, personal communication, Aug 14, 2018). A lawyer representing Resident A in a lawsuit wrote EPA and others stating that Mr. Smith was now among “idiots” that included “Marc Edwards, Mark Durno <EPA>, Miguel Del Toral, the EPA, Virginia Tech’s officers, employees and agents.” Smith now reflects that “coming out with the truth, was like trying to leave a cult” and that the “discovery of the lead sinker was a clear sign that citizen science had run amok” (S. Smith, personal communication, Oct 15, 2018). To his credit he wanted no part of it, paid a high price for doing the right thing, and eventually spoke to our graduate ethics class at Virginia Tech about his experiences (Edwards and Roy 2017e).
“If we overlook the ethical problems that [citizen science] raises we may end up unintentionally strengthening the lay–expert boundaries that [citizen science] was thought to overcome.” Hauke Riesch and Clive Potter (Riesch and Potter 2014)
“On one level, this story in Flint is about water. But on another level, it’s about trust in government, feeling like your voice matters and that elected leaders care about you. Pipes are hard to fix. Those other things are even harder.” Ari Shapiro, NPR’s All Things Considered (Shapiro and Gringlas 2018)
“Who can land on this earth and tell a Flint resident their water is fine? It’d have to be God.” Gina Luster, Flint resident (Glenza 2018)
The Flint Water Crisis, arguably a signature environmental injustice of the 21st Century, will be the subject of intense debate and analysis for years to come. In April 2018, the State of Michigan announced an end to distributing free bottled water to residents after water lead levels consistently registered below federal standards for four monitoring periods over two years, as supported by independent data from our own team and other researchers (Fonger 2017d; Tang et al. 2017; Masten 2018; MDEQ 2018b; Pieper et al. 2018; The Detroit News 2018). Nonetheless, the decision was widely portrayed as insensitive and premature by some residents who claim “chemicals and bacteria in the water are at an all-time high” and “I don’t trust the water. Period. It could be five years from now and I’ll still never drink this water” (Carmody 2018; Chavez 2018). From our perspective, every positive step forward by the agencies after the declaration of the Federal Emergency was countered by a two-step pushback by citizen science abusers, perpetuating a climate of victimization and a sense of distrust that will likely last generations.
This case study reveals both the promise and perils of citizen science in relation to scientific advancement, environmental justice, and achieving a public good. One viewpoint argues that citizen science is not about producing real knowledge, but “rejigging power relations” to get (often unpaid) citizens to produce data that benefits private corporations and academic researchers, with the citizen merely “mimicking scientific practice [without learning any science]” (Mirowski 2017), and that our own work in Flint exacerbated existing power imbalances by “replicating the very structural inequalities that render environmental injustices in the first place” (Lambrinidou 2016, 2017). We maintain that our citizen science collaboration, which revealed environmental injustice and possible crimes in Flint, was an ethical and appropriate emergency effort that “empower<ed> a public to critically use the tools of science for solving some of its problems, while also resisting the hegemony of the scientific framing of <MDEQ, USEPA and> others” illuminating the sometimes beneficial “politics of public participation in science” (Strasser et al. 2018). Lives literally depended upon our rapid research, public dissemination of the results, and investigative science model, serving as a possible road map if such an unfortunate situation were to occur again (Bates 2016; Edwards and Pruden 2016; Finnegan 2016).
This paper also demonstrates that no one is above the temptation to cheat, whether citizen or scientist, and that those who view citizens as incorruptible paragons of wisdom and virtue are at least as misguided as those who believe the same of scientists and engineers. It is important to note that Water Defense, TYT Politics, and Flint Residents A-B had no credible scientific reputation to lose when problems with their work were exposed. As a result, citizen science–in and of itself–may be even less capable of self-policing than conventional science. It is possible that the Post-Emergency Flint citizen science experience is an extreme anomaly not generalizable to normal citizen science projects. It is nonetheless important to highlight how citizen science abuse might jeopardize the public health and welfare, and the blowback directed against those who feel obligated to correct the scientific record.
As the US EPA and others embrace citizen science as a “core tenet of environmental protection” (NACEPT 2016; USEPA OIG 2018), important questions emerge from this case study that deserve to be answered:
The events described above, in combination with deeply rooted societal problems in post-industrial Flint, created a state of social and science anarchy that will have long-term repercussions for governance, regulation, environmental ethics, and citizen science.
1Sediment build-up is a function of source water, treated water chemistry, hardness and corrosion byproducts, is common in water heaters across the country, and not representative of the water at drinking taps.
We are grateful to LeeAnne Walters, Keri Webber, Jan Berryman, Patty Warner, and other Flint residents who demonstrated their commitment to truth-seeking throughout this ordeal. We would like to thank Dr. Amy Pruden, three anonymous reviewers, three editors of Citizen Science: Theory and Practice (CSTP)–Drs. Lisa Rasmussen and Caren Cooper and Rick Bonney–for helpful suggestions to the paper, and Mr. Scott Smith for sharing his experiences. We acknowledge the CitiSci 2017 Conference participants for their input during Ms. Walters and Dr. Edwards’ Keynote Event discussing the Flint case study. Thanks to Virginia Tech PhD student Anurag Mantha who collated data on Resident A’s test results. Finally, we gratefully acknowledge the support of Virginia Tech’s Open Access Subvention Funds (OASF) program toward funding CSTP’s open access publishing fees (£500).
MAE: During the time that this manuscript was being reviewed and revised, Dr. Marc Edwards became the subject of a letter writing campaign to Science, Technology, Engineering and Math (STEM) organizations, alleging that his behavior in Flint was unethical and deserving of investigation. Dr. Edwards has since filed a defamation lawsuit against Resident A, Dr. Lambrinidou, and another individual in relation to the letter and other events. The lawsuit Edwards vs. Schwartz et al. is ongoing. Dr. Edwards was also subpoenaed to testify as a fact witness in criminal cases against two State of Michigan employees, which are ongoing. Dr. Edwards and the Flint Water Study team’s records have been subpoenaed in hundreds of other lawsuits associated with the Flint Water Crisis. Dr. Edwards is not a party to any of these lawsuits, other than the defamation case.
SR: Dr. Siddhartha Roy collaborated with Resident A during the first Flint citizen science sampling event in August 2015. He also collaborated with Dr. Yanna Lambrinidou on a conference paper, teaching a graduate level class and a podcast interview (2012–15). Dr. Roy’s professional presentation on the case study presented in this paper at an academic conference (fPET 2018 on May 31 2018) was disrupted by Mr. Paul Schwartz. Schwartz also handed out copies of a letter, which Dr. Edwards alleged to be defamatory, to participants at this event. Dr. Lambrinidou and Resident A are both discussed in this paper and also named defendants, along with Mr. Schwartz, in Dr. Edwards’ lawsuit alleging personal defamation, which Dr. Roy is not a party to. However, the defendants’ lawyer, Mr. Bill Moran, sent Dr. Roy Twitter messages opining “[Dr. Edwards] should quietly exit this case,” and has repeatedly attacked him publicly on Twitter (e.g., “You are no victim, Sid. Grow the fuck up”).
This publication was partly funded and developed under Grant No. 8399375 “Untapping the Crowd: Consumer Detection and Control of Lead in Drinking Water” awarded by the U.S. Environmental Protection Agency to Virginia Tech. It has not been formally reviewed by EPA. The views expressed in this document are solely those of the authors and do not necessarily reflect those of the Agency. EPA does not endorse any products or commercial services mentioned in this publication.
ABC 12 Network. 2016. Marc Edwards: Wash hands to stop Shigellosis outbreak. ABC12. Oct 14. Available at: http://www.abc12.com/content/news/CDC-to-investigate-increase-in-Genesee-County-Shigella-cases-396815041.html [Last accessed 20 January 2018].
Acosta, R. 2016. Hand washing urged as Shigella bacteria cases triple in Genesee County. MLive. Sep 13. Available at: http://www.mlive.com/news/flint/index.ssf/2016/09/hand_washing_urged_as_bacteria.html [Last accessed 20 January 2018].
Adewunmi, B. 2017. Flint Isn’t Ready To Trust Anyone Yet. Buzzfeed News. May 24. Available at: https://www.buzzfeed.com/bimadewunmi/flint-isnt-ready-to-trust-anyone-yet [Last accessed 20 January 2018].
Alessa, L. 2009. The Other Way of Knowing. Cultural Survival. Sep 2009. Available at: https://www.culturalsurvival.org/publications/cultural-survival-quarterly/other-way-knowing [Last accessed 20 January 2018].
Allen, JM, Cuthbertson, AA, Liberatore, HK, Kimura, SY, Mantha, A, Edwards, MA and Richardson, SD. 2017. Showering in Flint, MI: Is there a DBP problem? Journal of Environmental Sciences, 58: 271–284. DOI: https://doi.org/10.1016/j.jes.2017.06.009
American Public Health Association; American Water Works Association; Water Environment Federation, (APHA, AWWA and WEF). 2017. Standard methods for examination of water and wastewater, 23rd ed.; American Public Health Association, American Water Works Association, & Water Environment Federation: Washington, D.C.
Andrews, T. 2016. Flint residents too scared of the water to wash. That’s making them sick. The Washington Post. Oct 4. Available at: https://www.washingtonpost.com/news/morning-mix/wp/2016/10/04/flint-residents-too-scared-of-the-water-to-wash-its-making-them-sick/ [Last accessed 20 January 2018].
Averett, N. 2017. New Blood: The Promise of Environmental Health Citizen Science Projects. Environmental Health Perspectives, 125(11). DOI: https://doi.org/10.1289/EHP2484
Backstrand, K. 2004. Scientisation vs. Civic Expertise in Environmental Governance: Eco-feminist, Eco-modern and Post-modern Responses. Environmental Politics, 13(4): 695–714. DOI: https://doi.org/10.1080/0964401042000274322
Barth, F. 1995. Other Knowledge and Other Ways of Knowing. Journal of Anthropological Research, 51(1): 65–68. DOI: https://doi.org/10.1086/jar.51.1.3630372
Bates, S. 2016. Flint water crisis: For young engineers, a lesson on the importance of listening. National Science Foundation. Mar 23. Available at: https://www.nsf.gov/discoveries/disc_summ.jsp?cntn_id=138060 [Last accessed 20 January 2018].
Bernstein, L and Dennis, B. 2016. Flint’s water crisis reveals government failures at every level. The Washington Post. Jan 24. Available at: https://www.washingtonpost.com/national/health-science/flints-water-crisis-reveals-government-failures-at-every-level/2016/01/23/03705f0c-c11e-11e5-bcda-62a36b394160_story.html [Last accessed 20 January 2018].
Bosman, J and Smith, M. 2016. Gov. Rick Snyder of Michigan Apologizes in Flint Water Crisis. The New York Times. Jan 19. Available at: https://www.nytimes.com/2016/01/20/us/obama-set-to-meet-with-mayor-of-flint-about-water-crisis.html [Last accessed 1 February 2018].
BrainyQuote. 2018. Dorothy Thomson Quote. Brainyquote.com. Available at: https://www.brainyquote.com/quotes/dorothy_thompson_393605 [Last accessed 1 February 2018].
Brockovich, E. 2015–18. Erin Brockovich. Twitter. Available at: https://twitter.com/ErinBrockovich [Last accessed 28 July 2018].
Brunner, RD. 2006. A paradigm for practice. Policy Sciences, 39(2), 135–167. DOI: https://doi.org/10.1007/s11077-006-9012-9
Business Wire. 2016. Aquaflex Update: Water Expert Scott Smith Launches Aquaflex™, Brings Clarity to Water Crisis. Business Wire. May 23. Available at: https://www.businesswire.com/news/home/20160513005406/en/Aquaflex-Update-Water-Expert-Scott-Smith-Launches [Last accessed 1 February 2018].
Cable News Network (CNN). 2016. Mark Ruffalo and Van Jones talk Flint & Obama’s visit. CNN Legal View with Ashleigh Banfield. Available at: https://www.youtube.com/watch?v=flCSxMIPQrU [Last accessed 1 February 2018].
Canepari, Z and Leduff, C. 2016. Flint and America’s Corroded Trust. Mother Jones. May–June. Available at: http://www.motherjones.com/environment/2016/04/flint-water-lead-crisis-snyder-mad-max/ [Last accessed 20 January 2018].
Carmody, S. 2018. In Flint, Residents Scramble To Get The Last Cases Of State-Provided Bottled Water. National Public Radio. Apr 10. Available at: https://www.npr.org/2018/04/10/601268214/in-flint-residents-scramble-to-get-the-last-cases-of-state-provided-bottled-wate [Last accessed 15 April 2018].
Chariton, J. 2017. Flint Poisoning: Would You Like to Deliver REAL Justice? Truth Against The Machine. Jul 31. Available at: https://truthagainstthemachine.com/flint-poisoning-like-deliver-real-justice/ [Last accessed 20 January 2018].
Chavez, N. 2018. Michigan will end Flint’s free bottled water program. CNN. Apr 7. Available at: https://www.cnn.com/2018/04/07/us/flint-michigan-water-bottle-program-ends/index.html [Last accessed 10 April 2018].
Chicago Access Network (CAN TV). 2018. Lies and Lead in Water: Poisonous Pipes in Midwest Cities. CAN TV. Feb 27. Available at: http://cantv.org/watch-now/lies-and-lead-in-water-poisonous-pipes-in-midwest-cities/ [Last accessed 22 March 2018].
City of Flint. 2014, Apr 25. City of Flint Officially Begins Using Flint River as Temporary Primary Water Source [Press Release]. Available at: https://www.documentcloud.org/documents/2696071-Snyder-Emails.html#document/p15 [Last accessed 22 March 2018].
Cole, TW and Fellows, KL. 2008. Risk Communication Failure: A Case Study of New Orleans and Hurricane Katrina. Southern Communication Journal, 73(3): 211–228. DOI: https://doi.org/10.1080/10417940802219702
Collins, HM and Evans, R. 2002. The Third Wave of Science Studies: Studies of Expertise and Experience. Social Studies of Science, 32(2): 235–296 DOI: https://doi.org/10.1177/0306312702032002003
Conat, R. 2017. Fishling lead sinkers found a Flint plumbing. ABC12. Sep 27. Available at: http://www.abc12.com/video?vid=448349533 [Last accessed 20 January 2018].
Corkins, M. 2017. Flint Water Activist’s Battle Continues Into Day 1,028, While US Cities Benefit. Observer. Feb 16. Available at: http://observer.com/2017/02/flint-water-protest-trump-womens-march/ [Last accessed 14 August 2018].
Curry, J. 2004. We are all confident idiots. Climate Etc. Nov 13. Available at: https://judithcurry.com/2014/11/13/we-are-all-confident-idiots/ [Last accessed 10 April 2018].
Dastagir, AE. 2017. People trust science. So why don’t they believe it? USA Today. Apr 20. Available at: https://www.usatoday.com/story/news/2017/04/20/science-march-war-truth-political-polarization/100636124/ [Last accessed 20 January 2018].
Davis, W. 2015. Water Quality Expert Come To Town. The Flint Courier News. Feb 19. Available at: http://theflintcouriernews.com/3144/ [Last accessed 27 July 2018].
Del Toral, MA. 2015. High lead levels in Flint, Michigan – interim report. WG-15J; U.S. Environmental Protection Agency Region 5. Chicago, IL. Available at: http://flintwaterstudy.org/wp-content/uploads/2015/11/Miguels-Memo.pdf [Last accessed 20 January 2018].
Delaney, A. 2016a. Mark Ruffalo’s Water Nonprofit Has Allied Itself With An Opportunistic Sponge Salesman. The Huffington Post. May 23. Available at: https://www.huffingtonpost.com/entry/mark-ruffalo-flint_us_5741ebc7e4b0613b512a8d25 [Last accessed 20 January 2018].
Delaney, A. 2016b. Mark Ruffalo Is Spreading Bogus Info In Flint, Scientist Says. The Huffington Post. May 16. Available at: http://www.huffingtonpost.com/entry/mark-ruffalo-flint_us_5739c801e4b077d4d6f37904 [Last accessed 20 January 2018].
Department of Justice (DOJ). 2014. Tonawanda Coke and Manager Sentenced for Violating the Clean Air Act and Resource Conservation and Recovery Act. U.S. Attorney’s Office, Department of Justice. Mar 19. Available at: https://www.justice.gov/usao-wdny/pr/tonawanda-coke-and-manager-sentenced-violating-clean-air-act-and-resource-conservation [Last accessed 20 January 2018].
Dosemagen, S and Rolfes, A. 2010. Oil Spill Crisis Map: Initial Analysis and Review. Louisiana Bucket Brigade and Tulane University. Available at: http://www.law.tulane.edu/uploadedFiles/News/News_Items/Current_year/Oil_Spill_Crisis_Map_Initial_Analysis_and_Review.pdf [Last accessed 20 January 2018].
Edwards, MA. 2016a. Institutional Scientific Misconduct at U.S. Public Health Agencies: How Malevolent Government Betrayed Flint, MI. Testimony to the U.S. Cong. Committee on Oversight and Government Reform on Examining Federal Administration of the Safe Drinking Water Act in Flint, Michigan Hearing. February 3, 2016. 112th Congress. 2nd session. Available at: https://oversight.house.gov/wp-content/uploads/2016/02/Edwards-VA-Tech-Statement-2-3-Flint-Water.pdf [Last accessed 20 January 2018].
Edwards, MA. 2016b. Failure of the U.S. Centers for Disease Control (CDC) and the U.S. Environmental Protection Agency (EPA) to Protect Children from Elevated Lead in Drinking Water: 2001–Present. Testimony to the U.S. Cong. Committee on Oversight and Government Reform on Examining Federal Administration of the Safe Drinking Water Act in Flint, Michigan Hearing Part II. March 15, 2016. 112th Congress. 2nd session. Available at: https://oversight.house.gov/wp-content/uploads/2016/03/Marc-Edwards-Final-3-15-2016.pdf [Last accessed 20 January 2018].
Edwards, MA. 2016c. In Flint Water Disaster Response, Ruffalo is a Bad Actor. Flint Water Study. May 23. Available at: http://flintwaterstudy.org/2016/05/in-flint-water-disaster-response-ruffalo-is-a-bad-actor/ [Last accessed 20 January 2018].
Edwards, MA. 2016d. A-List Actor But F-List Scientist: Mark Ruffalo Brings Fear And Misinformation To Flint. Flint Water Study. May 16. Available at: http://flintwaterstudy.org/2016/05/a-list-actor-but-f-list-scientist-mark-ruffalo-brings-fear-and-misinformation-to-flint/ [Last accessed 20 January 2018].
Edwards, MA. 2016e. Miguel Del Toral emerges from EPA’s House Arrest; FlintWaterStudy supports requested federal relief for Flint. Flint Water Study. Jan 16. Available at: http://flintwaterstudy.org/2016/01/miguel-del-toral-emerges-from-epas-house-arrest-flintwaterstudy-supports-requested-federal-relief-for-flint/ [Last accessed 15 April 2018].
Edwards, MA. 2016f. Clean Water for St. Joseph, LA: A Victory in the Battle Against Infrastructure Inequality. Flint Water Study. Dec 17. Available at: http://flintwaterstudy.org/2016/12/clean-water-for-st-joseph-la-a-victory-in-the-battle-against-infrastructure-inequality/ [Last accessed 20 January 2018].
Edwards, MA. 2017. Email to Bob Kaplan and Miguel Del Toral – Improper citizen sample according to 146.86(b)(2). Apr 8. Available at: http://flintwaterstudy.org/wp-content/uploads/2017/09/Documented-Problem-With-Wrong-Protocol.pdf [Last accessed 20 January 2018].
Edwards, MA. 2018b. Mr. Scott Smith and Flint Water Study. Flint Water Study. Jul 28. Available at: http://flintwaterstudy.org/2018/07/mr-scott-smith-and-flint-water-study/ [Last accessed 28 October 2018].
Edwards, MA and Pruden, A. 2016. We Helped Flint Residents Save Themselves and Are Proud of It—Staying in Our Ivory Tower Would Have Perpetuated Injustice. Environmental Science & Technology, 50(2): 12057–12057. DOI: https://doi.org/10.1021/acs.est.6b05347
Edwards, MA, Pruden, A, Roy, S and Rhoads, WJ. 2016. Engineers shall hold paramount the safety, health and welfare of the public — but not if it threatens our research funding? Oct 10. Available at: http://flintwaterstudy.org/2016/10/engineers-shall-hold-paramount-the-safety-health-and-welfare-of-the-public-but-not-if-it-threatens-our-research-funding/ [Last accessed 20 January 2018].
Edwards, MA and Roy, S. 2016. Should you bathe/shower in Flint water (or any water)? Flint Water Study. May 17. Available at: http://flintwaterstudy.org/2016/05/should-you-batheshower-in-flint-water-or-any-water/ [Last accessed 20 January 2018].
Edwards, MA and Roy, S. 2017a. TYT Network Threatens FlintWaterStudy: Demands Removal of Blog Post or Face Public Battle. Flint Water Study. Jun 7. Available at: http://flintwaterstudy.org/2017/06/tyt-threatens-flintwaterstudy-public-battle/ [Last accessed 20 January 2018].
Edwards, MA and Roy, S. 2017b. Ruffalo’s WATER DEFENSE EMPLOYEE Misleads Flint Residents: Supports Over-priced “Junk Science” Water Conditioners. Flint Water Study. Jun 17. Available at: http://flintwaterstudy.org/2017/05/ruffalos-water-defense-misleads-flint-residents-supports-over-priced-junk-science-water-conditioners/ [Last accessed 20 January 2018].
Edwards, MA and Roy, S. 2017c. Citizen Science in Flint: Triumph, Tragedy and Now Misconduct? Flint Water Study. Sep 26. Available at: http://flintwaterstudy.org/2017/09/citizen-science-in-flint-triumph-tragedy-and-now-misconduct/ [Last accessed 20 January 2018].
Edwards, MA and Roy, S. 2017d. Lead Sinker Story Timeline: Bizarre. Flint Water Study. Sep 29. Available at: http://flintwaterstudy.org/2017/09/lead-sinker-story-timeline-bizarre/ [Last accessed 20 January 2018].
Edwards, MA and Roy, S. 2017e. Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. Environmental Engineering Science, 34(1): 51–61. DOI: https://doi.org/10.1089/ees.2016.0223
Edwards, MA and Roy, S. 2018. Highlights of a Typical Week of “Citizen Engineering” in Flint. Flint Water Study. Jul 5. Available at: http://flintwaterstudy.org/2018/07/highlights-of-a-typical-week-of-citizen-engineering-in-flint/ [Last accessed 27 July 2018].
Egan, P. 2017a. These are the 15 people criminally charged in the Flint water crisis. Detroit Free Press. Jun 14. Available at: https://www.freep.com/story/news/local/michigan/flint-water-crisis/2017/06/14/flint-water-crisis-charges/397425001/ [Last accessed 20 January 2018].
Eller, JD. 2016. Social Science and Historical Perspectives: Society, Science, and Ways of Knowing. New York, NY: Routledge. DOI: https://doi.org/10.4324/9781315560434
Emery, A. 2016. Water Defense testing for harmful chemicals in water at Flint homes. MLive. Apr 24. Available at: http://www.mlive.com/news/flint/index.ssf/2016/04/water_defense_testing_for_harm.html [Last accessed 20 January 2018].
Enfield, N. 2017. We’re in a post-truth world with eroding trust and accountability. It can’t end well. The Guardian. Nov 17. Available at: https://www.theguardian.com/commentisfree/2017/nov/17/were-in-a-post-truth-world-with-eroding-trust-and-accountability-it-cant-end-well [Last accessed 21 November 2018].
Environmental Coffee House (ECH). 2017. Discussing the Flint Water Crisis with resident Vicki Marx. Facebook. Jul 29. Available at: https://www.facebook.com/environmentalcoffeehouse/videos/1074082592722224/ [Last accessed 20 January 2018].
European Commission. 2012. Chapter 3: Trust and Dialogue Between Science and Society. In: Innovation Union Competitiveness Report 2011. Luxembourg: Publication Office of the European Union. Available at: https://ec.europa.eu/research/innovation-union/pdf/competitiveness-report/2011/chapters/new_perspectives_smarter_policy_design_chapter_3.pdf.
Finnegan, W. 2016. Flint and The Long Struggle against Lead Poisoning. The New Yorker. Feb 4. Available at: https://www.newyorker.com/news/daily-comment/flint-and-the-long-struggle-against-lead-poisoning [Last accessed 20 January 2018].
Flint Water Class Action. 2016–17. Lawsuits Filed. Available at: http://www.flintwaterclassaction.com/lawsuits-filed/ [Last accessed 20 January 2018].
Flint Water Study. 2016. MacArthur Foundation 100&Change – Science as a Public Good. YouTube. Oct 9. Available at: https://www.youtube.com/watch?v=j8Y2Q7WPLOE [Last accessed 20 January 2018].
Freitag, A, Meyer, R and Whiteman, L. 2016. Strategies Employed by Citizen Science Programs to Increase the Credibility of Their Data. Citizen Science: Theory and Practice, 1(1): 2. DOI: https://doi.org/10.5334/cstp.6
French, R. 2016. Beyond bottled water: Huge checks, slow progress test patience in Flint. Bridge Magazine. Aug 23. Available at: http://www.bridgemi.com/children-families/beyond-bottled-water-huge-checks-slow-progress-test-patience-flint/ [Last accessed 1 February 2018].
Fonger, R. 2015a. Flint water problems: Switch aimed to save $5 million – but at what cost? MLive. May 29. Available at: www.mlive.com/news/flint/index.ssf/2015/01/flints_dilemma_how_much_to_spe.html [Last accessed 20 January 2018].
Fonger, R. 2015b. Erin Brockovich: Flint water system ‘failing,’ stop making excuses. MLive, Jan 21. Available at: https://www.mlive.com/news/flint/index.ssf/2015/01/erin_brockovich_twitter_fix_fl.html [Last accessed 27 July 2018].
Fonger, R. 2015c. Erin Brockovich investigator says tweaks can fix Flint River water. MLive. Feb 11. Available at: https://www.mlive.com/news/flint/index.ssf/2015/02/erin_brockovich_investigator_s.html [Last accessed 27 July 2018].
Fonger, R. 2015d. Erin Brockovich expert issues Flint water advice, including Detroit water cost analysis. MLive. Feb 18. Available at: https://www.mlive.com/news/flint/index.ssf/2015/02/erin_brockovich_water_expert_s.html [Last accessed 28 July 2018].
Fonger, R. 2015e. Erin Brockovich says Flint water an example of just how bad it can get. MLive. Sep 16. Available at: https://www.mlive.com/news/flint/index.ssf/2015/09/erin_brockovich.html [Last accessed 27 July 2018].
Fonger, R. 2016. Unity turns to mutiny as Flint water crisis activists battle each other. MLive. Jan 26. Available at: http://www.mlive.com/news/index.ssf/2016/05/unity_turns_into_mutiny_among.html [Last accessed 20 January 2018].
Fonger, R. 2017a. Flint water crisis group to Sen. Merkley: ‘Trust is our number one problem’. MLive. Oct 2. Available at: http://www.mlive.com/news/flint/index.ssf/2017/10/flint_water_round_table_to_sen.html [Last accessed 20 January 2018].
Fonger, R. 2017b. EPA says video appears to show fishing sinkers in faucet at home with high lead. MLive. Sep 27. Available at: http://www.mlive.com/news/flint/index.ssf/2017/09/epa_says_video_appears_to_show.html [Last accessed 20 January 2018].
Fonger, R. 2017c. State cites criminal probe, withholds emails related to lead sinker home in Flint. MLive. Oct 25. Available at: http://www.mlive.com/news/flint/index.ssf/2017/10/state_claims_lead_sinker_home.html [Last accessed 20 January 2018].
Fonger, R. 2017d. 71 Michigan water systems now have higher lead levels than Flint. MLive. Oct 30. Available at: http://www.mlive.com/news/flint/index.ssf/2017/10/71_michigan_water_systems_had.html [Last accessed 15 April 2018].
Gallup. 2016. Honesty/Ethics in Professions. Gallup Poll. Available at: http://news.gallup.com/poll/1654/honesty-ethics-professions.aspx [Last accessed 20 January 2018].
Geoghegan, H, Dyke, A, Pateman, R, West, S and Everett, G. 2016. Understanding motivations for citizen science. Final report on behalf of UKEOF, University of Reading, Stockholm Environment Institute (University of York) and University of the West of England.
Glenza, J. 2018. Flint crisis, four years on: What little trust is left continues to wash away. The Guardian. Apr 25. Available at: https://www.theguardian.com/us-news/2018/apr/25/flint-water-crisis-four-years-later [Last accessed 27 August 2018].
GMO Free News. 2015. Erin Brockovich Sent Us A Water Expert To Test The Flint River Water. YouTube. Oct 30. Available at: https://www.youtube.com/watch?v=Bbd7QrwkYMQ [Last accessed 27 July 2018].
Goodnough, A. 2016. In Flint, Fears of Showering Bring Desperate Measures. The New York Times. Apr 13. Available at: https://www.nytimes.com/2016/04/14/us/in-flint-rashes-stirfears-of-showering-as-scientists-hunt-forculprit.html [Last accessed 20 January 2018].
Goodnough, A and Atkinson, S. 2016. A Potent Side Effect of the Flint Water Crisis: Mental Health Problems. The New York Times. Apr 30. Available at: https://www.nytimes.com/2016/05/01/us/flint-michigan-water-crisis-mental-health.html [Last accessed 20 January 2018].
Goodwill, JE, WHagemann, MW, Edwards, MA and Reckhow, D. 2018. Quantifying and contextualizing disinfection byproducts during the Flint Water Crisis: A case study, and framework for broader application. Journal of Water Supply: Research and Technology-Aqua. DOI: https://doi.org/10.2166/aqua.2018.075
Gray, S, Sadler, A, Brown, A, Singer, A, Olabisi, LS, Lopez, M, Henderson, J, Gorman, M, Wallace, R, Kaplowitz, M and Introne, J. 2016. Flint resident perceptions about the causes, consequences, and solutions to the Flint Water Crisis. Community Foundation of Greater Flint. Dec 9. Available at: http://www.flintneighborhoodsunited.org/wp-content/uploads/2016/12/Voice-of-Flint-Policy-Brief-Final.pdf [Last accessed 20 January 2018].
Hall, H. 2013. Answering Our Critics: Part 1 of 2. Science-Based Medicine. Sep 24. Available at: https://sciencebasedmedicine.org/answering-our-critics-part-1-of-2/ [Last accessed 20 January 2018].
Hammond, C. 2017. In Flint, Fears of Showering Bring Desperate Measures. Facebook. Jun 23. Available at: https://www.facebook.com/TruthAgainstTheMachine/videos/466666763689182/ [Last accessed 20 January 2018].
Hanna-Attisha, M, LaChance, J, Sadler, RC and Champney Schnepp, A. 2016. Elevated blood lead levels in children associated with the Flint drinking water crisis: A spatial analysis of risk and public health response. American journal of public health, 106(2): 283–290. DOI: https://doi.org/10.2105/AJPH.2015.303003
Hanrahan, J. 2018. Waterborne Diseases. SHEA/CDC Outbreak Response Training Workshop 2018. Available at: http://ortp.shea-online.org/wp-content/uploads/2018/01/WED__230__Waterborne-Diseases.pdf [Last accessed 24 November 2018].
Harmon, MD. 2000. Quotes about ethics, media, etc. Mass Media Ethics. Jun 23. Available at: http://web.cci.utk.edu/~mdharmon/quotes8.html [Last accessed 20 January 2018].
Haze, M. 2017. Jordan Chariton Threatens To Sue The Young Turks And The Intercept. The Daily Haze. Nov 22. Available at: http://thedailyhaze.com/jordan-chariton-threatens-sue-young-turks-intercept/ [Last accessed 20 January 2018].
Hersher, R. 2016. Contaminated Water In St. Joseph, La., Leads To Public Health Emergency. NPR. Dec 16. Available at: https://n.pr/2gKrztF [Last accessed 20 January 2018].
Hohn, D. 2016. Flint Water Crisis and the ‘Troublemaker’ Scientist. The New York Times Magazine. Aug 16. Available at: https://www.nytimes.com/2016/08/21/magazine/flints-water-crisis-and-the-troublemaker-scientist.html [Last accessed 20 January 2018].
Holdren, JP. 2015. Memorandum to Heads of Executive Departments and Agencies: Addressing Societal and Scientific Challenges through Citizen Science and Crowdsourcing. White House OSTP. Sep 30. Available at: https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/holdren_citizen_science_memo_092915_0.pdf [Last accessed 25 March 2018].
Horgan, J. 2014. Advice to Young Science Writers: Ask “What Would Chomsky Think?” Scientific American. Dec 17. Available at: https://blogs.scientificamerican.com/cross-check/advice-to-young-science-writers-ask-8220-what-would-chomsky-think-8221/ [Last accessed 25 March 2018].
Jerome, S. 2016. Top Scientist: Ruffalo Should Be ‘Ashamed’ Over Flint Claims. Water Online. May 23. Available at: https://www.wateronline.com/doc/top-scientist-ruffalo-should-be-ashamed-over-flint-claims-0001 [Last accessed 20 January 2018].
Johnson, B. 2016b. Mark Ruffalo slammed for Flint statements. Fox News. May 19. Available at: http://www.foxnews.com/entertainment/2016/05/19/mark-ruffalo-slammed-for-flint-statements.html [Last accessed 20 January 2018].
Johnson, J. 2016a. Flint water problems: Switch aimed to save $5 million – but at what cost? MLive. Jan 28. Available at: http://www.mlive.com/news/flint/index.ssf/2016/01/flint_gov_appointed_committe_m.html [Last accessed 20 January 2018].
Justice League NYC. 2016. WATER DEFENSE ORG’s LEAD SCIENTIST SCOTT SMITH, FLINT RIVER. YouTube. Mar 7. Available at: https://www.youtube.com/watch?v=tuzx3adxxPc [Last accessed 20 January 2018].
Kennedy, B. 2016. Most Americans trust the military and scientists to act in the public’s interest. Pew Research. Oct 18. Available at: http://www.pewresearch.org/fact-tank/2016/10/18/most-americans-trust-the-military-and-scientists-to-act-in-the-publics-interest/ [Last accessed 20 January 2018].
Ketchum, WE. 2015. People take to streets to protest Flint water quality. MLive. Feb 14. Available at: https://www.mlive.com/news/flint/index.ssf/2015/02/flint_residents_protest_citys.html [Last accessed 27 July 2018].
Krauss, N, Malmfors, T and Slovic, P. 1992. Intuitive Toxicology: Expert and Lay Judgments of Chemical Risks. Risk Analysis, 12(2): 215–232. DOI: https://doi.org/10.1111/j.1539-6924.1992.tb00669.x
Lambrinidou, Y. 2016. On Listening, Science, and Justice: A Call for Exercising Care in What Lessons We Draw from Flint. Environ. Sci. Technol., 50(22): 12058–12059. [Last accessed 1 February 2018]. DOI: https://doi.org/10.1021/acs.est.6b04634
Lambrinidou, Y. 2017. Plenary Session II: What is the Role of Scientists in Supporting the Human Right to Water? In: AAAS Science and Human Rights Coalition Meeting: The Human Right to Water. Jan 26–27. Available at: https://www.aaas.org/events/science-and-human-rights-coalition-meeting-human-right-water [Last accessed 28 December 2018].
Lambrinidou, Y. 2016–18. Yanna Lambrinidou. Twitter. 2016–18. Available at: https://twitter.com/DrishtiEthics [Last accessed 12 April 2018].
Lambrinidou, Y, Rhoads, WJ, Roy, S, Heaney, E, Ratajczak, G and Ratajczak, J. 2014. Ethnography in Engineering Ethics Education: A Pedagogy for Transformational Listening. 121st American Society for Engineering Education (ASEE) Annual Conference & Exposition. Indianapolis, IN, June 15–18.
LIFETIME. 2017. It’s been 1,278 days. Facebook. Oct 24. Available at: https://www.facebook.com/lifetime/videos/10155398385719087/ [Last accessed 1 February 2018].
Lima, C. 2017. ‘Young Turks’ reporter vows to sue over his firing. Politico. Nov 22. Available at: https://www.politico.com/story/2017/11/22/jordan-chariton-young-turks-lawsuit-firing-259363 [Last accessed 1 February 2018].
Lower, S. 2013. “Catalytic” water treatment. Steve Lower’s Web Pages. Apr 30. Available at: http://www.chem1.com/CQ/catscams.html [Last accessed 1 February 2018].
Lurie, J. 2016. Meet the Mom Who Helped Expose Flint’s Toxic Water Nightmare. Mother Jones. Jan 21. Available at: https://www.motherjones.com/politics/2016/01/mother-exposed-flint-lead-contamination-water-crisis/ [Last accessed 1 February 2018].
Lynch, J. 2016. Va. Tech professor rips Ruffalo group’s Flint message. The Detroit News. May 16. Available at: http://www.detroitnews.com/story/news/michigan/flint-water-crisis/2016/05/16/professor-ruffalo-group-flint-message/84435156/ [Last accessed 1 February 2018].
Lynch, J and Carah, J. 2016. Group fears bathing risks in Flint water. The Detroit News. Apr 8. Available at: http://www.detroitnews.com/story/news/michigan/flint-water-crisis/2016/04/08/group-fears-bathing-risks-flint-water/82823590/ [Last accessed 1 February 2018].
Lynch, J and Chambers, J. 2017. Flint water has fallen below federal lead limit. The Detroit News. Jan 24. Available at: http://www.detroitnews.com/story/news/michigan/flint-water-crisis/2017/01/24/flint-water-lead-levels/96995682/ [Last accessed 1 February 2018].
Maddow, R. 2016. TIME Magazine’s 100 Most Influential People: Marc Edwards and Mona Hanna-Attisha. TIME Magazine. Apr 21. Available at: http://time.com/collection-post/4301337/marc-edwards-and-mona-hanna-attisha-2016-time-100/ [Last accessed 1 February 2018].
Marmot, M. 2017. Post-truth and science. The Lancet, 389(10068): 497–498. DOI: https://doi.org/10.1016/S0140-6736(17)30207-6
Marx, V. 2017. Well here are the results of my blood test. Facebook. Jul 26. Available at: https://www.facebook.com/photo.php?fbid=1595050693852642 [Last accessed 1 February 2018].
Masten, SJ. 2018. Independent Lead Testing in Flint, Michigan: Testing Period 1. Available at: http://flintwaterstudy.org/wp-content/uploads/2018/04/NRDC-REPORT-Phase1.pdf [Last accessed 15 April 2018].
Masten, SJ, Davies, SH and Mcelmurry, SP. 2016. Flint Water Crisis: What Happened and Why?. (not peer reviewed). Journal-American Water Works Association, 108(12): 22. DOI: https://doi.org/10.5942/jawwa.2016.108.0195
Matsa, KE, Mitchell, A and Stocking, G. 2017. Searching for News: The Flint Water Crisis. Pew Research Center. April 27. Available at: http://www.journalism.org/essay/searching-for-news/ [Last accessed 20 January 2018].
Mays, M. 2016a. Scott Smith from Water Defense – Bathing in Flint is NOT SAFE. YouTube. Apr 13. Available at: https://youtu.be/OwA6cPpdQgU [Last accessed 1 February 2018].
Mays, M. 2016b. #FlintWaterCrisis: My test results today: 160 ppb. Help!! Twitter. Sep 16. Available at: https://twitter.com/FlintGate/status/776913479160754176 [Last accessed 1 February 2018].
Mays, M. 2016–18. Melissa Mays. Facebook. 2016–18. Available at: https://www.facebook.com/indigrace/timeline?query=edwards [Last accessed 12 April 2018].
Mays, M. 2017. RESEARCH!!!! Facebook. Aug 7. Available at: https://www.facebook.com/indigrace/videos/10155662696309744/ [Last accessed 1 February 2018].
Mays, M. 2017–18. Melissa Mays. Twitter. 2017–18. Available at: https://twitter.com/FlintGate/ [Last accessed 27 July 2018].
Mcfarland, M. 2017. “We’re still being poisoned”: A chat with activist Melissa Mays about “Flint”. Salon, Oct 28. Available at: https://www.salon.com/2017/10/28/were-still-being-poisoned-a-chat-with-activist-melissa-mays-about-flint/ [Last accessed 20 January 2018].
Michigan Department of Environmental Quality (MDEQ). 2018b. City of Flint’s water quality restored, testing well below federal action level for nearly two years. Press Release. Apr 6. Available at: http://www.michigan.gov/snyder/0,4668,7-277-80388_80397-465766--,00.html [Last accessed 12 April 2018].
Michigan Department of Health and Human Services (MDHHS). 2016. MDHHS Reminds Residents Bathing is Safe for Children, Adults in Flint. State of Michigan website. Mar 14. Available at: http://www.michigan.gov/flintwater/0,6092,7-345-75251_75303-378971--,00.html [Last accessed 20 January 2018].
Mikulak, A. 2011. Mismatches between ‘scientific’ and ‘non-scientific’ ways of knowing and their contributions to public understanding of science. Integrative Psychological and Behavioral Science, 45(2): 201–215. DOI: https://doi.org/10.1007/s12124-011-9157-8
Milman, O and Glenza, J. 2016. At least 33 US cities used water testing ‘cheats’ over lead concerns. The Guardian. Jun 2. Available at: https://www.theguardian.com/environment/2016/jun/02/lead-water-testing-cheats-chicago-boston-philadelphia [Last accessed 20 January 2018].
Mirowski, P. 2017. Against Citizen Science. Aeon. Nov 20. Available at: https://aeon.co/essays/is-grassroots-citizen-science-a-front-for-big-business [Last accessed 20 January 2018].
Moore, AE and Mendes, M. 2017. “There’s No Way We’re Gonna Drink That”: Fighting for Clean Water in Flint. Truth-Out. Mar 21. Available at: http://www.truth-out.org/opinion/item/39917-i-m-not-gonna-drink-that [Last accessed 15 April 2018].
Morua, AR, Halvorsen, KE and Mayer, AS. 2011. Waterborne Disease-Related Risk Perceptions in the Sonora River Basin, Mexico. Risk Analysis, 31(5): 866–878. DOI: https://doi.org/10.1111/j.1539-6924.2010.01570.x
Murphy, C. 2016. Fighting to get clean water…. if the government wont we will. Facebook. Dec 20. Available at: https://www.facebook.com/murphy.murphy.73700/videos/vb.100011708118178/318616285205306/ [Last accessed 20 January 2018].
Murphy, C. 2017. Scott smith and mark durno epa testing side by side. Facebook. Apr 14. Available at: https://www.facebook.com/murphy.murphy.73700/videos/vb.100011708118178/381514305582170/ [Last accessed 20 January 2018].
National Advisory Council for Environmental Policy and Technology (NACEPT). 2016. Environmental Protection Belongs to the Public: A Vision for Citizen Science at EPA. Publication Number: EPA 219-R-16-001. Available at: https://www.epa.gov/faca/nacept-2016-report-environmental-protection-belongs-public-vision-citizen-science-epa [Last accessed 12 June 2018].
NBC News. 2016. Erin Brockovich sounds alarm on Flint water crisis. Available at: https://www.nbcnews.com/dateline/video/erin-brockovich-sounds-alarm-on-flint-water-crisis-605045827972 [Last accessed 28 July 2018].
Nichols, T. 2017. The Crisis of Expertise. Aeon. Jun 6. Available at: https://aeon.co/essays/its-time-to-reboot-the-relationship-between-expertise-and-democracy [Last accessed 20 January 2018].
NLP Aqua Solutions. 2016. The Best Water Solution – But Is It a Water Softener? NLP Website. Available at: http://www.nlpaqua.com/bestwater.html [Last accessed 20 January 2018].
Nowotny, H. 2003. Democratising expertise and socially robust knowledge. Science and Public Policy, 30(3): 151–156. DOI: https://doi.org/10.3152/147154303781780461
Nuzzo, R. 2015. Fooling ourselves. Nature, 526(7572): 182. DOI: https://doi.org/10.1038/526182a
Olson, TM, Wax, M, Yonts, J, Heidecorn, K, Haig, S, Yeoman, D, Hayes, Z, Raskin, L and Ellis, BR. 2017. Forensic Estimates of Lead Release from Lead Service Lines during the Water Crisis in Flint, Michigan. Environmental Science & Technology Letters, 4(9): 356–361. DOI: https://doi.org/10.1021/acs.estlett.7b00226
OPFLEXInventor. 2013. Steve Jobs & #OPFLEX & Scott Smith 2011. YouTube. Oct 13. Available at: https://www.youtube.com/watch?v=79MvQ8efPiY [Last accessed 20 January 2018].
Oransky, I and Marcus, A. 2017. Should scientists engage in activism? The Conversation. Feb 6. Available at: http://theconversation.com/should-scientists-engage-in-activism-72234 [Last accessed 20 January 2018].
Ottinger, G. 2010. Buckets of resistance: Standards and the effectiveness of citizen science. Science, Technology, & Human Values, 35(2): 244–270. DOI: https://doi.org/10.1177/0162243909337121
Ottinger, G. 2017. Scientific authority and models of change in two traditions of citizen science. In: Tyfield, D, Lave, R, Randalls, S and Thorpe, R (eds.), The Routledge Handbook of the Political Economy of Science, 351. New York, NY: Routledge. DOI: https://doi.org/10.4324/9781315685397-31
Parkinson, S. 2016. EPA to do more water sampling in Flint after meetings with advocates this weekend. WSMH. Mar 20. Available at: http://wsmh.com/news/local/epa-to-do-more-water-sampling-in-flint-after-meetings-with-advocates-this-weekend [Last accessed 20 January 2018].
Pellizzoni, L. 2011. The politics of facts: Local environmental conflicts and expertise. Environmental Politics, 20(6): 765–785. DOI: https://doi.org/10.1080/09644016.2011.617164
Pieper, KJ, Tang, M and Edwards, MA. 2017. Flint Water Crisis Caused By Interrupted Corrosion Control: Investigating “Ground Zero” Home. Environmental Science & Technology, 51(4): 2007–2014. DOI: https://doi.org/10.1021/acs.est.6b04034
Pierret, A. 2017. Flint woman dealing with lead level spike in blood, water. ABC12. Jul 27. Available at: http://www.abc12.com/content/news/Flint-woman-dealing-with-lead-level-spike-in-blood-and-water-437056953.html [Last accessed 1 February 2018].
Pigliucci, M. 2014. Are There ‘Other’ Ways of Knowing? Philosophy Now. Available at: https://philosophynow.org/issues/102/Are_There_Other_Ways_of_Knowing [Last accessed 1 February 2018].
Pigliucci, M and Boudry, M. 2013. The Dangers of Pseudoscience. The New York Times. Oct 10. Available at: https://opinionator.blogs.nytimes.com/2013/10/10/the-dangers-of-pseudoscience/ [Last accessed 31 March 2018].
PRWeb. 2016. Melissa Mays Hosts Press Conference Debunking Flint Michigan Water Quality Testing, on December 13, 2016. PRWeb. Dec 12. Available at: http://www.prweb.com/releases/2016/12/prweb13917220.htm [Last accessed 20 January 2018].
Resnik, DB, Elliott, KC and Miller, AK. 2015. A framework for addressing ethical issues in citizen science. Environmental Science & Policy, 54: 475–481. DOI: https://doi.org/10.1016/j.envsci.2015.05.008
Rhoads, WJ, Garner, E, Ji, P, Zhu, N, Parks, J, Schwake, DO, Pruden, A and Edwards, MA. 2017. Distribution System Operational Deficiencies Coincide with Reported Legionnaires’ Disease Clusters in Flint, Michigan. Environmental science & technology, 51(20): 11986–11995. DOI: https://doi.org/10.1021/acs.est.7b01589
Riesch, H and Potter, C. 2014. Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions. Public understanding of science, 23(1): 107–120. DOI: https://doi.org/10.1177/0963662513497324
Riley, D, Grunert, J, Jalali, Y and Adams, SG. 2016. Citizen Engineering: Disrupting Expertise in Classroom and Community. ASEE Annual Conference and Exposition. Jun 26–29 2016, New Orleans, LA. DOI: https://doi.org/10.18260/p.26495
Roy, S. 2017. The hand-in-hand spread of mistrust and misinformation in flint. American Scientist, 105(1): 22–26. DOI: https://doi.org/10.1511/2017.124.22
Roy, S and Edwards, MA. 2015. Chronological compilation of e-mails from MDEQ Freedom of Information Act (FOIA) request 6526 – 15 and 6525 – 15. 2015. Flint Water Study. Available at: http://flintwaterstudy.org/wp-content/uploads/2015/10/MDEQ-USEPA-Final.pdf [Last accessed 20 January 2018].
Roy, S and Edwards, MA. 2017. EXCLUSIVE! Mark Ruffalo’s WATER DEFENSE Sampling Methods Revealed. Flint Water Study. May 9. Available at: http://flintwaterstudy.org/2017/05/exclusive-mark-ruffalos-water-defense-sampling-methods-revealed/ [Last accessed 20 January 2018].
Roy, S and Edwards, MA. 2018a. Preventing another lead (Pb) in drinking water crisis: Lessons from the Washington D.C. and Flint MI contamination events. Current Opinion in Environmental Science & Health, 1–11. DOI: https://doi.org/10.1016/j.coesh.2018.10.002
Roy, S and Edwards, MA. 2018b. Is this Flint photo from 2015? Flint Water Study. June 29. Available at: http://flintwaterstudy.org/2018/06/is-this-flint-photo-from-2015/ [Last accessed 27 July 2018].
Ruffalo, M. 2015. Our Work in 2015 – Water Defense. Water Defense. Dec 22. Available at: https://waterdefense.org/our-work-in-2015/ [Last accessed 20 January 2018].
Ruffalo, M. 2016. Flint isn’t over. Keep your eye on the ball. Twitter. Sep 16. Available at: https://twitter.com/MarkRuffalo/status/776926174630993920 [Last accessed 1 February 2018].
Russian TV America (RT America). 2016. Flint residents still footing bill for poisoned water. YouTube. Jan 21. Available at: https://www.youtube.com/watch?v=lKophHV8rFY [Last accessed 27 July 2018].
Schwake, DO, Garner, E, Strom, OR, Pruden, A and Edwards, MA. 2016. Legionella DNA markers in tap water coincident with a Spike in Legionnaires’ disease in Flint, MI. Environmental Science & Technology Letters, 3(9): 311–315. DOI: https://doi.org/10.1021/acs.estlett.6b00192
Scipion, J. 2016, Dec 5. Is the Flint Water Crisis Getting Worse? FOXBusiness.com Available at: https://www.foxbusiness.com/features/is-the-flint-water-crisis-getting-worse [Last accessed 27 July 2018].
Shapiro, A and Gringlas, S. 2018, Oct 26. What’s Changed And What Hasn’t When It Comes To The Flint Water Crisis. NPR. Available at: https://www.npr.org/2018/10/26/661136990/whats-changed-and-what-hasnt-when-it-comes-to-the-flint-water-crisis [Last accessed 28 October 2018].
Shogren, E, Bartlett, S and Lombardi, K. 2011. N.Y. Plant’s Neighbors Expose Regulatory Gaps. NPR. Nov 10. Available at: https://www.npr.org/2011/11/10/142189390/tonawanda-provides-lessons-for-fighting-toxic-air [Last accessed 1 February 2018].
Smith, M. 2015. A Water Dilemma in Michigan: Cloudy or Costly? The New York Times. Mar 24. Available at: https://www.nytimes.com/2015/03/25/us/a-water-dilemma-in-michigan-cheaper-or-clearer.html [Last accessed 02 April 2019].
Smith, S. 2016a. Scott Smith. Twitter. Available at: https://twitter.com/WaterWarriorOne [Last accessed 20 January 2018].
Smith, S. 2016b. Scott Smith (Chief Technology Officer & Investigator) – Water Defense Statement 4/09/16. Available at: http://flintwaterstudy.org/wp-content/uploads/2016/05/Scott-Smith-Water-Defense-Statement-Flint-04-09-16-004.pdf [Last accessed 20 January 2018].
Smith, S. 2016c. Presentation on Potential Water Risks – Based on Real World Data. Florida Fracking Summit. Nov 2 2016. Available at: http://www.flfrackingsummit.org/wp-content/uploads/2016/11/Scott-Smith-Potential-Water-Risks-FL-Fracking-Presentation-11-2-16.pdf [Last accessed 20 January 2018].
Smith, S. 2016d. Please call Rachel Maddow. Instagram. Dec 4. Available at: https://www.instagram.com/p/BNm0gCxArkS/?taken-by=waterwarriorone [Last accessed 20 January 2018].
Smith, S. 2017. On plane to #flint. Instagram. Apr 14. Available at: https://www.instagram.com/p/BS4mMnGhvPs/?taken-by=waterwarriorone [Last accessed 20 January 2018].
Smith, S. 2018. Lessons I Learned in Flint and Clarifying the Facts — By Scott C. Smith. Flint Water Study. Jul 11. Available at: http://flintwaterstudy.org/2018/07/scott-smith-flint-guest-post/ [Last accessed 28 August 2018].
Snopes. 2016a. CPS in Flint Threatens Families Over Water Crisis? Jan 26. Available at: https://www.snopes.com/fact-check/flint-water-crisis-cps/ [Last accessed 27 July 2018].
Snopes. 2016b. Illegal for Flint Residents to Sell Homes. Snopes.com. Jan 28. Available at: https://www.snopes.com/fact-check/illegal-sell-homes-flint/ [Last accessed 27 July 2018].
Stange, KC. 2010. Ways of Knowing, Learning, and Developing. Annals of Family Medicine, 8(1): 4–10. DOI: https://doi.org/10.1370/afm.1082
Strasser, BJ, Baudry, J, Mahr, D, Sanchez, G and Tancoigne, E. 2018. “Citizen Science”? Rethinking Science and Public Participation. Science and Technology Studies. Available at: https://sciencetechnologystudies.journal.fi/forthcoming/article/60425/37504.
Strom, O, Rhoads, W, Garner, E, Pruden, A and Edwards, MA. 2016. Flint Water Press Conference – The Saginaw and Genesee County Shigella Outbreak. Flint Water Study. Available at: http://flintwaterstudy.org/2016/12/flint-water-study-press-conference-12-2-2016/ [Last accessed 20 January 2018].
Sullivan, L. 2016. Laura Sullivan. Facebook. Oct 6. Available at: https://www.facebook.com/dr.laura2/posts/844849792619 [Last accessed 1 February 2018].
Sullivan, L. 2017. Laura Sullivan. Twitter. 2017. Available at: https://twitter.com/drlaura2 [Last accessed 1 February 2018].
Tang, M, Pieper, KJ, Roy, S, Parks, J and Edwards, MA. 2017. Flint Water Press Conference – Flint resident sampling: August 2015 – August 2017. Flint Water Study. Available at: http://flintwaterstudy.org/2017/09/flint-water-study-press-conference-09-15-2017/ [Last accessed 20 January 2018].
The Detroit News. 2018. Editorial: Flint water is safe but trust lacking. The Detroit News. Apr 14. Available at: https://www.detroitnews.com/story/opinion/editorials/2018/04/14/flint-water-safe-bottled-michigan-lead-residents/33850609/ [Last accessed 15 April 2018].
The Late Show with Stephen Colbert. 2016. Erin Brockovich Explains The Flint Water Crisis. YouTube. Jan 29. Available at: https://www.youtube.com/watch?v=mWae__xM46o [Last accessed 27 July 2018].
The Steve Harvey Show. 2016. Is It Safe To Bathe In The Water?||STEVE HARVEY. YouTube. Mar 14. Available at: https://www.youtube.com/watch?v=oWcdFunBVww [Last accessed 27 July 2018].
The Young Turks (TYT). 2016a. Flint Being Under “Normal Lead Levels” Is a LIE. YouTube. Dec 27. Available at: https://youtu.be/mk1a2D4SiDY [Last accessed 20 January 2018].
The Young Turks (TYT). 2016b. Jordan Draws Potentially Poisonous Lead In 2-Year-Old Heater. YouTube. Dec 30. Available at: https://youtu.be/imtG5qczkHY [Last accessed 20 January 2018].
The Young Turks (TYT). 2016c. Flint Residents Poisoned, Then Blocked From Crisis Decisions. YouTube. Dec 30. Available at: https://youtu.be/P3eQXhaybgw [Last accessed 20 January 2018].
The Young Turks. (TYT). 2016d. Flint Poisons Citizens… Then Forgets Them. YouTube. Dec 26. Available at: https://youtu.be/NNdsHoru1PI [Last accessed 20 January 2018].
The Young Turks (TYT). 2017a. Flint Town Hall with Jordan Chariton. Facebook. Mar 28. Available at: https://www.facebook.com/TheYoungTurks/videos/vb.210277954204/10154507459024205/ [Last accessed 20 January 2018].
The Young Turks (TYT). 2017b. Jordan & Investigator Test Lead-Poisoned Indiana Homes. YouTube. Feb 21. Available at: https://youtu.be/mEI707fevko [Last accessed 20 January 2018].
The Young Turks (TYT). 2017c. Our Indiana Lead Test Results Are In…. YouTube. Mar 14. Available at: https://youtu.be/B5RKlO-D6s4 [Last accessed 20 January 2018].
The Young Turks (TYT). 2017d. Jordan EXPOSES EPA-Funded Lead Expert’s Lies. YouTube. Jun 8. Available at: https://youtu.be/tWc9J6vgEFk [Last accessed 20 January 2018].
The Young Turks (TYT). 2017e. EXCLUSIVE: Will The EPA Lie To Flint Residents? YouTube. Jan 9. Available at: https://youtu.be/FoKN45simAo [Last accessed 20 January 2018].
The Young Turks (TYT). 2017f. Why is the EPA Dodging Jordan’s Questions?!?!?!? YouTube. Sep 22. Available at: https://youtu.be/1es1DHICu0o [Last accessed 20 January 2018].
Truth Against The Machine (TATM). 2017. LIVE: Laura MacIntyre is at the Water Is Life Flint Water Summit. TATM. Sep 30. Available at: https://www.facebook.com/TruthAgainstTheMachine/videos/510413872647804/ [Last accessed 20 January 2018].
Union of Concerned Scientists (UCS). n.d. Science, Evolution, and Intelligent Design. Union of Concerned Scientists. Available at: https://www.ucsusa.org/scientific_integrity/what_you_can_do/science-evolution-and.html [Last accessed 20 January 2018].
U.S. Centers for Disease Control and Prevention (CDC). 2016a. Flint Rash Investigation Report. Aug. Available at: http://www.phe.gov/emergency/events/Flint/Documents/rash-report.pdf [Last accessed 20 January 2018].
U.S. Centers for Disease Control and Prevention (CDC). 2016b. Interim Findings from the Shigellosis Outbreak Investigation — Genesee and Saginaw Counties, Michigan, 2016. Dec 15. Available at: http://www.michigan.gov/documents/mdhhs/MI_Shigellosis_Dec_Update_CommunityPresentation_545873_7.pdf [Last accessed 20 January 2018].
U.S. Centers for Disease Control and Prevention (CDC). 2018. Shigella – Shigellosis|CDC. Jan 17. Available at: https://www.cdc.gov/shigella/index.html [Last accessed 20 January 2018].
U.S. Environmental Protection Agency Office of Inspector General (EPA OIG). 2016. Report #17-P-0004: Drinking Water Contamination in Flint, Michigan, Demonstrates a Need to Clarify EPA Authority to Issue Emergency Orders to Protect the Public. Oct 20. Available at: https://www.epa.gov/office-inspector-general/report-drinking-water-contamination-flint-michigan-demonstrates-need [Last accessed 20 January 2018].
U.S. Environmental Protection Agency Office of Inspector General (EPA OIG). 2018. Report #18-P-240: Report: EPA Needs a Comprehensive Vision and Strategy for Citizen Science that Aligns with Its Strategic Objectives on Public Participation. Sep 05. Available at: https://www.epa.gov/office-inspector-general/report-epa-needs-comprehensive-vision-and-strategy-citizen-science-aligns [Last accessed 22 October 2018].
Virginia Tech. 2016. University researchers hold Flint water press conference. YouTube. May 31. Available at: https://youtu.be/NHjmxxjQJ4E [Last accessed 20 January 2018].
Vosoughi, S, Roy, D and Aral, S. 2018. The spread of true and false news online. Science, 359(6380): 1146–1151. DOI: https://doi.org/10.1126/science.aap9559
Wakefield, MK. 2016. On the Ground This Week: HHS to Lead Federal Response in Flint. FEMA Blog. Jan 19. Available at: https://www.fema.gov/blog/2016-01-19/ground-week-hhs-lead-federal-response-flint/ [Last accessed 1 February 2018].
Walters, L. 2017. First You Said. Facebook. Apr 11. Available at: https://www.facebook.com/photo.php?fbid=1876455812380807 [Last accessed 20 January 2018].
Water Defense. 2016a. Mark Ruffalo Discusses Flint & Water Defense. YouTube. May 24. Available at: https://www.youtube.com/watch?v=1JiBCrjCRNM [Last accessed 20 January 2018].
Water Defense. 2016b. Water Defense Investigates Flint Contamination. YouTube. May 24. Available at: https://www.youtube.com/watch?v=6U9cDLRcBIk [Last accessed 20 January 2018].
Water Defense. 2016c. Water Defense on NBC Flint – 4 Segments with The Harringtons. YouTube. Mar 23. Available at: https://www.youtube.com/watch?v=dISL5F3HXRM [Last accessed 20 January 2018].
Water Defense. 2016d. Scott Smith – Flint News Conference – City Council Chambers 4.22.2016. YouTube. Apr 29. Available at: https://www.youtube.com/watch?v=Fz3EGyMwnA0 [Last accessed 20 January 2018].
Water Defense. n.d.1. For the record. Flint Water Defense Info. No date available. Available at: https://flintwaterdefenseinfo.wordpress.com/ [Last accessed 15 April 2018].
Water Defense. n.d.2. Frequently Asked Questions & Answers. Flint Water Defense Info. No date available. Available at: https://flintwaterdefenseinfo.wordpress.com/information/frequently-asked-questions-answers/ [Last accessed 20 January 2018].
Water Defense. n.d.3. Scientific Support Team. Flint Water Defense Info. No date available. Available at: https://flintwaterdefenseinfo.wordpress.com/information/who-we-are/scientific-support-team/ [Last accessed 15 April 2018].
Water You Fighting For. 2016. Skepticism, scared, trust in Flint has completely depleted. Facebook. Available at: https://www.facebook.com/WaterYouFightingFor/videos/953957681364216/ [Last accessed 27 July 2018].
Williams, R. 2016. Water experts say non-profit group’s Flint water test lacks credibility. Michigan Radio. Jun 7. Available at: http://michiganradio.org/post/water-experts-say-non-profit-groups-flint-water-test-lacks-credibility [Last accessed 20 January 2018].
Withstand With Wolves. 2017. Flint Michigan Residents Adam & Christina Murphy and how the water crisis has affected their lives. Facebook. May 4. Available at: https://www.facebook.com/withstandwithwolves/videos/294334194312385/ [Last accessed 20 January 2018].
WNEM. 2016. Nonprofit: Concerning levels of chemicals found in water. WNEM. Feb 18. Available at: http://www.wnem.com/story/31260610/nonprofit-concerning-levels-of-chemicals-found-in-water/ [Last accessed 20 January 2018].
Worth-Nelson, J. 2016. Shigella update: Bacteria not spread through Flint-area drinking water, CDC officials suggest. East Village Magazine. Nov 19. Available at: http://www.eastvillagemagazine.org/2016/11/19/shigella-update-bacteria-not-spread-through-flint-area-drinking-water-cdc-officials-suggest/ [Last accessed 20 January 2018].
Wynne, B. 1989. Sheepfarming after Chernobyl: A Case Study in Communicating Scientific Information, Environment: Science and Policy for Sustainable Development, 31(2): 10–39. DOI: https://doi.org/10.1080/00139157.1989.9928930
Yankelovich, D. 2005. Ferment and Change: Higher Education in 2015. Chronicle of Higher Education. Nov 25. Available at: https://www.chronicle.com/article/FermentChange-Higher/14934 [Last accessed 20 January 2018].