Editor's Note
An Editor’s Note relating to this publication can be found here: http://doi.org/10.5334/cstp.264
Introduction
“We’re in a post-truth world with eroding trust and accountability…. In our new normal, experts are dismissed, alternative facts are (sometimes flagrantly) offered, and public figures can offer opinions on pretty much anything. And thanks to social media, pretty much anyone can be a public figure.” Nick Enfield ()
This case study calls attention to lessons learned about the practice of citizen science during high profile emergency interventions and disasters involving environmental injustice. A citizen science collaboration among Flint residents, the Virginia Tech “Flint Water Study” team, and others first exposed contamination in Flint’s water supply in 2015. The resulting Federal Emergency declaration in January 2016 resulted in more than $600 million in relief funding, an acknowledged case of environmental injustice, and resignations/indictments of some public officials. But after the Federal Emergency was declared, ethical dilemmas associated with abuse of citizen science were encountered, conclusions of which were used to support unscientific public health messages that were in direct conflict with those of the relief agencies. A general state of science anarchy resulted which created further distrust and confusion, highlighting the lack of frameworks to police instances of unethical behavior by those claiming to be citizen scientists. While it is relatively easy for scientific experts to identify good vs. poor citizen science practices, Flint demonstrates that social justice advocates are sometimes willing to openly embrace faulty methods or even misconduct to achieve a populist objective, thereby creating a high likelihood of confrontation in future high profile cases like the Flint Water Crisis.
Origins of a Water Crisis
“It’s regular, good, pure drinking water, and it’s right in our backyard.” Dayne Walling, Former Mayor of Flint ()
To save about 5 million US dollars while constructing a new water pipeline, Michigan state-appointed emergency managers switched drinking water sources for the City of Flint from Lake Huron (purchased from the Detroit Water and Sewerage Department [DWSD]) to the local Flint River on April 25, 2014 (; ). A failure to implement federally mandated corrosion control treatment unleashed a “perfect storm” within Flint’s antiquated water distribution system of unlined iron water mains, lead/galvanized steel service pipes, and lead-bearing internal plumbing. The uncontrolled corrosion depleted the chlorine disinfectant residual in the system, elevating levels of microbes including Escherichia coli and Legionella pneumophila. An in-depth scientific analysis of deficiencies that caused Flint’s water quality problems can be found elsewhere (; ; ; ; ; ; ; ).
In parallel with the scientific and engineering problems, public trust was lost due to failures of government agencies at all levels (; ; ; ; ; ). Analysis of Google web searches illustrates that Flint residents began unusual online searches for information about their tap water soon after the switch in summer 2014 (). A series of red flags indicating growing problems were reported by consumers to appropriate local and state government officials during the time that Flint River water was being used (Apr 2014–Oct 2015), and these warnings were either ignored or in some cases covered up (). Exemplary problems included reports of red water, hair loss, skin rashes, three boil water advisories, rusting of engine parts at a General Motors factory, and violations of disinfection byproduct (DBP) regulations. But the crisis entered a new phase when high lead levels (greater than 100 ppb) were detected by a City of Flint official in the home of resident LeeAnne Walters ().
Uncovering the Flint Water Crisis through Citizen Science
“We proved that citizens and scientists working together could form a great alliance, and that grassroots science can have a sky-high impact.” LeeAnne Walters ()
After Walters (a.k.a. “Resident Zero”) discovered high water lead in her home (a.k.a. “Ground Zero”) through a sample collected by the City of Flint, she became a practicing citizen scientist who systematically educated herself about water lead issues and diagnosed the cause for elevated blood lead in her children. She also began collaborating with Miguel Del Toral, one of the foremost national experts of the federal Lead and Copper Rule (LCR) in the US Environmental Protection Agency (EPA) working in nearby Chicago. Walters and Del Toral unequivocally determined that there was no federally required lead corrosion control in the city’s water (; ). In April 2015, Walters sampled her tap water with the assistance of authors of this paper, demonstrating hazardous waste levels of water lead even after prolonged flushing (). Those data contributed to a detailed technical memo by Del Toral in late June 2015 which highlighted the imminent and substantial endangerment to Flint residents (). In early August 2015, Walters and other Flint residents learned that the Michigan Department of Environmental Quality (MDEQ)–which had the primary responsibility for safety of Flint water–intended to ignore the memo and cover up water lead problems without interference from senior management at the EPA (; ). Del Toral was effectively silenced by EPA ().
Given that local and federal agencies were not going to carry out their duty to protect Flint residents, a citizen science collaboration was launched by Virginia Tech in early August 2015. It involved the American Civil Liberties Union-Michigan, Walters, activist groups, and many others. Virginia Tech provided the technical plan, analytical support, and funding, while Flint residents donated their local knowledge, homes as test sites, and hard work. The collaboration started with a team from Virginia Tech driving to Flint and sampling the water distribution system, water heaters, and home plumbing for a wide array of potential chemical and microbial contaminants, which also laid a groundwork for the citizen coordinated citywide sampling event focused on water lead, which ultimately involved 269 geographically distributed Flint homes. Of the hundreds of analytes examined in the initial phase of testing in summer 2015 by the Virginia Tech team, only lead exceeded federal standards (Edwards 2015; ; Paynter 2016; ; ; ). That result, coupled with a later independent analysis by researchers at Hurley Medical Center demonstrating rising blood lead in some Flint children, led local medical authorities and, eventually, the State of Michigan to declare a water health advisory on October 1, 2015 and to switch back to Lake Huron water supplied by DWSD on October 15, 2015 (). These events are covered in the 2017 PBS NOVA documentary “Poisoned Water” (Smith and Thomson 2018).
In late 2015, high levels of Legionella pneumophila were detected inside large buildings in Flint by Virginia Tech (), and provision of these data to authorities helped to prompt revelations of a two year Legionnaires’ Disease outbreak in 2014 and 2015, with about a dozen deaths that occurred after the switch to Flint River water. Shortly thereafter (January 13, 2016), President Barack Obama declared a Federal Emergency (). The elevated lead in water, elevated lead in children’s blood, the Legionella outbreak, and the Federal Emergency created a media sensation that eventually mobilized hundreds of millions of US dollars of federal, state, and private relief funding for Flint ().
In the aftermath of the Federal Emergency declaration, the responsible and largely discredited government entities publicly accepted some blame and re-dedicated themselves to the relief effort, along with assistance mobilized by the Federal Emergency Management Agency (FEMA) and the National Guard, which distributed filters and bottled water (). The governor publicly apologized and set up a task force that included members of Virginia Tech, Hurley Medical Center, and other former vocal government critics to advise the recovery efforts (; ). Fifteen civil servants within the State of Michigan (including MDEQ) and City of Flint were charged for their role in the crisis ().
EPA, MDEQ, and the City of Flint then took additional measures to enhance disinfection and corrosion control beyond that normally present in treated water purchased from DWSD, to hasten recovery of the distribution system for both microbial and chemical contaminants. Subsequent sampling tracked the water system’s recovery in four citizen-led sampling campaigns in March 2016, July 2016, November 2016, and August 2017. Because Del Toral and Virginia Tech had proven that all official state water lead data before 2016 were collected using so-called “cheats” that effectively minimized detection of lead hazards at the tap, the citizen science datasets collected without cheats represented the only internally consistent measures of water lead levels during the water crisis and its aftermath. As a result, our follow-up citizen science team sampling efforts were funded by the federal government (i.e., US EPA) in March, July, and November 2016 to track the recovery (; ; ; ; ).
The citizen science sampling events and our other water quality research in Flint demonstrated expected improvements in tap water quality from the perspective of lead, Legionella, and metals that cause discolored water. Regulatory monitoring by MDEQ showed no problems with disinfection by-product regulations at any point during 2016, with 90th percentile lead levels below the EPA action level beginning in summer 2016. After 2015, the official MDEQ lead measurements were no longer in significant disagreement with our citizen science monitoring data (; ; ).
From a scientific and regulatory standpoint, Flint’s water met all existing federal standards by summer 2017, which by some definitions could mark an end to the public health crisis. But the problem of lost trust was an enormous issue (; ; ; ; ; ; ), creating a crisis of confidence () and an environment ripe for exploitation by rumors and fearmongering (). While the mistrust in government agencies was self-inflicted and deserved through at least early 2016, we argue herein that the credible work of relief agencies after that time were systematically undermined by those operating under a banner of citizen science.
Water Defense’s “Citizen Science” Program in Flint Coincides with Sharp Spike in Shigellosis Cases
Fears of Bathing and Showering
“You gotta understand you cannot be bathing in this water … a hot steamy shower–you’re simply asking for trouble.” Bob Bowcock ()
In January , environmental activist Erin Brockovich called attention to the ongoing problems with disinfection by-products (DBPs) in Flint, and sent her water expert Robert Bowcock to the city, who spoke to residents and recommended altering treatment (“fluoride, lime, and softener”) that could get “good water quality” in two months (, , , ; ; ). Bowcock did not recommend corrosion control but did suggest a switch back to Detroit water (). After President Obama’s Emergency Declaration, Brockovich continued to emphasize that residents should get filters “on their showerhead” () to reduce exposure to DBPs in showers, while Bowcock appeared on The Steve Harvey show alongside a local pediatrician and emphasized that “bathing in the water is not safe” (). At all times after the switch back to Detroit water, however, Flint had been meeting federal DBP standards and there was no basis to declare the water unsafe for bathing.
WaterBug™ testing, filters, and perceived conflicts of interest
“[We shouldn’t be having the debate of] whether or not my organization [Water Defense] is (sic) scientists, because we are admitting we are citizen scientists.… All we are doing is giving people information and there is nothing wrong with that. And the more information people have, the better off they are.” Mark Ruffalo ()
In late January 2016, months after the switch back to Lake Huron water from DWSD and just two weeks after President Obama’s declaration of a Federal Emergency had mobilized FEMA and the National Guard (; ; ; ), Mr. Scott Smith of actor Mark Ruffalo’s environmental nonprofit Water Defense came to Flint at the request of residents. Mr. Smith and Water Defense had recently launched WaterBug™–a green tentacled sponge product designed to “empower citizens to take water quality testing into their own hands and protect themselves” ().
Over the next five months, investigations by our team and others revealed that Mr. Smith had no formal scientific credentials to support a title of “Chief Scientist” and also had possible financial conflicts of interest. Of greatest concern for someone holding a high-profile public role with a 501(c)(3) nonprofit intervening in a Federal Emergency situation, he was: 1) inventor of foam materials used in the WaterBug™ and allegedly could receive royalty payments from the manufacturer for their purchase (); and 2) owned a for-profit company Aquaflex™ announced in May 2016, which was pitched to investors at the time as a maker of proprietary filtration systems to “satisfy unmet needs” as the Water Defense nonprofit “continue<d> to identify and diagnose water contamination sites.” While Mr. Smith publicly asserted at a Water Defense press conference in March 2016 that “The ultimate solution here is … to create green jobs, to create … solar powered [water] filtration [systems] … and help the economy in the process,” he did not prominently mention these possible financial conflicts or disclose the investor pitch to sell filters through his private Aquaflex™ venture or other parties (; ; ; ; ; ).
Citizen science sampling efforts, like all scientific sampling, must be rigorous and defensible and, whenever possible, use established protocols (e.g., from Standard Methods for the Examination of Water and Wastewater, 23rd ed.; ). They also must try to practice appropriate quality assurance/quality control, maintain data integrity (; ; ), and provide proper controls when appropriate to compare results to other geographic locations () or federal standards (e.g., from the US EPA and World Health Organization or WHO). For example, “proper” lead-in-water testing requires collecting 1 L water samples from the cold water kitchen or bathroom tap after a minimum 6 hour stagnation time according to EPA regulations (). In contrast, the proprietary Water Defense WaterBug™ had never been compared or calibrated against any accepted water testing standard, subjected to any peer-review, or even product-tested by standards organizations such as the National Sanitation Foundation.
It eventually became obvious that the floating WaterBug™ sponges, partly submerged in water, could collect chemicals from both water and ambient air, creating confusion when the detected contaminants were assumed to come from the water (; ). At one point, a Flint resident became concerned about bathing and showering in her home because the WaterBug™ detected high levels of acetone, which set off a search for sources of acetone contamination to the public water supply. However, the resident herself later determined that her bathroom contained multiple bottles of nail polish remover (100% acetone), which likely caused the false positive initially attributed to the water via the air (K. Webber, personal communication, Feb 4, 2018). In terms of microbial sample sterility, the resident claimed that Mr. Smith rubbed the WaterBug™ on his arm to demonstrate its relationship to skin testing, before placing it into the bathtub (S. Ganim, personal communication, Feb 13, 2018), consistent with other breaches of sterile microbial sampling protocols observable in Water Defense videos released online ().
During January to May 2016, Water Defense also routinely collected water samples in Flint homes from non-regulatory and improper sources including hose bibs, water meters, and water heaters and their sediment clean-out valves. All of these sampling locations create well-known false positives for lead, due to infrequently used plumbing components that are not designed to dispense water fit for human consumption because of known high lead risks (; ). As such they are termed by EPA as “improper” in relation to collecting data relevant to compliance with Federal law. While such samples can be used for diagnostic purposes, the resulting high lead data from these improper sites were publicly broadcast on social media as indicating serious and ongoing water lead concerns. Those data were complemented by the WaterBug™ test results in “detailed lab reports” containing as much as 97 pages of raw data listing hundreds of chemicals allegedly contaminating the water, which were presented without context as to what concentrations of these chemicals would be normal or “unsafe” in tap water (). Water Defense published press releases on their website with sensational titles including “Dangerous chemicals discovered in baths/showers of Flint, MI,” participated in town halls and press conferences, posted on social media with the hashtag #citizenscience, and were featured prominently in local media and Facebook groups by citizens (; ; ; ; ; ; ; ). From their very first video, taken before they had collected or analyzed a single Flint water sample, Water Defense appeared to have a bias toward finding data indicating that the water was “not safe for bathing and showering” ().
These messages were amplified on social media–where “falsehoods diffuse significantly farther, faster, deeper and more broadly than the truth” ()–and reached Flint residents during early 2016. For example, a TV news report on March 21, 2016 noted that “[Mr. Smith] for weeks has been urging people in Flint not to bath (sic) in <the water>” (). At this time, the Michigan Department of Health and Human Services’ (MDHHS) Chief Medical Executive Dr. Eden Wells tried to communicate the stance of all major agencies and researchers engaged in Flint relief efforts to Water Defense:
“On the bathing issue–both Dr. Mona [Hanna-Attisha] and the MDHHS state that based on what is currently known about the water system (including all the water tests, etc.), it is safe to bathe and shower. CDC and CDC ATSDR are in concurrence. Dr. Marc Edwards … is also in concurrence.” ()
Wells also shared details about an ongoing investigation of skin rashes in Flint by CDC that eventually showed a normal rash incidence at that time. However, Water Defense was combative in response to Wells and others and would not accept a scientific consensus opinion about bathing and showering safety during the Federal Emergency ().
The fact that Mr. Smith had no professional degrees in science or engineering () did not impede broadcasting of his messages and, assisted by Mr. Ruffalo’s star power, it may even have helped, but at some point the Chief Scientist title was dropped (; ; ; ). Mr. Ruffalo personally appeared on CNN, falsely stating that disinfection by-products (DBPs) were coming from Flint’s pipes damaged by corrosion, and reiterated his organization’s opinion that “[no one] can tell the people of Flint that it’s safe to bathe in [Flint] water,” again directly contradicting a unified message from all relief agencies that the Flint water was not more dangerous for bathing than other cities (; ; ). Independent professors from Stanford, University of Massachusetts-Amherst, and University of South Carolina with expertise in DBPs were recruited by our team to conduct their own investigations in Flint. Their findings also flatly contradicted Ruffalo’s false statement about DBPs originating from Flint pipes, which supported the relief agencies’ message to some extent ().
Water Defense messaging on bathing and showering and shigellosis cases in Flint
“As an ethical matter, pseudoscience is not—contrary to popular belief—merely a harmless pastime of the gullible; it often threatens people’s welfare, sometimes fatally so.” Maasimo Pigliucci and Marten Boudry ()
The government agencies whose earlier claims about water safety for lead were proved wrong by our citizen science collaboration had little or no credibility in Flint after late 2015 (). In fact, after our prior citizen science success, almost any group directly contradicting the agencies was given media coverage, with explicit reference to the prior vindication of whistleblowers Drs. Edwards and Hanna-Attisha, and Mr. Del Toral. Given their past failures, the agencies were also unwilling to directly contradict anyone openly undermining their expertise and authority. Distrust, fear, and anxiety toward the water by citizens was already justified, and residents continued to report skin rashes, hair loss, and other dermal ailments that they felt were due to water (; ). While the US Centers for Disease Control and Prevention (CDC) conducted a study that indicated higher than normal rates of rashes during the water crisis in 2014–2015, confirming residents’ beliefs about health risks from that time, they also reported normal rates of rashes after the water source was switched back to DWSD (). Nevertheless, our team personally received at least a dozen communications from worried residents who cited Water Defense data and associated press releases as justifications for stopping their normal bathing or showers.
An outbreak of Shigellosis began soon after Water Defense arrived in Flint (; ) (Figure 1). Shigellosis is a gastrointestinal illness caused by Shigella bacteria, often resulting in severe diarrhea, stomach cramps, and fever. The bacteria is most commonly spread through the fecal-oral pathway (e.g., by coming in contact with infected stool and/or soiled diapers), and only rarely from contaminated water in the US. The disease is best controlled by normal bathing/showering and regular handwashing with soap in disinfected water (). Flint is in Genesee County, which had a Shigella incidence 1.05 cases/month (35% lower than the national average) when the city was being served by Flint River water during the crisis (April 2014–October 2015), up from a rate of 0.47 cases/month during a comparable control period (April 2012–October 2013) before the water switch from DSWD (one-tailed paired t-test; p < 0.05). But shigellosis rose to 13.8 cases/month (6.7 times the national incidence rate from February to October 2016) in the immediate aftermath of the Federal Emergency in January 2016 and Water Defense’s arrival. Overlaying the timelines of monthly shigellosis incidence in 2016 with key public statements from Water Defense, Virginia Tech, health agencies (Genesee Co. Health Dept. or GCHD, and CDC), and some Flint residents during the outbreak is enlightening (Figure 1).
Shigellosis cases in Flint (2012–15 versus 2016) vis-à-vis conflicting public statements by Water Defense and others on bathing, showering, and Shigella in local, national, or social media. Monthly Shigella case data provided by Michigan Department of Health and Human Services (MDHHS). References (chronological as per graph): ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; .
What was the source of the outbreak? All water samples (150 total) collected by Virginia Tech from 30 Flint homes in June 2016 (a peak month of the shigellosis outbreak) were later found to be negative for Shigella, providing strong evidence that the outbreak was not from water (). A CDC-led study relying on survey questions, clinical dermatologic evaluations, genetic mapping, and water quality testing in Flint homes (n = 390) also later concluded that Shigella did not likely originate from Flint drinking water. From early 2016 the relief agencies and Virginia Tech considered it plausible that the misinformation on the dangers of bathing and showering compounded the legitimate fears of Flint residents, resulting in less effective hygiene that would make a Shigella outbreak more likely and of longer duration. Indeed, by August 2016, CDC revealed alarming statistics that nearly 80% of Flint residents surveyed from January 21 to April 29, 2016 reported that they had changed their bathing and showering habits including “showering less frequently” (75.3%), “taking shorter showers” (70.6%), and bathing in bottled or filtered water (; ). In a follow-up survey of Flint residents that had shigellosis (n = 24), 52% reported that they had changed their bathing and showering habits, a much higher percentage than changes in bathing reported in other affected households in Genesee and Saginaw Counties (J. Yoder, Personal Communication, Feb 1, 2018). Top motivating factors for these behavior changes were “media reports” and “health concerns” about bathing (; ).
In light of Water Defense’s lack of expertise and the possible harm to public health arising from their problematic messaging, our team attempted to reason privately with the group in April 2016, well before studies confirmed that the Shigella source was likely conventional transmission and not from the water. Without any explanation, Water Defense did not call into a scheduled teleconference meeting to discuss the issue. We then asked them to produce scientific evidence backing up their erroneous public statements about bathing and showering dangers, and informed them of the public health harm that could result if residents’ bathing habits were altered by their statements. At one point Erin Brockovich’s water expert Mr. Bowcock wrote us that while Water Defense had “spent big money on samples,” their DBP testing was “ridiculous,” and he “brokered an opportunity to fix this mess” but Water Defense refused (R. Bowcock, personal communication, May 16–19, 2016). Having tried to resolve the issue twice and failed, we informed Water Defense that we felt ethically obligated to publicly undermine their scientific credibility on our website. We even shared a draft blog to encourage them to correct their messaging and avoid a public confrontation, but after they refused, we posted the blog because we felt that their actions posed a direct threat to the public health and welfare (; ; ).
The resulting distasteful confrontation–between our citizen science team engaged in Flint and a non-profit backed by a high-profile Hollywood actor–did create negative publicity for Water Defense and Ruffalo, generating widespread national media coverage including The New York Times Magazine and other sources (; ; ; ; ; ). To further combat misinformation, we coordinated the 3rd and 4th party-independent EPA funded sampling events for DBPs in cold tap, hot tap, and shower water in Flint households, and eventually demonstrated that the DBP situation was completely normal (; ). We also sought written scientific opinions and analytical reports from third-party academic water experts and assisted journalistic investigations into Mr. Smith’s financial dealings (; ; ; ; ; ; ). These efforts debunked many claims and compromised the reputation of Water Defense, but they also hardened loyalty to Water Defense among some Flint activists who would not acknowledge their unscientific approach. A few residents viewed our actions as betraying common-cause activist unity against the County, State, and Federal government health authorities.
While a direct cause and effect relationship cannot be proven, there is indeed a strong temporal link consistent with the conventional hypothesis regarding spread of Shigella and the altered bathing habits of Flint residents (Figure 1). And at a minimum, considering the unified statement of scientific authorities (), Water Defense and others were sending the wrong message at the wrong time and in the wrong place about bathing and showering dangers, using an approach directly attacking the expertise of the relief agencies and scientists who had relevant experience and sound data. Following our confrontation, only 7 shigellosis cases were reported from January to June 2017, indicating that the risk had finally returned to levels considered normal. A few citizens, however, remain adamant that the Shigella outbreak came from the water (Table 1), demonstrating the remarkable staying power of bad science as illustrated by the persistent “vaccines cause autism” exemplar (; Mikulak 2015).
Table 1
Illustrative scientific claims by Flint Resident A.
| No. | Resident A Quotes | Reference |
|---|---|---|
| 1 | “[The State of Michigan is] willing to kill people [for land]” | CAN TV 2018 |
| 2 | “[Discolored water photo from 2015 shared in July 2018 proves that there is an ongoing] legal genocide [in Flint]” | |
| 3 | “Our showers are like gas
chambers” “Anything is going to release steam or heat […] and make our very well-insulated homes gas chambers” | |
| 4 | “We have shigella because we wash our hands” | CAN TV 2018 |
| 5 | “The [State-distributed lead] filters cause dysentery” | CAN TV 2018 |
| 6 | “The State of Michigan has blocked [a study showing lead filters were harmful] that Wayne State [University] has been trying to get … out for a year” | CAN TV 2018 |
| 7 | “I didn’t win an award for highest lead. Right now, [Resident B’s] got that … [Resident B] likes tiaras. You get a lead tiara, baby doll” | |
| 8 | “They also say you cannot get lead poisoning through showering. So, here’s [Resident B’s] blood lead level. 38.4 [μg/dL]” | |
| 9 | “This gorgeous #Flint baby is only 3 months old and was born w/high lead and e. coli b/c mom took baths while pregnant” | |
| 10 | “You get two times the exposure in a 10-minute shower than you do drinking two liters of the same water” <a Water Defense claim> | |
| 11 | “And the next person who suggests
our kids had higher lead levels prior to the Water Crisis is going
to get a public shaming for shoddy and LAZY research and
“reporting”” <Responding to actual data showing exactly that> | |
| 12 | “Residents have THOUSANDS of reasons (AND ppb of lead in our water) to NOT trust the “it’s all better” lie forced upon us” | |
| 13 | “Transdermal absorption of lead [from shower water] is an issue” <shared alongside a Medscape citation that states the opposite> | Edwards and Roy 2018 |
| 14 | “Shower filters are [quickly] spent because of the [corrosion control chemicals]” | |
Collaboration between Water Defense and “The Largest Online News Network in the World”
“Unfortunately, the media have trouble distinguishing between real science and propaganda cross-dressed as science.” Linda Bowles, columnist ()
Water Defense initially seemed undeterred by either bad publicity or worrisome news of the unfolding Shigella outbreak, and issued a press release stating that they “never suggested or implied that [their] concerns outweigh the dangers of not bathing” () even though media coverage (television and social media) interpreted their message otherwise (Figure 2). In December 2016, a political reporter named Jordan Chariton from The Young Turks Network (TYT Politics) began to work closely with Water Defense. In East Chicago, IN, Mr. Chariton carried out “water testing” under Mr. Smith’s guidance in the basement of one home, by scraping the rust buildup on the outside of a sewer pipe with a goal of finding “bacterial fungi” that the EPA could not find, while collecting water in unsanitary plastic bottles from a water sediment clean-out tap on a water heater (). The Chariton-Smith duo followed this with testing in at least three other East Chicago homes using the WaterBug™ sponge and other improper sampling methods, while livestreaming and/or broadly disseminating video interviews with residents presented with worrisome test results on YouTube and Facebook (; ; ).
A partial screengrab of a public YouTube video posted by a Flint resident (Resident A) using Water Defense data/assertions to declare “Bathing in Flint is NOT SAFE”. (image edited to remove Resident A’s real name from screenshot).
As a TYT Politics reporter, Mr. Chariton engaged in Flint between Dec 2016–Oct 2017 by holding interviews, conducting a Town Hall meeting, and uploading videos questioning Flint’s improving water quality based on anecdotal evidence from residents and prominently citing Mr. Smith’s WaterBug™ data (; ; ; ). Mr. Smith called out traditional Flint and national journalists as practicing “contrived reporting,” characterizing the multiple sources of evidence that Flint water quality was improving as “fake news,” and accusing normal media of being sellouts (). Chariton and Smith also pushed a conspiracy theory that the Virginia Tech citizen science team was no longer being truthful and had “sold out” residents because we received funding from the EPA and State of Michigan. TYT even hosted a prime-time news report centered on making an edited innocuous private phone conversation between the leader of our Flint Water Study citizen science team (Dr. Edwards) and a Flint resident (recorded without consent) into a conspiracy about hiding dangers from residents (; ). Chariton also attacked the EPA and State of Michigan repeatedly on air (; ; ; ).
In collaboration with some Flint residents, we again made a difficult decision to directly undermine this false messaging, which we deemed dangerous to the public welfare. To achieve this objective, we compiled vignettes from Water Defense and TYT Politics’ published YouTube videos to highlight obvious improper water sampling methods in an online blog post (). The videos documented invalid methods, improper sampling locations, unsterile protocols for microbial sampling, and misrepresentation of some test results–in essence, revealing the lack of rigor and poor scientific understanding by Water Defense and TYT. After the blog post went live, Mr. Chariton was visibly angry, and wrote a series of threatening emails demanding that we remove the blog post or face a “public dispute with the largest online news channel in the world,” which would include future feature stories that “prove [Dr. Edwards is] a hack” (). He also threatened lawsuits and at one point hired a lawyer. We did not delete the videos, and were indeed attacked continually online by Mr. Chariton, until he was finally fired from the network in November 2017 after acknowledging sexual relations with subordinates in Flint (; ). This chance event, and not the unscientific testing, erroneous reporting, or conspiracy theories, was the only reason that Chariton’s Flint reporting was temporarily halted. Indeed, after just a few months out of the spotlight, Chariton returned in 2018 as an unaffiliated journalist, declaring that the State of Michigan was perpetrating “ethnic cleansing” of Flint residents and renewing his public attacks on the State of Michigan and our team (Edwards and Roy 2018).
Perceived Conflict of Interest Involving Mr. Smith and Promoters of NLP Filters and Conditioners
“George Orwell first noted, the true genius in advertising is to sell you the solution and the problem.” Ben Goldacre, Bad Science ()
In December 2016, a Flint family began posting videos on Facebook, praising a whole-house filter “package deal” from a company called NLP Aqua Solutions that would cost US $11,300–$13,800 per Flint household to install (; ). Mr. Smith and NLP representatives appear in some of the videos (; ), including one where the resident read the following aloud from a document allegedly written by Mr. Smith:
“[NLP] is the first filtering technology I have reviewed with test results that makes me comfortable in saying the water is safe for drinking, showering, and bathing…. I look forward to taking a shower at your house and drinking the water after the treated NLP Aqua Solutions technology” (; S. Smith, personal communication, Jun 28, 2018)
Given Mr. Smith’s affiliation with Ruffalo’s Water Defense, this statement seemed to imply a celebrity endorsement of a for-profit, proprietary product, amidst the Federal Emergency and widespread fear of bathing/showering that Water Defense itself helped to create. The Flint family also referenced Smith as stating that their particular filtered water was “safer than bottled water” (). After determining that these filters cost 5–7 times more than other off-the-shelf filters certified to remove more contaminants, we noted that the package deal also included water treatment devices called “conditioners” (Figure 3) that the company sales literature stated could soften water without using any chemicals and could protect home plumbing and appliances from damage. The NLP company website made claims that this specially conditioned water boosted plant root growth and even increased milk production when consumed by cows. We tested this device in the laboratory and found that it did not perform as claimed, consistent with assertions by other Internet sources that this conditioner was “junk science” (; ; ; ).
An NLP water “conditioner” that claimed to soften water without any treatment chemicals did not perform as advertised in a battery of laboratory tests conducted by our team.
Our effort to expose what we then considered to be profit-minded, unscientific, and harmful activities by Mr. Smith while representing Water Defense resulted in numerous “Cease and Desist” letters from Mr. Ruffalo’s lawyers. We did not cease or desist, but eventually uncovered additional evidence of perceived financial conflicts, including social media postings on the filter package deal where US $2,100 was apparently earmarked for testing by Mr. Smith, and letterheads of lab reports given to Flint residents that had Mr. Smith’s for-profit “Aquaflex™” address on them (). These revelations finally resulted in Ruffalo’s nonprofit and Mr. Smith parting ways in June 2017, perhaps to protect Ruffalo’s celebrity or because their arrangement violated Water Defense’s commitment of not “hav<ing> any […] connection with [Aquaflex™]” (S. Ganim, Personal Communication, Feb 13, 2018; BJ Chisholm, Personal Communication, Feb 16, 2018; ).
Citizen Science Misconduct?
“Tell a lie once and all your truths become questionable.” Anonymous
In our initial sampling efforts in August 2015, the citizen science team of residents and scientists exercised extreme care to accurately collect samples, developing tamper-resistant sample kits, screening videos on proper sampling techniques to residents whenever possible, and collecting samples ourselves when we thought that residents might not follow the protocols. The citizens correctly anticipated that if our team found a citywide lead problem, both our motivation and methods would be questioned, and they wanted data to withstand scrutiny. After all, our team was directly challenging the power and conclusions of MDEQ and US EPA. When one reporter asked Virginia Tech “How do you know Flint residents are not just adding lead to the water to gain attention?”, we cited the care in sample collection shown by the residents and some high water lead samples collected exclusively by our Virginia Tech team.
After the initial triumphs following our citywide sampling event and unprecedented publicity, the tables were completely reversed. Nearly any sampling result put forth by residents, or any hypothesized link between ailments and tap water, could be broadcast in the national and international media frenzy. Water Defense, TYT Politics, and some citizens actively began to use improper sampling methods that obtained artificially “high” water lead results, which could be used to gain media attention or to push an agenda.
Case 1
Resident A confirmed to us in writing in 2015 that her home had a copper service line and modern lead-free plumbing (i.e., no lead solder) inside her house. The maximum water lead level detected at her kitchen faucet from five samples collected during the crisis period with highly corrosive Flint River water was 8 ppb. This was below the 15 ppb EPA action level and consistent with her initial claims that her plumbing had no lead solder or lead pipe. Her results changed dramatically (150 ppb) after she began to openly collaborate with Mr. Smith and collected samples from a hose bib in her basement, which is an improper sampling location according to US EPA because it creates false positive (high lead) results (). Despite willfully violating the written protocols in test kits provided by the State of Michigan or Virginia Tech, which clearly indicate to test only proper locations using the kits, Resident A’s high lead results were broadcast by Mr. Smith and Mr. Ruffalo on social media with no mention of the improper sampling approach. As time went on, her lead results rose to 160 ppb and then went “sky-high” to 1,740 ppb during our fourth round of citizen science testing (Figure 4). After getting these results, Resident A held press conferences, gave interviews, spoke at the 2017 Women’s March in Washington D.C. advertising her lead levels “in the 4-digits,” and also disseminated her results via gullible national media to assert that water quality in Flint was actually getting worse (; ; ; ; ; ; ; ; Mendes and Moore 2017). Our team reported her improper sampling to the US EPA and told Resident A that she could not engage in further citizen science sampling with our team ().
Resident A’s lead levels during the Flint Water Crisis. For every test, the top box indicates the sample collection location base on the resident’s statements and the bottom box shows who coordinated the testing [MDEQ, Virginia Tech = VT, or Water Defense = WD]. The highest lead value obtained per sampling event is stated above the horizontal axis.
Resident A also attempted to use these high lead results in a manner that illustrated a possible financial conflict of interest. For example, in one publicly accessible social media posting, she cited her high lead result, and desire to be given a free service line replacement, which at the time was prioritized for residents who had dangerous lead pipe (). After other residents called her out for this opportunism, and produced our written documentation showing that Resident A believed her service line was copper, Resident A deleted the posting. Over 2015–18, her claims of illness changed from rashes and hair loss (, The Tom Sumner Show 2016) to being poisoned and “lead and copper [permanently] stored in [her] brain” from water (Mays 2017–18, Mendes and Moore 2017). She insisted, without support, that “the longer [the government is] waiting to pull the [lead] pipes out, the bacteria’s worse, all the different cancer-causing byproducts are worse” (). She was also lead plaintiff on several lawsuits (Mays et al. v. City of Flint et al. Genesee County Circuit Court; Mays et al. v. Snyder et al. Michigan Court of Claims; and Mays et al. v. Snyder et al. US Federal Court for the Eastern District of Michigan) in which high lead data played a role (). At least one media report of her improper high lead test result from Virginia Tech was explicitly tied to her personal lawsuits (Mendes and Moore 2017).
Resident A publicly displayed questionable citizen science judgment even before she started working with Water Defense. For example, she posted a publicly accessible video on Facebook in January 2016, which was viewed 17,000 times, where a family member evaluated state-supplied bottled water using a “water tester.” The water tester was a Total Dissolved Solids meter that measured dissolved salts in water, not lead, but the family member claimed that it showed the bottled water was “horrible” for consumption (, ). During October 2015–January 2016, she claimed that “21,000 people” in Flint “lost their homes” because they couldn’t afford water bills, that Child Protective Services threatened to remove children from Flint households due to bad water, and that Flint residents were legally forbidden from selling their homes due to the water crisis–all of which were debunked by outside sources (, , , ). More recently, Resident A shared a 2015 Flint photo of a fire hydrant spewing discolored water on her public social media page, claiming it was actually from late-June 2018 (Figure 5) and alleging that it illustrated “legal genocide” of Flint residents (, ). The photo was liked and shared tens of thousands of times before it was debunked by our team and others (). After this was exposed, Resident A stated “Here is yet ANOTHER example of [the Flint Water Study team] taking it upon themselves to attack poisoned Flint residents and call them liars” (Edwards and Roy 2018). It has been argued elsewhere that some activists do not believe in or respond to scientific evidence, especially if it challenges their personal ideologies and political goals ().
Our online meme comparing Resident A’s fire hydrant photo from June 2018 supposedly showing horrible water (left) compared to the same photo from 2015 taken during the height of the Flint Water Crisis (right).
Other illustrative comments from this resident (see Table 1 for a representative list) included claims that the Flint water was transforming bathroom showers into “gas chambers” and, therefore, “until every single piece of plumbing is replaced, [Flint residents] are in danger” (; ). She openly attacked all the agencies, Virginia Tech, and other residents who presented data indicating that the water quality was improving. Throughout, this resident’s disruptive activities and statements were rarely refuted, in fact they were openly embraced, encouraged, and publicized by two supportive professors who were engaged in Flint (; ; ; ).
Case 2
A few months after we had reported Resident A to EPA for collecting improper samples in our federally funded citizen science project, Resident B had a water lead level sampling trend even more startling than Resident A. During the recovery phase of the system prior to compliance with the federal Lead and Copper Rule in July 2016, when there was still a citywide problem, 12 analyzed water samples from her home were between 5–44 ppb. However, 10 months after Resident A first publicized an approach to collecting improper samples (attributed to Mr. Smith) that obtained very high water lead data, Resident B’s lead levels reached and surpassed hazardous waste levels (>5,000 ppb) in July 2017 (Figure 6), triggering both alarm and widespread media coverage (). Resident B reportedly had a copper service line.
Resident B’s lead levels (2016–17) during and after the Flint Water Crisis (Source: MDEQ, publicly available social media postings).
Perplexed as to how this could be, we submitted a Freedom of Information Act (FOIA) request to the US EPA which revealed that Mr. Smith and a colleague discovered two lead fishing sinkers inside her plumbing in early August 2017 (). Before we made the FOIA public, Resident A, Mr. Chariton, and other residents had broadcast Resident B’s high lead results on social media, and her data were also cited at the premier of the LIFETIME movie to support the assertion that the water lead crisis was continuing in Flint (; ; Roy and Edwards 2017d). We also revealed that Resident B had possible financial conflicts of interest, as evidenced by launch of a personal GoFundMe online fundraiser that cited her high water lead results, and interviews in which she claimed that all “[water] mains, service lines, everybody’s […] hot water heaters, refrigerators, washers, everything” should be replaced (; ). Resident B is also part of a class action lawsuit along with Resident A ().
The timeline of her public social media postings also documented formation of her personal hypothesis that Flint residents could get lead poisoning simply by taking a shower (; ; Roy and Edwards 2017d). This hypothesis had been frequently cited publicly by Mr. Smith, despite the fact that it was contradicted by decades of sound research from reputable agencies, scientists, and engineers. Mr. Bowcock also called Smith’s hypothesis about lead poisoning from showers “ridiculous” (R. Bowcock, personal communication, May 16, 2016). Taking on a role of a biomedical research pioneer, Resident B put forth the lead poisoning by showering hypothesis in an interview to Mr. Chariton’s TATM channel on June 23, 2017, and then took a test proving her blood lead tested low at 2.7 μg/dL just two days later. Then, just six weeks later, she shared a blood test result indicating extreme lead poisoning (38.4 μg/dL) on social media by posting: “you most certainly can get lead poisoning by showering in toxic water!” (). A deluge of social media and local media coverage of this discovery followed. But importantly, at no point was any public mention made of the potentially important discovery of the lead fishing sinkers inside her plumbing by Resident A, Resident B, or Mr. Chariton.
After we published the FOIA with videos of the lead sinkers found in her plumbing and laid out the timeline of events online, including her GoFundMe launch (claiming that she was poisoned before obtaining her high blood lead test results), Chariton wrote that the sinkers had “NOTHING to do with [Resident B’s] high lead-levels” but her lawyer nonetheless barred her from talking to the media (; ; Roy and Edwards 2017d). Resident B’s lawyer first claimed that no lead sinkers were found inside her bathroom tap, but then retracted the statement and, finally, made a claim that the lead sinkers would not matter anyway because they would not leach any lead. We disproved that statement with our own experiments using lead sinkers, and one of us (MAE) even took a shower in toxic levels of lead with no detectable biological impacts (). Her lawyer then labeled the lead sinker story a “distraction” from the class action lawsuit that Resident B was party to and suggested that “a big defamation lawsuit” could be filed against us. The resident’s GoFundMe page disappeared within approximately two weeks of our blog posting (Roy and Edwards 2017d). The state has declined to release other emails associated with this case under the Freedom of Information Act, stating that this “could jeopardize an investigation by [EPA’s] Criminal Investigation Division” ().
Resident A (and members of her social network) repeatedly used Resident B’s astronomically high lead results to manufacture outrage regarding Flint’s water quality (e.g., ), even months after the lead sinker issue was exposed on our website (e.g., ). At the launch event of the LIFETIME movie “FLINT” in October 2017 featuring Resident A as the heroine and a false portrayal of Ms. Walters as a quitter with an unsupportive husband (), the high water lead data from Resident B were cited to assert in the media that “[Flint residents] are still being poisoned” (). Again, there was no mention of the fact that Mr. Smith had reported a discovery of lead sinkers in her plumbing to EPA. Resident B also never mentioned the lead sinkers at an MDEQ public hearing as late as March 2018 when reporting her high water lead results ().
Sobering Lessons on the Abuse of Citizen Science in Post Federal Emergency Flint
“There is nothing to fear except the persistent refusal to find out the truth.” Dorothy Thompson ()
“When the facts change I change my mind, sir. What do you do?” Attributed to John Maynard Keynes
The aforementioned actions of a nonprofit, an online news network, and a few residents (and unequivocal support of such actions from a few academics) suggest abuse of the citizen science approach to research including: Data falsification, failing to disclose and manage conflicts of interest, making scientific claims without evidence, or co-opting the term “citizen science” as a tool for activism. The events also highlight legitimate concerns, ethical dilemmas, and perverse incentives surrounding citizen science, at least in this unusual situation. The repeated and irresponsible practice of bad science to attack and disrupt expertise of government science agencies and others during the Flint Water Crisis relief effort endangered the public welfare and undermined trust in science itself.
We have been outspoken about the danger of perverse incentives in modern academia, where promotion and tenure are increasingly tied to quantitative metrics including research papers and funding dollars and which, in turn, increase the likelihood of misconduct and threaten scientific integrity (). Flint demonstrates that perverse incentives also exist for citizen scientists, especially if they find answers that lead to financial rewards and media attention (i.e., a LIFETIME movie portrayal, class action lawsuits, and GoFundMe campaigns). It is possible that such incentives are not as “anomalous” as previously assumed (), especially in cases involving lawsuits and environmental injustice.
Characteristics of quality citizen science projects
“The method of science, as stodgy and grumpy as it may seem, is far more important than the findings of science.” Carl Sagan
Freitag and colleagues () list 12 strategies to enhance credibility of citizen science projects, including prior expertise of project leaders, substantial training of citizen scientists, science advising to strengthen data collection, peer-oversight between citizen scientists, quality assurance protocols, and cross-comparison of citizen science data with data from professionals. This approach emphasizes rigor and objectivity and views science as a truth-seeking exercise, albeit subject to the same limitations and biases that all science is subject to. We evaluated our initial citizen science collaboration that exposed the Flint Water Crisis against Freitag et al.’s criteria and believe that it satisfied all 11 applicable criteria. The Water Defense-TYT Politics alliance, in contrast, arguably satisfied only two (see Table 2).
Table 2
Strategies from Freitag and colleagues () used to assess the initial Virginia Tech-Flint residents citizen science collaboration and Water Defense-TYT Politics alliance.
| No. | Strategies for demonstrating credibility | Assessment of VT-Flint citizen science collaboration | Assessment of Water Defense-TYT Politics alliance |
|---|---|---|---|
|
EARLY ACTIONS | |||
| 1 | Prior expertise (is there a “formalized minimum standard that volunteers must meet to participate?”: Yes, No) | Not Applicable; water samples for lead are normally collected by residents under the federal LCR | Not Applicable; anyone can “take water quality testing into their own hands” using WaterBug™ () |
| 2 | Training (“time investment” to train volunteers: None, Low, Medium, High) | Low; required viewing of YouTube sampling video | None for WaterBug™ |
| 3 | Science advising (“partnership with a university lab”, etc.: Yes, No) | Yes; with Virginia Tech scientists | Yes; but science advisors listed had no drinking water expertise (, ) |
|
IN THE FIELD | |||
| 4 | Ranking system (Volunteers designated as “experts” after gathering experience or passing tests: Yes, No) | Yes; for select residents. Flint residents paid by EPA to execute citizen sampling | No; Any citizens can be experts and “take water quality testing into their own hands” using WaterBug™ () |
| 5 | In-person oversight (“expert” volunteers to “directly oversee data collection” to minimize data collection errors: Yes, No) | Yes; cross-checking survey data between citizens | No; Mr. Smith did not initially have expertise in EPA protocols and collected many improper samples () |
| 6 | Retraining (Advancement of volunteer skills through more trainings, readings, etc.: Yes, Optional, No) | Yes; select residents sampling for chlorine | No; WaterBug™ was hypothesized as useful for all contaminants, including heavy metals, pathogens, DBPs |
| 7 | Technological aids (Simplifying data collection using technology: Yes, No) | Yes; Mostly using Excel | No; “independent lab” provided data dump in “lab reports” of up to 97 pages () |
|
IN THE OFFICE | |||
| 8 | Validation of observations: (Checks for human error and “statistics-driven flagging of incorrect data,” etc.: Yes, No) | Yes; Attempt for rigorous evaluation of collection methods for each sampled home | No |
| 9 | Cross-comparison (Comparing volunteer data to those collected by scientific experts to demonstrate credibility of “methods and data”: Yes, No) | Yes; samples collected from same locations by citizen scientists and third-party experts | No |
| 10 | Publication (external peer-review and/or publishing of data and findings: Yes, No) | Yes; several papers published in respected journals | Yes; peer-review of data by scientists (but none with drinking water expertise), no journal articles |
| 11 | Management use (Decision-makers use citizen science data: Yes, No) | Yes; informed Federal Emergency declaration and EPA funded work | No; Attempt was made to undermine decision-makers |
| 12 | Quality assurance protocol (Standard QA practice to “calibrate methods, technology, and practice over time”: Yes, No) | Yes; as per Standard Methods and EPA guidelines | No; WaterBug™ not vetted against established EPA/WHO guidelines |
Above and beyond the framework of Freitag, the Flint experience and additional research (e.g., ) has highlighted other (qualitative and ethical) characteristics of good citizen science which are lacking when citizen science can be hijacked by science anarchists and social justice advocates with agendas other than truth-seeking. These include embracing scientific rigor; willingly reporting evidence counter to one’s previous expectations or goals (i.e., intellectual honesty); working only in one’s area of competence; transparency; meeting obligations to correct misinformation; and openly addressing conflicts of interest.
Persistent attacks on merited expertise and the Dunning-Kruger Effect
“Democracy cannot function when every citizen is an expert. Yes, it is unbridled ego for experts to believe they can run a democracy while ignoring its voters; it is also, however, ignorant narcissism for laypeople to believe that they can maintain a large and advanced nation without listening to the voices of those more educated and experienced than themselves.” Tom Nichols ()
It could be argued that citizen science attempts to reduce the worrisome power and expertise gap between experts and non-experts through collaboration, respect for an individual’s experiences and observations (“other ways of knowing”), and working together to create new knowledge (; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ). These citizen science collaborations have often served as catalysts for scientific discovery and exposed abuses of power and scientific misconduct (Ottinger 2016; ), as illustrated in Tonawanda NY, Washington D.C., and Flint where local knowledge, experience, and advocacy were essential (; ; ; ; Cooper 2017). If scientists ignore individuals with local knowledge and expertise, the results can be disastrous (; Evans and Collins 2007).
In contrast, the disruptive actions of Water Defense, TYT Politics, and Flint Residents A-B could be considered an unqualified extreme of “anything goes” in a “citizen engineering” model, which was recently defined as an exercise by which “community members (“non-experts”) identify scientific questions and proceed through a formal process … sometimes but not always with the cooperation of trained scientists,” where “… disruption of engineering expertise is a central goal….” (). This model was best demonstrated when Resident A made several dubious scientific claims during a February 2018 public health panel (Table 1) at the University of Illinois Chicago, which aimed to “tell stories from community members […] to get scientific […] facts [about issues]” (). At this event, Resident A stated:
“[The State of Michigan is] willing to kill people [for their land]. They don’t care. […] They are going to clear that land out [and] knock the home down [for rich developers].” ()
without any contradiction from other panelists, moderators, or audience members. She also repeated the false narrative about bathing and showering dangers:
“We get told by a certain PhD that […] you poor, uneducated, dumb and dirty people in Flint. You have shigella because you didn’t wash your hands. Because you are too scared of the water. Excuse me. We have shigella because we wash our hands.” ()
Resident A used similar “dumb and dirty” quotes on her public Facebook page and in The New York Times Magazine (; ) to discredit the warnings from the county health department and other authorities about possible dangers of altered bathing habits. She attacked the Federal, State, and Country authorities in one post entitled “Blaming the Victims: The Lies our County, State and Federal Government Tell about Flint.” The post reiterated that government officials’ messaging about bathing was a “LIE” that “Flint is full of dumb and dirty people who get sick because they’re not bathing and washing their hands” (). She further stated “Shame on you people for saying garbage like that just to try to throw off the fact that the bacteria in our water is making us sick,” and that she was “ready to turn those shaming tables back around on you #ItsYourTurnNow” ().
Interestingly, Resident A was a contributing member of our initial Flint Citizen Science team until late 2015–early 2016 when she started to collaborate with Water Defense and TYT’s Jordan Chariton, whereas our team chose to work alongside the very agencies that we had previously attacked after they apologized. Resident A seemed to remain stuck in attack mode, even after the heroic EPA whistleblower Miguel Del Toral who originally worked with Walters was recruited to become the public face of the EPA response (; ).
We note that the best scientists, with decades of training, must always be on guard to minimize the likelihood of self-deception and politicization of their work (; ; ; ). A competent scientist has developed self-awareness, moral intuition, and a strong sense of humility, especially in their proven area of expertise (; Kruger and Dunning 1999; Klein 2016). Scientists are looked at as “purveyors of truth” (), which is indeed an ideal worth aspiring to while communicating both facts and uncertainties. This perception of scientists was also evident in a Spring 2016 Flint poll that showed “University Scientists” (i.e., Virginia Tech) as being “Helpful and Trusted” by Flint Residents () and the “whistle-blowers for Flint” (). The general public does trust scientific expertise, with Pew and Gallup polls in 2016 showing that more than 75% of Americans trust scientists to act in the larger interest and 65% rate engineers’ honesty and integrity as “High” or “Very High” (up from 48% in 1976), respectively (; ). In 45 years, “science is the one [public] institution that has not suffered any erosion of public confidence” (). Similar trends can be seen in Europe ().
On the other hand, citizen science abusers who openly embrace subjectivity and attack all expertise embody the Dunning-Kruger Effect (Figure 7), which is summarized: “Those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it” (Kruger and Dunning 1999). The chronicled examples illustrate a populist anti-elitist, “all opinions are equally valid” anarchist movement that aims to undermine expertise, scientific rigor, and organized knowledge. This is also embraced in the “Citizen Engineering” academic model that is occasionally taught to non-scientists on U.S. college campuses, which can be viewed as supporting anti-expertise social justice movements (; ; Edwards and Roy 2018; ). With no scientific background or legitimate experience in water quality or engineering, Mr. Smith (Water Defense), Mr. Chariton (TYT Politics), and Flint Residents A-B were truly “unskilled and unaware of [their incompetence]” (Kruger and Dunning 1999). When they took the spotlight and engaged in citizen science abuse in post-Federal Emergency Flint, the results certainly did achieve the goal of “disrupting expertise” in “communities,” but without any targeted desirable effect of making “crucial contributions that improve engineering practice” ().
The Dunning-Kruger Effect (illustrative simplified version). “Those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it” (Kruger and Dunning 1999).
To be clear, we are not opposed to activism that stems from legitimate science practiced by or with citizen scientists, sometimes termed “scientific authority-driven based citizen science” (Ottinger 2016; ). The citizen scientists of Flint, MI and St. Joseph’s, LA engaged in such activism with the expert assistance of Virginia Tech and Louisiana State University (; ), which led the states of Michigan and Louisiana to declare public health emergencies and provide millions in relief funds (; ). We also believe that we were morally and ethically obligated to call out bad expertise of authorities (e.g., MDEQ and US EPA) and agenda-driven science done to “reflect prejudices and interests–economic, political, ideological–of powerful groups” (), which led to grossly unethical and harmful actions (; ; ). In the aftermath of disease outbreaks and disasters that require multi-pronged trust-building efforts, the problem certainly cannot be fixed by simply bringing in more expertise (; Yearly 2000). But it also cannot be fixed by unrelenting and unfair attacks on expertise as documented in this paper. Finally, there is indeed “space for negotiation” () between traditional expertise and local knowledge to allow for co-creation of new knowledge with “credible” data quality that is “[fit] for use” (), without succumbing to citizen science anarchy where merited scientific expertise is disrupted with a goal of advancing populist agendas.
Citizen Science journey of Mr. Scott Smith
As we were making final revisions to this case study for publication, Mr. Smith (formerly of Water Defense) reached out to US EPA and expressed an interest in initiating:
“A private discussion with [Flint Water Study] about how his data is being misrepresented by the less-than-scrupulous activists in Flint” and supporting the official position of experts and agencies “on the progress of the Flint distribution system” (M. Durno, personal communication, June 14, 2018).
We later learned that Mr. Smith had previously sought to file a defamation case against Dr. Edwards, but was told that his best course of action would be to correct his errors publicly. He had previously written to EPA and others, threatening defamation cases against numerous EPA employees including heroic whistleblower Miguel Del Toral (). After considering the situation, we decided to work with Mr. Smith to correct the record, clear his name of several misunderstandings, and to obtain insights on his citizen science “lessons learned.”
Mr. Smith acknowledged mistakes with his testing, communication, and resident interactions, and admitted that some of his work could have been construed in a manner that changed residents’ bathing and showering habits. He noted that the many samples he took per EPA approved methods gave results consistent to those of scientists, but the WaterBug™ data were not comparable (S. Smith, personal communication, Nov 28, 2018). After writing a blog post on our website clarifying many issues (summarized in Exhibit 1), he then bravely owned up to his mistakes in the media including stating that “[Smith] says he made mistakes about dangers of bathing in Flint water” and “Ruffalo’s Former Associate Recants Flint Water Claims” (Delaney 2018, Fonger 2018). We openly applauded his candor and helped set the record straight on our website, and we amended the introduction of our prior adversarial postings accordingly (). We also acknowledged that there was no evidence that he profited from any business dealings in Flint, in spite of the many perceived conflicts of interest.
Exhibit 1: Lessons I Learned in Flint by Scott C. Smith (Abridged Version) (adapted from complete posting available at http://bit.do/scott-smith; edited and reorganized for clarity).
|
After doing what most would consider to be the “right thing” and apologizing, Mr. Smith was then viciously attacked by the same groups he was previously allied with. The Flint resident who claimed to have benefitted from the NLP filters wrote him: “How much money did it take for you to do this to us. People’s lives are more important than money. What happened to you. I’m letting the navio (sic) nation know you did this to us. No one is going to trust you again. So I hope you got a great pay check for this” (S. Smith, personal communication, Jul 11, 2018). Resident B, whose lead sinkers in plumbing were ethically and properly reported to EPA by Mr. Smith, promptly texted Mr. Smith “Et tu Brutus” (S. Smith, personal communication, Aug 14, 2018). Still another resident wondered if Smith was being “used” by Dr. Edwards (S. Smith, personal communication, Aug 14, 2018). A lawyer representing Resident A in a lawsuit wrote EPA and others stating that Mr. Smith was now among “idiots” that included “Marc Edwards, Mark Durno <EPA>, Miguel Del Toral, the EPA, Virginia Tech’s officers, employees and agents.” Smith now reflects that “coming out with the truth, was like trying to leave a cult” and that the “discovery of the lead sinker was a clear sign that citizen science had run amok” (S. Smith, personal communication, Oct 15, 2018). To his credit he wanted no part of it, paid a high price for doing the right thing, and eventually spoke to our graduate ethics class at Virginia Tech about his experiences ().
The Future
“If we overlook the ethical problems that [citizen science] raises we may end up unintentionally strengthening the lay–expert boundaries that [citizen science] was thought to overcome.” Hauke Riesch and Clive Potter ()
“On one level, this story in Flint is about water. But on another level, it’s about trust in government, feeling like your voice matters and that elected leaders care about you. Pipes are hard to fix. Those other things are even harder.” Ari Shapiro, NPR’s All Things Considered ()
“Who can land on this earth and tell a Flint resident their water is fine? It’d have to be God.” Gina Luster, Flint resident ()
The Flint Water Crisis, arguably a signature environmental injustice of the 21st Century, will be the subject of intense debate and analysis for years to come. In April 2018, the State of Michigan announced an end to distributing free bottled water to residents after water lead levels consistently registered below federal standards for four monitoring periods over two years, as supported by independent data from our own team and other researchers (; ; ; ; ; ). Nonetheless, the decision was widely portrayed as insensitive and premature by some residents who claim “chemicals and bacteria in the water are at an all-time high” and “I don’t trust the water. Period. It could be five years from now and I’ll still never drink this water” (; ). From our perspective, every positive step forward by the agencies after the declaration of the Federal Emergency was countered by a two-step pushback by citizen science abusers, perpetuating a climate of victimization and a sense of distrust that will likely last generations.
This case study reveals both the promise and perils of citizen science in relation to scientific advancement, environmental justice, and achieving a public good. One viewpoint argues that citizen science is not about producing real knowledge, but “rejigging power relations” to get (often unpaid) citizens to produce data that benefits private corporations and academic researchers, with the citizen merely “mimicking scientific practice [without learning any science]” (), and that our own work in Flint exacerbated existing power imbalances by “replicating the very structural inequalities that render environmental injustices in the first place” (, ). We maintain that our citizen science collaboration, which revealed environmental injustice and possible crimes in Flint, was an ethical and appropriate emergency effort that “empower<ed> a public to critically use the tools of science for solving some of its problems, while also resisting the hegemony of the scientific framing of <MDEQ, USEPA and> others” illuminating the sometimes beneficial “politics of public participation in science” (). Lives literally depended upon our rapid research, public dissemination of the results, and investigative science model, serving as a possible road map if such an unfortunate situation were to occur again (; Edwards and Pruden 2016; ).
This paper also demonstrates that no one is above the temptation to cheat, whether citizen or scientist, and that those who view citizens as incorruptible paragons of wisdom and virtue are at least as misguided as those who believe the same of scientists and engineers. It is important to note that Water Defense, TYT Politics, and Flint Residents A-B had no credible scientific reputation to lose when problems with their work were exposed. As a result, citizen science–in and of itself–may be even less capable of self-policing than conventional science. It is possible that the Post-Emergency Flint citizen science experience is an extreme anomaly not generalizable to normal citizen science projects. It is nonetheless important to highlight how citizen science abuse might jeopardize the public health and welfare, and the blowback directed against those who feel obligated to correct the scientific record.
As the US EPA and others embrace citizen science as a “core tenet of environmental protection” (; ), important questions emerge from this case study that deserve to be answered:
- Who can carry the banner of citizen science and for what purposes? In other words, who owns citizen science?
- How can one differentiate between good and bad expertise in citizen science? The stakes are much higher in public health crises, as are the consequences of citizen science abuse that may cause real harm.
- Who polices citizen science? Do experts and other citizens have an obligation to expose citizen science abuse and call out unethical actors, especially in environmental and public health emergencies? What protections or support, if any, can be provided to those who honestly report and call out abuse?
The events described above, in combination with deeply rooted societal problems in post-industrial Flint, created a state of social and science anarchy that will have long-term repercussions for governance, regulation, environmental ethics, and citizen science.