Citizen science outcomes

The outcomes of CS projects have been categorized by Shirk et al. (2012) into outcomes for research (e.g., scientific findings), outcomes for the system (e.g., influencing policies), and outcomes for individual participants. The latter outcomes, among others, may relate to the development of new skills, to increasing scientific literacy, to empowerment, and to changing attitudes and behavior (Land-Zandstra, Agnello, and Gültekin 2021; Haywood 2016; Stepenuck and Green 2015; Brouwer and Hessels 2019).

While scientific benefits of CS to researchers are well reported, system and individual outcomes, including behavior change, are frequently overlooked (Jakositz et al. 2020; Jordan et al. 2011). Indeed, based on their extensive literature review, Walker, Smigaj, and Tani (2021) conclude that about one quarter of all water CS papers reviewed had no mention of participant or community benefits, whereas about a third only list benefits without explanation or investigation. The numbers of CS studies that report behavior change is even more limited, with a few notable exceptions such as those that observe enhanced water conservation efforts (Egerer, Lin, and Philpott 2018; Harriden 2013), and those that observe the purchase of rain barrels and eco-friendly cleaning products (Church et al. 2019).

To address this gap, this paper explores how perceptions and behavior change after a particular intervention in the provision of drinking water, i.e., the installation of a central softener, and how, if at all, the impact of this intervention differs between water customers directly involved in a connected CS project, i.e., the group of citizen scientists, and those not involved in this project, i.e., the control group. As further explained below, in this project, CS participants conducted hardness and calcium precipitation measurements, both before and after the installation of the central softener. Although (i) a standard, and therefore identical, drinking water company letter was sent to both groups informing them about the softening intervention, and more importantly, (ii) both groups received tap water of an identical quality both before and after the installation of the central softener, we nevertheless expected that drinking water customers who had the opportunity to personally verify and observe these changes through accurate CS measurements would be more likely to react profoundly to those changes, compared with customers passively experiencing this change. Accordingly, and in line with previous work that found that CS can lead to increased awareness and behavior change (Brouwer and Hessels 2019; Church et al. 2019), our central hypothesis was that after the softening intervention, CS participants would show more profound changes in their quality perceptions and behavior.

Recruitment

In this project, following the classification of Brouwer and Hessels (2019), a combination of a targeted recruitment strategy and a generic recruitment strategy was used. First, targeted e-mail invitations were sent to the pool of respondents (n = 312) that: (i) participated in the original pre-survey; (ii) indicated no objection to receiving a related follow-up survey: and (iii) resided within the pilot study area. In total, 62 citizens replied with a positive response to this invitation, giving a 19.9% response rate. Additionally, targeted e-mail invitations were sent to a random sample (n = 1000) of households within the pilot study area. For this group, the positive response rate was merely 2.4%, equaling 24 people. In addition to this targeted invitation strategy, participants were recruited via Facebook-targeted advertising. Within a timespan of two weeks, 66 citizens registered via this Facebook campaign, amounting to a total of 152 registrations.

As depicted in Table 2, about half of the registered volunteering citizen scientists were younger than 45 (48%); the age group with the most respondents was 35–44 (28%). The majority of respondents were male (69% versus 31%). This figure may partly relate to the fact that the e-mails (target invitation strategy) were sent to the drinking water company’s contact persons, that is, persons in charge of paying the bill, which may be more often men. This, however, cannot be the full explanation since men, although less significantly, are also overrepresented in the Facebook campaign, where such bias doesn’t play a role. The majority of the respondents were well educated: 41% with a bachelor’s degree and 30% with a graduate degree as highest completed education. However, there is diversity also in this category; 26% had a secondary education, and a small percentage (3%) had primary school as highest completed educational level.

From a modern segmentation perspective, and in comparison with the 2018 baseline study, we found an overrepresentation of participants with the “aware and committed” perspective (38% versus 22%) and an underrepresentation of both the “down-to-earth and confident” (11% versus 20%) and the “egalitarian and solidary” perspective (28% versus 37%).

Citizen scientists’ activities

After the recruitment phase, including filling in the pre-survey, all registered citizen scientists were invited to attend one of the two “kick-off” meetings, which introduced the project and provided the opportunity to get acquainted with each other and to hand out the water test kit. The water test kit contained a drop test for the determination of hardness (Visocolor HE, Total Hardness H 20 F, Macherey-Nagel), a plastic measuring jug, a cooking pot, two pipettes, a tight plunger syringe, and step-by-step instructions.

During the course of the project, participants were invited to take measurements within two different 14-day measurement periods: before and after the installation of the central softener. The time interval between the two phases was planned to be 4 months. During the project realization, however, the COVID-19 pandemic, along with delays in the construction of the new system, led to the second measurement being delayed. In effect, the first measurement period, which started about 10 days after the kick-off meeting, took place in November 2019, whereas the second measurement took place in October 2020. Using a drop test and a simplified boiling-water test, the participants determined the hardness and lime deposition of their water. The drop test is based on a principle of titration by drop counting, and was explained in a step-by-step manner in both an instruction manual and a video. In short, people were asked to fill the test tube and to add 2 drops indicator solution, after which the test sample turns red. Next, people were asked to add titration solution dropwise while smoothly shaking the test tube until the solution turns completely green. At that point, people were asked to read off the total hardness indication (in mol/m³) from the syringe barrel. In each round, the drop test was carried out twice; once on water directly from the tap and once on water that had been boiled for 5 minutes. To this end, people were asked to measure 5ml of water into their distributed measuring jug and boil this water in the distributed pot without a lid. After boiling, people were asked to cool the water for at least 20 minutes, and again measure the remaining water in the jug. In addition to the second drop test, people were asked to evaluate the quantity of lime deposition in the test pot qualitatively by means of visual assessment. After each measurement round, the citizen scientists were asked to enter their measurement data in an online survey. Figure 2 visualizes some of the activities of the citizen scientists.

Acknowledging the importance of effective communication and feedback in CS (Capdevila et al. 2020), intermediate results were shared via the project website, along with two separate e-mails and a video explaining the analysis, quality control, and interpretation of the measurement results. In addition, CS participants were kept informed about the progress of the project, the reasons for the delay, and subsequent steps. After the analysis of second-round data, the results were shared via a webinar, giving ample opportunities for questions and feedback.

Citizen science water hardness measurements

In the first measurement round, the total number of participants that shared their data was 130 out of 152 registered participants, representing a participation rate of 86%. In the second round, 98 participants shared their data, representing a 65% participation rate. In line with the majority of CS studies in the water sciences, to ensure the production of scientifically valid data (Njue et al. 2019), all data points following the CS measurements were subjected to quality control to identify erroneous entries. The following data points were marked as erroneous: (i) registrations with improbably high boiling hardness (measured values of hardness larger than 50°fH in the first round, and 70°fH in the second round); (ii) values that were reported by citizen scientists themselves as measured “out of protocol”; and (iii) entries with a zero value, before or after boiling. In the first round, 2.3% of the observations flagged as erroneous required removal; in the second round, this was 7.1%. With a mean error rate for both phases being 4.7%, this result is considered on par with the error rate seen in sensor measurements for water, which typically assumes measurement confidence intervals of 95% (Da Silva 2013; EURAMET 2011). The quality of the CS dataset was further validated by a comparison with formal hardness measurements taken by the utility. As further detailed in Supplemental File 2: Data Validation, the validation concludes that the CS measurements have an average error (deviation) rate of 3.56% when compared with the utility measurements, which is considered on par with confidence intervals typically seen in preliminary sensor measurements (Aceves-Bueno et al. 2017; Sharma et al., 2014) and other studies suggesting that citizen scientists generate high-quality data comparable to professional data (e.g. Storey et al. 2016; Loperfido et al. 2010).

The first thing that becomes apparent from the CS measurements is the effect of the central softener. Figure 3 shows that the average measured hardness of water before boiling dropped from 43.7°fH in the first round (which, as listed in Table 1, is classified as hard to very hard) to 20.6°fH in the second round (classified as average hard), and from 17.5°fH to 11.4°fH following the boiling process. It should be noted these after-boiling data are provided after a volume correction for evaporation. This correction is needed because the density of minerals (Ca, Mg) within boiled water is higher due to evaporation, and entails a recalculation of the decreased volume so as to scale with the original volume of 5ml.

In this study, CS participants not only measured for the hardness of water, but also looked explicitly at the calcium precipitation (PCCP). From the large difference of hardness before and after boiling in the first round as depicted in Figure 3, one can deduce a rather high PCCP, whereas in the second round this difference was significantly smaller. As shown in Figure 4, the average PCCP indeed drops from 2.61 mol/m³ (SD 0.41 mol/ m³) in the first round to 0.91 mol/ m³ (SD 0.40 mol/ m³) in the second round. Accordingly, citizen scientists’ measurements indicate a PCCP drop after the installation of the central softeners of 1.69 mol/ m³.

In addition to this quantitative PCCP assessment, by using three sample pictures, participants were asked to qualitatively classify the amount of calcium precipitation in their own cooking pot as “a lot,” “limited,” or “none” following the boiling test. In accordance with the quantitative data, this classification methodology revealed a significant change. Whereas in the first round, a large majority (88.5%) classified the inside of their cooking pot as having a lot of calcium precipitation, in the second round, this percentage dropped to 8.2%. Concurrently, the percentage of participants classifying their pot as having a limited calcium precipitation increased from 11.5% to 86.6%.

Perception

With regards to the perceived hardness of water, we observe differences between the pre and post datasets (see Figure 5 for a visual comparison), that are strongly significant for both the CS and control group (p < 0.001 in both cases for the Wilcoxon-Mann-Whitney test). For instance, whereas in the pre-phase, 55% of the CS participants and 69% of the control group perceived their water to be very hard, in the post-phase this was reduced to respectively 0% and 6%. Likewise, we find that the percentage of respondents who regard their water as soft grew from 1% and 0% to 39% and 26%. The shift in perception appears to be stronger among the CS participants, both in the pre- (p = 0.043, Wilcoxon-Mann-Whitney test) and the post-phase (p = 0.018, Wilcoxon-Mann-Whitney test).

It is noteworthy that after the installation of the central softener, customers not only perceive their water as softer, but also as more healthy, and to a lesser degree, more tasteful (p < 0.001 between the pre- and post-phases in both datasets, Wilcoxon-Mann-Whitney test). For instance, the percentage of CS participants perceiving their water to be healthy grows from 58% to 86% and perceiving their water as having a good taste from 52% to 79%. These differences are summarized in Table 3. Interestingly, the CS participants seem to magnify these perception differences even more against the control datasets (p < 0.05 during the pre-phase, p < 0.1 during the post-phase). The lower statistical significance in this magnification can be attributed to the small number of CS participants, particularly in the post-phase. Accordingly, we observe strong reductions in the concerns about the quality of tap water between the two phases (p < 0.001 in both the control and CS group comparisons, Wilcoxon-Mann-Whitney test), which are magnified more between the CS and control group in the post-phase (p < 0.001, Wilcoxon-Mann-Whitney test) but interestingly not in the pre-phase (p > .1, Wilcoxon-Mann-Whitney test). We also observe growing concerns for rising prices (p < 0.001 for both groups, accordingly, Wilcoxon-Mann-Whitney test), which are also more profound in the CS participants compared with the control group, especially in the post-phase (p = 0.057 and p = 0.002 when comparing pre and post phases, accordingly, Wilcoxon-Mann-Whitney). This growing concern, however, cannot simply be attributed to the softener, as the study period coincided with the circumstances associated with the COVID-19 pandemic that led to major concerns about job security and concerns about how substantially more time indoors translates into higher utility bills.

The same pattern can be seen when looking at the calcium precipitation inconvenience that customers experience. Table 4 shows strong differences between the pre- and post-phase across all datasets (p < 0.001 for both warm and cold water, for both groups, Wilcoxon-Mann-Whitney test). Again, a stronger shift in perception among the CS participants is observed, especially in relation to the use of cold water (p = 0.02–0.05 and p < 0.001 when comparing control and CS groups for both phases and for warm and cold water respectively, Wilcoxon-Mann-Whitney test).

Perceived hardness vis-à-vis quality

As mentioned, a strong correlation between the perception of tap water quality and the perception of hardness was observed in the 2018 baseline study. We also observe this in our study for both the control group and for CS participants, as seen in Figure 6, with the vertical lines depicting the 95% confidence intervals of scoring. Interestingly, CS participants tended to give higher water quality scores regardless of their hardness perception, and this effect became more profound in the post-phase.

Figure 6 

Correlation between the perceived hardness and the quality of water.

Behavior

To elucidate to what extent, if at all, CS participation after the introduction of the softener has resulted in behavior change, we analyzed both the installation of individual water softener systems and the use of calcium-removal products. With the former, we observed a limited reduction. In the spring of 2019, 13% of the CS participants and 26% of customers within the control group had a softener installed in their household. In December 2020, respectively, 11% and 13% disconnected this softener. Furthermore, we found that a large majority had adjusted their personal softener (33% versus 26%) or were planning to do so (56% versus 40%).

Concerning the use of calcium removal products, we examined the qualitative responses related to the frequency of use on a series of household appliances, and thereupon estimated the number of uses per year f_i for each category i. On the basis of these assumptions, we then calculated the mean frequency (weighted against the number of responses) ${\overline{f}}_i$M1 \documentclass[10pt]{article} \usepackage{wasysym} \usepackage[substack]{amsmath} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage[mathscr]{eucal} \usepackage{mathrsfs} \usepackage{pmc} \usepackage[Euler]{upgreek} \pagestyle{empty} \oddsidemargin -1.0in \begin{document} \[ {\bar f_i} \] \end{document} , which represents the average uses of calcium removal products per person per year for a particular appliance.

As shown in Table 6, there is a clear frequency reduction of calcium-removal products among all devices following the installation of the softener in both groups (control and CS). This reduction is, however, significantly more pronounced among CS participants.