Selection of drains

As part of EPA’s Beach Report program (epa.vic.gov.au), short- and long-term microbial water quality results indicated that enterococci levels could be impacted during dry weather at some of the 36 Port Philip Bay sites monitored annually. Ten of these beach sites were selected for Drain Detectives monitoring (Figure 1).

In 2018–19, sixteen drains at five beaches were monitored. One drain per beach had signage installed, while citizen scientists monitored between two to four drains at each beach. In 2019–20, ten drains at ten beaches were monitored, with sensors installed at all beaches and signage and citizen scientists continuing to monitor the five beaches from year one.

The number of drains monitored per beach by trained citizen scientists was reduced in year two from monitoring between two to four drains per beach to just one per beach (i.e., five drains across five beaches). This was based on a review of year-one monitoring, which identified that there were many drains not receiving enough citizen science reports to determine frequency and pattern of drain flows. By using the existing pool of citizen scientists to monitor fewer drains, it was envisioned that more meaningful data would be collected at these drains to better determine frequency and patterns of dry-weather flows.

Signs at stormwater drains for community monitoring

The community could participate in Drain Detectives by using the drain monitoring instructional signs installed at five of the ten beaches in the project during the summer monitoring periods (year one: December 2018 through May 2019, and year two: December 2019 through March 2020). Signs for three beaches were designed by Life Saving Victoria. Different designs were tested and evaluated with a community user group to ensure they were fit for purpose. A council partner designed a sign for their two beaches in the project; this was preferred by the council to keep consistency with the design of their other signage on the foreshore.

At each of the five beaches, three signs were installed near a drain being targeted (i.e., fifteen signs in total). Signs were located at foreshore carparks, pathways close to the drain, and where possible, at the drain. Media articles, Tweets, and council newsletters were used for promoting the signs and to encourage community pollution reporting. All signs included a title, logos, a brief description of the project, and simple instructions for making a report using a smartphone, with reference to a data-collection platform for more information on how to locate the drain and what observations to make.

Trained citizen scientist monitoring at stormwater drains

Media articles, Twitter, and council newsletters were used for recruitment. The level of interest exceeded available places for citizen scientists in the Drain Detectives training program. Although the project had established a cap of 50 citizen scientists, 172 community members registered interest in participating. The response to the call for volunteers in year one highlighted the interest from the community to get involved in citizen science relating to beaches and water quality.

In total, the first 57 community members to register interest and 25 council officers were trained in drain monitoring. The number of trained volunteers could have been greater if resources permitted. People who registered Drain Detectives interest after the project cap was met were referred to the project’s drain signs as another way they could get involved in the drain monitoring.

The 82 citizen scientists were provided with training to make drain flow observations and to conduct water quality monitoring. The water quality sampling involved the use of a simple test kit to analyse ammonia levels in drain water (API ammonia test kit). Ammonia was used as a general indicator of faecal pollution after successfully being used in previous drain flow investigations where it was sensitive enough to detect faecal pollution in high concentrations (e.g., sewer spill or significant leak) (Amos and Sharley 2016). The test was an easy addition to collecting visual observations, as the sample was easy and relatively safe to collect, and a result could be obtained onsite.

Group training of citizen scientists was held at each of the five beaches targeted. In the second year of the project, no additional citizen scientists were recruited because resource limitations forced a cap on the number that the project could support. Refresher training was provided for citizen scientists through a training video.

Trained citizen scientists were provided with a standard operating procedure with drain location maps and location codes (codes used in the data-collection platform to identify specific drains). The procedure also had instructions for using the form in the data-collection platform, for making observations, on occupational health and safety requirements, on taking informative field photos, and on collecting and analysing the ammonia test-kit samples in a safe manner.

In the first year of the project (2018–19), trained citizen scientists were asked to monitor drains on days and times that suited them. During the later stages of this monitoring period, better coordination of citizen science monitoring was trialled. Citizen scientists were asked to fill out a monitoring schedule of days and times, indicating their intentions to monitor. The purpose of the schedule was to avoid overlap of monitoring between citizen scientists and to obtain greater consistency in monitoring across days and times. In the second year of the project, more direction on when to monitor was provided to the citizen scientists from the start of the monitoring period (December 2019–Mar 2020). Volunteers were emailed reminder requests to monitor drains when a week of dry weather was forecast, to target their monitoring efforts on periods that could provide the most useful data.

Smartphone monitoring of drains

Smartphones offered an ideal low-cost and readily available method for collecting data on whether drains were flowing. Smartphones allowed photos, written observations, and water quality data to be logged online by community reporters at drain signs and by the trained citizen scientists. When coupled with sensor monitoring, this provided an avenue to identify which drains were exhibiting regular dry-weather flows.

Sensor monitoring

Depth, conductivity, and temperature sensors were used to determine the frequency and patterns of dry-weather flows at 10 beach drains. The depth sensor was used to detect elevated flows and provide information on flow size and frequency. Conductivity and temperature provided information on the water quality associated with flows (i.e., the possible presence of sewage). Sensors were tested and developed by Monash University for drain environments in year one of the project. Sensors were then installed at 10 beaches in the second year of the project, during summer.

Signs at drains for community monitoring

Signs enabled 49 community members, across the two years, to become involved in the project who otherwise may not have participated. Most community members who participated made a single report (e.g., photo and observations); repeat use of the signs by community members was uncommon.

Reports from signs made a valuable contribution to the total number of reports received. Reports from community members across the five drain signs made up 15% and 16% of total reports received in year one and two, respectively. The number of reports from signs varied considerably between the five beaches. The number of reports were noticeably higher at drains where signs were close to the drain, so photos and observations could be taken at the sign, and the beach was likely popular with foreshore or beach walkers.

Trained citizen scientists

Citizen scientists made 341 reports over the project duration. Data on drain flows was collected across sixteen drains at five beaches in 2018–19, and five drains at five beaches in 2019–20. The number of reports across drains was variable, with some drains getting a considerably higher number of reports than others. A good example of this was in 2018–19, in which four drains received 44% of the reports over a six-month monitoring period (Figure 2). Not enough data was collected to determine definitive frequency of elevated flows, but the data was useful for indicating if there were continuous background flows, especially when combined with year-two data.

There was a decrease in the number of reports made between year one and two of the project (Figure 3). The average number of monthly reports decreased from 46 in 2018–19 to 15 in 2019–20. This decrease in reporting was consistent across sites and trained citizen scientists.

Most of the data was collected by a small proportion of trained citizen scientists. In year one, 36% of the citizen scientists who made reports were responsible for 82% of reports received. In year two, 44% of the citizen scientists who made reports were responsible for 71% of reports. Most trained citizen scientists either made no or only one report.

Citizen science data enhanced sensor monitoring of beach drains

The project fostered innovative science by using a combination of citizen science and sensors to monitor drains and to seek broad community participation in the project. No similar approach or project for water quality monitoring was evident in the literature.

The project demonstrated that trained citizen scientists with smartphones and a data-collection platform could regularly collect a range of data to support monitoring drains for water quality. The citizen science data enhanced sensor monitoring of drain flows, providing data on water colour, size of flow at outlet, ammonia levels, and whether flows were entering the Bay—data that could not be provided by sensors. Citizen science data was also good for indicating which drains had continuous background flows, whereas sensors were good for detecting and determining the extent of dry-weather pulses coming down drains.

Citizen science complemented the use of sensors. When data was combined from both sources over the two monitoring periods, a more complete set of data was available. For the five beaches with sensors and citizen science monitoring, the combined data was used to develop a risk profile for drains. The highest-risk drains were categorised as having continuous background flows or frequent pulses, with the dry-weather flows most likely entering the Bay. Continuous background flows were considered higher risk as they were most likely to occur during times people were swimming. Timing and size of pulses were also considered, with pulses occurring during daylight hours considered higher risk for people swimming. For the five beaches with sensors only, less data was available. The risk profiling was used to identify two drains that were potentially higher risk for swimmers. Further investigation with additional water quality sampling in the catchment of these drains identified the locations of previously unknown sources of dry-weather contamination sources. This was used by local authorities for targeted follow-up.

Citizen science addressed data gaps that came with using sensors in drains on their own, increasing the temporal and spatial scale of data collected. This finding was consistent with a variety of environmental monitoring in the literature (Lowry and Fienen 2013; Capdevila et al. 2020; Quinlivan, Chapman, and Sullivan 2020). Application of citizen science in Drain Detectives also enhanced what could be monitored, with citizen scientists collecting types of data that could not be provided by the sensors. Expansion of the type of data that can be collected as a result of citizen science has been demonstrated in water quality and in hydrological monitoring (Safford and Peters 2017; Starkey et al. 2017; Hegarty et al. 2021), as well as in monitoring of the Great Barrier Reef (Becken et al. 2019), with an emphasis on the value of combining citizen science and professionally collected data to better characterise the environment.

In-person engagement and regular communication is needed to maximise data collection

The use of trained citizen scientists was successful at gathering a large amount of data. Most of the data was collected through the trained citizen scientists, complemented by data from the broader community using signs. However, during the evaluation of the project, it was recognised that more data could have been collected had more action been taken to reduce the variability in data collection between drain sites, years, and citizen scientists.

The variability in reporting between drains and citizen scientists (Sauermann and Franzoni 2015; Scott and Frost 2017; Weeser et al. 2018; De Moor, Rijpma, and Prats Lopez 2019) and a drop off in number of reports during the life of monitoring (Rotman et al. 2014; Sauermann and Franzoni 2015) is common in citizen science. For Drain Detectives, this may have been due to changes in personal circumstances of volunteers, to not enough in-person engagement and communication to maintain interest or motivation, to the number and location of drains, or to unfavourable weather.

Regular and diverse engagement is important for motivating and retaining citizen scientists (De Moor, Rijpma, and Prats Lopez 2019). Not enough resourcing was built into the Drain Detectives project to engage more actively in person with trained citizen scientists and to provide more regular communication and feedback. The EPA Citizen Science Program was in its early stages when this project was scoped, and the time needed and importance of in-person engagement and communications was underestimated.

The only in-person contact with citizen scientists throughout the project was through the training sessions in year one. After that, communication was mainly through emails and the Yarra and Bay website (which no longer exists). Gathering citizen scientists together yearly to discuss data quality, to get feedback on their experiences, and to discuss findings and next steps may have helped to maintain the motivation and participation of citizen scientists; however; this was not undertaken because of resource constraints. An online survey was conducted to see what engagement the citizen scientists preferred, but only a small number of trained citizen scientists responded, and there was a mixed response on whether further engagement was wanted. The location and spread of the beaches across Port Phillip Bay made gathering all citizen scientists for a single in-person engagement activity challenging, and having single events per beach would have required a large amount of resourcing. Inviting citizen scientists to nominate their preferred engagement method(s) could have been built into the training sessions at the start of the project to make sure that feedback from most, if not all, volunteers would have been received. Greater resourcing for engagement and communications may have motivated and retained citizen scientists to continue to make reports, particularly in year two of the project. It would have also allowed the recruitment of more citizen scientists if needed.

Numerous studies and literature reviews have focussed on the importance of a range of factors for success in citizen science monitoring, with motivation and retention of citizen scientists through the life of monitoring featuring prominently (Alender 2016; Baruch et al. 2016; Compas and Wade 2018; Capdevila et al. 2020). An organisation’s communications and reporting back to citizen scientists, as well as in-person engagement and associated relationships, add to the experience of citizen scientists and their sense of making a valuable contribution (Kim et al. 2011; Alender 2016; Storey et al. 2016; Shupe 2017; Capdevila et al. 2020). This leads to citizen scientists being more likely to collect more data and stay engaged in the monitoring, resulting in a better outcome for citizen scientists and the project or monitoring program. Citizen science engagement and retention in a project can increase again if they are engaged in project co-design and co-interpretation (Shirk et al. 2012; Hidalgo et al. 2021).

Factoring the resources for in-person engagement and communications during the design phase of a project is important for getting the maximum amount of data and best outcomes for everyone involved.

Signs are valuable for data collection, but a range of factors need to be considered to maximise their use

The project demonstrated that data could be collected by untrained, local community members following directions on signs posted at drains. However, the value of this method for collecting a lot of data was limited by the low participation rate. This could be a limitation if signs are the sole method for data collection for a project. Low community engagement with signs placed at beaches and foreshore areas is not uncommon (Matthews, Andronaco, and Adams 2014; Morgan, Smith, and Matthews 2014; Rangsivek et al. 2019). Surveys (Morgan, Smith, and Matthews 2014) and anecdotal evidence within EPA regarding water quality signage indicate low awareness of signs and low transmission of important water quality information to the public, particularly in the early days of new signage. As Drain Detectives signs asked the community to participate in reporting drain flows causing pollution (i.e., a call for action), there was value in trialling whether the community would have a higher uptake for this type of sign versus the water quality signs mentioned above.

There was a low number of repeat users of the signs and highly variable use across sign locations. Variability may have been due to beach popularity and to the proximity of signs to the drain (with signs closer to drains getting higher use). These patterns of use were also found in hydrological monitoring using citizen science and signage, such as CrowdHydrology (Lowry et al. 2019).

There is a range of variables that may contribute to low sign use, such as beach use, sign location, other signs on the foreshore, sign design, and sign promotion. High-use coastal foreshores can contain a large number of bylaws, warnings, or amenity signs that look similar in design. An additional sign added to the foreshore may not attract much attention. Sign promotion was primarily through social media platforms such as Twitter and Facebook. Marketing was not available to boost awareness of the signs within the community. Promotion and marketing of citizen science projects is not an uncommon constraint within government-led citizen science projects (Roger, Turak, and Tegart 2019).

Another reason for the low uptake of the signs may have been the text-heavy design. Early in the project when signs were being designed, the team’s preference was to have a satellite aerial map of drain location and a photo of the drain, with brief instructions on the sign. This is a similar design to that used by CoastSnap in New South Wales, Australia, which monitors beach erosion (Harley et al. 2019). Because there was a risk that maps and photos would not print clearly on signs, the text-heavy design was used for Drain Detectives. In addition, the signs required participants to use a shortened URL to access a data-collection platform to get instructions, maps, and photos to locate the nearby drain and instructions for how to make reports. A QR code was considered instead, but internal advice was that they were not commonly used, although QR codes may be more commonly used now because of their use during the COVID-19 pandemic. A map, photo, and QR code on the sign may have attracted more attention or made the signs more engaging than text-only signs, but there is no way of knowing if this would have resulted in greater sign uptake for this project.

Signs may also be more successful depending on the type of project and the location of signs. For example, signs may be better suited to projects asking the community for simple, quick-to-collect data such as a single photo while standing at a sign, such as CoastSnap (Harley et al. 2019) or CrowdHydrology (Lowry et al. 2019). Drain Detectives requested a lot more data from sign users than these projects, including a photo and a range of observations. In addition, signs for CoastSnap and CrowdHydrology were located at the point where photos had to be taken. Drain Detectives signs were not always located directly at the drains as this was not always possible in a beach environment. This meant that some signs required the community to locate and walk over to a drain, take a photo, and fill out a field form for observations. This level of effort to make a report may have contributed to the low number of reports received from some signs.

The experience of using signs for Drain Detectives highlighted that there are many factors to consider before implementing this method for collecting data from the community. Ensuring a low effort for each report may increase the likelihood that the community will opportunistically take a photo or make an observation at a sign.