Brief history of NLNature.com

In 2009, we founded a citizen science website, NLNature.com, focused specifically on the flora and fauna of the province of Newfoundland and Labrador, Canada (hereafter NL). The initial purpose of NLNature was to examine ways to harness citizen science participation in online data collection (Lukyanenko, Parsons, and Wiersma 2011; Parsons et al. 2011). The province was chosen as a test case as part of a larger Canada-wide study (Tudge et al. 2012); its vast size (405,212 km²) but sparse population (~500,000 people) provided an interesting venue in which to investigate issues of participation, digital divides, and motivations. Approximately half of the population lives outside of the greater capital region, in small, rural (mostly coastal) towns. Many citizens hunt, fish, and forage to supplement their diets. Furthermore, the province of NL is a popular tourist destination; the Department of Tourism estimates 180,000 people visit per year (Government of Newfoundland and Labrador 2021). One attraction for visitors is the unique natural beauty of the province. Thus, while there is a small population of potential citizen scientists, a large proportion of these people spend a great deal of time outdoors and are keen observers of their surroundings. The site experienced its peak activity between 2011 and 2015. We took the site down in early 2022, owing to a lack of resources to keep it consistent with current web standards.

Data quality dimensions

The main purpose of NLNature.com was to serve as a vehicle to explore how to design data collection for an effective citizen science project. Although the founder is an ecologist, the intent of the site was never for rigorous data collection for use in ecological studies. As such, the data are “use agnostic” (Lukyanenko, Parsons, and Wiersma 2014). Instead of trying to collect data for a specific use, we designed the site to investigate whether and how we could motivate citizen science participation across a broad geographic region, and specifically, to conduct experiments to test questions about data quality, usability and engagement.

Data quality is one of the primary concerns with citizen science data (Baker et al. 2021), and has multiple dimensions (Wand and Wang 1996; Mesaglio et al. 2023). The common data quality dimensions include accuracy, precision, completeness, and timeliness. Our theoretical framework borrows from the field of information systems (IS), particularly platforms that rely on user-generated content (UGC), which can include product reviews, discussion boards, and citizen science. We focused specifically on the data quality dimensions of accuracy and completeness. Generally, we understood accuracy as the congruence between the labels and descriptions provided by the citizens and the standards for identification that represent state-of-the-art scientific knowledge in biology. Completeness was assessed as the depth and breadth of the descriptions of objects observed, as well as the fraction of objects captured in the database compared with the observations contributors came in contact with (the latter was naturally difficult to quantify, hence we employed estimations and used laboratory experiments with the live NLNature interface to simulate real-world sightings). For example, in a preliminary study, we asked non-biology students to identify images of plants and animals found in the province and to describe their attributes. We found that participants were able to accurately identify the images but only at class levels higher than species (e.g., they could identify something as a bird, but not what species of bird). Detailed results of the preliminary study are reported in Lukyanenko et al. (2014). We used this to inform design choices and rationale in the design of the online experiment (Lukyanenko, Parsons, and Wiersma 2016; Lukyanenko et al. 2017; Lukyanenko and Parsons 2020a; Lukyanenko and Parsons 2020b), which is summarized briefly below. The bulk of the work leading up to this writing made contributions to theories of software development, design science research, conceptual modeling, and data management.

Unanticipated contributions

We aim here to investigate what ecological value we realized from the site beyond the contributions to the IS field, as detailed elsewhere (Lukyanenko et al. 2014; Lukyanenko et al. 2016; Lukyanenko et al. 2019). Because we did not design the site with an ecological research question in mind, and we made no attempts to facilitate standardized data collection, the site represents a type of “surveillance monitoring” as opposed to “targeted monitoring” (Wiersma 2010; Wintle, Runge, and Bekessy 2010; Callaghan et al. 2018). Nonetheless, the site yielded some valuable ecological data, which are detailed below. We use these observations to discuss the pros and cons of use-agnostic online citizen science projects.

Beyond the generation of data, citizen science projects can bring intangible benefits to participants. This can include gaining knowledge (e.g., Crall et al. 2012), feeling connected to a community (e.g., Evans et al. 2005; Overdevest et al. 2004), and contributing to a collective challenge (McKinley et al. 2017). Lynch-O’Brien et al. (2021) discuss a theory for transference of impacts and describe five stages through which citizen scientists themselves transfer the impacts of their participation. Generally, individuals who can have impacts beyond a given citizen science project begin with a long-term interest in nature (stage 1), then join a citizen science project (stage 2). Over time, their participation in citizen science causes them to be attributed with expertise by peers (stage 3); they eventually acquire the role of an expert (stage 4) and may in turn influence change in others (stage 5) (Lynch-O’Brien et al. 2021). Through a summary of interviews and focus groups we conducted with NLNature participants, and reflections of the authors, we examine whether and how participants in the NLNature project realized some of these intangible benefits.

Website and experimental design

We designed NLNature.com to test how data quality (specifically accuracy and completeness) varied with different data entry constraints on user contribution. Participants contributed a sighting by clicking a point on a map that corresponded with the position where their sighting was made; this then opened up an interface where they were asked to contribute the date of the sighting and additional sighting information. During initial account set-up, participants were randomly assigned to one of two treatments. The online interface for each treatment was nearly identical; the single difference was whether users were constrained to a limited list of species names when identifying their sighting (the class-based interface), or if the field for entering their observation was open ended (the instance-based interface). In the class-based interface, users had to classify their observation based on a pre-defined list of 343 species (contributed by the first author and updated based on contributions from contributors) known to be extant in the province, or click “unknown.” In the instance-based interface, users could describe their sighting any way they wished. We describe the interface design in more detail in Lukyanenko and Parsons (2020a).

We promoted awareness of the site through extensive public outreach and media appearances between 2009 and 2010. We also further promoted the site via speaking tour across the province in 2015 2012, which targeted tourism businesses and interest groups (e.g., photography clubs, outdoors clubs, and naturalist groups).

To assess accuracy, we carried out a laboratory experiment similar to our initial studies, where we showed images of species from the province on the screen. Instead of pencil-and-paper responses, these users had to create an account on NLNature, and identify the species shown on the screen as best as possible using the online interface. The participants in the study were all non-biology majors. On a 7-point Likert scale (1 = strongly disagree; 7 = strongly agree), participants in the experiments gave themselves a mean rank of 2.2 (s.d. = 1.36) in response to the statement “I consider myself an expert in local (NL) flora, fauna and wildlife,” and 4.0 (s.d. = 1.77) in response to the statement “I am familiar with Newfoundland and Labrador’s plants and animals.” Additional information about the design of this experiment is detailed in Lukyanenko et al. (2019).

Data analysis

We analyzed the number of contributions made by participants by summarizing all observations contributed in each treatment (class-based and instance-based interfaces) after removing any non-species observations (e.g., “iceberg”) over a six-month period beginning May 1, 2013. We tested for significant difference in number of observations and number of novel contributions (i.e., unanticipated species that were not in the initial drop-down list of species expected to be present in the province) in the instance- versus class-based conditions using an exact permutation test. We assessed accuracy in the classroom experiment using MANCOVA to test for experimental differences between experimental groupings (degree of familiarity of species) and conditions (instance- versus class-based). See Lukyanenko et al. (2019) for details.

Focus groups

In 2015, we conducted individual interviews (n = 8) and focus group discussions (n = 2) with contributors to NLNature. We ran one focus group (5 participants) in the same city as the university campus, and one (2 participants) in a small town 2 hours outside the city. Of the 15 total participants, 5 identified as female and 10 as male. When asked about profession/occupation, 8 reported that they worked outside of biology/ecology; 5 said they worked in biology/ecology, and 3 did not provide information on occupation. The purpose of this was to conduct a qualitative analysis of responses to determine contributors’ motivations for participation, their perceptions of data quality on the site, and the applications to which they thought the data on NLNature could be used. A copy of the interview instrument is provided in the Supplemental File 1: Appendix A, and further details on interview methods and interviewee profiles can be found in Lukyanenko et al. (2017). Interviews were recorded and transcribed, and we qualitatively analyzed the text of the responses to assess for patterns as well as differences in responses.

Data quality

During the six-month online experiment on data quality (May–October 2013), there were 42 participants in the class-based treatment and 39 in the instance-based treatment. Our analysis of data quality showed that when forced to choose a species from a dropdown list, there were significantly fewer observations (87 in the class-based and 390 in the instance-based, p = 0.033) and fewer unanticipated observations (7 in the class-based and 119 in the instance-based, p = 0.007) (Figure 2). In at least six instances, users abandoned a post when they system did not accept their initial entry (analyzed via data logs). This suggests that contributors in the instance-based case might have been less confident in their species identification and defaulted to unknown or simply did not contribute rather than provide a guess that might be erroneous. However, we cannot verify this across all sightings, as not all contributors appended a photograph, and even if they did, not all species can be identified from a photograph. Thus, we rely on the lab experiment to assess the data quality dimension of accuracy and compare this between instance- and class-based data collection methods.

In the lab experiment, for which we knew the correct identity of the species, there was a significant difference in accuracy; the instance-based condition had higher accuracy than the class-based one (F = 156.6, p < 0.001), but significantly lower precision (F = 53.9, p < 0.001). This is because in the instance-based condition participants could identify at the taxonomic level they felt confident (thus we coded responses such as “gull” as correct for the image of the herring gull), but in the class-based condition, participants had to select from a drop-down species list that included many gulls (and other birds), forcing them to guess if they were not confident of the species identity. Further details on these experiments are provided in Lukyanenko et al. (2019).

Despite the limitations, the site did yield several valuable ecological observations. These include a record of a new species to the province (Fielden et al. 2015); sightings that expand the known range of two species in the province, including a species ranked as “imperiled” (S-2) (Clenche 2020); and a possible new species to science. We summarize these findings in more detail below.

Unanticipated contributions

Record of a new species to the province

The third author (ME) joined NLNature in 2013. In September 2013, he noticed a mosquito with unusual leg-band pattern in his backyard, about 1.4 km outside the capital city. He managed to take a photograph of it on his arm (Figure 3). It was the species Aedes japonicas japonicas, which researchers had first detected in Canada in 2001, but had not yet found in Newfoundland and Labrador. The species is native to eastern Asia and is a vector for West Nile virus in other parts of North America. ME asked the administrators of NLNature if the mosquito he had photographed was a potential disease vector. We passed the information to colleagues with mosquito expertise who were able to trap more individuals in the capital region and verify its presence. ME, along with entomology experts at Memorial University of Newfoundland, published a note describing this new species record for the province (Fielden et al. 2015).

Figure 3 

Aedes japonicas, a mosquito species native to eastern Asia, and discovered for the first time on the island of Newfoundland by a participant on the citizen science website NLNature.com. Photo by Mardon Erbland taken 24 September 2013 in the vicinity of Devereaux Lane, Outer Cove, NL.

Expansions of species distribution

In 2018, the second author (TC, member since 2013) was walking along the side of a favourite swimming location (48.115000 N, –53.751667 W) in rural Newfoundland and noticed an interesting-looking aquatic plant. He keyed it out as Nymphoides cordata (Little Floating-heart), and had the identification confirmed by a plant expert at the botanical gardens in St. John’s (Figure 4). This is an S-2 ranked (“imperiled”) species that was previously known from only eight localities on the island of Newfoundland, the nearest of which was 73.5 km from where TC discovered it. This greatly expands the known distribution of a rare species in the province. Since the initial discovery, TC found it in four other locations within the same watershed (Clenche 2020). His notation of such a significant finding led the lead author to notice it in a pond in a different watershed ~12 km (48.150556 N, –53.895000 W) from where TC discovered it.

Figure 4 

(a) The aquatic plant Nymphoides cordata and (b) its habitat as found 8 August 2018. The S-2 ranked species was only known from eight localities in the province until a participant on NLNature.com found it in several new locations. Photos by Tom Clenche.

In August 2016, TC was on a fishing trip in Labrador, and found an unusual freshwater clam in a river downstream from Igloo Lake (53.051038 N, –58.732001 W). He posted to NLNature, and alerted a local mollusk expert, who identified it as the Newfoundland floater clam (Pyganodon fragilis); this was the first record for this species anywhere within the Eagle River Watershed in Labrador. NatureServe lists it as S-2 (“imperiled”) in Labrador and “apparently stable” (S-4) on the island of Newfoundland. In August 2020, he found the same species in a pond on the island of Newfoundland (Shoal Pond, Carmanville; 49.368803 N, –54.164032 W) (Figure 5); again, the first record for that waterbody. He was with his father-in-law at the time, also an avid outdoorsperson. This led his father-in-law to find it in a different pond (Weirs Pond, Bonavista North; 49.218276 N, –54.382719W), also the first known occurrence of this species in this area. Thus, the sightings facilitated by NLNature.com have expanded our knowledge of the distribution of this species, which is currently confounded by uncertainty over its taxonomy (Martel et al. 2010) and a lack of survey effort. This species is likely quite common in the Atlantic region, but its distinctive features are noticeable only to someone with a keen eye and an interest in collecting bivalves. Thus citizen science observations could yield specimens useful to resolve the taxonomic uncertainty (Martel et al. 2010). Interestingly, another outdoorsperson, William Larkham Jr., also found P. fragilis in Labrador in 2020 and noted that it looked different from other freshwater bivalves. Larkham details his finding in two YouTube videos; the comments in the videos suggest that others think they have this species in their local watersheds as well.

Figure 5 

(a) Outer and (b) inner surface of a specimen of the Newfoundland floater clam Pyganodon fragilis found downstream from Igloo Lake, Labrador, NL, 23 August 2016. (c)Pyganodon fragilis found in Shoal Pond 23 August 2020. Photos by Tom Clenche.

Impacts beyond NLNature

Lynch-O’Brien et al. (2021) proposed a theory of transference of citizen science impacts by which participants can influence those around them and raise awareness of a particular issue. They documented its presence in an entomology citizen science project (Lynch-O’Brien et al. 2021). There is evidence that NLNature achieved transference of impacts in several cases. For example, TC was able to show his father-in-law the floater clam he found in Labrador, who then noted it in another pond; and TC’s documentation of the Little Floating-heart led two others to notice its presence in two different watersheds. This is an example of citizen scientists becoming recognized as experts by their peers (stage 3 in Lynch-O’Brien et al.’s 2021 framework). Both professional entomologists and the natural history museum’s curator emeritus have contacted TC about his findings, illustrating that he is acquiring the role of a local expert (stage 4 in Lynch-O’Brien et al. 2021). In a similar vein, ME is also recognized as a local expert. He notes that two of his observations introduced him to the world of amphipods and cyclopid copepods (stages 1 and 2 in Lynch-O’Brien et al. 2021). This spurred his current goal of finding DNA Barcodes for a sampling of freshwater crustaceans on the Northeast Avalon, including ostracods, anomopods, amphipods, and copepods. He recently collected amphipods for a doctoral candidate (stages 3 and 4 in Lynch-O’Brien et al. 2021) at Miami University in Ohio; DNA sequencing showed that two of his specimens were Hyalella wellborni, likely the first record of this species in the province.

The analysis of the 15 respondents from the focus groups and interviews provided evidence that many of the citizen scientists who interacted with NLNature passed through at least a few of the 5 stages for transference of impacts identified by Lynch-O’Brien et al. (2021). For example, when asked why they were motivated to participate, two spoke specifically of a long-term interest in the natural world, which Lynch-O’Brien et al. (2021) see as the first stage in their theory (Meldt 2017). Eight expressed that enjoyment of outdoor activities prompted them to interact with NLNature. Six expressed a motivation to join the citizen science project (stage 2 in Lynch-O’Brien et al.’s 2021 framework) in order to learn more about the natural world around them. Specifically, participants spoke about wanting to learn to identify species they did not know (Drechsel 2017; Lukyanenko et al. 2017; Meldt 2017), sharing their own observations with other people, and learning about the biodiversity of their home province (Meldt 2017). As well, they felt their data could be useful to track changes in the environment, especially unanticipated ones (Lukyanenko et al. 2017). Participants also noted that they felt that through active participation, they had earned the label “citizen scientist,” (stage 3 in Lynch-O’Brien et al. 2021), even if they had not identified themselves as such initially (Meldt 2017). Interestingly, at least three interview subjects expressed that did not feel they were necessarily credentialed as experts in the eyes of their peers on the site, but did see others on the site as experts, sometimes pointing to specific users as key participants they watched and learned from (Meldt 2017). Only two felt that they had acquired the roles of an expert (stage 4 in Lynch-O’Brien et al 2021), although some acknowledged that they had professional expertise in some domains of natural history (e.g., birds) but enjoyed participating on the site to learn about other taxa in which they did not have formal training (Drechsel 2017). In terms of using involvement in NLNature to influence change in others (stage 5 in Lynch-O’Brien et al. 2021), the main motivation participants had was to interact with others (both on the site and with experts who might be looking at the sightings contributed by citizens) to influences changes. Respondents expressed a high degree of motivation to contribute to research and to protect the natural beauty of the province (Meldt 2017).