Publisher's Note: There has been an amendment to the acknowledgements section of this article.
Terminology matters. People draw boundaries using language, choosing terms that include or exclude ideas, activities, and people (Gieryn 1999). As a quickly evolving and still nascent field, citizen science is already a broad concept, and inclusion (in the sense of broadening participation in science) is central in many of its meanings (Solomon 1993, Irwin 1995, Bonney 1996). Given this underlying tenet of inclusion, practitioners should give careful thought to the words they choose to describe activities, programs, and participants. Such care is important because terminology is also a part of how we construct our mental models of the world, what values we live by, and how we relate to each other (Haraway 1988, Barad 2007). Relationships between all members of research programs are particularly important for citizen science due to the large number of people involved, their varying motivations for involvement, and the power dynamics inherent in the way we produce knowledge. As an added challenge, terminology is never static and is constantly evolving. Terminology is particularly dynamic in citizen science, because the field is currently experiencing enormous expansion and a rapidly diversifying group of participants.
Our primary goal for this paper is to discuss the contemporary terminology of science involving the public. To do so, we summarize contexts for citizen science and different ideas of what citizen science is, making connections with and distinctions among related terms. Our secondary goal is to raise questions regarding the terminology for citizen science participants. We therefore identify common terms used, particularly as they relate to different types of citizen science initiatives and contexts. Finally, we note that some groups may prefer to avoid the term citizen science altogether. Because less familiar terms may be less useful for mobilizing resources to develop and maintain projects (e.g., funding, volunteers, media buzz, legal precedent), we propose that the definition of ‘citizen science’1 be as broad as possible, making it available for diverse groups to claim if they choose. We also share our perspective on what citizen science is not and offer suggestions for how to maintain coherent shared practice while allowing for plurality. We intend our suggestions to apply broadly to any currently popular set of terms as well as any future terms that may be used to describe kindred forms of public participation in scientific research.
This paper developed as a result of a lively conversation on the citsci-discussion-l email list, a Citizen Science Association (CSA) resource that was maintained by the Cornell Lab of Ornithology at the time this paper was developed.2 Initially, one list member raised the question of what to call the people involved in a citizen science project:
“I am talking to a lot of journalists this week, and I’m struggling with how to best differentiate between anonymous citizen scientists and people like me who have known credentials and training.” (C.C.M Kyba).
Our discussions moved from the citsci-discussion list to an email thread with a more targeted group of contributors who were interested in pursuing the development of a paper on the topic of terminology. Leadership then emerged to facilitate the synthesis of our discussions (first author); a shared Google doc was used as a tool to document and refine our discussion. The topic of word choice (for ‘participants,’ ‘scientists,’ and ‘citizen science’) further expanded into the question of what citizen science is, reflecting the entanglement of terminology and ontology, i.e., what we call things is linked to what we think they are. This paper therefore begins by establishing contexts for citizen science terminology, then discusses what citizen science is (including relationships to other terms), next offers a starting point for discussions around terminology used for participants, and finally concludes with thoughts about balancing plurality with coherent shared practice.
Although we intend to present some degree of consensus on terminology, we also seek to represent the diversity of our experiences, contexts, and perspectives. Not all of the opinions expressed in this paper are shared by all of the authors. We work in 11 countries around the world and represent a wide range of experiences with and commitments to citizen science (Table 1). Our terms, distinctions, definitions, and contexts originate from extensive collective experiences. While this paper does not exhaustively capture all perspectives of the larger citizen science community, we believe it is reasonably representative. We note, however, that most of the authors of this paper are academics and/or citizen science project leaders, many of whom are from the United States (U.S.) or Europe; we welcome comments from participants of and contributors to citizen science projects in other countries and regions.
|Author Name||Affiliation/Country||Commitment to Citizen science|
|Anne Bowser||Woodrow Wilson International Center for Scholars, USA||Supports the resources hosted on Citizenscience.gov; helps lead an international project on citizen science data and metadata interoperability and standardization; member of the board of the Citizen Science Association (CSA)|
|Jessica L. Cappadonna||Queensland University of Technology, Australia||Ecologist/PhD student who is studying how to engage citizen scientists with bioacoustics to find an elusive bird species; member of the Australian Citizen Science Association Management Committee|
|Luigi Ceccaroni||1000001 Labs, Spain||Coordinator of the Citclops project (http://www.citclops.eu), a 5 M€ European project about a citizens’ observatory for coastal and ocean monitoring; member of the European Citizen Science Association (ECSA) Board of Directors; editor of the book “Analyzing the Role of Citizen Science in Modern Research” (2017); chair of working group to “Improve data standardization and interoperability” of COST Action CA15212 “Citizen Science to promote creativity, scientific literacy, and innovation throughout Europe”|
|Caren Cooper||North Carolina State University and North Carolina Museum of Natural Sciences, USA||Co-authored ornithology/ecology papers with citizen science data (e.g., NestWatch, Christmas Bird Count, Project Feederwatch, Breeding Bird Survey); helped develop NestWatch, YardMap, and Celebrate Urban Birds; developed and manages the Sparrow Swap project; director of Research Partnerships with SciStarter; blogger of citizen science; founder and moderator of #CitSciChat; associate editor of CSA’s journal, Citizen Science: Theory & Practice|
|Brittany Davis||Allegheny College, USA||Environmental studies professor studying marine citizen science projects in Central America and the Caribbean|
|Daniel Dörler||Citizen Science Working Group at the University of Natural Resources and Life Sciences, Vienna, Austria||Zoologist currently completing a PhD on the ecology of an invasive slug in Austria using a citizen science approach; coordinator and founder of the citizen science networking platform “Österreich forscht” (http://www.citizen-science.at); organizer of the annual Austrian Citizen Science Conference|
|Ruth Duerr||Ronin Institute for Independent Scholarship, USA||Member of the Exchange for Local Observations and Knowledge of the Arctic (ELOKA) (http://www.eloka-arctic.org/), which facilitates the collection, preservation, exchange, and use of local observations and knowledge of the Arctic|
|MV Eitzel||University of California Santa Cruz, USA||Works on theories of epistemic justice at the Science and Justice Research Center; hands-on application to citizen science at the Statewide and Diablo California Naturalist Programs to develop stewardship of California’s ecosystems; and decolonial citizen science with The Muonde Trust’s community-based research team in rural Zimbabwe|
|Muki Haklay||University College London, UK||Researching participatory/co-created/co-production of citizen science, particularly in the context of local environmental issues, and indigenous and traditional ecological knowledge|
|Ed Harris||Scleroderma Education Project, USA||Medical educator and lead researcher on a new disease pathogenesis model for the rare autoimmune disease systemic scleroderma|
|Florian Heigl||Citizen Science Working Group at the University of Natural Resources and Life Sciences, Vienna, Austria||Agrobiologist and leader of “Project Roadkill” (http://www.roadkill.at/en), a citizen science project on road-killed animals in Austria, currently doing a PhD thesis on this project; coordinator and founder of the citizen science networking platform “Österreich forscht” (www.citizen-science.at); organizer of the annual Austrian Citizen Science Conference|
|Qijun Jiang||Wageningen University & Research, The Netherlands||Engaged in citizen science projects such as Amsterdam Smart Citizens Lab and Urban AirQ, which use bottom up and co-creation approaches that involve citizens in all steps of the process; member of the European Citizen Science Association (ECSA).|
|Tim Kiessling||Faculty of Ocean Sciences, Catholic University of the North, Chile||Marine biologist investigating the extent and impact of ocean litter pollution within the citizen science program “Científicos de la Basura” in Chile and Germany|
|Christopher Kyba||GFZ German Research Centre for Geosciences, Germany||Responsible for the “Loss of the Night” citizen science app and the “http://www.myskyatnight.com” webapp that allows users to view and analyze light pollution data|
|Chris Santos-Lang||Citizen Science Belleville, USA||Leads citizen science of a bible study that tests replicability of experiments to improve health, relationships, and well being for future generations (http://www.osf.io/fqn7v/wiki/home)|
|Anya Metcalfe||United States Geological Survey, Arizona, USA||Stream ecologist and entomologist who manages and utilizes a citizen science project across western U.S. examining aquatic insect distribution and emergence in a dam-regulated riverscape (http://www.gcmrc.gov)|
|Lesandro Ponciano||Federal University of Campina Grande and Pontifical Catholic University of Minas Gerais, Brazil||Computer science professor studying participation of volunteers in citizen science projects conducted through crowdsourcing and human computation systems, such as Zooniverse, Crowdcrafting, and Socientize platforms|
|Joseph Roche||Trinity College Dublin, Ireland||Astrophysicist and Assistant Professor in Science Education who lectures on citizen science for the M.Ed program in Science Education; on the science team for Sunspotter.org (Zooniverse project)|
|Andrea Sforzi||Maremma Natural History Museum, Italy||Director of the museum and responsible for the citizen science project (naturaesocialmapping.it) that includes a recording website and courses for citizen scientists and bioblitzes; member of the ECSA Board of Directors|
|Fraser Shilling||University of California, Davis, USA||Ecologist who co-directs the Road Ecology Center; trained water quality monitors for California’s longest running watershed-monitoring program (Yuba); co-developed the largest volunteer-contribution, roadkill-observing system in world; develops standard approaches and web-tools for public science; and tries not to use the term ‘citizen science’|
|Martin Thiel||Faculty of Ocean Sciences, Catholic University of the North; Millennium Nucleus Ecology and Sustainable Management of Oceanic Island (ESMOI); Center of Advanced Studies in Arid Zones (CEAZA), Coquimbo, Chile||Professor of Marine Biology, Director of the citizen science program “Científicos de la Basura” (http://www.cientificosdelabasura.cl), which has been investigating marine litter along the coast of Chile for the past 10 years, involving the participation of K-12 students and their teachers; associate editor of CSA’s journal, Citizen Science: Theory & Practice|
|Arika Virapongse||Ronin Institute for Independent Scholarship, USA||Social ecologist with expertise in conservation and development, including participatory approaches, and studying the role of citizen scientists in social-ecological observing and community development|
|Sarah West||Stockholm Environment Institute, University of York, UK||Designs, runs, evaluates, and conducts research about environmental citizen science projects in the UK and Kenya; projects include the OPAL project and Moors for the Future Community Science Project|
We intend this paper to be useful to a wide variety of people interested in any form of citizen science. We therefore explore the term ‘citizen science’ in a broad sense, including generation of any theory or hypothesis, research, scientific data collection, and/or data analysis in which the public (individuals or communities) participates. If readers prefer a different phrase for ‘citizen science,’ we encourage them to mentally substitute that phrase whenever we use the term here. Groups that are working to advocate and advance the development of citizen science, such as associations from around the world, may find our collection of terms and related starting points for discussion helpful for their own work on terminology, audience, and scope.3 Our definitions and taxonomy of terms related to citizen science can be useful as a reference to students, guidelines for journal editors, and to inform academic scientists as they engage with the public. Finally, we hope this paper can be useful to policymakers, funders who seek to support citizen science, and participants in citizen science including managers, planners, and practitioners.
To put our observations about citizen science terminology in perspective, we begin with a short discussion of how terminology matters, grounded in Science and Technology Studies (STS). We then review dictionary definitions of citizen science and provide some historical context. Finally, we discuss geopolitical and disciplinary contexts for the terminology of citizen science.
Scholars of science argue that language is not merely representational. Instead, language and materiality are co-produced; that is, the words that we use for what we observe are fashioned simultaneously with our perceptions of what those things are (Haraway 1988). Language is a sensitizing concept, or in psychology terms, language can prime us to see certain things and not others. This entanglement extends further. Barad (2007) states that how we know about the world (epistemology), what we believe exists in the world (ontology), and the values by which we live our lives (ethics), are not separate. Because science is a way of knowing about the world, and because naming things is simultaneous with our ontological conception of what we are naming, the role of language in Barad’s (2007) entangled ethics/epistemology/ontology is a critical point for consideration in citizen science. Furthermore, language is an important part of the construction of what is considered science vs. not-science (‘boundary-work,’ Gieryn 1999). Citizen science is typically intended to broaden participation in science, so we are essentially in the business of redefining or even disassembling boundaries. Therefore, attention to the terminology we use is an important part of effective practice. Because citizen science is a form of knowledge production, citizen science terminology has the power to allow some peoples’ knowledge to be included and the knowledge of others to be excluded. This power potentially presents epistemic (knowledge) justice issues (Fricker 2007) and has consequences for the quality of our understanding of the world (Haraway 1988).
As in many other participatory contexts, terminology can matter deeply to participants in citizen science projects. The terminology used to describe participants can potentially change the way they are treated or how they feel about themselves and their participation in the activity (Figure 1). The language used to refer to people, activities, and objects can have deep-seated resonance with racial inequities and histories of colonization. For example, the Standing Rock Sioux, as part of their resistance to the Dakota Access pipeline, object to being referred to as ‘stakeholders.’ Chairman Archambault said, “They would consider us a stakeholder, but we’re a nation” (quoted in Mufson 2016). Therefore, the use of terms like ‘stakeholder engagement’ is in conflict with how the Standing Rock Sioux view themselves, and in this case, the mismatch resulted in the tribe not participating in ‘stakeholder’ meetings. Or consider farmers in Colombia who, faced with social and armed conflict as well as militarized U.S.-Colombia anti-narcotics policy, are critical of the colonial legacies and capitalist influences of agricultural sciences (Lyons in press). Though these farmers conduct their own experimentation in their farming practices, they would not claim the term ‘citizen science’ to describe what they do when, for example, they use different ways of looking at and relating to soils than typical ‘scientific’ research. And in the U.S., the National Institutes of Health’s (NIH) International HapMap Project was designed to map human genome diversity from individuals around the world. Questions arose concerning what to label samples: If a sample was labeled ‘African’ or ‘Asian’ would the results be generalized as ‘all Africans’ or ‘all Asians?’ The NIH researchers spent considerable time and funding working with study participants to determine appropriate names, despite concerns that names could reproduce racial biases. Ultimately, the names suggested by participants were not used (Reardon in press).
Finally, as boundary-work, terminology selection is inherently strategic. This issue has been a part of our group’s discussions and has revealed our diversity of opinions. Some terms may be better for egalitarian purposes of democratizing knowledge production, while others may be better used for establishing the validity of citizen knowledge and authority to policy-makers. Still other terms are appropriate when working with indigenous people. We all came to this project with different goals in mind, and our orientations regarding terminology reflect this diversity. Different terms serve different goals, and different terms are appropriate for different audiences–e.g., prospective participants in projects vs. skeptical policy-makers vs. academics defending the concept of value-free science. In many ways, the range of goals and associated terms reflects the expansion of contemporary science beyond professional boundaries to include many facets of civil society.
In regard to definitions of terms, which activities fall under the guise of ‘citizen science’ is both nuanced and in a state of flux, although it is generally agreed that citizen science refers to the inclusion of members of the public in some aspect of scientific research. While the term ‘public’ or ‘the public’ is often used to describe the general population, well established research in public understanding of science and public engagement reminds us that the public cannot be assumed to be a monolithic entity (Chilvers and Kearnes 2015; Marres 2007).
The term ‘citizen science’ was added to the Oxford English Dictionary in 2014 as “Scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional scientists and scientific institutions” (OED 2016a). This definition of citizen science fails to consider the broader use of the term as initially coined by Alan Irwin (1995). Cooper and Lewenstein (2016) discuss these two meanings or ‘strands’ of citizen science. The first strand, from Irwin’s definition, emphasizes the responsibility of science to society, which they call “democratic” citizen science. At the other end of the spectrum they position the second strand, “participatory” citizen science, as practice in which people mostly contribute observations or efforts to the scientific enterprise, a meaning that originated with Rick Bonney’s (1996) work at the Cornell Lab of Ornithology. Bonney et al. (2016) suggest that future iterations of the OED definition should highlight the diversity, scale, and value of citizen science projects from both strands. Ceccaroni et al. (2017) focus on the convergence of these viewpoints to define citizen science in relation to civic education as work undertaken with citizen communities to advance science, foster a broad scientific mentality, and/or encourage democratic engagement, which helps society address complex modern problems.
Citizen science terms are dynamic and change over time, therefore
“it would be a shame for us to ignore widespread public interpretations of key terms and the reasons for those interpretations.” (Daniela Soleri, personal communication)
In that vein, using the word ‘citizen’ also can be an issue, as this word may be defined as “A legally recognized subject or national of a state” or “An inhabitant of a city or town” (OED 2016b). The first definition is problematic in some parts of the world where legal recognition is complex, and legal citizenship may not be relevant in many citizen science projects. The second definition appears to prioritize urban inhabitants. Citizenship can be more broadly construed, but the term remains problematic in practice; these difficulties also vary by country. While we cannot erase the potential problems associated with the term and we understand that many community members may not want to be called ‘citizens,’ in this paper we use the word ‘citizen’ as part of ‘citizen science’ because the term is familiar to most, and define ‘citizen’ here to mean a member of a broadly construed community.
The term ‘scientist’ was coined in 1833 (Yeo 1993) and slowly grew in use. The Eurocentric scientist-as-profession paradigm is relatively new, emerging slowly throughout the 17th to 19th centuries. Initially called ‘philosophers of science’ or ‘natural philosophers,’ individuals who pursued research made their living in another profession, were sponsored by a benefactor, or had independent means. Truth was accepted from “gentlemen” as reliable, in contrast to other groups, and technicians in gentlemen’s labs were regarded as “invisible” (Shapin 1994).
Sometimes these early scientists investigated topics by recruiting others through peer networks to assist with data collection and/or analyses (Miller-Rushing et al. 2012). For example, Charles Darwin was not a professional researcher but an unpaid companion on the Beagle, though he had medical training (Silvertown 2009). In 1874, many governments engaged prominent astronomers around the world to measure the Earth’s distance to the Sun in the “Transit of Venus” project (Ratcliff 2008). Some individuals outside the gentry also were able to contribute to discoveries. For example, Mary Anning–who had no training initially–found the first British Ichthyosaur (among other fascinating fossils) and became one of the most influential women in British paleontology, particularly posthumously (Hall 2002).
Citizen science as participatory data collection (part of Bonney’s 1996 definition) has existed for a long time without specific descriptive terminology. Members of the public have collected observations of nature for hundreds of years in such fields as archaeology, astronomy, and natural history (Silvertown 2009). In the late 1800s, amateurs were valued only for their observations rather than their ability to elucidate meaning from observations. An attitude prevailed that all observations, fossils, specimens, and so forth should be brought to a central place in England so that ‘scientists’ could use the materials and information to generate knowledge and advance science. So even after scientific research was professionalized and institutionalized, a partnership occurred between amateur and professional scientists (Miller-Rushing et al. 2012), although in some disciplines such as veterinary science, the quality of information collected by amateurs was progressively viewed with skepticism (Ruth Duerr, personal communication). In many cases, citizen science is often viewed through this lens of a partnership between amateur data collectors and professional elucidators.
As another example, the Audubon Christmas Bird Count in the U.S. was founded in 1900, and the United Kingdom (U.K.) also has a long and rich history of wildlife recording by volunteers; such existing programs are now labeled as ‘citizen science,’ which contributes to the recent perceived and actual rise in citizen science projects (Roy et al. 2012). ‘Citizen scientist’ (meaning scientist independent of institutions) was used at least as early as 1912 (Scott 1912). ‘Participatory Action Research’ (PAR), also termed ‘Community Action Research’ (CAR), became popular during the social movements in the 1960s and 1970s, especially in Latin America (McTaggart 1991, Torres 1992, Kindon et al. 2008). ‘Citizen science’ as a democratic concept was used in the 1990s (Solomon 1993; Irwin 1995), around the same time that Bonney first used ‘citizen science’ to describe his long-running participatory data collection projects (1996). ‘Crowdsourcing,’ as a term to describe an open call to a wide group to aid in some kind of labor, originated in WIRED magazine in a 2006 article; even in this first use, the term included examples of crowdsourced science and research (Howe 2006).
Citizen science is associated with, and often overlaps with, a variety of other names that may align with the context of an activity–such as the academic discipline, geopolitics, language, and culture–or features of research design, such as modes of engagement. In public health and environmental justice contexts, for example, ‘community-based participatory research’ (CBPR) is more commonly used (see, for example, Minkler and Wallerstein 2011), and terms for participants tend toward ‘resident,’ ‘neighbor,’ or ‘community member.’ Jason Corburn (2005) very carefully chooses the term ‘local knowledge’ for his study of community-based public health research. In geography, ‘volunteered geographic information’ is commonly used to describe engagement of large numbers of participants involved in the digital creation of geographic information (Goodchild 2007; Sieber and Haklay 2015), and despite the term’s focus on data, it is widely used to describe the ‘citizen science’ activity itself. The term ‘participant’ in medical research may refer to those participating in studies as human subjects of research, which contrasts with how the term is typically used in citizen science projects.
Differences in disciplinary contexts, vocabularies, and norms may make it difficult for researchers in one discipline to understand the engagement practices, methods, data, and impacts of researchers in another discipline. These differences also may create confusion among participants about the type and depth of their own involvement in the project. In some cases, developing standardized vocabularies for citizen science which articulate shared aspects, such as quality assurance/quality control concerns, may transcend disciplinary differences (Ceccaroni et al. 2017). However, these shared vocabularies must be co-developed by, and later used by, a wide range of relevant stakeholders.
Although the term ‘citizen science’ was coined in the U.S. and the U.K., the practice of scientists working together with other people occurs in many different countries, so various terms for this method exist (Table 2). What most of the terms have in common is their language-specific word for ‘citizen,’ in the sense of ‘inhabitant of a nation’ (sometimes associated with legal attributes, i.e., ‘civil rights’), and the translation of the term ‘science,’ which characterizes the scientific approach behind the activity. In many countries, for example Australia, Austria, Brazil, Chile, Ireland, and the Arctic regions, citizen science was established by grassroots activities through a bottom-up approach, and the terms that practitioners in these geographies use echo this grassroots development. In Europe, citizen science is also driven by universities, research centers, and museums. Governmental support and/or structures are available only very recently in some countries (e.g., Austria), whereas in Germany and the U.S., the government currently funds and sometimes even runs citizen science networking activities and projects. Considering the social diversity involved in the grassroots origins of citizen science in many countries, it may not be enough to simply translate the term, because the history, context, and practices must be looked at more closely. In some countries, for example Austria and Switzerland, the term is so novel and unusual that it is not translated at all, and the meaning of ‘citizen science’ is adapted to the country-specific context. To partially bridge this gap, the European Citizen Science Association (ECSA) is supporting the translation of their “Ten Principles of Citizen Science” (ECSA 2015) in as many languages as possible (27 languages by December 2016).
|Country||Citizen science terminology and context|
Traditional Ecological Knowledge (TEK)
|As noted in a Community White Paper (Noor Johnson et al. 2013), “Arctic Indigenous peoples have been systematically observing the environment for millennia” where such monitoring “plays a significant role in daily life, providing information that is critical to safe travel and successful hunting and harvesting activities.” The term ‘Citizen Science’ is rarely if ever used for research in these regions, perhaps due to the existence of the Arctic Council which has “promoted cooperation, coordination and interaction among the Arctic States, Arctic indigenous communities and other Arctic inhabitants on common Arctic issues, in particular on issues of sustainable development and environmental protection in the Arctic” for the last 20 years.|
|While members of the public have contributed to scientific research in Australia for decades, the term ‘citizen science’ and the ubiquity of such activities was relatively unknown until recently. Citizen science leaders recognized the need to connect this community, which led to the formation of the Australian Citizen Science Association (ACSA; http://www.citizenscience.org.au) in May 2014 and the first Australian citizen science conference in July 2015. To date, citizen science activities have been identified at community, regional, state, and national levels. For most projects, citizen scientists contribute observations of fauna, flora, and habitat, though a few projects exist in astronomy, meteorology, and seismology. Citizen science in Australia is also rapidly diversifying into new domains (e.g., online) and disciplines (e.g., biomedical sciences).|
|Citizen science has developed rapidly over the last 3–4 years in Austria, although the approach has been known for more than100 years. In 2014, the first Austrian online platform for citizen science projects (Österreich forscht, http://www.citizen-science.at) began to connect citizen science projects and actors to foster this method and to ensure quality. The platform is borne by citizen science project leaders, so it is independent from institutions. In parallel, a second platform (Centre for Citizen Science, http://www.zentrumfuercitizenscience.at) was developed at Österreichischer Austauschdienst.|
|The Citizen Science Movement (“Movimento Ciência Cidadã,” http://www.movimentocienciacidada.org) is an effort focused on democratizing access to Brazilian scientific production on topics of social interest. Some examples of citizen science projects are Farmer-Experimenter Groups, the ForestWatchers Project (http://www.forestwatchers.net), and Contribua (http://www.contribua.org). Participants are usually called volunteer (‘voluntário’) or participant (‘participante’).|
|Chile has a long-standing tradition of Participatory Action Research (Investigación-Acción Participativa), which became widespread during social movements of the 1960s and early 1970s. There is also a legacy of close collaboration with artisanal fishermen and small-scale farmers, using Traditional Ecological Knowledge (TEK), which has only recently been considered ‘citizen science.’|
公民科学 or 公众科学 (Simplified Chinese), 公民科學or公眾科學 (Traditional Chinese)
|The term ‘citizen science’ is translated to “公众科学” or “公眾科學” in simplified Chinese or traditional Chinese respectively. This translation is close to ‘public science’ in English. A more direct translation is ‘公民科学’ or ‘公民科學’ in simplified Chinese or traditional Chinese respectively.|
|The collaboration of citizens and scientists has a long tradition in Estonia, with roots beginning with the Estonian Naturalists’ Society, which was founded in 1853. However, the awareness and understanding of citizen science is still expanding in Estonian society. There are many parallel translations of the term citizen science: Kodanikuteadus translates to ‘citizen science’ and can be misunderstood as “science about being citizen,” which is rather formal; harrastusteadus translates to “hobby or amateur science,” rahvateadus translates to ‘people science,’ and huviteadus translates to ‘hobby or lay science.’ There are no existing associations for Estonian citizen science practitioners and there is no common website.|
|Citizen science in Europe is mainly represented by the activities of the European Citizen Science Association (ECSA), which is a non-profit association organized to encourage the growth of citizen science in Europe. It draws on 200 individual and organizational members from more than 28 countries across the European Union and beyond. Launched in 2013, ECSA has grown from an informal network of civic educators interested in citizen science into the reference network of citizen-science initiatives for Europe.|
|Citizen science in Germany has long been visible among prestigious local groups, but has rapidly increased in the past decade. The project GEWISS (BürGEr schaffen WISSen, literally translated as “citizens create knowledge,” is an initiative of different university and non-universitary organizations, funded by the German Ministry of Education and Research. GEWISS reflects, promotes, and supports citizen science in Germany. As of August 2016, the online platform buergerschaffenwissen.de lists 73 current German citizen science projects.|
|Citizen science in Ireland has grown in the last decade, but it is still a relatively unknown concept. Most of the citizen science projects that have taken place are localized, with few having the support needed to become national endeavors. The projects that do gain traction tend to be contributory, and are often led by environmental or biodiversity-focused organizations such as the Environmental Protection Agency, the National Biodiversity Data Centre, the Irish Wildlife Trust, and Science Gallery Dublin.|
|Although citizen science in Italy has become more common in the last few years, it is not a widespread concept. Defining citizen science in Italy relies on first discovering the existing citizen science projects, and this is hindered from a lack of clear terminology for this field. However, the international citizen science movement has recently activated some citizen science projects at the local, regional, and national scales of Italy. In 2015, an informal group called Citizen Science Italy was formed with the purpose of sharing experiences and developing the concept of citizen science. Most members, however, are observers or simply interested in supporting the development of citizen science in Italy. Nevertheless, Italy is among the most represented countries (i.e., number of members) in ECSA, demonstrating great interest and potential growth for this field in the future.|
|Dutch people are involved in a variety of citizen science projects, for example, air quality monitoring (Jiang et al. 2016) and noise monitoring and gas extraction-induced earthquakes (Carton and Ache 2017). These projects are often bottom-up in origin. The term ‘citizen science’ is ‘Burgerwetenschap’ in Dutch, but the English term is also widely used.|
|The context of citizen science in Spain is similar to other countries in Europe, except for the language used by contributors, which is mainly Spanish and Catalan, and to a lesser extent Euskera and Galician.|
|In the late 2000s, the term citizen science gained popularity as projects, such as OPAL, began to use the term. Many biological recording schemes rebranded themselves as citizen science (Roy et al 2012), as use of the term caught on in the media. Most citizen science projects in the UK are contributory, but there are some examples of co-created projects, for example, the ExCites group at UCL, which emerged from participatory action research.|
|Citizen science is widespread in the U.S., which has the highest percentage of members of the Citizen Science Association. However, citizen science activities are not particularly coordinated among host groups. One important and growing network is within U.S. government agencies, which are coordinated through the Federal Community of Practice for Crowdsourcing and Citizen Science (CCS) and the formal Agency Coordinators. In an effort to expand and accelerate the role of crowdsourcing and citizen science in the U.S. government, a collaborative group of agencies released the Federal Crowdsourcing and Citizen Science Toolkit (http://www.citizenscience.gov/) as a venue for finding, planning, and maintaining federal citizen science projects.|
|Most scientific work done by citizens in Zimbabwe is currently undocumented and occurring at a grassroots level, though there is little evidence of its existence. This work is conducted on such topics as traditional medicine for people and livestock, wild fruit and plant processing and preservation, civic construction, art, and climatology. Because communities do not recognize that they are doing ‘citizen science’ there is no word in Shona for the activity (and likely not in the other national languages of Zimbabwe). Zimbabwe’s low GDP is the main hindrance to formal scientific inquiry, so citizen science has potential to grow in the country as a way to generate information and solve problems.|
Our discussions raised questions regarding what terms to use to describe science, and the range of objectives and commitments of citizen science, which depended on the intent and practices of a given citizen science project. The degree of engagement of participants in different phases of science may affect the terminology used (or desired by participants). Public Participation in Scientific Research (PPSR) is a term explored by Bonney et al. (2009b) and Shirk et al. (2012) which covers a wide range of participatory approaches, including citizen science, crowdsourcing, participatory action research, community-based research, and volunteered geographic information. Within PPSR, these approaches also can be distinguished by the degree of participation and the stage of the scientific process at which people are involved (Shirk et al. 2012). The term has proven to be difficult to use and has gained less traction in the face of the already well-established ‘citizen science’; although in the U.S. some granting agencies, including the National Science Foundation (NSF), do use PPSR to describe the projects in their portfolio. ‘Community and citizen science’ is a more recent term intended to be more accessible and serve the same ‘umbrella’ purpose as PPSR (Ballard et al. 2016).
Many researchers and practitioners characterize citizen science instrumentally, that is, as a tool, method, or form of research collaboration (e.g., Bonney et al. 2009b; Wiggins and Crowston 2011; Follett and Streznov 2015). In these cases, citizen science is often contextualized within traditional, hierarchical science and policy-making processes. Advocates of the instrumental view often suggest that citizen science allows traditional scientific research practices to reach larger scales (e.g., geographically, sample size) than have ever been possible for many fields. Indeed, interest in citizen science has grown alongside the big data and open data phenomena, which are touted as the new future for many sciences (Auer et al. 2007; Dickinson et al. 2010). When considered as a tool, method, or form of research collaboration, citizen science is associated with the potential for significant benefits for volunteers, for example, for improving ecological literacy (Bonney et al. 2009b). Citizen science is often seen as an informal way to achieve both educational and scientific objectives, and can be seen as a parallel activity to the efforts of museums and science centers in informal education (Bonney et al. 2009a; Sforzi et al. In press).
Conversely, citizen science also has been characterized as part of a movement that democratizes the scientific research process (Irwin 1995), for example, by restoring public trust in science, re-orienting science toward coping with the complexity of environmental problems, and installing democratic governance of science, as Karin Bäckstrand (2003) noted. Increasingly, it is argued that policy development must be more evidence-based (Georgalakis et al. 2017) as well as more transparent and inclusive of the perspectives of constituents. Proponents of democratizing science may strive toward greater inclusion, suggesting that the public has the right and responsibility to set research agendas along with ‘professionals’ (e.g., Irwin 1995; Ramirez-Andreotta et al. 2015). This perspective is inherently political (Mueller and Tippins 2012). Elitism, whether or not intentional, permeates many science and policy-making processes, from setting research priorities and study design (e.g., often determined by institutional funding) to conducting analyses and interpretation (e.g., which may require specialized equipment or data literacy), and communication of results (e.g., through scholarly journals, many of which are behind paywalls). Citizen science can take place “in the peripheries of established institutions” and thus be considered a form of resistance to elitism (Kullenberg 2015). In this case citizen science is a sibling of the open science movement, which aims to purge barriers to accessing science. A truly democratic science would involve the public in all aspects of science, potentially combining open science and citizen science. Citizen science is also seen as a community engagement tool, particularly as governing entities face increasing pressure to include community in conventionally top-down decision-making processes. For example, the City of Boulder, Colorado, U.S. has prioritized a citizen science program as a way to engage communities in the city’s urban resilience strategy (City of Boulder 2016).
Finally, citizen science that qualifies as public science can be thought of as a knowledge-producing capacity of society and a path to evidence-based decision-making. Public science is conceived as being for the public good, sometimes (but not always) funded by the public through taxes and according to government priorities (Nielsen 2011). Citizen science can empower communities to advocate for their local environments through scientific research, for example, by gathering the evidence to articulate issues, share these results via social media with the public, and thereby influence decision makers to act on environmental problems. This type of citizen science is rooted within the principles of participatory action research (PAR), considered “a collaborative process of research, education, and action explicitly oriented toward social change. PAR involves academic researchers (usually full-time and paid) and non-academic co-researchers and participants (usually part-time on the project and not paid) working together to examine a problematic situation in order to change it for the better on participants’ own terms” (Kindon et al. 2008). Results of some such projects in the U.S. have revealed significant environmental degradation, which then resulted in broader investigations led by or supported by the federal government. For example, following citizen engagement in water quality monitoring during the recent Flint, Michigan water crisis, the U.S. Environmental Protection Agency (EPA) awarded $80,000 to a team of researchers at Virginia Tech to test the city’s water (Hohn 2016). As citizen science expands in public health and the social sciences, particularly in its community-based forms (Kindon et al. 2008), one can easily imagine societies experiencing similar benefits in these domains.
‘Science’ is not the only way to refer to systematic investigation. Irwin (1995) used the term ‘science’ broadly, “to encompass a whole worldview and a set of institutions within society” (p. 8). In principle, we also interpreted the term in many different ways: Knowledge production, model-building, explanation-generating, investigation, question-posing, or theorizing. In practice, citizen science projects in our collective experience could be referred to as citizen analysis, investigation, monitoring, or research. To give a sense of how these terms can have different meanings for different people, we encountered several different attitudes about the definition of ‘monitoring.’ From our initial email discussion:
“In the Wisconsin Citizen-based Monitoring Network, we use the term monitoring to mean continually tracking, or taking repeated measures. So, we view monitoring as a subset of citizen science that focuses on repeated data collection, often to look at trends over time, as opposed to other forms of citizen science which could be one-time experiments or looking at specific snapshots in time.” (Eva Lewandowski)
In contrast, another list member uses the term ‘monitoring’
“to raise awareness among academic researchers, for example, that they’re not really fostering citizen science if they’re not engaging [citizens] in formulating the questions or interpreting the results.” (Pam DiBona)
Conservation biologists often make distinctions between hypothesis-driven research and monitoring (Lovett et al. 2007). ‘Research’ is also a term that may be used outside of academia, e.g., in industry or by natural resource monitoring groups.4 The reason that these notions about the activity being conducted are important is that they are linked to the way the contribution is assessed and valued. For example, some academic researchers might view monitoring, or applied science, as scientific activity of a lesser value than new discoveries, while community members who want to address environmental issues may care more about the credibility of the results and less about publishing in a highly regarded journal or about calling their work ‘research’ (e.g., the example of the Colombian farmer).
Crowdsourcing designates the practice of obtaining needed services, ideas, or content by soliciting contributions from a large group of people, especially through online collaboration and participation. As it applies to scientific research projects, crowdsourcing can be considered science in which the general public participates (e.g., not just people with credentials), often without fully understanding the concepts or implications motivating a research project. For example, individuals contributing to crowdsourcing projects may contribute or analyze data, but they rarely determine the questions or initial motivations of research. This model is similar to the ‘contributory’ citizen science projects described by Shirk et al. (2012), and the Level 1 citizen science projects described by Haklay (2013). It is important to note that despite their reliance on micro-tasking and light engagement, there is evidence (Eveleigh et al. 2014) that some participants use the opportunity to develop deeper interest and engagement in science.
Citizen science in its contributory forms does have the potential to become a neoliberal tool, divesting the state of responsibility for important societal functions and delegating it to individuals (Brown 2015). Citizen scientist participants can be used for their labor without being given resources to do the work or authority to act on the outcome. But not all contributory projects do this, and well-designed crowdsourcing projects have the potential to contribute to democracy. Furthermore, not all citizen science projects need to make strides for social justice. All projects are still well advised to be aware of the concerns raised by STS critics of citizen science, though, and to give consideration to terminology and other justice aspects of their work.
In addition, while not all citizen science is crowdsourcing and not all crowdsourcing is citizen science, some authors are concerned that these two words may become synonymous, and that crowdsourcing projects do or may dominate citizen science funding, potentially displacing more engaged kinds of participatory research. We do not intend to dismiss crowdsourcing or contributory citizen science, but rather we seek to remind funders, journals, the popular press, and the public at large that many different kinds of activities and levels of engagement can be considered ‘citizen science.’ At the same time, we focus on the synergies between these terms and connecting both terms to other forms of open innovation, such as motivation through prizes and challenges, which can help a range of stakeholders understand and continue to value these forms of participatory research.
Given the broad vision of citizen science that we are suggesting, a few words are necessary to mark some boundaries. The OED definition and our discussion above indicate that the public should be involved in some aspect of the project. We do not consider projects to be citizen science if they use citizen data, biological samples, or labor without indicating what these are to be used for. Some form of transparency or informed consent should be a necessary part of the ethical conduct of citizen science projects. In addition, ECSA’s “Ten principles of citizen science” lay out the commitments that projects should have to their participants (ECSA 2015). In general, these principles position the public participants within the science program or project as allies and collaborators with professional scientists, given the same respect that a professional scientist would be accorded in conceiving of, implementing, interpreting, and publishing scientific results (see Sarna-Wojcicki et al. 2017 for a discussion about participatory work and coauthorship). Much as standards of ethics change over time, the ethical criteria that a project must meet to qualify as citizen science may shift over time–the ECSA principles may currently be ahead of their time and/or may someday become outdated, but the boundaries of citizen science are ethical boundaries, as we pointed out in our theoretical grounding (Gieryn 1999; Barad 2007).
As shown in the previous section, there are many different types of citizen science and associated initiatives. Similarly, the terms used to describe participants in citizen science also vary. Our original discussion had a pragmatic origin: What words do we use for people, when some words are clearly inappropriate for our particular situation? Ultimately we generated a large list of terms that varied in their popular and academic usage (Figure 2). In the following sections, we highlight some of the emergent themes in these terms, and we summarize some of their intentional and unintentional meanings in Tables 3 and 4.
|Citizen scientist, Scientist-citizen, public scientist, community scientist||Individual with formal science training who is actively engaged in the civic sphere and wants their work to both serve the greater good and do so transparently (Stilgoe 2009)||Citizen scientists investigated anecdotal evidence to construct hypotheses regarding developmental disorders that members of the public claimed were triggered by a MMR vaccine (Stilgoe 2009). Members of Union of Concerned Scientists’ Science Network (http://www.ucsusa.org/science-network)||‘Citizen Scientist’ is easily confused with more common meaning of public involvement in science|
|Civic educators||Individual who provides information and/or creates educational opportunities for others with the purpose of building a path for greater civic engagement||Researchers, teachers, scientists, issue advocates, journalists, reporters and political campaigners (Ceccaroni et al. 2016)||Closely associated with democratic values, inherently politically laden|
|Commercial||Individual trained in science with the goal of creating products for profit||Commercial fisher, Commercial scientist||Incentivized by financial profit, rather than ‘knowledge for the sake of knowledge’|
|Credentialed, Trained, Educated||Individual with formal scientific degrees and training||Faculty member at a university||Reinforces the value of formal scientific education|
|Elite||Individual with experience and/or privilege not shared by the general public||Only elite scientists may serve in some peer-review processes or are considered for tenure or funding||Typically excludes the general public, early-career scientists, and minorities; many scientists strive not to be elitist|
|Institutional, Academic, Laboratory||Individual employed by or affiliated with an academic institution, agency, company, or non-governmental organization||Tenured professor, Government scientist, Laboratory technician, Student||Scientists may not be affiliated with an institution or may not work in a laboratory|
|Professional, Paid, Employed||Individual working in a scientific occupation, profession, or holding a position for which they are paid||White collar professional, Professor, Employee||Some scientists may conduct participatory projects outside paid time|
|Researcher||Individual investigating a specific and identified scientific question||Research scientist, Research ecologist||Researchers are often interpreted strictly as academics|
|Scientist-activist||Individual with formal science training who applies their expertise to political agendas||Internationally, thousands of scientists participated in a “March for Science” on April 22, 2017 to show support for evidence-based policies in government||Can be perceived as having shed the ‘objectivity’ of science|
|Volunteer Scientist||An individual who is not paid for their participation in scientific pursuits||Graduate students||Implies that scientist is inexperienced or not worth formally hiring|
|Amateur, Hobbyist||Individual participating in science for non-fiscal personal gain||Amateur Astronomer, Amateur Naturalist||Implies that an individual is not a professional or expert|
|Anonymous, Non-identified||An individual participating in citizen science that is not identified by name||An anonymous contributor to http://www.myskyatnight.com||Does not credit participants|
|Citizen||An inhabitant of a particular town or city; a member of the general public in a defined geographic locale||American Citizen, Citizen Soldier||Can also be defined as “a legally recognized subject or national of a state, either native or naturalized,” which is misleading and potentially exclusive in the context of citizen science|
|Citizen Researcher, Individual Citizen Scientist||An individual leading an activity or performing independent or collaborative research as the lead investigator||Citizen researcher Edward Harris (a co-author of this paper) initiated the Scleroderma Education Project||Inherently separates projects from being considered traditional scientific research|
|Collaborator||An individual working together with a project leader||Participants on www.zooniverse.org are referred to as collaborators||Does not specify the level of collaboration (i.e., what part of the scientific enterprise)|
|Community, Community Researcher||Individuals who have joined a community, online or in person, to work toward a common research objective; members of a pre-existing community||Friends of Spy Pond monitoring program (Castleden et al 2008)||Implies parochial interest; implies familiarity among participants that may or may not exist; ‘community’ may be difficult to define|
|Contributor, Donor||A benefactor or contributor of money, goods, or other intellectual or physical products with value||Financial donor, Organ/tissue/blood donor, Intellectual property donor||Most often refers to individuals who donate money or materials, and not information; connotes a contributory model of citizen science with limited involvement of participants|
|Human Sensor||Individual who is part of a network by sending data and observations that are often taken and transmitted via modern communication tools, like smartphones, to a central database||The US Geological Survey has developed a smartphone app to record individual observations of Earthquakes (“Did you feel it?” https://earthquake.usgs.gov/data/dyfi/)||Has a historic stigma of scientists utilizing data on people (e.g., health, behavior, web history) without explicit consent|
|Indigenous/traditional/local knowledge expert or holder||An individual with place-based knowledge gained through lived experience or oral tradition||The indigenous Chukchi communities of Turvaurgin and Nutendli are documenting contemporary observations of climate change and comparing them to historic conditions on traditional hunting grounds (http://eloka-arctic.org)||The exact term used may be project specific and based on the expertise of the people involved, however, ‘traditional’ is less favored because the knowledge is dynamic; indigenous knowledge is viewed as different from science by both the holders of this knowledge and formally trained scientists|
|Lay Knowledge Holder, General Public||An individual who is not affiliated with a scientific establishment, but may possess specialized knowledge||People applying their knowledge of local geography to identify cities at night (http://www.citiesatnight.org)||‘Lay knowledge’ is broad and suggests that everyone has this|
|Layman||A person without specialized knowledge or training of a specific subject||A farmer providing novel pest specimens to agricultural research||Outdated term; was more commonly used during 19th century than in past 100 years|
|Participant||An individual involved in research||The Audubon Christmas Bird Count enlists thousands of participants to observe and quantify migratory birds||This term is used by those running a project; the term comes from participatory research and ‘volunteer’ may be preferred instead|
|Partner||An individual or organization working with a scientist; term is often used in community-based participatory research||Community partner||Power dynamics are rarely equal; term ‘ally’ may be favorable|
|Student, Pupil||An individual who is engaged because their classroom/school is participating in a citizen science project||Entire classrooms or schools are sometimes assigned participation in projects||These individuals are a captive audience and not truly volunteers in the democratizing sense of citizen science|
|Uncredentialed/non-credentialed Researcher, Nonacademic, Non-scientist||An individual participating in a citizen science project who lacks scientific credentials (Nielsen 2011)||Most online citizen science projects receive data and input from non-credentialed users||Definition based on what people are not or don’t have and relies on credential, affiliation, or scientist as a defining characteristic|
|VolunPeer||A volunteer, organization, or institution representing knowledge-building activities and collaborative enterprises||Twitter hashtag #volunpeers quickly connects individuals for asynchronous collaborative activities (http://www.meghaninmotion.com)||Not currently considered a word in any official dictionary|
|Volunteer||An individual who contributes unpaid labor or service to an enterprise, e.g., a science activity||Volunteer Monitor, Volunteer Recorder||A general term without being explicitly linked to science; some participants may be paid|
|Project-specific terms||Specific terms may be appropriate or preferred for specific projects or by specific participants||‘Biological Recorder’ (Biological Recording Centre), ‘Crafters’ (CrowdCrafting), ‘eBirder’ (eBird), ‘Local partner’ (MassBays), ‘NestWatcher’ (NestWatch), ‘Player’ (Foldit), ‘Zooites’ (Zooniverse)||Limited generalizability (but generalizability may not be desired)|
Scientists may have many different motivations for participating in or leading citizen science projects. Some may need labor to collect or analyze data (which can lead to contributory models of engagement, such as crowdsourcing). Others may strongly believe that science should serve the public interest, and want to incorporate egalitarian ideals into their work. Increasingly, these latter types of scientists are referred to as ‘public scientists’ or, as Stilgoe (2009) calls them, ‘citizen scientists’ in the sense of scientists who put forward their citizenship as central to their work. These individuals are often the leaders of a project, although calling them ‘project leaders’ is focused specifically on the person’s leadership role.
Some themes emerged in the terms used to describe scientists involved with citizen science (see Table 3). Some terms are related to employment (‘paid’ or ‘professional’), though these could be problematic in situations in which ‘citizens’ are paid to help with projects, or are professionals in other fields. ‘Professional’ is also difficult to define in and of itself. One suggestion from our group was that a ‘professional’ is ‘able to make a living by contributing to science,’ however that definition raises its own issues, for example, how to define ‘a living,’ particularly as the science community becomes increasingly fractionally employed (Arbesman 2012). Another set of terms referred to affiliation, for example ‘academic scientist’ or ‘institutional scientist,’ although these terms may not be appropriate for scientists working in industry or as private consultants. Finally, a last set of terms referenced training or the possession of a credential. Einstein’s great intellectual success while working as a patent clerk is a good example of a scientific contributor without a credential (Clark 1971).
Among the authors, we have examples where the ‘lead scientist’ of the project is a person without academic credentials (Edward Harris).5 While uncommon, a number of cases involve individuals with little or no formal education who have advanced science in diverse fields including astronomy, paleontology, and even medicine.6 In some cases, these self-taught individuals later went on to obtain formal credentials in their field, but in other cases they remained ‘amateurs’ as they continued to advance research in their field. Barriers to individuals such as these include complex concepts and research that is kept behind a paywall. Such barriers are slowly being broken down by online education (e.g., Khan Academy) and the rise of open science and open-access journals, but these are currently limited in their scope. Becoming a self-trained expert is a major challenge in any field, but that is only part of the battle. Learning how to navigate the written and unwritten rules, and even the politics of a particular field, are critical skills that are needed if one wants to become a researcher who helps to advance knowledge in that field.
Another relevant criticism of ‘trained scientist’ as the description of a project leader is that participants are also trained in some way; without their training, there could be no project.
“If the manner in which people think is important, then whether someone is a professional/employed/lab/institutional scientist may not matter so much as the fact that they are ‘formally educated in western science.’ This might most usefully be contrasted with experientially trained or traditional or local knowledge based researchers” (Daniela Soleri, personal communication).
Nevertheless, we do acknowledge that the level of training among those involved in a project may vary, but also that these differences may not directly correspond to one’s formal education.
A related topic that emerged as we discussed the terminology of citizen science was the question of whether ‘professional’ scientists’ were committed to civic action, whether they volunteered on their own or other projects, and whether they saw activism as a part of their work. Many scientists who lead citizen science projects identify as ‘public scientists who want their work to both serve the greater good and do so transparently (Irwin 1995; Stilgoe 2009).
We also generated a long list of terms to describe project participants (Table 4). This is unsurprising, as Geoghegan et al. (2016) found that citizen science stakeholders in the U.K. referred to environmental monitoring citizen scientists by using a range of terms, including volunteers, amateurs, amateur naturalists, natural historians, and biological recorders. ‘Amateur’ can be viewed differently depending on the discipline. For example, in astronomy, paleontology, and ornithology, the term ‘amateur,’ as in ‘amateur astronomer’ is widely used and accepted in a non-pejorative sense. However, anyone calling themselves an ‘amateur medical researcher’ would presumably be taken less seriously.
Members of the public may have varied motivations for participating in citizen science projects (West and Pateman 2016). Some may be drawn to a particular physical object of study or place, while others may enjoy contributing through crowdsourcing. Still others use science to defend their basic rights (e.g., Flint, Michigan water crisis and many other environmental justice projects). These motivations may change the terminology that is appropriate to describe their participation, or their own choices about what they want to be called.
The idea of citizenship is also worth a second look. Being a citizen of a nation state can be a source of pride, however, national citizenship can often be a sensitive topic and contested space. As such, practitioners may want to be aware of attitudes around citizenship in the geopolitical context they work in. The word ‘citizen’ may need to be explained or augmented to ensure that the right sense is received by potential participants. For example, John James Audubon was a French citizen living in the newly established U.S. when he first banded birds, painted them, and began studying their behavior–all without citizenship or credentials (Rhodes 2004).
Local, traditional, or indigenous knowledge holders are in some cases an important group of ‘citizen scientists.’ Local and Indigenous knowledge could be considered citizen science, but the term ‘science’ may or may not be appropriate for or acceptable to all groups. On the other hand, indigenous knowledge development may represent the first example of science by human societies, and indigenous people the first scientists (Snively and Corsiglia 2001). Outside authorities do not always consider traditional or local knowledge to be legitimate, although these sources of knowledge may be evidence-based and their validity well established. As one example, the community white paper “Strengthening community-based monitoring in the Arctic: Key challenges and opportunities” (Johnson et al. 2013) notes that “Arctic Indigenous peoples have been systematically observing the environment for millennia,” where monitoring “plays a significant role in daily life, providing information that is critical to safe travel and successful hunting and harvesting activities.” What has been missing until recently are efforts to document and connect all of these activities.7 Going well beyond environmental studies, such efforts include a wide variety of initiatives important to the communities involved such as mental health and wellness.8 It may also be inadvisable to refer to indigenous peoples as ‘citizens’ due to the legacies of colonialism.
In some cases, we found that terms focused mostly on what citizen scientists were not, e.g., non-credentialed, non-academic, or non-scientist. Terms such as non-professional or non-scientist are problematic, partly because they exclude professional scientists who participate in citizen science outside their area of expertise (OpenScientist 2011; Edwards 2014) and partly because these terms may result in devaluing participants or their work. Terms more favored in academic discussions of citizen science include ‘participant,’ ‘community member,’ ‘collaborator,’ and ‘partner.’ These terms come with their own problems, either because they are perceived as too academic (e.g., ‘participant’), may not be appropriate in all situations (what community do participants represent?), or may have unfortunate historical usage (collaborator in the sense of collusion with an enemy [OED 2016c]). Anonymity is another aspect that may factor into terminology. Falchi et al. (2016) found that the data from “scientists known to us” (i.e., professionals, but including specifically recruited amateur astronomers) had a greater variance, bias, and number of outliers than data from “scientists unknown to us” (i.e., citizen scientists, but likely including many professional scientist contributors).
We also found some terms that tended to be used for more general audiences, including ‘amateur,’ ‘hobbyist,’ ‘volunteer,’ ‘layperson,’ and ‘general public.’ While these terms may be neutral for most audiences, ‘volunteer’ can sometimes result in unfortunate power dynamics (Tim Vargo, personal communication) as participant contributions or skills may be devalued and may be associated with inaccurate stereotypes (Daniela Soleri, personal communication). Some project leaders avoid the term ‘volunteer’ to make it clear that the project aims to give something back rather than just taking free labor (Riesch and Potter 2013). In contrast, others embrace the term because of its neutrality when applied to credentialed and non-credentialed scientists co-participating in a voluntary fashion on a scientific process or project (e.g., the California Roadkill Observation System, Fraser Shilling, personal communication). Finally, participants or project managers may prefer terms that refer specifically to their project, e.g. ‘Zooites’ associated with Zooniverse projects.
One finding that emerges from our group discussions is that no one term works for everyone or every situation. We therefore share several strategies that could help for choosing terms.
The first strategy is to take an approach of generality. One option here is to simply use terms such as ‘public science’ or ‘citizen science,’ but include some recognition that the terminology may not be settled or that some audiences may find it problematic (perhaps a sentence or two, or a footnote). Another option is simply not to distinguish between the people involved in the project, using the same term for everyone. Several discussants on the email list raised this perspective:
“It is worth helping the barriers [to acceptance of citizen science results] to drop by not contributing to labeling that can lead to second class parsing. So, if we really believe that it is all just science and the actors change in and out … then maybe we should correspondingly name it–science and leave it at that.” (Fraser Shilling)
“It follows then that we don’t really need to make any further distinctions other than certain projects might require certain skills.” (Tim Vargo).
We stress that this approach is intended to provide greater recognition to volunteer contributions, not to give an excuse to ignore them (Cooper et al. 2014).
A second strategy is to take an approach of specificity. When writing about a project, practitioners could use ‘citizen science’ to connect with shared practice, but then define what they mean by the term and/or indicate an alternative term that fits their case better. This strategy is similar to when ‘scientist’ was coined to be a general term after such terms as chemist, physicist, and biologist had already existed; both the umbrella and specific terms are still in use today. One might distinguish among contributors based on characteristics pertaining to the work itself (e.g., anonymous vs. identified, light vs. heavy training, co-designers vs. not) or based on their specific roles (e.g., project leader, analyst, data-collector), regardless of their training, affiliation, or background. This option is attractive in part because it resembles the way that scientists often designate author roles in publications submitted to scholarly journals.
Any of the terms we have collected in this paper might be appropriate for one project but not another, so any set of terms can be used, as long as they are clearly defined. One could refer to citizen scientists by their specific expertise, for example ‘sea ice experts’ when working with that particular set of indigenous knowledge holders in the Arctic (Eicken 2010; Pulsifer et al. 2011). One could also ask the contributors what they want to be called, as different groups may have different preferences. Finally, in some areas, general principles have been produced for conducting research with specific communities, and these groups may publish information on how they want to be called, so the leader of the project should make an effort to research what pre-existing guidance is available (see for example the Convention on Biological Diversity 2011, Interagency Arctic Research Policy Committee 1995).
We hope that these terms and themes will spark a more systematic, ongoing, and inclusive discussion of terminology among the citizen science community. While we report primarily from personal experience, it is worthwhile in the future to conduct a more systematic review, consider how people involved in projects themselves interpret the meaning of terms, how interpretations translate into how participants are treated, and if people would prefer different terms if provided with alternatives. Two questions we raise are: 1) “Who gets to decide what people involved in all aspects of citizen science are called, and why?” and 2) “Who gets to decide what science consists of?” We believe that investigation into these questions is important for citizen science to address power imbalances in knowledge production. As we have pointed out in the theoretical background, terminology can have a profound effect on participants and has the power to include or exclude. For example, using terminology that unintentionally privileges the project leader may run counter to the democratizing intentions of a citizen science project, could influence how participants feel about the activity, and could affect the knowledge that is produced. Many other STS-based questions were raised during our discussions:
“What people are called reflects what they do (methods), how this mobilizes knowledge claims (truths), and what power these claims have in the world. Are the methods used by the differently named actors scientific, or are any engaged in questioning orthodox science? What claims about objectivity, neutrality, or reproducibility go with which forms of naming? In a world where authoritative science exists in symbiosis with other hegemonies, what is at stake in our question of names when it comes to changing the world (at any level)?” (Dan Mcquillan, personal communication)
Another set of investigations could revolve around the question “What are the ethical boundaries of citizen science?”
Future research also may allow us to gain a richer understanding of how language use may persuade or dissuade people from becoming engaged with citizen science. Existing efforts include a scientometric meta-analysis of published articles conducted to understand what citizen science entails (Kullenberg and Kasperowski 2016), and a semantic analysis of citizen science and related terms such as citizen sensing, crowdsourcing, and volunteered geographic information (Comber et al. 2014). Building on this work, a systematic bibliometric study carefully examining how terms have been used over time and in different languages could be illuminating, as could surveys of citizen science participants that explicitly ask them what terms they prefer.
Finally, it is worthwhile to explore how other analogous groups use terms (for example, people identifying as ‘makers’ or ‘hackers’). Terminology in those areas may encounter the same difficulties as we have with ‘citizen science,’ or there could be entirely new challenges illuminated by a wider exchange of ideas regarding terminology. We have found it exciting and generative in our discussions to include a wide range of authors, types of citizen science, geographic regions, and disciplines, and we imagine that broadening the conversation would only enrich our understanding of citizen science.
2Throughout this paper, double quotation marks (“) are used to refer to quotations from other sources. If these quotations are from individuals, italics are also used. Single quotation marks are used to call attention to key terms emphasized in the discussion to indicate that their meaning is not taken for granted.
3For example, we are aware of the Citizen Science Association’s Integrity, Diversity, and Equity working group’s current engagement with discussions on terminology, and we hope that this paper can offer useful context for its work.
4While the term ‘research’ may be understood differently by different populations, it is worth noting that some government agencies, such as the Department of Health and Human Services (HHS) in the US, do offer “official” definitions of research that are connected to regulatory and oversight functions.
5Harris was diagnosed with a rare, untreatable autoimmune disease in 1990, reviewed the published research and came up with a new disease model and treatment approach in 1993 that put his disease in remission within two years. He is now actively involved in research on this proposed disease model, has presented several papers at medical conferences, and is the lead author of a paper published in a peer-reviewed medical research journal.
6Another example: Kim Goodsell developed a rare genetic disease and learned genetics, allowing her to develop a treatment for her disease. She has presented her research at medical conferences and is now co-author on a recent publication in a medical journal. https://mosaicscience.com/story/diy-diagnosis-how-extreme-athlete-uncovered-her-genetic-flaw.
7See for example the Atlas of Community-Based Monitoring in a Changing Arctic where information about dozens of projects and hundreds of participating Arctic communities can be obtained: http://www.arcticcbm.org/index.html.
We acknowledge all of the original participants in the citsci-discussion-l email list for having generated a variety of interesting and inspiring ideas out of which this paper emerged. In particular, thanks go to Tim Vargo, Daniela Soleri, Dan Mcquillan, and Eva Lewandowski for contributing quotes from that discussion to this paper with permission. Comments from the following people led to improvements of the paper: Alice Ndlovu, Pam DiBona, Alessandro Campanaro, Veljo Runnel, Rebecca Jordan, Cleo Woelfle-Erskine, Bruno Strasser, Pen-Yuan Hsing, Jenny Reardon, Rick Bonney, and three anonymous reviewers. MV Eitzel was supported by the United States National Science Foundation under Grant No. 1415130. NSF had no involvement in study design; collection, analysis, and interpretation of data; writing of the paper; or the decision to submit for publication. M Haklay has received funding from the European Research Council (ERC) under the Horizon 2020 research and innovation programme (Grant agreement No. 694767).
The authors have no competing interests to declare.
Arbesman, S. (2012). The Rise of Fractional Scholarship and the Ronin Institute. WIRED Magazine, Available at: https://www.wired.com/2012/05/the-rise-of-fractional-scholarship-and-the-ronin-institute/ [Last accessed May 4th, 2017].
Auer, S.R., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R. and Ives, Z. (2007). DBpedia: A nucleus for a web of open data. The Semantic Web. Lecture Notes in Computer Science 4825: 722–735, 978-3-540-76297-3DOI: https://doi.org/10.1007/978-3-540-76298-0_52
Bäckstrand, K. (2003). Civic science for sustainability: reframing the role of experts, policy-makers and citizens in environmental governance. Global Environmental Politics 3(4): 24–41, DOI: https://doi.org/10.1162/152638003322757916
Barad, K. (2007). Meeting the universe halfway: Quantum physics and the entanglement of matter and meaning In: Duke University Press, DOI: https://doi.org/10.1215/9780822388128
Bonney, R., Ballard, H., Jordan, R., McCallie, E., Phillips, T., Shirk, J. and Wilderman, C.C. (2009a). Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education. A CAISE Inquiry Group Report In: Washington, D.C.: Center for Advancement of Informal Science Education (CAISE).
Bonney, R., Cooper, C. and Ballard, H. (2016). The Theory and Practice of Citizen Science: Launching a New Journal. Citizen Science: Theory and Practice 1(1)DOI: https://doi.org/10.5334/cstp.65
Bonney, R., Cooper, C.B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K.V. and Shirk, J. (2009b). Citizen science: a developing tool for expanding science knowledge and scientific literacy. BioScience 59(11): 977–984, DOI: https://doi.org/10.1525/bio.2009.59.11.9
Carton, L. and Ache, P. (2017). Citizen-sensor-networks to confront government decision-makers: Two lessons from the Netherlands. Journal of Environmental Management 196: 234–251, DOI: https://doi.org/10.1016/j.jenvman.2017.02.044
Castleden, H. and Garvin, T. (2008). Modifying Photovoice for community-based participatory Indigenous research. Social science & medicine 66(6): 1393–1405, DOI: https://doi.org/10.1016/j.socscimed.2007.11.030
Ceccaroni, L., Bowser, A. and Brenton, P. (2017). Civic Education and Citizen Science: Definitions, Categories, Knowledge Representation In: Ceccaroni, L. and Piera, J. eds. Analyzing the Role of Citizen Science in Modern Research. Hershey, PA: IGI Global, pp. 1–23, DOI: https://doi.org/10.4018/978-1-5225-0962-2.ch001
Comber, A., Schade, S., See, L., Mooney, P. and Foody, G. (2014). Semantic analysis of citizen sensing, crowdsourcing and VGI. Proceedings of the AGILE’14 International Conference on Geographic Information Science.
Convention on Biological Diversity (2011). Tkarihwaié: ri – code of ethical conduct to ensure respect for the cultural and intellectual heritage of indigenous and local communities relevant to the conservation and sustainable use of biological diversity In: Montreal, Canada: Secretariat of the Convention on Biological Diversity. Available at: http://www.cbd.int/traditional/code/ethicalconduct-brochure-en.pdf [Last accessed 19 December 2016].
Cooper, C.B., Shirk, J. and Zuckerberg, B. (2014). The invisible prevalence of citizen science in global research: migratory birds and climate change. PloS one 9(9): e106508.DOI: https://doi.org/10.1371/journal.pone.0106508
Dickinson, J.L., Zuckerberg, B. and Bonter, D.N. (2010). Citizen science as an ecological research tool: challenges and benefits. Annual review of ecology, evolution and systematics 41: 149–72, DOI: https://doi.org/10.1146/annurev-ecolsys-102209-144636
Edwards, R. (2014). Citizen science and lifelong learning. Studies in the Education of Adults 46(2): 132–144, DOI: https://doi.org/10.1080/02660830.2014.11661662
Eicken, H. (2010). Indigenous knowledge and sea ice science: What can we learn from indigenous ice users? In: SIKU: Knowing Our Ice. Springer Netherlands, pp. 357–376, DOI: https://doi.org/10.1007/978-90-481-8587-0_15
European Citizen Science Association (ECSA) (2015). “Ten principles of citizen science”. Available at: https://ecsa.citizen-science.net/sites/default/files/ecsa_ten_principles_of_citizen_science.pdf [last accessed May 4, 2017].
Eveleigh, A., Jennett, C., Blandford, A., Brohan, P. and Cox, A.L. (2014). Designing for dabblers and deterring drop-outs in citizen science. Proceedings of the 32nd annual ACM conference on Human factors in computing systems. April, ACM: 2985–2994, DOI: https://doi.org/10.1145/2556288.2557262
Falchi, F., Cinzano, P., Duriscoe, D., Kyba, C.C., Elvidge, C.D., Baugh, K., Portnov, B.A., Rybnikova, N.A. and Furgoni, R. (2016). The new world atlas of artificial night sky brightness. Science Advances 2(6): e1600377.DOI: https://doi.org/10.1126/sciadv.1600377
Follett, R. and Strezov, V. (2015). An analysis of citizen science based research: usage and publication patterns. PloS one 10(11): e0143687.DOI: https://doi.org/10.1371/journal.pone.0143687
Fricker, M. (2007). Epistemic injustice: Power and the ethics of knowing. Oxford University Press, DOI: https://doi.org/10.1093/acprof:oso/9780198237907.001.0001
Geoghegan, H., Dyke, A., Pateman, R., West, S. and Everett, G. (2016). Understanding motivations for citizen science In: Final report on behalf of UKEOF, University of Reading, Stockholm Environment Institute (University of York) and University of the West of England. Available at: http://www.ukeof.org.uk/resources/citizen-science-resources/MotivationsforCSREPORTFINALMay2016.pdf.
Georgalakis, J., Jessani, N., Oronje, R. and Ramalingam, B. eds. (2017). The Social Realities of Knowledge for Development Brighton: IDS/Impact Initiative. Available at: https://opendocs.ids.ac.uk/opendocs/bitstream/handle/123456789/12852/Social_Realities_of_Knowledge_for_Development_FullIssue.pdf?sequence=1.
Goodchild, M.F. (2007). Citizens as sensors: the world of volunteered geography. GeoJournal 69(4): 211–221, DOI: https://doi.org/10.1007/s10708-007-9111-y
Haklay, M. (2013). Citizen Science and Volunteered Geographic Information – overview and typology of participation In: Sui, D.Z., Elwood, S. and Goodchild, M.F. eds. Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI) in Theory and Practice. Berlin: Springer, pp. 105–122, DOI: https://doi.org/10.1007/978-94-007-4587-2_7
Hall, B.K. (2002). Palaeontology and Evolutionary Developmental Biology: A Science of the Nineteenth and Twenty–first Centuries. Palaeontology 45(4): 647–669, DOI: https://doi.org/10.1111/1475-4983.00253
Haraway, D. (1988). Situated knowledges: The science question in feminism and the privilege of partial perspective. Feminist studies 14(3): 575–599, DOI: https://doi.org/10.2307/3178066
Hohn, D. (2016). Flint’s water crisis and the troublemaker scientist. The New York Times Magazine, Available at: http://www.nytimes.com/2016/08/21/magazine/flints-water-crisis-and-the-troublemaker-scientist.html?_r=0 [Last accessed 16 August 2016].
Howe, J. (2006). “The Rise of Crowdsourcing”. WIRED Magazine, Available at: https://www.wired.com/2006/06/crowds/ [last accessed May 10, 2017].
Johnson, N., Alessa, L., Gearheard, S., Gofman, V., Kliskey, A., Pulsifer, P. and Svoboda, M. (2013). Strengthening community-based monitoring in the Arctic: Key challenges and opportunities, A Community White Paper Prepared for the Arctic Observing Summit. Available at: http://staging.eloka-arctic.org/sites/eloka-arctic.org/files/documents/cbm_white_paper.pdf.
Kullenberg, C. and Kasperowski, D. (2016). What Is Citizen Science?–A Scientometric Meta-Analysis. PloS one 11(1): e0147152.DOI: https://doi.org/10.1371/journal.pone.0147152
Lovett, G.M., Burns, D.A., Driscoll, C.T., Jenkins, J.C., Mitchell, M.J., Rustad, L., Shanley, J.B., Likens, G.E. and Haeuber, R. (2007). Who needs environmental monitoring?. Frontiers in Ecology and the Environment 5(5): 253–260, DOI: https://doi.org/10.1890/1540-9295(2007)5[253:WNEM]2.0.CO;2
Lyons, K. (). On the Situated Politics of Analytic Symmetry. Special issue on Engaging decoloniality and decolonization in and at the interface of science and technology studies, Catalyst: Feminism, Theory, Technoscience. In Press.
Marres, N. (2007). The issues deserve more credit pragmatist contributions to the study of public involvement in controversy. Social Studies of Science 37(5): 759–780, DOI: https://doi.org/10.1177/0306312706077367
McTaggart, R. (1991). Principles for participatory action research. Adult Education Quarterly 41(3): 168–187, DOI: https://doi.org/10.1177/0001848191041003003
Miller-Rushing, A., Primack, R. and Bonney, R. (2012). The history of public participation in ecological research. Frontiers in Ecology and the Environment 10(6): 285–290, DOI: https://doi.org/10.1890/110278
Mufson, S. (2016). A Dakota pipeline’s last stand. The Washington Post, Available at: https://www.washingtonpost.com/business/economy/a-dakota-pipelines-last-stand/2016/11/25/35a5dd32-b02c-11e6-be1c-8cec35b1ad25_story.html?utm_term=.0b499ba13e87 (Last accessed 25th April 2017).
OED (2016a). “citizen science”. Oxford English Dictionary, Available at: http://www.oed.com/view/Entry/33513?redirectedFrom=citizen+science#eid316619123 (Last accessed 5th December 2016).
OED (2016b). “citizen, n. and adj”. Oxford English Dictionary, Available at: http://www.oed.com/view/Entry/33513?redirectedFrom=citizen& (Last accessed 31st August 2016).
OED (2016c). “collaborator, n”. Oxford English Dictionary, Available at: http://www.oed.com/view/Entry/36198?redirectedFrom=collaborator (Last accessed 20th October 2016).
OpenScientist (2011). Finalizing a Definition of “Citizen Science” and “Citizen Scientists”. September 3 2011 Available at: http://www.openscientist.org/2011/09/finalizing-definition-of-citizen.html [Last accessed 19 December 2016].
Pulsifer, P.L., Laidler, G.J., Taylor, D.R. and Hayes, A. (2011). Towards an Indigenist data management program: Reflections on experiences developing an atlas of sea ice knowledge and use. The Canadian Geographer/Le Géographe canadien 55(1): 108–124, DOI: https://doi.org/10.1111/j.1541-0064.2010.00348.x
Ramirez-Andreotta, M.D., Brusseau, M.L., Artiola, J., Maier, R.M. and Gandolfi, A.J. (2015). Building a co-created citizen science program with gardeners neighboring a Superfund site: The Gardenroots case study. International public health journal 7(1): 13.
Riesch, H. and Potter, C. (2013). Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions. Public Understanding of Science, : 0963662513497324.DOI: https://doi.org/10.1177/0963662513497324
Roy, H.E., Pocock, M.J.O., Preston, C.D., Roy, D.B., Savage, J., Tweddle, J.C. and Robinson, L.D. (2012). Understanding Citizen Science and Environmental Monitoring. Final Report on behalf of UK Environmental Observation Framework. UK Environmental Observation Framework. Available at: https://www.ceh.ac.uk/sites/default/files/citizensciencereview.pdf.
Sarna-Wojcicki, D., Perret, M., Eitzel, M.V. and Fortmann, L. (2017). Where Are the Missing Coauthors? Authorship Practices in Participatory Research. Rural Sociology, DOI: https://doi.org/10.1111/ruso.12156 In Press.
Sforzi, A., Tweddle, J., Vogel, J., Lois, G., Wägele, W., Lakeman Fraser, P., Makuch, Z. and Vohland, K. (). Citizen science and the role of natural history museums. Book chapter n. 24 In: Haklay, M., Hecker, S., Makuch, Z., Bowser, A., Vogel, J. and Bonn, A. eds. Citizen Science – Innovation in Open Science, Society and Policy. in press.
Shirk, J.L., Ballard, H.L., Wilderman, C.C., Phillips, T., Wiggins, A., Jordan, R., McCallie, E., Minarchek, M., Lewenstein, B.V., Krasny, M.E. and Bonney, R. (2012). Public participation in scientific research: a framework for deliberate design. Ecology and Society 17(2): 29.DOI: https://doi.org/10.5751/ES-04705-170229
Sieber, R.E. and Haklay, M. (2015). The epistemology(s) of volunteered geographic information: a critique. Geo: Geography and Environment 2(2): 122–136, DOI: https://doi.org/10.1002/geo2.10
Silvertown, J. (2009). A new dawn for citizen science. Trends in ecology & evolution 24(9): 467–471, DOI: https://doi.org/10.1016/j.tree.2009.03.017
Snively, G. and Corsiglia, J. (2001). Discovering indigenous science: Implications for science education. Scientific Education 85: 6–34, DOI: https://doi.org/10.1002/1098-237X(200101)85:1
Torres, C.A. (1992). Participatory action research and popular education in Latin America. Qualitative studies in education 5(1): 51–62, DOI: https://doi.org/10.1080/0951839920050107
West, S.E. and Pateman, R.M. (2016). Recruiting and Retaining Participants in Citizen Science: What Can Be Learned from the Volunteering Literature?. Citizen Science: Theory and Practice 1(2): 15. 1–10.
Wiggins, A. and Crowston, K. (2011). From conservation to crowdsourcing: A typology of citizen science. System Sciences (HICSS), 2011 44th Hawaii international conference on (pp. 1–10). January, IEEEDOI: https://doi.org/10.1109/HICSS.2011.207
Yeo, R. (1993). Defining science: William Whewell, natural knowledge and public debate in early Victorian Britain. Cambridge University Press, 27DOI: https://doi.org/10.1017/CBO9780511521515