Start Submission

Reading: Project Categories to Guide Institutional Oversight of Responsible Conduct of Scientists Lea...


A- A+
Alt. Display


Project Categories to Guide Institutional Oversight of Responsible Conduct of Scientists Leading Citizen Science in the United States


Caren Cooper ,

North Carolina Museum of Natural Sciences, and North Carolina State University, Raleigh, North Carolina, US
About Caren
Caren Cooper is Assistant Director of the Biodiversity Research Lab at the North Carolina Museum of Natural Sciences. The Museum’s mission is “to illuminate the interdependency of humanity and nature” and citizen science plays a big role. In 2015, Cooper created a citizen science research lab, The Counter Culture, to promote community participation in avian research and pollution monitoring and mapping. Caren helped develop several citizen science projects at the Cornell Lab of Ornithology including YardMap, NestWatch, Celebrate Urban Birds, and the House Sparrow Project. Caren has over 45 publications, including research papers that relied on data from NestWatch, Project FeederWatch, Christmas Bird Count, Breeding Bird Survey, and My Yard Counts. Caren also collaborates with social scientists to study birdwatchers and citizen scientists to understand land stewardship behaviors. Caren is a Senior Fellow in the Environmental Leadership Program with a focus on science communication. She is writing a non-academic book about citizen science with The Overlook Press. Caren is a blogger with SciStarter, syndicated to Citizen Science Salon at Discover Magazine and CitSci at the PLOS Blogging Network. The Counter Culture and SciStarter host bi-monthly twitter-chats about citizen science. She can be followed on Twitter @CoopSciScoop.
X close

Lea Shanley,

University of Wisconsin, Madison, Wisconsin, US
X close

Teresa Scassa,

University of Ottawa, Ottawa, Ontario, CA
X close

Effy Vayena

Department of Health Sciences and Technology, ETH, Zurich, CH
X close


A growing number of individuals who are not professional scientists are working with professional scientists to contribute to scientific research through a broad spectrum of volunteer activities and roles. To guide the regulatory oversight of scientists carrying out citizen science projects, we draw distinctions among five categories of projects in which volunteer activities and roles vary based on the combination of setting (institutional or not), subject matter (human or not), and the norms and expectations of handling volunteers’ personally identifiable information. Each category has potentially different ethical considerations and forms of institutional (or regulatory) oversight. We identify and assign numeric labels to these categories rather than names to avoid confusion and value-laden connotations regarding terminology. We hope the absence of terminology will initiate conversations and encourage rapid evolution of necessary vocabulary in this area. We focus on Type 4 research, projects led in academia and in which volunteers are not the subject of the research, but provide personally identifiable information with expectations of non-confidentiality. Our preliminary data show that current Type 4 projects generally lack informed consent, and most do not provide details about their handling of personally identifiable data. We identify areas where federal guidelines, as well as existing institutional ethics review protocols for protection of human subjects in research, might be applied to some forms of citizen science in ways that could either support, or inadvertently undermine, the Common Rule (the US regulation regarding protection of human subjects). We illustrate these areas with examples of projects from Cooper’s lab. By highlighting the complex and distinct challenges of responsible conduct with each project type, we urge professional scientists, citizen scientists, regulators, and other stakeholders to jointly determine the type of institutional oversight that will best mitigate risks without stymying innovation and benefits. We hope that this essay will spark a lively discussion and refinement of concepts, research, and improved practices.

How to Cite: Cooper, C., Shanley, L., Scassa, T. and Vayena, E., 2019. Project Categories to Guide Institutional Oversight of Responsible Conduct of Scientists Leading Citizen Science in the United States. Citizen Science: Theory and Practice, 4(1), p.7. DOI:
  Published on 08 Mar 2019
 Accepted on 15 Feb 2019            Submitted on 10 Sep 2018


Globally, citizen science encompasses an enormous range of activities in which millions of people contribute to research in science, technology, engineering, and mathematic (STEM) fields. Often citizen science projects are facilitated by location-aware mobile phones equipped with apps, cameras, social networking platforms, and interactive web-mapping tools (Cartwright 2016). Increasing numbers of professional scientists and research institutions are incorporating citizen science into their research programs, resulting in the formation of professional associations to advance the science of citizen science (Bonney et al. 2014; Storksdieck et al. 2016).

Despite this increase in citizen science activity, however, in the United States the practice has not been subject to significant institutional oversight. While oversight of public participation in research has been developed in the context of medical, health, and psychology research about people (human subjects of research), it has not been developed for projects in which people voluntarily help to advance research on a variety of topics without a human focus. In this article we distinguish categories of citizen science projects to explore whether existing definitions and review processes for human subject research provide appropriate oversight for research by people about non-human animals, the natural and built environment, and space.

The US Food and Drug Administration (FDA) and the US Department of Health and Human Services (HHS) each has a definition of “human subjects research” (HSR). The FDA defines HSR as an experiment in which the experimental objects or controls or patients are humans. The HHS definition, on the other hand, does not rely on the subject matter topic of the research; instead, it states that a human subject is “a living individual about whom an investigator (whether professional or student) conducting research obtains: 1) Data through intervention or interaction with the individual, or 2) Identifiable private information.” Implicitly, the “investigator” in this definition either has or is seeking scientific credentials, and the “living individual” lacks such credentials.

In contrast, citizen science is an open collaboration where members of the public engage in the scientific process as active contributors, collaborators, or co-creators, undertaking activities similar to scientists (Shirk et al. 2012). Many citizen science projects do not fit the FDA definition of HSR, yet in most circumstances they do fit the HHS definition, because project design often involves the collection of personally identifiable information (PII). Thus, institutional or regulatory oversight of responsible conduct in citizen science must often consider the intersection of privacy rights and research protections. Yet, once a project is classified as HSR and regulatory protections are initiated, oversight influences more than simply the handling of PII, and may not recognize different norms and privacy expectations in citizen science compared to traditional HSR.

In HSR, such as in clinical trials of new medications, individuals (often called participants) may enroll voluntarily to be studied by professional scientists. Participants may share information about themselves in response to survey questions or submit to medical testing and other interventions. Traditional HSR in the US has involved serious violations of informed consent despite research codes in existence before the regulations, such as the Nuremberg Code and Declaration of Helsinki. Such violations prompted the development of regulations mandating paternalistic oversight of medical HSR via Institutional Review Boards (IRBs). Later, IRBs expanded to include oversight of social science and psychology research (Schrag 2010).

Citizen science offers ways for people to participate in and contribute to scientific research other than as conventional human subjects. Furthermore, the novel research paradigm of citizen science may raise ethical and other issues because it supports a less hierarchical approach to scientific research than conventional human subjects research. Although scholars have addressed research ethics in related fields (e.g., PPGIS; Picone and Lo Piccolo 2014) and have begun exploring ethical topics in citizen science (e.g., Bowser et al. 2014; Resnik et al. 2015; Rambaldi et al. 2006), most prior work has treated projects as homogeneous in design rather than conceptualizing projects based on nuances of oversight. Thus, we classify projects, summarize issues, identify gaps in institutional oversight within the United States, and propose next steps for beginning to meet these challenges.

Categories of Citizen Science Projects

We begin by identifying five categories of volunteer participation in and contributions to research beyond conventional human subject research based on the criteria in the federal regulations. The Common Rule, the US federal regulation for human subjects research across most federal agencies, requires oversight of research by professional scientists working within the auspices of institutions, thus our first criterion is whether the research is institutionally supported. The next two criteria, based on the HHS definition, are a) whether the research topic is human or non-human, and b) whether the project collects personally identifiable information and, if so, how it is handled (Figure 1). There is a lack of commonly accepted vocabulary to refer to these categories, but rather than assign new definitions to existing terms or invent new terminology, we label classifications by number to avoid confusion or unintended meanings (Figure 1). Two of these categories (Types 1 & 2) are clearly within the scope of existing IRB oversight, a third (Type 5) falls outside of institutional control, and a fourth (Type 3) is rare. We thus focus on Type 4, which covers a broad range of activities and raises novel ethical issues about the responsible conduct of research that does not fit comfortably within existing paradigms.

Figure 1 

Categories of citizen science projects. We categorized projects through several levels, first based on whether they are institutional research, second on whether the subject matter was human subjects or other subjects, and third related to the handling of personally identifiable information (PII).

Type 1

Internet and social media tools enable citizen science projects in which individuals can consent to engage as subjects of research with confidential identity. Examples of Type 1 projects include online surveys or tests (e.g., and apps to understand oneself (e.g., Mood Meter app). Research projects that fit Type 1 often do not use the term citizen science, perhaps because they present primarily as products and services for individuals to learn about themselves, and secondarily as research studies.

Type 2

Individuals can be subjects of research and also play an active role, disclosing their identity and associated data. Examples of Type 2 projects include those in which volunteers help to craft research questions about their personal environmental exposures, actively track and share their own biometrics collected through mobile apps and wearables, or collect and submit their own samples for DNA extraction and sequencing. Conventional oversight has adjusted to situations where research participants become active in studying themselves, because such situations can require new approaches to issues of confidentiality, dissemination of information, assessment of risks/benefits, and community-level informed consent, as covered elsewhere (Brown et al. 2010; Brody et al. 2007; Shore et al. 2015).

Type 3

Volunteers can assist in scientific studies with professional scientists on a variety of subjects of research other than themselves or human subjects generally, and provide no personally identifiable information. Globe at Night is an example of a Type 3 project in which volunteers provide observations without providing their name, email, or address. Other examples of Type 3 projects include those on the Zooniverse platform in which volunteers have and/or develop expertise as they actively learn and contribute text, annotations of specimens in museums and herbariums, and classifications of galaxies and other objects depicted in images, without providing names or email addresses to the online platform. The Zooniverse platform also enables hybrids of Type 3 and 4 projects: A volunteer can choose to experience the same project as either Type 3 or 4. Even when volunteers opt to disclose their identity to the Zooniverse platform, however, their PII is not shared with specific projects in which they enroll.

Type 4

Volunteers can help in research while not being traditional human subjects (i.e., not being the primary subjects of the research themselves), yet, in doing so, provide PII. The PII needed for Type 4 projects tends to be driven either by the management design (requiring names and emails for communications) or the research design (needing geolocations). Type 4 projects tend to rely on volunteers with local-expert knowledge to provide observations, photos, or sensor data, or to undertake collaborative community-based projects with scientists. This type of citizen science has yielded uncountable new discoveries (e.g., Cooper et al. 2014) as well as benefits to citizen scientists, their communities, and the research institutions involved. Studies of the social benefits of citizen science are still in their infancy as an emerging discipline, yet already point to increases in scientific literacy, scientific skills, social capital, community empowerment, and other social goods (NAS 2018; Jordan et al. 2015).

Type 5

Some citizen science projects are organized outside of research institutions and do not involve professional scientists. Type 5 projects have many manifestations. For instance, participant-led research may be health related and conducted by patients, typically using online social media (Vayena and Tasioulas 2013). Lifelogging, also known as self-tracking and Quantified Self, involves the use of wearable sensors and the Internet of Things, typically contributing data, sometimes unwittingly, to proprietary databases. DIYbio involves independent labs run by entrepreneurs and community organizers for those without scientific credentials to carry out genomics and molecular biology research. Amateur naturalists and amateur astronomers conduct scientific investigations independent of employment and institutions. Discussion about the institutional oversight of participant-led research is covered elsewhere in relation to health research involving human subjects (Vayena and Tasioulas 2013, Vayena et al. 2015), and DIYbio has developed ethical codes from conferences in Europe and North America (

Preliminary Assessment of Responsible Research with Type 4 Projects

To begin assessing the state of ethics oversight of Type 4 citizen science projects, we examined a sample of 47 citizen science projects on 38 platforms (details in Supplemental File). The projects had been previously selected as part of a different research endeavor that had no focus on ethics. We found that only 5% of them (2 projects) had the equivalent of informed consent (information about risks, benefits, and rights that were accepted by the user as conditional upon registering for the project); 44% (17 platforms with 25 projects) provided Terms of Service (75% of those dealt primarily with privacy issues, others with liability, and some with copyright in projects in which volunteers shared photographs); and 51% (20 projects/19 platforms) had neither informed consent nor Terms of Service. The two projects with informed consent involved topics related to human bodies (microbial samples from humans and records of mosquitoes biting humans). Thus, our preliminary analysis suggests limited oversight of citizen science projects in which the topic of the research is not people but where the project does involve the collection of personally identifiable information. One explanation for the low occurrence of informed consent is that the cultural context of citizen science is more similar to volunteering on ecological restoration projects, volunteering with charity organizations, or participating in education and outreach programs, than it is to human subject research.

Ethical Issues for Type 4 Citizen Science

Volunteers in Type 4 projects do not fit the FDA definition of human subject research because they are not the topic of the research. However, volunteers in Type 4 projects do fit the HHS definition of human subjects, because these projects involve the sharing of personally identifiable information. Nevertheless, our preliminary assessment shows that Type 4 projects fall through the regulatory cracks due to a gray area of interpretation: No guidelines specific to institutional oversight of citizen science projects exist. Some IRBs might view volunteers in Type 4 projects as human subjects, in either exempt or non-exempt regulatory categories, while others may not designate them as human subjects, viewing them instead as equivalent to student volunteers. Volunteers in Type 4 projects may provide names, emails, and other personally identifiable information such as geo-locations. Thus, although volunteers in Type 4 projects are typically perceived to be at a low risk, they may require protections due to imbalances in power, knowledge, and information, as well as the vulnerabilities of some volunteers and communities when participating in certain projects.

Resnik et al. (2015) reviewed four ethical issues potentially common to all projects: Data quality and validity (and related possibility of fraud), data sharing and intellectual property, conflict of interest, and exploitation. In Table 1, we contrast potential risks commonly assessed when reviewing human subjects research protocols with potential risks for volunteers in Type 4 projects. Not only do volunteers in Type 4 projects not fit the US government definition of human subjects, but the US government definition does not fit them: There is little overlap in the likely ethical and other issues faced by conventional human subjects and volunteers in Type 4 projects.

Table 1

The typical risks tend to be different for conventional human subjects than for volunteers in Type 4 citizen science projects.

Conventional HSR Volunteer in Type 4 project

Physical risks

Discomfort Manipulation from persuasive technology
Pain Risk of harm from data gathering activities

Psychological risks

Anxiety Loss of efficacy by not having access to research data
Sleep deprivation Distress from not having acknowledgement of contribution

Social/economic risks

Embarrassment Justice by not having access to benefits of volunteering
Loss of respect Intellectual property loss while solving problems and sharing ideas
Loss of wages
Loss of confidentiality

Invasion of privacy Loss of geo-privacy when gathering data
Loss of dignity

Five foundational ethical principles have commonly been applied across a range of professions: Respecting autonomy, doing no harm (non-maleficence), benefiting others (beneficence), being just (justice), and being faithful (fidelity) (Beauchamp and Childress 2001). The nuanced relationship between professional scientists and volunteers in Type 4 projects may impose new obligations on professional scientists and their institutions, some possibly construed as beneficence—such as providing training, acknowledging volunteer contributions, sharing research products—and some possibly construed as non-maleficence, such as protecting data privacy and mitigating risks of stigmatization. At the same time, power imbalances between professional scientists and volunteers in Type 4 projects may persist, raising ethical issues of justice and autonomy—such as sharing intellectual property rights (Scassa and Chung 2015). Given the benefits and risks in Type 4 projects, what constitutes responsible practice? What should institutional oversight entail, when should it apply, and who should conduct it? To date, answers to the above questions have been addressed with a variety of responses for the following central issues.

Shared intellectual property

Professional scientists have an obligation to address the issue of intellectual property by clarifying, or negotiating, intellectual property expectations of volunteers in Type 4 projects in a way that is respectful of their contributions and institutional and cultural norms. When research inputs are novel intellectual property, professional scientists have included Type 4 (and Type 2) participants as co-authors (e.g., Eiben et al. 2012). When research outputs are the creation of aggregate effort of the participants, professional scientists have adopted open access publication, open research data, and other open means of dissemination and use, which may be an emerging norm in citizen science (Resnik et al. 2015). Public recognition of participants is counter to placing primacy on the confidentiality of human subject research participants (see examples in Figure 2).

Figure 2 

Examples of Type 4 projects from the Cooper Citizen Science Lab. Volunteers in all projects share personally identifiable information, and NC State University IRB reviewed protocols for each as human subjects research. Projects are in chronological order of their development, which shows that the IRB and Cooper applied a progression of different norms to these projects.

Privacy and confidentiality

According to our preliminary assessment, it is rare for volunteers in Type 4 projects to experience an explicit consent process, and privacy risks are managed with data policies and Terms of Use, if managed at all. This area needs attention because most Type 4 projects lack expert guidance in data policies about appropriate access and use (Bowser et al. 2013). Citizen science projects are developing policy and technological solutions to help volunteers in Type 4 (and Type 2) projects manage their privacy and allow professional scientists to manage the risks of unwitting disclosures (McGuire and Gibbs 2006, Bowser et al. 2014). Privacy risks also are mitigated with increased digital literacy. In the rapidly changing area of online privacy, professional scientists have an obligation to inform volunteers in Type 4 projects of technical details of what personal information may be collected, how it may be used or shared, and what steps participants might take to limit misuse. Professional scientists may need to take additional steps to increase public understanding of the risks. Volunteers in Type 4 projects may be exposed to legal liability risks. As with privacy risks, they should be informed of the potential liability and projects should insure their volunteers.

Codes of ethics

Some citizen science projects have developed their own codes of ethical conduct. To address concerns about exploitation, Galaxy Zoo, a project of the online citizen science hub, created guidelines indicating that volunteers should: a) be treated as collaborators; b) not have their time wasted; and c) not be asked to do tasks that could be performed better by a computer (Prestopnik and Crowston 2012). In the 10 Core Principles of Citizen Science adopted by the European Citizen Science Association (ECSA), volunteers should not be asked to do tasks that otherwise could be accomplished easily by an algorithm. Further, citizen science projects should be designed so that the resulting data, code, applications, and technologies generated through the efforts of Type 4 participants are transparent, open, and accessible to the participants of the project and the broader public. The White House memorandum Addressing Society and Scientific Challenges through Citizen Science and Crowdsourcing (Holdren 2015) offers three core principles for the appropriate use of citizen science: Participating in scientific research should be fully voluntary, meaningful and beneficial to the participants, and acknowledged.

Guiding principles

While guiding principles for protecting research participants and human subject research participants draw on the Common Rule and the Belmont Report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (National Bioethics Advisory Committee, DHEW 1978; Belmont Report 1979; National Bioethics Advisory Committee; Council for International Organization of Medical Sciences), additional or different principles might be more suitable for protecting volunteers in Type 4 projects. For example, the volunteer crisis-mapping community, which includes activities similar to Type 4 projects, developed an ethical framework based on humanitarian principles adopted by the International Federation of Red Cross and Red Crescent Societies. The project Drugs from Dirt ( follows the Convention on Biological Diversity guidelines in handling of genetic resources. Non-professional biotechnologists developed a framework at in collaboration with The Wilson Center in Washington, DC, and others argue for a framework based on the 1948 Declaration of Human Rights (Vayena and Tasioulas 2015). Ethical standards developed by the former Association for Volunteer Administration (Association for Volunteer Administration 2005) and/or codes of ethics developed by the participatory mapping community also could guide citizen science ethics (Rambaldi et al. 2006).

The Institutional Oversight Process

Some ongoing citizen science projects with non-human subjects have been reviewed by the IRB process while others have not. We speculate that the lack of consistency in seeking review is likely a result of lack of guidance, agreement, or clarity on the research role of volunteers in Type 3 and 4 projects, variation in anonymity and confidentiality of personal identifiable data within projects, and a previous lack of language to distinguish different types of projects. It is important for the advancement of citizen science as a credible discipline that scientists, citizen scientists, and other stakeholders come together to deliberate the questions of institutional oversight.

The motivation for institutional oversight for research with human subjects is the need to protect research participants. IRBs or Ethics Review Committees assess the balance of risks and benefits, the characteristics of informed consent process, and the quality and competence of the researchers. US Federal legislation and international guidelines require researchers to obtain ethics approval for studies with human subjects (Council of International Organizations of Medical Sciences (CIOMS) 2002; US Department of Health, Education, and Welfare 1979). Unfortunately, some instances of IRB review have involved undue burden and are often viewed as overly paternalistic, even to the point of putting patients at risk (Edwards et al. 2004, Whitney and Schneider 2011, Chalmers 2011).

There is need to develop criteria that determine a) the necessity for IRB review or other external oversight mechanisms, b) what oversight should entail, c) what additional categories of projects may simplify the process, and d) what group would most appropriately carry it out. We suggest that any formal institutional option for oversight be flexible, adapting to the unique qualities of volunteers across types of projects and the cultural contexts of the communities involved, to prevent unnecessary bureaucratic delays. A formal, inflexible oversight review process may stymie both the dynamic and innovative nature of citizen science and the potential for professional scientists and volunteers in Type 4 projects to collaboratively develop their own normative principles.

We emphasize that those in Type 4 projects are defined as individuals who are volunteer contributors in a research role and are not themselves the topic of the research (Figure 1). As demonstrated in decades of community-based participatory research with human subject research participants, IRBs can inadvertently undermine beneficence, the ability of researchers to maximize overall good impacts (Brody et al. 2007). For example, conventional IRBs recommend against sharing results if there is any uncertainty in their interpretation; in this case, the traditional approach prioritizes the researcher’s role in minimizing potential harms, rather than participants’ ability to autonomously process information that is complex and uncertain. In contrast, in citizen science projects that take the form of community-based participatory research (CBPR) (Wilson et al. 2007; Minkler et al. 2003) or community-owned and community-managed research (COMR) (Wilson et al. 2008; Heaney et al. 2007), an ongoing non-hierarchical relationship between scientists and volunteers is essential for establishing trust, which is the basis for effective communication, and which allows approaches in which even uncertain results are reported back without causing harm (Brody et al. 2007).

More generally, IRBs may be inherently ill-suited for reviewing citizen science research: In a content analysis of ethics review processes, Flicker et al. (2007) found that IRBs tend to operate through a lens of biomedical research and do not appropriately accommodate CBPR.

For example, typical IRB restrictions on report-backs prioritize human subject confidentiality over access to information, but report-backs for volunteers in Type 2 and 4 projects are actually designed to reinforce Belmont principles of beneficence and respect for persons (Brody et al. 2007; Shore et al. 2015; Morello-Frosch et al. 2009). In all forms of citizen science, report-backs lessen knowledge disparities between participants and professional scientists.

We are concerned about the option of expanding the current IRB model beyond the context of human subject research participants and Type 2 volunteers. Even though the relationship between professional scientists and volunteers in Type 4 projects may reflect some power imbalances, the risks may be managed or mitigated through best practices, as in examples above (e.g., report-backs, open data, deeper engagement in research process, terms of service agreements, Zooinverse ethics). The popularity, interdisciplinarity, and novelty involved in Type 4 projects calls for broad discussion among stakeholders to further scope the range of benefits and risks, and options for institutional oversight.

Responsible Citizen Science: A Call for Dialogue

The ever-increasing role that volunteers in Type 4 projects will play in institutional research should urge professional scientists, citizen scientists, and other stakeholders to address ethical issues and related challenges in citizen science. Such a dialogue can take place under the auspices of stakeholder groups such as the Citizen Science Association and the US Federal Community of Practice on Crowdsourcing and Citizen Science, among others. We call for the development of: a) a framework to guide projects to different levels of institutional oversight based on their design, the role of volunteers, the handling of PII, and community context, and b) a Code of Ethics for the field of citizen science and/or a Citizen Scientists’ Bill of Rights explicitly stating general expectations of professional scientists and citizen scientists. Because Type 4 projects are highly varied in design and activities, and raise different ethical and related issues from projects with research participants (human subjects), we do not recommend a one-size-fits-all approach of institutional oversight. Dialogue on this matter is urgently needed. Proactive engagement will help to create a process that will help make Type 4 projects responsible, robust, and successful.

Supplementary File

The supplementary file for this article can be found as follows:

Supplementary Material

Citizen science projects reviewed for study on volunteer categories to guide ethics of professional conduct in citizen science. DOI:


We thank M. Edwards, D. Quigley, and L. Rasmussen for edits and discussion, and M. Sharova and R. Woodbridge for help with gathering preliminary data. This manuscript arose from student-inspired conversations at the Vespucci Institute summer school and was completed with help from the National Science Foundation CCE-STEM Award #1835352.

Competing Interests

The authors have no competing interests to declare.


  1. Association for Volunteer Administration. 2005. Professional Ethics in Volunteer Administration. Council for Certification in Volunteer Administration. 

  2. Beauchamp, TL and Childress, JF. 2001. Principles of Medical Ethics, Oxford University Press. 

  3. Bonney, R, Shirk, JL, Phillips, TB, Wiggins, A, Ballard, HL, Miller-Rushing, AJ and Parrish, JK. 2014. Next steps for citizen science. Science, 28: 1436–1437. DOI: 

  4. Bowser, A, Wiggins, A, Shanley, L, Preece, J and Henderson, S. 2014. Sharing data while protecting privacy in citizen science. Interactions Jan–Feb, 70–73. DOI: 

  5. Bowser, A, Wiggins, A and Stevens, RD. 2013. Data policies for public participation in scientific research: A primer. DataONE Public Participation in Scientific Research Working Group, Aug. 

  6. Brody, JG, Morello-Frosch, R, Brown, P, Rudel, RA, Altman, RG, Frye, M, Osimo, CA, Pérez, C and Seryak, L. 2007. “Is it safe?”: New ethics for reporting personal exposures to environmental chemicals. American Journal of Public Health, 97: 1547–1554. DOI: 

  7. Brown, P, Morello-Frosch, R, Brody, JG, Altman, RG, Rudel, RA, Senier, L, Pérez, C and Simpson, R. 2010. Institutional review board challenges related to community-based participatory research on human exposure to environmental toxins: A case study. Environmental Health, 9: 39. DOI: 

  8. Cartwright, J. 2016. Technology: Smartphone science. Nature, 531: 669–671. DOI: 

  9. Chalmers, D. 2011. Viewpoint: are the research ethics committees working in the best interests of participants in an increasingly globalized research environment? J Intern Med, 269: 392–395. DOI: 

  10. Cooper, CB, Shirk, J and Zuckerberg, B. 2014. The invisible prevalence of citizen science in global climate change research. PLoSONE, 9(9): e106508. DOI: 

  11. Council for International Organization of Medical Sciences. ( 

  12. Council of International Organizations of Medical Sciences (CIOMS). 2002. International ethical guidelines for biomedical research involving human subjects. Geneva: WHO. 

  13. Edwards, SJ, Kirchin, S and Huxtable, R. 2004. Research ethics committees and paternalism. J Med Ethics, 30: 88–91. DOI: 

  14. Eiben, CB, Siegel, JB, Bale, JB, Cooper, S, Khatib, F, Shen, BW, Players, F, Stoddard, BL, Popovic, Z and Baker, D. 2012. Increased Diels-Alderase activity through backbone remodeling guided by Foldit players. Nature Biotechnology, 30: 190–194. DOI: 

  15. Flicker, S, Travers, R, Guta, A, McDonald, S and Meagher, A. 2007. Ethical dilemmas in community-based participatory research: Recommendations for Institutional Review Boards. Journal of Urban Health: Bulletin of the New York Academy of Medicine, 84(4): 478–493. DOI: 

  16. Heaney, C, Wilson, SM and Wilson, OR. 2007. The West End Revitalization Association’s Community-Owned and -Managed Research Model: Development, Implementation, and Action. Progress in Community Health Partnerships: Research, Education and Action, 1(4): 339–350. DOI: 

  17. Holdren, JP. 2015. Memorandum to the Heads of Execuitve Departments and Agencies: Adressing Societal and Scientific Challenges through Citizen Science and Crowdsourcing, September 30, 2015. 

  18. Jordan, R, Crall, A, Gray, S, Phillips, T and Mellor, D. 2015. Citizen science as a distinct field of inquiry. BioScience, 65: 208–211. DOI: 

  19. McGuire, AL and Gibbs, RA. 2006. No Longer De-Identified. Science, 312: 370–371. DOI: 

  20. Minkler, M and Wallerstein, N. 2003. Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass. 

  21. Morello-Frosch, R, Brody, JG, Brown, P, et al. 2009. Toxic ignorance and right-to-know in biomonitoring results communication: A survey of scientists and study participants. Environmental Health, 8: 6. DOI: 

  22. National Academies of Sciences, Engineering, and Medicine. 2018. Learning Through Citizen Science: Enhancing Opportunities by Design. Washington, DC: The National Academies Press. DOI: 

  23. National Bioethics Advisory Committee. ( 

  24. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Department of Health, Education and Welfare (DHEW) (30 September 1978). 

  25. Picone, M and Lo Piccolo, F. 2014. Ethical E-Participation: Reasons for Introducing a ‘Qualitative Turn’ for PPGIS. International Journal of E-Planning Research, 3(4): 57–71. DOI: 

  26. Prestopnik, NR and Crowston, K. 2012. Citizen Science System Assemblages: Toward Greater Understanding of Technologies to Support Crowdsourced Science. iConference. Toronto, ON, Canada, 8–11 February, 2012. 

  27. Rambaldi, G, Chambers, R, McCall, M and Fox, J. 2006. Practical ethics for PGIS practitioners, facilitators, technology intermediaries and researchers. Participatory learning and action, 54(1): 106–113. 

  28. Resnik, DB, Elliott, KC and Miller, AK. 2015. A framework for addressing ethical issues in citizen science. Environmental Science & Policy, 54: 475–481. DOI: 

  29. Scassa, T and Chung, H. 2015. Managing Intellectual Property Rights in Citizen Science: A Guide for Researchers and Citizen Scientists, Wilson Center Commons Lab, Research Series Vol 03. 

  30. Schrag, ZM. 2010. Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965–2009. Johns Hopkins University Press. 

  31. Shore, N, Ford, A, Wat, E, Brayboy, M, Isaacs, M, Park, A, Strelnick, H and Seifer, SD. 2015. Community-Based Review of Research Across Diverse Community Contexts: Key Characteristics, Critical Issues, and Future Directions. American Journal of Public Health, 105: 1294–1301. DOI: 

  32. Storksdieck, M, Shirk, JL, Cappadonna, JL, Domroese, M, Göbel, C, Haklay, M, Vohland, K, et al. 2016. Associations for Citizen Science: Regional knowledge, global collaboration. Citizen Science: Theory & Practice, 1(2): 10. DOI: 

  33. US Department of Health, Education, and Welfare. 1979. The Belmont Report: Ethical principles and guidelines for the protection of human subjects of research Report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. Available: 

  34. Vayena, E, Brownsword, R, Edwards, SJ, Greshake, B, Kahn, JP, Ladher, N, Montgomery, J, O’Connor, D, O’Neill, O, Richards, MP, Rid, A, Sheehan, M, Wicks, P and Tasioulas, J. 2015. Research led by participants: A new social contract for a new kind of research. Journal of Medical Ethics. Published Online First: March 30, 2015. DOI: 

  35. Vayena, E and Tasioulas, J. 2013. Adapting standards: Ethical oversight of participant-led health research. PLOS Medicine, 10(3): e1001402. DOI: 

  36. Vayena, E and Tasioulas, J. 2015. “We the Scientists”: A human right to citizen science. Philos. & Technol, 28: 479–485. DOI: 

  37. Whitney, SN and Schneider, CE. 2011. Viewpoint: A method to estimate the cost in lives of ethics Med, 269: 396–402. 

  38. Wilson, OR, Bumpass, NG, Wilson, OM and Snipes, MH. 2008. The West End Revitalization Association (WERA)’s right to basic amenities movement: Voice and language of ownership and management of public health solutions in Mebane, North Carolina. Progress in Community Health Partnerships: Research, Education, and Action, 2(3): 237–243. DOI: 

  39. Wilson, SM, Wilson, OR, Heaney, CD and Cooper, C. 2007. Use of EPA Collaborative Problem-Solving Model to Obtain Environmental Justice in North Carolina. Progress in Community Health Partnerships: Research, Education and Action, 1(4): 327–338. DOI: