Citizen science goes by many names, especially when citizen science is applied to healthcare and biomedicine: There are patient researchers, patient-driven innovators, and more. Critiques of the term citizen science point out that “citizen” may not be an inclusive term, especially for those underrepresented in the scientific enterprise (Cooper et al. 2021), and this may be true in health and healthcare as well. The term citizen science itself is mostly often used inwardly facing (Eitzel et al. 2017). In healthcare, there may be as many participant-led (patient-led) projects in citizen science as there are traditional scientist-led projects, although estimates are challenging since many performing personal science or “do-it-yourself” efforts may not identify their work as citizen science. While citizen science projects outside of healthcare may be perceived as citizen science projects primarily because of participant contributions to data collecting (Elliott and Rosenberg 2019), in healthcare this is simply the traditional research model where patients are participants in clinical trials and studies in which they contribute data as participants (traditionally “subjects”). Instead, in health-related citizen science, the term citizen science is more often used to describe projects designed or led by patients (or caregivers) themselves, followed by collaborative “patient-centered” or “patient-involved” projects, rather than traditional researcher-led projects. Citizen science in healthcare can outpace the traditional medical establishment in knowledge creation (Wicks and Little 2013), and can also address limitations of traditional science itself, such as the current method of recruiting study participants through healthcare services that, as a result, excludes many people from participating and makes the output of such studies unrepresentative and nongeneralizable (Alwan 2021).
Citizen science can be effective at leveraging patient-generated knowledge in healthcare to discover new ideas or new ways to solve existing problems, or to generate new knowledge about diseases or management of conditions. One common example is that of a patient-designed automated insulin delivery (AID) system, which was created years before any AID system was available commercially. Through dissemination at scientific conferences (Lewis, Swain, and Donner 2018) and publications (Lewis and Leibrand 2016) about the efficacy of this patient innovation, the open source AID gained attention and collaborations grew, resulting in the world’s first randomized control trial of open source AID (Burnside et al. 2022), which further validates citizen science knowledge about the safety and efficacy of the algorithm and technology that has been used by thousands globally, despite the lack of regulatory approval or distribution by a manufacturer. This particular example is one in which dissemination played a key role in generating collaboration for further scientific research on relevant topics of importance to the community of people living with diabetes. This was also an intentional strategy by the community to disseminate their knowledge to healthcare providers and researchers through these channels, alongside parallel strategies to reach other people living with diabetes through different channels.
Citizen scientists in health-related research may find themselves facing many types of barriers for disseminating their knowledge through traditional channels. Some of the barriers are concrete; other barriers are perceived but no less significant. Citizen scientists may have to choose which parts of the boundary of traditional science to “bridge” (Ottinger 2010), and which to walk around. For example, in traditional research, a protocol is typically clearly defined and reviewed by an institutional review board prior to implementation. Given that much of citizen science involves iterative co-creation, this is at odds with standard ethical oversight procedures, and lack of traditional ethical oversight may itself be a perceived barrier to publication (Tzovaras et al. 2021), given that many publications will state it is a requirement for submission. However, unlike in conventional research, there is not necessarily a scientific or clinical institution that exposes participants to risks for the institution’s benefit (Fiske et al. 2019), so it is less clear to citizen scientists whether this is a hard requirement or could be walked around by instead including descriptions of the ethical community review and consent process to fulfill a journal’s review requirements when submitting to a traditional journal.
Additionally, some of the barriers to citizen science knowledge dissemination may be more concrete. Citizen science studies or documentation of such studies may be more likely to reflect interdisciplinary findings. Such interdisciplinary studies may be harder to publish (which may be perceived to be true of interdisciplinary work (Bammer 2016) outside of citizen science as well) and may attract more criticism, as one citizen scientist group found, perceiving they were a “less good fit” for the journals they sought to publish in, and that the publishing process itself was “very demanding” (Gadermaier et al. 2018). Similarly, in the past, traditional academic researchers perceive citizen science–related work is unlikely to lead to a high production of articles in refereed journals (Baum, MacDougall, and Smith 2006), so this lack of perceived fit is not unique to the citizen scientists themselves.
Citizen scientists face numerous barriers in both performing and disseminating their research, particularly through traditional scholarly research publications and in the healthcare space, although much of the literature focuses primarily on barriers to performing research itself. Although patient-led citizen science efforts have succeeded with dissemination of knowledge in some fields such as the study of diabetes (Asarani et al. 2021) or of Parkinson’s disease (Riggare 2020), or more recently “long COVID” (Crouch, Viduto, and Zangari 2022), the barriers are diffuse and diverse, such that likely only a small minority of patient researchers’ knowledge, findings, and insights ever become available in the research literature, although there have not been estimates to quantify how much is missed. However, it can be predicted or estimated that much of what is known by the true experts living with diseases and conditions does not become known by researchers and healthcare providers and practitioners. As such, opportunities to further evaluate innovations, resources, and new scientific findings that could change the management and treatments of some conditions are missed. Science and healthcare suffer as a result of this overlooked or unseen knowledge.
There is much to be learned from those who have overcome or worked around these barriers in healthcare, but little has been documented about the barriers faced by citizen scientists, their strategies, and potential solutions, specifically with regards to dissemination. While research has addressed lack of interest in certain groups of citizen scientists (Ganzevoort et al. 2017) to pursue publishing and dissemination, the previous examples described (such as open source AID for people with diabetes) and further described below highlight some of the differences in health and biomedicine, where dissemination effectively furthers and expands the work. Further research could expand on the effectiveness of such strategies and how they differ in health and biomedicine compared with other fields of citizen science.
To aid citizen scientists in health and biomedicine, this essay seeks to summarize some of the barriers to dissemination, in part so citizen scientists know these are legitimate barriers and they are not the only ones facing them, and also so that traditional researchers who want to support citizen scientists can identify strategies for addressing such barriers. This includes some of the barriers to doing research, such as lack of data, time, money, and skills needed to process and analyze data or perform other scientific experiments. If those are overcome, there are then numerous barriers to dissemination, such as the time it takes to format and publish in journals, and the publication costs. Additionally, the inability to address concrete or perceived institutional and publication gatekeeping expectations, such as lack of IRB, self-reported or community-tracked data, and a lack of traditional credentials, prevent many patient innovators from ever publishing their work in the scientific literature.
This essay therefore highlights the concrete and perceived barriers that citizen scientists face both in performing the work and in disseminating their work in healthcare, and describes some of the successful strategies citizen scientists have used to overcome them.
Barriers to citizen science in general have been fairly well documented, but there are some particular aspects of biomedicine and healthcare worth highlighting that may be unique to other fields in which citizen science may be done. Because citizen scientists in biomedicine and healthcare are more likely than in other fields to have a personal connection to the topic (e.g., a citizen scientist is a patient themselves, or a caregiver or a loved one of someone with lived experience), they may be more likely to lead and to perform the science and research themselves, rather than being only a data collector or a contributor. As a result, this provides numerous barriers that may (but not always) challenge citizen scientists.
For example, citizen scientists come from all walks of life and all backgrounds. To do scientific discovery and research, they may need to learn new skills such as data analysis and other scientific processes. Additionally, access to software, tools, or technology to do the research or analysis may be an issue. Citizen scientists may face cost barriers or knowledge barriers (e.g., knowing what tools exist). For example, a thematic analysis software commonly used by academic or traditional researchers, NVIVO [QSR International, Doncaster], was $956 USD for an individual, non-academic use license (Pricing Costs for NVivo). Citizen scientists may not be aware of alternative approaches, such as how to perform hand-coding of thematic analysis (high time cost but free) or alternative solutions such as Taguette, a free and open-source software for qualitative analysis (Rampin and Rampin 2021).
Additionally, data itself can be a barrier. Often, the citizen science work focuses on questions that are not addressed by traditional researchers, so data often does not yet exist and may need collecting. This starts with figuring out where to source or collect data ethically, which may include community survey data or data from medical devices, software, or wearables. Citizen scientists must then determine when and how to store data, including updates or additional data submissions. Traditional researchers may have labs or institutional storage options; citizen scientists must face the cost and establish processes themselves for local and/or cloud storage of the data. The process of accessing or updating data may also involve learning new technology, whether it be command line access to data (such as is needed to access projects with data on the Open Humans platform—for example, Greshake Tzovaras et al. 2019) or other technology. The size of the data may be much larger than traditional researchers’ data sets. In the diabetes field, “big data” is often published on when it includes 2–4 weeks of data by a few dozen people (such as in (Mosquera-Lopez et al. 2019; Rodríguez-Rodríguez et al. 2019); whereas the OpenAPS Data Commons, an anonymous dataset collected from and by the diabetes community of open source automated insulin delivery system users, contains more than 42,000 days’ worth of data (Papadopoulos, Salinas, and Crump 2021).
Once data is analyzed, writing itself can be a barrier when the target publication is a traditional academic or medical journal. Scientific writing is a particular skillset that citizen scientists may not already have, and it may be challenging. Challenges include constructing a manuscript for a journal’s publication process. (For example, this journal has a 35-page Author Guidelines document as of April 7, 2022 [Citizen Science: Theory and Practice, “Submissions Guidelines”].) If citizen scientists are collaborating with traditional scientists, they can be aided in writing and/or shaping their manuscript; but they may perceive that they cannot or should not solo author their publication. Similarly, conference or journal requirements for an IRB or ethical review board may present a barrier to citizen scientists. In some cases, this is not a hard requirement, and citizen scientists can demonstrate ethical research in other ways, but sometimes not saying “yes” to IRB review causes automatic rejection within the system. In other cases, IRB is irrelevant to the work being performed (e.g., due to retrospective analysis on anonymized data that is not required to have IRB approval in the US [Wilkins 2021]), but citizen scientists may not be aware of this and therefore do not complete the work or write it up for dissemination as a result. Similarly, it can be challenging to identify an appropriate journal where the work may fit.
Table 1 summarizes the types of barriers that citizen scientists may face more often in biomedicine and healthcare.
|Skills and tools||May need to learn new skills|
|May not have access ($) to software or analysis tools|
|Data||Data may not exist (yet) and may need collecting|
|Must figure out where and how to store data, including updates/new data submissions|
|Data storage may involve cost ($)|
|Data storage may involve learning new technology|
|Preparing for dissemination or publication||Requires written communication skills, sometimes in languages that are not the primary language|
|It takes time and effort to write up results for dissemination|
|Traditional publication submission process costs time and is burdensome; in some cases it costs money ($)|
(not concrete barriers but may still structurally appear to be concrete)
|Conferences or publications may cite a “requirement” of an IRB or ethical review board that is not a hard requirement and/or is not required for the type of research performed (e.g., retrospective, anonymized data analysis on community-donated data)|
|Conferences or publications are designed for credentialed submitters from institutions; submission may involve copious use of “n/a – not applicable” in credential or degree fields; institution; location; and other fields.|
In some cases, systems are designed to auto-reject or return submissions if they do not have a validated institution (e.g., many journal publication systems use Ringgold to validate institutions)
As a result of all these barriers, there may be any number of reasons why citizen scientist work is underrepresented in traditional journals and scientific conferences.
The process of writing takes time away from the research or work itself. Given the time cost of submitting to journals, it may or may not be perceived to be worth it. This is especially true if it significantly distracts the citizen scientist or detracts from their ongoing work to support themselves or the community that their work is focused on helping. If they tackle the cost of writing and submitting to a journal, they may receive a rejection and choose to resubmit, and be faced with the time cost of reformatting the metadata of their manuscript and the formatting of references, let alone other content changes to adjust the fit to a new journal. This is not a challenge unique to citizen scientists as traditional researchers also face these challenges; but given their work is not often funded, this can be one barrier too many. Similarly, there is often a financial cost to submit to journals or conferences. In some cases, scientific conferences require a fee to submit an abstract, even if it is not accepted. Journals, too, in some cases have submission fees, or more commonly article publishing charges (APCs) or open access (OA) charges for journals. Citizen scientists must often decide whether to self-pay these costs out of pocket to see their work disseminated.
Even if the cost and time barriers are addressed, there are still barriers left for citizen scientists to hurdle to achieve dissemination of their work in traditional channels. As previously described, citizen science is often multidisciplinary and may be already perceived to be of lower quality and “not a fit” for journals. A further challenge is that editors determine what is “important” in a field and is therefore prioritized for journal publication. There may be a disconnect between what patients and citizen scientists feel is important and what an editor—often a traditional researcher or clinician—assesses to be worth publishing. It can be an uphill battle for citizen scientists to even get their manuscripts over the first barrier of having their submission sent to peer review. Peer reviewers then may potentially also have similar subconscious or conscious biases regarding citizen science–related or –driven work, which can influence the outcome. And again, at the end of the process, even with positive peer review, an editor can decide not to publish if they decide the work doesn’t fit or is not a priority for the journal.
Another challenge is preprints. Citizen scientists may be more likely to post preprints and get contributions from fellow citizen scientists and early review and input from traditional researchers. However, preprints are still not widely allowed, and when they are allowed, some journals and/or editor teams are biased against research published in a preprint server. Additionally, preprints are not yet effective enough to receive the benefits of an audience that comes along with a peer-reviewed, published journal article, and/or the credibility for future citations of the work.
Many of these challenges are not unique to citizen scientists and are similar to barriers facing traditional researchers, especially newer career and under-represented researchers who are addressing previously overlooked or ignored areas of research. However, the barriers may be magnified and exacerbated for citizen scientists, who face all of the same barriers and challenges in addition to the magnification of the cost and time barriers, and the lack of institution and credentials while attempting to work in systems designed for those with institutions and credentials (for example, an entry for the “Institution” field within the journal management system is required to submit a paper to this journal, as is true for many other journals). While citizen scientists, as I often do, can enter “N/A” into the field, in some journal systems it is designed to auto-return the submission to the author because the institution (“N/A” or anything else) isn’t verified by their system as an institution. This typically requires ongoing communication with editorial assistants to resolve, and adds extra work to the submission process.
Table 2 summarizes some of these barriers as they apply to disseminating citizen science knowledge in biomedicine and healthcare areas.
|Time||It takes time to prepare a publication for submission, and if rejected, resubmission requires significant formatting changes even if the content of the manuscript stays the same|
|Money||There are often article publishing charges (APCs) or open access (OA) charges for journals|
|Gatekeeping||Priority about what is important is determined by those the system is designed for—traditional researchers and, in the case of biomedicine and healthcare, clinicians. There may be a disconnect between the research priorities of citizen science researchers and those of traditional researchers|
|There may be too big of a gap between what is being done or has been done, and the current work—or so perceived by the gatekeepers of publications if the current work submitted is not clearly linked to existing literature|
|Perceptions||While pre-prints are sometimes allowed (but sometimes not), some journal editors are biased toward research that is available in a pre-print server|
|Citizen science publications may face concerns and biases around self-reported or -collected data, even when there is no other data source established|
|Reviewers may recommend rejection of an article and suggest instead it appear on a patient website rather than in a journal, based on conscious biases around what should appear in a journal. Similarly, lack of credentials and institution may influence both editor and reviewer behavior.|
While multiple multi-faceted barriers face citizen scientists who seek to disseminate their work, there are lessons to be learned from those who have successfully done so.
As described in the introduction, there are many names and terms for citizen-science related work. This could involve quantified self (QS) or self-tracking work; personal science, n-of-1, self-research; community science, participatory science, community-driven research, community-based participatory research (CBPR), or other terms. Specifically in biomedicine and healthcare, indicative terms may include patient-driven, patient-innovator, patient-researcher, user driven, or similar. In some cases, the work itself may be citizen science yet not use any of these terms, so it may be hard to “see” other work that fits into this category.
For example, a recent scoping review by Reinius et al. (2022) focused on patient-driven innovation, using a broad definition of innovation and a narrow definition of patient driven, searching between 2008 and 2020 and finding 96 articles on 20 patient-driven innovations. (This of course would not include research that is not perceived as an innovation, which is highlighted in the limitations of Reinius et al.’s paper; therefore, this scoping review should not be used to assess the quantity or scope of success of citizen science work in healthcare.) Of these, open-source automated insulin delivery in diabetes and a platform, PatientsLikeMe, accounted for half of these articles. These therefore provide examples of strategies and lessons that other citizen scientists could leverage for disseminating their work.
There are numerous research publications related to PatientsLikeMe, for example (PatientsLikeMe Research Manuscripts Bibliography 2019). PatientsLikeMe drives individual, n-of-1 decision-making (Wicks et al. 2010), and citizen science at that level may not make it to many publications, because there is no incentive for many individuals to publish, nor may there be enough interest from traditional channels to publish such work. Despite this, PatientsLikeMe as a platform is cited in numerous publications on topics ranging from ALS (Frost and Massagli 2009), epilepsy (de la Loge et al. 2016), organ transplant (Wicks, Sulham, and Gnanasakthy 2014), to other topics. Because PatientsLikeMe is an online platform hosted by a company, they have had resources that the community of online platform participants have been able to leverage in the pursuit of disseminating knowledge from their online sub-communities. For example, Paul Wicks led the R&D platform for 13 years at PatientsLikeMe (Paul Wicks, no date), and co-authored many PatientsLikeMe publications. Given his employment by PatientsLikeMe, his publication track record shows he was an effective collaborator for communities in disseminating knowledge through traditional channels, especially in advocating that “science and research are a set of tools and thinking methods that can be applied by anyone” (Wicks 2018). This demonstrates that newer or internet-based organizations may be effective partners and collaborators for disseminating citizen science work. They may provide resources such as collaborators, funding for publications, and mentorship or other skill resources to aid a citizen scientist’s growth.
Not all successful citizen science projects involve organizations or institutions, though, and they are no less successful at dissemination. Nightscout, an open-source remote monitoring platform, was developed by a group of people living with diabetes and their loved ones (such as parents and partners) to enable remote monitoring of continuous glucose monitoring (CGM) data, years before remote monitoring was commercially available (Rivard, Lehoux, and Alami 2021). Nightscout was developed by individuals with no organizational involvement. However, it has frequently been written about in publications (as seen in Reinius et al.’s (2020) scoping review).
Similarly, the open-source automated insulin delivery (AID) systems were developed without organizational involvement (the first of which was OpenAPS [Lewis 2019]) and are also widely referenced in the literature. In Reinius et al.’s (2020) scoping review, it was identified in the list of patient innovation publications as the most widely published on disease-specific innovation. Initial publications and dissemination focused on its existence (Lewis and Leibrand 2016) and the novelty of patient-driven innovation (Lewis, Swain, and Donner 2018); subsequent publications expanded the focus to demonstrate advanced algorithm development (Lewis 2021), user-driven feature importance in AID development (Lewis et al. 2018), and overall scientific discovery in areas of diabetes that were newly exposed as a result of the rich, complex data streams created from users of these open source AID systems (Grant, Lewis, and Kriegsfeld 2021).
Why were articles on Nightscout and OpenAPS (and other open-source automated insulin delivery systems) so often published? These systems happen to be technology related to diabetes, which is a common condition and widely understood. The technology of remote monitoring and automated insulin delivery was not new, and eventually commercial options have come to market, so in addressing publication barriers like prioritization it may have been easier to tie this innovation and other research topics to existing work pointing to the directions of diabetes technology, as well as describing the change from the status quo, which citizen scientists often do effectively.
Additionally, because diabetes is a large topical area, there are likely more collaborators among traditional researchers who understand the problem or gap areas and are willing to support, mentor, or contribute to tackling some of the barriers citizen scientists face. The first publication on OpenAPS was from an invitation to submit a letter to the editor (Lewis and Leibrand 2016); subsequent publications were easier because there was an existing publication to cite that demonstrated the tie to existing work.
Similarly, invitations to submit articles to special issues or collections of journal publications may be an effective opportunity for citizen scientists to leverage. Special issues may be likely to be designed and led by editors and reviewers with pre-disposed interest in the topic areas who are more amenable to constructive review and to mentorship to support citizen science work including addressing barriers to the publication process.
Not all citizen science work will happen to fit into an established or common disease area such as diabetes. However, that does not mean that citizen science will not be able to be disseminated. In some cases, timely media attention could be leveraged to increase awareness, attention, and knowledge of editors and reviewers, which could aid in mitigating some of the biases and/or prioritization barriers that might otherwise exist. Long COVID-19 may be an example of this; there has been some successful citizen work by those experience long COVID-19, and they in fact have led the definition of “long COVID” as a disease or condition itself (“Long COVID: let patients help define long-lasting COVID symptoms,” 2020).
Additionally, leveraging mainstream media attention could make it easier to find interested collaborators among traditional researchers, especially when a disease is rare or less established as a traditional research field.
Table 3 highlights and summarizes some of these potential strategies and example of successful citizen science work that has been frequently published in conference and traditional publications.
|EXAMPLE STRATEGY||EXAMPLE PROJECT, TOPIC, ETC.||WHY THIS MIGHT BE EFFECTIVE|
|Have or leverage an established organization as an umbrella for citizen science projects||PatientsLikeMe||Organizational resources to address access and structural barriers, as well as additional skilled collaborators to contribute|
|Work on a well-known disease or topic||Nightscout, OpenAPS (open-source automated insulin delivery), and other diabetes-related projects||Easier to tie to existing work and describe the change from the status quo
More collaborators among traditional researchers who understand the problems or gaps
|Take advantage of timely media attention||Long COVID||Mainstream media attention also raises awareness among researchers and editors who may be less likely to gatekeep in journals as a result
This may also make it easier to find interested collaborators among traditional researchers
There are other ways to achieve dissemination of citizen science work, even outside of traditional publications. Sometimes this includes traditional scientific or medical conferences, but other venues such as blogs, patient conferences, or websites can be effective for disseminating citizen science work. Also, the appearance of this work in those venues can draw the attention of editors who may invite citizen scientists to submit work to a traditional journal (and thus advocate for and support addressing some of the previously described barriers).
If citizen scientists face these barriers for traditional journals, it may be possible to start elsewhere first and build up. Citizen scientists can leverage and cite the work of other citizen scientists, either within or outside of healthcare. It may feel like the area of work is new and novel—and it may be, but citing other citizen science work may help provide an example to address the gap between the status quo and where the new work lives. Filling in this gap with related or tangential citizen science work and showing the connection from the status quo and past work to the future, and where the new work fits, may aid traditional journals in better understanding and accepting novel citizen scientist work.
Similarly, impact factor is often considered by traditional academics, but there is some research showing that effective dissemination through social media channels and other avenues can be more impactful. Thus, if citizen scientists find a collaborative editor who is interested in publishing (whether through a special issue or regular channels), do not worry about the impact factor. Plan instead for an effective post-publication dissemination plan to raise awareness of the work. Scientific communication does not stop with publication or a single conference presentation; think omni-channel and consider parallel and multiple paths for content about the work to address multiple audiences, ranging from the scientific and medical communities of traditional journals to community members who are interested in understanding the work as well.
Additionally, don’t hesitate to seek community support, not only from a topical community (e.g., Alzheimer’s or cystic fibrosis), but also the broader community of citizen scientists. Many may have addressed the same barriers or found creative solutions for going around them, so citizen scientists may not have to recreate the wheel. Even if they took a different path, there may be lessons learned and/or resources to share. Also, citizen scientists can serve as peer reviewers for traditional publications, so consider suggesting fellow citizen scientists as suggested reviewers when submitting articles to traditional publications that allow for inputting suggested reviewers.
Citizen scientists are not different from traditional scientists in that they benefit from mentors. Mentorship exists for under-resourced professionals, and similarly mentorship for citizen scientists (either via fellow citizen scientists or traditional scientists) can encourage citizen scientists, help to increase or add to skills, fill resource gaps, and advocate for the work, including introductions to others and expanding networks, which can bring more opportunities to disseminate work in the future.
Finally, it is important to recognize that some partners for citizen science work—like mentors—may or may not be a good fit for the work, your personality, or what citizen scientists are currently focused on given the barriers they face. If someone approaches a citizen scientist, they should consider an honest assessment of how they can help, explicitly offer that help, and take it gracefully if the assistance is declined. Citizen scientists could ask interested collaborators, partners, or mentors to take the “Partner Readiness Quiz” (Readiness Quiz, no date) (or use it as a guide to self-assess the fit of the partnership). It can also aid citizen scientists in recognizing what help they do or do not need or want and can help them evaluate opportunities and partnerships accordingly.
There may be many barriers to citizen science in biomedicine and in healthcare, including those of disseminating the work effectively. These barriers can be addressed with effort, time, additional resources, upskilling, mentorship, or pure elbow grease and determination by citizen scientists. In other cases, the barriers can be worked around or the goal can be achieved through other methods. Citizen scientists may leverage existing organizations as umbrella partners for their work; tie their work to a broad topical area or disciplinary focus; or use mainstream or online media to help raise awareness about the need and fit of their work. All hands on deck are needed to address many pressing issues in biomedicine and healthcare, and citizen scientists can play a vital role in filling knowledge gaps and solving the pressing problems our communities face by disseminating their work. Future work should further highlight or compare experiences between novice traditional researchers and those of citizen scientists, as well as explore the differences between the experiences of citizen scientists in health and biomedicine and their experiences in other fields.
No financial conflict of interest exists. However, DL acknowledges that she has contributed and developed for both Nightscout and OpenAPS, open source projects in which her efforts are on a voluntary, un-paid basis, and some of this and her other unfunded citizen science work is cited within this paper.
DL is the sole author of this article and performed all authorship steps.
Alwan, NA. 2021. The road to addressing Long Covid. Science. DOI: https://doi.org/10.1126/science.abg7113
Asarani, NAM, Reynolds, AN, Elbalshy, M, Burnside, M, de Bock, M, Lewis, DM and Wheeler, BJ. 2021. Efficacy, safety, and user experience of DIY or open-source artificial pancreas systems: a systematic review. Acta Diabetologica. DOI: https://doi.org/10.1007/s00592-020-01623-4
Bammer, G. 2016. What constitutes appropriate peer review for interdisciplinary research? Palgrave Communications. DOI: https://doi.org/10.1057/palcomms.2016.17
Baum, F, MacDougall, C and Smith, D. 2006. Participatory action research. Journal of Epidemiology and Community Health, 60(10). DOI: https://doi.org/10.1136/jech.2004.028662
Burnside, MJ, Lewis, DM, Crocket, H, Meier, R, Williman, J, Sanders, OJ, Jefferies, CA, Faherty, AM, Paul, R, Lever, CS and Wheeler, BJ. 2022. 286-OR: The CREATE Trial: Randomized clinical trial comparing open-source automated insulin delivery with sensor augmented pump therapy in type 1 diabetes. Diabetes. DOI: https://doi.org/10.2337/db22-286-OR
Citizen Science: Theory and Practice. no date. “Submissions Guidelines.” Available at: https://theoryandpractice.citizenscienceassociation.org/about/submissions/ (Accessed: April 6, 2022).
Cooper, CB, Hawn, CL, Larson, LR, Parrish, JK, Bowser, G, Cavalier, D, Dunn, RR, Haklay, M, Gupta, KK, Jelks, NT and Johnson, VA. 2021. Inclusion in citizen science: The conundrum of rebranding. Science. DOI: https://doi.org/10.1126/science.abi6487
Crouch, LA, Viduto, V and Zangari, M. 2022. “Is long COVID taken seriously by primary care physicians? Evidence through citizen science.” Center for Open Science [Preprint]. DOI: https://doi.org/10.31235/osf.io/5sx79
Eitzel, MV, Cappadonna, JL, Santos-Lang, C, Duerr, RE, Virapongse, A, West, SE, Kyba, CC, Bowser, A, Cooper, CB, Sforzi, A and Metcalfe, AN. 2017. Citizen science terminology matters: Exploring key terms. Citizen Science: Theory and Practice, 2(1). DOI: https://doi.org/10.5334/cstp.96
Elliott, KC and Rosenberg, J. 2019. Philosophical foundations for citizen science, Citizen Science: Theory and Practice, 4(1). DOI: https://doi.org/10.5334/cstp.155
Fiske, A, Savio, LD, Prainsack, B and Buyx, A. 2019. Conceptual and ethical considerations for citizen science in biomedicine. In Personal Health Science. DOI: https://doi.org/10.1007/978-3-658-16428-7_10
Frost, J and Massagli, M. 2009. PatientsLikeMe the case for a data-centered patient community and how ALS patients use the community to inform treatment decisions and manage pulmonary health. Chronic Respiratory Disease. DOI: https://doi.org/10.1177/1479972309348655
Gadermaier, G, Dörler, D, Heigl, F, Mayr, S, Rüdisser, J, Brodschneider, R and Marizzi, C. 2018. Peer-reviewed publishing of results from citizen science projects. Journal of Science Communication. DOI: https://doi.org/10.22323/2.17030101
Ganzevoort, W, van den Born, RJ, Halffman, W and Turnhout, S. 2017. Sharing biodiversity data: citizen scientists’ concerns and motivations. Biodiversity and Conservation, 26(12). DOI: https://doi.org/10.1007/s10531-017-1391-z
Grant, AD, Lewis, DM and Kriegsfeld, LJ. 2021. Multi-timescale rhythmicity of blood glucose and insulin delivery reveals key advantages of hybrid closed loop therapy. Journal of Diabetes Science and Technology [Preprint]. DOI: https://doi.org/10.1177/1932296821994825
Greshake Tzovaras, B, Angrist, M, Arvai, K, Dulaney, M, Estrada-Galiñanes, V, Gunderson, B, Head, T, Lewis, D, Nov, O, Shaer, O and Tzovara, A. 2019. Open humans: A platform for participant-centered research and personal data exploration. GigaScience. DOI: https://doi.org/10.1093/gigascience/giz076
De la Loge, C, Dimova, S, Mueller, K, Phillips, G, Durgin, TL, Wicks, P and Borghs, S. 2016. PatientsLikeMe® Online Epilepsy Community: Patient characteristics and predictors of poor health-related quality of life. Epilepsy and Behavior, 63. DOI: https://doi.org/10.1016/j.yebeh.2016.07.035
Lewis, D. 2019. “History and perspective on DIY closed looping.” Journal of Diabetes Science and Technology. DOI: https://doi.org/10.1177/1932296818808307
Lewis, D. 2021. “How it started, how it is going: The future of artificial pancreas systems (automated insulin delivery systems).” Journal of Diabetes Science and Technology, 15(6). DOI: https://doi.org/10.1177/19322968211027558
Lewis, D and Leibrand, S. 2016. Real-world use of open source artificial pancreas systems. Journal of Diabetes Science and Technology. DOI: https://doi.org/10.1177/1932296816665635
Lewis, DM, Leibrand, S, Street, TJ and Phatak, SS. 2018. “Detecting insulin sensitivity changes for individuals with type 1 diabetes.” Diabetes, 67(Supplement_1). DOI: https://doi.org/10.2337/db18-79-LB
Lewis, DM, Swain, RS and Donner, TW. 2018. Improvements in A1C and Time-in-Range in DIY Closed-Loop (OpenAPS) Users. Diabetes, 67(Supplement_1). DOI: https://doi.org/10.2337/db18-352-OR
“Long COVID: let patients help define long-lasting COVID symptoms”. 2020. Nature. DOI: https://doi.org/10.1038/d41586-020-02796-2
Mosquera-Lopez, C, Dodier, R, Tyler, N, Resalat, N and Jacobs, P. 2019. Leveraging a Big Dataset to Develop a Recurrent Neural Network to Predict Adverse Glycemic Events in Type 1 Diabetes. IEEE Journal of Biomedical and Health Informatics [Preprint]. DOI: https://doi.org/10.1109/JBHI.2019.2911701
Ottinger, G. 2010. Buckets of resistance: Standards and the effectiveness of citizen science. Science Technology and Human Values, 35(2). DOI: https://doi.org/10.1177/0162243909337121
Papadopoulos, A, Salinas, J and Crump, C. 2021. Computational modeling approaches to characterize risk and achieve safe, effective, and trusted designs in the development of artificial intelligence and autonomous closed-loop medical systems. Proc. SPIE 11746, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications III, 1174623. DOI: https://doi.org/10.1117/12.2586101
PatientsLikeMe Research Manuscripts Bibliography. 2019. Available at: https://www.patientslikeme.com/research/publications (Accessed: April 6, 2022).
Paul Wicks. no date. Available at: https://www.bmj.com/about-bmj/advisory-panels/patient-panel-members/paul-wicks (Accessed: April 7, 2022).
Pricing Costs for NVivo. no date. Available at: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/buy-now (Accessed: April 7, 2022).
Rampin, R and Rampin, V. 2021. Taguette: open-source qualitative data analysis. Journal of Open Source Software, 6(68): 3522. DOI: https://doi.org/10.21105/joss.03522
Readiness Quiz. (no date). Available at: https://partner.openingpathways.org/quiz (Accessed: April 11, 2022).
Reinius, M, Mazzocato, P, Riggare, S, Bylund, A, Jansson, H, Øvretveit, J, Savage, C, Wannheden, C and Hasson, H. 2022. Patient-driven innovations reported in peer-reviewed journals: a scoping review. BMJ Open, 12. DOI: https://doi.org/10.1136/bmjopen-2021-053735
Riggare, S. 2020. Patient researchers — the missing link? Nature Medicine. DOI: https://doi.org/10.1038/s41591-020-1080-4
Rivard, L, Lehoux, P and Alami, H. 2021. It’s not just hacking for the sake of it: A qualitative study of health innovators’ views on patient-driven open innovations, quality and safety. BMJ Quality and Safety, 30(9). DOI: https://doi.org/10.1136/bmjqs-2020-011254
Rodríguez-Rodríguez, I, Chatzigiannakis, I, Rodríguez, JV, Maranghi, M, Gentili, M and Zamora-Izquierdo, MÁ. 2019. Utility of big data in predicting short-term blood glucose levels in type 1 diabetes mellitus through machine learning techniques. Sensors (Switzerland), 19(20). DOI: https://doi.org/10.3390/s19204482
Tzovaras, BG, Hidalgo, ES, Alexiou, K, Baldy, L, Morane, B, Bussod, I, Fribourg, M, Wac, K, Wolf, G and Ball, M. 2021. Using an individual-centered approach to gain insights from wearable data in the quantified flu platform: Netnography study. Journal of Medical Internet Research, 23(9). DOI: https://doi.org/10.2196/28116
Wicks, P. 2018. Patient, study thyself. BMC Medicine. DOI: https://doi.org/10.1186/s12916-018-1216-2
Wicks, P and Little, M. 2013. The virtuous circle of the quantifi ed self: A human computational approach to improved health outcomes. In: Handbook of Human Computation. DOI: https://doi.org/10.1007/978-1-4614-8806-4_12
Wicks, P, Massagli, M, Frost, J, Brownstein, C, Okun, S, Vaughan, T, Bradley, R and Heywood, J. 2010. Sharing health data for better outcomes on patientslikeme. Journal of Medical Internet Research. 12(2). DOI: https://doi.org/10.2196/jmir.1549
Wicks, P, Sulham, KA and Gnanasakthy, A. 2014. Quality of life in organ transplant recipients participating in an online transplant community. Patient, 7(1). DOI: https://doi.org/10.1007/s40271-013-0033-0
Wilkins, RB. 2021. Do You Know Me?: The Subtle Distinction Between “Anonymous” and “De-identified” Data in Clinical Research. Available at: https://www.wcgirb.com/insights/do-you-know-me-the-subtle-distinction-between-anonymous-and-de-identified-data-in-clinical-research/ (Accessed: April 7, 2022).