Introduction

Paul read Clusterbusters’ website, detailing how to use psychedelics as a treatment for cluster headache, with interest.¹ As a healthcare professional with a background in science, he found their grassroots research compelling. Plus, he seemed to have run out of treatment options–everything legal had failed to stop his cluster attacks. Still, taking psychedelic mushrooms made him nervous, having no prior experience with illegal drugs. So he followed every precaution outlined on their website. He grew and dried the correct strain of mushrooms and divided them into tiny capsules, each containing a half-gram of his harvest. He paid close attention to his “set” and “setting,” assuring that both his mindset and physical and social environment were conducive to a positive experience. He asked a friend who was an experienced psychedelic drug user to remain sober and “sit” with him, in case he needed help. When he finally dosed, he took a single pill and waited to feel an effect before taking another to ensure he ingested the minimum amount necessary. He told us in an interview that the results were a “miracle”–a claim supported with copious notes taken in a detailed diary.

Drug development is typically the purview of the pharmaceutical industry, in coordination with academic medical institutions. People creating biomedical knowledge outside traditional scientific institutions lack both the authority and–in this case–the resources needed to conduct biomedical research. Nevertheless, people can and often do manage to create useful biomedical knowledge in spite of these challenges. In this case, Clusterbusters, the organization who published the dosing protocol that Paul used as an instruction manual, created a standardized dosing protocol for the treatment of cluster headache from fungi grown illegally in their own homes, despite lacking the ability to create a standardized dose of psilocybin.²

Standardization is the “process of constructing uniformities across time and space through the generation of agreed-upon rules” (Timmermans and Epstein 2010). Standards serve a variety of functions in biomedical research, ranging from the pragmatic to the epistemological to the political. Standards convert “messy” varied outcomes into something predictable and reproducible, thereby ensuring reliability and validity (Knorr Cetina 1999). But they also provide science (and therefore the researchers who conduct this science) with power and authority, institutionalizing and legitimating the value of scientific work (Porter 1995; Brown 1993).

The gold standard for assuring that drugs are safe and effective in the regulated drug market is the randomized clinical trial (RCT) (Bothwell et al. 2016). In the regulated drug market, Phase II clinical trials typically use RCTs to determine an optimal dose and dosing strategy (Sheiner, Beal, and Sambol 1989). However, like many engaged in do-it-yourself (DIY) pharma, the people experimenting in these forums lacked the resources to conduct an RCT. An RCT cannot be performed without money, expert physicians, and access to a standardized pharmaceutical-grade drug to investigate, as well as a believable placebo. Nevertheless, as we demonstrate in this paper, Clusterbusters were still able to solve the pragmatic problem of how to dose themselves with a poorly understood, highly variable substance to treat the invisible, subjective experience of pain.

Clusterbusters’ success centers around their development of a workable dosing protocol. While RCTs testing the efficacy of psilocybin as a treatment for cluster headache have not yet been completed, case studies and survey data suggest this protocol may be more effective than any other preventive drug medication on the market for cluster headache (Sewell, Halpern, and Pope 2006; Schindler et al. 2015). In 2022, Yale University neurologist Dr. Emmanuelle Schindler completed data collection for a randomized controlled trial testing the efficacy of Clusterbusters’ protocol for the use of psilocybin in cluster headache (D’Souza 2016; Ellison 2021). In other words, both Yale University and the Federal Food and Drug Administration approved a clinical trial, in part, on the basis of Clusterbusters’ drug protocol. This point, in particular, strikes us as particularly salient given that the patent issued for the use of psilocybin as a therapy for cluster headache by the US Patent and Trademark Office fails to include the names of anyone affiliated with Clusterbusters (Sewell 2014).

Clusterbusters, we argue, developed a flexible and pragmatic standard that allows for a knowledge production process impossible within a standard RCT, by using their individual sense of embodiment to identify and refine their own dosing strategies. Clusterbusters thus represents an example of how citizen scientists can use standards for both the production of novel medical treatments and the creation of new medical knowledge. Their work slots conveniently into the typical progression of pharmaceutical research: In this case, Clusterbusters engage in a type of “online collective self-experimentation” (Kempner and Bailey 2019) that replicates what clinical researchers might call a “dose-ranging study,” that is, an experiment designed to produce the most effective and/or least harmful therapeutic dose. Schindler and her colleagues at Yale have taken the next logical step: testing the efficacy of these doses. Standardization thus plays an essential role in apparently non-standard informal research. Rather than draw a distinction between standardized and non-standardized medical knowledge, we argue that standardization characterizes both formal and informal knowledge production. For both formalized science and citizen science, standardization is necessary in order to produce portable, communicable knowledge that can be shared across contexts (Timmermans and Epstein 2010).

Our study demonstrates how standards that don’t suit formalized scientific epistemologies are nevertheless powerful tools for generating knowledge. This is a significant accomplishment, particularly given recent calls from psychedelic researchers to incorporate “real world data” (Carhart-Harris et al. 2022).

Background

Medical research into psychedelic drugs ceased following policy changes that made its clinical study nearly impossible post 1970 (Nutt, King and Nichols 2013; Oram 2016). However, today, psychedelic substances are viewed as potential solutions for a broad range of intractable health concerns, with psilocybin in particular decriminalized in Colorado, Oregon and Washington, DC (Noakes 2019; Kreps 2020). Despite the popular narrative that institutional scientists are behind this so-called psychedelic renaissance, much contemporary research on psychedelic substances may be considered a formal iteration of ancient indigenous knowledge (Sabina 2003; Tupper 2009; George et al. 2019) and/or of research that had previously been conducted underground by activists without formal training or the support of professional research institutions (Sewell 2008; Shroder 2014). As a result, citizen science has led to a variety of successful applications of hallucinogens.

Citizens have a long history of medical investigation on their own, particularly to develop treatments that institutionalized medicine has failed to provide. Such DIY medicine abounds among those who have traditionally been excluded from biomedical inquiry, such as AIDS activists and feminist health collectives (e.g., Murphy 2012 and Epstein 1995). People practicing DIY medicine sometimes resort to untested methods and marginalized forms of treatment, as in the case of the Wo/Men’s Alliance for Medical Marijuana (Chapkis and Webb 2008). In recent years, DIY medicine has occurred online as well, through a variety of informal channels. Many of these internet-based medical efforts are driven from the ground by patients, as in the case of the long Covid diagnosis, which emerged primarily from online patient support groups (Callard and Perego 2021). Others rely on informal networks of experts, like the DIY insulin movement and the Open Artificial Pancreas System (Burnside et al. 2020), or the vaccine experimentation collective RaDVaC (radvac.org 2022).

Additionally, noninstitutionalized scientists engage with scientists working within institutions in a variety of ways. Sometimes these relationships are oppositional, as when the legal and regulatory barriers of medicine explicitly foreclose needed treatments, placing activists in direct opposition to medical structures, as in the case of the abortion rights group the Jane Collective (Joffe et al. 2004). In other cases, DIY medical activists demand legitimacy and position themselves explicitly as credible sources of medical knowledge, such as the AIDS activism in the 1990s (Epstein 1995). As in the example of community-based air toxics monitoring, citizen scientists may actively work to build bridges with scientists, a form of “shadow mobilization” that gives them some semblance of legitimacy among experts (Frickel et al. 2015; Ottinger 2010).

Citizen scientists, however, often struggle to gain recognition for their labor from those who privilege knowledge generated from institutionalized scientists. Policymakers, journal editors, and other gatekeepers often find citizen science data to be of inferior quality, leading them to place lower value on its conclusions (Burgess et al. 2017), or they may consider citizen science to have lower ethical standards or involve less skill than formal science (Kosmala et al. 2016), regardless of whether these assessments are true. This difference in legitimacy may be exacerbated by the fact that citizen scientists often lack access to resources considered necessary to the production of sound science, such as laboratory equipment and money.

Thus, any lay production of medical knowledge grapples with a legitimacy challenge, in which its quality, ethics, and safety can be placed under potential question by formal researchers (e.g., Rasmussen et al. 2020). This skepticism emerges from some of the core legitimizing tenets of normal science, such as concerns over the objectivity, methodological rigor, and theoretical sophistication of citizen science (Elliott and Rosenberg 2019). One potential way for citizen scientists to counter this legitimacy challenge is to draw upon the epistemological tools of formal science, such as the notion of a standard.

Standards in Biomedical Research

Prior to the 1960s, self-experimentation in medicine was not only an accepted practice, but was sometimes even lauded as heroic (Herzig 2005). Scientific mores shifted dramatically in 1961, when it was discovered that thalidomide, a drug widely prescribed to pregnant women, had caused thousands of birth defects (Greene and Podolsky 2012). U.S. Congress reacted by passing the 1962 Kefauver-Harris Amendment to the Federal Food, Drug, and Cosmetics Act, which expanded the powers of the FDA to ensure that all drugs on the market could be proved both safe and efficacious (Turner 2012). Although the thalidomide scandal galvanized the passage of Kefauver-Harris, the amendment’s broader emphasis on safety and efficacy reflected an underlying shift toward evidence-based politics of health (Scroop 2007). Within drug development, this shift led to the hegemony of the RCT as a way to differentiate between legitimate and illegitimate knowledge. The RCT’s hegemonic status is signaled by the term gold standard, denoting a hierarchy of evidentiary processes that places RCTs at the top and other forms beneath it (Bothwell et al. 2016).

Shortly after the amendment’s passage, commentators noticed that the increased burden of proof for new drugs led to only the largest, most well-resourced firms pushing new drugs to market (Jadlow 1971). Today, potential interventions appear faster than the schedule of institutions producing evidence through RCTs can support (Bothwell et al. 2016). The Kefauver-Harris Amendment and the associated dominance of RCTs thus played a key role in locking innovative interventions out of the charmed circle of medical legitimacy, particularly those whose efficacy could not easily be supported by RCTs.

For medical practitioners, however, standards sometimes serve a different function. Often, practitioners work around standards even as they draw upon them. These workarounds demonstrate how weak normative pressure of standards is in actual practice relative to their its strength in knowledge production (Timmermans and Almeling 2009). For example, clinical practice often leans more toward “art” than the scientific specificity dictated in evidence-based guidelines (McGlynn et al 2003). Similarly, physicians are warned to be cautious with off-label prescriptions for which there is “a dearth of valid clinical studies” (Larriviere et al. 2009). The burden of judgment when prescribing novel psychoactive medications is thus placed onto individual doctors (Hall and Lucke 2010). In this context, standardization presents potential conflicts when the specifics of a patient’s subjective experience or preference cannot be readily addressed with a standard set of guidelines (Pfaff et al. 2010). Standards therefore function depending on context: When solving problems pragmatically, standards become more flexible, functioning as a guideline and a starting place rather than as a force delineating legitimate from illegitimate practice.

Standards take on this dual significance for citizen scientists as well. Citizen scientists may adopt standards as a way to build bridges with scientists, creating legitimacy among experts, as in the example of community-based air toxics monitoring (Ottinger 2010). However, in this paper, we argue that standardization’s more flexible forms can also enable citizen scientists to solve pragmatic health problems, much as medical professionals use flexible standards in clinical settings. In the next section, we describe one such group of citizen scientists, and consider the potential for alternate forms of standardization to enable unique forms of knowledge production.

Case and Methods

Our exploration of standardization draws upon a case study of Clusterbusters, an organization that represents an online network of people with cluster headache and their supporters who work together to develop innovative treatments for cluster headache, regardless of whether those treatments are legal or legitimated by medical consensus.

Cluster headache is a poorly understood and debilitating neurological disease characterized by excruciating attacks of head pain, each lasting between 15 to 180 minutes each (Burish et al. 2021). Eighty percent of people with cluster headache have episodic cluster headache, in which attacks occur daily for approximately two to eight consecutive weeks with a recurring seasonal onset. The remainder have chronic cluster headache, which means their daily attacks occur for at least a year with remission periods of less than three months.

Cluster headache affects about 1 in 1,000 American adults, which makes it about as prevalent as multiple sclerosis (Choong et al. 2017). People with cluster headache often seek alternative treatment because it can be difficult to find care in traditional healthcare settings. Diagnostic delays run an average of five years in the United States, and treatment options are limited (Rozen and Fishman 2012). Additionally, the severity of pain associated with cluster headache motivates many with the disease to consider a wide range of treatment options, no matter its their legality or risk (Andersson, Persson and Kjellgren 2017; Schindler et al. 2021).

People with cluster headache began discussing psychedelic therapies for cluster headache online in a patient support group called www.clusterheadaches.com (CH.com) in 1998. Robert “Bob” Wold formed Clusterbusters, a second, separate forum, in August 2002 to focus on the development of psychedelics as a novel treatment for cluster headache. Clusterbusters became a registered, tax-deductible nonprofit in 2006.

Clusterbusters is the primary organization advocating for the use of psychedelics as a treatment for cluster headache. Clusterbusters has served as the primary node in a network in which people produce knowledge about psychedelics as a treatment for cluster headache since its formation. Members meet in online forums, and in-person at annual conferences, advocacy events, and regional support groups. Clusterbusters’ online and in-person activities, therefore, represent key sites for knowledge production.

This paper draws upon a multi-sited digital ethnography, and includes multiple forms of data. Digital data includes online posts scraped from www.clusterheadaches.com and from Clusterbusters’ online discussion group between July 28, 1998 and December 31, 2005. This time frame bookmarks a highly productive moment in the development of psilocybin as a treatment for cluster headache, beginning with the original post that initiated this online collective self-experiment effort and ending at the approximate time that Harvard researchers completed data collection for their landmark peer-reviewed article published in Neurology (Sewell et al. 2006). We supplement this digital data with participant observation and interviews with key informants (conducted by JK). Please refer to Kempner and Bailey (2019) for a complete description of methods. Rutgers’ Institutional Review Board approved this research.

Our analysis of these data followed the inductive process associated with grounded theory, which relies on an iterative process of coding, re-coding, and writing memos to build and refine a conceptual scheme (Charmaz 2014). When appropriate, we also use these data as a searchable historical archive. Using ATLAS.ti, each author free coded texts independently before collaborating to identify emerging themes. Using the new themes as a coding scheme, we iterated on our first codes to capture an evolving analysis of how the Clusterbusters prepare mushrooms, dose themselves, and build knowledge.

Given the pseudonymous nature of online discussion, we are unable to speak with precision about the demographics of the posters in our dataset. However, fieldwork suggests that the majority are white adult men and women, primarily from the United States, the United Kingdom, Canada, and New Zealand. Many, but not all, have college degrees. Few appear to have university training in biomedical science. We indicate when pseudonyms are used in this paper to protect subjects.

Analysis

In August 2002, Clusterbusters’ founder, Bob Wold, suggested that the group begin their work by collecting everything they had already discovered about treating cluster headache with psychedelics and organizing it into a Frequently Asked Questions (FAQ) page that could be published online. This page, Wold explained, would serve an important task: “[t]he FAQ should include a complete set of links to ‘good, accurate’ information and be constantly updated” so that those who need treatment can find needed information.

A forum member responded with a selection of questions that they often hear, and which the FAQ would need to address. These questions revealed how difficult Wold’s project would become:

How do I obtain mushrooms?

Will my meds interact with psilocybin?

Do I need to detox from my meds for mushroom therapy?

Will mushroom therapy help chronic cluster sufferers as well as episodics?

How much psilocybin do I need to take in order to treat my CH’s?

How often will I need to dose?

Any FAQ able to answer these heretofore unanswered questions would become something akin to what institutionalized medicine might recognize as a standardized drug protocol.

Clusterbusters’ first version of their FAQ, published online in November 2002, involved the standardization of multiple sources of ambiguity: dosing amount and vector, potential drug interactions, legal issues, psychological mindset and physical setting, quantifying “trip levels,” and pain scales (Wold 2002). None were simple. Our analysis examines their methods for standardizing their doses of psilocybe mushroom.

A Standard Dose

People experimenting with psilocybe mushrooms in cluster headache forums had a pragmatic need to standardize each dose they consumed. Every cluster headache attack constitutes an event so traumatic it often triggers suicidal ideation (Lee et al. 2019). While all experimentation requires standardization in order to foster replication, desperation in this population is higher than in most (Andersson, Persson and Kjellgren 2017). A miscalculated dose might be ineffective. Too much could cause unpleasant side effects with potentially psychologically damaging consequences (Carbonaro et al. 2016). Additionally, it quickly became clear that group members would need to establish a way to standardize doses if they wanted to communicate their findings with each other and/or with establishment scientists.

The task of determining a dose created from a mushroom, however, proved difficult. A psilocybin-containing mushroom is different from psilocybin. Multiple species of mushrooms contain psilocybin. Some species grow large mushrooms, while others grow small mushrooms. Some species are reputed to contain a great deal of psilocybin, while others are weak in comparison. Additionally, each species has been cultivated into various strains, each reputed to contain various amounts of psilocybin.

Participants attributed variation in potency of mushrooms to a remarkable range of factors. Even defining a mushroom is difficult, with “one mushroom” representing an inexact metric. Does a dose of mushroom include only the fungus, or should it also include the mycelium, the root? Do some parts of the mushroom contain more psilocybin than others? Mushrooms can be grown in flushes, or harvests, from the same set of mycelium: Does the first flush differ in potency from the third? Members wondered whether the timing of their harvest might affect the potency of their doses. Did it matter, for example, whether the mushrooms they harvested had opened their caps yet? “The shrooms we harvested … after the caps opened and they blew their spores […] were less potent. If I harvested JUST before they opened they were pretty heavy duty,” notes one poster. Mushrooms are messy.

Additionally, members began noticing that storage and preparation of each dose could affect its potency. Eating more stems than caps or vice versa could produce radically different results. Or, as one participant accidentally discovered when he ate his mushrooms on a peanut butter sandwich, fat ingested with mushrooms could render the drug completely ineffective. An individual’s frame of mind could also completely transform how a person experienced the potency of a psilocybin dose. Members were encouraged to pay attention to their set and setting—a concept in psychedelic research that refers to an individual’s mindset (their set) and their physical and social setting (setting). Members were encouraged to take mushrooms with positive thoughts, moods, and expectations, by perhaps choosing music or activities that would make the psychedelic effects easier to manage.

Variation in dosage created problems. In the short term, too little of a dose might mean continued attacks. Taking too much might make for an uncomfortable psychedelic experience. More importantly, without a standard, Clusterbusters struggled to answer the many questions involved with turning mushrooms into medications. Clusterbusters, therefore, engaged multiple strategies in their efforts to standardize mushrooms.

Discussion and conclusion

Cluster headache is difficult to treat, and psilocybin mushrooms are difficult to use as a treatment. Clusterbusters do both, through a process of standardization. Clusterbusters combine mechanical objectivity and embodied standards to generate and standardize the subjective techniques of cultivating, preparing, and dosing with psychedelic mushrooms. Indeed, Clusterbusters’ knowledge is useful to them precisely because it is standardized, which allows them to create a reliable dose out of an unreliable substance. However, their knowledge does not bear the imprimatur of medical science to support dose effectiveness. Clusterbusters themselves do not engage in work that meets the “gold standard” of medical science.

Clusterbusters’ practices of standardization rely on epistemological and methodological tools that are ruled out by normal science. The dosing strategies developed through individual experience and posted on Clusterbusters’ website depend on subjective reports of individual experiences. This conflicts directly with the goals and practices of standardization within the scientific profession, which aims to reduce the influence of subjectivity on the knowledge-making process. Furthermore, the medical and scientific professions demand the use of RCTs as the gold standard for the assessment, yet Clusterbusters does not, and indeed cannot, engage in practices like double blinding or the use of control groups. Finally, Clusterbusters’ recommendations do not take the form of straightforward prescriptions, but rather recommendations for action that often become the subject of further debate rather than the closure implied by the specific doses used in RCTs. The existing and emerging scientific publications based on Clusterbusters’ work are illustrative. Multiple peer-reviewed journal articles have been produced describing the use of psilocybin by Clusterbusters (e.g., Sewell et al. 2006; Schindler et al. 2015; Schindler et al. 2021). However, these articles point out the need for further research under experimentally controlled settings.

Nevertheless, the work done by Clusterbusters highlights important advantages for citizen science. By taking a complex, poorly understood drug and standardizing it into a form that can be recommended to new users with no previous experience, Clusterbusters demonstrate some benefits of citizen science over formalized science. Perhaps most obviously, the lack of oversight in underground research means Clusterbusters can do research into illicit substances. Despite some anxiety and precautions around legality, citizen scientists can be more flexible with their research topics than formalized scientists. Furthermore, Clusterbusters create a useful, flexible standard that does not require the institutional or financial support that clinical research does. A clinical trial, for example, cannot be easily adjusted mid-course, while Clusterbusters members frequently adjust their doses to find suitable treatments across a range of bodies and conditions. Further, RCTs cannot easily meet the urgent need for relief demonstrated by members of Clusterbusters. The flexible standards of citizen science, however, can create interventions relatively quickly. Collectively, our analysis of Clusterbusters suggests that citizen science provides some important advantages over institutionalized science.

Clusterbusters’ embodied standard may lack the reliability necessary for use in institutionalized science, but clinical researchers have found it specific enough to adapt for use in randomized clinical trials. Thus, Clusterbusters demonstrate the dual functions of standardization. On the one hand, they build standards that can be worked around, useful as guideposts and starting points, but not as an authoritative list of prescriptions and prohibitions. On the other hand, their standards nevertheless facilitate reliable enough scientific communication to build legitimate bridges with scientists. By being both just flexible enough, and just reliable enough, standards can be used by citizen scientists to create novel interventions and support the production of knowledge, despite failing to meet institutionally recognized criteria.

Previous scholarship has highlighted the importance of standards primarily as a tool for citizen scientists to build bridges with scientific actors. In this analysis, we show that citizen scientists can use standards as a tool for their own knowledge production, both to meet immediate needs and to generate clinically viable knowledge for the future. As progressively more informal medical interventions, especially illicit ones, enter the public conversation around health, and as the ability for individuals to collectively use the tools of self-experimentation increases, we argue for the importance of focusing on citizen scientists as producers of knowledge—both in the standards that make knowledge creation possible, and as solvers of difficult public health problems.

Data Accessibility Statement

While the websites used to build our datasets are publicly available, much of the content on those sites may be sensitive. Thus, names have been changed in these quotes and the full dataset is not being released publicly.