Background
Early detection is a vital aspect of preventing the spread of infectious diseases (). One such closely monitored disease is influenza, which is responsible for 140,000 to 960,000 hospitalizations and 12,000 to 79,000 deaths per year in the United States (U.S.) (). Most infectious disease surveillance systems are based on outpatient visits to healthcare providers and laboratory-confirmed cases. Although these sentinel surveillance systems provide information on where, when, and what influenza viruses are circulating, they lack the ability to capture instances of disease when medical care is not sought. Furthermore, they are not capable of presenting data in real time when there is a delay between the onset of symptoms and receiving medical care or when cases are not immediately reported to local health departments by clinicians (; ).
Over the past two decades, the widespread proliferation of mobile devices and wearable technology has significantly changed the landscape of epidemiological data gathering and evolved into a field known as Digital Epidemiology (DE) (; ). DE provides an informal, complementary approach to traditional sentinel surveillance methods by leveraging data generated outside of the public health system through digital data sources such as Google internet searches (), Twitter posts (; ; ), and cloud-based electronic health records (Santillana et al. 2016). While these alternative data sources provide real time information about trends and general patterns of disease activity, the data is not generated specifically for epidemiologic purposes ().
One source of active digital data collection is online participatory syndromic surveillance systems (Smolinski et al. 2017). Through these systems, participants volunteer to report health information via online or mobile communication technologies on a weekly basis. The first of these systems, de Grote Griepmeting, or the Great Influenza Survey, started in 2003 in the Netherlands and Belgium (). Since that time, multiple systems have been established throughout Europe, Australia, the U.S., and Japan (; ; ; ). These systems actively engage the general public in reporting and provide timely information about disease trends within the community, thereby providing a mechanism for members of the community to become citizen scientists (; , ). Flu Near You (FNY) is a U.S.-based online participatory syndromic surveillance system that was developed in 2011 through collaboration between HealthMap of Boston Children’s Hospital and Ending Pandemics (formally Skoll Global Threats Fund) (; https://flunearyou.org). In exchange for participating in FNY, users can visualize local influenza-like illness (ILI) activity on maps, connect with local public health organizations, and find nearby locations offering influenza vaccines.
Two important issues for any citizen science system are motivating contributions from participants and ensuring the quality of these contributions (). For participatory syndromic surveillance systems, consistent participation by users is necessary to accurately track ILI over time and produce end-of-year estimates of ILI burden. However, many users of these systems report only once, and these one-time users are more likely to report ILI symptoms (). In August 2017, FNY added a new feature that allows website visitors to submit a health report without registering. Allowing reports without registration was aimed at reducing the initial barrier to participaton presented by the registration process and potentially reducing the bias from one-time reports.
Understanding the motivations of citizen science participants is key to attracting new participants and retaining old ones (). Starting with the 2015–2016 influenza season, FNY began administering user surveys at the end of each influenza season. These surveys collect information on reporting habits, on how users interact with the site, and on users’ motivations for joining and continuing to use FNY. The surveys also ask about demographic information not collected upon registration with the site, which allows for further understanding of the representativeness of the FNY user base. In addition, these surveys provide users with the opportunity to give feedback about their overall experiences with the platform.
Previous studies related to motivations for initial and continued engagement in citizen science projects have focused on intrinsic motivations, such as preexisting enthusiasm for the scientific topic and gaining a sense of helping, versus extrinsic motivations, such as game-based rewards and monetary compensation (; ; ; ; ; ; ). Results suggest that members with intrinsic motives have enhanced participation frequency and longer participation (; ; ). During the 2018–2019 influenza season, FNY partnered with Science Friday (SciFri), a weekly national public radio program that discusses topics relevant to science, nature, and technology (). The goal of this partnership was to provide SciFri listeners and current FNY users with factual information about influenza virus, symptoms, spread, and vaccination in the form of two half-hour interviews with influenza experts and high-risk patients broadcast as part of the public radio show and podcast. Online, broadcast, and SMS text content addressed misconceptions about influenza and vaccines and promoted a better understanding of disease surveillance and data literacy. This collaboration aimed to register new FNY users who are science enthusiasts with strong intrinsic motivations, and increase user participation through SMS text-based reminders.
In this paper, we summarize the effectiveness of the guest user feature in increasing the number of weekly symptom reports and in improving the frequency of registered user reporting. Specifically, we compare the total number of weekly reports, the total number of registered participants, and the median number of reports per participant before and after the implementation of this feature. We also summarize yearly user survey data for four influenza seasons (2015–2016 to 2018–2019) and compare results from year to year to demonstrate if and how the user base and motivations are changing. Finally, we compare the frequency of user reporting and the motivations of users who registered through the SciFri page with those of users who registered through the FNY website.
Methods
Data collection
Following registration, FNY users are sent a reminder email each Monday with a link to a brief weekly report. These reports first ask how the user is feeling. If a user reports “Not feeling well,” the user has the opportunity to select symptoms they experienced during the previous week, e.g., headache, body aches, diarrhea, fatigue, chills, nausea, shortness of breath, rash, and runny nose. (Figure 1). The user is then asked follow-up questions about the date of illness onset and whether the user sought a health professional for care. At the start of each season, users are also asked if they have received the influenza vaccine. We define participants as FNY users and their household members. Influenza surveillance seasons were defined as Morbidity and Mortality Weekly Report (MMWR) week 40 through week 20, which typically corresponds with the start of October through mid-May. Our study of FNY data received approval from the Boston Children Hospital’s Institutional Review Board.
Screenshots of Flu Near You.
Guest user analysis
The implementation of the guest user feature was analyzed to understand the frequency of user reporting and participant representation before and after this change. FNY data from two influenza seasons before the implementation (2015–2016 and 2016–2017) and two influenza seasons after the implementation (2017–2018 and 2018–2019) were included in the analysis. We summarized the number of participants, the number of users, the total number of FNY symptom reports, the median symptom reports per week, and the number of reports per participant for registered participants only across all four influenza seasons as well as for registered participants and guest users combined for the 2017–2018 and 2018–2019 influenza seasons. We also compared the demographics of registered participants across all four influenza seasons. Continuous variables are presented as median (25th percentile, 75th percentile), and categorical variables are presented as n (%). Significant differences across seasons were assessed using the Kruskal-Wallis test for continuous variables and Chi-square tests for categorical variables.
Survey data analysis
Surveys for each influenza season were created using SurveyMonkey or SurveyGizmo. Surveys were administered to registered FNY users in June of each study year. Prior to distribution, survey questions were fine-tuned with input from contributing FNY partners and stakeholders including individuals from local health departments, the Centers for Disease Control and Prevention (CDC), and universities. Surveys from the previous four influenza seasons, 2015–2016 through 2018–2019, were used in the analysis.
Only users with complete surveys were included in the analysis. Summaries of responses to selected survey questions were tabulated for each year. Questions included in the analysis were
- “How likely are you to recommend Flu Near You on a scale of 0 to 10 (with 0 being least likely and 10 being most likely)?”
- “Why did you sign up for Flu Near You?”
- and “What motivates you to continue using Flu Near You?”
Questions about users’ influenza vaccination status, whether FNY influenced them to get vaccinated, and whether FNY could influence those who were not vaccinated to receive a vaccine in the future were also analyzed. In addition, information about users’ reported level of educational attainment and occupation were summarized. Results were compared across each study year. Questions discontinued by the most recent FNY survey (2018–2019) were excluded from the analysis.
Each survey respondent’s number of symptom reports submitted to FNY per year was compared with the survey question “How frequently do you report to Flu Near You?” Survey respondents were compared with all registered FNY users to examine how they differed by number of reports. For this analysis, only users registered as of June 1, 2019 were included. When analyzing influenza seasons before the 2018–2019 season, users who registered after that season were excluded from the analysis. For example, when comparing 2015–2016 survey respondents with all FNY users, users who registered during the 2016–2017 season or later were excluded.
Science Friday listener analysis
SciFri listeners registered for FNY using a unique landing page that tagged users in the database as SciFri users. SciFri listeners also could sign up for weekly text reminders that included facts and tips related to influenza. All FNY participants were able to track the weekly percent ILI activity for the SciFri cohort, the FNY cohort, and the CDC Influenza-Like Illness Surveillance Network (CDC-ILINet) through a time-series visualization embedded on the FNY website (CDC 2019). We summarized the number of participants, the number of users, the total number of symptom reports, the median symptom reports per week, and the number of reports per participant for registered participants. Additionally, annual user survey responses from the 2018–2019 season from SciFri users were compared with those from users who did not register through SciFri. Chi-square tests of independence were used to assess differences in responses between these two groups.
Results
Guest user analysis
The descriptive statistics of FNY reporting for the four influenza seasons are shown in Table 1. The 2015–2016 influenza season had more registered participants (n = 47,893) compared with the 2016–2017 (n = 30,638), 2017–2018 (n = 31,558), and 2018–2019 (n = 29,849) influenza seasons. However, the 2015–2016 influenza season had the smallest median number of reports per registered participant—2 (1, 12). Following this season, the median number of reports per registered participant increased to 8 (1, 24). This change in reporting habits is captured in Figure 2, which shows that the relative percentage of registered participants who submitted only one report has decreased since the 2015–2016 influenza season. Although the 2018–2019 influenza season had the fewest number of registered participants, it had the greatest median number of reports per week from registered participants—11,561 (10,535, 12,022). Across all four seasons, approximately two-thirds of the registered participants were primary users. During the 2017–2018 influenza season, there were 132,504 reports from guest users. The number of reports from guest users during the 2018–2019 influenza season decreased to 55,890. Although these guest reports increased the median number of weekly reports, the median number of reports per participant decreased to 1.
Table 1
Descriptive statistics of Flu Near You (FNY) reports for the 2015–2016, 2016–2017, 2017–2018, and 2018–2019 influenza seasons. Continuous variables are displayed as median (25th quartile, 75th quartile), and categorical variables are displayed as n (%).
| VARIABLE | 2015–2016 | 2016–2017 | 2017–2018 | 2018–2019 | ||
|---|---|---|---|---|---|---|
| REGISTERED | REGISTERED | REGISTERED | REGISTERED + GUEST | REGISTERED | REGISTERED + GUEST | |
| Total number of participants1 | 47,893 | 30,638 | 31,558 | 164,062 | 29,849 | 85,739 |
| Number of users2 | 33,011 (68.9%) | 20,183 (65.9%) | 20,588 (65.2%) | 153,088 (93.3%) | 19,824 (66.4%) | 75,708 (88.3%) |
| Total number of reports | 378,919 | 333,628 | 350,636 | 483,142 | 372,363 | 428,280 |
| Weekly reports | 11,449 (10,834, 12,160) | 10,433 (9823, 11,290) | 10,416 (9895, 11,189) | 12,208 (11,178, 15,642) | 11,561 (10,535, 12,022) | 13,115 (11,379, 14,259) |
| Number of reports per participant | 2 (1, 12) | 4 (1, 23) | 5 (1, 23) | 1 (1, 1) | 8 (1, 24) | 1 (1, 2) |
1 Participants are defined as FNY users and registered household members.
2 Users are defined as individuals who submit a FNY report.
Distributions of the number of registered FNY participant reports during the (a) 2015–2016, (b) 2016–2017, (c) 2017–2018, and (d) 2018–2019 influenza seasons.
Table 2 shows the descriptive demographic statistics of registered FNY participants for all four influenza seasons. Since the 2015–2016 influenza season, the percentage of male participants increased significantly from 34% to 39% (p < 0.001). Although the median age of all participants increased from 50.8 (34.4, 62.3) to 52.8 (33.6, 64.4), the percentage of participants in the 5–17 age group increased significantly from 8.47% to 12.94% (p < 0.001).
Table 2
Descriptive statistics of registered Flu Near You (FNY) participant demographics for the 2015–2016, 2016–2017, 2017–2018, and 2018–2019 seasons. Continuous variables are displayed as median (25th quartile, 75th quartile), and categorical variables are displayed as n (%).
| VARIABLE | LEVEL | 2015–2016 | 2016–2017 | 2017–2018 | 2018–2019 | P-VALUE |
|---|---|---|---|---|---|---|
| Total | N | 47,893 | 30,638 | 31,558 | 29,849 | |
| Sex | Male | 16,258 (34.33%) | 11,295 (36.87%) | 11,715 (37.16%) | 11,660 (39.11%) | |
| Female | 29,679 (62.67%) | 18,636 (60.83%) | 18,712 (59.36%) | 17,506 (58.73%) | ||
| Unspecified | 1,418 (2.99%) | 707 (2.31%) | 1,097 (3.48%) | 644 (2.16%) | < 0.0011 | |
| Age | Median (IQR) | 50.8 (34.4, 62.3) | 53.6 (36.4, 64.1) | 52.8 (35.7, 64.3) | 52.8 (33.6, 64.4) | < 0.0012 |
| Age group | < 5 | 1,006 (2.1%) | 538 (1.76%) | 499 (1.58%) | 528 (1.77%) | |
| 5–17 | 4,055 (8.47%) | 2,914 (9.51%) | 3,302 (10.46%) | 3,861 (12.94%) | ||
| 18–49 | 16,349 (34.14%) | 9,310 (30.39%) | 9,889 (31.34%) | 8,806 (29.5%) | ||
| 50–64 | 14,516 (30.31%) | 10,038 (32.76%) | 9,774 (30.97%) | 9,055 (30.34%) | ||
| 65+ | 8,366 (17.47%) | 6,727 (21.96%) | 7,080 (22.43%) | 6,922 (23.19%) | ||
| Unspecified | 3,601 (7.52%) | 1,111 (3.63%) | 1,014 (3.21%) | 677 (2.27%) | < 0.0011 | |
1 Chi-square test.
2 Kruskal-Wallis test.
User survey data analysis
The analysis of survey data included 4,711 responses for the 2015–2016 influenza season, 1,333 responses for 2016–2017, 4,519 responses for 2017–2018, and 5,128 responses for 2018–2019. A summary of all survey questions with the number and proportion of responses per year is shown in Table 3. When asked how likely on a scale of 0 to 10 users were to recommend FNY to a friend or colleague, with 0 being least likely and 10 being most likely, survey respondents were likely to recommend the service overall, with 10 being the most common response for all four influenza seasons. Responses to this question by year can be seen in Figure 3. There was a clear increase in the number of participants who responded 10 to this question, as 1,957 out of 4,711 (41.54%) 2015–2016 respondents reported that they would rate their likeliness to recommend FNY a 10 compared with 2,711 out of 5,128 (52.87%) 2018–2019 respondents. The first influenza season where greater than half of participants rated their likeliness to recommend FNY a 10 was 2018–2019. Responses in the middle of the scale (3–8) decreased, as 399 out of 4,711 (8.19%) 2015–2016 respondents rated their likeliness to recommend FNY a 5 compared with 304 out of 5,128 (5.93%) 2018–2019 respondents, with decreases observed between each influenza season. Responses at the lowest end of the scale (0–2) were fairly consistent across years and were low overall; less than 2% of participants each year rated their likeliness to recommend FNY to a friend or colleague a 0, 1, or 2.
Table 3
2018–2019 user survey responses. Values are represented as n (%).
| RESPONSES N (%) | |||||
|---|---|---|---|---|---|
| INFLUENZA SEASON | |||||
| SURVEY QUESTION | RESPONSE | 2015–2016 | 2016–2017 | 2017–2018 | 2018–2019 |
| N | 4,711 | 1,333 | 4,519 | 5,128 | |
| On a scale of 0 to 10, with 0 being “very unlikely” and 10 being “very likely,” how likely would you be to recommend Flu Near You to a friend? | 5 or lower | 627 (13.31) | 111 (8.35) | 519 (11.48) | 542 (10.56) |
| 6 | 217 (4.61) | 46 (3.45) | 173 (3.83) | 179 (3.49) | |
| 7 | 476 (10.10) | 107 (8.03) | 330 (7.30) | 331 (6.45) | |
| 8 | 779 (16.54) | 209 (15.68) | 700 (15.49) | 709 (13.83) | |
| 9 | 655 (13.90) | 195 (14.63) | 623 (13.78) | 656 (12.79) | |
| 10 | 1,957 (41.54) | 665 (49.89) | 2174 (48.11) | 2,711 (52.87) | |
| How did you first hear about Flu Near You? | Doctor or healthcare worker | 77 (1.63) | 29 (2.18) | 60 (1.33) | 54 (1.05) |
| Family or friend | 391 (8.30) | 98 (7.35) | 288 (6.37) | 219 (4.27) | |
| News story (TV, radio, newspaper) | 589 (12.50) | 166 (12.45) | 346 (7.66) | 1,053 (20.52) | |
| School/university | 151 (3.21) | 27 (2.03) | 99 (2.19) | 83 (1.62) | |
| Workplace | 315 (6.69) | 89 (6.68) | 243 (5.38) | 189 (3.64) | |
| 1,031 (21.88) | 268 (20.11) | 377 (8.34) | 344 (6.71) | ||
| Internet search | 1,208 (25.64) | 323 (24.23) | 605 (13.39) | 534 (10.41) | |
| 20 (0.42) | 10 (0.75) | 14 (0.31) | 12 (0.23) | ||
| Other | 929 (19.72) | 323 (24.23) | 202 (4.47) | 453 (8.83) | |
| I don’t remember | N/A | N/A | 2,285 (50.56) | 2,187 (42.65) | |
| Why did you sign up for Flu Near You? Please select your primary motivation from the list below. | I consider disease tracking to be important and wanted to help | 3,312 (70.30) | 963 (72.24) | N/A | N/A |
| I want to know when people around me are sick | 998 (21.18) | 287 (21.53) | N/A | N/A | |
| I wanted to learn more about the flu | 123 (2.61) | 22 (1.65) | N/A | N/A | |
| I was sick and wanted to submit this information | 134 (2.84) | 27 (2.03) | N/A | N/A | |
| Other | 144 (3.06) | 34 (2.55) | N/A | N/A | |
| Why did you sign up for Flu Near You? Please check all that apply. | I was sick and wanted to submit this information | N/A | N/A | 275 (6.09) | 264 (5.15) |
| I wanted to learn more about the flu | N/A | N/A | 879 (19.45) | 971 (18.94) | |
| I want to know when people around me are sick | N/A | N/A | 2,302 (50.94) | 2,393 (46.67) | |
| I consider disease tracking to be important and wanted to help | N/A | N/A | 3,737 (82.70) | 4,270 (83.27) | |
| I wanted to take part in a citizen science project | N/A | N/A | 1,820 (40.27) | 2,658 (51.83) | |
| I joined as a part of a study or promotion from my area health department | N/A | N/A | 112 (2.45) | 112 (2.18) | |
| A friend or colleague recommended that I join | N/A | N/A | 211 (4.67) | 175 (3.41) | |
| Other | N/A | N/A | 115 (2.54) | 119 (2.32) | |
| Did you receive a seasonal influenza vaccine for the most recent flu season? | Yes | 3,671 (77.92) | 1,085 (81.40) | 3,649 (80.75) | 4,370 (85.22) |
| No | 1,025 (21.76) | 246 (18.45) | 857 (18.96) | 739 (14.41) | |
| I don’t remember | 15 (0.32) | 2 (0.15) | 13 (0.29) | 19 (0.37) | |
| Did participating in Flu Near You influence your decision to get a flu vaccine for this most recent flu season? | Planned to get vaccine, Flu Near You helped to remind | 555 (15.14) | 155 (14.29) | 453 (12.42) | 506 (11.59) |
| No-would have gotten vaccine regardless | 3,047 (83.12) | 910 (83.87) | 3,106 (85.17) | 3,750 (85.81) | |
| Probably would not have gotten vaccine without Flu Near You | 64 (1.76) | 20 (1.84) | 67 (1.84) | 90 (2.06) | |
| Not applicable | N/A | N/A | 21 (0.58) | 24 (0.55) | |
| Do you think that continued participation in Flu Near You might make you more likely to get an influenza vaccine in the future? | No plans to vaccinate | 419 (40.88) | 103 (41.87) | 388 (45.27) | 323 (43.71) |
| No—may vaccinate, not because of Flu Near You | 336 (32.70) | 77 (31.30) | 287 (33.49) | 221 (29.91) | |
| Yes | 270 (26.34) | 66 (26.83) | 182 (21.24) | 195 (26.39) | |
| For classification purposes only, please indicate your level of education. | Less than high school degree | 32 (0.68) | 13 (0.98) | 21 (0.46) | 14 (0.27) |
| High school degree | 962 (20.42) | 310 (23.26) | 786 (17.39) | 772 (15.05) | |
| College degree | 1934 (41.04) | 510 (38.26) | 1,786 (39.52) | 2,077 (40.50) | |
| Graduate degree | 1,744 (37.03) | 490 (36.76) | 1,843 (40.78) | 2,188 (42.67) | |
| Rather not say | 38 (0.81) | 10 (0.75) | 83 (1.84) | 77 (1.50) | |
| For classification purposes only, please indicate your occupation. | Accommodation and food service | 23 (0.49) | 2 (0.15) | 17 (0.38) | 19 (0.37) |
| Administrative and support | 115 (3.29) | 43 (3.22) | 161 (3.56) | 152 (2.96) | |
| Agriculture, forestry, fishing, and hunting | 22 (0.47) | 3 (0.23) | 27 (0.60) | 31 (0.60) | |
| Arts, entertainment, and recreation | 81 (1.72) | 20 (1.50) | 75 (1.66) | 109 (2.13) | |
| Construction | 35 (0.74) | 5 (0.38) | 25 | 34 (0.66) | |
| Educational services | 406 (8.62) | 117 (8.78) | 444 (9.83) | 534 (10.41) | |
| Finance and insurance | 89 (1.89) | 16 (1.20) | 72 (1.59) | 99 (1.93) | |
| Health care and social assistance | 880 (18.68) | 225 (16.88) | 725 (16.04) | 732 (14.27) | |
| Information | 101 (2.14) | 32 (2.40) | 124 (2.74) | 139 (2.71) | |
| Management of companies and enterprises | 52 (1.10) | 11 (0.83) | 59 (1.31) | 33 (0.64) | |
| Manufacturing | 60 (1.27) | 18 (1.35) | 61 (1.35) | 66 (1.29) | |
| Military/armed forces | 12 (0.25) | 1 (0.08) | 10 (0.22) | 13 (0.25) | |
| Mining, quarrying, and oil and gas extraction | 6 (0.13) | 1 (0.08) | 4 (0.09) | 2 (0.04) | |
| Other services | 280 (5.94) | 72 (5.40) | 218 (4.82) | 233 (4.54) | |
| Professional, scientific, and technical services | 443 (9.41) | 106 (7.95) | 436 (9.65) | 553 (10.78) | |
| Public administration | 81 (1.72) | 30 (2.25) | 80 (1.77) | 95 (1.85) | |
| Real estate and rental and leasing | 49 (1.04) | 12 (0.90) | 40 (0.86) | 49 (1.00) | |
| Retail trade | 85 (1.80) | 18 (1.35) | 83 (1.84) | 103 (2.01) | |
| Retired | 1,583 (33.61) | 544 (40.81) | 1,638 (36.25) | 1,909 (37.23) | |
| Student | 55 (1.17) | 12 (0.90) | 29 (0.64) | 39 (0.76) | |
| Transportation and warehousing | 23 (0.49) | 5 (0.38) | 32 (0.71) | 37 (0.72) | |
| Unemployed | 173 (3.67) | 34 (2.55) | 134 (2.97) | 119 (2.32) | |
| Utilities | 10 (0.21) | 4 (0.30) | 11 (0.24) | 16 (0.31) | |
| Wholesale trade | 6 (0.13) | 2 (0.15) | 14 (0.31) | 12 (0.23) | |
Proportion of responses to the survey question “How likely are you to recommend Flu Near You to a friend or colleague?” by influenza season. FNY: Flu Near You.
When asked how they first heard about FNY, “Internet search” was the most common response from 2015–2016 to 2017–2018. Many users said that they first heard about FNY through Facebook for the 2015–2016 and 2016–2017 influenza seasons (1,031 out of 4,711 [21.88%] and 268 out of 1,333 [20.11%], respectively), but only 377 out of 4,519 (8.34%) 2017–2018 respondents and 344 out of 5,128 (6.71%) 2018–2019 respondents reported Facebook as their source. “I can’t remember” was added as an option for the 2017–2018 influenza season, and about half chose this response during that season. It continued to be the most common response for the 2018–2019 influenza season, though there was a decrease from 2,285 out of 4,519 (50.56%) to 2,187 out of 5,128 (42.65%). During the 2018–2019 influenza season, 1,053 out of 5,128 (20.52%) respondents said that they had heard about FNY from a news story, which was an increase from the previous year, when this option was selected by 346 out of 4,519 (7.66%) users, and from the previous high of 589 out of 4,711 (12.50%) users for the 2015–2016 influenza season.
The most common reason that users reported signing up for FNY across all four influenza seasons was “I consider disease tracking to be important and wanted to help.” During the first two influenza seasons included in the study, users were asked to choose one option only, during which time the percentage of users who selected this response was 3,312 out of 4,711 (70.30%) for 2015–2016 and 963 out of 1,333 (72.24%) for 2016–2017. For the 2017–2018 season, additional responses were added, and users were able to select more than one response, at which point the percentage of those who chose this response increased to 3,737 out of 4,519 (82.70%). The second most common response until 2017–2018 was “I want to know when people around me are sick.” This response was overtaken during the most recent flu season by “I wanted to take part in a citizen science project,” a response added for the 2017–2018 influenza season. Users’ motivations to continue using FNY were similar to their reason for signing up. “Flu Near You provides valuable information that I want to contribute to” was the most common reason given in the first two seasons included the study, for which users had to choose only one response between two (the other being “To stay informed about disease activity in my area,” which received about one-third of responses for both years).
For the 2017–2018 and 2018–2019 influenza seasons, more options were added that were worded similarly to those for the question about signing up for FNY, and users were asked to select any response they found applicable. “I consider disease tracking to be important and I want to help” was the most common response. The second-most-common response in 2017–2018 was “I want to know when people around me are sick;” similar to the responses to the question about signing up for FNY, this response was overtaken by “I enjoy contributing to a citizen science project” for the 2018–2019 influenza season. There were 746 out of 2,923 (28.07%) users who selected this option and who reported first hearing about FNY from a news source, compared with 307 out of 2,205 (13.92%) who did not.
Users were asked to report whether they received an influenza vaccine during the most recent season. Most users across the four-year study period reported having been vaccinated, with a range of “yes” responses between 3,671 out of 4,711 (77.92%) for the 2015–2016 influenza season and 4,370 out of 5,128 (85.61%) in 2018–2019. The 2018–2019 influenza season had the highest proportion of “yes” responses, with an increase of 4.21% compared with the year with the second-highest proportion of “yes” responses (2016–2017). When users answered “yes” to this question, they were asked “Did participating in FNY influence your decision to get a flu vaccine for this most recent flu season?” The majority of respondents reported that they would have received a vaccine regardless of their participation in FNY; this response was consistent across influenza seasons, with a percentage range of 83.12% (2015–2016) to 85.51% (2018–2019). Between 11.89% and 15.14% of respondents across the four influenza seasons reported that they had been planning to get the vaccine and FNY helped remind them to do so. Each year, 1.59% to 2.06% of survey respondents reported that they would not have received the influenza vaccine if they were not reminded to do so from FNY. Users who responded “no” to the original question about their vaccine status were asked, “Do you think that continued participation in Flu Near You might make you more likely to get an influenza vaccine in the future?” The most common response to this question, with between 40.88% and 45.27% of unvaccinated users reporting, was “No, I did not get the vaccine and have no plans to do so in the future.” Between 21.24% and 26.83% of users reported that FNY may influence them to receive a vaccine in the future.
Overall, FNY survey respondents report high educational attainment. Fewer than 1% of users each year reported that they had less than a high school degree, and greater than 75% each year reported having at least a college degree. Between 36.76% and 42.67% of users each year reported having a graduate-level education. For occupation, the most common response selected each influenza season was “retired,” with 1,909 out of 5,128 (36.32%) 2018–2019 respondents reporting they were retired. The second most commonly selected occupational category was health care and social assistance, which received at least 10% of responses during all four influenza seasons. Other categories that received many responses each year were educational services and professional, scientific, and technical services.
Most users responded that they are just as likely to report to FNY when they are healthy as they are when they are sick during all four influenza seasons. However, as shown in Figure 4, the percentage of users who selected this option increased across each study year. For the 2015–2016 season, 3,122 out of 4,711 (66.28%) users reported that they were just as likely to report when healthy compared with 4,156 out of 5,128 (81.05%) for the most recent influenza season, and there were clear increases between each season. The percentage of users who report only when sick dropped from 1,532 out of 4,711 (32.53%) for the 2015–2016 influenza season to 642 out of 5,128 (18.37%) for the most recent season. Most users said that when they report symptoms, they report any symptoms that they’ve had during the week and not just those that they believe are associated with the flu. However, 22.59% to 24.46% of users still said that they report only influenza symptoms. There was a higher percentage of users during the most recent influenza season who said that they have never reported symptoms to FNY (500 out of 5,128, or 9.75%) compared with previous years, possibly reflecting the increase in those who report whether they are healthy or sick. In terms of what symptoms FNY users associate with influenza, the most definitive symptoms, with over 75% of users selecting these options during every influenza season, were fatigue, fever, and body aches. Chills/night sweats, sore throat, cough, and headache were also commonly associated with influenza; over 50% of users selected these as perceived influenza symptoms during every season.
Proportion of responses to the survey question “Do you consider yourself more or less likely to report to FNY when you have symptoms of illness?” by influenza season. FNY: Flu Near You.
Most survey respondents self-reported that they submitted reports to FNY “every week or almost every week,” with 3,991 out of 4,979 (80.16%) users selecting this option for the 2018–2019 influenza season. Survey responses to this question were compared with the actual number of reports that users submitted to FNY for the four influenza seasons, and it was found that answers to this question are fairly accurate. For the 2018–2019 season, users who said they submitted reports “every week or almost every week” submitted an average of 25.61 (6.87) reports out of the 33 weeks of the influenza season included in the analysis, which corresponds to a mean participation rate of approximately 78%. This value is compared with a participation rate of approximately 46% for those who said they reported “at least twice a month,” 18% for those who said they reported “sometimes,” and 8% for those who said they report to FNY “not very often” or “never.” Similar trends were also seen for the 2015–2016, 2016–2017, and 2017–2019 influenza seasons. These results are detailed in Table 4.
Table 4
Number of responses to the question “How often do you report to Flu Near You?” Mean number of reports out of 521 and mean participation rate2 for survey respondents for four influenza seasons. Number of responses is reported as n (%), and average number of reports is presented as mean (standard deviation).
| RESPONSE | ||||
|---|---|---|---|---|
| NEVER/NOT VERY OFTEN | SOMETIMES | MOST OF THE TIME | EVERY WEEK OR ALMOST EVERY WEEK | |
| 2015–2016 | ||||
| Responses | 57 (1.55) | 163 (4.44) | 698 (19.02) | 2,752 (74.99) |
| Mean number of reports | 5.77 (6.81) | 10.25 (9.72) | 24.98 (12.82) | 38.26 (11.83) |
| Mean participation rate | 11.10% | 19.71% | 48.03% | 73.58% |
| 2016–2017 | ||||
| Responses | 24 (1.81) | 44 (3.31) | 203 (15.23) | 1,057 (79.59) |
| Mean (sd) number of reports | 2.50 (3.38) | 7.86 (8.15) | 25.01 (14.02) | 41.06 (10.64) |
| Mean participation rate | 4.81% | 15.12% | 48.11% | 78.96% |
| 2017–2018 | ||||
| Responses | 63 (1.40) | 227 (5.03) | 765 (16.96) | 3,456 (76.61) |
| Mean (sd) number of reports | 5.83 (8.56) | 10.93 (10.41) | 26.23 (13.07) | 42.90 (9.47) |
| Mean participation rate | 11.20% | 21.03% | 50.44% | 82.51% |
| 2018–2019 | ||||
| Responses | 49 (0.98) | 221 (4.44) | 718 (14.42) | 3,991 (80.16) |
| Mean (sd) number of reports | 2.6078 (4.9156) | 5.9012 (5.9623) | 15.0615 (8.7417) | 25.6110 (6.8659) |
| Mean participation rate | 7.90% | 17.89% | 45.64% | 77.61% |
1 For 2018–2019, the average number of reports is out of 33, not 52.
2 Defined as average number of reports divided by 52 (33 for 2018–2019).
Science Friday listener analysis
During the study period, 3,196 SciFri listeners registered for FNY, and 2,982 registrants submitted at least one symptom report. An additional 588 household members were registered by users, bringing the total SciFri cohort to 3,570 participants. Overall, 45,702 symptom reports were submitted by SciFri participants, and the median number of symptom reports per week was 1,644 (1,535, 1,710). SciFri users submitted a median of 11 reports (2, 21) during the 2018–2019 season, which corresponds with a mean participation rate of 40.74%. As shown in Figure 5, symptom reports submitted by SciFri participants made up approximately 10–15% of the total weekly FNY symptom reports. Compared with non-SciFri-registered FNY users who submitted at least one symptom report during the 2018–2019 influenza season, SciFri users were younger—55.2 (35.3, 66.3) versus 57.8 (44.8, 66.6) and had a greater percentage of males (38.5% versus 29.5%).
Number of symptom reports per week for all registered Flu Near You (FNY) participants and guests, registered FNY participants, and SciFri participants during the 2018–2019 season.
There were 990 out of 2983 (33.2%) SciFri users who completed the annual 2018–2019 user survey, and they represented 19.31% of all 2018–2019 survey respondents during this year. This cohort was compared with survey respondents who did not sign up through SciFri (Table 5). SciFri users rated their likeliness to recommend FNY similarly to non-SciFri users and also gave high scores overall; 519 out of 990 (52.42%) rated their likeliness to recommend the service a 10, compared with 2,129 out of 4,138 (52.97%) non-SciFri users. There were 88 out of 990 (8.89%) SciFri users and 454 out of 4,138 (10.97%) non-SciFri users who gave a score of 5 or lower.
Table 5
2018–2019 user survey responses and p-values for SciFri users versus non-SciFri users. Values are represented as n (%).
| SURVEY QUESTION | RESPONSE | SCIFRI (n = 990) | NON-SCIFRI (n = 4,138) | P-VALUE |
|---|---|---|---|---|
| On a scale of 0 to 10, with 0 being “very unlikely” and 10 being “very likely,” how likely would you be to recommend Flu Near You to a friend? | 5 or lower | 88 (8.89) | 454 (10.97) | 0.072 |
| 6 | 43 (4.34) | 136 (3.29) | ||
| 7 | 57 (5.76) | 274 (6.62) | ||
| 8 | 138 (13.94) | 571 (13.80) | ||
| 9 | 145 (14.65) | 511 (12.35) | ||
| 10 | 519 (52.42) | 2,192 (52.97) | ||
| Why did you sign up for Flu Near You? Please check all that apply. | I was sick and wanted to submit this information | 18 (1.82) | 246 (5.94) | < 0.001 |
| I wanted to learn more about the flu | 133 (13.43) | 838 (20.25) | < 0.001 | |
| I want to know when people around me are sick | 283 (28.56) | 2,110 (50.99) | < 0.001 | |
| I consider disease tracking to be important and wanted to help | 885 (86.36) | 3,385 (81.80) | < 0.001 | |
| I wanted to take part in a citizen science project | 768 (77.58) | 1,890 (45.67) | < 0.001 | |
| I joined as a part of a study or promotion from my area health department | 8 (0.81) | 104 (2.51) | 0.001 | |
| A friend or colleague recommended that I join | 7 (0.71) | 168 (4.06) | < 0.001 | |
| Other | 22 (2.22) | 97 (2.34) | 0.911 | |
| Did you receive a seasonal influenza vaccine for the most recent flu season? | Yes | 881 (88.99) | 3,489 (84.32) | 0.001 |
| No | 106 (10.71) | 633 (15.30) | ||
| I don’t remember | 3 (0.30) | 16 (0.39) | ||
| For classification purposes only, please indicate your level of education. | Less than high school degree | 0 | 14 (0.34) | < 0.001 |
| High school degree | 69 (6.96) | 703 (16.99) | ||
| College degree | 390 (39.39) | 1,687 (40.77) | ||
| Graduate degree | 519 (52.42) | 1,669 (40.33) | ||
| Rather not say | 12 (1.21) | 65 (1.57) | ||
The reasons for joining the site differed between SciFri users and non-SciFri users. While believing in the importance of disease tracking was the most commonly selected reason in both groups, non-SciFri users weremore likely to select “I want to know why people around me are sick;” 2,129 out of 4,138 (50.99%) non-SciFri users selected this option, compared with 283 out of 990 (28.56%) SciFri users. More SciFri users (768 out of 990, or 77.58%) reported being motivated by wanting to participate in a citizen science project than non-SciFri users (1,890 out of 4,138, or 45.67%).
SciFri users were slightly more likely to report having been vaccinated for influenza than non-SciFri users. A total of 881 out of 990 (88.99%) users reported that they had received a vaccine for the most recent influenza season compared with 3,489 out of 4,138 (84.32%) non-SciFri users. SciFri users are highly educated: 519 out of 990 (52.42%) survey respondents who signed up from SciFri had graduate degrees, compared with 1,669 out of 4,138 (40.33%) survey respondents who did not sign up from SciFri.
Discussion
Citizen science projects have the potential to complement traditional sentinel surveillance systems by reaching populations that do not access health care systems and areas with limited surveillance data; however, the success of participatory syndromic surveillance systems depends on participants who are dedicated to providing accurate data (). Citizen science initiatives play a unique role in supporting public health, particularly for disease surveillance activities (). This role is two-fold in that the data that individuals provide increases our collective understanding of disease risk and transmission patterns while the direct engagement of individuals offers an opportunity to provide information about disease risks and empowers citizens to address needs or concerns in their communities ().
The number of new participants that register for FNY varies each season and is typically influenced by the severity of the influenza season, media coverage, and advertising efforts, which vary each season. For example, more than 7,000 individuals registered in January 2013 after FNY was featured in a prominent national news program. Although the direct impact of the guest implementation is unknown due to the confounding factors listed above, the number of registered participants, the total number of reports, and the median number of reports per week were similar to the previous two years. However, the median number of reports per registered participant increased, and the percentage of registered participants who submitted only one report decreased after guest reporting was introduced. The guest reports also increased the total number of reports during each influenza season. Because of these promising results, FNY will continue the using the guest feature.
FNY survey respondents are highly engaged with the platform based on their higher rates of participation compared with the overall FNY user base. FNY survey respondents generally had altruistic reasons for signing up for and for continuing to participate in the project. Their primary motivation both for joining and continuing to use the site across all four influenza seasons is their belief in the importance of disease tracking, to which they want to meaningfully contribute by submitting their weekly symptom reports. Since desire to participate in a citizen science project was added as a motivation that users could select on the survey for the 2017–2018 season, about half of users chose this for a motivation for joining the site, and this option overtook the more self-serving reason of “I want to know when people around me are sick” during the 2018–2019 season as the second-most-common response. This was especially apparent in the SciFri cohort; 77.58% reported that they signed up for FNY because of a desire to participate in citizen science, whereas only 28.56% reported that they were motivated by wanting to know when people around them are sick. These findings support previous research that users with intrinsic motivations have greater participation frequency and duration (; ; ).
The careful analysis and comparison across years of surveys will drive future development of surveys and recruitment efforts. For example, the option “I don’t remember” will be removed from the question “How did you first hear about FNY?” in future surveys. Also, because most survey respondents stated they were very likely to recommend FNY to a friend, a recruitment campaign that asks users to invite two or three friends at the start of influenza season may be an effective low-cost way to build the user base. This approach was successful for the Australian system, FluTracking.net ().
One limitation of the survey data analysis is that the FNY survey respondents may not be a representative sample of all FNY users. Specifically, the median number of reports submitted within an influenza season by FNY survey respondents was larger than the overall FNY user base. In addition, FNY survey respondents had a higher percentage of vaccination compared with the overall FNY user base. They are also highly educated compared with the general U.S. population. According to U.S. census data from 2018, 35% of adults over the age of 25 have Bachelor’s degrees, and 13% had masters and/or doctorate and/or professional degrees; whereas 80% of FNY survey respondents during the same year (2017–2018 survey) reported having at least a college degree, and 41% of those respondents reported also having a graduate degree (). Importantly, FNY and other systems are also not representative of the general population in sex and age distribution. Females and middle-aged individuals are over-represented in these systems (; ; ). Of note, SciFri users were younger and had a greater percentage of males compared with non-SciFri users.
Despite these limitations, participatory surveillance projects such as FNY engage citizen scientists to contribute their information to support scientific research while providing resources for individuals to learn about disease risk and to track symptoms in the community. The FNY population provides valuable, longitudinal data for both researchers and public health officials to understand patterns of disease in near-real time. This type of data collection directly from the public compliments traditional sentinel surveillance measures and provides a flexible surveillance method for emerging diseases. For example, in response to the COVID-19 pandemic, the FNY team pivoted the crowdsourced reporting structure of the platform to create COVID Near You (CNY), a symptom surveillance tool aimed to use citizen-reported health information to supplement tracking of COVID-19. This platform aims to increase understanding of the burden of COVID-19 across North America, identify hotspots of disease activity, and inform the public on reported symptom behaviors, in the absence of population-level and representative testing (COVID Near ).
In the future, we will assess the impact of the COVID-19 pandemic on user behaviors. Specifically, we will compare the number of new registrations, the frequency of user reporting, and the motivations of users before, during, and after this global pandemic. We will also compare user motivations and reporting behaviors of FNY users with CNY users over the same period. Of particular interest is whether the reporting habits of CNY users who use the SMS text-based response system differ from FNY and CNY users who use the email-based response system.
Conclusions
Following the addition of an option that allows FNY website visitors to submit health reports as guests without registering, we found that the number of registered users who submit only one symptom report decreased and the total number of weekly reports submitted increased. FNY survey respondents are highly engaged with the platform because of their belief in the importance of infectious disease tracking and their interest in participating in citizen science projects. The FNY partnership with SciFri succeeded in registering users highly engaged with science, and these users had notably higher levels of interest in citizen science. Engagement of users is an important aspect of maintaining citizen science projects and ensuring that they are effective in collecting and presenting data.
Data Accessibility Statement
Data is available upon request for public health research. Contact flunearyou@healthmap.org for information.