Planning

CitSciVirtual 2021 was planned and organized by a volunteer-led conference program committee supported by two CSA staff members and a contracted conference coordinator. The program committee included 29 volunteers (including authors Hils and Silka), a conference program committee chair (author Cawood), and an emeritus conference program committee chair (author Hjarding). Within the broader program committee, there were four subcommittees responsible for making and implementing decisions related to keynote talks, posters, workshops, and special events. Additionally, three coordination committees provided guidance and support for the conference on topics related to 1) equity and development of Community Guidelines (including author Kelty), 2) communications, and 3) engagement technologies.

The decision to hold CitSciVirtual 2021 over the full month of May was based on committee members’ experiences attending other virtual conferences during the pandemic. They reported finding it difficult to fully focus on the conference when professional and personal obligations were amplified by the pandemic. They often felt burnt out after sitting through up to eight hours of conference programming, especially on consecutive days. The program committee attempted to lower this barrier to participation by designing shorter daily blocks of content across a month in hopes that attendees might be able to spend one or two focused hours several times per week engaged in conference activities.

In the early stages of CitSciVirtual planning, the committee focused on establishing the conference goals and the value for attendees (Table 1). The six goals were driven not only by the desire to develop a program that would have direct benefits to the c*science community, but also to be responsive to the needs of the likely attendees, as expressed through surveys conducted by CSA in May and June 2020 (CSA 2020).

The conference was designed to prioritize interactions and dialogue among participants and to help participants build new skills. This priority was established in response to insights from Metimeter check-in polls during the 2021 conference, from 2021 CSA Connect comments during and post-conference, and from 2019 and 2021 post-conference surveys (CSA 2019b; Fischer and Cho 2021). These goals were major drivers in the decision not to have individual talks. Instead, the program committee focused on more interactive experiences for attendees such as discussion-based poster sessions and hands-on workshops (Supplemental File 1: Appendix A). They also moved away from the traditional conference abstract submission format which is not familiar to those outside of academic spaces, by opting for simple user-friendly Google Forms (Supplemental File 2: CitSciVirtual 2021 poster proposal submission form and Supplemental File 3: CitSciVirtual 2021 workshop proposal submission form) and clear guidelines for submission and specific questions (Supplemental File 4: CitSciVirtual 2021 poster submission guidelines and Supplemental File 5: CitSciVirtual 2021 workshop submission guidelines).

Platform

In tandem with conference planning, CSA was also in the process of implementing a new membership platform, CSA Connect (www.connect.citizenscience.org), that would allow members to connect more easily and could host a variety of content like short courses and live events. CSA Connect became the conference’s primary platform for schedules and event reminders, asynchronous material, including posters and workshop content, and social interactions. Within the platform, participants were able to upload and comment on posters, develop courses and workshops, chat with other participants, connect to live events, and watch/replay recorded events. To prepare attendees for the new interface and to help presenters construct interactive content, the program committee provided orientations to the space in the month before CitSciVirtual as well as a live help desk available throughout the conference. However, both attendees and organizers faced challenges navigating the new platform and implementing all available features. The project evaluation results indicated that frustrations with the platform were a major drawback to the conference, with more than 30% of evaluation respondents mentioning challenges connecting with others on the platform (Fischer and Cho 2021).

Program

The CitSciVirtual program was a mix of keynotes, interactive poster sessions, workshops, and social events. All virtual content was available through the conference platform to paid CitSciVirtual 2021 attendees. Keynotes were streamed live, posted publicly, and did not require conference registration to view.

CSA Connect synched with Zoom, the video conference software popularized in the months before this conference. Each conference session could host up to 100 participants and system-generated, simultaneous closed captions were available. Live sessions, video recordings, and alternate audio/visual files provided personalized options for presenters to convey their message, and most sessions were recorded using Zoom. Keynote and workshop sessions wove together pre-recorded and live content to allow smooth viewing for participants. Additionally, poster presenters shared recorded audio and video to guide attendees through their projects as though they were in-person. A benefit of the pre-recorded elements, especially for posters, was that presenters could pre-record in multiple languages; about 10% of posters were recorded in languages other than English, including recordings in French, Spanish, Portuguese, and Italian.

Goal 1: Networking and connection opportunities

CitSciVirtual 2021 attracted a variety of participants. Over half (52%) identified their primary field as ecology or conservation, 35% percent identified as members of the environmental justice community, and 33% as part of the education community. Many participants were affiliated with higher education institutions (51%), although there was significant participation from people who work in nonprofits (26%), nature centers (25%), and informal learning spaces (25%). Most of the attendees (84%) are not new to the field, and noted their engagement in c*science for three or more years. Despite efforts to focus on justice, equity, diversity, and inclusion (JEDI) in the design of the event, CitSciVirtual was diverse only in some respects. Attendees did not represent a wide range of racial, gender, or educational diversity. The vast majority of attendees were white (74%), identified as women (77%), and had a graduate degree (80%).

Though the conference was homogenous in some respects, the majority of attendees (74%) reported that they had opportunities to engage with people from other disciplines. Additionally, most (60%) said that they had opportunities to engage with people from diverse racial and ethnic backgrounds during the conference. While interacting with like-minded colleagues is beneficial, conference attendees also value opportunities to explore beyond their educational, geographic, discipline, and institutional silos of knowledge (Rich et al. 2020). Breaking these barriers increases the quality and consistency of best practices in the realm of c*science.

The interactive and collaborative sessions were successful, with more than 80% of survey respondents saying that they would like to see more discussion-focused sessions at conferences going forward. Presenters in the collaborative poster sessions reported enjoying the opportunity to connect with others working on similar topics. Feedback shared with program committee members and CSA staff indicated that overall, individual talk sessions were not missed, and there is a strong preference for interactive and collaborative sessions. It was noted that the dedicated facilitator was an integral element to the success of these sessions. However, some reported missing the opportunity to engage deeply with a specific poster and presenter. Although these opportunities were available through the CSA Connect platform, these functions (finding event links, liking, sharing, and messaging) and were not well utilized.

One area of improvement is the desire for more informal opportunities to meet with and connect with other attendees. Although the program committee tried to address this, not everyone was aware of or able to participate in the live events or able to utilize the available CSA Connect functions. In retrospect, the conference could have focused more on networking opportunities that wove the diverse CSA community together and allowed for critical and exploratory discussions beyond the bounds of our own echo chambers.

Goal 2: Skill-building opportunities

While the conference planning committee attempted to ensure ample opportunities for skill building with the focus on workshops during CitSciVirtual, this did not always work as intended. While there was inspiring content created and presented by workshop organizers, overall, this often did not translate into meaningful experiences for workshop attendees or presenters. Several factors may have contributed to this shortfall such as how asynchronous material was presented, expectations that attendees would prepare in advance, and the ways that attendees were expected to engage in the live workshops. To highlight some of the rich content created for the workshops, CSA staff assisted presenters that reworked workshop content into a stand-alone webinar or mini-course offered through CSA.

One area for improvement is providing additional support to emerging researchers in the c*sciences. Our conference attempted peer connection sessions specifically for students, but generally these events were poorly attended. Potentially transitioning to a peer-organized model for these interactive events could better meet the needs of emerging researchers in virtual spaces and help improve attendance (Newman et al. 2021; Sarabipour et al. 2021). Specific support could include allocating a proportion of presentations for emerging researchers alongside senior researchers and within primetime slots (Rich et al. 2020; Sarabipour et al. 2021) while guaranteeing feedback for all interested presenters through a poster competition or interactive poster discussions. Other options could be to encourage teams of presenters to highlight emerging researchers, which may free up more time for established researchers to provide feedback, mentorship, and guidance for other attendees’ research and career development (Sarabipour et al. 2021).

Goal 3: Feedback opportunities

Virtual posters did provide a unique opportunity for participants to give and receive feedback on their work before, during, and after the conference. Conversations appeared on poster discussion threads before the conference officially opened, and some conversations continued after the official end of the conference. Asynchronous commenting on posters or private messaging enabled engagement with other participants that was agnostic of time zones. This format also supported engagement in a variety of languages other than English.

However, feedback experiences were mixed in terms of opportunities for participants to share and receive feedback on their work. On one hand, virtual spaces reduce thresholds for participation for some individuals by increasing time to craft communications and control over interactions with other participants (Vervoort et al. 2020; Sarabipour 2020; Sarabipour et al. 2021). On the other hand, the visibility and permanence of communication in virtual spaces with unknown sizes of audiences of unfamiliar others may lead some presenters to be more reserved in their communication to avoid risk of plagiarism or embarrassment (noted as the Hawthorne Effect by Stein et al. 2021). Instituting more options for blinding and anonymity may help reduce this barrier with a potential sacrifice in transparency. Special protections can be instituted to protect unpublished/preliminary data by abstaining from posting presentation materials from social media platforms or providing options for non-recorded sessions that are not uploaded to CSA Connect (e.g., Spencer, Doran, and Burcham 2021).

One main area of improvement is to increase the chances for feedback even though the platform can be challenging. On the basis of results from CitSciVirtual 2021, during the 2022 conference, the Conference Program Committee planned pre-conference sessions for poster presenters to share their virtual poster, in which they could get feedback on their poster while also giving feedback to others or gaining new ideas for their work. More importantly, the CSA Connect algorithm shows the most recent comments at the top of the platform. To increase poster topic visibility, certain people created “playlists” of a group of posters around a specific topic. Finally, content should remain up for attendees to view in their own time after the conference ends, including keynote speeches and workshop sessions, as well as all virtual posters.

Goals 4 and 5: Inclusion and accessibility

Virtual conferences can provide a venue for participation that has the potential to expand the reach of an in-person conference, and we saw some of this in CitSciVirtual. Presenters and attendees came from all over the world, and the number of countries represented nearly doubled from previous years of in-person conferences. There were also presenters from nontraditional backgrounds, including high school students and individuals researching medical issues of personal significance, that are not typical of traditional academic conferences. The vast majority of attendees (more than 90% for workshops and more than 80% for poster collaborative sessions) reported feeling welcome and included in live sessions.

Despite this, the new platform resulted in a significant accessibility barrier for some. Many users struggled to find information and engage effectively through CSA Connect. There was a help desk and informational articles available throughout the event and CSA staff held tutorials on how to navigate the site. Even with these aids available, many attendees found the platform frustrating and had difficulty finding the links to the live Zoom sessions or the event schedule, as well as difficulty trying to download and/or watch the asynchronous materials. The CSA Connect platform, Mighty Networks, was also not compatible with screen readers, and most individual presenters did not include closed captions despite encouragement from event organizers. These shortfalls were felt by some attendees and presenters who prefer languages other than English or rely more heavily on either audio or visual content. Furthermore, the internet connections were not reliable for all attendees and presenters, leading to exclusion from synchronous content. The digital gap specifically affected our rural or international attendees, and we expect internet connectivity issues disproportionately impacted other underrepresented and minoritized populations as noted by other pandemic-era conference organizers (Spencer et al. 2021). The availability of prerecorded presentations and recorded live sessions helped to overcome connectivity challenges and language barriers. But most hosts during live sessions forgot to turn on the closed captions, and much of the richness of discussion occurred in simultaneous sessions without as much activity in asynchronous chat features.

The intention to provide a supportive and productive virtual environment for all attendees was not fulfilled in transition to the virtual conference space, which was a common experience for research conferences even before the pandemic (De Saá-Pérez et al. 2015; Nielsen, Bloch, and Schiebinger 2018). Accessibility concerns persisted after the transition to a virtual conference even though we reduced the safety and physical accessibility concerns about travel options and venues where conferences are usually held (Shirk, Cooper, and Bowser 2018; Sarabipour et al. 2021). Additionally, there was a reduced registration fee scaled to reduce the financial barrier to participation for those of specific identity-groups defined by CSA (Supplemental File 1: Appendix B). CSA continues to update their registration fee scales to be more inclusive of its audience. For example, CSA has reduced rates for students, retirees, K–12 educators, environmental justice and Indigenous practitioners, and project volunteers. But we must acknowledge that these fees are only one part of the cost to fully participate in a conference, and diverse participants benefit from a conference that transcends traditional meeting formats.

After summing the accessibility and inclusivity challenges faced during the transition to virtual conferences and more generally in academic conferences (reviewed by Sarabipour et al. 2021), we conclude that one area of improvement is to acknowledge the limited voices that could participate in this conference and think about steps that would allow them to participate in the future. The conference did incorporate Black voices in the HBCU session and with the kick-off keynote speaker, and some Indigenous voices with the workshop bundle, “Participatory Research and Indigenous Knowledge.” However, anecdotal evidence collected during sessions and social events identified shortcomings in conference elements that were intended to embrace disabled, student and early-career, and international communities. For the 2022 conference, CSA intentionally targeted student and early-career people with a mentorship program to guide them through the conference. Furthermore, for the 2023 hybrid conference, CSA should invite the local and regional high schools and colleges to include more Latinx and Indigenous voices.

Goal 6: Virtual and in-person conference synergy

CitSciVirtual 2021 significantly impacted the planning process for all CSA future events, as well as CSA operations and policies. One outcome of the event was the Code of Conduct, which has been shared with several different communities and supported by the CSA community (Supplemental File 9: CitSciVirtual 2021 community guidelines). Building on lessons learned from CitSciVirtual, the 2022 CSA conference included a virtual core program accessible to attendees around the world, as well as regional (in-person or virtual) events to help participants connect to their local c*science community. The major themes that emerged from submissions and discussions during CitSciVirtual, including increasing diversity and inclusion in c*science, forming partnerships, and sustaining projects through time, formed the basis for the 2022 conference core program.