Interest development

Hidi and Renninger (2006) proposed a four-phase model of interest development and suggested that educational interventions can support development of interest for particular content. Triggered situational interest, stimulated externally by specific situations or learning environments, may be a precursor to reengage similar content leading to maintained situational interest. Maintained interest may evolve into a more enduring self-regulated individual interest in an early emerging phase and a later well-developed phase. Various learning interventions, including videos and digital games, have been shown to trigger positive affective responses that support development of content interests. Science television has a long proven history of fostering children’s interest in science content (Chen 1994; Fisch 2004; Mares et al. 1999; Steinke et al. 2009).

In the last two decades children’s public television series have increasingly included multiple platform experiences, making digital games and offline activities available in conjunction with online and broadcast video. Recent studies of the impact of such multimedia components reveal significant triggering of content-specific interest in youth. An experimental study of Cyberchase showed that fourth graders exposed to math multimedia components demonstrated more interest in school math activities compared to those not exposed to the materials (Fisch et al. 2010). Elementary school children who experienced Spyhounds multimedia components significantly increased their interest in science and in doing science activities at home (Paulsen and Andrews 2014). With respect to the SciGirls format in particular, evaluation that focused on exposure to Season Two episodes and website activities showed a positive influence on the interest of fifth grade girls in carrying out science investigations (Flagg 2012) and in doing hands-on robotics engineering (Flagg 2013).

Applying Hidi and Renninger’s (2006) construct of situational interest leads to the hypothesis for this study that experiencing citizen science via the multimedia platform of SciGirls will trigger interest that is carried into the subsequent FrogWatch USA field session, thereby generating greater interest in the treatment group than in the control group in the citizen science session itself and in doing other citizen science projects.

Sources of self-efficacy

In extending the theory of interest development to informal learning settings, Renninger (2007) suggested that “fun” activities may support successful interactions with science that increase feelings of science self-efficacy. Self-efficacy is a theoretical construct of the social cognitive psychologist Albert Bandura (1977) and pertains to a belief in one’s ability to perform in a given domain of activity. Bandura proposed that people form self-efficacy beliefs as they interpret information from four sources: Physiological state, social persuasion, mastery experience, and vicarious experience. The latter two sources are of interest in this study.

Based on results of engaging in activities themselves (i.e., mastery experience), youth develop beliefs about how well they will perform in similar subsequent activities (Britner and Pajares 2006). Applied to technology-enhanced learning environments such as websites and games, users who have successful interactions may acquire more positive feelings about their competence to do similar tasks. Vicarious experience refers to learning from observations of others performing or modeling tasks in one’s immediate environment or in mass media (Bandura 2001). Self-efficacy beliefs of viewers appear to be most influenced by models whom viewers perceive to be similar to themselves and models who perform tasks that are novel to viewers. (Bandura 1997; Zeldin et al. 2008). Applied to the video medium, the viewer who sees characters similar to themselves succeeding in unfamiliar tasks may conclude: If they can do it, so can I (Steinke et al. 2009).

As children experience success or failure with tasks and as they observe others modeling tasks, they may adjust their perceptions of their own efficacy for related activities. Studies of science, math, engineering, or technology (STEM) self-efficacy have involved mostly high school and older students and have applied correlational or predictive analyses (as reviewed by Usher and Pajares 2008). Few studies have employed treatment interventions to examine if and how science efficacy beliefs may be modified in children. Studies of change in children’s science self-efficacy in the digital gaming literature have focused on mastery experiences with immersive virtual worlds. Fifth graders demonstrated significant increases in science self-efficacy after playing in a virtual world about landforms and ecosystems (Meluso et al. 2012). Sixth graders increased significantly in science self-efficacy after exposure to a problem-based multimedia program about the solar system (Liu et al. 2006). Experience of virtual worlds also significantly influenced middle school students’ beliefs in their ability to do scientific inquiry (Bergey et al. 2015; Chen et al. 2014).

In addition, few studies have investigated how children’s STEM self-efficacy beliefs are influenced through vicarious experiences of observing media-based models. Schunk and Hanson (1985) found that children who watched videos of same-age, same-gender peers successfully modeling subtraction operations improved significantly in subtraction self-efficacy compared with those who observed a teacher model or no model. Fourth graders exposed to Cyberchase episodes in which characters model math problem solving sustained their pre to post levels of self-efficacy regarding school math problems, whereas a control group exposed to a history video series declined in their school math confidence ratings (Fisch et al. 2010). Viewing of Season One episodes of SciGirls significantly influenced fifth grade girls’ feelings of competence for doing some but not all engineering design tasks, compared to a control group that viewed a literacy video series (Flagg 2010).

The treatment intervention of the study reported here provides vicarious experiences with citizen science via SciGirls videos showing same-age, same-gender, ethnically diverse peers successfully doing citizen science. The intervention also provides mastery experiences with citizen science via games of animal identification and data collection. The study tests the hypothesis that treatment girls, who are exposed to both vicarious and mastery experiences of SciGirls, will demonstrate greater feelings of efficacy with respect to a subsequent citizen science session (FrogWatch USA) and future citizen science activities, compared to control girls not exposed to SciGirls prior to FrogWatch USA.

Learning

According to Renninger and Hidi (2006), for situational interest to move into later phases of interest, content knowledge needs to develop concurrently. Participant content knowledge generated through citizen science projects ranges from acquisition of specific facts and concepts to more sophisticated understanding of the process and methods of scientific research (Dickinson and Bonney 2012; Shirk et al. 2012). Rather than focusing on facts or process specific to FrogWatch USA, the content acquisition assessed in this study is knowledge of the general practice of citizen science, with the belief that youth who acquire awareness and understanding of what the citizen science community does will be more interested in future citizen science participation.

Descriptions of contributory-type citizen science projects typically emphasize four common characteristics: (a) members of the public are active participants; (b) following a specific citizen science protocol is critical to contributing to valid science research; (c) professional scientists use citizen science data to generate knowledge about real-world problems; and (d) a virtual collaboration occurs where participants and scientists share data online (Bonney et al. 2009; Bonney et al. 2014b; Dickinson and Bonney 2012; Wiggins and Crowston 2011). These four unique features of citizen science practice are communicated in different ways in the FrogWatch USA sessions and in the SciGirls multimedia experience. In the FrogWatch USA sessions, girls learn about citizen science practice via a leader-presented PowerPoint and participant activities followed by field-based frog call data collection. In SciGirls multimedia, the features are embedded in narrative form via video stories and in interactive experiences via games of animal observation and identification of animal characteristics.

One might assume that providing multiple examples of concepts via different learning platforms can yield a better understanding of content; however, little research has examined this issue with children. Fisch’s 2013 review of three informal education multimedia projects for children points out that different combinations of learning platforms (video, web, hands-on) produced greater effects than a no-media control group but that more media did not necessarily produce the best impact. For example, using the multiple media platforms of Cyberchase, Fisch et al. (2010) found that exposure to narrative-based video plus online games showed more consistent effects on students’ mathematical problem solving compared with video alone, games alone, video plus games plus hands-on activities, and no exposure. Fisch and his colleagues speculated that cross-platform impact depends on providing multiple examples of a concept in different contexts to increase the possibility of transfer to new situations. The Cyberchase study found evidence that transfer of learning occurs across media such that children can apply their learning from one medium to their learning in a second medium when they are closely aligned in content (Fisch et al. 2010). With exposure to SciGirls videos and games, the treatment group experiences multiple exposures to citizen science practice and could transfer that learning to enrich their subsequent experience of the leader-guided FrogWatch USA sessions. Consequently, we hypothesize that the treatment group will demonstrate a better understanding of the features of the practice of citizen science compared to the control group girls.

Experimental design

In this mixed methods embedded experimental study (Creswell and Plano Clark 2011), quantitative data serve to address the hypotheses about treatment and control groups, whereas qualitative data serve to illuminate the statistical results by exploring how treatment participants perceived the impact of SciGirls. Graduating fifth grade girls were randomly assigned to a treatment group or control group. The treatment group experienced SciGirls web activities at home prior to a live FrogWatch USA session. The control group attended the FrogWatch USA session without prior SciGirls experience. After their frog session, all girls completed an online survey and subsequent phone interview. To equalize participant experiences, the control group accessed the SciGirls multimedia after their survey and interview.

Sites

To represent the six citizen science projects covered by SciGirls multimedia, one project was chosen for the study’s live citizen science experience. FrogWatch USA was selected because its content and female leader availability best fit the launch schedule of materials on the SciGirls website. Female leaders were preferred for this study because the videos feature female adult mentors. Five FrogWatch USA sites participated: Fort Wayne Children’s Zoo, Fort Wayne, IN; Greenville Zoo, Greenville, SC; Hiram College, Hiram, OH; Nashville Zoo/Owl’s Hill Nature Sanctuary, Nashville, TN; and Roger Williams Park Zoo, Providence, RI. Site institutions and leaders received stipends to cover expenses and staff time.

Participants

Recruitment. Sites recruited participants through a researcher-provided parent information letter distributed to neighborhood schools and home-school networks. The letter invited graduating fifth grade girls to participate during a weekend at the end of the school year in a free multimedia enrichment experience in nature and science involving the local FrogWatch USA institution and PBSKids.org. The letter described project requirements and referred interested parties to an online application and permission form to be completed by both the child and a parent. Recruitment conformed to IRB requirements (E&I Review Services, #13086-01).

Through the online application, parents and children confirmed their interest and availability and gave permission or assent to participate in (a) enrichment activities of a FrogWatch USA session on either one of two weekend days and (b) online PBSKids.org activities during the week before or after the frog sessions. The application also obtained parent permission and child assent for children to participate in post online surveys and a phone interview. To prevent unintended early access to the SciGirls website, recruitment communications mentioned only PBSKids.org. Participants learned about SciGirls just prior to when their group, either treatment or control, was to participate in the SciGirls portion of the study. Participants received a small monetary incentive upon completion of all activities and data collection.

Stratified random assignment. To support individual random assignment to either the treatment group or control group, girls had to confirm availability for both Saturday and Sunday FrogWatch USA session days. Stratifying on self-reported minority status, girls were randomly assigned to groups for each site by coin toss. A total of 108 girls were confirmed to participate: 54 for each group. A total of 98 attended FrogWatch USA sessions: 49 for each group. The 10 girls who did not attend their assigned session reported no-show reasons related to unforeseen scheduling conflicts (e.g., illness, soccer playoff) rather than to session content or study requirements.

Age. The sample of 98 fifth grade girls included 10-year-olds (27%), 11-year-olds (69%), and 12-year-olds (4%), with an average age of 10.8 years. Age distribution and mean age did not differ significantly between treatment and control groups.

Minority representation. Based on 2013 U.S. census tables, racial or ethnic minorities comprise 45% of the population of fifth grade female students. To represent this population, the study sample included 38% who self-identified as a minority, including 12% Black/African American, 9% Hispanic/Latina, 3% Asian, and 13% multiethnic. The term minority in this study refers to the ethnically diverse subsample, whereas non-minority refers to girls who self-identified only as white. Stratified random assignment equally distributed minority girls into the groups for treatment (n = 18) and control (n = 19).

Preexisting interest in nature and science. Despite individual random assignment, groups might still differ in their preexisting interest in nature and science, which could influence group post comparisons. Consequently, an interest scale was developed and validated for inclusion in the online recruitment application. In the Girls’ Interest in Nature and Science Scale (GINSS, Flagg 2015), the following nine statements were presented in random orders with 5-point Likert ratings from strongly disagree to strongly agree: (a) “It’s fun to do science activities,” (b) “It’s fun to collect things from outdoors,” (c) “I want to understand how things in nature work,” (d) “I like to observe birds, butterflies, bugs or other things in nature,” (e) “I like to identify things in nature,” (f) “I enjoy watching nature shows,” (g) “I like to hear about new discoveries in science,” (h) “I enjoy reading about science,” (i) “I like talking about science topics with others.”

Analysis of the scale’s psychometric properties with a sample of 212 nationally distributed fifth grade girls assessed in school classrooms showed a unidimensional scale with loadings greater than 0.60 on one factor and high scale reliability (ordinal alpha = .88; Gaderman et al. 2012). For the study sample of 98 girls, high scale reliability was also obtained (ordinal alpha = .87). The mean and median summated GINSS scores for both the treatment and control groups were 4.4 out of 5, and thus groups did not differ significantly. Thus, while the girls participating in the study’s enrichment experiences rated their interest in nature and science quite high, the treatment and control groups were equivalent in their interest upon starting the study. GINSS scores also did not differ significantly by age group or minority/non-minority status.

Materials and procedure

SciGirls multimedia experience. Distributed over a 10 day period prior to their FrogWatch USA session, treatment girls spent about two hours completing the SciGirls experience. To verify treatment fidelity, 100% of study participants confirmed their viewing of the assigned videos that included “Frog Whisperers” and “Feathered Friends” and one of two other videos: either “Sky Girls” or “Flower Power.” These videos are associated respectively with the citizen science projects of FrogWatch USA, Celebrate Urban Birds, S’COOL, and Nature’s Notebook (videos available at http://goo.gl/VTN73b).

Each SciGirls episode presents a detailed story of a group of girls who participate in the respective citizen science experience. As an example of the video narratives, “Frog Whisperers” begins by introducing the group: Two African American girls, one Latina girl, and one Caucasian girl. The girls meet their mentor, a FrogWatch USA program leader, who teaches them how to recognize different frog calls. She takes them to a nearby stream to explore frog habitats, where they are amazed at the experience of holding a live frog. As night falls, they hike to a rural stream and record the frogs they hear, following the FrogWatch USA protocol. The next night they visit an urban park to monitor frogs so they can compare the two locations. They are surprised how many fewer frogs they hear in the urban park. The next morning they enter their data into the FrogWatch.FieldScope.org website and enjoy a Skype call with a scientist who will use their data. Finally, the girls prepare a presentation to share their FrogWatch USA experience with the public at a local Earth Day celebration.

In addition to viewing three videos, participants played two games. “Creature Features” is a classic platformer game in which players listen for and locate frogs and birds with particular animal features as they move through four environments (pbskids.org/scigirls/games/creature-features). The game relates to the citizen science projects of FrogWatch USA and Celebrate Urban Birds. “Rule the Roost” is a simplified citizen science experience in which participants join an online team to complete a real-world data collection task that changes monthly (pbskids.org/scigirls/games/rule-the-roost). The question of the month during the study period was “When looking for robins, does time of day matter?” which relates to the “Feathered Friends” SciGirls episode and the Celebrate Urban Birds citizen science project. A majority of treatment girls reported completing “Creature Features” (90%) and entering data into “Rule the Roost” (76%). Girls who did not complete games noted website usability difficulties.

FrogWatch USA session. At each of the five sites, the same FrogWatch USA-certified leaders ran both the treatment and control group sessions to limit a teacher effect. Three sites were randomly assigned to run treatment groups on Saturday and control groups on Sunday; two sites reversed this order. Leaders did not know which day involved which group of girls nor had leaders viewed any of the SciGirls materials prior to their sessions. Eight to 12 girls attended each session (M = 9.8). Leaders attempted to make the two FrogWatch USA sessions equivalent in content and experience, so if something occurred or did not occur during Saturday’s session then an effort was made to replicate it on Sunday’s session. For example, one site discovered, caught, and discussed a snake in the rafters on Saturday; and then on Sunday, the site also presented a snake to the girls. Or when technology did not operate on Saturday, then that same technology also was not used on Sunday. Fortunately such unique occurrences happened only on Saturdays and accordingly could be added to or subtracted from the Sunday sessions.

The FrogWatch USA staff of the Association of Zoos and Aquariums provided leaders with an agenda to follow for a 2.5 hour session. The session included (1) introduction and overview; (2) activity to hear calls of four local frog species; (3) PowerPoint and Q&A on citizen science and background of who, what, where, when, why, and how for frog call monitoring; (4) PowerPoint on data collection protocol and importance; (5) snack break; (6) activity to learn frog call intensity levels; (7) activity to learn and practice identification of at least four local frog calls; (8) real data collection in the field; (9) return to classroom for concluding review.

To establish implementation fidelity, sessions were observed with an agenda checklist. Although teaching styles varied across sites, observations confirmed that the treatment and control groups at each site received the same content presentations for approximately the same periods of time. A few differences occurred among sites related to technology and weather; for example, one site that unexpectedly lost Internet access presented PowerPoint information on the leader’s own tablet for both sessions, and one site that had bad weather for both sessions substituted a prerecorded audiotape of local frogs to simulate the data collection activity.

Measures

Frog Activity Interest Scale. To assess the hypothesis about interest in the FrogWatch USA session and in doing other citizen science programs, the interest/enjoyment subscale from the Intrinsic Motivation Inventory (Deci and Ryan n.d.) was modified. The interest/enjoyment subscale includes agree-disagree statements that are often reworded to refer to specific activities. The subscale has shown high reliability for a variety of youth participants in different settings and target activities (e.g., Hong and Masood 2014; Wilde and Urhahne 2008). To assess scale reliability, a pilot study was implemented with 82 fifth grade girls in CA, FL, and NC who had participated in NASA’s S’COOL citizen science project. Four statements from the interest/enjoyment subscale were modified to refer to the S’COOL activity (e.g., “I had a lot of fun while doing S’COOL”). In an online survey, the statements were integrated with other post activity statements and presented in random order for each respondent, who rated on a 5-point disagree-agree scale. Reliability was high for the pilot interest scale (Cronbach’s α = .85). All statements contributed positively to scale reliability and were retained. For this study, the scale statements were modified to refer to the frog activity: (a) “I enjoyed doing FrogWatch very much,” (b) “I had a lot of fun while doing FrogWatch.” (c) “I thought FrogWatch was very interesting,” (d) “I would like to do more citizen science activities.” The statements for the Frog Activity Interest Scale were presented in an online post survey in random order for each respondent. Data for the study sample of 98 girls produced high scale reliability (Cronbach’s α = .85).

SciGirls Activity Interest Scale. A second activity interest scale was implemented to assess the treatment group’s interest in their SciGirls experience. The scale instructions reminded respondents of their SciGirls experience as “videos and games,” and the four scale statements were modified to refer to SciGirls: (1) “I enjoyed doing SciGirls very much;” (2) “I had a lot of fun while doing SciGirls;” (3) “I thought SciGirls was very interesting;” (4) “I would like to do more on the SciGirls website.” These statements were presented to treatment girls in an online post survey in random order for each respondent (Cronbach’s α = .88, n = 49).

Frog Activity Self-Efficacy Scale. To assess the hypothesis about feelings of efficacy with respect to the citizen science activity of FrogWatch USA, we modified the youth science self-efficacy scale being developed by the Cornell Lab of Ornithology DEVISE project. We drew on four statements with the highest reliability (Cronbach’s α = .82; e.g., “I’m good at learning science topics”), as reported in Cornell’s validation process with 106 male and female fifth graders (N. Porticella, personal communication, March 10, 2015). For the pilot study, the statements were modified to specify the S’COOL citizen science activity in place of general “science topics” and “science activities,” because self-efficacy is assessed typically at the level of the specific task (Schunk and Meece 2005). The statements were integrated randomly into a list of other S’COOL statements in an online post survey and administered to 82 fifth grade girls (Cronbach’s α = .72). All statements contributed positively to scale reliability and were retained. For this study, the four scale statements were modified to refer to the frog activity: (a) “I was good at following instructions for FrogWatch,” (b) “I could learn about frog and toad calls as quickly as others my age,” (c) “I could do FrogWatch just as well as others my age,” (d) “I was good at learning about frog and toad calls.” Statements for the Frog Activity Self-Efficacy Scale were presented to the study sample in the online post survey in random order for each respondent (Cronbach’s α = .70, N = 98).

Online surveys. Following their FrogWatch USA session, both treatment and control girls completed at home an online survey via Survey Monkey (http://www.surveymonkey.com). For both groups, surveys were completed within an average of 1.7 days after the frog session. In the survey, both treatment and control girls completed the Frog Activity Interest and Self-Efficacy Scales. They rated questions about interest in finding out more about other citizen science projects, likelihood to look for another citizen science project to do in the future, and belief in one’s ability to be good at doing other citizen science projects. They also answered an open-ended question about what they learned about citizen science. Treatment girls additionally checked off which SciGirls activities they had completed, answered the SciGirls Activity Interest Scale, and identified how much and how the onscreen video girls were like them and not like them.

Additional survey questions addressed the appeal of FrogWatch USA and SciGirls, and control girls completed a survey after their delayed SciGirls experience. These results are not included in this paper but were provided as formative feedback to the institutions running the projects.

Structured phone interview. Within an average of two days after completing their FrogWatch USA session post survey, both treatment and control girls answered phone interview questions that were worded the same for everyone to obtain reliable information across sites. Eight questions, with no follow-up probes, elicited information to assess girls’ understanding of the four common features of citizen science practice: (a) anyone can participate, (b) participants use the same protocol so data can be combined and be high quality, (c) data can help real scientists come to real conclusions, and (d) a wide community of scientists and volunteers work together and share data to which the public, as well as scientists, have access. To illuminate the interest and efficacy quantitative ratings, the treatment girls responded to four additional questions addressing how their SciGirls experience affected their interest in participating in and their perceived ability to do FrogWatch USA and also other citizen science projects generally.

Data analysis

An alpha level of .05 was used for all statistical tests. The study’s rating scales are ordinal measures with non-normal score distributions; therefore, non-parametric one-tailed Mann-Whitney U tests were employed for treatment and control group comparisons. Rather than comparing means as the parametric t-test does, the non-parametric Mann-Whitney U test compares medians or mean rankings of responses of the two independent groups to determine if they reflect the presence of a beyond-chance difference in the larger populations that they represent (de Winter and Dodou 2010). Comparisons between minority and non-minority subsamples with two-tailed Mann-Whitney U tests constituted exploratory analyses because of small sample sizes. To measure the strength of associations among quantitative ratings, the non-parametric Spearman’s rank order correlation was used.

Qualitative data comprised transcriptions of open-ended survey and interview questions. After training on responses from 10 randomly chosen participants, the author and a second researcher, who did not participate in data collection, independently coded participant data that were presented in random order without group identification. Coding differences were clarified by discussion with a third researcher. Coding of the treatment groups’ qualitative responses to questions about interest, self-efficacy, and similarity to video girls yielded Cohen’s unweighted kappa coefficients ranging from .81 to 1.0. Content analysis verified that all four learning outcomes were addressed in the SciGirls videos and games as well as during each of the FrogWatch USA sessions. Coding of participant responses into the four learning outcomes yielded Cohen’s kappa of .89. Group differences of learning mean scores were analyzed with t-tests.

Interest

Interest in FrogWatch USA experience. The hypothesis was supported that the SciGirls multimedia experience would trigger higher interest in the FrogWatch USA session for the treatment group compared to the control group. Results from a Mann-Whitney U test presented in Table 1 show that mean ranks of the 5-point Frog Activity Interest Scale were significantly higher for the treatment group than for the control group. Also, for the treatment group, interest activated by prior exposure to SciGirls multimedia was moderately and significantly associated with interest in the subsequent FrogWatch USA session, as revealed by the correlation between Frog Activity Interest ratings and SciGirls Activity Interest ratings, r_s(47) = .58, p < .001.

Additionally, within each of the treatment and control groups, the subsamples of ethnically diverse minority girls reported significantly greater interest in their FrogWatch USA session compared with non-minority girls (Table 1).

In the treatment group’s post interview, a plurality of the 49 girls (47%) pointed out that their interest in the FrogWatch USA session increased because the SciGirls “Frog Whisperers” video prepared them for their own frog citizen science session (e.g., “The video about the frogs – it showed me what kind of stuff we might be doing, so it kinda made me excited.”). Treatment girls also felt that SciGirls videos showed them that FrogWatch USA would be fun (31%), and they learned why citizen science data collection is important (10%). Some treatment girls referenced increased interest in FrogWatch USA due to the SciGirls games, explaining that “Creature Features” exposed them to frog listening and frog facts prior to the FrogWatch USA session (18%), and the activity in “Rule the Roost” introduced them to citizen science data collection procedures (8%). A small number of treatment girls (10%) did not feel that their SciGirls experience influenced their interest in their subsequent FrogWatch USA session.

Interest in pursuing other citizen science projects. The study’s interest hypothesis also posited that after their FrogWatch USA session, groups would differ in their interest in pursuing other citizen science projects, but that was not supported statistically in treatment and control comparisons. Both treatment and control groups rated themselves moderately interested in finding out more about citizen science projects (Table 2) and somewhat likely to look for a future citizen science project to do (Table 3). However, the treatment had a significant impact on minority girls: Within the treatment group, minority girls compared with their non-minority counterparts showed significantly greater interest in finding out more about citizen science projects (Table 2) and significantly greater likelihood to look for a citizen science project to do in the future (Table 3). Within the control group, minority and non-minority subsamples did not differ significantly in their interest ratings in pursuing citizen science (Tables 2 and 3).

Interest in finding out more about other citizen science projects and likelihood to look for a future citizen science project to do were significantly correlated with interest in their FrogWatch USA session for both treatment and control groups (Table 4, column 2). For the treatment group, interest in their prior SciGirls experience was significantly correlated both with interest in finding out more about other citizen science projects and with likelihood to look for another project to do in the future (Table 4, column 3). In addition, significant correlations in Table 4, column 4, show that the more the treatment group felt that the girls in the SciGirls videos were like them, the greater their interest in finding out more about other citizen science projects and the greater their likelihood to look for another project to do in the future.

In the treatment group’s post interview about the influence of SciGirls on interest in doing citizen science projects other than FrogWatch USA, 22% of girls reported that fun portrayals of citizen science by the video girls increased their interest (e.g., “It looked really fun when you saw it.”). Another 22% reported being influenced mainly by video models of how girls could participate in citizen science (e.g., “It kinda showed me what they did and what I could do too.”). Treatment girls also described wanting to help scientists and their community as the video girls did (14%) and observed that citizen science activities presented in SciGirls can help them learn different types of science (14%). Some treatment participants (14%) agreed that SciGirls increased interest in other citizen science projects but could not explain how, and 14% did not feel that SciGirls influenced their interest.

Self-efficacy

Perceived efficacy in FrogWatch USA session. The hypothesis was not supported that treatment girls would demonstrate greater feelings of efficacy with respect to the FrogWatch USA session compared to control girls. In their ratings on the 5-point Frog Activity Self-Efficacy Scale, both the treatment group (Mdn = 4.75) and the control group (Mdn = 4.5) displayed high confidence about their abilities in the frog citizen science session. Also minority and non-minority girls did not differ significantly in their ratings within either group.

Although treatment and control groups did not differ significantly in their quantitative efficacy ratings, the majority of treatment girls reported in their post interview that SciGirls multimedia helped them do better in their frog session. Treatment girls explained that “Frog Whisperers” video and “Creature Features” game increased their knowledge of frogs and their calls (39%), made them aware of what would happen in the frog session (20%), or taught them about the data collection protocol (16%); for example:

“When we were doing Name That Tune [in FrogWatch USA session], I knew some of the sounds because of the feature creature game. The FrogWatch video, it helped me more because it was showing the girls – they were listening to frogs. It showed the part where you have to wait 30 minutes after sunset and then wait 2 minutes for the frogs to settle in and then 3 minutes more of silence to listen to the frogs.”

Another 18% of treatment girls agreed that SciGirls helped them do better in the FrogWatch USA session but could not describe that influence, and 6% did not feel that SciGirls influenced them.

Perceived efficacy in other citizen science projects. The treatment group (Mdn = 3 out of 4) did not differ significantly from the control group (Mdn = 3) in their ratings of how much their respective experiences made them believe that they could be good at doing other citizen science projects (Table 5). However, within the treatment group, exposure to SciGirls had a significantly stronger impact on minority girls’ ratings compared to non-minority girls that they would be good at doing other citizen science projects (Table 5).

In their post interview, 22% of treatment girls felt that they learned from SciGirls videos about the process of other projects and that made them believe they could do a good job in a different citizen science project (e.g., “I know the process I need to go through. For the birds, I do the same thing like for the frog, just at a different time.”). Treatment girls (16%) also explained that they could do other citizen science projects because the video girls were like them (e.g., “I saw a lot of girls my age doing it, so I thought I could do it too.”). Treatment girls described being motivated to help scientists by participating in citizen science (8%) or felt that they learned science information from SciGirls that would help them do a good job in a different citizen science project (8%). Another 16% of treatment girls were positive about the impact of SciGirls but could not describe what in their experience influenced their belief that they could do a good job in another citizen science project, and 29% did not feel that SciGirls influenced their feelings of future efficacy (e.g., “I already thought I could do a really good job on a citizen science project.”).

Perceived similarity to video girls. Treatment girls felt they were like the video girls a lot (18%), mostly (49%), a little (29%) or not at all (4%). They explained that like the video girls, they both liked nature and/or science (49%), had the same personal interests or activities (20%), were the same age (12%), liked going outdoors (10%), liked to learn (10%), liked to have fun (6%), or liked helping animals or the environment (4%). Many fewer treatment girls reported how the video girls were not like them. These participants said they did not share the personal interests or activities of the video girls (10%); did not like going outside like the video girls (8%), or were shy, unlike the outgoing video girls (6%).

Of the 13 video girls appearing in the four SciGirls episodes, nine (69%) are minorities, including African American, Latina, and Asian. One third (33%) of minority treatment girls felt that the video girls were a lot like them compared to 10% of non-minority treatment girls. Minority viewers felt they were significantly more like the video girls than non-minority viewers (Mdn = 3, M rank = 30.39 vs. Mdn = 3, M rank = 21.87, respectively) z(49) = 2.18, p < .05. Minorities also expressed significantly stronger interest than non-minorities on the SciGirls Activity Interest Scale (Mdn = 4.8, M rank = 31.86 vs. Mdn = 4, M rank = 21.08, respectively) z(49) = 2.61, p < .01.

Learning

Girls’ survey and interview responses about the practice of citizen science were coded as showing or not showing understanding of each of four citizen science features, yielding a learning score of 0 to 4. As predicted in the study’s learning hypothesis, the treatment group (M = 2.84, SD = 0.83) displayed significantly better understanding of citizen science features compared with the control group (M = 2.47, SD = 1.24), one-tailed t(83) = 1.72, p = .04. Minority and non-minority subsamples did not differ in their learning results.

Treatment girls demonstrated significantly better understanding of the four citizen science features that: (a) anyone can participate; e.g., “Anybody can participate in a program like this;” (b) participants use the same protocol so data can be combined and be higher quality; e.g., “Your data wouldn’t be as correct if you didn’t use the same procedure;” c) data can help real scientists come to real conclusions; e.g., “Citizen scientists can collect data and report it and that helps the professional scientists so they would know what’s going on in the areas where the citizens are;” and (d) a wide community of scientists and volunteers work together and share data to which the public, as well as scientists, have access; e.g., “I get to participate in activities that can share my data with other people and see how my data is related to others in my area.”

Interest

As predicted by the interest model of Hidi and Renninger (2006) and the study’s hypothesis, the intervention of SciGirls successfully triggered situational interest that carried into the subsequent FrogWatch USA session. Although both groups rated their quantitative interest in the FrogWatch USA session very high, those exposed to the SciGirls multimedia experience displayed a significantly higher content-specific interest than those not exposed. Treatment girls reflected that their interest increased because the SciGirls videos and games prepared them for their session, showed them that the citizen science experience would be fun, and explained why citizen science data collection is important. These interest triggers parallel sources of interest summarized by Renninger and Hidi (2011), including features such as novel information, intensity, and meaningfulness.

The study also looked at whether interest in citizen science projects became more general as a result of the informal enrichment experiences. Quantitative ratings did not reveal significant differences between treatment and control groups. Interest in pursuing other citizen science projects was strongly and significantly correlated with interest in the FrogWatch USA session, implying the importance of the appeal of the live project-based activity in raising citizen science interest generally. Interest triggered by prior experience of SciGirls, as indicated by quantitative ratings, did not appear strong enough to move the treatment girls significantly further than control girls toward a second level of maintained situational interest in which children reengage in similar content (Hidi & Renninger 2006). Nonetheless, the treatment group’s quantitative interest in pursuing other citizen science projects was strongly and significantly associated with high interest in their prior SciGirls experience and their perception that the girls in the videos are very similar to themselves. Treatment girls noted that the videos portrayed how girls like themselves can participate in fun citizen science projects.

Self-efficacy

The SciGirls multimedia components provided vicarious citizen science experiences via peer models in the videos and mastery experiences through the “Creature Features” and “Rule the Roost” games. Both of these sources were hypothesized to influence girls’ perceptions of their own efficacy in the FrogWatch USA session and in other different citizen science projects. Treatment girls identified with the video girls in a variety of ways, and they felt they did better in the FrogWatch USA session because SciGirls video and games prepared them in advance for the frog content and activities. Additionally, treatment girls felt that they could do a good job in a different citizen science project because they learned from SciGirls about the process of other citizen science projects and saw girls like themselves in the videos successfully participating in other projects.

Although the qualitative data showed a self-perceived influence on citizen science efficacy of prior exposure to the SciGirls’ experiences, results from the quantitative ratings of efficacy in citizen science did not reveal statistically significant differences between the treatment and control girls, contrary to the hypothesis. Both groups rated their self-efficacy very high for the FrogWatch USA session and for how well they believed they could perform in other citizen science projects. Several possibilities could explain why no significant quantitative difference was found for self-efficacy between treatment and control girls. The first possibility is that the vicarious peer-models and mastery game experiences of SciGirls are not truly vehicles that influence self-efficacy. It may also be that both groups of girls entered the study with high feelings of general science self-efficacy that were then manifested in the high citizen science efficacy ratings, irrespective of their treatment or control experiences. Further, the quantitative measures may lack sensitivity to discriminate differences at the upper levels of efficacy beliefs; such a ceiling effect may lead to a potentially erroneous conclusion that a treatment has no effect (Cramer and Howitt 2004). Alternatively, it is possible that the FrogWatch USA leaders were so effective in making girls feel successful in the citizen science session that prior exposure to SciGirls could not add significantly to the impact of the live mastery experience. Supporting this interpretation is Britner and Pajares’ (2006) finding that for (largely white) middle school girls, mastery experience was a stronger predictor of science self-efficacy than vicarious experience. However, the authors point out that vicarious experience still contributes to positive self-efficacy, particularly when models are perceived as similar to the observers, as was the case for this study’s treatment girls.

Learning

Confirming the learning hypothesis, the experiences of the treatment girls led to a significantly better understanding of the features of the practice of citizen science compared to control girls. The treatment group learned about citizen science practice via SciGirls’ videos and games and transferred that learning to enrich their understanding of citizen science as presented in the leader-guided FrogWatch USA session. Successful interactions in “Creature Features” and “Rule the Roost” as well as exposure to multiple instances of citizen science projects in SciGirls videos helped treatment girls understand that anyone can participate in citizen science; that participants use the same protocol so data can be combined and be high quality; that citizen science data can help real scientists come to real conclusions; and that citizen science brings together a wide community of scientists and volunteers to work together and share data to which the public, as well as scientists, have access. These findings contribute to the learning literature that multiple examples and different experiences of concepts via a variety of platforms and settings can yield a better understanding of content (Fisch 2013).

Impact on minority girls

Treatment group results that showed a stronger impact of SciGirls on a heterogeneous collection of minority girls (i.e., African American, Latina, Asian, multiethnic) are tantalizing and invite further research and replication. Within the treatment group that was exposed to SciGirls, minority girls demonstrated significantly higher interest than non-minorities in finding out more about other citizen science projects; greater likelihood to look for a future citizen science project to do; and stronger belief in their efficacy to be good at doing other citizen science projects. Minority treatment girls also liked their SciGirls experience more and felt they were more like the video girls compared with their non-minority counterparts. These same variables did not show significant minority vs. non-minority differences in the control group that was not exposed to SciGirls. The method of random assignment to groups, stratified by minority status, permits us to conclude that the SciGirls multimedia experience contributes to the minority girls’ more positive responses; yet this conclusion must be qualified by the small sample size.

The very few studies that have explored the development of STEM interest or self-efficacy specifically with ethnically diverse youth give some support for our findings. An intervention with African American and Latina middle school girls utilized story narratives about groups of female peers to motivate engagement with e-textile projects (Erete et al. 2015). The authors’ observations suggest that narratives triggered interest in science activities because the girls identified with the characters and the real-world context. Similarly, after exposure to the SciGirls narratives in this study, the minority fifth grade girls explained their increased interest in future citizen science activities because they learned about girls similar to themselves doing fun activities helpful to their community. Usher and Pajares’ extensive 2008 review of sources of self-efficacy notes only two studies that examined the relationship of STEM self-efficacy and ethnic background in youth. These studies suggest that ethnically diverse youth might interpret the sources of self-efficacy differently than their white counterparts, with more emphasis on vicarious experiences. Within our treatment group, the minority girls appeared to be more responsive than the non-minorities to the vicarious experiences of seeing same-age, same-gender video girls of various ethnicities doing novel citizen science activities.

The study also showed that minority girls in both treatment and control groups reported significantly greater interest in their FrogWatch USA citizen science session compared with non-minority girls, even though prior interest in nature and science was equivalent for the subsamples in each group. One can only speculate on mediating variables to explain these results; for example, a recent survey regarding out-of-school STEM programs revealed that minority parents have a more positive opinion of such programs than non-minority parents (Afterschool Alliance 2015), hinting at a possible influence of parental attitude on youth interest in the FrogWatch USA experience.

Limitations

To meet the requirements of a controlled experimental design, the study managed delivery of video, game, and citizen science experiences in a manner that may not entirely reflect natural usage, although effort was made to support typical encounters both at home and at the FrogWatch USA sites. The sample chosen for the study limits generalization for findings to preteen girls, even though both SciGirls video and games, and certainly FrogWatch USA sessions, are experienced by boys too. It would be intriguing to see if similar experiences by boys yield significantly different results. The girls in this study were also very interested in nature and science prior to participating, so the impact of SciGirls materials may be different for children with lower interest.

Implications for citizen science field

The study findings indicate that using SciGirls multimedia prior to a FrogWatch USA session improves engagement and learning in citizen science for preteen girls. For logistical reasons, FrogWatch USA was chosen to represent the contributory model of citizen science projects covered in the SciGirls project. Other highlighted projects in Season Three of SciGirls include Celebrate Urban Birds, Monarch Larva Monitoring Project, S’COOL, Nature’s Notebook, and Seafloor Explorer. Although each of these projects has unique elements, if we can generalize from the FrogWatch USA findings, prior exposure of preteen girls to the SciGirls materials associated with a respective project may increase project interest, help prepare girls for their citizen science sessions, and improve understanding of citizen science practice.

Studies of gender disparities in interest in STEM careers at the start of high school conclude that more effective efforts are needed to engage girls at earlier ages (Sadler et al. 2012). Informal and formal educators working with preteen girls could use SciGirls’ resources – video in particular – independently of specific citizen science projects to introduce the practice of citizen science generally and generate interest and participation in citizen science more broadly. Exposing young girls to the process and fun of citizen science via video will increase not only knowledge about citizen science, as demonstrated by the study’s results, but also might stimulate participation in citizen science projects not highlighted in the videos.

This study’s findings of significant impact on minority girls’ interest and efficacy in citizen science are certainly not definitive but underscore the notion that the potential of media to address ethnic issues in informal science education is ripe for future investigation. Further research with larger samples might focus on ethnically diverse or homogeneous groups and collection of a much wider variety of background information and mediating variables to shed light on how peer-oriented multimedia influence youth outcomes, how minorities might respond differently to contributory model citizen science experiences, and how groups differ in their pathways of science interest and self-efficacy.

Finally, to contribute to potential comparisons across citizen science and science programs more generally, the Girls’ Interest in Nature and Science Scale (GINSS), developed and validated as part of this study, is available for use by others (Flagg 2015).

Conclusion

The SciGirls multimedia experience contributed significantly to girls’ experience of citizen science. Exposure to SciGirls triggered interest that was carried into a subsequent live citizen science session and increased preteen girls’ understanding of the unique practice of citizen science, with a special influence on minority girls’ interest and self-efficacy. SciGirls multimedia shows youth the process and practice of citizen science, demonstrates the fun of citizen science, and presents peers with whom girls can identify.