Knowledge-building as described by discourse and social network analysis

The relationships among the knowledge found in discourse are “the most important indicators of learning activity” for a knowledge-building community (Oshima, Scardamalia, and Bereiter 1996; p. 132). Discourse analysis of knowledge-building projects with upper elementary and middle school students has focused on the types of knowledge connections that students make (Oshima et al. 1996), the conceptual advancement provided by students during online discourse (Hakkarainen and Sintonen 2002), the levels of explanations that students use to build knowledge (Hakkarainen 2003), and the importance of questions (Lai and Law 2013). The evolution of these schemes has resulted in a recent paper that documents “good moves” in knowledge-building, highlighting seven dialogic actions that promote knowledge creation and critical thinking (Bereiter and Scardamalia 2016).

Discourse analysis of online knowledge-building among youth has indicated that even primary school students can work together to build knowledge (Chen, Scardamalia, and Bereiter 2015), and that this process results in enhanced understanding of science content (Resendes et al. 2015). Lin and Chan (2018) found that fifth grade students’ problem-centered posts were predictive of enhanced understanding of epistemic cognition (i.e., the understanding of how people think about the nature of knowledge and knowing in science), while theory-building posts were predictive of gains in understanding science content. Lia and Law (2013) identified a developmental difference between how middle schoolers and high schoolers used questions and explanations to advance knowledge. Though both groups asked sophisticated questions in their online discourse, these questions resulted in higher-level explanations only among older students.

Though the utility of social network analysis to study knowledge-building has been tested and verified (Oshima, Oshima, and Matsuzawa 2012), studies that examine the relationships between students to assess knowledge-building are rare. Using this relational approach, Zhang et al. (2009) found that opportunistic groups that evolved based on a common goal provided the most distributed network structure, such that students engaged with a wider range of participants across the community and at a level similar to that of their teacher. Palonen and Hakkarainen (2013) used social network analysis to understand the interactions between students and found that there were large individual differences in the number of contributions made by elementary school students, and that females facilitated the majority of knowledge building.

Participation in online citizen science communities, as described by discourse and social network analysis

Like the knowledge-building literature, discourse analysis of online citizen science projects is more prevalent than the use of social network analysis to study volunteer interactions. Discourse analysis of discussion board posts has been conducted to learn more about users who participate in citizen science, the relationship between the amount of discourse and data contributions, and whether online discussions help users to feel a sense of community (Aristeidou et al. 2015, 2017; Huang, Cheng, and Huan et al. 2018; Liberatore et al. 2018; Luczak-Roesch et al. 2014; Tinati et al. 2014; Tinati, Luczak-Toesch, and Simperl et al. 2015; Tinati, Luczak-Toesch, and Simperl et al. 2016).

One of the most consistent findings across these studies is that a small number of volunteers do the majority of the work. This pattern is true regarding both the amount of data submitted and for participation in online discussion boards (Aristeidou et al. 2017; Curtis 2015, 2018; Luczak-Roesch et al. 2014). The presence of a “super user” group is such a constant that researchers have used the phenomenon to categorize new projects and to make recommendations about leveraging these groups to help sustain online citizen science communities (Ono, Ikkati, and Enoto 2018).

Of the studies in the citizen science literature, the approach used by Huang et al. (2018) is the most similar to that used by knowledge-building researchers. Examining the scientific discourse of citizen scientists as they engaged in modeling and problem solving online, Huang and colleagues found that the timing of facilitator input was most important in supporting collaboration when they initiated and framed discussions and as they helped to wrap up discussions. In addition, the use of leading questions and sharing strategies to further the investigation each played a critical role for participants in advancing knowledge related to planning and problem solving. This evidence is similarly found in the network-based analysis of the Weather-It project, which demonstrated that some volunteers actively engaged in the project by creating new investigations and data, while others simply commented on the mission and data posts provided by others (Aristeidou et al. 2015). Earlier work examining the relational data from WeatherBlur has suggested that the project’s online community model was successful at breaking down traditional hierarchies of learning (Kermish-Allen, Peterman, and Bevc 2018). This study extends these findings by answering two research questions:

1. Which forms of discourse encourage knowledge-building in online citizen science investigations?
2. Which users, if any, are the principal drivers of online citizen science investigations?

Study context

WeatherBlur consists of a non-hierarchical knowledge-building community in which participants function as both co-creators of knowledge and active learners via citizen science projects. Working as a community of practice (Wenger 2000), students, teachers, scientists, and community members come together and work toward a common goal, sharing actions and interactions to promote knowledge-building in relation to local citizen science investigations.

There are no predefined investigation topics in WeatherBlur. Instead, investigations are formed through online discourse as participants engage in an iWonder process to propose and hone questions that can be defined as a formal investigation. Once the investigation is defined, the community works to design data-collection protocols by building on existing methods, leveraging both the place-based and scientific expertise of community members. Anyone in the online community can then join the investigation to participate in knowledge-building.

The qualitative and relational data for this study were collected as part of an investigation that focused on the November 2016 king tide in the United States, Maine. King tides are defined as the high tides that occur when the moon is closest to the Earth. This investigation topic was initiated by a fourth grade student interested in learning more about how the high water from the king tide would affect local beaches, roads, and buildings in her community. Her post prompted a larger discussion about the potential impact in other communities and how the effects could be measured. A scientist in the WeatherBlur community joined the group based on his interest in adding hyperlocal data to the few existing tide monitoring stations up and down the coast which contribute to a statewide tide-gauge database. Participants worked together during an initial three-week formative discussion period to refine the research question and define the data-collection protocols that supported the formal investigation. The King Tide Investigation was designed to gather three types of data: Photographs and videos of sites before, during and after the king tide; measurements of a high tide before the king tide, along with calculations of mean sea level; and the height of the king tide with some basic wind and other weather information. Student participants often completed these tasks as a class.

Online discourse within the investigation took place over seven weeks, beginning with planning conversations that occurred almost exclusively between teachers and scientists. Then, on the day after the king tide, a staff member asked students to share their observations and findings. Students provided a number of responses, including those posted in response to a class assignment to share data collection procedures and observations. The final contributions to the King Tide Investigation were also posted via a class assignment in which students uploaded data presentations to describe their results.

Participants

Thirty users participated in the King Tide Investigation, including two scientists, one facilitator, four teachers, two community members, and 21 students. Students were in grades 3–5. Five schools participated in the investigation, including three small, island-based schools with grade level sizes ranging from one to 14 students. The two remaining schools were more traditional in size. Most students participated in the investigation as an entire class. The sample for this study includes only students who made online contributions to the seven-week investigation. During that time, participants contributed either online discourse or photos via the online community’s discussion board. A total of 53 conversations generated 250 comments.

All online activity was recorded by server-side activity logs, including comments, responses, and photos posted. These served as the dataset for the present study. The activity log data were downloaded by the research team at the end of the investigation and compiled into a series of comma-separated files for coding and analysis. Discussions were identified through a unique numeric “parent” identifier that was used to link each individual comment to prior and related discussion posts; this function allowed the research team to see the growth and flow of discussion over time. Row-separated time-stamped entries of each comment were used to define the unit of analysis for both the discourse analysis and social network analysis. Comments also were clustered into discussion “threads” as a second unit of analysis.

Discourse analysis

A discourse analysis was conducted to replicate the approach used to study knowledge building. Online posts were coded and analyzed using NVivo 11. Two series of codes were of interest. The first series documented the context during which online discourse occurred. The second was used to document how knowledge was shared and built through the online community. Codes were assigned on a turn-by-turn basis to discussion board posts. Consensus coding was conducted for all posts. Two researchers coded each post independently and then compared codes; all disagreements were discussed and the final code was agreed upon.

Social network analysis

Social network analysis was used to identify the patterns and interactions of discourse and exchange between users in the online community over time and to identify the principal drivers of the investigation (i.e., those individuals who were most important to the functioning and growth of the network). Each comment in the activity log contained information that uniquely identified who posted the comment (i.e., the sender) and the user of the “parent” comment to which the post referenced (the receiver). This combination of sender-receiver (e.g., Sue responded to Kate’s comment) represents one unit of analysis. The social network was generated from the listing of this information captured in the log. Analyses were conducted in the statnet suite of packages in the R software platform (Wasserman and Faust 1994; Handcock et al. 2008; R Core Team 2016).

Once the network was created, the analysis examined not only how individual users participated in the investigation, but also how their participation contributed to the larger discussion and the discussions of other users. The data were first analyzed to identify variations across individual users, evaluating the number and types of connections they had with others. Two primary measures were used to document the number of comments that users made and received (degree centrality), and how often users were placed in a position between two other users and thus with the potential to serve as a “go-between” (betweenness centrality) in the discussions (Freeman 1978).

The second part of the analysis measured the interactions and patterns of exchange between two or more users, reporting on the reciprocity and core-periphery structure of the network (Wasserman and Faust 1994). Reciprocity measures the mutual exchange in the knowledge-building process. For example, if Sue posts a question and receives a comment from Kate, reciprocity reports how often Sue responds to Kate to complete the social interaction. This reflects an incremental measure of knowledge-building, whereby reciprocity is only one part of the larger process.

The core-periphery analysis was used to differentiate the various “layers” of the network to identify principal drivers. To uncover the “layers,” the analysis calculated k-index (k_s) scores to provide an individual measure of a user’s position relative to the “core” or center of the investigation. The k-index reflects a user’s aggregate number of connections, centrality in the network, and “role” as go-between in relation to others, relative to size of the network. This measure was calculated for each user and the user’s scores were then ranked. The greater the index value, the more critical the role a user played in the overall network. Relative to network position, the results of this analysis offer a more nuanced understanding of the individual and collective dynamics contributing to user discussions.

Research question 1: Which forms of discourse encourage knowledge-building in online citizen science investigations?

To answer the first research question, trends were compared between discourse that was used to initiate new conversation versus discourse that responded to existing conversation. The totals for each set of codes, along with the results for chi square tests that compared the portion of instances used to initiate versus respond to conversation, are presented in Table 2. A Bonferroni procedure was used to control for Type 1 error; results were considered significant at the .003 level.

A total of 53 conversations were initiated during the investigation. An additional 197 posts were made during the investigation. Most were coded as responses (n = 150; 75%) resulting in a total of 203 comments that were compared in the analysis. As shown in Table 2, background knowledge was the context coded most often, followed by data. The only context used differently across initiations and responses was analysis, which was used more often to initiate than to respond to conversation.

Knowledge connections seemed to function in distinct ways depending on whether a comment was an initiation or a response. Knowledge connections to initiate a conversation were significantly more likely to widen then deepen the conversation, while responses were significantly more likely to deepen the conversation only. For example, the photo in Figure 1 was presented by a student who initiated conversation with the following widening, then deepening post: “We graphed the high and low tides for a month before the king tide. We noticed the tides made a pattern. It kind of looks like a mountain.” In response, a scientist deepened the conversation by saying, “This is an excellent graph…. The camera angle is good too because it allows the eye to look at the trends easily. You might make an overlay of the phases of the moon to see if that helps explain the bumps and valleys. Your explanation can also talk about why the width of the blue band varies.”

The level of explanation provided in posts did not vary significantly across initiations and responses. Facts were coded most often, followed by full and then partial explanations. Differences were found in the use of some conceptual advancement to initiate rather than respond to conversation. The following post from a teacher provides an example of an initiation with some advancement: “Check out the photographs taken by [a] drone over Brown’s Boatyard, North Haven, Maine, during the King Tide on Tuesday, November 15, 2016. There are more under observations.”

Photos were also used significantly more often to initiate rather than respond to existing conversation. Figure 2 shows a photo of a map that was posted by a scientist in preparation for the king tide event, with the following comment: “Here comes a low pressure center (L) up the coast from the south. It is not a very big storm but could push a little more water ashore during the king tide.”

The use of questions did not vary across initiations and responses. When questions did occur they did not seem to advance the conversation; fewer than half resulted in an answer (44%). Most questions asked for a simple clarification such as the following example: “This looks great. Where is it located?” to ask about a water gauge. Explanation-seeking questions also were common, such as the following question from a teacher, “I wonder how the predicted wind is going to affect the zero tide tomorrow. We will still try to calibrate our gauge at 1:37 PM, but I am wondering if the reading should be considered suspect. Any thoughts?” In summary, given that initiations were defined as new topics that yielded at least one response, conversation that focuses on analysis, the use of widening, then deepening comments, and providing some advancement can each be considered strategies that encourage new online conversation. Deepening comments offer a strategy for sustaining conversation once initiated.

Research question 2: Which users, if any, are the principal drivers of online citizen science investigations?

The results presented thus far have focused on how different types of discourse functioned within the context of the King Tide Investigation. This section focuses on the participants who contributed those responses. Given the small sample size and uneven distribution of sub-groups, statistical analyses were not conducted by group for the discourse analysis. Percentages are reported throughout for descriptive purposes.

First, differences in the King Tide Investigation were explored by user group. Online conversation was initiated most often by students (42%), followed by teachers (36%), and then scientists (22%). Similar portions of the responses were posted by scientists and students (38% and 36% of responses coded, respectively); slightly fewer were posted by teachers (26% of all responses).

Looking at the context of online dialogue, each user group dominated different portions and thus played a distinct role in the investigation. Over half of the background knowledge, for example, was provided by scientists (55%). Scientists were also most likely to ask questions (60% of the instances recorded). Discussion of procedures was dominated by teachers (54%). Teachers also submitted approximately half of the photos that were posted to the online discussion (52%). Teachers and students were equally likely to share data (43% and 41%, respectively). Students provided the majority of the data presentations that were coded to analysis (59%).

With regard to knowledge-building constructs, user groups differed slightly in their overall use of knowledge connections (Table 3; overall constructs are in bold text). Scientists and teachers made a similar portion of the contributions to this construct, and at a higher level than students. The use of specific knowledge connections varied by user group. Over half of the widen responses were from teachers, and students contributed more than half of the deepen, then widen posts. Scientists contributed the greatest portion of deepen comments, and widen, then deepen comments were provided at equal levels by scientists and teachers, and in greater portion compared to students.

The largest portion of levels of explanation was provided by students, with scientists and teachers providing similar lower portions. All groups contributed a similar portion of facts. Scientists and students provided a greater portion of both partial and full explanations compared to teachers.

Scientists provided the greatest portion of comments that promoted conceptual advancement overall, followed by teachers, and then students. Teachers contributed the highest portion of comments coded as some advancement, followed by scientists, and then students. Scientists provided the majority of comments coded as strong advancement, followed by students.

Though these results show some variability by user group, the number and type of contributions seem similar overall. Students and teachers, for example, were similar in the number of initiations provided, and students and scientists were similar in the number of responses. There was also variability by group in the knowledge-building constructs used, but these differences were not based on the complexity of the discourse. These findings provide evidence to support the non-hierarchical design of the online learning community overall.

Next, social network analysis was used to document the importance of different users and groups to the King Tide Investigation. Each user had an equal opportunity to post to the online discussion space. In Figure 3, each symbol represents a different user and the shape reflects a different user type. The size of the symbol reflects the total number of comments posted (i.e., sent) by each user, and the arrowed line represents who commented to whom, including self-responses (loops). Collectively, this network diagram illustrates the exchange of posts and comments between users on the discussion board.

As the figure shows, several users were very active, posting frequently; the top three users included one participant from each user group: Scientist A (#5 in Figure 3) posted 55 times, Teacher B (#15) made 45 postings, and Student C (#21) made a total of 21 comments within the discussion (out-degree centrality). These values differ from the number of comments or replies received by users (in-degree centrality). For example, Scientist A received 34 responses (in-degree) versus the 55 comments made as part of the investigation discussion. In contrast, some users received more comments than they posted, e.g., Scientist B (#4) received 18 responses with respect to her 11 posted comments.

The mutual exchange of information reflects another critical aspect of the knowledge-building process. As the arrows in the figure show, many interactions were one-way and not reciprocated. Of the discussions initiated by users in this investigation, 35% were mutual exchanges versus 39% that were one-way (asymmetric) and 26% that were self-responses. The overall measure of edgewise reciprocity among users was r⁴ = 1.568, which can be interpreted as the relative log-odds of a discussion post or comment given a reciprocation, versus the baseline probability (50/50) of a post or comment occurring in the network (Butts 2008). In other words, where all users have an equal opportunity to respond to posts, a comment by User A to User B is about 56% more likely to receive a response from User B than other users in the discussion space, “closing the loop” on the discussion and helping to build the investigation.

On average, betweenness centrality was 16.17, with a median of 0.23 (interquartile range of 0 to 6.93), indicating that many users were frequently positioned in the network between other users. This position contributes to the user’s potential to serve as a broker, or go-between, for other users based on his or her position in the network. The interactions between users are also important when considering the exchanges that contribute to the building of knowledge. The users found to have the highest betweenness centrality scores were Scientist A (#5, C_B = 183.7), Teacher B (#15, C_B = 94.0), and Student C (#21, C_B = 55.2), the same users who posted the highest number of comments. The next three scores were all students (#27 – C_B = 54.8, #23 – C_B = 25.3, and #18 – C_B = 23.4) who took an active role and served as the go-between in discussions between teachers and scientists.

When these results are combined with the outcomes of the core-periphery analysis and the user rankings, the larger picture of the characteristics of principal drivers in the network emerges. While Student B (#19) may not have been the most “chatty,” making seven comments and receiving nine, the location of these posts occurred at critical junctures within the larger discussion space and engaged some of the most active users, including Scientist A (#5), Teacher B (#15), and Student C (#21). Student B (#19) also frequently served as a go-between for two different user groups (teachers and scientists). Both of these results contribute to Student B’s higher k-index ranking (k_s index score = 14). The results are similar for Student D (#27), who also ranked highly (first quartile, k_s = 14) with modest levels of commenting (n = 11) and frequent brokerage engagement with other user types.

Looking at user attributes across quartiles based on k-index scores (range: 0–23), each of the top three contains a mix of both adult and student users, providing evidence to support the non-hierarchical nature of the project’s design. The top quartile includes Teacher B, Scientists A and B, and a staff member from the lead organization, each of whom have an equally high k-index score, indicating that they are the innermost core of the network. Four students from School 1 round out the top quartile (k_s range: 14–23), including Student C.

When quartile membership was compared with the discourse coding, a clear pattern emerged. The number of posts was highest for those in the first quartile (i.e., those with the highest k-index scores), followed by those in the second, followed by those in the third, and then those in the fourth (Figure 4). This pattern was consistent for the amount of discourse coded overall, as well as for each knowledge-building construct. In each case, there was a precipitous drop between the amount of discourse coded for the first and remaining quartiles, with only a few comments posted by members in the second through fourth quartiles. This pattern was also reflected in the number of questions posed, the knowledge connections made, and the amount of conceptual advancement. Users in the first quartile made the greatest number of posts containing categories of both higher-level discourse (e.g., full explanation, strong advancement, perpetuation) as well as those that did not contribute to meaningful knowledge-building (e.g., no knowledge, no advancement, beyond scope).

To learn more about how individual users functioned to drive the investigation, a final set of analyses were conducted to focus on individuals in the upper quartile. This revealed that half of the posts across the entire investigation were made by Scientist A and Teacher B, who contributed 25% and 24% of the posts to the investigation, respectively. Given the prevalence and quality of posts by these two members, it is not surprising that they were among the highest k-index rankings. As such, the differential pattern of results between the first and remaining quartiles in Figure 4 is due, in part, to these two individuals. Though the difference between the first quartile and others is dampened if these users are removed, the overall pattern remains.

Reviewing the individual k-index scores of others in the first quartile revealed a second possible profile for those who drive an investigation. Scientist B had a k-index score equal to that of Scientist A and Teacher B, though the number of comments submitted by Scientist B was far lower. Looking at the remaining members of the first quartile revealed that Student B also had a lower number of posts, relative to others in the quartile, but a high k-index score. To try to understand more about how these members accomplished such high scores, we compared the overall profile of comments used by these low frequency members and other members from the first quartile. Two differences were noted. The two low frequency members posted a balanced number of initiations and responses during the investigation, while all other first quartile members posted many more responses than initiations. Comments from low frequency members were also the most successful at generating a response from other members of the investigation. There were no notable differences in how low frequency members used specific types of discourse.