A statistical analysis of wh-scope responses to embedded wh-phrases in Gyeongsang Korean

The interrogative sentence with a wh-phrase in the embedded clause used in the experiment was the following.

Although discussions of how to deal with the nature of syntactic constraints in syntax are beyond the scope of this study, it is important to note that a sentence with a wh-phrase in the embedded sentence must be read with appropriate intonation to be legitimately perceived as having the matrix wh-scope. This type of intonation is so called “wh-intonation” (Deguchi & Kitagawa, 2002; Hwang, 2006, 2007, 2011, 2015; Ishihara, 2002, 2004; Jung, 2010; Kubo, 2005; Miyagawa, 2004). However, even with the interrogative ending marker, “-nkiko,” listeners can perceive the sentence as a yes/no-question as shown in Yun (2022) and Park et al. (2020).

The judgement of whether it is a yes/no or a wh-question mostly depends on the prosodic prominence of the phrase right after a wh-phrase. Yun (2019) argued that the dephrasing of the prosodic boundary after the wh-phrase ensures the matrix wh-scope. In Gyeongsang Korean, dephrasing manifests acoustically as either a compression of the pitch contour after the wh-phrase or a continuous rise in the pitch from the wh-phrase and the maintenance of a flat pitch contour or a gentle rise until it crosses the boundary between a prosodic phrase with an embedded verb and a complementizer and the next prosodic phrase, composed of the matrix verb phrase and the interrogative ending marker. Hwang (2015) describes this type of wh-intonation was as “a high plateau.”

Whether the acoustic manifestation of dephrasing is a compression or a high plateau as shown in Figure 1, a common characteristic is an inconspicuous pitch prominence across the whole span of the prosodic phrase with the embedded verb phrase and the complementizer, “-ʥ˳i” (Yun et al., 2020). There is a difference in pitch contour between the compression and the high plateau after the prosodic phrase. In the compression, the pitch stays low or further descends towards the end of the sentence, meaning the common characteristic still holds. However, in the high plateau, it decreases drastically at the end of the last prosodic, specifically at the last syllable of the interrogative ending marker. This makes the pitch contour look conspicuous in that prosodic phrase.

Figure 1. Two styles of wh-intonation. Adapted from Yun (2022: 3) with CC-BY-NC.

Download Original Figure

The objective of the dephrasing of prosodic boundaries is to make a sequence of prosodic phrases bear only one pitch prominence. In other words, the prosodic boundaries indicated as “C,” and “D” in Figure 1 do not function as active prosodic boundaries. If they are active, the prosodic phrases, the accentual phrases in this case, between the two boundaries would have a pitch peak because Gyeongsang Korean is a lexical pitch-accent language (Jun et al., 2006).

As illustrated in the left panel in Figure 1, the evident pitch prominence is on the wh-phrase and the pitch then becomes compressed all the way through the end of the sentence. On the other hand, in the right panel, the pitch starts to rise from the wh-phrase, maintains its level, and then drops at the last syllable, causing the pitch prominence found in the last prosodic phrase. Among the three prosodic phrases from the wh-phrase included to the final one, only the second does not have a pitch peak in both types of wh-intonation.

Although it is obvious that the interrogative ending marker, “-nkiko” clearly triggers speakers to produce either a compression or a high plateau type of wh-intonation, the actual utterances of the sentence in a previous experiment with 40 speakers were not as expected (Yun et al., 2020: 48-49). Even though prominence levels of the second prosodic phrase were statistically lower in the productions of the wh-question with the matrix wh-scope ending marker than of a yes/no-question with the embedded wh-scope ending marker, there were different levels of prominence in the phrase among the utterances of the wh-question, which may have induced the embedded wh-scope interpretation by listeners.

It is a logical assumption that an interrogative sentence with a wh-phrase in the embedded clause will be perceived as a wh-question, if the sentence is produced with either type of wh-intonation. However, according to the result of a comprehension test, the percentage of responses selecting a wh-question and thus the matrix wh-scope was not 100%, judging from the boxplots in Hwang (2015: 54). The number of responses selecting a yes/no-question was also considerable in Lee & Yun (2018), with 13% of the responses choosing a yes/no-question. Park et al. (2020) reported that 17.3% of responses chose a yes/no-question. Just over 10% of the responses were marginalized as errors due to stimuli made up of a typical wh-intonation.

As Yun et al. (2020) noted, in productions of wh-intonation, pitch contours cannot be categorized as simply one of the two wh-intonation patterns. In this study, the pitch contours varied over the three prosodic phrases, a wh-phrase with a case marker, an embedded verb with a complementizer, and the matrix verb with an interrogative ending marker. There were cases where pitch contours were similar to the intonation pattern imposed on a yes/no-question. This suggests that there must be a gap between the definition of wh-intonation and the actual acoustic realization of wh-intonation. Dephrasing of prosodic boundaries after a wh-phrase implies wh-scope interpretation relies on the listener’s judgement on whether prosodic boundaries exist. A question arises as to how much increases or decreases in the pitch imposed on the corresponding prosodic phrases can be regarded as dephrasing. Simply labelling it as dephrasing does not provide acoustically concrete criteria for what the dephrasing actually means. It is thus logical to examine the validation of wh-intonation in various productions of wh-intonation.

Yun (2022) reported that interpretations of wh-scope were inversely correlated to the prominence level of the second prosodic phrase which is composed of an embedded verb with a complementizer. Through a multiple regression analysis, the number of responses indicating a wh-question increased as the prominence level of the second prosodic phrase decreased. The prominence level was calculated by subtracting the lowest fundamental frequency (f0) of the embedded complementizer from the highest f0 in the whole span of the second prosodic phrase. Yun (2022) also discovered that the matrix wh-scope was a categorical perception, and the perceptual boundaries varied greatly among subjects.

However, the previous study could not handle responses in a more statistically appropriate way, not only because the response was the binary either “wh-question” or “yes/no-question,” but also because there were three trials that a multiple regression analysis could not account for. Another study based on the same experimental data examined response time depending on the categories of stimuli and types of responses (Yun, 2023). The stimuli had either an interrogative ending marker for a yes/no-question or one for a wh-question, and the types of responses were either a yes/no-question or a wh-question. The study focused on the response time differences in matched and mismatched groups between the categories of stimuli and the types of responses, but it did not relate response time directly to the binary responses.

One of the major arguments in Yun (2023) is that the perception of the matrix wh-scope depends on two weighted factors: morpho-syntactic constraints and prosodic structural integrity. This framework resolved variations such as a wide range of perceptual boundaries among subjects and a gradual or a steep change of subjects’ preferences regarding either of the response types. The argument relied on a plot representing a moving average of the matrix wh-scope interpretation across trials for each subject. However, the methodology on which this argument was based is not statistically robust enough to support a grammatical model using these two weighted factors for the perception of wh-scope interpretation. A more elaborate statistical analysis was thus needed to verify the findings.

In contrast to other studies of the matrix wh-scope interpretation in which participants were instructed to produce intended intonations when stimuli were prepared, the participants recorded in Yun et al. (2020) and whose utterances were used as stimuli in Yun (2022, 2023) underwent no external intervention regarding how they should produce given sentences. The only guidance provided was a context to induce the matrix wh-scope interpretation, along with the final interrogative ending marker, “-nkiko.” However, the two perception studies lacked a more statistically elaborate analysis. This paper aims to offer more precise statistical insights from these studies by employing logistic mixed-effects regression models.

There were two groups of participants: one for production of stimuli and the other for perception. For the production, a total of 40 native speakers of Gyeongsang Korean (20 males and 20 females) read a script containing the target sentence along with context. They read the script twice for repeated measures analysis of production, but only the first utterance of the target sentence was extracted from the speech and used as the perception test stimuli. The stimuli were calibrated to ensure consistent intensity across all of them. Each stimulus was equalized to a root mean square level of 70 decibels (dB).¹ Praat was used for the extraction and calibration of the stimuli (Boersma & Weenink, 2022).

For the perception test, a total of 24 native speakers (3 males and 21 females) of the same dialect participated. All the participants were university students at the time of the test. They listened to the stimuli using over-ear headphones (K271MKⅡ; AKG) connected to a laptop computer running OpenSesame, a freeware program for creating experiments in psychology and other fields (Mathôt et al., 2012).

In the actual test, stimuli consisted of sentences featuring the interrogative ending marker “-na.” As previously mentioned, this marker makes the sentence a yes/no-question, interpreting the wh-scope of the wh-phrase in the embedded clause as having embedded wh-scope. A total of 80 target stimuli (40 for the matrix wh-scope and 40 for the embedded wh-scope) were included, along with 119 fillers, comprising one session of the test. Each session was repeated three times with a 10-minute break between sessions.² To control for order effects and ensure unbiased responses, all the stimuli across all the sessions were randomized using OpenSesame’s built-in randomization algorithm.

After listening to each stimulus, participants indicated their response by pressing either the left Shift-key for a yes/no-question or the right Shift-key for a wh-question. Response time and key responses was recorded for each participant.

As reported in the previous multiple regression analysis conducted by Yun (2022), prominence levels of prosodic phrases, specifically the second accentual phrase, comprising an embedded verb and a complementizer, were related to dephrasing of prosodic boundaries. In subsection 2.2, the calculation of prominence levels was briefly described. It is worth noting that the f0 values used in the calculation were not raw; instead, all f0 values were first normalized followed by the computation of the prominence level for each accentual phrase.

Each response time was examined to exclude responses with excessive hesitation, defined as lasting over 10 seconds. A total of 12 responses met this criterion and were subsequently removed from the analysis. The remaining response times were transformed using the natural logarithm and normalized in preparation for further logistic regression analysis. In total, the dataset comprised 2,868 samples, calculated as 24 subjects multiplied by 40 stimuli per trial, across 3 trials minus the 12 excluded responses.

Logistic regression is a statistical technique used to model the relationship between a binary outcome variable and one or more predictor variables. In this study, logistic mixed-effects regression was used to analyze the effect of prominence levels of the second prosodic phrase on the probability of “yes” responses. This approach is particularly suited for the repeated measures design of this study because it enables the modeling of both fixed effects (e.g., the main effect of the predictor) and random effects (e.g., individual differences in the responses of the participants). By accounting for random effects, the model provided more accurate estimates of the fixed effects and reduced the risk of biased results due to participant variability.

For further analysis, responses of “wh-question” and “yes/no-question” were encoded as 1 and 0, respectively, to facilitate modeling within the logistic regression framework. In addition, random effects were included to account for variability among participants and stimuli. R was used for statistical analysis (R Core Team, 2024).

There has been no previous report on the distribution of responses for wh-questions or yes/no-questions in the literature. Table 1 presents the counts of each response for each trial. Subjects are in descending order based on their total number of responses of wh-questions across trials.

The total number of responses for wh-questions increased with each repetition of the test trial. Particularly, there was a significant increase in the second trial, followed by a further increase in the third trial. However, the variance in responses is not as noticeable as observed in the second trial. It is noteworthy that subjects who frequently responded with wh-questions showed minimal instances of switching to yes/no-questions. In contrast, there was a notable increase in the number of responses changing from yes/no-questions to wh-questions during the second trial.

Since all stimuli consisted of productions of wh-questions that ended with the matrix wh-scope, marked by the interrogative ending marker “-nkiko,” it is reasonable to assume that these stimuli were predominantly perceived as wh-questions. Despite variations in pitch patterns throughout the productions, there was a tendency towards perceiving more stimuli as wh-questions. While some subjects consistently exhibited similar numbers of responses to both wh-questions and yes/no-questions across the trials, the response patterns of subjects from S19 and below differed notably. In the first trial, they predominantly responded with yes/no-questions, but in the second trial, the number of responses indicating wh-questions roughly doubled for most subjects, except for S24, who appeared to maintain a preference for yes/no-questions over wh-questions.

Based on these response patterns, subjects can be categorized into three types. The first group demonstrated a resilient acceptance of wh-intonation. They consistently selected wh-questions as responses, even when presented with stimuli with pitch patterns different from typical wh-intonation. The second group exhibited a more balanced judgment of wh-intonation. They tended to change their responses depending on pitch patterns, particularly the prominence level of the second prosodic phrase, which comprises an embedded verb and a complementizer. The final group tended to respond with yes/no-questions without paying attention to the interrogative ending marker.

Table 1 clearly implies two things. First, variances among subjects must be taken into account in the statistical model as a random factor to produce a better understanding of subjects’ perceptual behavior regarding response patterns. Second, test trials need to be included in the model as an independent variable, as they show different enumerations of wh-question responses. More importantly, it is necessary to confirm whether varying patterns of responses between trials are related to each subject’s preference for either type of response. In other words, random slopes for trial by subjects need to be analyzed.

In this section, possible independent variables for logistic regression models are discussed and different models are compared to find a model to have a better fit to the responses. As reported in Yun (2022), the prominence level of the second prosodic phrase is the most effective predictor of responses to wh-questions. Other prominence levels of the remaining two prosodic phrases are viable predictors and belong in models with different combinations of other independent variables. However, before comparing models, the possible independent variables must be checked to ensure their potentials as predictors in the theoretical framework.

Yun (2019) claimed that wh-intonation is dephrasing of prosodic boundaries after the wh-phrase. As illustrated in Figure 1, the two patterns of pitch contour satisfy this description of wh-intonation. The compression pattern is prominent in the first prosodic phrase, while in the high plateau, the final prosodic phrase is prominent. Since this study did not differentiate these two patterns, one would not expect the prominence levels of both first and last prosodic phrases to be consistent. The only common characteristic, and thus a reliable parameter for responses of wh-questions in any pattern, is the prominence level of the second prosodic phrase.

In the previous multiple regression analysis conducted in Yun (2022: 6), the pitch peak of the second prosodic phrase showed a minor impact on the number of the matrix wh-scope responses (β=13.4, p=.030). It is worth including this predictor into a model and comparing the model to a model without it. As seen in Table 1, the number of the matrix wh-scope responses in each trial was quite different, particularly in the second trial where there were more matrix wh-scope responses. Therefore, the trial as a factor was included in all the models created. For random intercepts, both subjects and stimuli were included.

As expected, none of the models, including either or both prominence levels of the first and the last prosodic phrases alongside the prominence level of the second prosodic phrase, performed better than a simple model that included the prominence level of the second prosodic phrases. This simple model was compared with another model that included the pitch peak of the same prosodic phrase.

Table 2 presents the Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC) of the two models: Model 1, the simple model, and Model 2, which included the pitch peak. In addition, the table includes the same models with random intercepts for subjects and stimuli, as well as models replacing the random intercept for subjects with random slopes for trial by subjects.

A model with lower values of both AIC and BIC indicates a better balance between accuracy and simplicity. In this respect, it is difficult to determine which of the two models is a better fit to the responses. However, models with random slopes for trial by subjects performed better compared to models with random intercepts for subjects. While the difference in AIC between the two models is relatively small, Model 1 holds a slight advantage in BIC. Moreover, Model 1 has one fewer predictor than Model 2. When models achieve comparable fit, selecting the model with fewer parameters is generally recommended.

Models of the interactions between the various combinations of the main three predictors were examined: pitch peaks, prominence levels, and trials (e.g., pitch×prominence, pitch×trials, prominence× trials, pitch×prominence×trials), but no improvement was found in any of the models. In the next subsection, the fixed effects of Model 1 will be reported. Also, response time will be checked with regard to the binary responses.

This section deals with the fixed effects estimates of predictors in Model 1, This model included random slopes for trial by subjects as it demonstrated a better fit than the model with intercepts for subjects. The fixed effect of the main predictor, the prominence level of the second prosodic phrase, was elucidated through a plot illustrating predicted probabilities against this key predictor, as depicted in Figure 2. Detailed estimates are provided in Table 3, which also encompasses the estimate of the trial factor.

Figure 2. Predicted probabilities of responses against prominence level across trials: (a) overall response pattern, (b) responses for trial 1, (c) responses for trial 2, (d) responses for trial 3.

Download Original Figure

In Table 3, the intercept term in the model represents the estimated log odds of selecting a response with a wh-question when all other predictor variables are zero. Although the estimated intercept is not statistically significant at the level of 0.05, the actual coefficient was found to be 1.40 (95% CI: lower –0.11, upper 2.91) indicating a higher likelihood of selecting a response with a wh-question. Across all trials, a total of 1,630 responses out of 2,860 were wh-questions, suggesting a preference for this response type.

Interestingly, while subjects selected almost an equal number of responses for yes/no- or wh-questions in the first trial, as shown in Table 1, Figure 2 (b) illustrates that the predicted probability curve starts lower than the starting points in (c) and (d), indicating a lower likelihood of initially selecting responses with wh-questions. However, the movement of the curve from (b) to (c) and then to (d), is upward, suggesting an increasing tendency to select responses with wh-questions when the test was repeated. This observation provides insights into the subjects’ changing response patterns over the course of the experiment.

The coefficient of the main predictor, the prominence level of the second prosodic phrase was estimated to be –1.22 (95% CI: lower –1.88, upper –0.56) indicating a negative association with the likelihood of selecting wh-questions while controlling for other factors. This suggests that as the prominence level increases, there is a corresponding increase in the likelihood of selecting responses with yes/no-questions.

Keeping the prominence level of the second prosodic phrase low makes the left prosodic boundary, which is right after the prosodic phrase containing the embedded wh-phrase, disappear. In other words, this dephrases two independent prosodic phrases, resulting in a single prosodic phrase. This is in line with Jun & Oh (1996), who observed that a wh-phrase and the next verb phrase stay in the same accentual phrase.

Test trials also influenced the response patterns as Table 3 shows. There was a larger difference in the number of responses selecting wh-questions between the first and the second trial than between the second and third trial. There was a discernible shift from initial responses of yes/no-questions to responses of wh-questions in the second trial. While this trend continued in the third trial, it was not as pronounced as in the second trial. The percentage of responses selecting wh-questions in the first trial stood at 50.3%, which increased significantly to 58.8% in the second trial and further to 61.2% in the third trial. This corresponds to differences of 8.5 and 2.4 percentage points between the trials.

Upon examining the response numbers of each subject in Table 1, it became apparent that the shift from yes/no-questions to wh-questions was not uniform across all the subjects. While some subjects exhibited noticeable changes in their responses, others maintained consistent pattern throughout the trials. This indicated that further analysis of the random factors pertaining to subjects and trials was necessary to elucidate the subtle interplay between response patterns and individual subjects, shedding light on the extent of variability in response shift across trials.

Finally, an independent logistic model was employed to investigate whether response time could effectively predict response types. Response time was included as an independent predictor in Model 1, which incorporated random slopes for trial by subjects. However, the fixed effect of response time had no significant impact on the selection of wh-questions as responses (β=0.09, SE=0.07, p=0.20).

This section explores response patterns across trials and their variation among individual subjects by analyzing the random effects within the previously described logistic model. Additionally, random slopes for trial by stimuli are also investigated with an additional model. Initially, all the models created for comparison included random intercepts for both subjects and stimuli. However, given the observed inconsistency in response pattern changes across trials among subjects, the random intercept for subject was modified to illustrate how variation in responses patterns across trial changed across different subjects. Consequently, Model 1 with random slopes for trial by subjects demonstrated the best fit among the models, as shown in Table 3.

Figure 3 illustrates the two distinct preferences: one towards selecting wh-questions and the other towards yes/no-questions, with a third group exhibiting an intermediate pattern. Based on insights from a plot of moving averages of wh-question responses for each stimulus, Yun (2023) proposed a hypothesis regarding the response behaviors of subjects. He suggested that two primary weighted factors—morpho-syntactic constraints and prosodic structural integrity—underlie the decision-making processes of subjects. Participants who consistently chose wh-questions may therefore have based their decisions on morpho-syntactically constrained information. While acknowledging the importance of wh-intonation for wh-questions, these individuals may have primarily focused on the interrogative ending marker, which evidently signaled to listeners that the sentence was a wh-question. Consequently, they may have paid less attention to variations in wh-intonation, leading to a resilient acceptance of wh-intonation.

Figure 3. Estimated random effects for trial by subjects. Subjects were sorted based on total wh-question responses across trials.

Download Original Figure

On the other hand, participants who exhibited a preference for yes/no-questions may have based their responses on the proper production of the sentence in terms of wh-intonation, indicating that prosodic structural integrity superseded the morpho-syntactic constraints. This pattern was predominantly observed in the first trial, but participants’ preferences shifted towards responses with wh-questions in the subsequent trials, suggesting an increased awareness of the morpho-syntactic information transferred by the ending markers, while still considering prosodic structural integrity as significant. However, participant S24, who appears to have overlooked the conveyed meaning of the given interrogative ending markers maintained his/her preference throughout the trials. Yun (2023), in his analysis, did not account for changes in response patterns across trials, thus failing to recognize that individual preferences may weaken, albeit not to the same extent as observed in the intermediate group.

Morpho-syntactic constraints and prosodic structural integrity can be balanced through weighting. When one factor is prioritized, it tends to diminish the influence of the other. It is thus likely that the intermediate group maintained a balance between these factors, exhibiting no strong preference for either wh-questions or yes/no-questions. In addition, the random effects for trial were relatively small in this group compared to those observed in the group with clear preferences for yes/no-questions in the initial trial. This suggests that the responses of participants in the intermediate group remained relatively stable across trials, unlike those in the group with a preference for yes/no-questions, whose responses varied more noticeably over subsequent trials.

Although the prominence level of the second prosodic phrase had a significant effect on overall response patterns, an examination of the random effects of stimuli was conducted. Response patterns across trials were also explored across stimuli using an additional logistic model. Random slopes for trial by stimuli were initially incorporated into Model 1 along with random slopes for trial by subjects, but the model did not converge. Consequently, the random slopes for trial by subjects were replaced with random intercepts for subjects. As a result, the random slopes for trial by stimuli were included in the additional model to investigate the random effects of trials based on the random intercepts of stimuli.

In Figure 4, the stimuli on the x-axis are sorted based on the prominence level of the second prosodic phrase, with stimuli on the left side more likely to elicit wh-question responses, indicated by relatively high positive random intercepts. Conversely, stimuli on the right side show negative values, suggesting a higher likelihood of eliciting yes/no-question responses. However, there is an alternating pattern of random intercepts for stimuli between these extreme cases, indicating the prominence level of the second prosodic phrase may not strongly determine response patterns. The random effects for trial based on stimuli reveal that stimuli more likely to elicit one type of response tended to maintain that tendency across trials. Notably, stimuli with low random slopes and intercepts did not exhibit a clear tendency towards either response type. Moreover, these random response patterns remained consistent across trials.

Figure 4. Estimated random effects for trial by stimuli. Stimuli were sorted based on the prominence level of the second prosodic phrase.

Download Original Figure