1. INTRODUCTION
Stuttering is a disorder of speech fluency that impacts the communication of an individual [1]. There is evidence to suggest that the relationship between fluency disorders and anxiety may commence in childhood [2]. In addition to the observable features of the disorder (e.g., repeated movements, fixed postures, superfluous behaviors, etc.), it is well-reported that stuttering can have a potential negative impact on the quality of life of the stutterers [3], [4], [5], particularly social functioning and mental health [6], [7]. Stuttering can also impact an individual’s job performance, academic achievement, and employment opportunities [8], [9], [10].
The Unhelpful Thoughts and Beliefs about Stuttering scale (UTBAS) is a tool that can be used to screen for indicators of speech-related social anxiety in adults who stutter [11], [12]. The original UTBAS scale (developed in English) includes 66 items that comprehensively measure an individual’s thoughts and beliefs about their own stuttering. Completion of the original scale is time intensive, therefore a brief version was developed to use in clinical settings. The brief version of the scale, UTBAS-6, includes 6 items and has been proven to be a valid and reliable tool [1]. To date, the UTBAS-6 scale has been translated into other languages such as Japanese (UTBAS-J) [13] and Turkish (UTBAS-TR) [14]. Translation of the tool into Turkish followed the recommended translation process [14], however, the forward translation process reported by Chu et al. [13] was different from the recommendation that at least two people should translate the scale independently [15]. Further, the pilot test was carried out on 5 people who did not stutter [13]. Both studies did not report neither content validity nor face validity.
To date, speech related social anxiety for Vietnamese adults who stutter has not been investigated. However, in order to conduct such research, a culturally and linguistically relevant assessment tool is required but is not yet available. The purpose of this study is to evaluate the validity and reliability of the Vietnamese version of the UTBAS-6 scale in adults who stutter.
2. MATERIALS AND METHOD
The study was carried out nationwide online from 01/2/2021 to 30/4/2021 for Vietnamese adults who stutter.
It is recommended that a study evaluating the validity and reliability of a scale requires a sample size of 5-10 times the number of items on the scale [16], [17]. Therefore, 34 adults who stutter were recruited for this study. Participants were recruited via the online self-help group for people who stutter in Vietnam, called “Echo Vietnam”. This group was established to connect individuals who stutter into a community to provide support, share exercises to improve their communication and connect with other communities of people who stutter (Facebook address of Echo Vietnam: https://www.facebook.com/groups/133766986662954).
Study information was sent directly to the Echo Vietnam group members and posted on the Facebook page of the Rehabilitation department, Faculty of Nursing-Medical technology, University of Medicine and Pharmacy at Ho Chi Minh City. Potential participants who were interested in the study were required to contact a researcher via e-mail/phone call/message, and those who met the following criteria were included in the study: Vietnamese adults aged 18 years or older, diagnosed as developmental stuttering by a Vietnamese trained speech and language therapists experienced in management of stuttering in adults. Participants were excluded if diagnosed as acquired stuttering or having lived outside Vietnam in a period of 12 months or more.
The UTBAS-6 [1] includes six items (Table 1), scored under three subscales: Frequency of unhelpful thoughts (Frequency subscale), degree of belief in these thoughts (Belief subscale), and degree of anxiety from these thoughts (Anxiety subscale). Each subscale contains same six questions, and respondents would rate each question on a 5-point Likert scale (1=never or not at all, 2=rarely or a little, 3=sometimes or somewhat, 4=often or a lot, 5=always or totally). The total score (the sum of 6 item scores) is converted to deciles, Iverach et al. recommends that where a score falls within or above the fifth decile, appropriate referrals to relevant psychological professionals take place [1]. However, the authors also caution that scores below the fifth decile do not necessarily indicate that a psychological issue may not be present and that clinicians additionally use other information to make clinical decisions regarding psychological referral as required [18].
The translation and cross-cultural adaptation process used in this study were based on standard guidelines from Beaton et al. [15]. The procedure consisted of five steps: forward translation, backward translation, synthesis, expert panel review, and pretesting (Figure 1).
The original version of the UTBAS-6 scale was independently forward translated into Vietnamese by two independent translators. The 2 translated versions were designated as V1 and V2. These were synthesized into V3 by the researchers and reviewed by Vietnamese experts to become the pre-final V4 version. The V4 was backward translated into English by two Vietnamese translators who were independent of the study and did not know the original version. The two backward translated E1 and E2 versions were synthesized into the E3 version. This version was sent to five experts who worked in the field of speech and language therapy (Vietnamese and international speech therapists) and were experienced in management of stuttering [17]. The experts were asked to rate the “Clarity” (semantic and idiomatic equivalence) and “Relevance” (experiential and conceptual equivalence) of the Vietnamese-translated UTBAS-6 scale using a 4-point Likert scale (1=cannot be used, not relevant/clear, 2=cannot be used, item needs some revisions, 3=quite relevant/clear, may be used with some minor revisions, 4=highly relevant/clear). The experts were also asked for any modification to ensure equivalence between the translated version and the original scale [19], [20] (Table 1). A final translated version was confirmed once consensus achieved.
The Vietnamese UTBAS-6 scale was sent to all consenting participants via e-mail. Participants completed a scale online, and were also asked to suggest modifications to improve the translated scale for face validity [21]. To ascertain test-retest reliability, 3 weeks after the first survey participants were requested to complete the same online survey.
To evaluate the face validity of the Vietnamese version of the UTBAS-6 scale, participants were requested to answer questions “The items were relevant to your stuttering” and “The scale was easy for you to understand”. The answers were based on a Likert 5-point scale (1=strongly disagree, 2=disagree, 3=slightly agree, 4=agree, 5=strongly agree), and if level l or 2 was chosen, the participants would be asked for the reasons and any suggestions to modify the scale.
Epidata Entry Client software was used for data entry and statistical analysis were conducted using Stata 14.0.
This analysis was based on the expert’s rating when reviewing the E3 version. The content validity was determined via the content validity index (CVI). There were two kinds of CVI: Item-CVI (I-CVI) and Scale-level CVI (S-CVI) [19]. I-CVI was computed as the number of experts giving ratings at level 3 (Quite relevant/clear) or level 4 (Highly relevant/clear) to an item divided by the total number of experts. S-CVI was calculated using the number of items in a tool that have achieved a rating of 3 or 4 values. There were two methods for calculating S-CVI, one was the Universal Agreement among experts (S-CVI/UA), and the other was the Average CVI (S-CVI/Ave) [20]. S-CVI/UA was calculated by adding all items with I-CVI rated at level 3 or 4 divided by the total number of items, while S-CVI/Ave which was a less conservative method was calculated by taking the sum of the I-CVIs divided by the total number of items. I-CVI score of >0.79 is considered excellent, between 0.7 and 0.79 is average, and <0.7 is poor [20]. The content validity was excellent when S-CVI/UA ≥0.8 and S-CVI/Ave ≥0.9 [20]. A kappa statistic was conducted to control risk of chance agreement [20]. Criteria for evaluating kappa is that the values >0.74 = excellent, 0.6-0.74 = good, 0.4-0.59 = fair [22].
Face validity was determined by the percentage of each level rated by the participants to the questions whether the Vietnamese version was relevant to stuttering, or easy to understand. Ten out of 34 participants answered these questions.
Construct validity was determined through convergent and divergent validity, also evaluated through the relation between the three subscales. A subscale had convergent validity when its items have correlation coefficients >0.4 [23], [24]. An item had discriminatory value (i.e. between the Frequency scale and the Belief scale) when the correlation coefficient for that item and the rest of items in a scale (i.e. the Frequency scale) was lower than the correlation coefficients of that item and the remaining items in the Belief scale [25].
Cronbach’s alpha was used to examine the internal consistency of the scale. The scale has internal consistency when the value of alpha ≥0.7 [26]. Test-retest reliability was evaluated by intraclass coefficient correlation (ICC). ICC was interpreted by the rule of thumb: >0.9 = Very good, 0.75-0.9 = Good, 0.5-0.75 = Moderate, <0.5 = Poor [27].
3. RESULTS
Most of participants were young adult males with a high educational level and living in urban areas. The mean age (±SD) was 27.2 (±5.4) (Table 2).
In terms of “Relevance”, there were 5 items that had the values of I-CVI and kappa as 1.00. This suggested that there was very good equivalence between the translation and the original (Table 3), except item 5, two experts rated it at level 2 and suggested some revisions to have the same meaning with the original. In terms of “Clarity”, I-CVIs of the Vietnamese version of the UTBAS-6 scale were in the range of 0.80-1.00 (I-CVI values >0.79), the kappa values of each item were in the range 0.76-1.00 (>0.74), which indicated very good agreement of expert panel [20]. S-CVIs of “Relevance” and “Clarity” of the scale were in the range of 0.83-1.00, indicating that the scale had a high consensus with the expert panel.
All ten participants stated that the Vietnamese version of the UTBAS-6 scale was related to stuttering and easy to understand (Table 4) but suggested changes to the term “mức độ thường xuyên” (frequency) clearer in the Vietnamese version.
| Relevance | Clarity | |||
|---|---|---|---|---|
| Number of raters | % | Number of raters | % | |
| Level 1 | 0 | 0 | 0 | 0 |
| Level 2 | 0 | 0 | 0 | 0 |
| Level 3 | 1 | 10 | 4 | 40 |
| Level 4 | 5 | 50 | 4 | 40 |
| Level 5 | 4 | 40 | 2 | 20 |
Convergent validity: Table 5 presents the correlation coefficients in each scale, and between each subscale with the total scale. The cells of the table were numbered from 1 to 9 in parentheses. Cell 1 showed that the Frequency scale had 4/6 items (66.7%) with correlation coefficients greater than 0.4, those were items 2, 4, 5, and 6. All 6 items (100%) of the Belief or Anxiety scales had correlation coefficients greater than 0.4 as seen in cells 5 and 9 respectively. When analyzing the three subscales within the total scale, there were 16/18 items (88.9%) that had correlation coefficients greater than 0.4 (cells 1, 5, and 9).
Divergent validity: The correlation coefficients of items 1, 3, and 4 of the Frequency scale with the Belief scale (cell 2) and the Anxiety scales (cell 3) were greater than those with the Frequency scale (cell 1) (Table 5), indicating a poor divergence. Similarly, items 4 and 6 of the Belief scale showed poor divergence with the Anxiety scale, as seen in cell 5 and 6. Meanwhile, all items of the Belief scale indicated good divergence with the Frequency scale, as seen in cell 4 and 5. The Anxiety scale indicated good divergence with the Belief scale since all the correlation coefficients between the Anxiety scale and the Belief scale (cell 8) were less than those of the Anxiety scale with itself (cell 9). However, items 5 and 6 of the Anxiety scale indicated poor divergence with the Frequency scale. In summary, only 11 out of 18 items (61.1%) of the total scale had good divergent validity. The Frequency, Belief, Anxiety, and total scales were strongly correlated with all correlation coefficients greater than 0.6 (Table 6).
| Frequency | Belief | Anxiety | Total scale | |
|---|---|---|---|---|
| Frequency | 1 | |||
| Belief | 0.61 | 1 | ||
| Anxiety | 0.77 | 0.71 | 1 | |
| Total scale | 0.88 | 0.87 | 0.93 | 1 |
Alpha coefficients of the Frequency, Belief, Anxiety scales, and the total scale were greater than 0.7 which indicated that the scale had internal consistent reliability (Table 7).
ICCs of the Frequency and Anxiety scales were greater than 0.7, which showed that 2 of these subscales had test-retest reliability. However, the Belief scale had poor test-retest reliability with an ICC of 0.52, and the confidence interval of 95% for ICC was (-0.07 to 0.78) (Table 8). In summary, the test-retest reliability of the scales was moderate [27].
| Scale | ICC (Confidence interval 95% for ICC) |
|---|---|
| Frequency | 0.74 (0.42 - 0.89) |
| Belief | 0.52 [(-0.07) - 0.78] |
| Anxiety | 0.71 (0.36 - 0.87) |
| Total scale | 0.72 (0.37 - 0.88) |
4. DISCUSSION
The validation results of the Vietnamese version of the UTBAS-6 scale indicated that the scale has high content and face validity. The convergent validity was at 88.9% of the items, and the divergent validity was at 61.1% of the items. Alpha coefficients of three subscales and total scale were greater than 0.7, and test-retest reliability was moderate.
The survey sample with 34 subjects was quite small, and not necessarily representative of the community of stutters in Vietnam. However, it was interesting to see some common characteristics of adults who stutter among the study group that were consistent with findings from previous studies. Previous studies revealed the number of men who stutter was higher than women [28]. Most of participants were single possibly because of their young age, however, stuttering was a less attractive feature which could be a barrier for establishing intimate relationships [29].
The content validity of the Vietnamese version of the UTBAS-6 scale was considered excellent based on the value of the corresponding index. The evaluation of the content validity of a scale needs a panel with at least five experts working in the field in which the scale is used to have the appropriate level of control over the chance of consensus [19], [22], and experts should be people speaking target language of study [20]. However, speech and language therapy is a new profession in Vietnam and few people work in the stuttering field. The expert panel of this study included two Vietnamese and three Australian speech-language pathologists ensured the content validity of the Vietnamese version of the UTBAS-6 scale.
Both Turkish and Japanese studies did not report any information regarding face validity of the UTBAS-6 [13], [14]. In this study, face validity was determined through feedback from participants on the translated scale. Although only ten participants responded to the face validity related questions, all agreed that all items of the scale related to stuttering and the scale was understandable and clear.
The UTBAS-J [13] and UTBAS-TR [14] determined construct validity by comparing with other scales. Because there were no other available Vietnamese scales to use in a comparison, this study assessed construct validity by using convergent and divergent validities. The Vietnamese version of the UTBAS-6 scale had good convergent validity, but the divergent validity was moderate due to the impact of small sample size and some factors.
Some participants had received treatment and their stuttering condition improved, and this could influence their thoughts and beliefs about stuttering. The severity of stuttering might be higher among non-participants, therefore, the choice of an answer for a question might deviate toward mild stuttering. In addition, when evaluating face validity, some participants suggested that it should be made clear the term “mức độ thường xuyên” (how frequently) in the Vietnamese version. An unclear term definition might result in non-valid data. Although generally assessed as having face validity, it is necessary to review each item.
This study was one of four studies in a research project about stuttering in Vietnam, and online self-rating survey answering 3 sets of quantitative questionnaires at the same time might influence the quality of raw data. Furthermore, using the same question to measure three different domains may confuse the respondents.
Alpha coefficients in this study were smaller than those of the UTBAS-TR scale [14] and the UTBAS-J scale [13]. However, these two studies validated the full UTBAS scale, so they had much more items and a larger sample size. Although fewer items, the Vietnamese version of the UTBAS-6 scale showed good internal consistency reliability. Paired t-test was used in the two above studies to test the difference in mean scores between time 1 and time 2, but ICC was used in this study because the sample size was small (26 respondents were enrolled in the analysis) and furthermore ICC was more appropriate than paired t-test for evaluating test-retest reliability. However, given a wide confidence interval of ICC estimate a study with a larger sample size is needed.
This study strictly followed the recommended translation process to ensure the equivalence between the Vietnamese version of the UTBAS-6 scale and the English original. Beside content validity and construct validity, the study also assessed face validity to confirm the validity of the Vietnamese version. People who stutter usually avoid communication situations, so online made it more accessible for them. However, this study had some limitations. As discussed in construct validity, the study sample size was small, not representative, biased toward mild stuttering subjects, and information bias might occur.
Conclusion
This is the first study among Vietnamese adults who stutter providing preliminary information of the validity and reliability of the Vietnamese version of the UTBAS-6 scale. The results of the study indicated that the scale is valid and reliable. However, further studies with a representative and larger sample size are needed to ensure the accuracy of the findings.
