Translation and Validation of the Health Literacy Score-14 Questionnaire for Vietnamese Patients with Diabetes

Abstract

Introduction: Health literacy (HL) is a crucial indicator for health promotion and diabetes care improvement, but the available measurements are mostly in English. This study aimed to translate and validate the 14-item Health Literacy Scale (HLS-14) questionnaire from English to Vietnamese for patients with diabetes in Vietnam.

Methods: We translated HLS-14 into Vietnamese in accordance with the World Health Organization guidelines and conducted a cross-sectional survey among 571 outpatients with type 2 diabetes using the HLS-14 Vietnamese version (HLS-14 VN). The reliability and validity of the tool were assessed using Cronbach’s alpha, composite reliability (CR), average variance extracted (AVE), and maximum shared variance (MSV), and confirmatory analysis was conducted.

Results: Cronbach’s alpha coefficients for the three subscales as in the original version were 0.931, 0.810, and 0.928 for functional HL, communicative HL, and critical HL, respectively. However, AVE for critical HL was 0.488, which improved to 0.516 after the removal of one item in the communicative HL. For all subscales in the revised 13-item version (HLS-13 VN), CR was above 0.8, AVE was above 0.5, and MSV was less than AVE. Confirmatory analysis of HLS-13 VN revealed an acceptable fit with comparative fit index of 0.983, goodness-of-fit index of 0.963, and root mean squared error of approximation of 0.058.

Conclusions: The reliability and validity of HLS-13 VN were confirmed. The tool is suitable for use in clinical settings in Vietnam to assess multidimensional HL in patients with type 2 diabetes.

Key words: health literacy, diabetes, questionnaire design, validity, Vietnam

Introduction

Having an adequate level of ability to understand health information motivates people to better understand their own health and empowers them in self-management ^{(1), (2)}. Thus, individuals are recommended to have an adequate level of heath literacy (HL), which is defined by the World Health Organization as “the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health” ⁽³⁾. It has long been known that inadequate HL exerts negative effects on health outcomes and healthcare service utility ⁽⁴⁾.

In the modern age, HL requires not only skills in reading, writing, and numeracy but also an ability to use the Internet ⁽¹⁾. Various tools to screen the HL levels of people and patients have been designed and used ⁽⁵⁾. For short- and long-term diabetes control, although routine screening is not recommended, clinicians should be aware of both the HL level of their patients and the importance of their own HL skills to convey health information in an accessible manner. HL promotion requires bilateral communication between service providers and patients so as to provide individualized services to users. In addition, familiarity with HL is recommended not only for clinicians at health and medical institutes but also for all public health professionals to provide a driving force for health promotion and support community health and policymakers to reinforce HL promotion at the organizational and policy levels ^{(6), (7)}.

Nutbeam developed an HL model including three domains: functional HL for examining reading and writing skills, communicative HL for extracting and communicating information, and critical HL for analyzing and making decisions ⁽⁸⁾. Using Nutbeam’s conceptual framework as a basis, Ishikawa et al. devised an instrument called the 14-item Health Literacy Scale (HLS-14) to be applied among patients with diabetes ⁽⁹⁾. HLS-14 was then revised and validated for use among the general public in Japan by Suka et al. ⁽¹⁰⁾.

In Vietnam, HL is considered a new approach to respond to a rapid increase in noncommunicable diseases, particularly diabetes mellitus ⁽¹¹⁾. As a chronic progressive disease, it requires patients to perform self-management in their daily activities. Their HL skills can be the foundation for both treatment adherence and self-efficacy in disease management ^{(2), (12), (13)}. Recently, the comprehensive short-form HL survey tool (HLS-SF12) was developed from the 47-item European Health Literacy Questionnaire (HLS-EU Q47) and validated in the Vietnamese general population ⁽¹⁴⁾. However, the HLS-SF12 has not been applied in clinical settings targeting patient groups, such as those with diabetes. Therefore, this study aimed to translate and validate the HLS-14, which has been validated for Japanese patients with diabetes, for use among Vietnamese patients with diabetes.

Materials and Methods

Translation process

The original version of the questionnaire was translated into Japanese, and its English version was published ⁽⁹⁾. We followed the translation processes proposed in the World Health Organization (WHO)’s 2016 guidelines ⁽¹⁵⁾. Two translators independently translated the questionnaire from English into Vietnamese, then two principal investigators compared and discussed criteria such as clarity, common language use, and conceptual equivalence to create a draft Vietnamese version. A total of 10 volunteer patients with diabetes evaluated the draft to identify words and questions that were difficult to understand or answer and made suggestions for improvements. Some minor changes were made to develop the preliminary Vietnamese version, which was backtranslated into English by two health professionals working independently. Subsequently, we reviewed and discussed the differences between the translations and the original content, and the finalized Vietnamese version (HLS-14 VN) was approved by all research team members before being subjected to a validity study.

Study setting and participants

This was a hospital-based cross-sectional study conducted at the outpatient department of People’s Hospital 115, Ho Chi Minh City, Vietnam, between June 2020 and February 2021. People’s Hospital 115 is one of the largest tertiary general hospitals in Ho Chi Minh City, with 1,600 beds. Participants were selected from patients who fulfilled the inclusion criteria: (1) having type 2 diabetes and (2) aged 18 years or above. Patients who had difficulty communicating in the Vietnamese language as a result of cognitive, hearing, and/or visual impairment were excluded.

Participants were interviewed by health professionals who had been informed of the study aims and methods. Demographic data, clinical parameters, and items related to diabetes were collected using a standardized self-administered questionnaire. The HLS-14 VN contained five items each for functional and communicative HL and four for critical HL. A five-point Likert scale was employed for the answer options for each item: strongly disagree, disagree, not sure, agree, and strongly agree. The scores for the items in each subscale were summed to obtain subscale totals.

Data analysis

The reliability of the HLS-14 VN was evaluated using Cronbach’s alpha coefficient ⁽¹⁶⁾, corrected item-total (IT) correlation, and Cronbach’s alpha if item was deleted (i.e., reevaluated Cronbach’s alpha after scale items are individually removed). The IT correlation indicates the strength of the association between each item and the summated score for all other items. Items with an IT correlation below 0.30 are considered not to be satisfactorily correlated with the entire scale, suggesting the need to remove them ⁽¹⁷⁾. Items for which Cronbach’s alpha increases if the item is deleted from the scale are also removed.

Exploratory factor analysis (EFA) was conducted to establish the factor structure. To determine whether the measures are suitable for factor analysis, the Kaiser-Meyer-Olkin (KMO) test was employed to measure sampling adequacy, and Bartlett’s test of sphericity was used to assess the factorability of the data. A KMO value close to 1 and a significant statistical result for Bartlett’s sphericity test indicate suitability for factor analysis. Principle component analysis with promax rotation was conducted to establish the factor structure of the instrument. Eigenvalues were determined for each factor, representing the total amount of variance explained by each factor, with the total amount of variability equal to the number of original variables in the analysis. The Kaiser rule was adopted, which states that only those factors with eigenvalues greater than 1 should be retained. Furthermore, a general rule was applied to retain the factors that contribute at least 70% of the total variance.

Construct validity was assessed using confirmatory factor analysis (CFA) based on the original three-factor model ⁽⁹⁾. The following indices were used: ratio of chi-squared minimum and degree of freedom (CMIN/DF), goodness-of-fit index (GFI), comparative fit index (CFI), Tucker-Lewis’s coefficient index (TLI), root mean squared error of approximation (RMSEA), and probability of close fit (PCLOSE). The following values indicated a good fit: CMIN/DF < 3; GFI, CFI, and TLI > 0.90; RMSEA < 0.08; and PCLOSE > 0.05 ⁽¹⁸⁾. The modification indices were calculated to improve model fit by allowing correlations between within-factor error terms.

Convergent validity assesses the extent to which multiple measures of a construct are associated. In this study, convergent validity of the factors was assessed using the measures of factor loading (FL) of items, composite reliability (CR), and average variance extracted (AVE) ⁽¹⁹⁾. CR value ≥ 0.70 and AVE value ≥ 0.50 were considered satisfactory whereas FL value > 0.5 was acceptable ⁽²⁰⁾. Discriminant validity refers to the degree to which a measure does not correlate with another measure whose underlying construct is theoretically unrelated to it. To evaluate discriminant validity, the square root of AVE of each factor was compared with the correlation coefficient between factors (ρ_ij) as well as AVE and maximum shared variance (MSV). Discriminant validity was established when AVE < MSV and the square root of AVE > all of (ρ_ij) in the same row and column ⁽¹⁹⁾.

All analyses were conducted using IBM SPSS version 16.0, whereas CFA analysis was conducted using IBM SPSS Amos version 20.0.

Ethical considerations

The study was reviewed and approved by the Scientific and Ethical Committee of People’s Hospital 115 (Number 1903/QD-BV115 dated 27/09/2019). Written informed consent was obtained from all participants.

Results

Following the translation process, there were no substantive changes in the content of the HLS-14 VN compared with the original version. However, in some instances, wording was revised to improve ease of understanding in Vietnamese. For example, the verb “read” from items fhl1, fhl4, and fhl5 could be confused with the meaning “read and understand”; therefore, this was elucidated in the HLS-14 VN. For the functional HL scale, volunteer patients suggested the inclusion of instruction cards or leaflets as examples of literature that needs to be understood. Some minor formatting adjustments were also made to improve the questionnaire layout.

Of the 575 eligible subjects, 571 completed the questionnaire (response rate: 99.3%). As presented in Table 1, the mean age of the respondents was 59 years, and 275 (48.2%) of them were men. The proportions of those who did not smoke and drink were as high as 79.1% and 72.2%, respectively, but the proportion of those doing regular physical activities was only 16.3%. In terms of disease status, the mean diabetes duration was 5.5 years and the mean HbA1c was 8.6%.

The mean scores of functional, communicative, and critical HL were 3.57, 3.99, and 4.33, and Cronbach’s alpha values were 0.931, 0.810, and 0.928, respectively (Table 2). Cronbach’s alpha if item deleted only increased in value when the first item of communicative HL (chl1) was omitted; this item also showed the lowest IT correlation. Notably, the KMO index was 0.85 and Bartlett’s test of sphericity was significant (n = 571, χ² = 5827, df = 91; p < 0.0001). After extraction and rotation, three components in the dataset had eigenvalues greater than 1. The first component explained 39.89% of the total variance, with an eigenvalue of 5.58. The second component explained 63.41% variance, with an eigenvalue of 3.29. Furthermore, the third component explained 9.48% variance, with an eigenvalue of 1.33. The overall result exhibited 72.88% of the total estimated variance with the three domains. Table 3 demonstrates that the FL of all items was within the range of 0.711-0.960.

The indicators in Table 4 suggest that Ishikawa’s original 14-item version ⁽⁹⁾ (Model 1) did not satisfy Hu et al.’s criteria for good fit ⁽¹⁸⁾. The AVE for critical HL in the 14-item version was 0.488, indicating that the convergent validity for the subscale was unsatisfactory (Table 5). After checking the standardized regression weight, the first item of communicative HL was discarded with the lowest value of 0.533. The modified 13-item version (Model 2) satisfied Hu et al.’s criteria (Table 4; GFI = 0.963, CFI = 0.983, and RMSEA = 0.058), and for all subscales, CR was above 0.8, AVE was above 0.5, and MSV was less than AVE (Table 5). In addition, recalculated Cronbach’s alpha coefficients were 0.93, 0.81, and 0.94 for functional, communicative, and critical HL, respectively. Figure 1 presents the path diagram of the final 13-item version (HLS-13 VN).

Figure 1. Path diagram of the confirmatory factor model for the 13-item version (HLS-13 VN)
Rectangles indicate the observed variables (items) and ellipses the latent constructs (factors).
Values on the single-head arrows leading from the factors to the items are standardized FLs.
Values on the curved double-headed arrows are correlations between factors/errors.
fhl: functioning health literacy, chl: communicative health literacy, crhl: critical health literacy, df: degree of freedom, GFI: goodness-of-fit index, CFI: comparative fit index, TLI: Tucker-Lewis index, RMSEA: root mean squared error of approximation, PCLOSE: probability of close fit.

» Full size image

Discussion

The newly developed HLS-13 VN is a unique tool for assessing the three dimensions of HL level among Vietnamese patients with diabetes and can be used to promote communication between diabetes care providers and patients in the Vietnamese context. The translation process was performed in accordance with the WHO translation protocol, and some changes were made in the translated version to reflect differences in culture and language (e.g., the word “read” in items fhl1, fhl4, and fhl5). Furthermore, a small group of volunteer patients was interviewed about difficult-to-understand words, phrases, or questions to make further minor modifications. Carefully discussing the translation in this manner could help improve the quality of a local version of a survey instrument ⁽²¹⁾. Regarding the survey method, we received important feedback from patients about avoiding morning as the time for questionnaire completion (many outpatients indicated that it was difficult for them to concentrate on the survey as a result of undertaking routine fasting for their morning visit to the hospital) and that it would be helpful to receive a handout explaining how to correctly fill in the survey. This feedback shows that it is important to interactively communicate with patients from the survey planning and preparation stages.

In terms of EFA, KMO value of over 0.8 and a significant Bartlett’s test result indicated that our dataset was adequate for factor analysis ⁽²²⁾. Eigenvalues greater than 1 for three factors represented the amount of total variance in each item that can be explained by the principal component. Furthermore, these three factors accounted for most of the total variability in the dataset. By then applying CFA to Ishikawa et al.’s original HLS-14 ⁽⁹⁾, it was found that five (CMIN/DF, GFI, TLI, RMSEA, and PCLOSE) out of six fit indices (except CFI) indicated a poor fit. Such differences between the original and translated versions of HL measurement were reported in a previous study ⁽²³⁾. To improve the model fit, the first communicative HL item (chl1: “Collected information from various sources”) was deleted to increase all fit index values to acceptable levels. A similar modification was reported for the HLS-14 Brazilian-Portuguese version ⁽²⁴⁾. Diabetes-related information comes from diverse sources, including physicians, healthcare professionals, peer patients, diabetes education programs, relatives and friends, mass media, the Internet, leaflets, and newsletters. At present, many patients with diabetes can easily find health information in the Internet. Vietnam is also a country where the Internet is widely used; the number of users in 2021 was reported to be 69 million out of the total population of 96 million ⁽²⁵⁾. With digital information becoming the single most common information source, a survey item asking regarding the collection of information from various sources may no longer be necessary.

In addition to the aforementioned good model fit, our revised HLS-13 VN exhibited satisfactory convergent and discriminant validity. The four indices (CR, MSV, AVE, and square root of AVE) indicated that each subscale was unidimensional and did not have overlap items and that Cronbach’s alpha coefficient exhibited good internal consistency for all three subscales. A similar finding was obtained in the development of Iranian ⁽²⁶⁾, Dutch ⁽²⁷⁾, and Brazilian-Portuguese ⁽²⁴⁾ versions. Our study adds to the existing evidence that for translated versions of the scale, the modified HLS-13 is suitable for HL measurement among patients. People with diabetes need to have the ability to continue self-learning, self-care, and self-management after being provided with oral and written instructions by health professionals ⁽²⁸⁾. The HLS-13 VN lists the practical skills needed to use health information in daily life. We recommend wider usage of the tool at health and medical institutes to make healthcare providers more aware of their clients’ HL capacity and better able to meet their needs. It would also help patients become aware of the importance of managing their own health.

This study has several methodological limitations. First, the validation sample consisted of outpatients with diabetes at a single hospital during the COVID-19 outbreak. The generalizability of the present findings is therefore limited. Second, those with inadequate Vietnamese language skills are often the most vulnerable portion of the population, requiring special attention and additional care in terms of health promotion. Exclusion of this group from the target population could affect the study implications. A wider and repeated application and evaluation of the devised instrument are warranted ⁽²⁹⁾.

Practical implications

The HLS-14 is used to measure different domains of HL. For Vietnamese healthcare professionals, the shorter version of the HLS-13 VN is easy to apply. The questionnaire can be widely disseminated by using digital tools such as a QR code linked to an Internet-based survey. If healthcare providers can develop an understanding of the HL level of their patients with diabetes, this allows them to improve services and communication. For decision-makers from organizations and (local) governments, this instrument can play a beneficial role in monitoring health promotion programs and optimizing diabetes care strategies.

Conclusion

To the best of our knowledge, this is the first study on a multidimensional HL assessment tool adapted for Vietnamese patients with diabetes. The HLS-13 VN exhibited sufficient reliability and validity. The tool is suitable for use in clinical settings to raise awareness among healthcare professionals of the HL needs of their clients, enabling them to improve and individualize care. Further research is warranted among more diverse populations.

Article Information

Conflicts of Interest

None

Acknowledgement

This study was conducted as a part of the Japan International Cooperation Agency’s Partnership Program (Local Government Type) “Capacity building toward evidence-based medicine among health care professionals at University of Medicine and Pharmacy, Ho Chi Minh City and its related institutes” (PI: AG) and a 2020 Grant-in-Aid for Scientific Research (C) (PI: HY, No. 20K10539). We thank Oliver Stanyon for editing the draft of this manuscript.

Author Contributions

KTV planned the study, collected and analyzed data, and wrote the manuscript. HY, KTN, GA, and TN offered advice for study planning and data analysis and contributed to the finalization of the manuscript. All authors have read and approved the final draft of the manuscript.

Approval by Institutional Review Board (IRB)

This study was reviewed and approved by the Scientific and Ethical Committee of People’s Hospital 115 (Number 1903/QD-BV115 dated 27/09/2019).

Disclaimer

Aya Goto is one of the Editors of JMA Journal and on the journal’s Editorial Staff. She was not involved in the editorial evaluation or decision to accept this article for publication at all.

References

1. 1.

   Inoue M, Takahashi M, Kai I. Impact of communicative and critical health literacy on understanding of diabetes care and self-efficacy in diabetes management: a cross-sectional study of primary care in Japan. BMC Fam Pract. 2013;14:40.

   • View Article
   • PubMed

2. 2.

   Xu XY, Leung AYM, Chau PH. Health literacy, self-efficacy, and associated factors among patients with diabetes. Health Lit Res Pract. 2018;2(2):e67-77.

   • View Article
   • PubMed

3. 3.

   World Health Organization. Health promotion glossary [Internet]. 1998 Jun [cited 2023 Aug 11]. Available from: https://www.who.int/publications-detail-redirect/WHO-HPR-HEP-98.1

4. 4.

   Berkman ND, Sheridan SL, Donahue KE, et al. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155(2):97-107.

   • View Article
   • PubMed

5. 5.

   Hersh L, Salzman B, Snyderman D. Health literacy in primary care practice. Am Fam Physician. 2015;92(2):118-24.

   • PubMed

6. 6.

   Nutbeam D, McGill B, Premkumar P. Improving health literacy in community populations: a review of progress. Health Promot Int. 2018;33(5):901-11.

   • View Article
   • PubMed

7. 7.

   Rudd RE. Needed action in health literacy. J Health Psychol. 2013;18(8):1004-10.

   • View Article
   • PubMed

8. 8.

   Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promot Int. 2000;15(3):259-67.

9. 9.

   Ishikawa H, Takeuchi T, Yano E. Measuring functional, communicative, and critical health literacy among diabetic patients. Diabetes Care. 2008;31(5):874-9.

   • View Article
   • PubMed

10. 10.

    Suka M, Odajima T, Kasai M, et al. The 14-item health literacy scale for Japanese adults (HLS-14). Environ Health Prev Med. 2013;18(5):407-15.

    • View Article
    • PubMed

11. 11.

    Nguyen TT, Trevisan M. Vietnam a country in transition: health challenges. BMJ Nutr Prev Health. 2020;3(1):60-6.

    • View Article
    • PubMed

12. 12.

    Chima CC, Abdelaziz A, Asuzu C, et al. Impact of health literacy on medication engagement among adults with diabetes in the United States: a systematic review. Diabetes Educ. 2020;46(4):335-49.

    • View Article
    • PubMed

13. 13.

    Marciano L, Camerini AL, Schulz PJ. The role of health literacy in diabetes knowledge, self-care, and glycemic control: a meta-analysis. J Gen Intern Med. 2019;34(6):1007-17.

    • View Article
    • PubMed

14. 14.

    Duong TV, Nguyen TTP, Pham KM, et al. Validation of the short-form health literacy questionnaire (HLS-SF12) and its determinants among people living in rural areas in Vietnam. Int J Environ Res Public Health. 2019;16(18):3346.

    • View Article
    • PubMed

15. 15.

    World Health Organization. Process of translation and adaptation of instruments [Internet]. 2016 [cited 2023 Jan 31]. Available from: https://www.coursehero.com/file/30372721/WHO-Process-of-translation-and-adaptation-of-instrumentspdf/

16. 16.

    Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297-334.

17. 17.

    Cristobal E, Flavián C, Guinaliu M. Perceived e-service quality (PeSQ): measurement validation and effects on consumer satisfaction and web site loyalty. Manag Serv Qual. 2007;17(3):317-40.

18. 18.

    Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1-55.

19. 19.

    Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res. 1981;18(1):39-50.

20. 20.

    Hair JF, Black WC, Babin BJ, et al. Multivariate Data Analysis. 7th ed. New York: Pearson; 2010.

21. 21.

    Wagner AK, Gandek B, Aaronson NK, et al. Cross-cultural comparisons of the content of SF-36 translations across 10 countries: results from the IQOLA project. International quality of life assessment. J Clin Epidemiol. 1998;51(11):925-32.

22. 22.

    Kaiser HF. An index of factorial simplicity. Psychometrika. 1974;39(1):31-6.

23. 23.

    Dwinger S, Kriston L, Härter M, et al. Translation and validation of a multidimensional instrument to assess health literacy. Health Expect. 2015;18(6):2776-86.

    • View Article
    • PubMed

24. 24.

    Batista MJ, Marques ACP, Silva Junior MF, et al. Translation, cross-cultural adaptation and psychometric evaluation of Brazilian Portuguese version of the 14-item health literacy scale. Cien Saude Colet. 2020;25(7):2847-57.

    • View Article
    • PubMed

25. 25.

    Nguyen MN. Internet usage in Vietnam [Internet]. Statista; 2022 Jul [cited 2023 Aug 11]. Available from: https://www.statista.com/topics/6231/internet-usage-in-vietnam/

26. 26.

    Reisi M, Mostafavi F, Javadzede H, et al. The functional, communicative, and critical health literacy (FCCHL) scales: cross-cultural adaptation and the psychometric properties of the Iranian version. Iran Red Crescent Med J. 2017;19(1):e29700.

27. 27.

    Van der Vaart R, Drossaert CH, Taal E, et al. Validation of the Dutch functional, communicative and critical health literacy scales. Patient Educ Couns. 2012;89(1):82-8.

    • View Article
    • PubMed

28. 28.

    White RO, Wolff K, Cavanaugh KL, et al. Addressing health literacy and numeracy to improve diabetes education and care. Diabetes Spectr. 2010;23(4):238-43.

    • View Article
    • PubMed

29. 29.

    Lee EH, Kim CJ, Lee J, et al. Self-administered health literacy instruments for people with diabetes: systematic review of measurement properties. J Adv Nurs. 2017;73(9):2035-48.

    • View Article
    • PubMed