The modification, translation, and validation of the Malaysian version of Score for Allergic Rhinitis (SFAR) questionnaire

Abstract

Background

Score for Allergic Rhinitis (SFAR) is a validated self-administered questionnaire to assess for allergic rhinitis (AR) in a population setting but was designed for a temperate climate.

Objective

This study aims to modify the SFAR for the tropical climate, translate the modified SFAR from English to Malay Language, and validate this Malaysian version of SFAR (MySFAR).

Methods

This was a cross-sectional study at an outpatient Otorhinolaryngology clinic in a tertiary center. There were 2 phases in the study: (1) the translation and validation of SFAR and (2) the testing of diagnostic accuracy. Two different groups of participants were recruited for the respective phase.

Results

In phase 1, the total MySFAR score showed good discriminant validity between AR and healthy controls (13.44 ± 1.58 v 1.00 ± 2.12, p < 0.01). The internal consistency and test–retest reliability of MySFAR was excellent with Cronbach’s alpha 0.92 (95% CI 0.90–0.94) and intraclass correlation coefficient of 0.97, p < 0.01. In phase 2, MySFAR gave an AUC of 0.98 (95% CI = 0.96–1.00, p < 0.01), and a cut-off score of > 9 (J = 0.92) was determined based on the highest Youden index. This cut-off was 97.8% sensitive and 93.9% specific to predicting allergic rhinitis from non-allergic rhinitis.

Conclusion

The present study showed good validity and reliability of MySFAR among the Malaysian population. The cut-off score of > 9 was able to predict allergic rhinitis. This would be a useful screening tool for allergic rhinitis population studies in tropical countries.

Background

Allergic rhinitis (AR) is a symptomatic disorder of the nose induced after allergen exposure due to an IgE-mediated inflammation of the membranes lining the nose [1]. It is characterized by at least two of the following symptoms for more than 1 h on most days: nasal pruritus, sneezing, watery rhinorrhea, and nasal obstruction with or without conjunctivitis [1]. Several allergens such as pollens, molds, animal dander, and dust mites can trigger this immune response [2]. The diagnosis of allergic rhinitis depends on a suggestive history confirmed by standard allergy tests which are the skin prick test (SPT) or the serum-specific immunoglobulin E (IgE) test [1, 3]. However, these tests are not always freely available.

The Score for Allergic Rhinitis (SFAR) is a validated questionnaire introduced in 2002 by the Annesi-Maesano group to identify AR as well as to differentiate AR from non-allergic causes of rhinitis (NAR) by using a quantitative scoring system. There are a total of 8 questions which assess for (1) symptoms of rhinitis; (2) eye itchiness; (3) allergen which triggers nasal symptoms; (4) months or seasons the symptoms usually occur; (5) perceived allergy status; (6) prior allergy test results; (7) prior diagnosis of allergy by a medical professional; and (8) family history of allergy. Each question is assigned either one or two marks for a positive answer with a total score of 0–16 marks. A cut-off point of ≥ 7 was found to best discriminate between AR and non-AR with a sensitivity of 74%, specificity of 83%, positive predictive value of 84%, and negative predictive value of 74% [1, 4]. This questionnaire was used in many countries for prevalence study including a few Middle East countries, Africa, Turkey, and also France [5,6,7,8,9,10]. It was also translated into Chinese and Arabic versions and validated without changing the items within the questionnaire with satisfactory internal consistency and good reliability [11, 12]. However, this questionnaire was designed for a temperate climate. The questions regarding the seasonality of symptoms are not suitable for a tropical climate in Malaysia. Therefore, modifications are needed to make it more applicable to a tropical climate.

Malaysia is a multiethnic country with a different culture but learning the same language: Bahasa Malaysia. For the locals to understand better, the SFAR needs to be translated into Malay version which is our national language. The aim of this study is to adapt it according to our tropical climate and validate the questionnaire against a positive skin prick test as the gold standard to confirm allergic rhinitis after translation. This is to provide a validated tool to assess AR among the Malaysian population in the future.

Methods

This was a cross-sectional study carried out at the outpatient Otorhinolaryngology, Head and Neck Surgery (ORL-HNS) clinic, Universiti Kebangsaan Medical Centre (UKMMC), from June 2020 to May 2022. Ethical approval was obtained from the UKMMC ethic committee (JEP-2020–348). Informed consent was obtained from all participants. The study involved 2 phases: (1) the translation and validation of SFAR and (2) testing of diagnostic accuracy. Both phases comply with the STROBE and STARD guidelines.

Phase 1: Translation and validation

Modification procedure

Permission was obtained from the original author of SFAR through email for modification, translation, and validation of the questionnaire [4]. An experienced panel consisting of four otorhinolaryngologists who have worked in Malaysia for at least 5 years evaluated the original SFAR and assess the relevance of each question for the Malaysian population.

Question 3 which assessed the presence of symptoms during the months of the year as well as seasonality (options of symptoms during autumn/winter/spring/summer) was found to be less suitable for the tropical climate. In the original scoring system, 1 point is awarded for having seasonality and one more point for having perennial symptoms. The options winter/spring/summer/autumn was removed and the 1 point for having seasonality of symptoms was deducted. However, the option for months of the year was kept to assess for perennial symptoms. One point will be awarded if the participant ticks more than 9 months a year indicating perennial symptoms [2]. Based on the literature, an itchy nose was found to be a suggestive symptom of AR compared to NAR. The question “In the past 12 months, has this nose problem been accompanied by nasal itchiness?” with the answer option of either yes/no was added. This question was added to replace the score for seasonality and a positive response will be awarded 1 point [13]. All the other questions from the original SFAR were retained and the total score remained at 16.

Translation procedure

Translation began with 2 groups of people fluent in both Malay and English languages: (1) doctor/clinician who understood medical terms and (2) professional who did not practice medicine. Forward translation began first from the original English version into the Malay version by 1 person from each group of people, then backward translation by another person from each group into the English version again. The backward-translated questionnaire was reviewed and assessed to ensure the original meaning of the questionnaire was preserved. The words or terms in both questionnaires were merged to produce the pre-final questionnaire in the Malay language: the Malaysian version of SFAR (MySFAR).

Content validity

The experienced panel re-assessed the MySFAR which consisted of 9 items and one sub-item. The panelist was each asked to review each translated item and to rate its clarity for use in local settings using a score from 0 to 10 (0 being very unclear and 10 being very clear). If any reviewer gave a score of less than 10, the item was discussed and further modifications to the wording were made to improve its clarity.

Face validity

The MySFAR was then pre-tested among 25 patients with AR. These patients were not included in the validation study later. They evaluated all the items in MySFAR, whether all sentences and options were clear and easily understood. Some items were reworded based on the comments from these participants. The workflow of modification, translation, and validation procedure is presented in Fig. 1.

Fig. 1

Flow of study beginning from the translation procedure, content validation, and face validation to produce the Malaysian version score for allergic rhinitis (MySFAR) questionnaire

Discriminant validity, internal consistency, and test–retest reliability

The final version of MySFAR was then answered by a total of 100 participants which included two groups of patients: 50 patients with AR and 50 healthy controls. These participants were recruited from the Otorhinolaryngology outpatient clinic. Patients in the AR group included adults (> 18 years old) with two or more rhinitis symptoms for more than 3 months and a confirmed positive SPT result towards aeroallergen. Patients with chronic rhinosinusitis and cognitive or psychiatric disorders were excluded from the study. The healthy volunteer group included participants who do not have rhinitis symptoms and a negative SPT towards aeroallergens. They were recruited from hospital staff and patients who visited the clinic. Those without rhinitis symptoms but who had positive SPT were excluded.

Both groups were invited to answer the MySFAR questionnaire after SPT in the clinic. The participants repeated filling in the questionnaire after 2-week interval. This was done to assess for discriminant validity, internal consistency, and test–retest reliability of the MySFAR.

Statistical analysis

The calculation of sample size for the validity of the MySFAR was based on the rule of thumb of 10 subjects per item. However, a minimum of 100 subjects were required to ensure the stability of variance–covariance matrix [14]. Discriminant validity was evaluated by comparing the mean ± standard deviation (SD) of total MySFAR between two groups by t test. Also, the score for each item in the questionnaire was compared between groups using chi-square or Kendall’s tau-b test. The internal consistency of the questionnaire and items was determined using the Cronbach alpha, a value of 0.5–0.7 was acceptable [15]. The test–retest reliability of the total MySFAR score was assessed by the intraclass correlation coefficient. An ICC value between 0.5 and 0.75 indicated moderate reliability, values between 0.75 and 0.9 indicated good reliability, and values larger than 0.9 indicated excellent reliability [16]. The data is presented as proportion (95% confidence interval).

Phase 2: Testing of diagnostic accuracy

Participants

All adults (≥ 18 years old) in the Otorhinolaryngology Clinic who complained of nasal symptoms (nose block, runny nose, sneezing, or itchy nose) for more than 3 months were recruited in the study. The patient selection was by convenient sampling method. Patients were excluded if they refuse to participate, were not literate in the Malay language, and were contraindicated for SPT.

MySFAR

All participants answered a self-administered questionnaire (MySFAR) which had been translated and validated.

Skin prick test

All participants had SPT results done within 1 year of recruitment. The allergens were aeroallergens including dust mites (Dermatophagoides pteronyssinus, Dermatophagoides farinae, Blomia tropicalis), cat fur, cockroach, grass, and mold. The participant was considered positive to the allergen when the largest diameter of wheal showed ≥ 3 mm. A positive SPT result must fulfill the criteria: 0 mm on negative control, ≥ 3 mm on positive control, and ≥ 3 mm on one of the aeroallergens [3]. Allergic rhinitis was defined as the presence of 2 or more nasal symptoms with a positive SPT. Patients with nasal symptoms but negative skin prick test was defined to have non-allergic rhinitis (NAR). A list of medications that can interfere with the result of SPT was given to patients, and those medications were withheld for a specific duration prior to SPT.

Statistical analysis

Data were analyzed using SPSS. The mean age between AR and NAR was compared using t test, categorical data were analyzed using the chi-square test, and the ordinal data were analyzed with Kendall’s b test. A receiver operator characteristics (ROC) curve was used to assess the total score of SFAR over a range of cut-off values to predict AR. The optimum cut-off score was calculated using the Youden index based on the coordinate from ROC. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratio for positive result (LR +) and negative result (LR −) were then calculated using the selected cut-off score.

Results

Phase 1

Content validity

Items 3, 4, 5, 7, and 8 scored less than 10 from the panelists’ rating in the first translation were further modified for better clarity based on expert opinion to adapt better in the local setting to give the final version of MySFAR. Item 3 sentence structure was further modified to address the symptom of itchy-watery eyes in the past 12 months as a persistent symptom that accompanied nasal symptoms to prevent confusion with intermittent infective conjunctivitis. Item 4 sentence structure was modified for a better understanding of the frequency of symptoms (perennial or intermittent). The options in item 5 for house dust and house dust mites were combined as the layperson may get confused between symptoms triggered by house dust or house dust mites, especially during house cleaning. In item 7, the term “allergy” in the local Malay language (alahan) was added instead of a translated scientific term (alergi) in the question. The same was done for item 8, the terms “asthma,” “eczema,” and “allergic rhinitis” were modified into the local Malay language (lelah, keradangan kulit, and alahan hidung) which was easier to understand. By improving the sentence structure and terms in the items above, the questionnaire was clearer for better local adaptation.

Face validity

All 25 patients with AR and positive SPT gave feedback and indicated that the items were generally clear and easily understood. Hence, no further modification was done to the questionnaire.

Discriminant validity, internal consistency, and test–retest reliability

The characteristics of the AR and healthy control groups are presented in Table 1. There was no significant difference in gender, age, race, educational level, and occupation between the groups. The total MySFAR score showed good discrimination validity between AR and healthy controls (13.44 ± 1.58 v 1.00 ± 2.12, p < 0.01) which was also reflected in each item (Table 2). The MySFAR showed excellent internal consistency with Cronbach’s alpha of 0.92 (95% CI 0.90–0.94). The corrected item-total correlation values for each item were more than 0.4 (ranged 0.41 to 0.94), indicating that each item correlates well with the total score. The intraclass correlation coefficient of total MySFAR was 0.97 (95% CI 0.95–0.98), indicating an excellent test–retest reliability.

Table 1 Summary of the demographic data of participants in the present study

Table 2 Comparison of the MySFAR scores between allergic rhinitis patients and healthy control

Phase 2

MySFAR was answered by a total of 122 participants, 89 patients with AR and 33 patients with NAR. There were 61% of females, and the mean age was 36.3 ± 13. The characteristics of the AR and NAR groups are presented in Table 3. The AR group was younger and had more symptoms of rhinorrhea, sneezing, and nasal itchiness. Among the AR, 72% were sensitized to dust mites, 31% to cockroach, 30% to cat, 6% to mold, and 2% to grass.

Table 3 Summary of the demographic data of participants in the present study

The total scores of the MySFAR plotted in ROC curves (Fig. 2) gave an AUC of 0.98 (95% CI = 0.96–1.00, p < 0.01). From the coordinates of the curve, Youden’s index (sensitivity + specificity–1) was computed to identify the optimum cut-off value of MySFAR (Table 4). The cut-off value of MySFAR with the highest Youden index was > 9 (J = 0.92). This cut-off value gave sensitivity and specificity of 97.8% and 93.9% respectively, PPV and NPV of 97.8% and 93.9%, and LR + and LR − of 16.29 and 0.02 respectively. The sensitivity and specificity of other cut-off values are represented in Fig. 3.

Fig. 2

Receiver operator characteristic (ROC) curve for the Malaysian Version of Score for Allergic Rhinitis (MySFAR)

Table 4 Coordinates of curve and Youden’s index

Fig. 3

Sensitivity and specificity of the Malaysian Version of Score for Allergic Rhinitis (MySFAR)

Discussion

The MySFAR is the Malaysian language version of the original SFAR which has been modified for the tropical climate and validated. This study demonstrated that after modification the MySFAR retained its psychometric properties comparable to the original SFAR and other translated versions. Therefore, this questionnaire is a valid and reliable tool to screen for AR in prevalence studies.

Translation and validation

The Malaysian SFAR (MySFAR) questionnaire which is in Malay language and modified for the tropical climate in Malaysia showed excellent consistency and reliability consistent with previous studies [5, 9, 11]. A self-administered questionnaire like SFAR provides an easier way of screening and estimating the prevalence of AR among a population [4]. In order to be utilized locally in Malaysia, a cross-cultural translation and adaptation are essential for a better understanding of the study population. This will enhance the results of future studies on AR. Malay is the national language in Malaysia. It is the common language among all races in Malaysia for communication. The MySFAR can also be utilized by physicians to identify patients with AR in the outpatient clinic setting.

The most important modification was to adapt the questionnaire to a tropical climate. In order to adapt to this condition, the original options for the temperate climate in the questionnaire were removed. A question regarding the presence of nasal pruritus is added, as this is one of the symptoms which points toward AR [1, 13]. This also allowed us to retain the original total score of 16 and keep the other score weights that were validated in the original questionnaire. Both forward and backward translations involved 2 groups of people who were fluent in both Malay and English languages. The doctor/clinician group resembles the translators who understand the terms, concepts, and objectives of the questionnaire, while the non-medical professional resembles the naïve group, who are not familiar with medical jargon. This can help to avoid the presence of medical jargon and preserve the objectives of the MySFAR questionnaire simultaneously.

Content validity is important as it displays that the questionnaire is fully representative of what it aims to measure. The relevance of each item and sub-item was already assessed by a panel of experts before the translation process although no scoring was done. Then, the content of the Malay translation was judged by a panel of experts using a semi-quantitative measure to better identify the items which needed further modification. Through the scoring process of 0 to 10, the items which showed a score of less than 10 from any rater were modified in a meeting. All experts must agree that the modified translated item was clear to understand while conveying the original information. The clearer translated items can ensure better comprehension.

The present study shows that the results of MySFAR have good reliability. The Cronbach alpha of the total MySFAR score is 0.92. This is comparable with the previous translation and validation studies which depicted Cronbach’s alpha of 0.69 to 0.83 [5, 8, 11, 12]. It reflects that MySFAR has good internal consistency in detecting AR even after modification [17]. Test–retest reliability for the total MySFAR in this present study shows a value of 0.97, which indicates that the result of MySFAR is reproducible. This is comparable with other studies showing an r value of 0.83 and 0.86 [11, 12].

All patients defined as having allergic rhinitis (at least two symptoms, either sneezing, runny nose, blocked nose, or itchy nose) in this study for more than 3 months confirmed by a positive SPT. It is an objective test that provides evidence of sensitization and confirms the diagnosis of suspected type I allergy especially in AR [4]. It is used to identify the specific allergen and as the gold standard in diagnosing allergies (sensitivity of 68–100% and specificity of 70–91%). In contrast, serum-specific IgE has the disadvantage of delayed results, being more expensive, and potential of high false-positive results [18]. A nasal provocation test, however, is suitable for diagnostic doubt (when the SPT is negative but history and examination are strongly suggestive of allergic rhinitis). It is more relevant in the diagnosis of occupational allergic rhinitis [1]. SPT result is therefore used as a criterion in the present study to define the patients with AR and healthy individuals without allergic utterly. MySFAR has good discriminant validity by comparing the total MySFAR scores of AR and control groups. It shows that MySFAR was able to discriminate between subjects with and without AR as a screening tool. This has been shown in previous studies as well [5, 8, 11, 12]. Hence, this questionnaire is ideal to screen for AR as SPT may not be feasible and practical, especially in primary care settings. MySFAR could be utilized for studies on the prevalence of AR as similar to previous studies [6, 7, 10].

Diagnostic validation

In this diagnostic validation study, the MySFAR gave an AUC of 0.98 which indicates excellent performance [19]. MySFAR is able to identify AR among patients with any chronic nasal symptoms with sensitivity and specificity of 97.8% and 93.9% respectively. This was comparable with another study in the adaptation of SFAR [12]. MySFAR score of > 9 also gave NPV of 97.8% and this may be utilized in a clinical setting to reduce the need to perform a SPT. The cut-off value was revised to > 9 in the present study as compared to the original value of ≥ 7 [4]. On top of clinical diagnosis, this present study applied SPT results to differentiate AR and NAR patients. The shift in the new cut-off value could be due to the item regarding the previous history of positive allergic test including SPT. As the original paper allocated 2 marks on the history of the positive allergic test, a high cut-off value was noticed in the current study to differentiate between 2 groups of patients. If the original value of ≥ 7 was adopted, a sensitivity of 98.9% was obtained but the specificity would be compromised to 69.7%. In contrast, a lower cut-off value of > 6 was proposed in another study [12]. The new value was obtained based on the clinical diagnosis from the family medicine specialist and otorhinolaryngologist separately. A lower cut-off value was attributed to the item on allergen where the pollen was not sufficient to trigger symptoms.

The strength of this study is the utilization of SPT as an objective test to confirm allergy compared to another study that utilized clinical diagnosis [12]. Thus, the cut-off value to identify allergic rhinitis in this present study was identified based on both clinical diagnosis and SPT result. This increased the level of confidence in diagnosing allergic rhinitis compared to other studies of translation and validation [11, 12].

Limitations

The limitation of this study is that SPT cannot identify patients with local allergic rhinitis (LAR) who will have negative skin prick test despite having IgE-mediated nasal inflammation in the nasal cavity [20]. LAR requires a nasal allergen provocation test or nasal secretion IgE test for its diagnosis but its incidence is low among the Asian population [21]. Another limitation of this study is that a self-administrated questionnaire has the disadvantage of misreporting before confirmation of symptoms or diagnosis by a physician. Patients’ perceptions can be different on the severity and progression of the disease [22]. Self-administered questionnaire also has a higher non-completion rate which may be contributed by their uncertainty in the diagnosis before consultation [23]. MySFAR is designed to identify AR but not its severity or frequency of symptoms. Therefore, this questionnaire is not suitable for assessing the severity of AR and follow-up for patients who are diagnosed with AR. Finally, this questionnaire is in Malay which is not suitable for the Chinese and Indian population who do not understand Malay. In Malaysia, Mandarin and Tamil are languages among different races. However, Malay is the national language understood by most Malaysians. There is also a validated Mandarin version that was published although in a temperate climate [12]. Further work on translation into Tamil may be done in future study.

Conclusion

MySFAR is a validated and reliable questionnaire with good sensitivity and specificity. It may be used in epidemiological studies in tropical regions to identify AR and also to screen for allergic rhinitis in a clinical setting.