Test-retest assessment of the olfactory test reliability (Odorized Markers Test)

Summary

Aim: The aim of this study was to determine the test-retest reliability of the Odorized Markers Test (OMT). This psychophysical test is a tool for screening anosmia based on the identification of odorants. The test is used for the Czech population. Examination technique consists of two parts. First, subjects are asked to spontaneously name the odorants and second, subjects have to choose the most proper name for an odorant from four descriptors. Methods: The participants included 150 subjects: 77 healthy subjects and 73 patients with smell disorders. Olfactory functions were assessed using the OMT. Retest was performed on an average interval of 2 days. Results: The coefficient of correlation between the first test and retest was 0.91. The test-retest showed that the controls reached higher scores than patients with olfactory impairment (P < 0.001). The first part of the test showed higher degree of internal consistency than the second part. The results for the first testing were α = 0.9 and 0.56, respectively, and for retesting α = 0.89 and 0.65, respectively. Anosmia was found in 26 (17%) and 27 (18%) patients on test and retest, respectively. Conclusion: Positive significant correlation was found between the first test and retest. Internal consistency was satisfactory in both the first test and retest. Based on the results of our study, OMT is a reliable and valid test for olfactory evaluation in clinical practice.

Keywords:

screening – anosmia – test – olfaction – smell disorders

Introduction

The Odorized Markers Test (OMT) is a tool for screening assessment of olfaction in the Czech population. It is used daily in clinical practice. There are normal values available for subjects in the Czech population [1]. The influence of colors on screening anosmia was not found [2]. The OMT can be performed by patients themselves [3]. The self-administered procedure is reducing the demands on medical staff and the test is decreasing the risk of infection, too. Olfactory dysfunction has been observed as one of the frequent symptoms in the SARS-CoV-2 infection [4]. The selfadministered OMT can be used to evaluate COVID-19 patients in home quarantine or in hospital isolation. However, data for test-retest reliability are not available for OMT despite the fact that this provides an important information on the reliability of OMT. On the other hand, there are several studies which assessed sense of smell using the OMT [5– 8]. The participants suffering from sinonasal diseases achieved lower scores when compared to healthy subjects. The difference between scores of patients with post-traumatic olfactory loss and healthy controls was also significant [5]. In healthy individuals, women reached a higher score than men, but the difference was not statistically significant. This study showed that three olfaction tests (OMT, Sniffin’ Sticks test – identification subtest, Test of Odor Pleasantness [TOP]) demonstrated a significant mutual correlation and a negative correlation with age [7]. OMT can be used to screen olfaction in patients with idiopathic Parkinson’s disease. Patients with Parkinson’s disease scored significantly lower than healthy participants [8]. On the other hand, it was not possible to discriminate patients with Parkinson’s disease from patients with other etiology of olfactory loss (post-traumatic, post-viral and others) using the OMT [6].

Examination technique consisted of two parts. The subjects are asked to spontaneously name the odorants and then they must choose the most proper name for an odorant from four descriptors. The spontaneous naming of the odorant is not a common technique in psychophysical olfactory tests. Therefore, we decided to further evaluate OMT reliability. The test-retest reliability was investigated in healthy subjects and patients with olfactory dysfunction.

Materials and methods

Participants

All subjects agreed to participate in the study and signed the informed consent. Olfactory functions were assessed using the OMT in 150 subjects. Mean age of participants was 41 years, standard deviation was 20 (age range 11–89) years. 61 (41%) men and 89 (59%) women took a part in our study. 27 (18%) participants were smokers and 42 (28%) participants stated their workplace as high-risk for smell deterioration (chemical and dust workers). Healthy people as well as patients suffering from olfactory disorders were tested. Descriptive statistic of the 3 tested groups is depicted in Tab. 1.

The first group included healthy participants. They did not have any problems with their sense of smell at the time of testing and they did not suffer from any disease with a possible impact on sense of smell. The second group included patients with sinonasal olfactory dysfunction. The third group included patients with other etiology of smell disorders (post-traumatic, post-viral, patients with head and neck cancer, patients with tracheostomy).

Procedure

In the first part of the examination, participants were questioned about their illness especially nasal problems, smoking, subjective problems with taste, and working environment. They were also asked to provide a subjective evaluation using a Visual Analogue Scale (VAS) of their olfaction and nasal patency.

The second part included examination of olfaction. The sense of smell was tested using the OMT. The follow-up investigation was performed in an average interval of 2 days. OMT includes 6 colored and odorized pens (perfumed markers). The black marker, presented as the first one smells like a liquorice, the second yellow one like a lemon, brown one like a cinnamon, blue one like a blueberry, green one like an apple, and the red one like a strawberry. OMT is based on spontaneous naming and odor identification from a list of 4 options. In the first part, participants were asked to name the odors they smelled and to identify each marker by a different name. They scored 1 point for naming each of the odors differently. If they were unable to name the odor at all or gave the same name to more than 1 odor, they scored 0 points. Next, the participants were asked to select 1 correct answer from a list of 4 descriptors. They gained 1 point for every correct identification (Tab. 2). The minimum and maximum possible scores were 0 and 12 points, respectively [5,7]. The score from 9 to 12 points means a normal value, the score from 6 to 8 points indicates hyposmia and 0 to 5 points indicates anosmia [1].

An open end of each felt-tip pen was placed 2 cm in front of both nostrils, where it was held for about 4 s. The interval between odor presentations was approximately 15 s. Testing was performed in a quiet room with an adequate ventilation.

Statistical analysis

The data were analyzed by using a statistical program software NCSS 2019 (Kaysville, UT, USA) and a statistical analysis and graphics software Statistica 12 StatSoft (Prague, Czech Republic). Correlation analyses were performed using the Pearson correlation method. The nonparametric Kruskal-Wallis post-hoc using a Dunn test with Bonferroni modification were used for the identification of the results depending on the etiology. The Student T-test was used for the assessment of differences between test-retest results. The alpha level was set at 0.05. The standardized Cronbach alpha coefficient for internal consistency was used.

Results

Median of subjective evaluation using the VAS (0–10) was 8 for olfaction and 9 for nasal patency. 15 (10%) participants reported subjective problems with sense of taste.

The correlation of subjective evaluation of olfaction using the VAS and nasal patency using the VAS and OMT was 0.65. and 0.47, respectively.

Test–retest reliability of the total OMT score

The coefficient of correlation between the first test and retest was 0.91 (Fig. 1). The mean value was 8.41 and 8.72 for the first test and retest, respectively. The difference of mean value was not significantly different (P = 0.40). The controls reached higher scores than patients with olfactory impairment. The median and T-values are depicted in Tab. 3. The total loss of smell was found in 26 (17%) and 27 (18%) patients on the first test and retest, respectively.

The standardized Cronbach alpha coefficient for internal consistency at the first examination of OMT items was 0.90 for the first part and 0.56 for the second part. Considering the retest, Cronbach alpha coefficient was 0.89 and 0.65 for the first and the second part, respectively. The participants reached higher scores on the first part of the test. The coefficient of correlation on the first part based on spontaneous naming of odors between the first test and retest was 0.90 (Fig. 2). The mean score on the first part of the test was 5.11 and 5.13 (P = 0.93) for the first test and retest, respectively. The coefficient of correlation on the second part based on forcedchoice technique between the first test and retest was 0.67 (Fig. 3). The mean value for the second part of the test was 3.76 for the first test and 4.14 for the retest (P = 0.12).

Healthy subjects

Women reached higher scores than men in the first test as well as in the retest. Average scores for the first test were 11 and 10 (P = 0.27) for women and men, respectively; and 11 and 9 points (P = 0.07) for the retest for women and men, respectively. The correlation coefficient of the age and the results of the first test and retest was –0.4369 and –0.4976, respectively.

Discussion

OMT is a psychophysical method for screening anosmia in clinical practice used in the Czech Republic [5].

There are studies on test-retest reliability of internationally used olfactory tests such as Sniffin’ Sticks and The University of Pennsylvania Smell Identification Test (UPSIT).

The Sniffin’ Sticks test is a comprehensible psychophysical test. It is a tool used in a daily clinical practice as well as scientific research for the assessment of olfactory performance. According to the results of the study performed by Haehner et al [9], the follow-up investigation was performed in 69 controls within an average interval of 4 days. Test–retest reliability was found to be r = 0.88 for Sniffin’ Sticks (odor identification) [9]. In our study, positive significant correlation was found between the first test and retest (r = 0.91).

Sniffin’ Sticks test allows a precise evaluation of olfactory function especially when different olfactory tasks are assessed using individual subtests. Patients scored significantly worse than control subjects (P < 0.001) on the identification test [9]. Our study showed that OMT was a valuable method considering evaluation of olfactory loss with different etiologies. Healthy participants reached significantly higher scores than patients with smell disorders in both the first test and retest.

In the study by Oleszkiewicz et al [10], there was also a significant yet small main effect of sex, suggesting that on average, females outperformed males on the overall transition dyspnea index score on the Sniffin’ Sticks test [10]. The same result was confirmed in our study in controls, where the women reached higher scores than men. However, the results were not statistically significant. Studies on the Sniffin’ Sticks test proved that age played and important role when olfaction is evaluated [9]. The most pronounced decrease was at the age of 61–70 years [10]. There was a negative correlation of the results of the OMT in healthy participants and their age in our study. These results confirmed the decreased ability of identification in older healthy subjects. The result was proved for the first test and the retest. UPSIT test-retest reliability supports the utility of UPSIT as a reliable test not only over short time intervals but also over long intervals in cognitively intact older adults, thereby providing support for clinical and research utility [11]. Results of a study performed by Pospíchalová et al [8] indicated that patients with Parkinson’s disease reached a significantly lower score on OMT compared to healthy participants. The standardized Cronbach alpha coefficient for internal consistency (α = 0.83 and 0.78 for the first test and retest, respectively) was used for UPSIT [11]. A similar result for internal consistency was confirmed in our study for OMT. The first part, based on the spontaneous naming of the odor, showed a higher degree of internal consistency than the second part of the test based on the forced-choice technique. The internal consistency was high in both the first test and retest.

Conclusion

Positive significant correlation was found between the first test and retest. Internal consistency was satisfactory in both the first test and retest. The test-retest showed that the controls reached higher scores than patients with olfactory impairment. Based on the results of our study, OMT is a reliable and valid test for olfactory evaluation in clinical practice.

Ethical approval

This study was performed in line with the principles of the Declaration of Helsinki. The methodology for this study was approved by the Human Research Ethics committee of Pardubice Hospital (503/15.1. 2016).

Informed consent was obtained from all of the individual participants of the study.

Funding

This study was funded by the University of Pardubice, Faculty of Health Studies (IGS, grant number IGSFZS_ 2021_001) and TA ČR GAMA 2 (TP01010012).

Conflict of interest

The authors declare that they have no conflict of interest.

The Editorial Board declares that the manuscript met the ICMJE “uniform requirements” for biomedical papers.

Redakční rada potvrzuje, že rukopis práce splnil ICMJE kritéria pro publikace zasílané do biomedicínských časopisů.

Mgr. Pavlína Brothánková, PhD

Faculty of Health Studies University of Pardubice Průmyslová 395 532 10 Pardubice Czech Republice

e-mail: pavlina.brothankova@upce.cz

Accepted for review: 15. 11. 2021

Accepted for print: 7. 2. 2022