Objective(s): The cutting-edge assessment of voice disorders includes objective and subjective methods in the daily clinical practice. The latter assessment is usually performed through the administration of self-reported questionnaires. Voice Handicap Index (VHI) is one of the most widely used tools both in clinical practice and in research level. This tool-questionnaire was employed in this research along with the Voice Evaluation Template (VEF). In turn, the aim of this study was to analyse and produce the cut-off points of VHI for voice-disordered patients in Greece by using Receiver Operating Characteristic Curves (ROC Curves).
Methods: Sixty-three participants (40 non-dysphonic and 23 with different types of dysphonia) were classified by ENT (Ear, Nose, and Throat) doctors and SLPs (Speech-Language Pathologists). The Hellenic VHI along with the translated Greek version of the VEF was administered to the subjects of this research.
Results: The voice-disordered subjects exhibited higher overall VHI scores (in total and in its 3 subdomains) compared to the control group. Statistical significant differences were found between dysphonic and non-dysphonic participants for all VHI’s construct domains. The cut-off point of VHI total score was estimated at the value of 14.50 (sensitivity: 0.870, 1-specificity: 0.000). Moreover, the cut-off points of the three subdomains were computed as 7.50 for functional (sensitivity: 0.783, 1-specificity: 0.000), 8.50 for physical (sensitivity: 0.739, 1-specificity: 0.000) and 8.50 for emotional domain (sensitivity: 0.783, 1-specificity: 0.050).
Conclusion: The preliminary statistical and ROC data analysis of VHI concluded that by using this type of assessment method, populations with or without voice disorders (in Greece) can be distinguished. Albeit this tool is a non-interventional method it could consequently offer an adequate screening and monitoring capability.
Voice-disordered populations are regularly examined by Otolaryngologists during their daily clinical practice. The etiology of a voice disorder may vary due to changes of the laryngeal structure [1] [2] [3] [4] [5]. For the diagnosis of a voice disorder there is a plenty of clinical voice assessments [6] , different types of protocols [7] , methods [8] and classifications [9] in the literature. From 2001, the European Laryngeal Society (ELS) [10] recommended guidelines for a functional assessment of a voice pathology. The guidelines included laryngeal imaging [11] [12] [13] [14] , clinical evaluation (acoustic, perceptual and aerodynamic measurements) [15] - [21] and moreover the voice self-assessment by the patient himself [10] [21] [22] [23] [24].
Particularly, the objective evaluation of voice function can provide information about the voice impact on patient’s quality of life [10] while it determines the severity of the disability that the patient perceives [10] [24]. For this type of non-interventional voice disorders evaluation, a number of questionnaires were developed [21] [22] [23] [24]. Specifically, Jacobson, et al. [25] developed the Voice Handicap Index (VHI) in 1997. Since its first release, VHI has been translated in many languages [26] - [33] and has been used vastly in research level [34] [35] [36] [37]. The VHI was also cross-culturally translated and adapted into Greek language [38] and in turn was used in research level relevant to Greek population [39] [40] [41] [42].
Furthermore, many studies reported that self-assessment methods may be considered as potential screening tools for distinguishing between dysphonic and non-dysphonic individuals [43] - [48]. The threshold’s value of differentiation is called cut-off point. This cut-off point can be obtained through ROC analysis [49]. The maximum value of 1.0 for sensitivity and at the same time a lowest value of almost 0 for 1-specificity, indicate an instrument’s efficiency to perform perfect classifications [49]. This kind of methodology was developed by electrical and radar engineers during Second World War for detecting enemy threats [49]. Thereinafter, the ROC curve methodology was adapted by medical sciences to evaluate the discrimination ability of an assessment tool between typical population and subjects exhibiting pathology [50]. Over the last few years, self-perceived questionnaires’ cut-off points are calculated for people with voice disorders [45] [46] [47] and particularly from VHI results [43] [44] [51] [52] [53].
Specifically, this research aims to provide preliminary data for the VHI’s cut-off points relevant to Greek voice-disordered patients.
2. Materials and Methods2.1. Participants
Sixty-three participants (23 dysphonic patients and 40 non-dysphonic subjects) enrolled in this study. The mean age of the total sample was calculated equal to 41.24 yrs (SD = 11.60) with males’ mean age at 43.90 yrs (SD = 12.17) while for females at 41.81 yrs (SD = 11.13). The mean age of control group was equal to 38.04 yrs (SD = 12.22) which was ranged from 22 to 42 years of age, with males mean age being equal to 38.81 yrs (SD = 12.31) and for females being equal to 37.56 yrs (SD = 11.80). The mean age of patients with voice disorders (VDP) was equal to 45.19 yrs (SD = 10.99 years) which was ranged from 26 to 49 years of age, with males mean age being equal to 45.67 yrs (SD = 10.05) and with females’ age being equal to 42.59 yrs (SD = 11.26). All demographic data are presented in Table 1.
All subjects were classified by Ear-Nose-Throat (ENT) doctor and a Speech Language Pathologist (SLP). The patient subgroups were diagnosed via video laryngeal endoscopy-stroboscopy. The VDP subgroup did not experience previous medical condition that affected their voice during recruitment. The twenty-three (23) dysphonic patients were under the supervision of a private ENT and SLP office in Ioannina, Greece.
All patients were split into three broad diagnostic subgroups. The first subgroup was the “Laryngeal Mass Lesions (LML)” (6 patients), from which three (3) were diagnosed with vocal nodules, two (2) with vocal polyps and one (1) with leukoplakia. The second subgroup included “Laryngeal Inflammatory Disorders (LID)” (7 patients), from which two (2) patients were diagnosed with Reinke’s edema and four (4) with chronic laryngitis. The third subgroup included ten (10) “Neurogenic Voice Disorders (NVD)” patients which were diagnosed with voice disorder due to hypokinetic dysarthria.
The control group data was collected from people who accompanied patients and from subjects from the School of Health and Welfare Professions, TEI of Epirus. The participants non-selection was based on the following criteria for the last two weeks before data collection: 1) any occurrence of upper or lower respiratory system disorder or any laryngeal/vocal complaints; 2) the existence of symptoms of gastroesophageal reflux (GERD) and/or laryngopharyngeal reflux (LRP) disease; 3) the undergone voice therapy and/or voice disorders existence in the past; 4) any history of alcohol and/or drugs abuse; 5) living or/and working environments with smoke or/and dust presence, exposure to chemicals, external noise, and/or allergens, etc., and other factors that affected voice.
Twenty (20) of the sixty (60) non-dysphonic patients (controls) were excluded because they did not meet the study’s criteria. The nine (9) of them had a history of gastroesophageal reflux (GERD) and/or laryngopharyngeal reflux (LRP) disease with history of voice disorders in the past, three (3) cases had a history of alcohol abuse and in one (1) case the use of recreational drugs. Also two (2) cases had a history of exposure to noisy-dusty industrial environments and the remaining five (5) cases had a history of laryngopharyngeal reflux (LRP) disease with exposure to smoke or/and dust.
2.2. Data Collection
Before the enrolment of all subjects, they were informed for the confidentiality of obtained data and they signed a consensus letter. The Hellenic VHI [38] and the translation of Greek Voice Evaluation Template (VET) [54] were administered to all participants. VHI is a subjective tool consisted of 30-questions spilt into three domains [functional (VHI-F), physical (VHI-P) and emotional (VHI-E)]. Each domain’s score is ranging from 0 to 40. The three domains sum is the VHI total score (VHI-T) that ranges from 0 to 120. VET is a consensus template relevant to voice that the American Speech Hearing Association (ASHA) developed for being used in the daily clinical practice.
2.3. Statistical Analysis
The distribution finding of variables was performed with Kolmogorov-Smirnov and Shapiro-Wilk tests. All skewed variables (VHI scores) were expressed through median (interquartile range), and all normal distributed variables were expressed through mean and standard deviations (SD). A Kruskal-Wallis H test was used for the comparison of the four study groups (non-dysphonic participants, LML patients, LID patients and NVD patients). Also, a Mann-Whitney U test was conducted for the comparison of the two study groups (dysphonic and non-dysphonic participants). Finally, for estimating the cut-off points, ROC curve analysis was conducted for VHI and for its three domains. All reported P values were two-tailed and the statistical significance was set at P < 0.05. The analysis was performed using SPSS statistical software (version 19.0, Armonk, NY, USA).
3. Results
Sample consisted of 63 participants (40 controls and 23 VDP). The VDP group had a mean educational level of M = 12.45 yrs (SD = 3.43) while the control group a mean of 12.98 yrs (SD = 5.09). The VDP group smoked for an average of M = 14.09 yrs (SD = 5.12) while the control group did not smoke [M = 0.00 (SD = 0.00)]. The majority of this sample’s subjects lived in urban areas and most of them were married (Table 1).
The VDP subgroup had a significant higher overall VHI total score compared to non-smokers, U = 70.000, P < 0.001. Similar statistically significant differences of medians were found for VHI-F (U = 113.500, P < 0.001), VHI-P (U = 127.500, P < 0.001) and for VHI-E (U = 118.500, P < 0.001). The VDP subgroup (in all comparisons) exhibited the higher achieved scores (Table 2). The results of VHI (Figure 1) and its three domains are represented with box plots (Figures 2-4).
Α Kruskal-Wallis test was conducted for comparing subgroups’ medians of all VHI total scores and their domains. Particularly, significant statistical differences were observed between non-dysphonic and dysphonic groups for VHI-T score [H (3) = 37.520, P < 0.001] with mean rank of 22.25 for non-dysphonic participants, 52.25 for LML patients, 35.50 for LID patients and 56.40 NVD patients. Likewise, computations were conducted concluding to VHI-F domain [H (3) = 29.780, P < 0.001] with mean rank of 23.34 for non-dysphonic participants, 52.75 for LML patients, 34.29 for LID patients and 52.60 NVD patients. Similarly, results included VHI-P domain [H (3) = 30.838, P < 0.001] with mean rank of 23.69 for non-dysphonic participants, with mean rank of 47.08 for LML patients, of 31.57 for LID patients and of 56.50 for NVD patients. Finally, for VHI-E
Subjects’ demographic data
Dysphonic Patients (N = 23)
Non-Dysphonic Patients (N = 40)
Mean (SD)
Mean (SD)
Age
38.31 (12.31)
38.04 (12.22)
Educational level
12.45 (3.43)
12.98 (5.09)
Years of smoking
14.09 (5.12)
-
Marital status
Married
13
18
Single
10
22
Area of living
Urban area
23
26
Suburban area
-
8
Rural area
-
6
Comparisons of medians between controls and VDP for VHI total score and VHI domains
Controls (N = 40)
VDP (N = 23)
Mann-Whitney U
P level
Median (IQR)
Median (IQR)
Total
13.00 (12.00 - 13.75)
45.00 (20.00 - 53.00)
70.000
<0.001*
Functional
4.00 (3.00 - 5.00)
14.00 (8.00 - 18.00)
113.500
<0.001*
Physical
4.00 (3.00 - 5.00)
18.00 (6.00 - 23.00)
127.500
<0.001*
Emotional
5.00 (3.00 - 5.00)
14.00 (7.00 - 16.00)
118.500
<0.001*
*p level at P < 0.05; Abbreviations: IQR, interquartile range; VHI, Voice Handicap Index.
domain [H (3) = 29.530, P < 0.001] was computed the mean rank of 23.46 for non-dysphonic participants, the mean rank of 55.17 for LML patients, of 34.71 for LID patients and 50.35 for NVD patients.
A ROC analysis was calculated in order to determine the cut-off points of VHI-T and of its three domains. A statistical significant positive discrimination between VDP and controls was observed. Specifically, a very high effect was detected for VHI-T (AUC 0.924, P < 0.001), VHI-F (AUC 0.877, P < 0.001), VHI-P (AUC 0.861, P < 0.001) and VHI-Ε domain (AUC 0.829, P < 0.001) (Table 3).
The cut-off point of VHI-T score was found equal to 14.50 with sensitivity of 0.870 and 1-specificity of 0.000 (Figure 5), the VHI-F cut-off point was equal to
Coordinates for VDP and controls curve for VHI total score and VHI domains
AUC
SE
P level
95% CI
Total
0.924
0.046
<0.001*
0.834 - 1.000
Functional
0.877
0.053
<0.001*
0.772 - 0.981
Physical
0.861
0.058
<0.001*
0.747 - 0.975
Emotional
0.871
0.057
<0.001*
0.760 - 0.982
*p level at P < 0.05; Abbreviations: AUC, area under curve; CI, confidence interval; SE, standard error; VHI, Voice Handicap Index.
7.50 with sensitivity of 0.783 and 1-specificity of 0.000 (Figure 6). Also, the VHI-P cut-off point was equal to 8.50 with sensitivity of 0.739 and 1-specificity of 0.000 (Figure 7) and VHI-E cut-off point was equal to 8.50 with sensitivity of 0.783 and 1-specificity of 0.050 (Figure 8). All values of figures’ axis are presented as percentages (e.g. sensitivity of 0.870 corresponds to sensitivity of 87%).
4. Discussion
This study indicated that participants with voice disorders had significant higher median scores in VHI-T and its three domains compared to non-dysphonic
patients. Specifically, VHI-T median score was estimated at the value of 45.00 which was approximately the same with other research [25] [26] [28] [32] [33] [38]. The VHI-F domain was computed at the value of 14.00 and it was almost similar to research with voice disordered population [25] [26] [28] [32] [33] [38]. Likewise, VHI-P domain exhibited a value of 18.00 which was almost in agreement with other works [25] [26] [28] [32] [33] [38]. Similarly, the VHI-E median was calculated at the value of 14.00 which was in agreement with Jacobson et al. (VHI-E = 13.33) [25] , Behlau et al. (VHI-E = 13.90) [28] and Trinite & Sokolovs research (VHI-E = 16.11) [33]. These aforementioned results indicate the potentiality of the VHI utility being probably a strong screening tool.
Only a few only studies determined the cut-off points of VHI’s ROC curves for dysphonic patients [43] [44] [47] [51] - [57] as well as for other populations [52] [53]. All these studies used the original versions of VHI in their languages and the results showed that the cut-off values varied from 12 to 20 [43] [44] [47] [51] - [57]. In the aforementioned studies, the AUCs were satisfactory. Van Gogh et al. [57] used the Dutch version of VHI to compare patients with cancer at the glottic area and benign laryngeal lesions, while he set a threshold of VHI-T equal to 15 with sensitivity of 0.97 and 1-specificity of 0.86, which is similar result to this research. Additionally, Moradi et al. [44] with the use the cross-cultural Persian version of VHI concluded that the cut-off value was equal to 14.5 with a sensitivity of 92% and a specificity of 95% which is also equal to this research.
Furthermore, the German Version of VHI [55] calculated a cut-off value of 12 for VHI-T and this is approximately in agreement with the Greek threshold. Likewise, the same cut-off value was obtained (VHI-T = 12) for the Polish version of VHI with sensitivity of 0.98 and specificity of 0.95 and again was approximately in agreement with the Greek cut-off value [43]. Behrman, Rutledge, Hembree and Sheridan [56] analyzed data from 156 questionnaires that were answered only by women and found a VHI-T cut-off score of 11.5 and this is again close enough to the Greek obtained value of 14 for VHI-T subdomain.
On the contrary, other research groups have found higher cut-off scores in comparison to this preliminary study. Particularly, the Swedish team calculated a cut-off value of VHI-T = 20 with a sensitivity of 0.77 and a specificity of 0.87 [58]. However, they underlined limitations to their study. Moreover, a cut-off value of 19 (95% sensitivity) refers to the Norwegian version of VHI [29]. Finally, a research in American population with organic dysphonia after thyroidectomy found a cut-off value of VHI-T equal to 18 [51] and this score is higher compared to the score of this study. Probably, these higher scores are due to differences in the studied population [47].
Finally, is interestingly enough that the above studies computed only cut-off points for VHI-T and not for the rest of its domain (VHI-F, VHI-P, VHI-E) which has a good diagnostic value as it was underlined in former studies [52] [53]. Additionally, to the above, only Karlsen et al. [29] underlined that VHI-F seemed to score accurately as a diagnostic discrimination between dysphonic and non-dysphonic condition in their study and this result is in agreement to this research. Nevertheless, additional sampling is needed with the inclusion of furthermore subjects to this preliminary research in order to produce a large sample under test.
5. Conclusion
The aim of this study was to estimate the VHI’s cut-off points (including its three domains) for patients with different types of voice disorders. VHI can profoundly distinguish the voice self-perception between dysphonic and non-dysphonic individuals. The utility could be used as a screening and self-monitoring tool for dysphonic patients. The proper use of VHI’s cut-off points could help a voice clinician to probably foresee patients’ progress in the short and long run while assisting the clinician to better customize a treatment planning. Finally, further research is suggested to be conducted in larger populations with deviant voice problems in Greece while the factor of the type of building environment should be considered for research [59]. Additional research of measuring physical voice characteristics could be conducted with the employment of advanced wireless technologies [60] [61] [62] [63].
Conflict of Interest
All authors declare no conflicts of interest in this manuscript.
Cite this paper
Tafiadis, D., Kosma, E.I., Chronopoulos, S.K., Voniati, L. and Ziavra, N. (2018) A Preliminary Receiver Operating Characteristic Analysis on Voice Handicap Index Results of the Greek Voice-Disordered Patients. International Journal of Otolaryngology and Head & Neck Surgery, 7, 98-114. https://doi.org/10.4236/ijohns.2018.73013
Abbreviations List
ASHA = American Speech Hearing Association
ENT = Ear Nose Throat
ELS = European Laryngeal Society
GERD = Gastroesophageal Reflux
LID = Laryngeal Inflammatory Disorders
LMS = Laryngeal Mass Lesions
LRP = Laryngopharyngeal Reflux
NVD = Neurogenic Voice Disorders
SLP = Speech Language Pathologist
ROC = Receiver Operating Characteristic
TEI of Epirus = Technological Educational Institute of Epirus
VEF = Voice Evaluation Template
VHI = Voice Handicap Index
VHI-E = Voice Handicap Index―Emotional
VHI-P = Voice Handicap Index―Physical
VHI-F = Voice Handicap Index―Functional
VHI-T = Voice Handicap Index―Total score
VDP = Voice Disorders Patients
ReferencesCmelak, A.J., Li, S., Murphy, B., Burkey, B., Adams, G.L., Cannon, M. and Forastiere, A.A. (2006) Locally Advanced Resectable Larynx (L) or Oropharynx (OP) Cancer: Updated Results of Organ Preservation Trial ECOG 2399. Journal of Clinical Oncology, 24, 5527-5527.Rajani Kanth, T., Nanayya, R., Satyaprabhakar, Y., Blandeena, K. and Murthy, P. (2006) Giant Supraglottic Schwannoma. Indian Journal of Otolaryngology and Head and Neck Surgery, 58, 397-398. https://doi.org/10.1007/BF03049610Sonkhya, N., Mehta, R., Sonkhya, D., Gupta, S. and Faujdar, M. (2013) Primary Histoplasmosis of Larynx: A Case Series and Review of Literature. International Journal of Otolaryngology and Head & Neck Surgery, 2, 47. https://doi.org/10.4236/ijohns.2013.22012Mohyuddin, A. and Sandhu, G. (2017) Reinke’s Oedema Presenting as Stridor: Implications for Otolaryngologists in Difficult Airway. International Journal of Otolaryngology and Head &Amp; Neck Surgery, 6, 11-15. https://doi.org/10.4236/ijohns.2017.62003Cooper, J., Pajak, T., Forastiere, A., Jacobs, J., Campbell, B., Saxman, S., et al. (2004) Postoperative Concurrent Radiotherapy and Chemotherapy for High-Risk Squamous-Cell Carcinoma of the Head and Neck. New England Journal of Medicine, 350, 1937-1944. https://doi.org/10.1056/NEJMoa032646Roy, N., Barkmeier-Kraemer, J., Eadie, T., Sivasankar, M., Mehta, D., Paul, D. and Hillman, R. (2013) Evidence-Based Clinical Voice Assessment: A Systematic Review. American Journal of Speech-Language Pathology, 22, 212-226. https://doi.org/10.1044/1058-0360(2012/12-0014)American Speech-Language-Hearing Association (2015) Recommended Protocols for Instrumental Assessment of Voice [Draft Summary, Expert Panel to Develop a Protocol for Instrumental Assessment of Vocal Function]. American Speech-Language-Hearing Association, Rockville, MD.Chiu, P., Shih, C., Tsai, Y. and Chiang, R. (2013) Laryngeal Ultrasound for Assessment of Supraglottic Laryngitis with Abscess. International Journal of Phonosurgery & Laryngology, 3, 24-27. https://doi.org/10.5005/jp-journals-10023-1055Sellam, V. and Jagadeesan, J. (2014) Classification of Normal And Pathological Voice Using SVM and RBFNN. Journal of Signal and Information Processing, 5, 1. https://doi.org/10.4236/jsip.2014.51001Dejonckere, P., Bradley, P., Clemente, P., Cornut, G., Crevier-Buchman, L., Friedrich, G., et al. (2001) A Basic Protocol for Functional Assessment of Voice Pathology, Especially for Investigating the Efficacy of (Phonosurgical) Treatments and Evaluating New Assessment Techniques. European Archives of Oto-Rhino-Laryngology, 258, 77-82. https://doi.org/10.1007/s004050000299Sataloff, R., Spiegel, J., Carroll, L., Schiebel, B., Darby, K. and Rulnick, R. (1988) Strobovideolaryngoscopy in Professional Voice Users: Results and Clinical Value. Journal of Voice, 1, 359-364. https://doi.org/10.1016/S0892-1997(88)80012-2Emilia Elias, M., Thayer Sataloff, R., Rosen, D., Heuer, R. and Spiegel, J. (1997) Normal Strobovideolaryngoscopy: Variability in Healthy Singers. Journal of Voice, 11, 104-107. https://doi.org/10.1016/S0892-1997(97)80030-6Woo, P. (1996) Quantification of Videostrobolaryngoscopic Findings-Measurements of the Normal Glottal Cycle. The Laryngoscope, 106, 1-27. https://doi.org/10.1097/00005537-199603001-00001Sataloff, R.T., Mandel, S., Mann, E.A. and Ludlow, C.L. (2003) Laryngeal Electromyography: An Evidence-Based Review. Muscle & Nerve, 28, 767-772. https://doi.org/10.1002/mus.10503Sapienza, C. (1997) Aerodynamic and Acoustic Characteristics of the Adult African American Voice. Journal of Voice, 11, 410-416. https://doi.org/10.1016/S0892-1997(97)80036-7Rosen, C.A., Lombard, L.E. and Murry, T. (2000) Acoustic, Aerodynamic, and Videostroboscopic Features of Bilateral Vocal Fold Lesions. Annals of Otology, Rhinology & Laryngology, 109, 823-828. https://doi.org/10.1177/000348940010900907Joshi, A. and Watts, C.R. (2017) Phonation Quotient in Women: A Measure of Vocal Efficiency Using Three Aerodynamic Instruments. Journal of Voice, 31, 161-167.Lovato, A., De Colle, W., Giacomelli, L., Piacente, A., Righetto, L., Marioni, G. and de Filippis, C. (2016) Multi-Dimensional Voice Program (MDVP) vs Praat for Assessing Euphonic Subjects: A Preliminary Study on the Gender-Discriminating Power of Acoustic Analysis Software. Journal of Voice, 30, 765.e1-765.e5. https://doi.org/10.1016/j.jvoice.2015.10.012Gaskill, C.S., Awan, J.A., Watts, C.R. and Awan, S.N. (2017) Acoustic and Perceptual Classification of Within-Sample Normal, Intermittently Dysphonic, and Consistently Dysphonic Voice Types. Journal of Voice, 31, 218-228. https://doi.org/10.1016/j.jvoice.2016.04.016Yamasaki, R., Madazio, G., Leão, S.H., Padovani, M., Azevedo, R. and Behlau, M. (2017) Auditory-Perceptual Evaluation of Normal and Dysphonic Voices Using the Voice Deviation Scale. Journal of Voice, 31, 67-71.https://doi.org/10.1016/j.jvoice.2016.01.004Murry, T., Medrado, R., Hogikyan, N.D. and Aviv, J.E. (2004) The Relationship between Ratings of Voice Quality and Quality of Life Measures. Journal of Voice, 18, 183-192. https://doi.org/10.1016/j.jvoice.2003.11.003Dejonckere, P. (2009) Assessment of Voice and Respiratory Function. In: Remacle, M. and Eckel, H., Eds., Surgery of Larynx and Trachea, Springer, Berlin, Heidelberg, 11-26. https://doi.org/10.1007/978-3-540-79136-2_2Hogikyan, N.D. and Sethuraman, G. (1999) Validation of an Instrument to Measure Voice-Related Quality of Life (V-RQOL). Journal of Voice, 13, 557-569. https://doi.org/10.1016/S0892-1997(99)80010-1Deary, I., Wilson, J., Carding, P. and MacKenzie, K. (2003) VoiSS: A Patient-Derived Voice Symptom Scale. Journal of Psychosomatic Research, 54, 483-489. https://doi.org/10.1016/S0022-3999(02)00469-5Jacobson, B.H., Johnson, A., Grywalski, C., Silbergleit, A., Jacobson, G., Benninger, M.S. and Newman, C.W. (1997) The Voice Handicap Index (VHI): Development and Validation. American Journal of Speech-Language Pathology, 6, 66-70. https://doi.org/10.1044/1058-0360.0603.66Schindler, A., Ottaviani, F., Mozzanica, F., Bachmann, C., Favero, E., Schettino, I. and Ruoppolo, G. (2010) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Italian. Journal of Voice, 24, 708-714.https://doi.org/10.1016/j.jvoice.2009.05.006Malki, K.H., Mesallam, T.A., Farahat, M., Bukhari, M. and Murry, T. (2010) Validation and Cultural Modification of Arabic Voice Handicap Index. European Archives of Oto-Rhino-Laryngology, 267, 1743-1751. https://doi.org/10.1007/s00405-010-1296-xBehlau, M., dos Santos, L.D.M.A. and Oliveira, G. (2011) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Brazilian Portuguese. Journal of Voice, 25, 354-359. https://doi.org/10.1016/j.jvoice.2009.09.007Karlsen, T., Grieg, A.R.H., Heimdal, J.H. and Aarstad, H.J. (2012) Cross-Cultural Adaption and Translation of the Voice Handicap Index into Norwegian. Folia Phoniatrica et Logopaedica, 64, 234-240. https://doi.org/10.1159/000343080Taguchi, A., Mise, K., Nishikubo, K., Hyodo, M. and Shiromoto, O. (2012) Japanese Version of Voice Handicap Index for Subjective Evaluation of Voice Disorder. Journal of Voice, 26, 668.e15-668.e19. https://doi.org/10.1016/j.jvoice.2011.11.005Bonetti, A. and Bonetti, L. (2013) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Croatian. Journal of Voice, 27, 130.e7-130.e14.https://doi.org/10.1016/j.jvoice.2012.07.006Moradi, N., Pourshahbaz, A., Soltani, M., Javadipour, S., Hashemi, H. and Soltaninejad, N. (2013) Cross-Cultural Equivalence and Evaluation of Psychometric Properties of Voice Handicap Index into Persian. Journal of Voice, 27, 258.e15-258.e22.https://doi.org/10.1016/j.jvoice.2012.09.006Trinite, B. and Sokolovs, J. (2014) Adaptation and Validation of the Voice Handicap Index in Latvian. Journal of Voice, 28, 452-457. https://doi.org/10.1016/j.jvoice.2014.01.008Rosen, C.A., Murry, T., Zinn, A., Zullo, T. and Sonbolian, M. (2000) Voice Handicap Index Change Following Treatment of Voice Disorders. Journal of Voice, 14, 619-623. https://doi.org/10.1016/S0892-1997(00)80017-XBouwers, F. and Dikkers, F.G. (2009) A Retrospective Study Concerning the Psychosocial Impact of Voice Disorders: Voice Handicap Index Change in Patients with Benign Voice Disorders after Treatment (Measured with the Dutch Version of the VHI). Journal of Voice, 23, 218-224. https://doi.org/10.1016/j.jvoice.2007.08.007Meulenbroek, L.F.P., Van Opstal, M.J.C.M., Claes, L., Marres, H.A.M. and de Jong, F.I.C.R.S. (2012) The Impact of the Voice in Relation to Psychosomatic Well-Being after Education in Female Student Teachers: A Longitudinal, Descriptive Study. Journal of Psychosomatic Research, 72, 230-235. https://doi.org/10.1016/j.jpsychores.2011.11.016da Rocha, L.M. and de Mattos Souza, L.D. (2013) Voice Handicap Index Associated with Common Mental Disorders in Elementary School Teachers. Journal of Voice, 27, 595-602. https://doi.org/10.1016/j.jvoice.2012.10.001Helidoni, M.E., Murry, T., Moschandreas, J., Lionis, C., Printza, A. and Velegrakis, G.A. (2010) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Greek. Journal of Voice, 24, 221-227. https://doi.org/10.1016/j.jvoice.2008.06.005Tafiadis, D., Kosma, E.I., Chronopoulos, S.K., Papadopoulos, A., Toki, E.I., Vassiliki, S. and Ziavra, N. (2017) Acoustic and Perceived Measurements Certifying Tango as Voice Treatment Method. Journal of Voice, 32, 256.e13-256.e24. https://doi.org/10.1016/j.jvoice.2017.05.016Tafiadis, D., Chronopoulos, S.K., Siafaka, V., Drosos, K., Kosma, E.I., Toki, E.I. and Ziavra, N. (2017) Comparison of Voice Handicap Index Scores between Female Students of Speech Therapy and Other Health Professions. Journal of Voice, 31, 583-588. https://doi.org/10.1016/j.jvoice.2017.01.013Tafiadis, D., Toki, E.I., Miller, K.J. and Ziavra, N. (2017) Effects of Early Smoking Habits on Young Adult Female Voices in Greece. Journal of Voice, 31, 728-732.https://doi.org/10.1016/j.jvoice.2017.03.012Tafiadis, D., Tatsis, G., Ziavra, N. and Toki, E.I. (2017) Voice Data on Female Smokers: Coherence between the Voice Handicap Index and Acoustic Voice Parameters. AIMS Medical Science, 4, 151-163. https://doi.org/10.3934/medsci.2017.2.151Niebudek-Bogusz, E., Kuzańska, A., Woznicka, E. and Sliwinska-Kowalska, M. (2011) Assessment of the Voice Handicap Index as a Screening Tool in Dysphonic Patients. Folia Phoniatrica et Logopaedica, 63, 269-272. https://doi.org/10.1159/000324214Moradi, N., Pourshahbaz, A., Soltani, M. and Javadipour, S. (2013) Cutoff Point at Voice Handicap Index Used to Screen Voice Disorders among Persian Speakers. Journal of Voice, 27, 130.e1-130.e5. https://doi.org/10.1016/j.jvoice.2012.08.007Fava, G., Paolillo, N.P., Oliveira, G. and Behlau, M. (2015) Cross-Cultural Adaptation, Validation, and Cutoff Point of the Italian Version of the Voice Activity and Participation Profile: Profilo di Attività e Partecipazione Vocale. Journal of Voice, 29, 130.e11-130.e19. https://doi.org/10.1016/j.jvoice.2014.04.008Moreti, F., Zambon, F., Oliveira, G. and Behlau, M. (2014) Cross-Cultural Adaptation, Validation, and Cutoff Values of the Brazilian Version of the Voice Symptom Scale—VoiSS. Journal of Voice, 28, 458-468. https://doi.org/10.1016/j.jvoice.2013.11.009Behlau, M., Madazio, G., Moreti, F., Oliveira, G., dos Santos, L.D.M.A., Paulinelli, B.R. and de Barros Couto Jr., E. (2016) Efficiency and Cutoff Values of Self-Assessment Instruments on the Impact of a Voice Problem. Journal of Voice, 30, 506.e9-506.e18. https://doi.org/10.1016/j.jvoice.2015.05.022Musio, D., Felice, F., Bigelli, C., Bulzonetti, N., Guarnaccia, R., Tombolini, M., et al. (2014) An Analysis of the Global and Partial Voice Handicap Index in Patients with Early Glottic Carcinoma Treated with High Level of Irradiation. Journal of Cancer Therapy, 5, 21-27. https://doi.org/10.4236/jct.2014.51003Lehr, R. and Pong, A. (2003) ROC Curve. In: Encyclopedia of Biopharmaceutical Statistics, Informa Healthcare, 884-891. https://doi.org/10.1201/b14760-122Morrison, A.M. (2005) Receiver Operating Characteristic (ROC) Curve Preparation—A Tutorial. Massachusetts Water Resources Authority, Environmental Quality Department.Solomon, N.P., Helou, L.B., Henry, L.R., Howard, R.S., Coppit, G., Shaha, A.R. and Stojadinovic, A. (2013) Utility of the Voice Handicap Index as an Indicator of Postthyroidectomy Voice Dysfunction. Journal of Voice, 27, 348-354.https://doi.org/10.1016/j.jvoice.2012.10.012Tafiadis, D., Kosma, E., Chronopoulos, S., Papadopoulos, A., Drosos, K., Siafaka, V., et al. (2018) Voice Handicap Index and Interpretation of the Cutoff Points Using Receiver Operating Characteristic Curve as Screening for Young Adult Female Smokers. Journal of Voice, 32, 64-69. https://doi.org/10.1016/j.jvoice.2017.03.009Tafiadis, D., Chronopoulos, S.K., Kosma, E.I., Voniati, L., Raptis, V., Siafaka, V. and Ziavra, N. (2017) Using Receiver Operating Characteristic Curve to Define the Cutoff Points of Voice Handicap Index Applied to Young Adult Male Smokers. Journal of Voice, in press. https://doi.org/10.1016/j.jvoice.2017.06.007American Speech Hearing Association. Voice Evaluation Template. https://www.asha.org/uploadedFiles/slp/healthcare/AATVoiceEvaluation.pdfGräβel, E., Hoppe, U. and Rosanowski, F. (2007) Graduierung des Voice-Handicap-Index. HNO, 56, 1221-1228. https://doi.org/10.1007/s00106-007-1594-9Behrman, A., Rutledge, J., Hembree, A. and Sheridan, S. (2008) Vocal Hygiene Education, Voice Production Therapy, and the Role of Patient Adherence: A Treatment Effectiveness Study in Women with Phonotrauma. Journal of Speech, Language, and Hearing Research, 51, 350-366. https://doi.org/10.1044/1092-4388(2008/026)Van Gogh, C.L., Mahieu, H.F., Kuik, D.J., Rinkel, R.N., Langendijk, J.A. and Verdonck-de Leeuw, I.M. (2007) Voice in Early Glottic Cancer Compared to Benign Voice Pathology. European Archives of Oto-Rhino-Laryngology, 264, 1033-1038. https://doi.org/10.1007/s00405-007-0313-1Ohlsson, A.C. and Dotevall, H. (2009) Voice Handicap Index in Swedish. Logopedics Phoniatrics Vocology, 34, 60-66. https://doi.org/10.1080/14015430902839185Chronopoulos, S., Kosma, E., Tafiadis, D., Dimopoulos, D., Raptis, V., Karvounis, E.C., Angelidis, P. and Kostarakis, P. (2018) Reduced Ecological Footprints of Modern Facilities Introducing the Implementation of Advanced Wireless Technologies, and Human Resources’ Benefits. Communications and Network, 10, 11-29. https://doi.org/10.4236/cn.2018.101002Toki, E.I., Plachouras, K., Tatsis, G., Chronopoulos, S.K., Tafiadis, D., Ziavra, N. and Siafaka, V. (2017) The Design of a Mobile System for Voice e-Assessment and Vocal Hygiene e-Training. In: Auer, M. and Tsiatsos, T., Eds., Advances in Intelligent Systems and Computing Book Series (AISC), Vol. 725, Springer, Cham, 167-174.Raptis, V., Tatsis, G., Chronopoulos, S.K., Mallios, S. and Kostarakis, P. (2013) Development and Experimental Measurements of a Tunable Antenna. Communications and Network, 5, 220-224. https://doi.org/10.4236/cn.2013.53026Chronopoulos, S.K., Christofilakis, V., Tatsis, G. and Kostarakis, P. (2016) Preliminary BER Study of a TC-OFDM System Operating under Noisy Conditions. Journal of Engineering Science and Technology Review, 9, 13-16.Chronopoulos, S.K., Christofilakis, V., Tatsis, G. and Kostarakis, P. (2015) Performance of Turbo Coded OFDM under the Presence of Various Noise Types. Wireless Personal Communications, 87, 1319-1336. https://doi.org/10.1007/s11277-015-3055-1