BACKGROUND: Mental disorders are often a cause of long-term sickness absence (LTSA). Kessler 6 (K6) and the Brief Job Stress Questionnaire (BJSQ) are short mental health assessment questionnaires and widely used for assessing mental health in the workplace. OBJECTIVE: The purpose of this study is to evaluate the screening performance of these tools for the prediction of LTSA due to mental disorders (LTSA-MD). METHODS: K6 and BJSQ were administered to 19,321 public servants, of whom 131 took LTSA-MD during the study period. A receiver operating characteristic curve was drawn to estimate the area under the curve (AUC), sensitivity, and specificity with optimal cut-off points for K6 and four BJSQ subscales (Job Stressors; Stress Responses; Social Support; and Job Stressors + Social Support). RESULTS: K6 showed the highest AUC, and K6 and the Stress Responses subscale of the BJSQ showed higher AUC and sensitivity than other BJSQ subscales. CONCLUSIONS: The psychological distress experienced at the time,
i.e. K6 and the Stress Responses, was the best predictors of LTSA-MD. Using these instant effective tools will enable us to identify high-risk workers, employ suitable interventions to improve mental health at an early stage, and reduce the incidence of LTSA-MD.
Mental illness is a major global burden, accounting for 14% of all diseases [1] . In Japan, psychiatric and behavioral morbidity was 203/100,000 in 2014 [2] , and there is a continuous increasing trend [3] . Over 60% of workers in Japan experience intense worry or stress related to employment [3] , and mental health problems are growing in 37.6% of enterprises [4] . Therefore, mental health care for employees and measures to address these trends are important issues in the workplace.
Mental illness is often disabling, leading to decreased productivity at work and long- term sickness absence (LTSA). Mental disorders are the cause of 40% of all sickness absence in the United Kingdom [5] , and 41% of sickness absence among men and 30% among women in Sweden [6] . In Japan, sickness absence due to mental disorders rose from 34% in 2001 to 63% in 2006 among national public servants [7] . In 2013, the Japanese Ministry of Health, Labour and Welfare reported that in about 10% of companies, some employees took more than 1 month of LTSA, or retirement, because of mental disorders in the past year [8] . The cost of productivity loss attributable to LTSA is enormous, and it is necessary to predict LTSA as early as possible.
To minimize the social and individual effects of sickness absence due to mental disorders, early detection of mental disorders is essential [9] [10] [11] . Many self-admi- nistered questionnaires have been developed for screening mental disorders in the general population [12] [13] [14] [15] [16] . The Brief Job Stress Questionnaire (BJSQ) was developed by a research group commissioned by the Japan Ministry of Health, Labour and Welfare [17] . BJSQ is a simplified questionnaire for employees with 57 items relating to job stress. Additionally, from December 2015, an amendment to the Industrial Safety and Health Act required companies regularly employing 50 workers or more to offer an annual “Stress Check Program” using the BJSQ in Japan. This amendment aims to prevent work-related mental illness by helping employees identify of job stressors and their impact, and helping employers improve the work environment through collective analysis of BJSQ data. The BJSQ has three subscales: Job Stressors; Stress Response; Social Support. These are based on the National Institute for Occupational Safety and Health (NIOSH) stress model, and are checked in the screening for highly stressed workers in the Stress Check Program.
Very short screening questionnaires, such as the Kessler 6 (K6), which contains just six questions on depression and anxiety, are increasingly used in community settings [18] . The self- or interviewer-administered K6 questionnaire was developed to detect general psychological distress [18] . Because of its brevity (six items to be completed within 2 - 3 min), K6 has a great advantage over other well-known scales.
Previous reports have demonstrated that K6 and BJSQ can effectively detect major depression [19] [20] . However, few studies to date have examined the ability of these brief screening questionnaires to predict LTSA-MD. We hypothesized that K6 and BJSQ can instantly predict LTSA-MD. The aim of this study was to evaluate the screening performance of the K6 and BJSQ subscales for LTSA-MD.
2. Materials and Methods2.1. Subjects
This was a retrospective cohort study that examined the screening performance of K6 and BJSQ subscales in association with LTSA in Japanese workers. City “A” of the Kinki region in Japan administered mental health check by K6 and BJSQ [21] [22] to strengthen the mental health of public servants once a year. In the present study, 21,369 public servants belonging to the municipal office or the ward office in city “A” participated the mental health check in 2011. We got the date of mental health check with the corporation of city “A”. After excluding 2048 participants because they didn’t complete questionnaires, a total of 19,321 public servants who completed the questionnaires were analyzed. No participant was taking LTSA at the time of completion of the questionnaires. We then obtained lists of workers on LTSA-MD between 2011 and 2012. LTSA was defined as sickness absence lasting more than 90 days and confirmed by medical certification. The International Classification of Diseases, Tenth Revision (ICD-10) codes [23] were used to diagnose and classify mental disorders (F code) that resulted in LTSA.
2.2. Ethics Statement
The Human Subjects Review Committee at Osaka City University approved the protocol of this study (authorization number: 2969). No written informed consent was required because the data already existed. The health care center of City A collected the K6 and BJSQ data of workers during an annual mental health check to evaluate and improve the psychological work environment. All participant data were anonymized and de-identified before we obtained them under encrypted IDs.
2.3. Demographic Variables
The demographic variables were sex, age, job title (manager/non-manager), and job category (clerical/technical/professional/other).
2.4. Kessler Psychological Distress Scale 6 (K6)
K6 data of workers in City A was obtained at the annual health check. K6 consisted of six items measured on a 5-point scale (0 - 4). The total score (0 - 24) is an indicator of severity of mental illness [24] and mood and anxiety disorders [25] , with higher scores indicating greater severity. K6 was translated into Japanese, and its validity to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) mood and anxiety disorders was confirmed by a lay interviewer-administered diagnostic interview in the community sample [25] . The reliability and validity of the original versions have been evaluated repeatedly in the USA and Australia [24] [25] . The efficacy of the K6 scale was also reported in the detection of major depression and dysthymia according to the DSM-IV [26] . The screening performance of K6 scales in detecting 30- day DSM-IV mood and anxiety disorders was excellent, with area under the receiver operating characteristic curve (AUC) values as high as 0.94 for K6 [27] .
2.5. Brief Job Stress Questionnaire
The BJSQ was derived from questions in the Job Content Questionnaire and the NIOSH generic job stress psychometric instrument [28] . The BJSQ is a reliable and valid measure of mental health in Japanese workers [21] [22] [29] . The questionnaire contains 57 items with 4 point Likert-type responses ranging from “agree” (4) to “disagree” (1) and comprises four subscales: Job Stressors (17 items with a total score of 17 - 68), Stress Responses (29 items; 29 - 116), Social Support (9 items; 9 - 36), and Work and Life Satisfaction (2 items; not used in the present study). Higher scores indicate greater stress, and reverse scoring was used where necessary.
For Job Stressors, we obtained scores of quantitative job overload (3 items), qualitative job overload (3 items), physical demands (1 item), job control (3 items), skill underutilization (1 item), interpersonal conflict (3 items), poor physical environment (1 item), job suitability (1 item) and job reward (1 item). For Stress Responses, scores were obtained for vigor (3 items), hostility (3 items), fatigue (3 items), insecurity (3 items), depression (6 items), and physical stress (11 items). The Social Support subscale comprised supervisor support (3 items), coworker support (3 items), and family support (3 items).
In the Stress Check Program, workers were identified as possible “high-stress” subjects based on BJSQ scores, and were recommended to receive an interview by a doctor if one of the following two criteria were met: A, high stress reactivity (i.e. Stress Responses score > 76), or B, moderate stress reactivity but high job stress or low social support in the workplace (i.e. Stress Responses score > 62, and Job Stressors + Social Support score > 75) [30] . We therefore used three BJSQ subscale scores, plus the sum of two (Job Stressors + Social Support) to screen LTSA in the present study.
In addition, we calculated the sensitivity and specificity of the Stress Check Program criteria (A and B) for the prediction of LTSA-MD.
2.6. Statistical Analysis
Cronbach’s alpha coefficient was calculated for K6 and BJSQ, and the distribution of the scores was obtained for the workers in City A. Receiver operating characteristic (ROC) curves were drawn for each scale to differentiate workers who were on LTSA from those who were not, by calculating the AUC. The AUC indicates the probability that the mental health symptom score correctly identifies a LTSA worker, and is a measure of the discriminative ability of the K6 and BJSQ subscales. Assuming AUC = 0.50 represents no discrimination above chance, AUC ≥ 0.90 represents excellent discriminative ability, 0.80 - 0.89 good, 0.70 - 0.79 fair, and 0.60 - 0.69 represents poor discriminative ability. AUC < 0.60 indicates insufficient discrimination [31] . The optimal cut-off points of K6 and BJSQ for LTSA were calculated using the Youden method [32] , and the “shortest distance to upper left corner” method was used to select a cut-off point that would yield a position closest to the upper left corner on the ROC [33] . Sensitivity and specificity were compared across the scales based on the optimal cut-off points. All analyses were conducted using IBM SPSS version 22 (IBM, USA) and JMP 8 (SAS, USA).
3. Results3.1. Participant Characteristics
Table 1 shows the subject characteristics. Of the 19,321 participants, 13,479 (69.8%)
Participant characteristics
Total
Male
Female
Number of workers
19321
13479
5842
Age
18 - 29
1721 (8.9%)
773
948
30 - 39
5734 (29.7%)
3984
1750
40 - 49
6161 (31.9%)
4424
1737
50 - 59
4731 (24.5%)
3459
1272
≥60
974 (5.0%)
856
118
Job title
Non-manager
12781 (66.2%)
9055
3726
Manager
6540 (33.8%)
4424
2116
Job category
Clerical worker
8932 (46.2%)
6804
2128
Technical worker
5730 (29.7%)
5205
525
Professional worker
3125 (16.2%)
2195
930
Other categories
1534 (7.9%)
540
994
were male and 5842 (30.2%) female. The mean age ± standard deviation was 43.3 ± 10.0 years for the whole sample, 44.3 ± 9.8 for males and 41.0 ± 10.2 for females. There were 131 workers who had taken LTSA-MD (0.68%) in 2011-2012.
3.2. Characteristics of LTSA-MD
The most frequent ICD-10 diagnostic code of workers who had taken LTSA was F3: Mood disorders; this accounted for 97 workers (74.0%). The second most frequent diagnostic code was F4: Stress-related and somatoform disorders (31 workers; 23.7%), followed by F1: Mental and behavioral disorders due to psychoactive substance use (two workers; 1.5%) and F5: Eating disorders (one worker; 0.8%).
3.3. K6 and BJSQ Scores
Questionnaire scores of workers on LTSA were higher (representing greater stress) than those of workers not on LTSA (Table 2). Cronbach’s alpha coefficient was 0.90 for K6 and 0.84 for BJSQ.
3.4. ROC Curves and AUC
Figure 1 shows the ROC curves representing K6 and the subscales in BJSQ; Table 3 shows their AUC, optimal cut-off point, sensitivity and specificity. The “shortest distance to upper left corner” method was used to identify the optimal cut-off point for each scale. In K6, the areas under the ROC curves for LTSA-MD was 0.72 (95% CI: 0.68 - 0.77); in the BJSQ subscales, the areas under the ROC curves of Stress Response, Job
Kessler 6 and Brief Job Stress Questionnaire subscales scores with workers on long-term sickness absence and without long-term sickness absence
Sickness absence (+)
Sickness absence (−)
K6
9.1 ± 7.0
4.2 ± 4.6
BJSQ
Job stressors
45.0 ± 7.7
41.9 ± 6.5
Stress responses
68.8 ± 18.9
56.6 ± 14.6
Social supports
20.7 ± 5.8
19.2 ± 5.0
Job stressors + Social supports
65.8 ± 11.6
61.1 ± 9.4
Data are expressed as mean ± standard deviation; K6: Kessler 6, BJSQ: Brief Job Stress Questionnaire; Sickness absence (+) means the workers on long-term (>90 days) sickness absence due to mental disorders during 2011 and 2012; Sickness absence (−) means the workers without long-term sickness absence due to mental disorders during 2011 and 2012.
Area under the curve, cut-off points, the sensitivity and the specificity of Kessler 6 and Brief Job Stress Questionnaire subscales
AUC
Cut-off point
Sensitivity
1-Specificity
K6
0.72
6/7
56.5%
24.6%
BJSQ
Job stressors
0.62
46/47
45.0%
22.8%
Stress responses
0.69
61/62
63.4%
32.9%
Social supports
0.58
24/25
27.5%
13.8%
Job stressors + Social supports
0.63
63/64
61.1%
38.4%
AUC: Area under the curve. K6: Kessler 6, BJSQ: Brief Job Stress Questionnaire.
The receiver operating characteristic (ROC) curve for K6 and Brief Job Stress Questionnaire subscales differentiating workers with sick leave from those without. The receiver operating characteristic (ROC) curve for K6 (bold solid line), Job Stressors (fine solid line), Stress Responses (bold dotted line) and Social Support (fine dotted line) differentiating workers with sick leave from those without. K6: Kessler 6; Job Stressors, Stress Responses, Social Support: subscale scores in the Brief Job Stress Questionnaire
Stressors, Social Support, and Job Stressors + Social Support for LTSA-MD were 0.69 (95% CI: 0.64 - 0.74), 0.62 (0.57 - 0.68), 0.58 (0.53 - 0.63), and 0.63 (0.58 - 0.68), respectively. AUC for combining Stress Responses and Social Support showed no advantage over each AUC alone. K6 showed the greatest AUC for screening sickness absence, followed by Stress Responses, Job Stressors + Social Support, Job Stressors, and Social Support. The AUC for the “Job stressors + Social support” subscale was smaller than that for K6.
3.5. Applying Stress Check Program Criteria
Criterion A was satisfied by 1992 workers (10.3%), criterion B by 1,066 workers (5.5%), and both by 671 workers (3.5%). The Stress Check Program identified 2387 possible high-stress workers (12.3%), of whom 47 (0.3%) had taken LTSA-MD. The sensitivity of the criteria used in the Stress Check Program was 35.9%, and 1-specificity was 12.2%.
4. Discussion
The aim of the present study was to evaluate the screening performance of K6 and BJSQ for LTSA-MD. We used a large sample of 19,321 public servants to explore this question. Our study identified the most reliable predictors of LTSA-MD incidence from these brief screening questionnaires. Our data show that, of the questionnaires investigated, K6 and the Stress Response subscale of the BJSQ are the best predictors of LTSA- MD.
The number of workers taking LTSA-MD was 131 (0.68%) in the present study. In Japan, the prevalence of LTSA-MD among public servants is 0.97% [34] , and in Sweden, the annual cumulative sickness absence accounts for 1.7% of the employed population [35] . The incidence of sickness absence in the present study is lower than that reported elsewhere. This may be because, despite the large number of studies investigating LTSA, there is no international consensus on its definition. Here, we defined LTSA as taking leave from work for more than 90 days; however, many other studies define LTSA as a period of absence shorter than 60 days. It may be that workers on sick leave in our study are more severely affected than those in other studies, and require a longer period of absence.
The optimal cut-off point for K6 in the present study was 6/7. In previous screening studies, the optimal cut off point was 4/5 for mood/anxiety disorders in the general population [20] and 12/13 for severe mental illness [24] ; thus, the K6 cut-off point in the present study was between the results obtained in previous studies. In addition to the possibility that the severity of mental disorders may have been exceptionally high in our study, it is also possible that public servants in Japan take sickness absence before their mental illness becomes severe. Public service roles in Japan are considered very stable, and the system of taking sickness absence is easier to use than in the private sector.
Of the questionnaires examined, the highest AUC, indicating the best screening performance for LTSA-MD, was observed in K6 (0.72). The Stress Responses subset of the BJSQ was the second highest (AUC = 0.69), followed by Job Stressors and Social Support. Combination of Job Stressors + Social Support did not result in a notable improvement in either individual AUC. With the optimal cut-off points, the sensitivity was 56.5% for K6 and the specificity was 75.4%. Sensitivity and specificity were comparable between K6, Stress Responses, and Job Stressors + Social Support. K6 may be easier to complete than the BJSQ because it has far fewer items.
K6 and the Stress Responses subscale of the BJSQ were the best predictors of LTSA- MD. In the NIOSH stress model (Figure 2), job stressors reflecting work conditions lead to acute reactions, or strains, in workers [36] . These acute reactions represent more or less temporary affective, physiological, and behavioral responses. Such short-term strains are supposed to effect on longer-term indicators of mental and physical health. Acute reactions lead to sickness absence. This model includes three moderators for job stressor: individual factors, non-work factors, and buffer factors. These factors cause differences in the way workers exposed to the same job stressors perceive and/or react to them. In the present study, K6 and Stress Responses, which reflect the psychological distress at the time, showed the highest AUC for LTSA-MD detection, and correspond to the acute reaction of the NIOSH stress model. In addition, the acute reactions resulted in sickness absence. LTSA corresponds to illness in the NIOSH stress model, and Job Stressors and Social Support correspond to moderators of this model. Our results indicate that, in the prediction of LTSA-MD, assessments using the K6 and Stress Responses questionnaires are more reliable than using data from the Job Stressors or Social Support subscales, because K6 and Stress Responses are closer to illness in the NIOSH stress model, and LTSA is the final result.
The sensitivity of BJSQ according to the cut-off point in the Stress Check Program was lower than that of K6. This may be because our study focuses on workers taking LTSA, whereas the Stress Check Program focuses on possible high-stress workers and includes those with lower stress to enable primary prevention. Highly stressed workers tend to take sickness absence [6] , and this might progress to LTSA-MD. LTSA-MD can
Model of job stress and health by the National Institute for Occupational Safety and Health (based on Hurrell Jr. J.J., McLaney M. A. (1988) Exposure to job stress―a new psychometric instrument. Scand J Work Environ Health, 14, Suppl 1, 27-28). K6: Kessler 6; BJSQ: Brief Job Stress Questionnaire
be predicted as 35.9% from the highly stressed workers identified by the Stress Check Program criteria, or as 56.5% from K6 with a cut-off of 6/7. In terms of preventing LTSA-MD, our data indicate that assessing workers by the K6 or Stress Responses scales is more effective than other methods. The method of calculation in the Stress Check Program is simply the addition of the Job Stressor and Stress Response scores, subtracting the Social Support score as a buffer. In addition, each question is considered to be of similar importance, meaning that the subscales with more questions, such as Job Stressor, have a greater weighting. In reality, job stressors have various weights, and buffer factors may work not only in a subtractive, but also a divisive manner. However, the purpose of the Stress Check Program was primary prevention. Therefore, it is unlikely that the whole BJSQ questionnaire would be more suitable for the prediction of LTSA-MD in the present study. Because K6 and Stress Responses assess similar criteria in workers, it is rare that both questionnaires are used together. So, in the Stress Check Program, focusing on Stress Responses in the BJSQ might enable us to manage LTSA in the early stages.
This study has several limitations. First, the data were obtained from public servants in one city in Japan, which may make it difficult to generalize the findings to other job categories, and other regions or countries. Second, we did not consider the time between completion of the questionnaires and the start of sickness absence. Third, we studied only two of several short screening questionnaires examining mental disorders; future studies should address other questionnaires. Fourth, there may be a sex difference in the screening performance of K6 and BJSQ; we were not able to investigate this effect in the present study. Finally, all data were collected by self-reporting, meaning that the results may be influenced by personality differences or response tendencies.
K6 and BJSQ, in particular the Stress Responses subscale, which reflects psychological distress at the time, can effectively predict LTSA-MD in workers, and enable us to provide support for mental disorders at an early stage. Health care staff can monitor workers’ responses to the K6 or Stress Responses questionnaires and support them by providing information about visiting clinics or risk of LTSA-MD, or their supervisors can adjust the environment around them to relieve their stressors. We believe that as a result, the number of workers with sickness absence will be decreased by these brief screening questionnaires and consequent early intervention.
Acknowledgements
We would like to thank Dr. Kyoko Izumotani for her helpful comments on the data analysis.
Conflicts of Interest
The authors declare that they have no conflict of interest.
Cite this paper
Ishimoto, H., Iwasaki, S. and Inoue, K. (2016) Brief Screening Tools for Long-Term Sickness Absence Due to Mental Disorders in Public Servants. Health, 8, 1453-1464. http://dx.doi.org/10.4236/health.2016.814145
ReferencesMurray, C.J.L. and Lopez, A.D. (1996) A Comprehensive Assessment of Mortality and Disability from Diseases, Injuries, and Risk Factors in 1990 and Projected to 2020. Harvard University Press, Cambridge, MA.Ministry of Health Labour and Welfare. Patient Survey 2014.
http://www.mhlw.go.jp/english/database/db-hss/dl/sps_2014_02.pdfMinistry of Health Labour and Welfare. Annual Health, Labour and Welfare Report 2009-2010 Section 7. http://www.mhlw.go.jp/english/wp/wp-hw4/dl/honbun/2_2_7.pdfJapan Productivity Center. Sixth Questionnaire about the Action for Mental Health Problems among Japanese Enterprises. (In Japanese)
http://activity.jpc-net.jp/detail/mhr/activity001359.htmlShiels, C., Gabbay, M.B. and Ford, F.M. (2004) Patient Factors Associated with Duration of Certified Sickness Absence and Transition to Long-Term Incapacity. British Journal of General Practice, 54, 86-91.Mather, L., Bergstrom, G., Blom, V. and Svedberg, P. (2015) High Job Demands, Job Strain, and Iso-Strain are Risk Factors for Sick Leave due to Mental Disorders: A Prospective Swedish Twin Study With a 5-Year Follow-Up. Journal of Occupational and Environmental Medicine, 57, 858-865. http://dx.doi.org/10.1097/JOM.0000000000000504The National Personnel Authority. The Survey of National Public Servants Who Take Sick Leave in 2011. (In Japanese) http://www.jinji.go.jp/kisya/1303/23tyouki.pdfMinistry of Health Labour and Welfare. Survey on State of Employees’ Health 2012. (In Japanese) http://www.mhlw.go.jp/toukei/list/dl/h25-46-50_01.pdfSakamoto, S., Tanaka, E., Neichi, K. and Ono, Y. (2004) Where Is Help Sought for Depression or Suicidal Ideation in an Elderly Population Living in a Rural Area of Japan? Psychiatry and Clinical Neurosciences, 58, 522-530.
http://dx.doi.org/10.1111/j.1440-1819.2004.01295.xOno, Y. (2004) Suicide Prevention Program for the Elderly: The Experience in Japan. The Keio Journal of Medicine, 53, 1-6. http://dx.doi.org/10.2302/kjm.53.1Chida, F., Okayama, A., Nishi, N. and Sakai, A. (2004) Factor Analysis of Zung Scale Scores in a Japanese General Population. Psychiatry and Clinical Neurosciences, 58, 420-426.
http://dx.doi.org/10.1111/j.1440-1819.2004.01277.xGoldberg, D. and Williams, P. (1988) A User’s Guide to the General Health Questionnaire. Nfer-Nelson, Berkshire.Zigmond, A.S. and Snaith, R.P. (1983) The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica, 67, 361-370.
http://dx.doi.org/10.1111/j.1600-0447.1983.tb09716.xWeissman, M.M., Sholomskas, D., Pottenger, M., Prusoff, B.A. and Locke, B.Z. (1977) Assessing Depressive Symptoms in Five Psychiatric Populations: A Validation Study. American Journal of Epidemiology, 106, 203-214.Zung, W.W. (1965) A Self-Rating Depression Scale. Archives of General Psychiatry, 12, 63-70. http://dx.doi.org/10.1001/archpsyc.1965.01720310065008Langner, T.S. (1962) A Twenty-Two Item Screening Score of Psychiatric Symptoms Indicating Impairment. Journal of Health and Human Behavior, 3, 269-276.
http://dx.doi.org/10.2307/2948599Shimomitsu, T., Ohno, H., Maruta, T. and Tanigawa, T. (2000) Investigation Reserch Report Concerning Prevention of Disease Related to Work in 1997 the Minisitory of Labor: III Stress Measurement Research Group Report, 101-169.Kessler, R.C., Andrews, G., Colpe, L.J., Hiripi, E., Mroczek, D.K., Normand, S.L., et al. (2002) Short Screening Scales to Monitor Population Prevalence and Trends in Non-Specific Psychological Distress. Psychological Medicine, 32, 959-976.
http://dx.doi.org/10.1017/S0033291702006074Wada, K., Sairenchi, T., Haruyama, Y., Taneichi, H., Ishikawa, Y. and Muto, T. (2013) Relationship between the Onset of Depression and Stress Response Measured by the Brief Job Stress Questionnaire among Japanese Employees: A Cohort Study. PLoS ONE, 8, e56319.
http://dx.doi.org/10.1371/journal.pone.0056319Sakurai, K., Nishi, A., Kondo, K., Yanagida, K. and Kawakami, N. (2011) Screening Performance of K6/K10 and Other Screening Instruments for Mood and Anxiety Disorders in Japan. Psychiatry and Clinical Neurosciences, 65, 434-441.
http://dx.doi.org/10.1111/j.1440-1819.2011.02236.xShimomitsu, T., Haratani, T., Nakamura, K., Kawakami, N.H., Hiro, H., Arai, M., et al. (2000) Final Development of the Brief Job Stress Questionnaire Mainly Used for Assessment of the Individuals. Tokyo Medical University, Tokyo.Kawakami, N., Kobayashi, F., Araki, S., Haratani, T. and Furui, H. (1995) Assessment of Job Stress Dimensions Based on the Job Demands-Control Model of Employees of Telecommunication and Electric Power Companies in Japan: Reliability and Validity of the Japanese Version of the Job Content Questionnaire. International Journal of Behavioral Medicine, 2, 358-375. http://dx.doi.org/10.1207/s15327558ijbm0204_5World Health Organization (1993) ICD-10: International Statistical Classification of Diseases and Related Health Problems. 10th Revision, Geneva.Kessler, R.C., Barker, P.R., Colpe, L.J., Epstein, J.F., Gfroerer, J.C., Hiripi, E., et al. (2003) Screening for Serious Mental Illness in the General Population. Archives of General Psychiatry, 60, 184-189. http://dx.doi.org/10.1001/archpsyc.60.2.184Furukawa, T.A., Kessler, R.C., Slade, T. and Andrews, G. (2003) The Performance of the K6 and K10 Screening Scales for Psychological Distress in the Australian National Survey of Mental Health and Well-Being. Psychological Medicine, 33, 357-362.
http://dx.doi.org/10.1017/S0033291702006700American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorder. 4th Edition, Washington DC.Furukawa, T.A., Kawakami, N., Saitoh, M., Ono, Y., Nakane, Y., Nakamura, Y., et al. (2008) The Performance of the Japanese Version of the K6 and K10 in the World Mental Health Survey Japan. International Journal of Methods in Psychiatric Research, 17, 152-158.
http://dx.doi.org/10.1002/mpr.257Harada, H., Suwazono, Y., Sakata, K., Okubo, Y., Oishi, M., Uetani, M., et al. (2005) Three-Shift System Increases Job-Related Stress in Japanese Workers. Journal of Occupational Health, 47, 397-404. http://dx.doi.org/10.1539/joh.47.397Endo, M., Muto, T., Haruyama, Y., Yuhara, M., Sairenchi, T. and Kato, R. (2015) Risk Factors of Recurrent Sickness Absence Due to Depression: A Two-Year Cohort Study among Japanese Employees. International Archives of Occupational and Environmental Health, 88, 75-83. http://dx.doi.org/10.1007/s00420-014-0939-4Kawakami, N. and Tsutsumi, A. (2016) The Stress Check Program: A New National Policy for Monitoring and Screening Psychosocial Stress in the Workplace in Japan. Journal of Occupational Health, 58, 1-6. http://dx.doi.org/10.1539/joh.15-0001-ERFan, J., Upadhye, S. and Worster, A. (2006) Understanding Receiver Operating Characteristic (ROC) Curves. Canadian Journal of Emergency Medicine, 8, 19-20.Youden, W. (1950) An Index for Rating Diagnostic Tests. Cancer, 3, 32-35.
http://dx.doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3Holmes, W.C. (1998) A Short, Psychiatric, Case-Finding Measure for HIV Seropositive Outpatients: Performance Characteristics of the 5-Item Mental Health Subscale of the SF-20 in a Male, Seropositive Sample. Medical Care, 36, 237-243.
http://dx.doi.org/10.1097/00005650-199802000-00012The National Personnel Authority (2006) The Survey of National Public Servants Who Take Sick Leave in 2006. (In Japansese)
http://www.js-mental.org/images/11/data2008jinji.pdfHensing, G., Alexanderson, K., Akerlind, I. and Bjurulf, P. (1995) Sick-Leave Due to Minor Psychiatric Morbidity: Role of Sex Integration. Social Psychiatry and Psychiatric Epidemiology, 30, 39-43. http://dx.doi.org/10.1007/BF00784433Hurrell Jr., J.J. and McLaney, M.A. (1988) Exposure to Job Stress—A New Psychometric Instrument. Scandinavian Journal of Work, Environment & Health, 14, 27-28.