Logo-aim
Arch Iran Med. 24(11):811-821. doi: 10.34172/aim.2021.121

Original Article

Psychological and Psychosocial Impact and Related Factors during the COVID-19 Pandemic among Iranian Oil Refineries Personnel: A longitudinal study

Ahmad Soltanzadeh 1 ORCID logo, Mohammad Ghasemi 2 ORCID logo, Mansoureh Sadeghi-Yarandi 3 ORCID logo, Hamedeh Golmohammadpour 4, Maedeh Maddahi 5, Sepehr Sadeghi-Yarandi 6 ORCID logo, Saba Kalantary 4, Mohsen Sadeghi-Yarandi 4, * ORCID logo

Author information:
1Department of Occupational Safety and Health Engineering, Health Faculty, Qom University of Medical Sciences, Qom, Iran
2Health Research Center, Lifestyle Institute, Baqiyatallah University of Medical Sciences, Tehran, Iran
3Department of Counseling, Faculty of Humanities, Science and Research Branch, Islamic Azad University, Tehran, Iran
4Department of Occupational Health Engineering, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
5Department of Environment, University of Applied Sciences and Technology, Karaj, Iran
6Department of Psychology, York University, Toronto, Canada

*Corresponding Author: Mohsen Sadeghi-Yarandi, MSc; Department of Occupational Health Engineering, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran. Tel: +98912-087-28-98, Fax: +982188954781, Email: M-Sadeghiy@alumnus.tums.ac.ir

Abstract

Background:

The COVID-19 outbreak is currently a critical challenge, and proper perception of the mental health situation of individuals in the working environments has become a crucial issue. This study aimed to investigate the psychological and psychosocial impact during the COVID-19 pandemic among Iranian oil refineries personnel.

Methods:

This longitudinal study was conducted in two phases from November 2019 to July 2020 among the 850 employees of three refineries in Iran. To study the mental workload, job burnout, lifestyle, occupational stress, general health, and memory quotient of personnel, the NASA-TLX questionnaire, Maslach Burnout Inventory (MBI) questionnaire, Walker’s health-promoting lifestyle questionnaire, health and safety executive (HSE) job stress questionnaire, general health questionnaire, and Wechsler memory scale were used, respectively. Data were analyzed using paired sample t test, independent sample t test, and one-way ANOVA in SPSS version 25.

Results:

The study results revealed that the mean age of studied personnel was 34.62±13.48 years. There was a meaningful correlation between the values of mental workload, job burnout, occupational stress, general health, healthy lifestyle, and memory quotient of personnel before and during the COVID-19 pandemic (P value<0.05).

Conclusion:

The COVID-19 pandemic can significantly affect the psychological and psychosocial parameters of employees in working environments. Therefore, implementing psychological interventions is absolutely necessary to promote personnel’s mental health during the COVID-19 pandemic.

Keywords: COVID-19, Oil refinery, Psychological parameters, Psychosocial parameters, SARS-CoV2

Copyright and License Information

© 2021 The Author(s).
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Cite this article as: Soltanzadeh A, Ghasemi M, Sadeghi-Yarandi M, Golmahammadpour H, Maddahi M, Sadeghi-Yarandi S, et al. Psychological and psychosocial impact and related factors during the covid-19 pandemic among iranian oil refineries personnel: a longitudinal study. Arch Iran Med. 2021;24(11):811-821. doi: 10.34172/aim.2021.121


Introduction

In December 2019, a viral disease outbreak was reported in Wuhan, Hubei province, China.1

The novel coronavirus disease 2019 (nCOVID-19) is a severe acute respiratory syndrome that is closely related to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)2 Unfortunately, this virus has spread rapidly around the world due to its high infectivity and infected all countries in a short period of time (less than four months),3 as the World Health Organization announced a pandemic condition on March 11, 2020.4 COVID-19 mostly affects the respiratory system of people with the disease and is spreading catastrophically.5 Older people with underlying diseases are more prone to becoming infected with severe symptoms and consequences.6 COVID-19, similarly to SARS, is a kind of betacoronavirus that can be spread to humans via intermediate organisms like bats.7 Clinical manifestations after infection include fever, chills, painful throat, cough, trouble breathing, vomiting, and diarrhea.6,8

At the moment of writing, the global cases of COVID-19 are regularly growing all over the world. On August 13, 2020, the total number of individuals presenting with confirmed COVID-19 was 20 439 814 people, with 744 385 deaths.9 In Iran, the first official announcement of deaths from COVID-19 was made on February 19, 2020.4 According to a report released by the World Health Organization (WHO), 333 699 people in Iran have coronavirus disease as of August 20, 2020. A total of 5118 people have died due to COVID-19 so far.9

The outbreak of COVID-19 may be stressful for people. Fear and stress about a disease can be very severe and cause intense emotions in every age range. A novel study conducted in China revealed that the COVID-19 outbreak is a significant international public health concern and poses an important challenge for psychological health.10 Generally, when facing the extraordinary situation of the public health crisis, bodies are more susceptible to various psychological and mental problems.11 Underlying parameters like gender, specific somatic symptoms (like myalgia, dizziness, coryza), chronic diseases, and poor health situation were significantly associated with a broader psychological impact of the outbreak and severe stress levels, anxiety, and depression.10 Previous studies have revealed a wide variety of psychological and psychosocial influences on people at the individual, social, and global levels during outbreaks of infection.12 The COVID-19 pandemic is currently a significant concern, and a timely understanding of the mental health situation of people has become a crucial issue.10

Psychological and psychosocial parameters (like workload, occupational stress, burnout, lifestyle, functional memory, etc) stand among the most critical parameters in determining the occupational achievement of employees in workplaces.13-15 Past research has revealed that the epidemic of viral diseases negatively affects psychological parameters such as workload, irritability, sleep disorders, stress, anxiety, and depression in the workplace.16

All of those mentioned above can affect people’s job performance in work environments and cause a person’s disability in performing their occupational tasks. This situation is causing mental health problems like stress, anxiety, depressive symptoms, insomnia, fear, and anger. These mental health disorders not only affect the workers’ awareness, perception, and decision-making ability but could also exert a lasting effect on their overall health and cause many accidents in the workplace.17,18

Therefore, due to the increasing number of confirmed cases and mortality, the high impact of the COVID-19 outbreak on the psychological and psychosocial parameters of employees in the workplace and also the need to control the mentioned risk factors in the workplace as economic bases of the countries, and lack of similar studies in the working environment, the present study aimed to investigate the psychological and psychosocial impact and related factors during the COVID-19 pandemic among Iranian oil refineries personnel.


Materials and Methods

Study Design

This longitudinal study was performed in two phases from November 2019 to July 2020 among the employees of three oil refineries in southern Iran. It should be noted that the first phase of the study was carried before the COVID-19 pandemic in the form of a cross-sectional study. However, after the pandemic of COVID-19 and investigating the trend of the changes in psychological and psychosocial parameters among the participants, this research project was performed as a longitudinal study in two phases. The statistical population included all operational employees working in three oil refineries in Iran (894 employees). In the present study, all operational employees were selected as a sample. Therefore, the sample size was determined to be 894 employees and was selected using the census method. In the first phase of the study, 850 employees entered the study (response rate was 95%). In the second phase of the study, 823 employees participated in the study (response rate was 96.8%). The first phase of the study was conducted in November 2019 and before the COVID-19 outbreak in Iran. The second phase of the study was performed in July 2020 (during the COVID-19 outbreak in Iran) to investigate the effects of the COVID-19 epidemic on changes in some psychological and psychosocial parameters among the studied employees. Inclusion criteria included at least one year of work experience, and exclusion criteria were chronic mental illness, taking sedatives, and lack of consent for participation in the study. It should be noted that in order to reduce the effect of confounding variables, the most critical factors affecting psychological factors were examined. It was found that the most important risk factors affecting employees’ psychological factors such as daily workload, level of social support by colleagues and supervisors, working hours per day, work shift planning, salary and income, type of duties and workstation, and the physical condition of the work environment were constant during the study in the three studied refineries. Participants were able to leave the study if they were not willing to continue. Before beginning the research and completing the questionnaires and tests, all the required information about completing the questionnaires and how to perform the tests were explained to the personnel. The flow diagram of the study is presented in Figure 1. Demographic data were extracted using a self-administrated questionnaire, and to obtain the main parameter, the following tools were applied.

aim-24-811-g001
Figure 1.

Flow Diagram of the Study.


NASA-TLX Questionnaire

This questionnaire was used to assess mental workload. NASA-TLX is a multidimensional questionnaire that provides an overall workload based on a weighted average of six scales of mental demand, physical demand, temporal demand, effort, overall performance, and frustration level. The participant scores any of the six dimensions from zero to one hundred according to their working conditions. The worker chooses the option that is most relevant to the activity between the two questions. Each selection is equal to a weighted score for that item. By multiplying the weight of each dimension of the workload (ranging from 0‒1) to the scale score of each dimension (ranging from 0 to 100), the total workload is calculated numerically from 0 to 100. If the overall workload score is less than 50, the risk level is low, and if it is above 50, the risk level is high. The validity and reliability of this questionnaire have been approved in past studies in Iran.19-22

Maslach Burnout Inventory

We used the Maslach burnout inventory (MBI) to study job burnout, one of the most common tools for measuring burnout. MBI is designed to evaluate the three dimensions of job burnout: emotional exhaustion (nine parts), depersonalization (eight parts), and professional efficacy (four parts). The questions are answered based on the frequency with which the worker experiences these feeling, with a 7-point rating scale (ranging from 0 “never” to 6 “every day”). The validity and reliability of this questionnaire have been confirmed in prior studies in Iran.23,24

Walker’s Health-Promoting Lifestyle Questionnaire

This questionnaire was first developed by Walker and colleagues25 and consists of 52 questions that measure six dimensions of spiritual growth, health responsibility, interpersonal relationships, stress management, exercise and physical activity, and nutrition with 8 or 9 questions. Each question has 4-point Likert scale options, including never, sometimes, most of the time, and always. High scores demonstrate a better health-promoting lifestyle. A score of 52 to 104 is classified as poor lifestyle, a score of 104 to 156 is classified as an average lifestyle, and a score of 158 to 208 is classified as good lifestyle. The validity and reliability of this questionnaire have been confirmed in prior studies in Iran.26

General Health Questionnaire – 28

The 28-item general health questionnaire form was adopted in 1979 by Goldberg and Hillier.27 The General Health Questionnaire–28 (GHQ-28) has been divided into four dimensions, including somatic symptoms (parts 1–7), anxiety/insomnia (parts 8‒14), social dysfunction (parts 15‒21), and severe depression (parts 22‒28). This test generally shows a person’s mental and physical condition. The test scores are based on a four-point Likert scale, with scores varying from zero to three. Finally, the range of obtained scores is from zero to 84. Applying this method, a total score of 23/24 is the threshold for the appearance of distress. Total scores were divided into complete health, the threshold of disease, and deterioration of the disease. High scores on this test indicate a worse health situation.28 The validity and reliability of this questionnaire have been confirmed in prior studies in Iran.29,30

Wechsler Memory Scale

The Wechsler memory scale (WMS) was developed by David Wechsler in 1945. WMS is a fast, easy, and practical memory examination. This test includes seven subtests. Test 1 includes six simple questions of personal and current information. Test 2 consists of five simple questions and is planned to test the subject’s immediate orientation. Test 3 is labeled mental control and consists of three sub-items. Test 4 is named logical memory and consists of two memory passages. Test 5 involves repeating the numbers forward and backward. Test 6 is for testing visual memory, and test 7 is labeled associate learning. The total memory score is obtained from the sum of the scores of each of the subtests. According to the main form of the test, it is possible to add a fixed score to the sum of the test scores based on the age of the participants. Finally, according to the available tables, each person’s memory quotient can be obtained. The validity and reliability of this test have been confirmed in previous studies in Iran.31-33 In the present study, the test time for each individual was 10 to 15 minutes.

HSE Job Stress Questionnaire

The UK’s health and safety executive (HSE) questionnaire was used to assess employees’ job stress levels. This questionnaire consists of 35 items and seven dimensions. These seven dimensions include 1. Demands: includes issues such as workload, workplace features, 2. Control:how much a person has choice and authority in doing his or her work, 3. Support provided by management, 4. Support provided by colleagues, 5. Relationships at work, 6. Role: understanding job roles by employees, and 7. Organizational change. The answers to the questions were made on a five-point Likert scale. High scores indicate low and appropriate stress and job pressure, and low scores indicate exposure to higher risks of stressors (high levels of stress).34 The validity and reliability of this questionnaire have been confirmed in prior studies in Iran.35,36

Statistical Analysis

After data collection, the data collected through the questionnaire and test was ultimately analyzed using IBM SPSS statistical software version 25.0 for Windows (SPSS Inc Chicago, Illinois). Kolmogorov-Smirnov test and normal probability plot were applied to check the normality of data distribution. The findings revealed that the distribution of data was normal in all cases (P value > 0.05). Descriptive statistics (mean, standard deviation, median, interquartile range and, frequency) were presented. Statisticalanalysis was performed using paired sample t test [to compare the mean values (before and during COVID-19 outbreak)], independent sample t test (differences in a mean between two independent groups), and one-way ANOVA (differences in a mean between three or more independent groups) at the significance level of 0.05. Homogeneity of variance was examined by Levene’s test (P value < 0.05).


Results

The present study results revealed that the mean age, working hour per day and body mass index of studied personnel were 34.62 ± 13.48 years, 7.41 ± 1.13 hour and 24.81 ± 5.73 kg.m -2, respectively. Furthermore, 81% of the participants were male and 19% were female. The majority (71%) of the employees were married, and 16% of the population under survey had underlying disease. According to the contract type, 63%, 24% and 13% of employees were full-time, part-time and temporary, respectively. Also, 34% of staff had a history of smoking. The other demographic values are provided in Table 1.


Table 1. Demographics of Participants
Demographic variables (unit) Value
Age (Year) [Mean (SD)] 34.62 (13.48)
Gender [Percentage (Number)] Male 81 (689)
Female 19 (161)
Work experience (Years) [Median (IQR)] 13.50 (8.00)
Working hour per day (Hour) [Mean (SD)] 7.41 (1.13)
Contract [Percentage (Number)] Full-time 63 (536)
Part-time 24 (204)
Temporary 13 (110)
Sleeping hour per day (Hour) [Mean (SD)] 6.14 (2.39)
Underlying disease [Percentage (Number)] Yes 16 (136)
No 84 (714)
Taking medication [Percentage (Number)] Yes 21 (179)
No 79 (671)
Height (cm) [Mean (SD)] 170.54 (9.36)
Weight (kg) [Mean (SD)] 79.92 (11.34)
BMI (kg.m -2) [Mean (SD)] 24.81 (5.73)
BMI level [Percentage (Number)] Normal (18.5 ‒ 24.9 kg.m -2) 61 (519)
Over weight (25 ‒ 29.9 kg.m -2) 34 (289)
Obesity class 1 (30 ‒ 34.9 kg.m -2) 5 (42)
Marital status [Percentage (Number)] Single 29 (247)
Married 71 (603)
Education level [Percentage (Number)] Diploma 40 (340)
Associate Degree 16 (136)
Bachelor 31 (263)
Master and higher 13 (111)
Smoking [Percentage (Number)] Yes 34 (289)
No 66 (561)

SD, Standard Deviation; IQR, Interquartile Range; BMI, Body mass index.

Mental Workload

A study of mental workload based on NASA-TLX revealed that the average scores of mental workload before and during the COVID-19 outbreak were 58.43 ± 11.68 and 66.38 ± 13.70, respectively, and a meaningful relationship was observed between the mentioned values (P value < 0.05). A significant relationship was found between the mean scores of mental demand, overall performance, and frustration level dimensions before and during the COVID-19 outbreak (P value < 0.05). It was found that the mean scores of all mental workload dimensions, except the temporal demand and overall performance, were increased during the COVID-19 epidemic. The highest increase was seen in the values of mental demand and frustration level components (Table 2).


Table 2. Mean scores of dimensions of mental workload based on NASA-TLX index among studied employees
Dimensions Before nCOVID-19Outbreak During nCOVID-19 Outbreak 95% Confidence Intervals of the Difference P Value
Mean (SD) Mean (SD) Lower Upper
Mental demand 43.11 13.76 51.74 18.36 13.24 23.30 0.016
Physical demand 62.19 16.44 63.35 18.70 0.44 2.48 0.614
Temporal demand 54.70 17.44 52.79 19.27 -1.26 -5.13 0.215
Effort 53.61 14.84 53.94 17.47 0.12 1.09 0.300
Overall performance 54.81 15.67 48.11 16.52 -5.44 -15.36 0.007
Frustration level 43.72 18.07 52.57 19.14 10.78 24.55 0.001
Mental workload overall score 58.43 11.68 66.38 13.17 6.48 16.46 0.011

nCOVID-19, 2019 novel coronavirus disease; SD, Standard Deviation.

Job Burnout

The mean job burnout scores before and during the COVID-19 outbreak were 39.75 ± 15.61 and 46.77 ± 16.31, respectively, and a significant relationship was found between the mentioned values (P value < 0.05). A significant relationship was recognized between the mean scores of all three dimensions of job burnout (emotional exhaustion, depersonalization, professional efficacy) before and during the COVID-19 epidemic (P value < 0.05). The finding demonstrated that during the COVID-19 outbreak, the mean scores of emotional exhaustion and depersonalization dimensions were increased, and the average score of the professional efficacy dimension was decreased. The other values of job burnout are presented in Table 3.


Table 3. Mean Scores of Dimensions of Job Burnout among Studied Employees
Dimensions Before nCOVID-19 Outbreak During nCOVID-19 Outbreak 95% Confidence Intervals
of the Difference
P Value
Mean (SD) Min Max Mean (SD) Min Max Lower Upper
Emotional exhaustion 21.11 8.86 0 41 29.67 9.83 5 45 23.46 48.63 0.004
Depersonalization 8.36 6.42 0 20 12.75 8.74 0 22 25.70 50.36 0.015
Professional efficacy 28.58 10.71 8 42 23.78 8.51 6 45 8.44 19.42 0.034
Job burnout overall score 39.75 15.61 22 59 46.77 16.31 22 75 10.52 19.87 0.001

nCOVID-19, 2019 novel coronavirus disease; SD, Standard Deviation.

Lifestyle

The findings of the lifestyle survey revealed that the mean scores of lifestyle among all the participants before and during the COVID-19 outbreak were 129.16 ± 19.70 and 118.14 ± 16.07, respectively, and the COVID-19 outbreak has reduced the lifestyle mean values of individuals. A meaningful correlation was observed between the mean scores of health responsibility, stress management, and physical activity dimensions before and during the COVID-19 outbreak (P value < 0.05). The results showed that the COVID-19 outbreak increased the average scores of spiritual growth and health responsibility dimensions and reduced the mean scores of interpersonal relationships, stress management, physical activity, and nutrition dimensions (Table 4).


Table 4. Mean Scores of Dimensions of Lifestyle Among Studied Employees
Dimensions Before nCOVID-19Outbreak During nCOVID-19 Outbreak 95% Confidence Intervals of the Difference P Value
Mean (SD) Mean (SD) Lower Upper
Spiritual growth 29.73 (3.16) 30.66 (4.82) 0.98 5.36 0.903
Health responsibility 30.28 (4.30) 38.14 (6.74) 15.58 31.67 0.001
Interpersonal relationships 21.16 (3.25) 19.88 (2.91) -2.48 -8.59 0.106
Stress management 15.12 (3.14) 10.41 (2.39) -21.77 -33.68 0.026
Physical activity 17.32 (4.13) 12.58 (3.17) -18.58 -28.31 0.015
Nutrition 18.13 (4.74) 17.65 (3.81) -1.55 -3.11 0.211
Lifestyle overall score 129.16 (19.70) 118.14 (16.07) -3.28 -8.63 0.031

SD, Standard Deviation; nCOVID-19, 2019 novel coronavirus disease.

General Health and Wechsler Memory Scale

The results showed that the employee’s general health status was affected by the COVID-19 outbreak. The mean scores of general health among all subjects before and during the COVID-19 outbreak were 45.13 ± 11.65 and 51.41 ± 12.89, respectively, and there was a meaningful relationship between the mentioned values (P value < 0.05). Moreover, a significant relationship was found between the mean scores of anxiety/insomnia and severe depression components before and during the COVID-19 epidemic (P value < 0.05). The findings revealed that the COVID-19 outbreak had a minor effect on the somatic symptoms dimension and the most significant effect was on anxiety/insomnia and severe depression dimensions among studied personnel.

The finding of the functional working memory test of individuals using the Wechsler memory scale demonstrated that the average scores of memory quotient of personnel before and during the COVID-19 epidemic were 85.36 ± 15.19 and 80.64 ± 13.39, respectively and the COVID-19 outbreak has significantly reduced worker’s memory quotient (P value < 0.05). The other values related to general health components and the Wechsler memory scale are shown in Figure 2.

aim-24-811-g002
Figure 2.

Average Scores of Different Dimensions of General Health and Wechsler Memory Scale (before nCOVID-19 Pandemic vs. during nCOVID-19 Pandemic)


Occupational Stress

Examination of occupational stress values using the UK’s HSE questionnaire revealed that the mean scores of occupational stress before and during the COVID-19 outbreak were 81.83 ± 18.37 and 69.94 ± 14.45, respectively, and the COVID-19 pandemic significantly increased the levels of worker’s occupational stress (P value < 0.05) (Table 5).


Table 5. Mean Score of Occupational Stress among Studied Workers
Variable Before nCOVID-19Outbreak During nCOVID-19Outbreak 95% Confidence Intervals of the Difference P Value
Mean (SD) Mean (SD) Lower Upper
Occupational stress score 81.83 18.37 69.94 14.45 -9.13 -17.52 0.003
Occupational stress level (percent) Good 40 31 - 0.001
Mild stress 39 40
Moderate stress 19 14
Severe stress 2 11
Very severe stress 0 4

nCOVID-19, 2019 novel coronavirus disease; SD, Standard Deviation.

In addition to those mentioned above, it was determined that age, gender, work experience, type of contract, history of underlying diseases, history of taking medication, and smoking were among the most critical parameters in determining the effect of the COVID-19 outbreak on mental workload, job burnout, lifestyle, occupational stress, general health and, memory quotient values. The results related to the study of psychological and psychosocial parameters in terms of employee demographic variables are given in Table 6.


Table 6. Mean Scores of Physical, Psychological and Psychosocial Parameters Based on the Demographic Variables of the Studied Workers
Before nCOVID-19 Outbreak (Mean±SD) During nCOVID-19 Outbreak (Mean±SD)
Variables MWL JB LS GH OS WMS MWL JB LS GH OS WMS
Age (y) 19-27 59.41 ± 13.21 40.58 ± 16.97 119 ± 20.26 46.61 ± 12.91 84.64 ± 19.61 86.12 ± 11.57 63.71 ± 13.89 48.29 ± 18.52 118.64 ± 18.31 49.70 ± 10.68 72.34 ± 13.26 81.57 ± 9.98
28-35 56.37 ± 11.44 35.49 ± 13.61 130.11 ± 18.22 43.21 ± 11.52 84.10 ± 18.47 85.31 ± 12.15 57.04 ± 11.73 34.63 ± 12.81 121.64 ± 16.67 52.27 ± 12.69 71.68 ± 12.69 81.08 ± 10.18
 > 36 58.51 ± 12.73 41.64 ± 15.73 129.37 ± 19.38 45.28 ± 11.44 79.27 ± 17.57 83.11 ± 10.69 62.21 ± 14.82 47.59 ± 16.57 119.37 ± 15.97 50.87 ± 11.67 68.54 ± 14.72 79.13 ± 9.14
P value 0.041 0.049 0.037 0.030 0.011 0.046 0.037 0.013 0.041 0.010 0.024 0.039
Gender Male 58.46 ± 9.89 40.14 ± 14.69 128.14 ± 18.64 46.12 ± 11.36 79.64 ± 17.55 85.31 ± 12.79 65.82 ± 11.34 44.67 ± 15.90 119.37 ± 17.34 51.38 ± 12.34 70.67 ± 15.07 80.51 ± 10.51
Female 60.42 ± 10.57 39.13 ± 15.10 132.43 ± 19.13 45.11 ± 12.07 82.21 ± 18.51 84.81 ± 11.91 66.81 ± 12.80 43.72 ± 16.20 118.09 ± 16.32 50.91 ± 12.69 68.73 ± 14.91 81.37 ± 11.61
P-value 0.030 0.239 0.023 0.219 0.022 0.129 0.044 0.136 0.128 0.311 0.036 0.234
Work experience 1-5 years 60.73 ± 13.24 37.43 ± 13.54 121.37 ± 16.84 45.16 ± 10.73 81.72 ± 18.36 86.13 ± 13.25 67.14 ± 13.58 42.23 ± 15.31 115.67 ± 14.67 49.82 ± 12.68 69.15 ± 14.33 81.31 ± 10.67
 > 6 years 57.68 ± 12.36 42.64 ± 16.67 133.83 ± 19.67 46.61 ± 11.97 81.97 ± 18.9 85.44 ± 12.61 62.41 ± 10.72 48.64 ± 16.83 121.64 ± 16.44 52.6712.81 69.81 ± 14.20 80.14 ± 10.21
P value 0.046 0.001 0.001 0.289 0.384 0.237 0.023 0.001 0.003 0.037* 0.419 0.451
Contract Full-time 57.43 ± 10.40 38.43 ± 14.83 130.67 ± 19.67 42.76 ± 9.67 78.55 ± 16.71 85.00 ± 12.81 58.12 ± 11.13 44.35 ± 15.70 122.64 ± 15.97 48.61 ± 11.37 68.09 ± 13.28 80.34 ± 10.67
Part-time 58.36 ± 9.73 38.14 ± 15.24 128.64 ± 17.29 43.67 ± 10.82 79.33 ± 17.24 85.12 ± 11.57 63.29 ± 13.67 44.32 ± 14.67 120.51 ± 15.50 52.67 ± 13.14 68.38 ± 13.70 81.07 ± 9.61
Temporary 58.44 ± 10.29 39.37 ± 17.34 125.34 ± 16.51 47.55 ± 11.82 83.16 ± 18.42 83.09 ± 10.37 68.59 ± 13.78 48.67 ± 17.52 114.31 ± 9.64 52.11 ± 12.67 70.68 ± 14.79 79.23 ± 10.51
P value 0.129 0.120 0.012 0.028 0.011 0.048 0.028 0.003 0.001 0.041* 0.044 0.038
Underlying disease Yes 59.72 ± 11.29 39.94 ± 17.30 128.64 ± 17.10 47.51 ± 11.37 83.28 ± 19.36 84.33 ± 13.09 68.55 ± 13.84 48.79 ± 16.90 118.74 ± 16.52 54.61 ± 13.23 68.75 ± 15.09 81.31 ± 10.73
No 58.14 ± 9.37 39.43 ± 16.40 131.67 ± 20.43 43.67 ± 10.71 80.16 ± 17.67 85.12 ± 12.79 60.81 ± 10.61 45.31 ± 14.81 118.12 ± 15.87 50.78 ± 10.67 72.41 ± 13.25 80.79 ± 11.59
P value 0.347 0.451 0.042 0.001 0.037 0.419 0.001 0.031 0.327 0.021* 0.010 0.628
Taking medication Yes 60.27 ± 11.47 39.46 ± 15.82 128.44 ± 18.26 46.50 ± 10.67 82.64 ± 19.30 85.16 ± 12.33 65.70 ± 11.19 47.68 ± 16.09 119.31 ± 15.84 53.28 ± 12.67 69.68 ± 14.26 80.64 ± 10.14
No 60.11 ± 10.82 40.13 ± 14.61 129.67 ± 19.26 45.03 ± 11.20 80.51 ± 17.67 84.79 ± 11.89 62.97 ± 10.81 44.67 ± 15.34 120.68 ± 17.34 51.31 ± 11.31 71.22 ± 13.71 80.93 ± 11.07
P value 0.451 0.219 0.219 0.119 0.079 0.523 0.039 0.027 0.314 0.036* 0.068 0.419
Smoking Yes 58.34 ± 11.13 39.64 ± 14.51 125.74 ± 17.69 46.32 ± 10.58 82.67 ± 18.71 85.00 ± 12.29 62.73 ± 14.28 45.87 ± 16.98 114.67 ± 15.82 51.81 ± 12.39 68.11 ± 13.51 80.50 ± 10.33
No 59.64 ± 11.71 40.98 ± 15.67 132.51 ± 20.36 44.19 ± 12.09 81.47 ± 18.39 84.75 ± 11.58 63.46 ± 9.84 47.1 ± 15.53 123.17 ± 14.61 51.30 ± 12.67 69.30 ± 14.09 81.41 ± 9.57
P value 0.128 0.167 0.001 0.049 0.214 0.628 0.088 0.127 0.001 0.641 0.139 0.498

MWL, Mental workload score; JB, Job burnout score; LS, Lifestyle score; GH, General health score; OS, Occupational stress score; WMS, Wechsler memory score; SD, Standard Deviation.


Discussion

Study of mental workload based on NASA-TLX revealed that the mean scores of mental workload before and during the COVID-19 outbreak were 58.43 ± 11.68 and 66.38 ± 13.70, respectively (Table 2), and the COVID-19 outbreak has significantly enhanced the amount of mental workload of employees. It was found that the values of mental workload and frustration level increased significantly during the COVID-19 epidemic. These findings show that the COVID-19 outbreak has increased the amount of mental burden imposed on individuals and has also increased frustration, insecurity, and stress in employees in performing their duties. It was also found that the employees’ overall performance levels decreased significantly during the COVID-19 epidemic, and the personnel felt less satisfied with their performance in the workplace. Past studies have also pointed out that the outbreak of viral illnesses can increase the mental workload of individuals, which is consistent with the results of the present study.16,17

Before and during the COVID-19 outbreak, the mean job burnout scores were 39.75 ± 15.61 and 46.77 ± 16.31, respectively, and a significant relationship was found between the mentioned values (Table 3). A study of different dimensions of job burnout revealed that the COVID-19 outbreak increased the mean scores of emotional exhaustion and depersonalization and reduced the mean scores of the professional efficacy dimension. These findings confirm that the personnel in the workplace suffer from confusion and stress due to the COVID-19 epidemic, do not have a proper understanding of their new tasks, and feel less successful in performing their duties. All of these, as mentioned above, will increase the burnout of employees over time. A study conducted by Kim et al revealed that burnout and stress levels increased following the outbreak of viral illnesses such as the Middle East respiratory syndrome coronavirus (MERS-CoV) and could seriously impair a person’s job performance37 which is in line with the results of the present study.

The findings of the lifestyle survey revealed that the mean scores of lifestyle among all the participants before and during the COVID-19 outbreak were 129.16 ± 19.70 and 118.14 ± 16.07, respectively, and the COVID-19 outbreak has reduced the healthy lifestyle mean values of individuals (Table 4). It was found that the COVID-19 outbreak has increased the score of the health responsibility dimension, and employees are more sensitive to maintaining their health. The number of interpersonal relationships decreased significantly during the COVID-19 outbreak. Among the reasons for this are fear of contracting the disease and maintaining social distance to prevent the spread of the disease. Moreover, it has been determined that depression and anxiety are more likely to occur and worsen in the inadequacy of interpersonal interaction.11 It was also found that the rate of exercise decreased significantly during the COVID-19 epidemic. The reason for this can be the closure of gyms and clubs, quarantine at home, and fear of contracting an infection in case of physical activity outside home or with others. Previous studies have shown that people’s physical activity decreases during the spread of viral diseases such as H1N1. In contrast, a review of research on physical training and viral infections shows that regular, moderate-intensity aerobic exercise (65‒80% VO2max) improves the body’s immune system and reduces people’s stress levels.38 There was no significant difference between the nutritional status of employees before and during the COVID-19 outbreak, which can be due to lack of knowledge about proper nutritional habits during the epidemic of the COVID-19 or the economic situation of individuals.

The results showed that the employee’s general health status was affected by the COVID-19 outbreak. The mean scores of general health among all subjects before and during the COVID-19 outbreak were 45.13 ± 11.65 and 51.41 ± 12.89, respectively (Figure 2). The finding revealed that the COVID-19 outbreak significantly affects employees’ physical and mental condition. It was also found that the COVID-19 outbreak hurt the dimensions of anxiety/insomnia and severe depression in individuals and significantly increased the values of these parameters. Also, an examination of occupational stress values showed that the mean scores of occupational stress before and during the COVID-19 outbreak were 81.83 ± 18.37 and 69.94 ± 14.45, respectively, and the COVID-19 outbreak significantly increased the levels of worker’s occupational stress (Table 5).

Previous studies revealed that the spread of viral diseases such as MERS-COV and COVID-19 increases the levels of anxiety, stress, insomnia, and depression.37,39,40 The results of the study conducted by Wang et al in China showed that during the initial phase of the COVID-19 epidemic in China, in total, 53.8% of respondents rated the psychological impact of the outbreak as moderate or severe; 16.5% reported moderate to severe depressive symptoms, and 28.8% reported moderate to severe anxiety symptoms.10 These symptoms can be much more severe in working environments due to higher workload and stress levels.

Researchers in past studies have also pointed out that during the outbreak of viral diseases, due to people’s fear of contracting an infection, increasing social distance, insufficient support from the government and lack of access to adequate medical care and food, and ultimately fatigue and boredom, many symptoms of mental disease such as emotional distress, depression, stress, irritability, insomnia, decreased attention, post-traumatic stress disorder and anger are observed.16

The average scores of memory quotient of the personnel before and during the COVID-19 epidemic were 85.36 ± 15.19 and 80.64 ± 13.39, respectively, and the COVID-19 outbreak has significantly reduced workers’ memory quotient (Figure 2). One of the reasons for this is the decrease in the mental focus of employees due to the concerns caused by the outbreak of COVID-19. The study conducted by Kanmogne et al showed that the spread of viral diseases could reduce attention/working memory, learning, and memory in adults, which is consistent with the results of the present study.41

The results of the study regarding the average scores of physical, psychological, and psychosocial parameters based on the demographic variables demonstrated that the most significant effect of the COVID-19 outbreak on increasing mental workload and job burnout was in the age range of 19 to 27 years and more than 36 years. The reasons may be the inadequate work experience of younger personnel and the inability to control stress, the physical condition of older personnel, and fear of contracting an infection during the COVID-19 pandemic. It was also found that the COVID-19 outbreak had the most negligible impact on the lifestyle of young employees in the age range of 19 to 27. It was observed that the COVID-19 outbreak was more effective in the amount of mental workload in workers with part-time and temporary contracts. Due to the unfavorable economic conditions of the company at the time of the COVID-19 outbreak and the possibility of terminating the contracts of several employees, the personnel with part-time and temporary contracts feel more mental workload.

Moreover, the COVID-19 outbreak had the greatest impact on reducing the quality of lifestyle of workers with temporary contracts. One of the reasons for this issue is the decrease in their income due to the decrease in working hours. Moreover, the rate of increase in the average job burnout and occupational stress values in employees with part-time and temporary contracts was higher than employees with permanent and full-time contracts.

Following the outbreak of COVID-19, the personnel with a history of underlying and chronic illnesses and using medications experienced more increased mental workload, job burnout, and occupational stress compared to the other personnel. One of the reasons for this is the greater fear of these employees regarding becoming infected with the coronavirus disease, because these people are more susceptible to infection. Also, female employees’ levels of occupational stress increased significantly due to the outbreak of COVID-19 compared to male employees (Table 6).

One of the limitations of the present study was the impossibility of examining other psychosocial and psychometric parameters and the impossibility of conducting an intervention study due to time constraints. Therefore, it is suggested that researchers conduct interventional studies in the future in order to improve the status of the most important psychological and psychosocial parameters in the workplace during the coronavirus pandemic and also to evaluate the effectiveness of intervention measures. Another limitation of the present study was the potential threats to internal validity. The COVID-19 outbreak is probably “causing” a lot of these changes, but based on our design alone, we cannot be sure, and a lot of other things are happening in the world between those two data points (both related and unrelated to the pandemic). For the first time in the workplace, the present study was conducted to examine changes in the most important psychological and psychosocial parameters during the COVID-19 pandemic. The findings of this study can provide novel insight into the effects of the COVID-19 pandemic on psychological and psychosocial variables and associated risk factors among employees in different working environments.

Ultimately, the findings showed that the COVID-19 outbreak affected the psychological and psychosocial parameters of the subjects; so, it is essential to adopt measures to control these parameters such as increasing social support for employees, providing the financial demands of workers during the COVID-19 outbreak, providing appropriate protective equipment (such as face shields, masks, gloves, etc) following the guidelines provided by the relevant organizations (WHO, OSHA, NIOSH, etc), providing educational content to make employees more familiar with disease prevention strategies and stress control, and maintaining peace of mind during an epidemic and increasing telecommuting until the disease is reduced.

In conclusion, the results of the present study revealed that the COVID-19 outbreak could affect psychological and psychosocial parameters such as mental workload, job burnout, healthy lifestyle, general health, occupational stress, and functional working memory of employees in working environments. Therefore, implementing psychological interventions is very necessary to promote the mental health of personnel during the COVID-19 epidemic.


Authors’ Contribution

AS and Mohsen.SY: Project development, data collection, manuscript writing, final approval of the submitted version. HG, MM and SK: Data analysis, SSY, MGH and Mansoureh.SY: Data Collection, manuscript writing, final approval of the submitted version.

Conflict of Interest Disclosures

The authors declare that there is no conflict of interest.

Ethical Statement

Ethical approval for this study was obtained from the Ethics Committee of Birjand University of Medical Sciences (Ethics code: IR.BUMS.REC.1399.114).

Funding

This study was supported by Birjand University of Medical Sciences & Health Services under grant number 5402.


References

  1. Zhai Y, Du X. Mental health care for international Chinese students affected by the COVID-19 outbreak. Lancet Psychiatry 2020; 7(4):e22. doi: 10.1016/s2215-0366(20)30089-4 [Crossref] [ Google Scholar]
  2. Zhu H, Wei L, Niu P. The novel coronavirus outbreak in Wuhan, China. Glob Health Res Policy 2020; 5:6. doi: 10.1186/s41256-020-00135-6 [Crossref] [ Google Scholar]
  3. Remuzzi A, Remuzzi G. COVID-19 and Italy: what next?. Lancet 2020; 395(10231):1225-8. doi: 10.1016/s0140-6736(20)30627-9 [Crossref] [ Google Scholar]
  4. Takian A, Raoofi A, Kazempour-Ardebili S. COVID-19 battle during the toughest sanctions against Iran. Lancet 2020; 395(10229):1035-6. doi: 10.1016/s0140-6736(20)30668-1 [Crossref] [ Google Scholar]
  5. Lai CC, Shih TP, Ko WC, Tang HJ, Hsueh PR. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease-2019 (COVID-19): the epidemic and the challenges. Int J Antimicrob Agents 2020; 55(3):105924. doi: 10.1016/j.ijantimicag.2020.105924 [Crossref] [ Google Scholar]
  6. Chen N, Zhou M, Dong X, Qu J, Gong F, Han Y. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet 2020; 395(10223):507-13. doi: 10.1016/s0140-6736(20)30211-7 [Crossref] [ Google Scholar]
  7. Paules CI, Marston HD, Fauci AS. Coronavirus infections-more than just the common cold. JAMA 2020; 323(8):707-8. doi: 10.1001/jama.2020.0757 [Crossref] [ Google Scholar]
  8. World Health Organization (WHO). WHO Director-General’s Opening Remarks at the Media Briefing on COVID-19 - 3 March 2020. WHO; 2020.
  9. World Health Organization (WHO). Coronavirus Disease (‎COVID-19)‎: Situation Report, 206. WHO; 2020.
  10. Wang C, Pan R, Wan X, Tan Y, Xu L, Ho CS. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int J Environ Res Public Health 2020; 17(5):1729. doi: 10.3390/ijerph17051729 [Crossref] [ Google Scholar]
  11. Xiao C. A novel approach of consultation on 2019 novel coronavirus (COVID-19)-related psychological and mental problems: structured letter therapy. Psychiatry Investig 2020; 17(2):175-6. doi: 10.30773/pi.2020.0047 [Crossref] [ Google Scholar]
  12. Hall RC, Hall RC, Chapman MJ. The 1995 Kikwit Ebola outbreak: lessons hospitals and physicians can apply to future viral epidemics. Gen Hosp Psychiatry 2008; 30(5):446-52. doi: 10.1016/j.genhosppsych.2008.05.003 [Crossref] [ Google Scholar]
  13. Wang Z, Liu H, Yu H, Wu Y, Chang S, Wang L. Associations between occupational stress, burnout and well-being among manufacturing workers: mediating roles of psychological capital and self-esteem. BMC Psychiatry 2017; 17(1):364. doi: 10.1186/s12888-017-1533-6 [Crossref] [ Google Scholar]
  14. Allahyari T, Khanehshenas F, Khalkhali H. Explaining the relationship between psychosocial stress and job performance in banking industry: an integrated model. J Occup Hyg Eng 2017; 4(1):18-25. doi: 10.21859/johe-04013.[Persian] [Crossref] [ Google Scholar]
  15. Mollaei M, Allahyari T, Arsalani N, Khalkhali H. An investigation of the relationship between psychosocial work factors and cognitive failures in nursing. Iran Occup Health 2018; 15(3):89-102. [ Google Scholar]
  16. Shahyad S, Mohammadi MT. Psychological impacts of COVID-19 outbreak on mental health status of society individuals: a narrative review. J Mil Med 2020; 22(2):184-92. doi: 10.30491/jmm.22.2.184.[Persian] [Crossref] [ Google Scholar]
  17. Kang L, Li Y, Hu S, Chen M, Yang C, Yang BX. The mental health of medical workers in Wuhan, China dealing with the 2019 novel coronavirus. Lancet Psychiatry 2020; 7(3):e14. doi: 10.1016/s2215-0366(20)30047-x [Crossref] [ Google Scholar]
  18. Kim KW, Park SJ, Lim HS, Cho HH. Safety climate and occupational stress according to occupational accidents experience and employment type in shipbuilding industry of Korea. Saf Health Work 2017; 8(3):290-5. doi: 10.1016/j.shaw.2017.08.002 [Crossref] [ Google Scholar]
  19. Basahel AM. Physical demand, fatigue and shift work in the installation and maintenance of window air-conditioner units. Life Sci J 2016; 13(5):46-55. doi: 10.7537/marslsj13051605 [Crossref] [ Google Scholar]
  20. Khandan M, Mirshekari F, Koorsani E, Mosaferchi S, Koohpaei A. Subjective workload and musculoskeletal disorders among workers of a manufacturing company in Iran. Biotechnol Health Sci 2018; 5(1):e13599. [ Google Scholar]
  21. Puspawardhani EH, Suryoputro MR, Sari AD, Kurnia RD, Purnomo H. Mental workload analysis using NASA-TLX method between various level of work in plastic injection division of manufacturing company. In: Arezes P, ed. Advances in Safety Management and Human Factors. Cham: Springer; 2016. p. 311-9. 10.1007/978-3-319-41929-9_29
  22. Lowndes BR, Forsyth KL, Blocker RC, Dean PG, Truty MJ, Heller SF. NASA-TLX assessment of surgeon workload variation across specialties. Ann Surg 2020; 271(4):686-92. doi: 10.1097/sla.0000000000003058 [Crossref] [ Google Scholar]
  23. Kabir MJ, Heidari A, Etemad K, Gashti AB, Jafari N, Honarvar MR. Job burnout, job satisfaction, and related factors among health care workers in Golestan province, Iran. Electron Physician 2016; 8(9):2924-30. doi: 10.19082/2924 [Crossref] [ Google Scholar]
  24. Sepehri Shamloo Z, Hashemian SS, Khoshsima H, Shahverdi A, Khodadost M, Modares Gharavi M. Validity and reliability of the Persian version of the Maslach burnout inventory (general survey version) in Iranian population. Iran J Psychiatry Behav Sci 2017; 11(2):e8168. doi: 10.5812/ijpbs.8168 [Crossref] [ Google Scholar]
  25. Walker SN, Sechrist KR, Pender NJ. The health-promoting lifestyle profile: development and psychometric characteristics. Nurs Res 1987; 36(2):76-81. [ Google Scholar]
  26. Mohammadi M, Ramezankhani A, Mohammadi S, Zahed S, Khabiri F, Khodakarim S. The predictors of metabolic syndrome based on Walker Health-Promoting lifestyle in Iran 2016. Diabetes Metab Syndr 2017; 11 Suppl 2:S745-S9. doi: 10.1016/j.dsx.2017.05.009 [Crossref] [ Google Scholar]
  27. Goldberg DP, Hillier VF. A scaled version of the General Health Questionnaire. Psychol Med 1979; 9(1):139-45. doi: 10.1017/s0033291700021644 [Crossref] [ Google Scholar]
  28. Noorbala AA, Faghihzadeh S, Kamali K, Bagheri Yazdi SA, Hajebi A, Mousavi MT. Mental health survey of the Iranian adult population in 2015. Arch Iran Med 2017; 20(3):128-34. [ Google Scholar]
  29. Nazifi M, Mokarami H, Akbaritabar A, Faraji Kujerdi M, Tabrizi R, Rahi A. Reliability, validity and factor structure of the Persian translation of General Health Questionnaire (GHQ-28) in hospitals of Kerman University of Medical Sciences. J Adv Biomed Sci 2014; 3(4):336-42. [ Google Scholar]
  30. Taghavi S. Validity and reliability of the General Health Questionnaire (GHQ-28) in college students of Shiraz University. J Psychol 2002; 5(4):381-98. [ Google Scholar]
  31. Ryan JJ, Morris J, Yaffa S, Peterson L. Test-retest reliability of the Wechsler Memory Scale, Form I. J Clin Psychol 1981; 37(4):847-8. doi: 10.1002/1097-4679(198110)37:4<847::aid-jclp2270370429>3.0.co;2-k [Crossref] [ Google Scholar]
  32. Wechsler D. A standardized memory scale for clinical use. J Psychol 1945; 19(1):87-95. doi: 10.1080/00223980.1945.9917223 [Crossref] [ Google Scholar]
  33. Prigatano GP. Wechsler Memory Scale: a selective review of the literature. J Clin Psychol 1978; 34(4):816-32. doi: 10.1002/1097-4679(197810) [Crossref] [ Google Scholar]
  34. Cousins * R, MacKay CJ, Clarke SD, Kelly C, Kelly PJ, McCaig RH. ‘Management standards’ work-related stress in the UK: practical development. Work Stress 2004; 18(2):113-36. doi: 10.1080/02678370410001734322 [Crossref] [ Google Scholar]
  35. Azad-Marzabadi E, Gholami Fesharaki M. Reliability and validity assessment for the HSE job stress questionnaire. Int J Behav Sci 2011; 4(4):291-7. [ Google Scholar]
  36. Sahraian A, Davidi F, Bazrafshan A, Javadpour A. Occupational stress among hospital nurses: comparison of internal, surgical, and psychiatric wards. Int J Community Based Nurs Midwifery 2013; 1(4):182-90. [ Google Scholar]
  37. Kim JS, Choi JS. Factors influencing emergency nurses’ burnout during an outbreak of Middle East respiratory syndrome coronavirus in Korea. Asian Nurs Res (Korean Soc Nurs Sci) 2016; 10(4):295-9. doi: 10.1016/j.anr.2016.10.002 [Crossref] [ Google Scholar]
  38. Shirvani H, Rostamkhani F. Exercise considerations during coronavirus disease 2019 (COVID-19) outbreak: a narrative review. J Mil Med 2020; 22(2):161-8. doi: 10.30491/jmm.22.2.161.[Persian] [Crossref] [ Google Scholar]
  39. Lima CKT, de Medeiros Carvalho PM, de Araújo Araruna Silva Lima I, de Oliveira Nunes JV, Saraiva JS, de Souza RI. The emotional impact of coronavirus 2019-nCoV (new coronavirus disease). Psychiatry Res 2020; 287:112915. doi: 10.1016/j.psychres.2020.112915 [Crossref] [ Google Scholar]
  40. Sadeghi Yarandi M, Gholami A, Ghasemi M, Sadeghi Yarandi M, Ghasemi Koozekonan A, Soltanzadeh A. Investigating the psychological consequences of the COVID-19 outbreak in the occupational society. J Mil Med 2020; 22(6):562-9. doi: 10.30491/jmm.22.6.562.[Persian] [Crossref] [ Google Scholar]
  41. Kanmogne GD, Fonsah JY, Umlauf A, Moul J, Doh RF, Kengne AM. Attention/working memory, learning and memory in adult Cameroonians: normative data, effects of HIV infection and viral genotype. J Int Neuropsychol Soc 2020; 26(6):607-23. doi: 10.1017/s1355617720000120 [Crossref] [ Google Scholar]
Submitted: 31 Dec 2020
Revised: 22 Apr 2021
Accepted: 28 Apr 2021
First published online: 01 Nov 2021
EndNote EndNote

(Enw Format - Win & Mac)

BibTeX BibTeX

(Bib Format - Win & Mac)

Bookends Bookends

(Ris Format - Mac only)

EasyBib EasyBib

(Ris Format - Win & Mac)

Medlars Medlars

(Txt Format - Win & Mac)

Mendeley Web Mendeley Web
Mendeley Mendeley

(Ris Format - Win & Mac)

Papers Papers

(Ris Format - Win & Mac)

ProCite ProCite

(Ris Format - Win & Mac)

Reference Manager Reference Manager

(Ris Format - Win only)

Refworks Refworks

(Refworks Format - Win & Mac)

Zotero Zotero

(Ris Format - FireFox Plugin)

Abstract View: 2201
PDF Download: 1026
Full Text View: 43