Systematic Review

Frequency of the Statistical Methods and Relation with Acceptance Period in Archives of Iranian Medicine Articles: A Review from 2015–2019

Cite this article as: Bahariniya S, Madadizadeh F. Frequency of the statistical methods and relation with acceptance period in archives of iranian medicine articles: a review from 2015–2019. Arch Iran Med. 2022;25(4):267-273. doi: 10.34172/aim.2022.43

Introduction

Statistical methods are recognized as essential and vital tools in biomedical research. In modern medical research, the use of advanced and extensive statistical methods is very important. Improper use of statistical methods can reduce the quality of biomedical research, and lead to poor interpretation and wrong conclusions. On the other hand, the use of complex and inefficient statistical methods solely for the purpose of superficial improvement of the study can be considered as an immoral approach.^1-5

Medical journals with a wide coverage of various topics in the field of medicine and public health have many enthusiasts around the world. In Iran, the Archives of Iranian Medicine (AIM) is a valuable journal and a reputable clinical resource that has been working since 1998 and has been publishing articles monthly since 2012. The journal welcomes biomedical experiences and clinical research on common diseases in the region, as well as analysis of factors that may moderate the management of diseases and related medical problems. The impact factor (IF) of this journal in 2019 was about 0.996 and the 5 years IF was 1.46, It also had an H-index of about 45 and a CiteScore of about 2.20.

Printing high quality articles is the main goal of all journals, especially journals in the field of medical sciences. To achieve this goal, it is necessary to evaluate the articles published in the journal, review the quality of articles, identify strengths and weaknesses of the journal and eliminate problems and shortcomings in future issues.⁶

The use of advanced statistical methods works as a double-edged sword that, on the one hand, leads to deeper analysis and more valuable findings, and on the other hand, makes it difficult to interpret data; so the presence of biostatistics experts can be helpful at this stage. Improper use of statistical methods can lead to a loss of researchers’ effort and a waste of time and investment. One of the key factors to improve the quality of medical journals is the acceptance of articles that have used valid, advanced and effective statistical methods. Reputable international journals that seek promotion, in order to increase the quality of the articles and to ensure the validity of the statistical methods used in the articles, conduct detailed reviews by judges with statistical expertise. The type of statistical methods used in articles may play an important role in attracting the attention and decision of the journal editors regarding the acceptance or rejection of articles.^7-10

Some authors believe that if they use more advanced statistical methods instead of just conventional and simple methods, they will have a better chance of publishing the article and will spend less time in the judging process. So, according to this belief, they start with using advanced statistical methods and sometimes get stuck in interpreting the outputs of these methods, and finally submit their article with incorrect interpretations to the journal. On the other hand, knowledge of the variety of statistical methods used and the trend of the average refereeing time, type of study, statistical community and applications can be useful for self-evaluation and promotion of the journal and identify gaps in science and increase the variety of articles. In this regard, review articles have recently been published focusing on the statistical methods used in the original articles of scientific journals.^11-16

Due to the importance of the content mentioned in the previous paragraphs, the present study aimed to identify and review statistical methods, sample size, study design, statistical population, and type of software used in the original articles and investigate their relationship with the acceptance period (interval between the received and accepted dates mentioned in the articles) of articles published in the Archives of Iranian Medicine in the period 2015 to 2019. Statistical review of journal articles is usually done over a period of 5 or 10 years.^3,4 At the time of conducting this study (June 2020), AIM’s 2020 issues were not yet complete, so this year was not included in our study. Therefore, the period from 2015 to 2019 was selected as our study period.

Materials and Methods

This review study was conducted in June 2020 on AIM Journal articles. All original articles published in a period of 5 years (2015–2019) from volumes 18 to 22 and issues 1 to 12 of the Archives of Iranian Medicine were reviewed. Each article was reviewed by a three-member team consisting of a biostatistics expert and two expert researchers in the field of medical research. Six variables such as statistical methods, sample size, study population, software, study design and acceptance period were extracted from the articles.

In order to perform descriptive statistics, considering that it was possible to use more than one method in each article, the multiple response analysis (MRA) technique was used and the frequency and frequency percentage were reported. MRA is one of the valuable statistical methods for analyzing questions with the possibility of more than one answer. In the output of this method, which we used in the present study, unlike simple descriptive statistics tables, a table is provided containing two absolute/relative frequencies, one for the sum of responses and one for cases. In our study, the answers were the tests used in articles and the cases were original articles. In order to investigate the relationship between the type of statistical methods, software, statistical population and type of study with the acceptance period of the articles, first the normality of errors in quantitative variables was investigated by the Kolmogorov-Smirnov test and after rejecting the null hypothesis and confirmation of non-normality in errors, the Kruskal Wallis test and Spearman correlation coefficient were used. Statistical analyses were performed using SPSS version 26 with a significance level of 5%.

Results

A total of 742 articles were published during the study period, of which 423 (57%) were original articles that were included in our study. Other published articles such as Case Reports, Reviews, Letters to the Editor, Editorials, Brief Reports, Opinions, Book Reviews, Systematic Reviews and others were not included in the study (Table 1).

All statistical methods used in the original articles published in the Archives of Iranian Medicine during the study period of (5 years: 2015–2019) are summarized in Table 2.

In the period of the present study (from 2015–2019), the statistical population reported in the articles mostly consisted of patients (164 cases, 38.8%) and human samples at the community level (155 cases, 36.6%). The least use pertained to statistical data with 1.4% (6 cases); the results of the Kruskal-Wallis test showed that the mean acceptance period in different statistical populations was not significantly different (P = 0.148) (Table 3).

During the study period, most studies had a sample size of less than 500 people (51.5%, 218 cases). In 12.3% (52 cases) of the studies, the sample size was unknown; the results of the Kruskal-Wallis test showed that the mean acceptance period in different sample sizes was not significantly different (P = 0.356) (Table 4).

The study design in most articles was analytical-observational (233 cases, 55.1%) and only 3.3% (14 cases) were qualitative; the results of the Kruskal-Wallis test showed that the mean acceptance period in different study designs was not significantly different (P = 0.439) (Table 5).

Based on the results, most articles had selected the SPSS software for analysis (337 cases and 79.7%); the results of the Kruskal-Wallis test showed that the mean acceptance period in different types of software was not significantly different (P = 0.072) (Table 6).

The results of our review showed that the time of receiving and accepting the article and mentioning it in the main file of the article was done exactly since the first issue published in 2018. The results showed that the mean (SD) and median (IQR) acceptance period of the articles were 200.99 (106.856) and a 194 (134.25) days, respectively.

The results of the Kruskal-Wallis test showed that there was a significant difference in the acceptance period across months (P = 0.001), such that the longest time (541 days) pertained to the second month of 2018, after which the acceptance period diminished and reached 180 days at the end of 2018. Also, the declining trend continued and reached 164 days at the end of 2019. There was no significant relationship between the acceptance period and the study design (P = 0.568), statistical population type, (P = 0.148), type of software (P = 0.072), or type of statistical methods (P = 0.252). The results of the Spearman correlation coefficient test did not show a significant relationship between sample size and acceptance period (r = 0.13, P = 0.482).

Figure 1 shows the trend of the acceptance period of articles reviewed in our study. There was a downward trend during both years.

Close

Figure 1.

Median Length of Acceptance Period During 2 Years Based on Issues 1 to 12.

Figure 1.

Median Length of Acceptance Period During 2 Years Based on Issues 1 to 12.

The findings of MRA showed that generally, among all the statistical methods that were used in the articles, the highest rate of use pertained to descriptive statistics (30.3% and 227 cases), chi-square test (14.2% and 130 cases), t test (12.6% and 115 cases), and regression models (11.5% and 105 cases). Among all articles, 68.9% used descriptive statistics, 32.3% used chi-square test, 28.6% used ttest, 26.1% used regression models, 19.2% used one-way analysis of variance, and 18.9% used non-parametric methods (Table 7).

The trend of using simple statistical methods (such as descriptive statistics, t test, chi-square, Fisher’s exact test, McNemar, one-way analysis of variance, Mann–Whitney U test, Kruskal-Wallis test) and advanced statistical methods (ANCOVA, MANOVA, COX regression, multiple regression, SEM etc.) is presented in Figure 2. The average use of advanced statistical methods increased during the years under review.

Close

Figure 2.

Mean Number of Times to Use Complex and Simple Statistical Methods During 5 Years.

Figure 2.

Mean Number of Times to Use Complex and Simple Statistical Methods During 5 Years.

Discussion

In recent years, the philosophy of using statistical methods in biomedical research has undergone massive changes. Meanwhile, some researchers believe that if they use more sophisticated and advanced statistical methods to analyze research data, they will have a better chance to publish the article in a short time.

The findings of this study, which were obtained by reviewing the AIM journal in the period 2015 to 2019 and based on 423 original articles, showed that patient and community-level samples (including women, men, mothers, girls, adults, children, students, infants, smokers, the elderly and middle-aged, families, adolescents, young people, urban and rural residents, transplant recipients, blood donors, healthy people in the community, obese people, prisoners, couples, spouses and others) were the most frequent statistical population used in the reviewed articles. This shows the special attention of the leading AIM journal to accepting articles on the topics of diseases and problems in the community. Due to the possibility of missing statistical data, despite the great effort of researchers, the manuscript may be rejected by reviewers. So, according to the results of the present study, the lowest frequency of statistical populations pertained to statistical data such as information, documents and reports.

The cumulative frequency in Table 4 shows that 36.2% of the sample sizes were more than 500 people. It may be thought that the larger the sample size of the study, the less time it will take for the article to be accepted. The results of our study showed that this assumption is wrong and there was no relationship between the acceptance period and the article’s sample size.

The findings of the present study in mentioning the type of study design in the studied articles showed that most of the study designs were analytical-observational. Up to 55% of the studies were analytical-observational. This confirms the special attention of AIM to this type of studies. Qualitative studies involve the collection and analysis of non-numerical data to understand concepts, opinions, or experiences. These studies are used to gather in-depth insights into an issue or generate new ideas for research. According to the results, only 3.3% of the studies were qualitative. Given that qualitative research can play a major role in health research, it requires more attention from the editors of the journal to this issue.¹⁷

SPSS is one of the most popular statistical analysis software among researchers. Although it has very good coverage of common statistical methods, it is not a complete software, and researchers must use programming software or semi-programming software to use more advanced statistical methods. The findings of the present study showed that among the articles published in this journal, the majority of articles used SPSS and a smaller percentage of published articles used programming (such as R, Win Bugs, Matlab, Python, etc.) or semi-programming software (such as Minitab, Stata, etc.).¹⁸ A possible reason for this could be the features of being user-friendly and the availability and easy learning of SPSS.

According to Table 7, the MRA results showed that among both the total tests and the articles, the highest use of statistical methods pertained to descriptive statistics and the chi-square test. In the present study, the Fisher exact test, Hardy-Weinberg, Wald test and Trend analysis were considered as the chi-square test. Frequent use of statistical tests in the findings of similar studies^19-21 in line with our study showed that the most common statistical method used in the articles pertained to descriptive statistics and the chi-square test.

The chi-square test is one of the most important statistical tests used to evaluate the relationship between two qualitative variables.²⁰ This test is one of the available statistical methods, user-friendly and easily used in most statistical software. Therefore, it has many fans. Our results showed that the use of advanced and specialized statistical methods in 2019 was reduced compared to previous years. This requires the attention of editors to accept articles with more advanced statistical methods.

Based on the results of the present study, from the last issue published in 2017, the time of receipt and the time of acceptance are mentioned on the first page of the articles. Mentioning the time of receiving and accepting articles in the published articles is very important and in order to promote AIM journal, it is necessary for the editors of the journal to pay attention to such issues.

Accuracy in judging and accepting articles in a short period of time will pave the way for a journal to move forward. On the other hand, the authors tend to submit their articles to journals that have a shorter acceptance period. In general, as shown in Figure 1, fortunately, the time of acceptance of articles by AIM had a declining trend during 24 months, and this issue, along with the accuracy of referees, can pave the way for the future progress of the journal.

The results of the Kruskal-Wallis test showed that there was no significant difference in the median time of acceptance of articles in terms of the type of test, sample size, type of statistical community, study design and type of software. This highlights the misconception of those researchers who believe that by using more advanced statistical tests, a larger sample size, more sensitive statistical populations, more sophisticated study designs, and more advanced statistical software can publish their article in a shorter time. The results of the present study were consistent with the results of similar studies.²²

In terms of strengths, this study examined and analyzed more variables compared to similar studies. One of the main limitations of this study was lack of access to articles rejected by the journal and only accepted articles were reviewed. Otherwise, based on statistical analysis of logistic regression, we could calculate the odds of the acceptance of an article based on each of the variables studied in this study, such as sample size, type of statistical method, type of statistical population, etc. Another limitation was lack of access to information on the refereeing time of all articles (accepted and rejected), which, if accessible, we could have definitely performed a more valuable analysis. To calculate the time required to publish the articles, the difference between the time of receipt and the time of acceptance that was listed on the first page of the article, was considered.

In conclusion, it took about 28 weeks from submission to acceptance in AIM journal. Given that the acceptance period is an important item for researchers to choose a journal and on the other hand, it can be a key item to promote journal indexing, it seems to be beneficial to reduce it and more attention is required from the editors of the journal. Most of the articles were of the observational-analytical design with patient as the statistical population. The statistical methods of most articles consisted of simple tests. The frequency of advanced statistical methods was decreasing during the study period. Acceptance of articles with advanced statistical methods seems necessary. Contrary to what the researchers thought, the type of test, sample size, type of community, type of study design or type of software were not effective factors regarding the acceptance period.

References

1. Zapf A, Rauch G, Kieser M. Why do you need a biostatistician?. BMC Med Res Methodol 2020; 20(1):23. doi: 10.1186/s12874-020-0916-4 [Crossref] [ Google Scholar]
2. Tsiamalou P, Brotis A. Biostatistics as a Tool for Medical Research: What are we Doing Wrong?. Mediterr J Rheumatol 2019; 30(4):196-200. doi: 10.31138/mjr.30.4.196 [Crossref] [ Google Scholar]
3. Parsons NR, Price CL, Hiskens R, Achten J, Costa ML. An evaluation of the quality of statistical design and analysis of published medical research: results from a systematic survey of general orthopaedic journals. BMC Med Res Methodol 2012; 12:60. doi: 10.1186/1471-2288-12-60 [Crossref] [ Google Scholar]
4. Choi E, Lyu J, Park J, Kim HY. Statistical methods used in articles published by the Journal of Periodontal and Implant Science. J Periodontal Implant Sci 2014; 44(6):288-92. doi: 10.5051/jpis.2014.44.6.288 [Crossref] [ Google Scholar]
5. Kumar A, Misra DK. A review on the statistical methods and implementation to homogeneity assessment of certified reference materials in relation to uncertainty. MAPAN 2020; 35(3):457-70. doi: 10.1007/s12647-020-00383-4 [Crossref] [ Google Scholar]
6. Archives of Iranian Medicine (AIM) website. http://www.aimjournal.ir.
7. Ibrahimi E. Data analysis and documentation of statistics in biomedical research papers in Albania. Biostatistics Epidemiol Int J 2018; 1(1):18-20. doi: 10.30881/beij.00006 [Crossref] [ Google Scholar]
8. Nieminen P. Ten points for high-quality statistical reporting and data presentation. Appl Sci 2020; 10(11):3885. doi: 10.3390/app10113885 [Crossref] [ Google Scholar]
9. Sato Y, Gosho M, Nagashima K, Takahashi S, Ware JH, Laird NM. Statistical methods in the journal—an update. N Engl J Med 2017; 376(11):1086-7. doi: 10.1056/NEJMc1616211 [Crossref] [ Google Scholar]
10. Otwombe KN, Petzold M, Martinson N, Chirwa T. A review of the study designs and statistical methods used in the determination of predictors of all-cause mortality in HIV-infected cohorts: 2002-2011. PLoS One 2014; 9(2):e87356. doi: 10.1371/journal.pone.0087356 [Crossref] [ Google Scholar]
11. Popovic MM, Yissar D, Joksimovic L, Thang M, Micieli J, Buys YM. Frequency of statistical methods in ophthalmology journal articles: a review. Expert Rev Ophthalmol 2020; 15(3):183-91. doi: 10.1080/17469899.2020.1764350 [Crossref] [ Google Scholar]
12. Gosho M, Sato Y, Nagashima K, Takahashi S. Trends in study design and the statistical methods employed in a leading general medicine journal. J Clin Pharm Ther 2018; 43(1):36-44. doi: 10.1111/jcpt.12605 [Crossref] [ Google Scholar]
13. Zhang J, Wang Y, Zhao Y. Investigation on the statistical methods in research studies of library and information science. Electron Libr 2017; 35(6):1070-86. doi: 10.1108/el-02-2016-0042 [Crossref] [ Google Scholar]
14. Kim J, Yoon S, Kang JJ, Han K, Kim JM, Kim SK. Research designs and statistical methods trends in the annals of rehabilitation medicine. Ann Rehabil Med 2017; 41(3):475-82. doi: 10.5535/arm.2017.41.3.475 [Crossref] [ Google Scholar]
15. Diong J, Butler AA, Gandevia SC, Héroux ME. Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PLoS One 2018; 13(8):e0202121. doi: 10.1371/journal.pone.0202121 [Crossref] [ Google Scholar]
16. Lang TA, Altman DG. Basic statistical reporting for articles published in biomedical journals: the “Statistical Analyses and Methods in the Published Literature” or the SAMPL Guidelines. Int J Nurs Stud 2015; 52(1):5-9. doi: 10.1016/j.ijnurstu.2014.09.006 [Crossref] [ Google Scholar]
17. Pathak V, Jena B, Kalra S. Qualitative research. Perspect Clin Res 2013; 4(3):192. doi: 10.4103/2229-3485.115389 [Crossref] [ Google Scholar]
18. van Wingerde B, van Ginkel J. SPSS syntax for combining results of principal component analysis of multiply imputed data sets using generalized procrustes analysis. Appl Psychol Meas 2021; 45(3):231-2. doi: 10.1177/0146621621990757 [Crossref] [ Google Scholar]
19. Goldin J, Zhu W, Sayre JW. A review of the statistical analysis used in papers published in Clinical Radiology and British Journal of Radiology. Clin Radiol 1996; 51(1):47-50. doi: 10.1016/s0009-9260(96)80219-4 [Crossref] [ Google Scholar]
20. Barbosa FT, de Souza DA. Frequency of the adequate use of statistical tests of hypothesis in original articles published in the Revista Brasileira de Anestesiologia between January 2008 and December 2009. Rev Bras Anestesiol 2010; 60(5):528-36. doi: 10.1016/s0034-7094(10)70064-7 [Crossref] [ Google Scholar]
21. Emerson JD, Colditz GA. Use of statistical analysis in the New England Journal of Medicine. N Engl J Med 1983; 309(12):709-13. doi: 10.1056/nejm198309223091206 [Crossref] [ Google Scholar]
22. Bahariniya S, Madadizadeh F. Review of the statistical methods used in original articles published in Iranian Journal of Public Health from 2015-2019: a review article. Iran J Public Health 2021; 50(8):1577-85. doi: 10.18502/ijph.v50i8.6803 [Crossref] [ Google Scholar]