Is an article in a top journal a top article?

Financial Management (Financial Management Association), Winter, 2004 by Stanley D. Smith

This study ranks 15 leading finance journals by the average number of Social Sciences Citation Index cites per articles for articles published in 1996. It also defines a "top article," compared to an "article in a top journal. " Using different criteria for top articles, I examine the Type I error (a "top " article is rejected by a particular decision rule, e.g., in top three journals) and the Type II error (a "non-top" article is accepted as a top article) for each journal and combinations of the journals. Due to the high error rates, the results suggest that identifying top articles requires looking beyond the Top 3 journals, as well as examining each article more carefully for its intrinsic quality.

**********

A colleague was recently told by his dean that to be promoted to professor, he needed to have more publications in the "top three" finance journals. Another colleague, an assistant professor at another university, was told that only articles in the top three journals would count towards his promotion and tenure decision. But the top three journals were not the same in these two cases. Assistant professors are often told that they need publications in the top three journals or "top N" journals.

The purpose of this article is to investigate whether a "top N journals" approach provides a reasonable decision rule in terms of identifying top articles in the finance literature. I use Type 1 and II errors to evaluate the accuracy of these decision rules. I define a Type I error as one when a "top" article is rejected by the decision rule. I call this a Reject Top Article (RTA) error. 1 define a Type ]I error as one when a "non-top" article is accepted as a top article. I call Type II errors an Accept Non-Top Article (ANTA) error.

In this study, 1 rank 15 leading finance journals by the average number of Social Sciences Citation Index (SSCI) cites per articles from 1996 to January 2004 for articles published in those 15 journals in 1996. Recent studies often use Journal Citation Reports (JCR) impact factors to rank journals. The JCR impact factor for a journal is the number of cites in year t for articles published in years t-I and t-2 divided by the number of articles published in years t-1 and t-2. Given the possibly long life over which an article can be cited, as measured by citations the impact factor is a relatively short-term measure of the impact. Although I use one year of publications, 1996, the number of citations is over a much longer period than the JCR impact factors.

I also define a "top article" (the average number of cites is above the median, mean, 90'h percentile, or 95'h percentile for a set of leading finance journals) as opposed to an "article in a top journal." The results show that using the top three (JF, JFE, RFS) approach to identify top articles (based on the mean number of cites per article) leads to an RTA error 44% of the time and an ANTA error 33% of the time. I provide similar information for the 15 individual journals and for different levels of citations. I find that at the 90th and 95th percentiles, the RTA error decreases but the ANTA error increases dramatically. I examine the trade-off-between these two errors. My results demonstrate that if someone is interested in identifying top articles then the RTA and ANTA errors associated with a top three journals approach suggest that one should look at a broader set of journals and should examine each article more closely for its intrinsic quality, rather than the general quality of its journal.

The article is organized as follows. Section I discusses the literature and method. Section II presents the results. Section III summarizes and concludes.

I. Literature and Method

I examine the 5,979 SSCI cites (as collected on February 7-12, 2004) for 626 articles published in 1996 in 15 leading finance journals. The 15 journals include 11 of a core 16 set of journals, as defined and studied by Chan, Chen, and Steiner (2002) and four other journals included in other similar studies.

Chan et al. (2002) use Journal of Finance-equivalent sized pages in their 16 finance journals from 1990 to 2001 to rank finance programs. The set of 11 journals from their set of core 16 journals comprises Journal of Finance (JF), Journal of Financial Economics (JFE), Review of Financial Studies (RFS), Journal of Financial and Quantitative Analysis (JFQA), Journal of Business (JB), Journal of Financial Intermediation (JFI), Journal of International Money and Finance (JIMF), Financial Management (FM), Journal of Banking and Finance (JBF), Journal of Futures Markets (JFM), and Journal of Portfolio Management (JPM). I do not include Chan et al.'s other journals [Financial Analysts Journal (FAJ), Journal o[Financial Services Research (JFSR), Journal of Financial Research (J FR), Journal of Business Finance & Accounting (JBFA), and Financial Review (FR)] because these journals are not covered or fully covered by the SSCI during the sample period.

Chan et al. (2002) also rank programs based on the top three of JF, JFE and RFS. They also examine four other journals covered by the SSCI, Journal of Money Credit and Banking (JMCB), Journal of Risk and Insurance (JRI), Real Estate Economics (REE), and Journal of Real Estate Finance and Economics (JREFE). These journals examine a broad range of finance areas that consists of money and banking, real estate finance, and insurance.