Journal Ranking for Biological & Marine Sciences

This purpose of this page is to give an explanation of the various methods which have been used to rank scientific journals, with particular reference to biology and marine biology-related titles. The information should help the reader to assess the value or ranking of journals in their field of research, and to evaluate which ones are the most suitable in which to publish. 

Contents:

Introduction: PBRF and Journal Ranking
Journal Citation Reports
   Impact Factor
   Immediacy Index
   Cited Half-life
   Citing Half-life
   JCR categories for biology and marine sciences
   Reasons that impact factors may vary between journals of similar quality
   Use and mis-use of Impact Factors
Eigenfactor.org
Journal-Ranking.com
 

Introduction: PBRF and Journal Ranking

The Tertiary Education Commission (TEC) is responsible for leading the government's relationship with the Tertiary education sector, and for policy development and implementation. There is some pressure to publish in journals that can be seen to be of high standing in their area, although assessing the quality of a journal objectively and without bias is not easy (especially for specialist titles) as there are many factors involved, some of which are very subjective. In past years, the main resource used to assess journal rankings by the TEC has been Journal Citation Reports (JCR) published by Thomson in their ISI Web of Knowledge database (of which Web of Science forms a part). JCR can be accessed from the database listings on the library homepage under J:

.

Top of Page

Journal Citation Reports

JCR provides Impact Factor (IF) information and is published in two editions, 1) Science and Technology, and 2) Social Sciences. Currently, ISI indexes about  8700 journals (there are over 250 000 journals worldwide), of which 6100 are in the so called ‘hard’ sciences, 1860 are in the social sciences and 1200 are in the arts and humanities. Of these, the JCR Science and Technology edition includes data from some 5900 titles while the Social Sciences edition includes data from 1700 titles. Each annual database collection contains the previous year's publication data (loaded and made available each July) which is the accumulated and tabulated citation data and article counts used to measure various parameters, including:

  What are the largest journals?
    Journals are ranked by the number of published articles

  What journals are most often used?
    Journals are ranked by the number of times they are cited in other journals

  Which journals have the highest impacts?
    Journals are ranked by their Impact Factor (IF) which allows some evaluation and comparison of a journal's relative
importance to other titles in the same category

  What are the hottest journals?
    JCR's Immediacy Index can indicate which journals are publishing the latest highest cited research in rapidly moving fields
 

For each journal title covered in JCR, the following information is collected and/or tabulated
Citation and article counts

• Impact Factor

• Immediacy Index

• Cited Half-life

• Citing Half-life

• Source Data Listing

• Citing Journal Listing

• Cited Journal Listing

• Subject Category

• Publisher Information

• Journal Title Changes

Top of Page

Impact Factor

A journal's Impact Factor is defined as the number of times a journal is cited within the previous two years of publication divided by the total number of articles published by that journal over that period. It is intended to measure how often on average authors cite moderately recent articles from a particular journal.

The term 'article' is undefined but includes review articles. Journals often have referenced, peer-reviewed papers that are not considered 'articles' by Thomson. Letters or editorials are not counted as articles (even if they generate many citations) but citations to them are included in the calculation. Hence a journal which has many letters in its correspondence section and which generate large numbers of citations to them will tend to have an over-estimated impact factor.

Recently, in recognition that literature from some sciences have very long half lives (eg. mathematics, palaeontology), JCR now includes a five-year impact factor calculation.

Immediacy Index

The Immediacy Index of  a journal is intended to measure how often on average authors cite very recent articles from that journal, and hence how rapidly the average paper from that journal is adopted or accepted into the literature.

Cited Half-life

The Cited Half-life of a journal is the calculated point (or age) in years, above which 50% of the citations to the journal are more recent, and below which 50% of the citations are older. This is intended to measure how long, on average, articles from journals continue to receive citations. This will change year by year, and the results are presented by Thomson graphically.

For example, all citations to Nature Genetics in 2004 may be broken down as:
                                          48.4 % of citations are to issues for 2000 or more recent
                                          12.2 % of citations are to 1999 issues
                                          39.4 % of citations are to issues prior to 1999
So in 2004, Nature Genetics had a cited half-life of 5.1 years (the following year it was 4.7 years).

All citations made to American Journal of Mathematics in 2004 can be broken down to:
                                          18.3 % of citations are to issues for 1995 or more recent
                                          81.7 % to citations are to issues earlier than 1995
So in 2004, American Journal of Mathematics had a cited half life of > 10 years.
 

Citing Half-life

The Citing Half-life of a journal is the calculated point (or age) in years, above which 50% of the citations made by authors of articles in that journal are more recent, and below which 50% are older. The intention is to give a relative measure of the age of articles being cited by articles within the journal.

Top of Page

JCR categories for biology and marine sciences

The following Biological/Marine subject categories and corresponding number of journal titles, as at 2007, are included in Journal Citation Reports (note that many biology journals fall into more than one category so will be represented numerically more than once):

Biochemical research methods 56
Biochemistry and molecular biology 262
Biodiversity conservation 24
Biology 65
Biophysics 66
Biotechnology and applied microbiology 140
Cell biology 156
Chemistry, organic 56
Crystallography 23
Developmental biology 34
Ecology 114
Entomology 69
Environmental sciences 144
Evolutionary biology 35
Fisheries 41
Genetics and heredity 131
Horticulture 21
Immunology 117
Limnology 17
Marine and freshwater biology 79
Mycology 17
Oceanography 48
Ornithology 19
Physiology 78
Plant sciences 147
Zoology 115
 

Reasons that impact factors may vary between journals of similar quality

Within each JCR subject category, there is a rough correlation between the journal's Impact Factor (IF) and what is perceived to be journal quality: the inference is that the more prestigious journals will have higher Impact Factors. However, there are many reasons, both deliberate and incidental, why two journals of very similar quality may have different IFs.
 

• Books, book chapters and non-listed journals are excluded from ISI data.

• All citations to non-citable items (letters, editorials) are included as numerators in the IF calculations.

• The denominator of the IF only contains articles designated by ISI as primary research articles or review articles, and this excludes certain sections of
   so called front matter' (such as Nature's 'News and Views' ).

• Publications from some non-English speaking countries, including those with applied biomedical industries, are excluded from ISI data
  (for example, the Cuban National Health System journals published by Editorial de Ciencias Médicas).

• The data is biased toward US publications and English language.

• IFs are often very biased estimates, and the proportion of this bias can vary widely even within the same JCR subject category.

• IFs are not statistically representative of individual articles (ie. they actually correlate poorly with citation rates of individual articles).

• Journals in categories with a high rate of research or developing technologies (eg biotechnology) often contain content that is rendered redundant or is superseded soon after publication, and are therefore more likely to have higher IFs (a high number of citations in a short period of time), higher Immediacy Indexes, and shorter Cited Half-lives.

• Journals where content is more permanent (natural history, taxonomy, mathematics, palaeontology) contain content that is more permanent and less likely to become obsolete, and for these reasons will tend to have lower IFs and Immediacy Indexes, and long Cited Half-lives (they are more likely to be cited long after publication).

• Self citations are included in citing journal data (the number of citations a journal has in its reference lists) but are excluded in ‘cited only’ journal data.

• Journals with many review articles generally have high IFs as reviews are more likely to be widely cited than individual articles of original research,
  regardless of the quality of their content.

• Ease of access (electronic versions) increases IFs.

• A citation may signify utility for the person citing it, rather than relevance (a constant temptation with reference software such as Endnote and Refworks).

• Journals can raise their IFs by publishing fewer articles or by changing the categories of the sections in which the articles are published.

• The citation rate of a single article can determine the IF of an entire journal.

• Long-term studies (particularly those involving several seasons of field or trial results) will have low IFs.

• Studies authored by a group rarely receive individual author citations (eg. the major International Stroke Trial of 1997 only received 1 citation in 1999 according to ISI).

• Some articles have high citations rates simply because they represent flawed science and are being cited as such by critics of the original article (cold fusion and the South Korean stem cell research articles being cases in point).

• Articles are often mis-cited (error rates of 25% are not uncommon) and these errors are replicated by IF data. A 1999 study of a well known botany serial found errors in 45.5% of all citations: half had one error per citation and one citation had eight errors.

Top of Page

Use and mis-use of Impact Factors

Although IFs have been used since the 1950s, it was only in the 1990s that any serious attention was given to them by the wider scientific community. One of the more unfortunate drawbacks of IFs (and one that was probably totally unexpected by JCR’s originator Eugene Garfield) is that it has altered scientists’ behaviour towards publishing their work. Many researchers now seem intent on only publishing in journals with perceived high IFs (ie. the so-called ‘free-ride’ syndrome) at the expense of specialist journals that may be far more appropriate vehicles for their research. Unfortunately for these authors, there is no free ride. It is the citation rate of the articles that generate a journal’s IF, not vice versa. Bibliometric studies have shown no difference at all in citation rates of similar articles published in journals of widely differing IFs (from 0.5 to 8.0), but most authors would still prefer to publish in a journal that is expected to have a high impact factor in the belief that this will somehow flow on to the citation rate of their article. Editors too have been tempted to use IFs to their own end - the editor of a certain medical journal was found to be routinely sending letters to submitting authors requesting that they increase their citations to papers published in that journal.

Thomson employees are charged with designating articles as primary, review or 'front matter' using a variety of criteria (it has been suggested that publishers can sometimes negotiate with Thomson to change these categories if they so wish). The details of how this is carried out are not available to the public, a fact that has been viewed with some concern by authors such as Rossner et al (2007); they noted that the journal Current Biology had an IF of 7 in 2002 and 11.9 in 2003. The denominator of the IF dropped from 1032 articles in 2002 to 634 articles in 2003, in spite of the fact that the number of articles published by that journal actually increased over the same period. There was also concern expressed that citations to retracted articles were counted in the IFs (the case in point being given as W.S. Hwang's stem cell papers published in Science from 2004-2005 and subsequently retracted by the journal but not before having generated a total of 419 citations). The same authors also pointed out that because the IF is a mean, it can be easily skewed by one blockbuster paper. The example given was the initial human genome paper published in Nature which received some 5904 citations as at Nov 2007. Indeed, Nature itself noted that 89% of citations to Nature articles referred to 25% of their published papers. Rossner et al favour the provision of a median IF calculation rather than a mean (the median would typically be much lower than the mean).

For biologists, the risk of putting too great a stock in the value of IFs is greater than it is for some other sciences - in 2001 ISI claimed that 90% of chemistry journals were being covered by their journal selection, but coverage of biology titles over the same period was estimated at only 30%. It has been estimated that the average scientific article will only ever be cited 1.5 times during its life, and up to 50% of articles may never be cited. Which is not to say that IFs don't have their worth when used correctly and in conjunction with other factors, but they are only one facet of providing information on data ‘value’ and are really only meaningful in a comparative sense when used in conjunction with subject- and journal-specific criteria as well as peer review. ISI has become aware of the pitfalls that mis-use of IFs has caused in recent years, and now has a cautionary note on its online ‘help’ site that sounds not unlike a surgeon general’s warning to smokers printed on cigarette packets.

“Citation data are not meant to replace informed peer review. Additionally, careful attention should be paid to the many conditions that can influence citation rates, such as language, journal history and format, publication schedule, and subject speciality. The number of articles given for journals listed in the JCR include only original research and review articles.       Editorials, letters, news items, and meeting abstracts are not included in article counts because they are not generally cited. Journals publishing in non-English languages or using non-Roman alphabets may be less accessible to researchers worldwide, which can influence their citation patterns. This should be taken into account in any comparative journal citation analysis” (JCR Online help “Using JCR Wisely” http://admin.isiknowledge.com/JCR/help/h_using.htm).

Yet just as with smokers, it is proving difficult to change people’s habits. Marketing promotions of new journals will make the most of high IFs wherever and whenever possible, as it is often seen as definitive numerical evidence of a journal’s perceived standing or popularity. Finland directly links state funding of university hospitals to publication points of staff articles (which in turn are based on direct IF comparisons) and German universities have also been guilty of using IFs to calculate funding in a similar fashion. Editors, researchers and authors need to understand how IFs are generated and what their limitations are, because their mis-use can unduly advantage review journals as well as be detrimental towards specialist niche journals.
 

Top of Page

Eigenfactor.org

Another database that ranks journals in the biological sciences area is Eigenfactor.org. which is sometimes seen by users as an antidote to the shortcomings of Journal Citation Reports. See: http://www.eigenfactor.org/about.php  Eigenfactor.org was set up and is maintained by Carl and Ted Bergstrom from the Department of Biology at the University of Washington. This database uses algorithms to calculate influence rankings for over 7000 science and social science journals (in fact the same ones listed by ISI in their Journal Citation Reports) as well other material such as books, newspapers, PhD theses and popular magazines. 

There are a number of features of Eigenfactor.org that make it different to, and potentially more advantageous than, Journal Citation Reports.

• Eigenfactor.org ranks journals in a similar manner to how search engines such as Goggle rank websites, by using networking theory where citations replace hyperlinks.

• The so called Eigenfactor indicates the amount of use any given journal is getting by scholars, while a second measure called Article Influence more closely correlates to the Impact Factor used by ISI.

• As well as citation data, information on journal prices and value are also included, and cost effectiveness of journals is measured.

• Eigenfactor.org takes into consideration the fact that different disciplines have different standards for citation ranking and different time scales over which citations might be expected to occur. By using the whole citation network, Eigenfactor.org tries to allow for more meaningful comparisons between research areas.

• The time factor used in citation data calculation is five years (as opposed to two in Journal Citation Reports) so as to account for the longer citation half lives of journals in disciplines such as mathematics/taxonomy/palaeontology.

• The variances in assumed 'prestige' of journals is taken into account during data manipulation, so that the ranking adjusts for differences in citation patterns among different disciplines (ie. the expected citation rates and half-lives within each discipline).
   

Journal-Ranking.com

A third journal ranking database that includes biology titles is Journal-Ranking.com which is a new resource (launched Jan 2007) run out of Hong Kong University of Science and Technology accessed at:  http://www.journal-ranking.com/ranking/web/index.html
Certain features can only be accessed with registration. The site does not give much detailed information on how rankings are compiled, but the authors state that their intention in setting up was to address the deficiencies of the ISI citation index rankings, which assigns all citations equal weight. Journal-Ranking.com note that a citation by a paper in a top journal may outweigh many citations from lesser journals. Another stated drawback of the Impact Factor (IF) used by ISI is that it does not apportion suitable credit to so-called indirect citations (the example given is that of a notable theory published in a leading maths journal which is then extended to applied journals, and which in turn would be more likely to attract citations than would the original paper).

Like Eigenfactor, Journal-Ranking.com make use of Google's PageRank methodology which uses an extended version of the Pinski & Natin invariant method of ranking and sorting. The graph theoretic properties and axiom analysis of the PageRank model are extended to rank individual journal influences so as to produce a similar ranking order to that of the PageRank algorithm.

Journal-Ranking.com ranks journals by the following parameters

• PII = ranking of the title among all journals

• JII  = Journal Influence Index

• PII  = Paper (ie. article) Influence Index which is the JII divided by the article number multiplied by 1000

• B2  = IF as used by ISI

• B4  = IF measurement using article counts from the preceding four years rather than two as used by ISI

• B6  = IF measurement using article counts from the preceding six years rather than two as used by ISI

References
Amin M & Mabe M, 2000. Pers in Pub 1 (Oct): 1-729
Bloch S & Walter G, 2001. Aus NZ J Psych 35: 563-568
Hecht F et al, 1998. Cancer Genet Cytogenet 104: 77-81 
Seglen PO, 1997. BMJ 314 (7079): 498-502
Rossner M, Van Epps H & Hill E, 2007. J Cell Bio 179 (6): 1091-1092

Top of Page

File last updated: November 28, 2011
Comments and suggestions to :John Lavas