IMPACT FACTOR CONTROVERSY

I am glad that Professor Asghar Qadir at last has come out in public with his point of view vis-a-vis the controversial Impact Factor (The News, August 7, 2000). Soon after my first article on the Impact Factor, Rating Pakistani Scientists, was published (The News, October 28, 1999), I wrote an open letter to Professor Asghar Qadir asking him to give his professional opinion. There was no response from him until my second article, The Impact Factor and Mathematicians (The News, July 19, 2000). It is important that mathematicians within and outside Pakistan know the stand he has taken. 

My first article highlighted a number of crucial defects in the criterion. Professor Qadir's admittance that the criterion of Impact Factor has many flaws makes my case for rejection of the criterion more justifiable. 

I need not repeat myself here in highlighting the defects, deficiencies, and deceptions of the criterion. However, I would like to add something new in my arguments against the Impact Factor. Before that I wish to reply Professor Qadir's query: I obtained the figure of 233 mathematicians from the compendium published by the UGC in 1999. Incidentally, I borrowed it from his office. The figure is correct.

Those who are for the Impact Factor, consistently put forward the following arguments: Yes, there are many deficiencies, but there must be a standard yardstick to gauge the work of a scientist, and the Impact Factor is the best yardstick under the circumstances.

I have substantiated with numerous examples in my first article that the criterion is defective, deficient, and deceptive. No body has given counter arguments so far, not even Professor Qadir in his article. In the first place, using the Impact Factor as a "yardstick" for gauging scientists' work is absurd. The sentence "but there must be a standard yardstick to gauge the work of a scientist" is essentially a politically motivated statement.

The idea of using the Impact Factor to catalogue Pakistani scientists in a book, called The Leading Scientists of Pakistan (produced by the Pakistan Council for Science and Technology) has probably descended from heaven. It has thus become a holy cow.

If the real intention was to have a standard yardstick to rate Pakistani scientists, as it is claimed, then the usual and proper way is to form task committees of scientists representing respective scientific communities. They should have been given ample time to come up with a criterion for each branch. Such a criterion would have taken care of all the relevant parameters. Instead, the "yardstick" has descended from heaven in the form of a book, The Leading Scientists of Pakistan, produced by the Pakistan Council for Science and Technology.

There seems to have some ulterior motive behind. The very purpose of using this criterion of the Impact Factor. Professor Qadir has compared the Impact Factor with Democracy. This seems a naive comparison to me. I think it is wiser if one compares it with the situation in undivided South Asia, when English suddenly replaced Urdu as the court language and many scholars all of a sudden became munshis. The Impact Factor has the same rationale and potential.

I have exchanged ideas on this subject with a number of scientists in Pakistan and abroad. Most of them have endorsed my viewpoint that the Impact Factor is not the right for gauging one's scholarship. A couple of eminent scholars from abroad have totally rejected it. One such scholar says that it is too bizarre that one's status could be determined by the arbitrary assignments of numbers to the Journals in which one happened to publish.  The Science Citation Index is sometimes used to give an indication of status of an individual by considering the number of citations of that individual's papers, but he has never encountered the method of assessment which is used in The Leading Scientists of Pakistan. The use of the SCI Journal Citation Reports is an example of a seriously flawed basis for making comparisons.  The only real criterion of an individual is quality and that, he says, does not admit of simple numerical assessment.

Another foreign scholar says that the document "Leading Scientists of Pakistan" produced by the PCST is certainly very disturbing and also surprising.  He says that most intelligent scientists and administrators are well aware of two facts: one, that we do not yet have reliable bibliographic measures for comparing or making absolute ratings of the value of the work done by research workers; two, that in any event, bibliographic measures appropriate in one field are inappropriate in others.  He adds that an impact value based on the simple measurement of how many times a journal is cited makes no sense as a measure of the quality of the papers published in it, let alone the quality of the mathematicians publishing there.

Mr D.A.Pendlebury, the Manager of Contract Research at Institute of Scientific Information (ISI), Philadelphia, states on behalf of the inventor of the Impact Factor, Professor E.Garfield, that the Impact Factor was devised by ISI to compare journals within the same field -- not to compare the work of individuals. One does not really measure the Impact Factor of individual papers.

The application of the criterion has given misleading results. There are many Pakistani scientists who are well placed internationally but their Impact Factor is low, whereas there are many Pakistani scientists whose Impact factor is higher than the Nobel Laureates, Fellows of the Royal Society, and Field Medallists. There must be something wrong here.

Professor Muzaffar Ali Qureshi has said something very relevant in the 7th paragraph of his article Fashioning a New System, (The News, August 4,2000. He states " We see their (scientists) experiments reported in foreign journals which they (scientists) like to claim promotions and medallions. But we (social scientists) do not see their impact in improving industrial production and product quality". 

Before the use of the criterion of the Impact Factor, scholars who received national and international awards were genuine recipients with may be a few exceptions. Since the use of the Impact Factor in 1999, the situation has deteriorated. In fact, some internationally recognised scholars are outcast because of this "yardstick" and some undeserving ones have been awarded. Scholarship does not and should not hinge on one thing only, that is, the quantity and quality of research papers only. There are other qualities as well which makes one a good scholar. Competence and eminence are not necessary and sufficient to each other. Pakistan's number one need is good and honest governance, not the quality and quantity of research papers.

If one insists on having a numerical formula to rate Pakistani scientists, then call a genuine debate on the issue and devise some reasonable criterion for each branch separately.