EFFECTS OF HEC ON SCIENCE IN PAKISTAN

Excellence in science and technology depends upon the quality of education, especially at higher levels. The Higher Education Commission (HEC) realized that without developing education in general and science and technology in particular, our country will continue to lag behind.

HEC has thus brought about a paradigm change in the policy on tertiary education in order to bring about a change in the quality of our education in general and science and technology in particular.

A greater proportion of our GNP is now being spent on HEC itself, on “science projects”, “scientific laboratories” (some of which can easily be converted into small restaurants), and  “awards and rewards”. Social sciences are completely ignored. Many branches of science have simply been considered useless through propagating a utilitarian approach. Some branches of mathematics have been annihilated. A typical scientific cult has been created through appointments with huge salaries. Too much interference in the affairs of universities has created cronyism, which has destroyed the scholarship and merit by and large.

While we are spending a huge amount in these endeavors, we would like to make sure at the same time that the money spent is well utilized and the objective is achieved. We therefore have to evolve an adequate system of assessment.

One way to assess the quality of our highly educated work force is to monitor their research. The HEC uses an index called the Impact Factor to measure the quality of research in science. Since HEC and the Pakistan Science Council often use this data for faculty assessments, it is but natural to expect that it should give us a fair assessment of one’s scholarship.

However, it is evident now that this index gives a misleading impression of the relative standing of certain scientific journals.

Quantifying scholarship, in my opinion, is not an easy assignment.  But we must also at the same time remember that scientifically developed nations must have evolved some kind of system, which appreciates quality amongst scientists and technologists and discourages substandard scientific and technological research.  Perhaps this is one of the reasons that the money and effort spent upon science and technology in these countries is paying back effectively. Of course it is reasonable to think that this is why they are doing so well in science and technology. 

Below I would like to describe, taking the subject of mathematics as an example, why the Impact Factor, as it is calculated today, is not suitable for scientific journals and I would like to open a discussion on how to change the present method of IF calculation to reflect the realities of research.

First, let me explain how the impact index is computed. The IF of a journal for year Y (say, for Y = 2003 is the ratio C/N, where C is the number of citations during year Y (i.e., 2003 in our example) of papers published in a particular journal in years Y- 1 and Y-2 (i.e., 2002 and 2001). N is the number of published papers during these two years (Y – 1 and Y – 2) in this journal.

The controversy over the use of the Impact Factor as a criterion for ranking scientists began with the publication of a list in 1999 by the Pakistan Council of Science and Technology (PCST) entitled The Leading Scientists of Pakistan. PCST published a similar list the following year entitled Scientific Research in Pakistan.

A campaign against the use of the Impact Factor (IF) to rate scientists was launched as early as in 1999 when I returned from Brunei Darussalam after about five and a half years abroad and wrote an article against it in The News on 28 Oct 1999. Another article by me opposing the Impact Factor also appeared in The News on 19 July 2000.

On 19 August 2000, my letter was published in The News in response to an article by Professor Asghar Qadir in The News, favouring the government's use of the Impact Factor to rate scientists. Another critique against the Impact Factor by Dr Shahid Siddiqui was also published in Dawn of 6^th May 2001.

On 3 April 2001, I was invited to speak on the pernicious effects of the Impact Factor on scientists and science in Pakistan at a seminar at the Ghulam Ishaq Khan Institute. I opined that the use of the Impact Factor would adversely affect the already poor state of science in Pakistan. The seminar generated an interesting discussion during the question and answer session.

In August 2001, the use of the Impact Factor to measure the worth of scientists in Pakistan was criticized at the forum of the 2^nd Pure Mathematics Conference 2001 in Islamabad. Mathematicians at the conference objected to the fact that a criterion, which was originally designed to measure the worth of scientific journals, was being used to measure the worth of scientists in Pakistan. This they said, in effect rendered mathematicians as the least productive of scientists in Pakistan. The Impact Factor was denounced as an inaccurate and poor attempt to quantify the subjective matter of praising the worth of a scientist.

Meanwhile in January 2001, I approached the Federal Ombudsman and filed an application against the government's use of the Impact Factor to rate scientists. A hearing was thus held in which both sides argued their respective viewpoints. Finally in September 2001, the Federal Ombudsman issued its verdict in favour of the complainants.

In its findings dated 26 September 2001, the Federal Ombudsman wrote: "...it is acknowledged that the Impact factor criterion has its limitations, which are all the more significant in the case of disciplines of physics or mathematics. It would certainly be unfair that those in the aforementioned disciplines be judged wholly and solely on the basis of this criterion for the purpose of promotions, awards, research grants, etc. The same would amount to injustice and hence, mal-administration as defined in Article 2(2) of President's Order No 1 of 1983. It is therefore recommended that the Impact Factor criterion may not be taken as the sole criterion for the assessment of individual scientists, specially those in the discipline of mathematics..."

Proponents of the Impact Factor readily admit that the use of the criterion to assess the work of scientists has its defects. The then minister of science and technology, Prof Atta-ur-Rahman, had himself admitted "the drawbacks which the Impact Factor assessment process has" in a reply letter dated 10 August 2000 to me regarding the Impact Factor.

The then Dean of the Faculty of Natural Sciences, Quaid-i-Azam University, Professor Qadir, in his article in The News published on 7 August 2000 advocating the use of the Impact Factor, also wrote that there were "many problems" associated with the Impact Factor.

The main counter argument of the proponents of the Impact Factor is that a rough yardstick for assessing the quality of scientific productivity in Pakistan is better than no yardstick at all. Both Professor Atta-ur-Rahman and Professor Qadir have stated this.

Critics however strongly believe that the negative impact and damage to science arising from the Impact Factor's "drawbacks" and "many problems" will more than counter any advantage which the proponents say using the Impact Factor will have in Pakistan. I have been looking into the question of the validity of information gleaned from the Journals Impact Index and have noted that, for the mathematical community at least, this index gives, in some cases, a misleading impression of the relative standing of journals. This data is now being used by our universities in faculty assessments for financial and academic awards, for awarding projects, etc.

Researchers who work in mathematical areas, which are though very theoretical but have some resemblance with applied sciences, have wider range of journals to publish in them. Not only that it increases the probability of getting a paper published but also the process of getting it published is much faster than the journals which are exclusively meant for mathematics. These journals have much greater Impact Factors than the mathematical journals not because they are qualitatively better but because they have a wider readership and the time spent from acceptance of a paper to its publication is much shorter.

Also, PCST uses the list of Impact Factors published by Thomson Company, which lists only 255 mathematical journals. Whereas the MathSciNet of the AMS has created its own Citation database which contains more that 1530 journals with non zero impact factors.This puts mathematicians in general and pure mathematicians in particular at a very disadvantageous position. As a consequence, the number of young researchers in pure mathematics is on the decline. Mathematics, especially fundamental research, has to be saved. It is with this aim that I am trying to correct the existing system of evaluation. 

At the 10^th National Seminar organized by the Pakistan Mathematical Society, I explained what the impact factor is and described why the impact factor, as it is calculated today, is not suitable for mathematical journals and how to change the present method of impact factor calculation to reflect the realities of mathematics research. Ms Aneesa Tahirkheli, the State Minister of Education, who was the chief guest at the seminar, agreed that the use of Impact Factor needs some modifications to suit the very nature of mathematics. She agreed to form a group of representatives from the Ministry of Education, the Higher Education Commission, and the Pakistan Mathematical Society, to sit together and help bring the necessary changes for improving the system of assessment of education and research in mathematics.

The deplorable state of mathematics in general and pure mathematics in particular is worsening rapidly. The wrong policies of HEC towards mathematics have already started revealing the damage caused to the proper growth of mathematics. Recently introduced condition for PhD students to produce at least one research paper with non-zero impact factor will deter students from doing PhD from Pakistan universities. A PhD degree from a good foreign university has become more attractive for they require less rules and consume less time than a degree from a Pakistani university.