The observer couldn’t believe what she was hearing as she watched one of the subgroups consisting of 5 panelists in NSERC’s new conference model. “The applicant has a couple of papers in the Journal of … . Does anyone know anything about this journal?” said one. “Oh this must be one of these obscure journals”, replied another panelist.
The observer –who is also on the panel but not on this particular subgroup– couldn’t control herself and asked for permission to provide the information she had. The journal in question is one of the top journals for the discipline, information that could be crucial to the case. The NSERC staffer present denied the permission!
Why is an academic panelist, donating many hours of hard-earned time to NSERC, asking permission of an administrator to correct an error in the discussion?
Welcome to the rapidly warping universe of our granting councils, where bureaucrats are starting to rule supreme, and where scientific evaluation is becoming an exercise in policing, legalese and loopholes.
The bureaucratic grip on the Tri-council’s granting processes is becoming so alarming that it is time to start drawing the line. Leaving aside the myriad Tri-council programs that are now solely decided by bureaucrats (a list of which will be provided in a future post), the bureaucrats have now honed in on what used to be the jewel in the crown of Canada’s granting system: The NSERC Discovery Grant program.
First, the bureaucrats introduced a “binning system”, which is built on a fragmented decision-making process, the so-called conference model. It is this fragmentation, which is causing the volatility and the lack of uniformity and consistency that is leading senior scientists like Don Fraser to speak out.
With their implementation of this binning system having HQP supervision at its core, the bureaucrats have effectively changed the very nature of the DG program, from one that promotes and supports research excellence to just another one of their many programs that sponsor training. A list and an evaluation of these programs will also be given in a future post.
Even the best-intentioned can’t predict all the consequences of their actions. Let’s hope that the bureaucrats did not foresee the negative impact on early-career scientists, and on universities without or with minimal graduate programs. Did they not anticipate that by adding three scores (for Research, Quality of Proposal and HQP), a scientist in HQP-deprived Regina would have to be a much better researcher than one in HQP-rich University of Toronto, in order to get the same level of funding?
After creating an absurdly rigid binning system, bureaucrats next try to dictate the so-called funding to binning map, sometimes overruling the panelists’ recommendation. They aim to enforce cut-offs below which there is no funding (again having a dramatic impact on smaller institutions), and then they interfere by trying to push down the upper half, supposedly to improve the standards.
But the absolutely worst part in this new era of bureaucratic rule may be the amount of “policing” of the review process that is coming with it.
Indeed, listen to the tale of this absent-minded mathematician who had forgotten to include and mention one of his best papers in his grant application. One frustrated panelist, who knew how good the applicant was, jumped in joy when he saw that the paper was mentioned by one of the external reviewers. Like a lawyer looking for a loophole to make his case, this piece of evidence can now be used in the evaluation/trial.
Further examples of excessive control include citations of particular papers, or information about specific University programs, which, unless explicitly stated in the application, cannot be used by panelists. In one instance, a trivial google search showed that an applicant was supervising 50% of all graduate students in his small department. However, since this information was not included in the official documents nor mentioned by an external referee, and since the HQP numbers looked small compared to other applicants, he received a low HQP rating. Kafka would have loved the absurdity of this exercise: knowingly, panelists make the wrong decision, forced by purely bureaucratic rules!
As well as the new importance of the art of finding loopholes at NSERC, we now hear about the relevance of mastering the correct (i.e., allowable) vocabulary in order to make a point. For example, NSERC does not allow leadership or administrative duties to be considered as a factor –for example as a reason for delay in research activities– in reviewing a grant applicant. The loophole here is to talk about the “impact of the applicant” on the community during his/her tenure on the job.
This system is set to repeat itself every year. The process for appointing evaluation group members is completely opaque, but NSERC’s bureaucratic goals and ways are at odds with obtaining qualified reviewers. In addition to issues of balance on gender, geography, and language, there is a reluctance to appoint reviewers from departments that have a large number of applicants, as this raises the number of conflicts of interest. The inevitable result is that the largest, and often most highly-rated departments in the country are under-represented on review committees.
Compare this to what happens on NSF panels, where top scientists convene and discuss in ways they see fit the merit of each case, with no interference whatsoever from the program officer, who also happens to be a scientist seconded from his/her institution. There is no controlling and policing of the proceedings by a bureaucrat.
We ask the bureaucrats to pull back and to make way for a process where reigns an atmosphere of trust in the wisdom, integrity and expertise of the outstanding scientists who volunteer their time and effort for a most valuable endeavour. It is time to draw a line in the sand and call on Canada’s scientists to regain control of the country’s scientific evaluation process.