NSERC’s President Suzanne Fortier will be making a presentation on June 03 at the Edmonton meeting of the Canadian Mathematical Society. The unexpected visit is undoubtedly related to the public letter by 16 members of the Evaluation Group 1508, which came on the heels of the open letter to the minister of industry by 327 mathematical scientists, including 27 Canada Research Chairs and 35 fellows of the Royal Society of Canada. Both public letters, as well as various individual interventions, convey serious concerns about the new evaluation procedures of the Discovery Grants. Dr. Fortier’s willingness to address these concerns head on, and in direct discussions with rank-and-file researchers, is surely welcome.
Suzanne Fortier has been touring the country for the past two years trying to garner the support of University Presidents and Vice-Presidents Research for her plans for “reforming” the Discovery Grant program. Her general message, one that she will very likely repeat in Edmonton, is simple and convincing: Her quest for reform is anchored on the following principles.
- The level of a grant should be commensurate with scientific or engineering merit,
- Within a given discipline group, proposals with similar scientific merit should have similar grant levels regardless of the applicant’s granting history with NSERC.
- Bin levels, budget permitting in a given competition year, are expected to be in a similar range from year to year.
- Need to separate the process of assessing scientific or engineering merit from assigning funding.
Who would argue against such general principles? Yet, the research community is up in arms against NSERC’s new ways, and puzzled senior university administrators are hurriedly trying to address the demands of their revolting researchers, who have been filing appeals in droves.
It is high time to initiate a real discussion with Dr. Fortier and with the senior administrators whose support she is seeking. Let us not waste time on the noble intentions and generalities that we all agree upon. And no one questions the good intentions that NSERC’s leadership had in mind when they initiated such a “reform”. Let’s focus instead on the very details where the real devil is, namely the unintended –but by now well documented– flaws of the new Discovery Grant adjudication process, and the way with which it is undermining every one of the above stated principles.
The message of the research community to Madame Fortier should be clear and simple. The new system developed by the Vice-President for the Discovery Grant Program and her staff is seriously flawed. It goes against the very principles it is supposed to uphold. It is much less transparent than the system it replaced. It amplifies and institutionalizes all the flaws of the older system that it was supposed to fix. It trivializes the good work of the Evaluation Groups. It over-empowers executive committees while keeping them somewhat blindfolded. It has led to too many wrong, unfair and reckless decisions in a very short period of time. It has allowed for the circumvention of any notion of accountability.
The anomalies in this year’s competition within Group 1508 for the mathematical sciences provide a long list of illustrative examples of why the new system is broken. They are well documented and will be presented to you in Edmonton on behalf of the Liaison committee. But these aberrations are not only relevant to Group 1508 and the mathematical sciences. They also apply to other Evaluation Groups.
The new so-called “conference model” limits the use of needed expertise, fragmentizes decision-making, prevents comparisons and rankings, disallows calibration, and circumvents any attempt at accountability. The cookie-cutter binning system may work for certain disciplines but does not mesh with the research values and priorities of others. The new system has effectively altered the mandate of the Discovery Grant program from one that supports advanced research to another that is mostly concerned with training. Its evaluation grid is too coarse to make sound and fair slotting decisions. The voting system leads to highly inaccurate and random results. The rule that protect older –and often fatter– grants of certain applicants (those with at least one E) institutionalizes rigidity and a sense of entitlement that are often blamed on the older system. It creates grossly distorted funding decisions (such as those in bin F funded at a higher level than some in bin C).
Another serious problem seems to originate from the newly “empowered” executive committees. Selected by NSERC staff, these ECs have already run the gamut of excesses. They have managed –often unwittingly- to distort and override decisions of the evaluation groups. They have used their absolute powers to dictate success rates. They have managed to break the “sanctity of bins”, and they have proceeded to segregate within the same research community by assigning differential dollar values within the same bin. As I said before, “You know there is a serious problem, when the members of NSERC’s Evaluation Groups are the first to call foul, and announce they are shocked, surprised and offended by the results of the latest Discovery Grant competition — the one they just finished running.”
So, yes Dr. Fortier, we support your principles for the Discovery Grants, and we ask you to make sure that they are correctly upheld. The newly devised system is flawed in so many ways and is not doing the job. Please ask the staff at the Discovery program to listen to the research communities, to take their input into consideration and to go back to the drawing board.
Piece of Mind 
Just returned from the Chemical Society of Canada’s meeting in Montreal, where I attended a session featuring Isabelle Blain. Unfortunately, I lost my temper and left before the end of the meeting. In response to my question about why the DG success rate for “large” universities was about 12% higher than for “medium”, which in turn was higher than for “small”, Ms. Blain told me that this was eminently reasonable because the researchers at “large” universities were just better than the rest of us. She suggested several reasons for this: 1) that “large” universities are more selective in their hiring; 2) that “large” universities have more dynamic intellectual environments than smaller places; 3) that “large” universities have better facilities than smaller places.
I found her responses downright insulting, but it also struck me that her logic contained the self-fulfilling prophecy that is the problem with the “new” DG system. Of course with more money, the environment and facilities will be better-supported. But is that a justification for giving those people still more money? The rich get richer…
I can see that “Excellence of the Researcher” is akin to the medieval concept of “Noblesse”. By virtue of where you are you must necessarily be better than other people who lack your advantages, and hence deserving of still more advantages. I was naive to think that in a modern democratic society we would be evaluated on our own achievements and merits.
The other thing that angered me in Ms. Blain’s remarks was that she denied that the numbers of HQP was the important criterion in assessing this category, but could not give a satisfactory reason why my HQP record was insufficient. I have never had the resources to support many students, but I have placed recent PhD grads into postdoctorals with a Nobel Prize winner in one case, and with a rising star in the US in another case. My MSc grads are almost all working in industry, in one case as the Director of an entire firm. So I can’t see what more I could have done to prove the QUALITY of my HQP – but if quantity was not the criterion why did I not get a higher score?
Other people in the room had similar questions and challenges for Ms. Blain. As I said, I lost my temper and spoke harsh words before leaving, but I don’t regret expressing my thoughts. These bureaucrats have to see and hear what their messing about has accomplished – they have ruined a marvellous system that gave Canadians excellent value for money in terms of technical output and the education of young people.