… wrote a Canadian scientist describing reactions to the posts of “Piece of Mind” on NSERC’s new evaluation system. I am reproducing (with his/her permission) the following email: You are definitely not alone!
“First of all, I am sure that I speak on behalf of the overwhelming majority of our Canadian colleagues in saying a big THANK YOU for the time, thought, energy, creativity and courage that you are putting into your posts on “Piece of Mind”. I have been reading your posts very carefully, particularly those about NSERC, and I agree with all the criticisms that you have voiced on the binning system.
I have seen people reacting to the results of the 2009 competition with a mixture of anxiety and resignation. Colleagues’ reactions to receiving serious cuts in their grants following a period that corresponds to their five best years of research and student supervision, were one of total disbelief, followed by anger. A colleague in the mathematical sciences got his NSERC grant cut by 100%, soon after receiving an invitation to speak at the International Congress of Mathematicians (ICM) in Hyderabad, India. I am told that this is the ultimate recognition of a mathematician’s work in the past 4 years.
Of course, it is not enough to criticize; one must propose viable solutions to improve the system and that is precisely what you have done. I cannot thank you enough for that. I won’t repeat the excellent arguments that you have made on the absurd and deeply troubling consequences of a system where decisions are made by small groups of five, within the rigid framework of the “binning” system (the word itself makes me cringe). The volatility and absurd consequences of such a system, where the median vote carries so much weight, are manifest to anyone who has had any serious experience with the complex process of the adjudication of grants.
I am very disappointed by NSERC’s responses to your posts, particularly by the response to the post in which you propose a formula to improve the “binning” mechanism. Contrary to what NSERC claims in its response, it is not possible to “calibrate” things so as to ensure real consistency within and between EGs when the groups are so small. Good decisions are made following an in-depth discussion by a group scientists whose combined expertise and experience is large enough, and who have the overall picture of the pool of applicants in any given year. This cannot be achieved with small EGs.
Furthermore, the HQP variable combined with the binning system is leading people to play an unhealthy “numbers” game, and is unjustly penalizing colleagues in smaller universities, as well as those colleagues who work on difficult, long term projects.”