Finally, the long-awaited announcement from the Council of Canadian Academies: “The Minister of Industry, on behalf of the Natural Sciences and Engineering Research Council (NSERC), asked the Council of Canadian Academies to examine the international practices and supporting evidence used to assess performance of research in the natural sciences and engineering disciplines.”

That’s weird! A blue-ribbon panel for that?

We thought this was the committee that was supposed to review the adequacy of, and make recommendations about,  the funding in the various Evaluation Groups (EG) within NSERC’s Discovery Grants program.

The terms of reference give more hints:

What do the scientific evidence and the approaches used by other funding agencies globally have to offer, in terms of performance indicators and related best practices in the context of research in the natural sciences and engineering, carried out at universities, colleges, and polytechnics?
More precisely:
1. What existing qualitative and quantitative indicators and metrics are relevant to budget allocation in the context of support for research in the natural sciences and engineering, and how can they be categorized (e.g. shelf life; cross-disciplinary and international comparability; relevance to interdisciplinary vs. focused disciplinary areas; and applicability to emerging vs. established research areas)?

2. What are international best practices in the construction, methodological review, and use of quantitative and qualitative indicators for research evaluation and budget allocation in support of research in the natural sciences and engineering?

3. Considering the foregoing, and in light of the Government of Canada’s Science and Technology Strategy and NSERC’s objectives for the support of research, what key considerations (e.g., risks, advantages/disadvantages, behavioural and institutional consequences) and principles emerge in determining defensible use and balance/ weighting of performance indicators/metrics for budget allocation?

Have you ever seen “more defensive” terms of reference? What about if the panelists have few new ideas on their own?

Oh! Not to worry. With Rita Colwell in charge, something good will happen.