Computing Ed Research – Guzdial's Take

2 Comments Add your own

• 1. Monica McGill  |  June 28, 2022 at 10:11 am

  Hello! Thanks for sharing this.

  Will this information be added to the current ICER website for 2022 and for future conferences going forwarded? It will make it easier and consistent to share.
  Can authors self-nominate? Self-nominations may open this up to a greater diversity of both authors and papers for consideration.
  Given that ICER seems to have only recently started to open up to broadening participation among researchers, has consideration been given to what this means about what types of quality work have not been lifted up as high as others and what this means for this award? (I know, tough question, and you likely already considered–but I’d love to hear how consideration is being made/operationalized–particularly with no predefined rubric.)

  Thanks!
  Monica

  Reply

  • 2. Mark Guzdial  |  July 9, 2022 at 9:52 am

    Response from Quintin

    Dear all,

    This is a second invitation to nominate for the ICER Lasting Impact Award, details of which are at the foot of this message. Note there are two kinds of nomination – quite separate:

    You can nominate a paper to be considered by the judging panel for the award.
    You can nominate yourself to be a judge on that panel.

    Monica McGill helpfully raised a number of questions on the SIGCSE Members list about the original invitation, and I’ve included the response I made to her on that list below, as others may well have been uncertain about the process as well. The original invitation is below that.

    Please do nominate a paper. Of course this is serious, ICER is an important conference, and awards are significant – but – they can also be warm-hearted and an opportunity to celebrate the work of our community – those whose work has really meant something to us, has enriched our lives, our understanding and our own research, our teaching and hence the lives of our students, and our passion for this work. You will know – and you can express this in your nomination. Go for it! Remember the crazy, off-beat, John Henry award, now of ICER history, and think – “To which ICER paper from over 10 years ago should we really have given the John Henry award – and the associated headgear – because, well, just look at the influence it’s had on me and others?”

    Deadline: next Sunday 17th July AoE.

    warmest regards, and looking forward to receiving your nominations,

    Quintin, and the ICER steering committee

    ——————————————

    (Response to Monica’s message, hopefully with clarifications (!), on the SIGCSE members list)

    Many thanks to Monica for raising these questions. The format and rules for the Award were approved by the SIGCSE Board last year and we are just arranging an ICER landing page, separate from pages for any particular instance of the conference, where these kinds of policy documents can be posted.

    While sending this post which I hope will address Monica’s questions – may I take this opportunity to remind everyone to consider nominating a paper, OR nominating themselves to stand as a judge – hopefully any uncertainties anyone may have had over the original invitation, appended at the bottom of this message, will be clarified below.

    Can a person nominate their own paper and provide rationale or are only judges nominating the paper and providing rationale? If a person can nominate their own paper, how do they do so? If a person can nominate another person’s paper, are there restrictions on current or past students and/or close kin? If only judges nominate, then how do you ensure comprehensive coverage of potential papers outside of the judges area of expertise?

    In the approved Award rules, there can be no self-nominations. No member of the Steering Committee can nominate a paper. Anyone else can nominate a paper. As noted in the invitation to nominate, below, this requires supporters and a rationale.

    The issue of paper nomination is entirely separate from nomination of judges, which IS a self-nominating process. We are running a little late for this award and so put these two processes together in one email – and this has perhaps been confusing.

    Note also that the judges may choose not to make the award at all, if they find that the papers nominated do not meet the description of the award.

    Given that ICER seems to have only recently started to open up to broadening participation among researchers, has consideration been given to what this means about what types of quality work that have not been lifted up as high as others and what this means for this award?

    This is an ICER award, approved by the SIGCSE Board. Nominated papers can only be drawn from the pool of papers accepted to ICER in the relevant period. Any paper within that boundary can be nominated, and the nominators can make any case for impact that they deem worthy – we have no preconceived notions about what constitutes impact per se, and are thus open to all nominations from the community.

    Given that no pre-established rubric will be used, how will the judges ensure that their personal biases and positions in the community will not sway the outcome? (Isn’t best practice to use a rubric when grading and reviewing papers in general to help offset these biases and ensure all are judged more equally?)

    It is exactly because this is a new award, and these are difficult decisions to make, that we have not jumped to a rubric. The judges, who can step forward from any part of the community, will clearly have to consider these issues carefully in coming to a judgement, and their rationale will be made public, as I noted in the initial email, as will the names of the judges. We aim for the process to be as transparent as possible, and we will invite discussion afterwards such that the approach taken can be improved in the light of feedback received.

    Any organisation is only as good as the folk who choose to stand up and serve, of course. How do any of us work to avoid bias in what we think and do? I am confident that the judges, whoever they are, will strongly challenge one another in all of their thinking in coming to a decision. And the rationale for that decision will be published at the end of the process.

    Thanks. I’m definitely not opposed to a lasting impact award and I appreciate the volunteers who are working to make this happen.

    I’m only concerned about both the operational practice of designating the award and doing so in a manner that considers papers that may have provided significant impact to others across a very diverse community and set of learners. The organizers have likely already thought about these concerns as well. It could just be me, but they aren’t apparent to me in the call.

    As I say, anyone can nominate another’s papers (with the caveat that the ICER Steering Committee cannot nominate, and my papers cannot be nominated), and separately, anyone can step forward as a judge.

    Again, many thanks to Monica for raising these important questions,

    AND – please do go ahead and nominate a paper, OR nominate yourself as a judge!! 😉

    with best regards,

    Quintin.

    Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed