Enhancing Peer Review
Below is an email sent to all ASHE members from 2020 ASHE President KerryAnn O'Meara about enhancing the peer review process:
January 21, 2020
January 21, 2020
Happy New Year! I hope 2020 brings you new opportunities, challenges, friendships, and joy.
The theme of the 2020 conference in New Orleans is “Advancing Full Participation.” We are excited to explore this theme leading up to and as part of our 45th annual conference. As ASHE president, I would like us to think about advancing the full participation of our members throughout the year in multiple ways, such as enactment of the strategic plan, development of new ethics policies and practices, and finding more ways to connect our scholarship with efforts to improve equity and inclusion in higher education.
As a scholarly association, one of the most important things we do as members is support each other’s scholarship through peer review and feedback. Through the strategic planning process and preparations for the 2020 conference, we realized there are several ways to enhance the conference peer review and feedback process through full participation of our ASHE community.
Several important contexts shape this observation:
(1) The number of proposals ASHE receives each year has increased at a greater rate than the number of reviewers. The increase in the number of proposals is good news and a sign of participation. However, only about half of those eligible to review research and scholarly papers (e.g., those members with an earned terminal degree) are actually reviewing, contributing to the high number of reviews assigned to those who volunteer.
(2) Feedback from strategic planning and conference evaluations suggests we can improve the proposal review process. Evaluations showed that some proposal authors submitted papers without receiving substantive feedback in their reviews.
(3) Although most authors submit 4 or fewer proposals, some submit more than 10 proposals, despite the fact that we limit ASHE members to appearing on the conference program 5 times in any role (presenter, chair, discussant, etc.).
(4) Feedback from strategic planning and conference evaluations also suggests that there were conference sessions where discussants did not receive completed papers, and where participants desired stronger feedback from discussants.
To improve peer review in ASHE, there needs to be full engagement in proposal reviews, paper quality, and discussant feedback from our community. Doing this will increase the pool of reviewers to ensure every research and scholarly proposal is assigned to at least one reviewer who is familiar with the method and at least one who is familiar with the content of the paper. Building on Past President Professor Kris Renn’s 2019 invitation for our field to engage in more “generous thinking,” I think we can step up to engage more fully as a community in improving our peer review process.
I have worked with our executive director, board, and 2020 program co-chairs to consider strategies to enhance participation in peer review, with the goal of enhancing the quality of reviews and feedback provided. As part of the 2020 proposal submission and review process, as well as the period leading up to the 2020 conference we will launch the following strategies:
(1) To increase the number of reviewers, and thereby reduce the number of reviews completed by each individual, we will ask every ASHE member with an earned doctorate/terminal degree who submits a research and/or scholarly paper, regardless of order of authorship, to complete a minimum of 3 proposal reviews.
(2) To ensure reviewers are equipped with information and tools necessary to read proposals and provide quality feedback, we will have webinars and resources for our members on quality reviews—with concrete examples of how to structure constructive feedback. Additionally, to enhance reviewer reliability and give authors better substantive feedback, we will work to strengthen the clarity of the proposal review rubric and guidelines for comments to authors and section chairs who make the accept/reject decision.
(3) We will limit proposal submissions to 4 proposals per person across all proposal types. As in past practice, we will still allow members to be on the program 5 times (e.g. members can have 4 accepted proposals and serve as a discussant or chair). Pre-Conference proposals and sessions are not included in the 5 time appearance on the program count, only sessions at the main conference.
(4) To enhance the quality of discussant feedback—while respecting different strategies for providing feedback--we will strengthen guidance given to discussants via a recorded webinar and resources and strengthen guidelines requiring presenters to submit papers in reasonable time for discussants to review them.
I want to encourage participation from ASHE members in all of these strategies. Here are a few ways:
-Whether or not you submit a proposal to the conference, we hope you will engage somehow as a reviewer, chair, or discussant.
-If you have examples or thoughts on excellent feedback that you have received via proposal review or from a discussant to share, please let myself or Corbin Campbell orJulie Park (Co-Chairs, 2020 Program Committee) know so we can highlight them.
-If you have a strategy for being a good discussant that has been praised by colleagues and are willing to share, please let us know! I saw several excellent models at the 2019 conference—all different, but all providing useful feedback and discussion.
We look forward to preparing for and engaging in New Orleans in 2020 as full participants as well. Soon we will share plans for a webinar that leads us into the complex history of NOLA, land acknowledgement, and what some of this has meant for access and success in higher education.
The Call for Proposals will come out in early February, proposals will be due in late April, and notifications will be sent out in late June.
Thank you for being a member of our ASHE community. I hope you will accept this invitation to enhance the feedback we provide to scholars on their work and ultimately to strengthen the value we provide to members as part of our peer review process.