Proposal Review Process

To provide greater transparency on the 2024 General Conference and Pre-Conferences proposal review process, we are outlining the steps in this 2024 proposal review process. As is a regular practice, Program Committee members ensure consistency and clarity in reviews submitted.

Assignment of Reviewers

  • Beginning on the submission deadline date, Section and Pre-Conference Chairs will review applications for reviewers to their respective section or pre-conference. They ensure applicants are current ASHE members and meet the requirements to be a reviewer (e.g., have a terminal degree, are a doctoral candidate, are a doctoral student, or have significant research experience) and have provided enough information (e.g., recent publications, methodological expertise) to allow them to appropriately match them to a proposal(s).

  • Once volunteers are accepted, Section and Pre-Conference Chairs will begin to review proposals. This typically begins with a skim of the title, abstract, keywords, and the proposal’s suggested reviewer pool. Chairs will begin to match reviewers to proposals. The system shows chairs the number of reviews each person has volunteered to complete across all sections and pre-conferences and limits the total number of assignments to the maximum the person signed up for or 12 if they volunteered for more than 12.

  • If a Section and Pre-Conference Chair is unable to match a proposal to a reviewer in the applicant pool, they may reach out to someone who is qualified to be a reviewer to ask them to serve. This often happens with niche methods or research topics.

  • The guidance provided to chairs regarding reviewer assignment is: “While having all three reviewers with methodological and topical expertise for each proposal is ideal, it’s seldom possible. We suggest that you assign at least one person who has methodological expertise and one person who has topical expertise to the proposal. If these qualifications are met, your third reviewer may be more of a “generalist.”

  • For 2024, the reviewer assignment criteria are as follows:

    • Interactive Symposiums, Self-Designed Paper Sessions, Research Papers, Scholarly Papers, and PVDS sessions should have at least 2 doctoral/terminal degree holders (e.g. PhD, EdD, JD, etc.) and may have one doctoral candidate or a member with practical expertise relevant to the proposal who does not have a terminal degree (e.g., a policy maker or policy researcher with 10+ years experience).

    • Posters and Works in Progress sessions must have at least one doctoral/terminal degree holder.

  • Once Section and Pre-Conference Chairs have completed reviewer assignments (over the course of about a two-week period), the Program Committee Co-Chairs and ASHE Staff review assignments. This is a check to ensure reviewers meet the established criteria and no reviewer is assigned to more reviews than they volunteered for and no more than 12 overall.

  • Also during this process, if a Section and Pre-Conference Chair is listed on a proposal in their section, the proposal is assigned to the Program Committee Co-Chairs for reviewer assignment in accordance with the ASHE Conflict of Interest Policy. In this instance, Program Committee Co-Chairs will also make an accept/reject a recommendation to the chairs based on reviewer feedback; the ASHE Staff will mask the reviews in the Conference Portal so that the respective chair will only have access to the feedback but not information about the reviewers.

Peer Review

  • Once reviews are sent out to reviewers, reviewers will once again have the opportunity to return reviews that they either feel they are not qualified to review (e.g., there was a mismatch in the reviewer assignment) and/or they believe they have a conflict of interest (e.g., they recognize the work or writing of a close colleague). 

  • Chairs monitor reviewer progress throughout the review period of about two weeks. When reviewers are unable to complete reviews, chairs tap into “back up” reviewers as needed. These reviewers must also be qualified to review.

Making Accept/Reject Decisions

​After chairs have made their accept/reject decisions and created sessions, the Program Committee Co-Chairs and ASHE Staff review any outliers (e.g., proposals above 4.0 that were rejected and proposals below 3.5 that were accepted), ensure allocations were met, and review proposals that chairs wanted to accept but did not have space. 

  • When the review period is closed, chairs begin to review the reviews. This includes determining alignment in a single review between the scores, comments, and recommendation as well as between reviews. If there is a mismatch within a single review or between reviewers, the chair may either accept the recommendation of two of the three reviews or assign a fourth reviewer.

  • If in reviewing comments, chairs are made aware of potential plagiarism or inappropriate comments, this is immediately referred to the ASHE Executive Director for investigation. In cases of the latter, the review may be deleted and assigned to a fourth reviewer; the other reviews of this reviewer will also be examined.

  • The guidance provided to chairs is: “You should strongly consider proposals that receive a score of 4.5 and higher for acceptance. Proposals that receive scores of 3.5 and lower should be strongly considered for rejection. Proposals which receive scores of 3.51 to 4.49 should be evaluated more closely.” In instances when a chair wants to reject a proposal with an average score above 4.0 or accept a proposal with an average score below 3.5, they must consult with the Program Committee Co-Chairs.

  • Section chairs are provided with an allocation based on the number of submissions to their section. For example, if the International Higher Education section receives 6% of the submissions, they will be allocated 6% of sessions within the general conference. The number of sessions is determined by the number of rooms available (for an in-person conference) and the number of time slots. Pre-Conference Chairs work with the ASHE Staff on their allocation based upon room availability and time. In both instances, chairs are granted the ability to create sessions as they deem appropriate. For example, a session may be one interactive symposium or three research/scholarly papers or performance visual digital scholarship submissions. Posters and works in progress have a different allocation based on space and time.

  • After initial accept and reject decisions are made, chairs begin to create sessions from paper submissions. In some instances, an individual submission that would otherwise be accepted may not fit into a created session or there may not be space for a session. When this happens, the chairs provide this information to the Program Committee Co-Chairs in the rare case that space becomes available. This is when chairs also assign discussants and session chairs based on those who volunteered and recommendations of submitters.

  • ​Participants may be included in a maximum of four General Conference proposals total regardless of role (e.g., co-author, symposium participant). A participant may appear on the conference program no more than five times in total across all roles and formats (e.g., chair or symposium participant) during the General Conference. These limits apply to the General Conference and do not include pre-conferences. Please keep this in mind as you submit proposals and agree to participate in collaborative submissions.

  • In early July, notifications are sent out by the ASHE Staff to submitters and assigned chairs and discussants.

​Scheduling

  • Once decisions are sent out, the ASHE Staff begins to build the General Conference schedule. This is a month-long process that involves ensuring presenters are not double-booked and that there is topical variety within time blocks within the space and time available. With over 150 sessions and over 1,000 presenters, this is a detailed process and as such we are unable to accept requests for presentation times.

  • Pre-Conference Chairs create their schedule with assistance from the ASHE Staff. Pre-Conference Chairs typically create their schedule prior to accept/reject decisions being sent out based on room and time availability. Just as with the General Conference, we are unable to accept requests for presentation times for Pre-Conferences.