Abstract judgement is frequently the most time consuming and data-intensive stage of the abstract management process. Traditionally, this process involves:
Thankfully, there are ways to help you streamline this messy, cumbersome process. To help you discover better ways of teaching your judges how to review abstracts for conferences, we’ve put together these 7 tips that speed up the abstract judgement process:
If you know your reviewers’ areas of expertise (or use a platform where they can indicate this), you can separate them into groups that will grade submissions in the same topic area. Judges with extra knowledge in the abstract’s area will be able to assign grades quicker and easier. As a side benefit, this also gives you more consistency and fairness in the review process, since the abstract judgement in each topic is done by the same sets of eyes.
An intelligent abstract submission form can do a lot of your work before the abstracts get to your judges. A system like OpenWater can set up eligibility criteria (for example, post-graduate degree required, or only PDF attachments allowed) so that no submission can be completed without matching your base level of expectations.
Your judges may be doing more assessment than you actually need them to do. Are they writing personalized comments on every abstract submission? Do they need to grade seven different categories with a 1-10 score, when four categories with a simple 1-5 score might do the trick? Could you have them rank their top 3 submissions rather than their top 5?
Multiple rounds of review don’t have to slow down your process. If the first round has simple standards with a straightforward yes/no assessment or single 1-5 score, you may be able to distill your pool to just the highest quality abstracts before the abstract judgement process kicks into high gear. The second, smaller round will increase the value of time spent on things like specific text comments.
There may be fields of the abstract submission form that would only create static for your reviewers to skim through. If you make those portions invisible to reviewers—with a capable abstract management software—there will be less to distract them from what’s most important. This can protect presenters from unintentional biases at the same time.
An abstract judgement software may be able to configure customizable “rules,” each with an assigned point value, and then automatically check the abstracts against your rules to assign a score.
For example, in OpenWater, you could promote or penalize submissions based on submission date:
These auto-scores could act as your first wave of judging, or as a supplementary score that allows human judges to focus on higher-value portions of the abstract.
Likely, your reviewers aren’t doing abstract judgement full time, and they’ll probably have competing priorities. You can help them manage their time and keep track of deadlines by sending out email blasts that alert them to their progress. Some abstract management systems have built-in functions for monitoring individual reviewer progress and mass emailing reviewers who are falling behind. Automated personalized reminders can make a difference for presenters, too.