Reviewer Guidelines


The Role of Reviewers

The goal of RSS is to promote high-quality robotics research. In the face of a changing publication landscape we have this year made some significant changes to the review process in order to retain the quality and integrity of the conference. In particular we have made changes to encourage hypothesis driven research, publication of negative results; and to reduce the disproportionate representation of benchmark-focused research. As a reviewer your responsibility is to write an informative review that is helpful to the Area Chair and the Authors of the manuscript; your review will serve as a primary input to the Area Chair to make a decision about the paper.

The main changes to the submission process for this year from a reviewer’s perspective are the following,

  • We want to encourage research that is based on a scientific narrative and is driven by a hypothesis. This means that as a reviewer you will review Extended Abstracts, not finished papers. Your aim is to judge the paper’s hypothesis in terms of how relevant and impactful it is and how feasible it is to provide support for the hypothesis or not.
  • Your task as a reviewer is to provide support to the Area Chair who will make the decision about the submission. If the submission moves forward in the process the Area Chair will provide the authors with a Metareview that draws upon the opinions of all the reviewers.
  • The post-rebuttal phase for submissions that are invited to proceed will include an item dubbed an “Extension Charter”. Your review includes an opportunity to shape the evolution of the work by providing suggestions in the form of a course of actions. (Further detail appears below.)
  • The reviews for the accepted papers will be made publicly available after the final decisions have been made, and the reviewer’s name will be associated with the reviews.

Review

Below is the structure of the review that you are expected to provide to the Area Chair for each paper. The review should flag any papers that contain ‘hallucinated’ references, or that do not comply with any of the rules stipulated in the Call for Papers (such as Author Anonymity).

Value of Hypothesis (numeric score in range [1-4])

  1. Limited Interest - hypothesis is of limited interest and is not likely to have impact on the community
  2. Possibly Interesting - hypothesis unclear and or value of hypothesis not sufficiently motivated or clear
  3. Interesting - interesting hypothesis that will have impact on a limited subfield of the community
  4. Strongly Interesting - excellent hypothesis that will have significant impact on the community

Approach and Hypothesis: Justification of score for the interest of hypothesis and its possible implications for the robotics community. You are encouraged to provide comments and feedback, including concrete variations on the hypothesis statement in order to improve its value to the community.

Feasibility of Conclusion (numeric score in range [1-4])

  1. Weak - it will be very challenging to provide support for the hypothesis
  2. Limited - only part of the hypothesis can be supported
  3. Good - the hypothesis can be supported in a manner that provides evidence for a conclusion
  4. Very Strong - the hypothesis can clearly be supported and provide very strong evidence for a conclusion

Feasibility of Conclusion: Justification of score detailing if the planned experimental evaluation is likely to provide support or falsification of the hypothesis and whether there is a clear plan on how this will be executed and interpreted into a conclusion.

Extension Charter: Provide a list of points that you want the AC to integrate into the Extension Charter including motivation of how this suggested course of actions will support the paper’s hypothesis and the conclusions that can be drawn as a consequence. Importantly, a hypothesis is judged by its scientific value, meaning that ‘negative’ results are valued equally with positive ones.

Confidential Comments: Please use this space include any sensitive items of context you feel necessary to interpret your review, and other confidential concerns regarding anonymity, plagiarism, formatting, etc.