Review Process

Process Overview

We will follow a process inspired by CHI2024, which is similar to the RA-L review process. This is a two-stage process with a threshold decision in round 1, where those above the threshold have 3 weeks to revise and resubmit (and rebut any comments as part of author response), followed by an AC meeting to make final decisions on resubmitted papers in round 2.

Desk Reject

Upon initial submission, the Program Committee chairs (PCs) and Area Chairs (ACs) will determine whether the submission is appropriate for RSS or should be desk rejected.

Examples of desk rejects include:

  • Incomplete submissions: This includes papers with obvious placeholder titles as well as papers that end abruptly and are obviously incomplete.
  • Non-anonymized papers (for non-demo contributions):
    • Leaving the author names in the paper or having a description or an acknowledgements section that reveals authors or the institution (e.g., places where user studies were conducted and specific supporting grant information).
    • Not following anonymization guidelines such that deanonymization is obvious.
  • Incorrectly formatted paper:
    • All submissions must use the template specified in the Call for Submissions page. Any submissions with other templates will be desk rejected.
  • Out of scope for the conference
  • Not written in English.

Desk rejected submissions will not be assigned to a reviewer, and their authors will receive a brief note about the rejection. Criteria for Desk Rejection may be applied to revised papers in the revise and resubmit process.

AC Assignment

For the submissions that are not desk rejected, PCs will assign papers to ACs based on the paper keywords while avoiding conflicts of interest and balancing the load across ACs. For non-demo papers, the ACs and the external reviewers are never aware of the authorship of the papers they review.

Round 1 Review Process

The AC will manage the review process and assign three reviewers to assess each submission that undergoes full review. Reviewers should have high expertise on the topic of the paper(s) they are asked to review. As part of the review process the PCs reserve the right to get additional input from other ACs for any paper so as to make the best decision for acceptance. External reviewers will be asked to provide a detailed review of the submission and will be asked to provide a first round recommendation. The AC will, based on the external reviews and their own assessment of the paper, provide a meta-review and first round recommendation. A recommendation will be one of: Accept (A), Weak Accept (WA), Weak Reject (WR) and Reject (R). Based on the reviewer and AC recommendations, papers will then fall into two groups:

Revise and Resubmit: Any paper that receives at least one recommendation of WA or higher from either the reviewers or the AC will be given the opportunity to make changes and submit a track-changed version along with an author response to the reviews. If your reviews are positive, this may not involve many changes, but more borderline reviews may involve more work. Authors that meet this minimum threshold will have 3 weeks to respond as they see fit. Note: at this stage, no papers are accepted, and a significant portion of R&R papers may be rejected during the second round of reviewing.

Reject: Papers where reviewer and AC recommendations are all R or WR will be considered below the threshold to revise and resubmit, and so will be rejected at the completion of Round 1. The time frame for the Revise and Resubmit process is limited to a fixed length. A submission will thus be rejected if the reviewers feel that the required revisions would not be achievable within the time frame. A rejection in round 1 should be taken to indicate that the work is not deemed to be ready for this year’s RSS, not that it has no place at RSS at all.

Resubmission and Round 2 Review Process

Authors of the papers with the Revise and Resubmit recommendations are invited to submit a revision of their submission, and a response to reviewers, by the resubmission deadline. Note that author lists may not be changed.

Revisions should track changes to highlight the changes to the document, and also include a response letter on how the reviewer comments have been addressed. No resubmission will be allowed after the deadline. Note: authors of non-demo papers should check all resubmission files carefully for anonymity, including that highlighted changes are not attributed to an author. Failure to maintain anonymity could result in a desk reject of the whole submission.

All the resubmissions will be reviewed by the original AC and external reviewers. The AC reserves the right to invite new reviewers if needed. The results of the current review(s) will be shared with the new reviewers in such cases. The result of the second round review is either Accept with Minor Revisions or Reject. Authors do not have a chance for another round of major revisions.