Reviewing

(last update: November 15th, 2024)

Here are some reviewing stats for nerds.

Method

The following stats are computed by accounting for all the reviews I worked on (either as a reviewer, subreviewer, or PC member) since 2020. They only include reviews for “full papers” (for journals, conferences, or workshops), and do not consider either (i) research grants, (ii) revisions/shepherding of papers, (iii) theses – all of which are also documents that I will gladly review, but which would skew the results.

To compute the numbers presented below, I devised a Jupyter Notebook which analyzes all the reviews I wrote (which I meticulously store on my own devices prior[a]As such, these numbers represent the review “at submission time”, and do not account for edits or changes that I may make afterward. to submitting them) and extracts some metrics.

If you are interested in knowing the venues I review for, you can look at my Service page.

Overview

First, a couple of figures that showcase my reviewing efforts.

Observation. Starting from mid-2022, I changed the way I review papers. Specifically, I have begun “quoting” (verbatim) some of the statements written in the paper, and use such quotations as basis for my remarks. Despite being quite time-consuming, I found that such an approach can be very helpful – both for myself, but also (hopefully) for the authors, since it precisely shows the parts of the paper that are problematic (according to my own judgement). Of course, such an approach also leads to longer reviews.[b]I do not filter-out the quoted statements, since it would be impossible to clearly distinguish them from the rest of the content.

Detailed metrics

Below are some specific metrics per year.