In this note, we propose a new approach for benchmarking computational problems and their solvers. The proposed methods are based on special construction of the paired comparison matrices of the problems and solvers; in other words, the scorematrices of the problems and the solvers. Having these matrices at our disposal, different rating/ranking methods can be used. We illustrate our approach using rating/ranking methods originated for ranking sports teams. The proposed approach is illustrated using an example to demonstrate its viability and suitability for applications.

Keywords : Benchmarking; Software; Solvers; Problems; Testing; Ranking Method.