Sudoku Grader

With the standardized grading scale finished, it is time to begin analysis of some of the major news publications’ and Sudoku providers’ grading systems. Four possible publications to consider are USA Weekend, The Columbus Dispatch, NYTimes.com, and The Giant Gift Book of Sudoku by Will Shortz. These are two print news publications, a website, and a Sudoku book. Sampling puzzles from each source and running them through Sudoku Grader, a Sudoku grading program written by myself as prescribed in this essay, some interesting patterns emerge in the data. The data, available in Appendix A has been summarized in Table 1.

First of all, it is clear that different publications’ grading scales do not line up. Also, it is immediately apparent that puzzles written for print news publications tend to be graded on a scale far easier than those available through other means. A puzzle rated “5 of 5” in The Columbus Dispatch is significantly less difficult than a “Difficult” puzzle from NYTimes.com or a “Demanding” puzzle from Will Shortz, as confirmed by a two-sample t-test (Appendix B). While other statistical conclusions are difficult to make due to limited sample sizes and unequal variances, it is worth noting some other trends in the data. First of all, every single USA Weekend or Columbus Dispatch puzzle rated had at most a strategic difficult score of one. The vast majority of these puzzles had a strategic difficulty score of 0, requiring only Slicing and Slotting and Simple Singles. This means that such print news publications do not include puzzles requiring strategies more advanced than Hidden Singles. Likely an effort to avoid alienating casual Sudoku solvers, only publishing easier puzzles contributes to confusion between the grading scales.

Ideally, the proposed grading scale would be better able to differentiate between the news publications’ difficulty levels. Were the system more precise, perhaps USA Weekend’s “Easy”, “Medium”, and “Hard” ratings would not grade to 1. But these subtle differences between difficulty levels are not an issue of the grader’s accuracy. Instead the problem is in lack of precision. If the grading scale used the set of real numbers 0-10 rather than the set of integers 0-10, finer distinctions could be made. That said, having ten levels of difficulty should offer enough distinction between difficulties. Anything more might be superfluous.

With the grading scale implemented and tested, one final question remains: why hasn’t a standardized grading system been adopted yet? As evidenced by this paper, creating the standardized and transparent grading system need not be overly complex. The issue is that different individuals perceive difficulty in different ways, and different publications do not want to alienate their target audiences. Readers of The Columbus Dispatch might be disappointed to learn that their puzzles are relatively easy compared to those in other sources by seeing a low standardized score, despite the fact that the target audience for a newspaper’s Sudoku puzzles is less sophisticated than those that search for difficult puzzles in published Sudoku collections.

A .05 alpha-level is used for all tests.

Test 1

H₀ : Sudoku Grader grade for “5 of 5” Dispatch puzzles = Sudoku Grader grade for

Test 2

H₀ : Sudoku Grader grade for “5 of 5” Dispatch puzzles = Sudoku Grader grade for