I get the impression that good journal editors will get at least one reviewer who is skilled in the methodology used in the paper. The importance of this reviewer role would presumably vary with the statistical or other methodological complexity of the paper.
That said, reviewing is well known to be imperfect particularly when it comes to checking all the possible smaller errors that can be made. Errors in statistical analysis and reporting are wide-spread in published articles in psychology. For example, Bakker and Wicherts (2011) did a review, where to quote the abstract
we checked the consistency of reported
test statistics, degrees of freedom, and p values in a random sample
of high- and low-impact psychology journals... On the basis of 281
articles ... [we estimate that] around 18% of statistical results in the
psychological literature are incorrectly reported. Inconsistencies
were more common in low-impact journals than in high-impact journals.
Moreover, around 15% of the articles contained at least one
statistical conclusion that proved, upon recalculation, to be
incorrect; that is, recalculation rendered the previously significant
result insignificant, or vice versa.
References
- Bakker, M., & Wicherts, J. M. (2011). The (mis) reporting of statistical results in psychology journals. Behavior Research Methods, 43(3), 666-678. FULL-TEXT