The discourse on quality in education is permanently present in social sciences. The problem involves outstanding authorities, politicians and specialists in various fields. The article is devoted to the analysis of external evaluation - in terms of meeting purposes attributed to it by the education authorities. The authors attempt to find answers to the following questions: 1. Does the external evaluation procedure enable collection of data giving rise to doing a reliable and comparable evaluation of schools? 2. Can the analysis based on this material help to improve schools and other educational institutions? The results show that the procedure and methodology of external evaluation is not conducive to achieving pedagogical supervision system objectives.