The Ukrainian system of education quality assurance is just being developed. For many years it was a priority for stakeholders in education, yet instead we were focusing on fighting corruption, modernising the structure of education, and similar.
With the implementation of the New Ukrainian School Reform in 2016 the government clearly stated the necessity of high quality education, transparent measures of educational outcomes and public awareness of the state of the system. This reform aims to shift from a system of control and punishment to a system of support and encouragement. This includes relationships between different stakeholders like the state and schools, teachers and parents, communities and others.
How the system works
Our two state organisations share responsibility for working with country-level aspects of quality assurance. They are the State Service of Education Quality of Ukraine (SSEQU) and the Ukrainian Centre for Educational Quality Assessment (UCEQA). The SSEQU was established in 2017. Its goal is to externally evaluate and support educational institutions in ensuring the quality of education. The UCEQA delivers external independent evaluation of students’ learning outcomes at a certain educational level, and conducts monitoring studies of the quality of education in Ukraine.
A key procedure in the external evaluation system is institutional audit. This process involves a systematic analysis of school activities to identify strengths and weaknesses, as well as the development of recommendations for schools’ further development. Audit methods draw upon international standards and best practices. It allows Ukrainian schools to continually enhance their practices and meet the demands of the modern world. The audits cover key areas such as the learning environment, student assessment systems, pedagogy, and school management processes, against a set of meticulously crafted criteria to ensure a comprehensive assessment.
Reports of institutional audits are made accessible to the public through school websites and a SSEQU official web platform. SSEQU encourages schools to conduct regular (annual) self-assessment of their activities and supports them in this process. To facilitate this, a comprehensive detailed methodology with tools known as the “Alphabet for the Principal” has been designed. This is a resource book that empowers schools to conduct self-assessments aligned with the criteria utilised in institutional audits. By using the Alphabet, a school can evaluate its current state and design strategies for advancement.
Both institutional audit and self-assessment involve collecting information from various sources. This includes attending classes, analysing school documents, inspecting the school environment, and surveying students, their parents, teachers, and school leaders. All these processes take place according to standardised procedures and utilise uniform tools.
Every few years (usually every two or three years), recommendations for ensuring the quality of education and educational activities for schools are reviewed and published in the form of a new edition of the Alphabet for the Principle (editions for 2019 (printed in 2020) and 2021 have already been published, and work on the third edition underway).
Data is key to good assurance
To simplify and improve the reliability of institutional audits and self-assessments, SSEQU in collaboration with the Czech School Inspection and with the support of the Czech Development Agency, has developed the EvaluEd information system. This system automates data collection and analysis and recommendation preparation, tailored to the specific context of each school. Full operation of EvaluEd will begin in 2024 – each school will have free access to self-assessment, and SSEQU will conduct institutional audits using the system.
EvaluEd allows accumulating and analysing the data on the activities from all Ukrainian schools, delivering significant benefits to stakeholders at all levels of the Ukrainian education system. Schools can periodically conduct self-assessment and track the dynamics of their educational quality, while government bodies gain access to reliable data to improve and develop evidence-based policies. Moreover, schools have the option to publicly share the results of their self-assessments. This, along with the public results of institutional audits, increases the transparency and accountability of schools and enhances public trust in them.
Another important aspect of educational quality is enhancement: obtaining the data about students’ learning outcomes and using it for improvement. The second organisation for quality assurance, the UCEQA, is responsible for building the system of educational assessments allowing collecting and analysing the information on students’ outcomes.
The organisation recently went through a significant mindset transformation and re-launched as an institution conducting admission examinations and getting new functions in educational research.
In the early 2000s society did not have any faith in the admission processes in Ukraine, the system of higher education was claimed to be corrupt and in need of deep transformation. Therefore, in 2007 with the support of USAID UCEQA was launched. Since then, the institution has been holding examinations for entering bachelors’ and masters’ degree programs.
The results of these assessments became the only data source on students’ learning outcomes. However, this data source was very limited. Firstly, because not all school graduates considered entering universities. Secondly, because the information was received after students were now longer at schools, and it was difficult for teachers to use the data properly. Thirdly, because there was no information available on the earlier stages of school education, it was almost important to estimate the impact of particular practices on students’ learning outcomes. Not having any other reliable source of data for many years the results of admission examinations were used to make conclusions on the state of secondary education.
National data analysis and international comparisons
In 2007 Ukraine made an attempt to gather reliable nation-wide data on the quality of education – the country took part in Trends in International Mathematics and Science Study (TIMSS). However, due to the political instability in 2011 Ukraine stopped its participation in the study.
Only in 2017, the attempt to start systematic data collection on students’ learning outcomes was made – the two sample-based studies were launched in Ukraine. They were the National Study of Primary School Education Quality (NSPSEQ) and the Program on International Student Assessment (PISA).
NSPSEQ was held in 2018, then in 2021 and is going to be conducted in 2024 again. Within the study, fourth graders are assessed in mathematics and reading, students also respond to questionnaires. Ukraine joined PISA in the 2018 cycle, despite the full-scale war, it was able to collect data in 2022 and it is preparing for the next cycle (PISA 2025).
The data from these studies showed a significant impact of the Covid-19 pandemic and the war on the learning outcomes of students. It shows that the impact of these factors differs for different groups of students. According to PISA 2022 students from rural areas run behind their urban peers for more than four school years. The findings have become shocking for many Ukrainians, but not for the educators in the country. It is obvious that psychological stress, electricity blackouts, and long air alerts are causing great damage to education in Ukraine.
UCEQA sees the data from such large surveys as of primary interest to policy makers. However, the institution also aims to support individual teachers and schools with the findings. Reports on both studies include detailed methodological explanations on particular students’ challenges and how to overcome them in a class. It is clear though that the implications of the sample-based studies are limited, because they do not allow addressing students’ individual needs, they also do not support SSEQU in communication with schools.
UCEQA is aiming to solve this issue by launching nationwide assessments at the key stages of the Ukrainian educational system, mainly at the end of the fourth, ninth and twelfth years of schooling. These will be computer-based assessments with the main goal to provide students, their teachers, and school principals, local and state policymakers the detailed information on students’ learning outcomes. Implementing such assessments in the country with a large population going through the war will be very difficult and will require collaboration between different stakeholders, full political and financial support.
Even if the plans of SSEQU and UCEQA are fulfilled, there is much to be done in the system of educational quality assurance. Despite the challenges Ukraine is facing now stakeholders are working on educational system transformation. The war has changed both the political and social landscapes of the country, but education is seen as one of the most important instruments for supporting the sustainable society and building the resilience of the state.
