- For personal development, gaining insight into the possibilities and limitations that pave the way for us in the future is necessary. In the past, a builder's son would inevitably become a builder himself, regardless of his capabilities. Psychologist Adriaan de Groot put an end to this arbitrary sense of succession by developing a child assessment test that later became known as the Final Examination for Primary Education. The social status and opinions of the educator were no longer all-encompassing. While the son of a builder could certainly become a builder himself, he could also become a scholar, if he had the intellectual capabilities. This test was the start of the Cito mission - in this way, visitors are introduced to their work by the experts of an institution that has become an authority in the field of testing and evaluation in education over the past half-century.
In October 2022, representatives of the Ministry of Education, the Institute for Education Quality and Evaluation and the Institute for the Improvement of Education spent four days at the CITO Institute, to learn more about their way of working and, in particular, the procedures for preparing tests and evaluation methods.
Since 1968, CITO has been conducting tests at various levels - annual tests at the end of educational cycles and national tests. Their methodology for preparing tests for final exams is based on eight steps. The methods used during the preparation of the tests within the state matura piloting were similar, which is not surprising because the CITO Institute was one of the institutions within the consortium that provides technical support for the implementation of the State Matura Project.
- The first step is always determining the test's purpose. After that, the test specification is made - what the test consists of, the levels, areas, types of tasks, and the number of points are defined. That's exactly how we started - says Aleksandra Rosić, an expert on the State Matura Project, who, for the past three years, coordinated working groups for the preparation of exam materials for general education subjects.
In the Netherlands, too, exam materials are prepared by school teachers, and faculty professors mainly do professional and methodological reviews. The most significant difference in preparing the material is the pre-testing of tasks. In this phase, the validity of the tasks is checked - and the Dutch use national tests for this, in which they anchor the tasks that will appear on the state exam in three or five years. When students take the national test, these tasks are evaluated separately, a psychometric analysis is done, and based on that, the tasks are improved or revised. The tests and the correct sequence of tasks are only constructed after that. In creating the key, they have a series of answers they received from the students as part of the pre-test, so they can offer a more extensive selection of potential responses in the key, which is very important for essay-type tasks. Proofreading and adaptation of tests for students with disabilities are the next two steps, similar to what was done under the State Matura Project.
- The evaluation of results and the preparation of detailed reports, especially for working groups, are also considered critical activities at the CITO Institute - these reports become the zero step for the subsequent testing. The group receives complete psychometrics for each task; they see what can be done on the next test and then start the process over again. We insisted on this as part of the State Matura Project; our psychometricians analysed each task, and that information was the basis for the last round of training that was held - points out Aleksandra Rosić.
The Dutch National Institute is very developed; they have a large number of employees, they produce quality materials, and for the last two years, all their tests, except national ones, have been digital. The Institute for Education Quality and Evaluation, our reference institution for national and international testing, is just preparing to fully take over all the responsibilities assigned to it to implement the state matura exam. The experiences of Dutch colleagues will be vital to them, but time and resources are necessary to reach such a high level of development.
- The digital way of testing offers greater possibilities because, for example, in vocational education or exams in physics or chemistry, there is a simulation of experiments within which students can try several times, get results and draw conclusions. Their tasks are increasingly contextual and conceptual, and they examine the way of thinking. Tasks are reviewed automatically in the digital environment as soon as the test is completed. Those contextual tasks and digitalisation are the future of testing because the digitalised test can also be personalised - in relation to the answer to the first question, the system assigns the following question and begins to classify the examinee into the appropriate level - explains Aleksandra Rosić.
Critical
assessment of work and results is what CITO Institute experts consider key to
quality. Innovation in work also means occasionally accepting risks and the
fact that mistakes can be made along the way. Our institutions are taking the
first steps to state matura, and other countries' experiences are essential to
determine the guidelines as soon as possible and reaching the goal faster.