Sirats Santa Cruz: “In the research, we defined a framework for the design, implementation and analysis of evaluation systems”
Sirats Santa Cruz: “In the research, we defined a framework for the design, implementation and analysis of evaluation systems”
Sirats Santa Cruz: “In the research, we defined a framework for the design, implementation and analysis of evaluation systems”
Bikain “cum laude” kalifikazioa lortu du Sirats Santa Cruzek martxoan Eskoriatzako campusean aurkeztutako "A programme level assessment system in the Faculty of Humanities and Education of Mondragon Unibertsitatea: A qualitative study of perceptions of student teachers, teacher educators, mentors, and coordinators" tesiak, nazioarteko aipamenarekin. Egilearekin hitz egin dugu ikerketari buruz.
On March 3, Faculty of Humanities and Education Sciences researcher Sirats Santa Cruz (Oñati, 1992) defended her doctoral dissertation, A programme level assessment system in the Faculty of Humanities and Education of Mondragon University: A qualitative study of perceptions of student teachers, teacher educators, mentors, and coordinators. The event was held on the Eskoriatza campus. We spoke with the author about her research, which earned a rating of Outstanding “cum laude”.
What is the main topic of your doctoral dissertation?
The dissertation discusses the evaluation system that competency-based education requires, an evaluation system used in higher education. To this end, we analyzed the evaluation system used in the Bachelor’s Degrees in Education in the Faculty of Humanities and Education Sciences, exploring the perceptions of the main agents involved in this evaluation system. This research work has allowed us, on the one hand, to construct a framework for the design, implementation and analysis of evaluation systems at the level of qualifications or initial teacher training programs and, on the other hand, to make a series of proposals to be taken into account in the future.
In your research, you focused on the evaluation system required by competency-based education. Why did you decide to investigate this subject? What was your starting point?
The design and implementation of evaluation systems has been a thorny issue in higher education in recent decades. The conceptualization, design and implementation of summative evaluation, educational evaluation... has been the subject of debate in the scientific literature. But not only in literature, but also in practice. In fact, various studies have shown that the competency-based evaluation system has been the element with the greatest deficit in the implementation of the Bologna process in Spain. And, in part, I’ve also experienced it. As a teacher in the Bachelor’s degree programs in Early Childhood Education and Primary Education and in the Master’s degree program in Teaching Habilitation for the Exercise of the Profession of Secondary Education, Vocational Training and Teaching of Languages, I had difficulties in evaluating the development of my students’ skills. All this led me to investigate this topic.
You conducted 23 discussion groups and 16 interviews for your dissertation, with 45 students, 35 professors, 6 mentors and 2 coordinators. In what ways were they involved?
Excellent. It was clear to us that we wanted to collect the perceptions of the main agents involved in the evaluation system, from all of them. So all the agents were invited to participate in the research and more than 85% participation was obtained. But not only that: all the participants in the discussion groups and interviews shared their experiences and feelings with us very openly and generously. It’s thanks to all of them that we were able to carry out this research work, so I’d like to take this opportunity to thank all of them once again for their participation.
You explored the perceptions of the main agents involved in the evaluation system, identifying the main areas of agreement and disagreement among their perceptions. What do the agents say? What differences in perception are there among students, teachers, mentors and coordinators?
It’s not easy to summarize here what the participants say, since in the doctoral dissertation we needed more than 350 pages to present it in detail. What I would highlight is that the perceptions of all agents showed consensus on the following attributes related to the design and implementation of the evaluation system: 1) assumptions, 2) alignment, 3) organization, 4) shared understanding, 5) evaluation process, 6) participation, 7) impact factors, 8) proposals for the future and 9) feelings. In other words, the perceptions of students, teachers, mentors and coordinators agree that these 9 attributes are the key attributes to be taken into account when designing and implementing an evaluation system. Therefore, these results suggest that evaluation systems at the level of qualifications or initial teacher training programs should be designed and implemented with these key attributes in mind.
However, the agents’ perceptions diverge if these attributes are investigated in more detail. To give an example, the coordinators and some students, teachers and mentors feel that they have an understanding of the objective of the evaluation system, while other students, teachers and mentors do not. Furthermore, the results show that that understanding is not entirely shared among those who perceive it. In fact, students, teachers and mentors agree that the evaluation system has a summative, formative objective, but students and coordinators also feel that it has the objective of learning to evaluate (which is not the case with the teachers and mentors).
What are the main conclusions of your work? What contribution does your research make?
I would say that there are three main contributions. First, a framework was constructed for the design, implementation and analysis of evaluation systems at the level of qualifications or initial teacher training programs. Second, this work contributes to the debate on the existing evaluation in higher education, since each of the sections of the theoretical framework has been conceptualized in great depth. Finally, it makes a special contribution to the evaluation system of the degree programs in Early Childhood Education and Primary Education of the Faculty of Humanities and Education Sciences, which is the subject of this research, since several proposals are made for future consideration. Among other things, we propose that the curricular map of the degrees be completed and continuously reviewed, and that we construct an understanding of the evaluation system that is shared among the main agents (from the results, it can be deduced that this shared understanding is not fully perceived at present). In addition, in the context of initial teacher training degrees, we recommend that special attention be paid to student agency and to assessment literacy, as the students in these degree programs are future teachers and evaluators.
How did you find the process of writing your dissertation?
In short, I think it was a great learning experience, with the fluctuations that learning processes can have. I learned as a researcher, I learned as an education professional, and I learned as a person. The process was long and I also had to live through a few moments of crisis in which I had to decide whether to go ahead with my dissertation or abandon it. I don’t regret my decision to move forward at all. Without a doubt, I’ve taken everything on board that I learned along the way.
