SEFI Podcast Season 6 Episode 3: Kurt Coppens from KU Leuven on Feedback Literacy with Natalie Wint and Neil Cooke

Feedback is a somewhat perennial topic within higher education (HE), with increasing emphasis on students as independent learners who should engage as active participants to fulfil the role of a self-regulated learner.

In this episode we spoke to Dr. Kurt Coppens who works within the ETHER (Engineering Technology Education Research) research group, at the Faculty of Engineering Technology, KU Leuven, Belgium and whose PhD focused on the feedback literacy of engineering students.

The remainder of the article will summarise key discussion points.

Feedback as a gift

Kurt explains that the purpose of feedback information is to help us understand our strong points and areas for improvement, with the goal of improvement. He compares feedback information to gift. To benefit from the gift, you need the (feedback literacy) skills to unwrap it, understand its value, and use it to its fullest potential. Feedback literacy is thus the set of skills and understandings you need to engage with feedback effectively. Kurt highlights the work of Carless and Boud, who outlined four key features of feedback literacy: appreciation; making judgements; managing affect; and taking action.

• Appreciation: Students need to understand and appreciate the role of feedback in improving work; that they have an active role in the process; and that feedback information comes in different forms from different sources.
• Making judgements: Students should be able to make sound academic judgement about their own work and that of others.
• Managing affect: It is important for students to handle their emotions when receiving feedback.
• Taking action: It is important for students to understand that they can only benefit from the feedback information if they take action by considering it and make informed decisions about their work. 

Kurt then introduces the framework of Molloy, Boud and Henderson which he uses in his own research and which includes 31 characteristics of feedback literacy behaviour, derived from a large empirical dataset. These characteristics are split into 7 groups. For example, one is focused on understanding the role and purpose of feedback and committing to feedback as improvement, with others related to appreciating feedback as an active process, and eliciting information to improve learning and keeping an ongoing dialogue. Other groups cover processing feedback information, judgement and acknowledging emotions.

Why is feedback still worthy of study?

Kurt explains that satisfaction surveys typically show that students often feel dissatisfied with their feedback experiences compared to other aspects of their learning. At the same time, educators feel that students don’t care about the feedback they provide, and that they primarily focus on their grades.

He says that one of the reasons for this disconnect could be the traditional view of feedback that both teachers and students often have, which mostly limits feedback to an input mechanism that is mainly teacher-centred rather than considering it as a shared responsibility between teachers and students. Feedback should go beyond being just a gift from teachers to help students improve. Instead, it should shift towards a learning-centred process that actively promotes student learning.

PhD aims

Kurt defines the aims of his PhD work as answering the following questions:

1. How does the feedback literacy of engineering students evolve through their curriculum?
2. How do feedback experiences influence feedback literacy among engineering students?
3. What is the relation between self-reflection and feedback literacy among engineering students

Starting the research

Kurt explains that he started his research by using the Feedback Orientation Scale by Linderbaum and Levy. However, after collecting a first round of quantitative data, he noticed some reliability issues with one of the subscales so decided to start collecting reflective logs to gain deeper insights. In these reflective logs he asked the students to reflect upon a feedback experience they perceived to have had the most impact on their learning process and the data was thematically analysed using the empirical framework for feedback literacy based on the 31 characteristics of Molloy et al. Students were grouped into three categories: a first group of students showing none of the identified feedback literacy characteristics, a second group of students demonstrating a basic understanding and appreciation of feedback, and a third group showing advanced feedback literacy characteristics.

The findings

Kurt explains that one of the first findings from the Feedback Orientation Scale data was that students generally scored themselves highly for items related to feedback utility and accountability, while items related to feedback self-efficacy scored the lowest. These findings suggest that students believe that feedback is useful in achieving goals and that they feel responsible for reacting to and following up on feedback, but that they are less confident in their ability to interpret and respond to feedback appropriately.

Comparing the data with reflective logs

Data collected from the reflective logs showed that most first-year students acknowledge that feedback is an active process and that they need to act in some way. However, only about half of the reflective logs mentioned more advanced uses of their feedback, with most students considering feedback as information given to them after submitting a piece of work. They thus fail to identify feedback they can use on their own, such as exemplars or assessment criteria, or they don’t recognise these as a form of feedback information that can help them improve. Kurt tells us that it is therefore important to explicitly introduce students to contemporary, learner-centred definitions of feedback so they can better recognise and benefit from the multitude of feedback opportunities available to them. In terms of the feedback literacy characteristics, the logs showed that students with higher reflection levels demonstrated more advanced feedback literacy characteristics. Students thus need more guidance and practice in developing their reflective skills which should help them further develop their feedback literacy and start recognising more feedback opportunities.

Change over time

More data was then collected, with the Feedback Orientation Scale being used at the beginning of each semester and reflective logs being collected at the end of each semester. The Self-Reflection and Insight Scale by Grant et al., was then introduced to measure students’ awareness of engaging in self-reflection during their learning. Kurt explains that the reflective logs showed a significant increase in reflection levels throughout the students’ first year (that was not reflected by the Feedback Orientation Scale) but also showed a significant negative relation with the data from the Self-Reflection and Insight Scale. A course-related self-reflection grade that students received during a problem-based learning course in their second semester was therefore used to benchmark the findings. This data supported that collected through the reflective logs.

Similarly, the data from the Feedback Orientation Scale did not show growth during a three year longitudinal study, whereas the reflective logs did, this suggesting that the Feedback Orientation Scale is unable to capture the context-specific nuances of students’ everyday feedback experiences.

So, what do all the findings mean together?

Kurt explains that the data from the reflective logs, the course-related grades, and the two surveys were brought together, and it was concluded that the growth in feedback literacy and self-reflection observed in reflective logs, reflects the actual student behaviour over the year whereas the two survey instruments show students’ intents rather than their actual engagement in feedback and self-reflection. This finding highlights the need to supplement quantitative data with qualitative data.

What does this mean for educators?

Kurt references the work of Woitt et al. which focuses on feedback attitudes and feedback practices. Based on this, he thus suggests that lecturers educate students about the function of feedback and, based on the findings of Noble et al., that they support students in understanding their proactive role in feedback practices, making different opportunities for feedback more explicit. Focusing on ‘feedback practices’ involves continuously challenging students in their feedback encounters, for example, by using more peer feedback.

What’s next?

Kurt explains that there is a need to investigate the apparent gap between student intentions measured and their actual behaviour, as well as conducting more longitudinal research. He also suggests partnering with students to secure their commitment to consistently participate in these longitudinal studies. Such studies could examine the didactical implementations of feedback practices, such as extensively incorporating peer feedback and using reflective activities, to learn how it influences actual student behaviour.

Kurt’s work

Kurt’s Thesis can be found by following the link below: https://kuleuven.limo.libis.be/discovery/search?query=any,contains,LIRIAS4154722&tab=LIRIAS&search_scope=lirias_profile&vid=32KUL_KUL:Lirias&offset=0

Coppens, K., Van den Broeck, L., Winstone, N., & Langie, G. (2023). Capturing student feedback literacy using reflective logs. European Journal of Engineering Education, 48(4), 653–666. https://doi.org/10.1080/03043797.2023.2185501

Coppens, K., Van den Broeck, L., Winstone, N., & Langie, G. (2024). A mixed method approach to exploring feedback literacy through student self-reflection. Assessment & Evaluation in Higher Education, 50(2), 173–186. https://doi.org/10.1080/02602938.2024.2373792

Other Key Resources

Carless, D., & Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354

Grant, A. M., Franklin, J., & Langford, P. (2002). The Self-Reflection and Insight Scale: A new measure of private self-consciousness. Social Behavior and Personality: An International Journal, 30(8), 821–835. https://doi.org/10.2224/sbp.2002.30.8.821.

Linderbaum, B. A., & Levy, P. E. (2010). The development and validation of the Feedback Orientation Scale (FOS). Journal of Management, 36(6), 1372–1405. https://doi.org/10.1177/0149206310373145

Molloy, E., Boud, D., & Henderson, M. (2019). Developing a learning-centred framework for feedback literacy. Assessment & Evaluation in Higher Education, 45(4), 527–540. https://doi.org/10.1080/02602938.2019.1667955

Noble, C., Billett, S., Armit, L., Collier, L., Hilder, J., Sly, C., & Molloy, E. (2020). “It’s yours to take”: generating learner feedback literacy in the workplace. Advances in health sciences education : theory and practice, 25(1), 55–74. https://doi.org/10.1007/s10459-019-09905-5

Woitt, S., Weidlich, J., Jivet, I., Orhan Göksün, D., Drachsler, H., & Kalz, M. (2023). Students’ feedback literacy in higher education: an initial scale validation study. Teaching in Higher Education, 30(1), 257–276. https://doi.org/10.1080/13562517.2023.2263838