Evaluation with Relative Feedback. A Randomised Experiment at Rey Juan Carlos University

Abstract

Feedback on grades from activities and exams allows students to have information about how their effort translates into a score, which can affect their endeavour in the future and, therefore, their performance. This research aims to determine whether increased knowledge in ongoing assessment throughout the semester enhances performance at the end of the course. To this end, a randomized control experiment is conducted in the subject of Financial Economic Analysis II in the Marketing degree at Rey Juan Carlos University. The treatment group received relative feedback regarding their percentile rank within the grade distribution of their class group, while the control group only received their individual numerical grade.

The results indicate that the average grades between both groups are very similar, suggesting that relative feedback does not lead to an improvement in the results of the treated group. Further analysis of moderating factors and expanding the sample size is necessary to determine if the effects on mean values are obscuring opposing effects between higher and lower performing quartiles.

References

Abadie, A. (2020). Statistical non significance in empirical economics. American Economic Review, 110(2), 193-208.

Azmat, G., & Iriberri, N. (2010). The importance of relative performance feedback information: Evidence from a natural experiment using high school students. Journal of Public Economics, 94(7-8), 435-452.

Bandiera, O., Larcinese, V., & Rasul, I. (2015). Blissful ignorance? A natural experiment on the effect of feedback on students' performance. Labour Economics, 34, 13-25.

Barankay, I. (2012). Rank incentives: Evidence from a randomized workplace experiment. The Wharton School, mimeo.

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74.

Blader, S., Gartenberg, C., & Prat, A. (2015). The contingent effect of management practices. Columbia Business School Research, 15-48.

Cabrales, A., Azmat, G., Bagues, M., & Iriberri, N. (2019). What you don't know... Can't hurt you? A natural field experiment on relative performance feedback in higher education. Management Science, 65(8), 3449-3947.

Connolly, P., Keenan, C., & Urbanska, K. (2018). The trials of evidence-based practice in education: A systematic review of randomised controlled trials in education research 1980–2016. Educational Research, 60(3), 276-291.

Dunn, K. E., & Mulvenon, S. W. (2009). A critical review of research on formative assessment: The limited scientific evidence on the impact of formative assessment in education. Practical Assessment, Research & Evaluation, 14(1), 1-11.

Fraile, A., López-Pastor, V., Castejón, J., & Romero, R. (2013). La evaluación formativa en docencia universitaria y el rendimiento académico del alumnado. Aula Abierta, 41(2), 23-34.

Fuch, L. S., & Fuch, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53(3), 199-208.

Gill, D., Kissová, Z., Lee, J., & Prowse, V. (2018). First-place loving and last-place loathing: How rank in the distribution of performance affects effort provision. Management Science, 65(2), 494-507.

Gil-Flores, J. (2012). La evaluación del aprendizaje en la universidad según la experiencia de los estudiantes. Estudios Sobre Educación, 22, 133-153.

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 88-112.

Hidalgo, M. E. (2021). Reflexiones acerca de la evaluación formativa en el contexto universitario. Revista Internacional de Pedagogía e Innovación Educativa, 1(1), 189-210.

Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30(4), 28-37.

Kosfeld, M., & Neckermann, S. (2011). Getting more work for nothing? Symbolic awards and worker performance. American Economic Journal: Microeconomics, 3(3), 86-99.

López-Pastor, V. (2012). Evaluación formativa y compartida en la universidad: clarificación de conceptos y propuestas de intervención desde la Red Interuniversitaria de Evaluación Formativa. Psychology, Society, & Education, 4(1), 117-130.

Macías, C. (2023). Random experiment on relative performance feedback in higher education at URJC. In J. Sainz & I. Sanz (Eds.), Addressing inequities in modern educational assessment (pp. 45-802). Springer: Cham.

Macías, C., & Santero, R. (2022). El sistema educativo en el ámbito rural español. Un análisis comparativo con la educación en la ciudad en España. Revista Universitaria Europea, 37, 215-248.

Molinos, M. del C., & López Pastor, V. (2009). Evaluación formativa y compartida en educación superior: Propuestas, técnicas, instrumentos y experiencias. Estudios Sobre Educación, 16(1), 201.

Moore, D. A., & Cain, D. M. (2007). Overconfidence and underconfidence: When and why people underestimate (and overestimate) the competition. Organizational Behavior and Human Decision Processes, 103, 197-213.

Nyquist, J. B. (2003). The benefits of reconstruing feedback as a larger system of formative assessment: A meta-analysis. (Unpublished master’s thesis). Vanderbilt University.

Styles, B., & Torgerson, C. (2018). Randomised controlled trials (RCTs) in education research–methodological debates, questions, challenges. Educational Research, 60(3), 255-264.

Tran, A., & Zeckhauser, R. (2012). Rank as an inherent incentive: Evidence from a field experiment. Journal of Public Economics, 96(9), 645-650.

Williams, J., & Kane, D. (2009). Assessment and feedback: Institutional experiences of student feedback, 1996 to 2007. Higher Education Quarterly, 63, 264-286.