Dimensiones de la evaluación de universidades según escalas de valoración: Percepción de los expertos latinoamericanos

Francisco Ganga Contreras; Juan Bautista Abello Romero; Patricio Viancos-González; Walter Saéz-San Martin; Luis Andrade Ríos

doi:10.31637/epsir-2024-1679

Authors

Francisco Ganga Contreras Universidad de Tarapacá
Juan Bautista Abello Romero University of Santiago Chile https://orcid.org/0000-0003-3064-3456
Patricio Viancos-González University of Tarapacá https://orcid.org/0000-0003-4550-3608
Walter Saéz-San Martin University of Tarapacá https://orcid.org/0000-0003-4346-2304
Luis Andrade Ríos Universidad del Alba https://orcid.org/0009-0002-0397-4516

DOI:

https://doi.org/10.31637/epsir-2024-1679

Keywords:

University Governance, Rankings, Higher Education Institutions, Performance Dimensions, Educational Evaluation, Higher Education, Public Policies, Universities

Abstract

Introduction: Academic indexes for evaluating universities have emerged in higher education to satisfy the demand for information and foster transparency, although they have also generated criticism. This study explores the perception of experts from Latin American universities on the dimensions used in these rankings, in order to guide future proposals. Methodology: The research is descriptive and quantitative in nature. Methodologies of thirteen rankings were analysed and a Likert scale questionnaire was applied to 712 participants from 13 Latin American countries. Results: Respondents rated the dimensions evaluated positively, although there were significant variations according to hierarchical role and country of origin. Academics prioritise research less than middle managers, and notable differences were found, especially in Argentina. Conclusions: Perceptions of the rankings vary considerably according to position and national context, suggesting the need to adapt evaluation criteria to regional realities in order to improve the relevance of these indices in Latin America.

Downloads

Download data is not yet available.

Author Biographies

Juan Bautista Abello Romero, University of Santiago Chile

D. in Management Sciences and Master in Management Sciences from the University of Santiago, and DEA in Management from the University of Seville. Member of the Accounting and Auditing Department.

Patricio Viancos-González, University of Tarapacá

D. in Educational Policy and Management from the Universidad de Playa Ancha de Ciencias de la Educación. Public Administrator and Bachelor in Public Administration from the Universidad de Valparaíso. He is currently a researcher at the University of Tarapacá, Chile. Member of the network of university governance researchers and IESED-Chile.

Walter Saéz-San Martin, University of Tarapacá

D. student in Higher Education Policy and Management at the Universidad Nacional de Tres de Febrero (Argentina), with degrees in Public Administration and Public Administration from the Universidad de Valparaíso (Chile). Research assistant at the Universidad de Tarapacá and in the Fondecyt Regular Nº1220740 of the Universidad de Santiago de Chile. He works in lines of research on university rankings, quality assurance and university governance.

Luis Andrade Ríos, Universidad del Alba

Sociologist from the Pontificia Universidad Católica de Chile. Methodological advisor in regular Fondecyt projects 1131134, 1161353, 1220740 and 1231766. Currently he is the methodological expert for IESED-Chile and for the Universidad de los Lagos.

References

Abello-romero, J. B., Sáez San Martin, W., & Mancilla, C. (2021). Evaluación del desempeño de las universidades: el aporte de los rankings mundiales. Hallazgos, 18(35), 55-75. https://doi.org/10.15332/2422409x.5792 DOI: https://doi.org/10.15332/2422409X.5792

Abramo, G., & D’Angelo, C. A. (2007). Measuring science: Irresistible temptations, easy shortcuts and dangerous consequences. Current Science, 93(6), 762–766. http://www.jstor.org/stable/24099115

Abramo, G., & D’Angelo, C. A. (2015). Evaluating university research: Same performance indicator, different rankings. Journal of Informetrics, 9(3), 514–525. https://doi.org/10.1016/j.joi.2015.04.002 DOI: https://doi.org/10.1016/j.joi.2015.04.002

Abramo, G., & D’Angelo, C. A. (2021). The different responses of universities to introduction of performance-based research funding. Research Evaluation, 30(4). https://doi.org/10.1093/reseval/rvab022 DOI: https://doi.org/10.1093/reseval/rvab022

Abramo, G., Cicero, T., & D’Angelo, C. A. (2013). Individual research performance: A proposal for comparing apples to oranges. Journal of Informetrics, 7(2), 528–539. https://doi.org/10.1016/j.joi.2013.01.013 DOI: https://doi.org/10.1016/j.joi.2013.01.013

Abramo, G., D’Angelo, A. C., & Murgia, G. (2017). The relationship among research productivity, research collaboration, and their determinants. Journal of Informetrics, 11(4), 1016–1030. https://doi.org/10.1016/j.joi.2017.09.007 DOI: https://doi.org/10.1016/j.joi.2017.09.007

Abramo, G., D’Angelo, C. A., & Di Costa, F. (2019). When research assessment exercises leave room for opportunistic behavior by the subjects under evaluation. Journal of Informetrics, 13(3), 830–840. https://doi.org/10.1016/j.joi.2019.07.006 DOI: https://doi.org/10.1016/j.joi.2019.07.006

Agasisti, T., Egorov, A., & Serebrennikov, P. (2023). Universities’ efficiency and the socioeconomic characteristics of their environment — Evidence from an empirical analysis. Socio-Economic Planning Sciences, 85. https://doi.org/10.1016/j.seps.2022.101445 DOI: https://doi.org/10.1016/j.seps.2022.101445

Alcaide-Aranda, L. I. del C. y Aguilar Alcaide, C. E. (2023). Liderazgo Escolar y sus Efectos Organizacionales en Procesos de Mejora Educativa. Revista de Comunicación de la SEECI, 56, 282–301. https://doi.org/10.15198/seeci.2023.56.e837 DOI: https://doi.org/10.15198/seeci.2023.56.e837

Altbach, P. G. (2008). Funciones complejas de las universidades en la era de la globalización. In La educación superior en el mundo: Educación superior, Nuevos retos y roles emergentes para el desarrollo humano y social (pp. 5-14).

América Economía (2017). Ranking Universidades de Chile: variables, ponderadores y validaciones. Documento de trabajo, Vol. 2. Santiago de Chile: América Economía Intelligence.

Ansmann, M., & Seyfried, M. (2022). Isomorphism and organizational performance: evidence from quality management in higher education. Quality Assurance in Education, 30(1), 135–149. https://doi.org/10.1108/QAE-07-2021-0114 DOI: https://doi.org/10.1108/QAE-07-2021-0114

Araya-Castillo, L., Burgos, M., González, P., Rivera, Y., Barrientos-Báez, A., Yáñez-Jara, V., Ganga-Contreras, F.A. y Sáez, W. (2023). Procrastination in University Students: A Proposal of a Theoretical Model. Behavioral Sciences,13(2), 128. https://doi.org/10.3390/bs13020128 DOI: https://doi.org/10.3390/bs13020128

ARWU- Academic Ranking of World Universities (2017). ShanghaiRanking's Academic Ranking of World Universities Methodology 2017. Disponible en: https://www.shanghairanking.com/methodology/arwu/2017 DOI: https://doi.org/10.1201/9781315155890-6

Ascanio Rengifo, R. (2022). Profesionales universitarios venezolanos que han emigrado y la pérdida del capital intelectual para el país. Revista de Ciencias de la Comunicación e Información, 27, 1-14. https://doi.org/10.35742/rcci.2022.27.e133 DOI: https://doi.org/10.35742/rcci.2022.27.e133

Barquero Cabrero, J. D., Caldevilla Domínguez, D., Barrientos Báez, A. & Gonzálvez Vallés, J. E. (2022). Social networks as a vehicle for happiness management in university governance. Corporate Governance, 22(3), 521-535. https://doi.org/10.1108/CG-05-2021-0182 DOI: https://doi.org/10.1108/CG-05-2021-0182

Barrientos-Báez, A., Caldevilla-Domínguez, D. y Gallego-Jiménez, M. G. (2020). Innovación universitaria en ecologías de aprendizaje. Enseñanza inclusiva en la educación superior: grado en turismo. Octaedro.

Bensalem, E. (2022). El impacto del disfrute y la ansiedad en la voluntad de comunicarse de los estudiantes del idioma inglés. Vivat Academia, Revista de Comunicación, 155, 91-111. https://doi.org/10.15178/va.2022.155.e1310 DOI: https://doi.org/10.15178/va.2022.155.e1310

Bornmann, L., Gralka, S., Anegón, F. D. M., & Wohlrabe, K. (2023). Efficiency of universities and research-focused institutions worldwide: The introduction of a new input indicator reflecting institutional staff numbers. Journal of Informetrics, 17(2). https://doi.org/10.1016/j.joi.2023.101400 DOI: https://doi.org/10.1016/j.joi.2023.101400

Brito, A. C. M., Silva, F. N., & Amancio, D. R. (2023). Analyzing the influence of prolific collaborations on authors productivity and visibility. Scientometrics, 128(4). https://doi.org/10.1007/s11192-023-04669-7 DOI: https://doi.org/10.1007/s11192-023-04669-7

Brunner JJ. y Ganga-Contreras F. (2016). Dinámicas de transformación en la Educación Superior latinoamericana: Desafíos para la gobernanza. Revista Opción. Año 32. N° 80.

Carvalho, T., & Videira, P. (2019). Losing autonomy? Restructuring higher education institutions governance and relations between teaching and non-teaching staff. Studies in Higher Education, 44(4). https://doi.org/10.1080/03075079.2017.1401059 DOI: https://doi.org/10.1080/03075079.2017.1401059

Chirikov, I. (2022). Does conflict of interest distort global university rankings? Higher Education. https://doi.org/10.1007/s10734-022-00942-5 DOI: https://doi.org/10.1007/s10734-022-00942-5

Cueva Estrada, J. M., Sumba Nacipucha, N., Meleán Romero, R., Artigas Morales, W., Cueva Ortiz, N. y Plaza Quimis, M. (2023a). Gestión del contenido en redes sociales, por revistas científicas indexadas en SciELO España. Revista de Comunicación de la SEECI, 56, 194–213. https://doi.org/10.15198/seeci.2023.56.e834 DOI: https://doi.org/10.15198/seeci.2023.56.e834

Cueva Estrada, J., Sumba Nacipucha, N., Paredes Floril, P., Sanchez-Bayon, A. y Carbo Guerrero, K. (2023b). Correlación entre Facebook y Google Scholar en el impacto de revistas científicas. Revista Latina de Comunicación Social, 81, 314–331. https://doi.org/10.4185/RLCS-2023-1897 DOI: https://doi.org/10.4185/RLCS-2023-1897

CWTS Leiden (2017). Information about the CWTS Leiden Ranking. Disponible en: https://www.leidenranking.com/information/general

De Albeniz, I. M. y Galarraga. A. (2022). Dilemas paradigmáticos y procesos de innovación en la cultura gastronómica del País Vasco. European Public & Social Innovation Review, 7(2), 84-98. https://pub.sinnergiak.org/esir/article/view/203 DOI: https://doi.org/10.31637/epsir.22-2.7

De Philippis, M. (2021). Multi-Task Agents and Incentives: The Case of Teaching and Research for University Professors. Economic Journal, 131(636). https://doi.org/10.1093/ej/ueaa119 DOI: https://doi.org/10.1093/ej/ueaa119

De Vicente Domínguez, A. M., Cea Esteruela, N. y Carballeda Camacho, M. R. (2022). La guía de expertos en las universidades privadas españolas: análisis de su presencia y gestión para la tranferencia del conocimiento científico. Revista de Ciencias de la Comunicación E Información, 27, 77-91. https://doi.org/10.35742/rcci.2022.27.e247 DOI: https://doi.org/10.35742/rcci.2022.27.e247

De-Moya-Anegón, F; Herrán-Páez, E; Bustos-González, A; Corera-Álvarez, El; Tibaná-Herrera, G (2017). Ranking Iberoamericano de instituciones de educación superior. SIR IBER 2017. Barcelona, España: Ediciones Profesionales de la Información SL. ISBN: 978 84 697 6288 2 http://doi.org/10.3145/sir-iber-2017 DOI: https://doi.org/10.3145/sir-iber-2017

Dobbins, M., Knill, C., & Vögtle, E. (2011). An analytical framework for the cross-country comparison of higher education governance. Higher Education, 62(5), 665–683. DOI: https://doi.org/10.1007/s10734-011-9412-4

Donina, D., & Paleari, S. (2019). New public management: global reform script or conceptual stretching? Analysis of university governance structures in the Napoleonic administrative tradition. Higher Education, 78(2). https://doi.org/10.1007/s10734-018-0338-y DOI: https://doi.org/10.1007/s10734-018-0338-y

Drucker, J., & Goldstein, H. (2007). Assessing the regional economic development impacts of universities: A review of current approaches. International Regional Science Review, 30(1), 20–46. https://doi.org/10.1177/0160017606296731 DOI: https://doi.org/10.1177/0160017606296731

Easley, J., Strawderman, L., Babski-Reeves, K., Bullington, S., & Smith, B. (2021). Perceived quality factors in higher education. Quality in Higher Education. https://doi.org/10.1080/13538322.2021.1909210 DOI: https://doi.org/10.1080/13538322.2021.1909210

Espeland, W. N., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40. https://doi.org/10.1086/517897 DOI: https://doi.org/10.1086/517897

Etzkowitz, H. (2016). The Entrepreneurial University: Vision and Metrics. Industry and Higher Education, 30(2), 83–97. https://doi.org/10.5367/ihe.2016.0303 DOI: https://doi.org/10.5367/ihe.2016.0303

Flores-Orozco, J., Villaseñor-Becerra, J., & Moreno-Arellano, C. (2015). Perspectivas actuales sobre los rankings mundiales de universidades. Revista de la educación Superior, XLIV(175), 41-67. https://doi.org/10.1016/j.resu.2015.09.001 DOI: https://doi.org/10.1016/j.resu.2015.09.001

Gadd, E., Holmes, R., & Shearer, J. (2021). Developing a method for evaluating global university rankings. Scholarly Assessment Reports, 3(1). https://doi.org/10.29024/sar.31 DOI: https://doi.org/10.29024/sar.31

Ganga-Contreras F., Quiroz, J. y P. Fossatti (2017). Isomorfismo Organizacional (IO): Breves aproximaciones teóricas y algunas aplicaciones a la Educación Superior. Revista Espacios. Vol. 38. N° 20. pp. 31-43.

Ganga-Contreras, F., Sáez San Martín, W., Rodríguez-Ponce, E., Calderón, A.-I., & Wandercil, M. (2018). Universidades Públicas de Chile y su Desempeño en los Rankings Académicos Nacionales. Fronteiras: Journal of Social, Technological and Environmental Science, 7(3), 316-341. https://doi.org/https://doi.org/10.21664/2238-8869.2018v7i3.p316-341 DOI: https://doi.org/10.21664/2238-8869.2018v7i3.p316-341

Ganga-Contreras, F., Sáez-San Martin, S., Viancos, P., & Abello, J (2023). Rankings universitarios, ni ángeles ni demonios: críticas y usos por parte de grupos de interés. Journal of the Academy, (9), 158-183. https://doi.org/10.47058/joa9.10 DOI: https://doi.org/10.47058/joa9.10

Ganga-Contreras, F., Sáez-San Martín, W y Viancos, P (2019). Rankings de universidades como herramientas de evaluación institucional: un análisis a las metodologías utilizadas por tres instrumentos internacionales. Revista Inclusiones Vol. 6 N°4. pp. 367-382.

Ganga-Contreras, F., Suárez-Amaya, W., Valderrama-Hidalgo, C. y Salazar-Botello, C. (2022). Rankings universitarios y su influencia en el desempeño: Comparando perspectivas expertas de Chile y Venezuela. Retos Revista de Ciencias de la Administración y Economía, 12(24), pp. 200-210. https://doi.org/10.17163/ret.n24.2022.01 DOI: https://doi.org/10.17163/ret.n24.2022.01

Grupo de Estudios Avanzados Universitas (2017). Ranking general de calidad de la docencia de pregrado 2017. Disponible en: http://infografias.elmercurio.com/ESPECIALuniversidades2017/pdf1/Ranking_Docencia.pdf

Guba, K., Zheleznov, A., & Chechik, E. (2023). Evaluating grant proposals: Lessons from using metrics as screening device. Journal of Data and Information Science, 8(2). https://doi.org/10.2478/jdis-2023-0010 DOI: https://doi.org/10.2478/jdis-2023-0010

Guerra Guerra, A., & Flores Polán, M. G. (2023). Fomento de la Innovación Social: La España vaciada. European Public & Social Innovation Review, 8(1), 73-94. https://pub.sinnergiak.org/esir/article/view/193 DOI: https://doi.org/10.31637/epsir-2023-193

Hazelkorn, E. (2011). Rankings and the reshaping of higher education: The battle for world-class excellence. In Rankings and the Reshaping of Higher Education: The Battle for World-Class Excellence. https://doi.org/10.1057/9780230306394 DOI: https://doi.org/10.1057/9780230306394

Lee, J. J., Vance, H., Stensaker, B., & Ghosh, S. (2020). Global rankings at a local cost? The strategic pursuit of status and the third mission. Comparative Education, 56(2), 236–256. https://doi.org/10.1080/03050068.2020.1741195 DOI: https://doi.org/10.1080/03050068.2020.1741195

Lee, S., & Bozeman, B. (2005). The impact of research collaboration on scientific productivity. In Social Studies of Science (Vol. 35, Issue 5, pp. 673–702). https://doi.org/10.1177/0306312705052359 DOI: https://doi.org/10.1177/0306312705052359

Luque-Martínez, T., del Barrio-García, S., Doña-Toledo, L., & Faraoni, N. (2023). Value generation and economic impact of the University of Granada. Studies in Higher Education, 48(3), 369–382. https://doi.org/10.1080/03075079.2022.2138848 DOI: https://doi.org/10.1080/03075079.2022.2138848

Lynch, K. (2015). Control by numbers: new managerialism and ranking in higher education. Critical Studies in Education, 56(2), 190–207. https://doi.org/10.1080/17508487.2014.949811 DOI: https://doi.org/10.1080/17508487.2014.949811

Matarín Rodríguez-Peral, E., Gómez Franco, T. y García Manso, A. (2022). Aplicación del Proceso Analítico Jerárquico en la comunicación digital de los organismos públicos durante la pandemia de la COVID-19. Revista Latina de Comunicación Social, 80, 89-117. https://doi.org/10.4185/RLCS-2022-1532 DOI: https://doi.org/10.4185/RLCS-2022-1532

Mohammed, A. M. S., Ukai, T., & Hall, M. W. (2022). University campuses’ role in accelerating the natural urban transformation process. Bulletin of Geography. Socio-Economic Series, 58, 75–96. https://doi.org/10.12775/bgss-2022-0035 DOI: https://doi.org/10.12775/bgss-2022-0035

NTU Ranking (2017). Indicators Perfomance Ranking of Scientific Papers for World Universities. Disponible en: http://nturanking.csti.tw/methodoloyg/indicators

Pereira, R., & Franco, M. (2023). University-Firm Cooperation and Regional Development: Proposal of a Model of Analysis. Journal of the Knowledge Economy, 14(2), 676–690. https://doi.org/10.1007/s13132-022-00947-6 DOI: https://doi.org/10.1007/s13132-022-00947-6

Print, M., & Hattie, J. (1997). Measuring quality in universities: An approach to weighting research productivity. Higher Education, 33(4), 453–469. https://doi.org/10.1023/A:1002956407943 DOI: https://doi.org/10.1023/A:1002956407943

Quacquarelli Symonds (2017). QS World University Rankings 2018 Supplement. Disponible en: https://www.topuniversities.com/student-info/qs-guides/qs-world-university-rankings-2018-supplement

Qué Pasa (2017). Ranking de Universiaddes 2017, Qué Pasa- La Tarcera. Disponible en: http://especiales.latercera.com/2017/universidades/article.html?id=0

Reyes, C. (2016). Medición de la calidad universitaria en Chile: La influencia de los rankings. Calidad en la Educación(44), 158-196. doi:10.4067/S0718-45652016000100007 DOI: https://doi.org/10.4067/S0718-45652016000100007

Sáez, W., Ganga-Contreras, F., Calderon, A. I., & Wandercil, M. (2023). ¿Universidades de Clase Mundial? Las universidades chilenas en los rankings académicos internacionales. Revista Meta: Avaliação, 15(46), 28. https://doi.org/10.22347/2175-2753v15i46.3892 DOI: https://doi.org/10.22347/2175-2753v15i46.3892

Sánchez-Barrioluengo, M., & Benneworth, P. (2019). Is the entrepreneurial university also regionally engaged? Analysing the influence of university’s structural configuration on third mission performance. Technological Forecasting and Social Change, 141, 206–218. https://doi.org/10.1016/j.techfore.2018.10.017 DOI: https://doi.org/10.1016/j.techfore.2018.10.017

Santos, J. M. R. C. A., Varela, C., & Martínez-Galán, E. (2022). A Framework for the Management of Research and Innovation Projects in Academic Settings. Journal of Research Administration, 53(2), 60–84.

Scimago (2017). Ranking Methodology. Disponible en: https://www.scimagoir.com/methodology.php

Scott, J. C. (2006). The mission of the university: Medieval to postmodern transformations. Journal of Higher Education, 77(1). https://doi.org/10.1080/00221546.2006.11778917 DOI: https://doi.org/10.1353/jhe.2006.0007

Sigl, L., Felt, U., & Fochler, M. (2020). “I am Primarily Paid for Publishing…”: The Narrative Framing of Societal Responsibilities in Academic Life Science Research. Science and Engineering Ethics, 26(3), 1569–1593. https://doi.org/10.1007/s11948-020-00191-8 DOI: https://doi.org/10.1007/s11948-020-00191-8

Song, J., & Yang, Z. (2023). Striving Transition for University Academics: The Academic Role Identity of Young Postdocs at Universities in China. SAGE Open, 13(1). https://doi.org/10.1177/21582440231153054 DOI: https://doi.org/10.1177/21582440231153054

Suárez-Amaya, Wendolin, & Ganga-Contreras, Francisco, & Barrios Fuentes, Eliana, & Pereira Burgos, Morela, & Villegas-Villegas, Francisco (2021). PERSPECTIVA DE LOS ACADÉMICOS VENEZOLANOS RESPECTO DE LOS RANKINGS UNIVERSITARIOS. Interciencia, 46(2),72-78.[fecha de Consulta 1 de Septiembre de 2021]. ISSN: 0378-1844. Disponible en: https://www.redalyc.org/articulo.oa?id=33966129004

Taşbaşı, K. (2023). Objecıón a la investigación generacional en los estudios de comunicacıón: una crítica teórica. Vivat Academia, Revista de Comunicación, 157, 1–24. https://doi.org/10.15178/va.2024.157.e1504 DOI: https://doi.org/10.15178/va.2024.157.e1504

Thompson-Whitside, S. (2016). Zen and the art of university rankings in art and design. She Ji= The Journal of Design, Economics, and Innovation, 2(3), 243-255. https://doi.org/:10.1016/j.sheji.2017.01.001 DOI: https://doi.org/10.1016/j.sheji.2017.01.001

Times Higher Education (2017). World University Rankings 2018 Methodology. Disponible en: https://www.timeshighereducation.com/world-university-rankings/methodology-world-university-rankings-2018

Troncoso, E.; Ganga-Contreras, F.; Briceño, M. (2022). Incentive Policies for Scientific Publications in the State Universities of Chile. Publications, 10(20). https://doi.org/10.3390/publications 10020020 DOI: https://doi.org/10.3390/publications10020020

U.S. News (2018). How U.S. News Calculated the Best Global Universities Rankings. Disponible en: https://www.usnews.com/education/best-global-universities/articles/methodology

U-Multirank (2017). U-Multirank's approach to university rankings. Disponible en: https://www.umultirank.org/about/methodology/our-approach/

URAP Research Laboratory (2017). Methodology: How we do. Disponible en: https://urapcenter.org/Methodology

Zagorac, I. (2017). The idea of the university and the role of philosophy. In Diacovensia (Vol. 25, Issue 4, pp. 531–547). https://doi.org/10.31823/d.25.4.2 DOI: https://doi.org/10.31823/d.25.4.2