Challenges of impact assessment in Social Innovation: A qualitative study from two European rural regions
Desafíos de la evaluación de impacto en la Innovación Social: Un estudio cualitativo de dos regiones rurales europeas
Keywords:
social innovation, impact assessment, rural regions, local development initiatives, Austria, PortugalAbstract
Over recent years, various approaches to assessing the impacts of social innovation (SI) have developed without a uniform method having arisen. There are some issues around how impacts can be assessed, connected with the questions on the nature of impacts, the levels of analysis and effects of a purely positivist approach to impact assessment. While attempting to assess such impacts, various SI initiatives face the diversity of challenges. To this end, the aim of the article is to investigate the experience of said initiatives promoting social innovation related to impact assessment of social innovation. The paper is based on an empirical study conducted with the local development associations and local action groups involved in social innovation projects in two rural regions of Austria and Portugal. The results indicate that, despite recognising the importance of impact assessment regarding social innovation activities and the opportunities it provides, local organisations in question face many challenges in assessing the impacts of social innovation, including conceptual and practical difficulties.
Resumen
En los últimos años se han desarrollado diversos enfoques para evaluar los impactos de la innovación social (IS) sin que haya surgido un método uniforme. Existen algunas cuestiones en torno a la forma de evaluar los impactos, relacionadas con las preguntas sobre la naturaleza de los impactos, los niveles de análisis y los efectos de un enfoque puramente positivista de la evaluación de impactos. Al intentar evaluar dichos impactos, varias iniciativas de IS se enfrentan a la diversidad de desafíos. Para ello, el objetivo del artículo es investigar la experiencia de dichas iniciativas que promueven la innovación social en relación con la evaluación del impacto de la innovación social. El artículo se basa en un estudio empírico realizado con asociaciones de desarrollo local y grupos de acción local que participan en proyectos de innovación social en dos regiones rurales de Austria y Portugal. Los resultados indican que, a pesar de reconocer la importancia de la evaluación de impacto en relación con las actividades de innovación social y las oportunidades que ofrece, las organizaciones locales en cuestión se enfrentan a muchos retos a la hora de evaluar los impactos de la innovación social, incluyendo dificultades conceptuales y prácticas.
Palabras clave: innovación social; evaluación de impacto; regiones rurales; iniciativas de desarrollo local; Austria; Portugal.
Downloads
References
Antadze, N., & Westley, F. R. (2012). Impact metrics for social innovation: Barriers or bridges to radical change? Journal of Social Entrepreneurship, 3(2), 133-150. https://doi.org/10.1080/19420676.2012.726005
Arena, M., Azzone, G., & Bengo, I. (2015). Performance measurement for social enterprises. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 26(2), 649-672. https://www.jstor.org/stable/43654699
Arya, A., & Mittendorf, B. (2015). Supply chain consequences of subsidies for corporate social responsibility. Production and Operations Management, 24(8), 1346-1357. https://doi.org/10.1111/poms.12326
Avelino, F., Wittmayer, J. M., Pel, B., Weaver, P., Dumitru, A., Haxeltine, A., ... & O'Riordan, T. (2019). Transformative social innovation and (dis) empowerment. Technological Forecasting and Social Change, 145, 195-206. https://doi.org/10.1016/j.techfore.2017.05.002
Avelino, F. (2021). Theories of power and social change. Power contestations and their implications for research on social change and innovation. Journal of Political Power, 1-24. https://doi.org/10.1080/2158379X.2021.1875307
Baturina, D., & Bežovan, G. (2015). Social innovation impact-review. Seventh Framework Programme, Third Sector Impact. Brussels.
BEPA. (2010). Empowering People, Driving Change. Social Innovation in the European Union. Bureau of European Policy Advisers and European Commission.
Bock, B. B. (2016). Rural marginalisation and the role of social innovation; a turn towards nexogenous development and rural reconnection. Sociologia Ruralis, 56(4), 552-573. https://doi.org/10.1111/soru.12119
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.
Bund, E., Gerhard, U., Hoelscher, M., & Mildenberger, G. (2015). A methodological framework for measuring social innovation. Historical Social Research/Historische Sozialforschung, 48-78.
Burdge, R. J., & Vanclay, F. (1996). Social impact assessment: a contribution to the state of the art series. Impact Assessment, 14(1), 59-86. https://doi.org/10.1080/07349165.1996.9725886
Chatzichristos, G., & Nagopoulos, N. (2020). Regional Institutional Arenas for Social Innovation: mixed methods research. Journal of Social Entrepreneurship, 1-23. https://doi.org/10.1080/19420676.2019.1705378
Clark, C., Rosenzweig, W., Long, D., & Olsen, S. (2004). Double Bottom Line Project Report: Assessing Social Impact in Double Line Ventures. New York. Available online at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.516.916&rep=rep1&type=pdf (accessed 21 November).
Cunha, J., & Benneworth, P. (2020). How to measure the impact of social innovation initiatives? International Review on Public and Nonprofit Marketing, 17(1), 59-75. https://doi.org/10.1007/s12208-019-00240-4
Di Iacovo, F., Moruzzo, R., Rossignoli, C., & Scarpellini, P. (2014). Transition management and social innovation in rural areas: Lessons from social farming. The Journal of Agricultural Education and Extension, 20(3), 327-347. https://doi.org/10.1080/1389224X.2014.887761
Ebrahim, A., & Rangan, V. K. (2010). The limits of nonprofit impact: A contingency framework for measuring social performance (No. 10-099). Harvard Business School.
Ebrahim, A., & Rangan, V. K. (2014). What impact? A framework for measuring the scale and scope of social performance. California management review, 56(3), 118-141. https://doi.org/10.1525/cmr.2014.56.3.118
Epstein, M.J., & Yuthas, K. (2014). Measuring and improving social impacts: A guide for nonprofits, companies, and impact investors. Berrett-Koehler Publishers. Available online: https://www.bkconnection.com/static/Measuring_and_Improving_Social_Impacts_EXCERPT.pdf (accessed 23 April 2021).
European Commission. (2014). Proposed approaches to social impact measurement in European Commission legislation and in practice relating to EuSEFs and the EaSI: GECES sub-group on impact measurement. European Commission Publications Office. https://data.europa.eu/doi/10.2767/29691
Eurostat. (2019). Available online: https://ec.europa.eu/eurostat/statistics-explained/index.php/Archive:Rural_development_statistics_by_urban-rural_typology (accessed 12 December 2019).
Fougère, M., & Meriläinen, E. (2021). Exposing three dark sides of social innovation through critical perspectives on resilience. Industry and Innovation, 28(1), 1-18. https://doi.org/10.1080/13662716.2019.1709420
Goldstein, J., Hazy, J. K., & Silberstang, J. (2010). A complexity science model of social innovation in social enterprise. Journal of social entrepreneurship, 1(1), 101-125. https://doi.org/10.1080/19420671003629763
Grieco, C. (2015). Assessing social impact of social enterprises: Does one size really fit all? Springer.
Howaldt, J., Domanski, D., & Kaletka, C. (2016). Social innovation: Towards a new innovation paradigm. RAM. Revista de Administração Mackenzie, 17(6), 20-44. https://doi.org/10.1590/1678-69712016/administracao.v17n6p20-44
Howaldt, J., Kaletka, C., Schröder, A., & Zimgiebl, M. (Eds.). (2018). Atlas of social innovation: new practices for a better future. Technische Universität Dortmund, ZWE Sozialforschungsstelle.
Köhler, J., Geels, F. W., Kern, F., Markard, J., Onsongo, E., Wieczorek, A., ... & Wells, P. (2019). An agenda for sustainability transitions research: State of the art and future directions. Environmental innovation and societal transitions, 31, 1-32. https://doi.org/10.1016/j.eist.2019.01.004
Kolodinsky, R. W., Madden, T. M., Zisk, D. S., & Henkel, E. T. (2010). Attitudes about corporate social responsibility: Business student predictors. Journal of Business Ethics, 91(2), 167-181. https://doi.org/10.1007/s10551-009-0075-3
Krlev, G., Bund, E., & Mildenberger, G. (2014). Measuring what matters—Indicators of social innovativeness on the national level. Information Systems Management, 31(3), 200-224. https://doi.org/10.1080/10580530.2014.923265
Liket, K. C., & Maas, K. (2015). Nonprofit organizational effectiveness: Analysis of best practices. Nonprofit and Voluntary Sector Quarterly, 44(2), 268-296. https://doi.org/10.1177/0899764013510064
Liket, K. C., Rey-Garcia, M., & Maas, K. E. (2014). Why aren’t evaluations working and what to do about it: A framework for negotiating meaningful evaluation in nonprofits. American Journal of Evaluation, 35(2), 171-188. https://doi.org/10.1177/1098214013517736
Ludvig, A., Weiss, G., Sarkki, S., Nijnik, M., & Živojinović, I. (2018). Mapping European and forest related policies supporting social innovation for rural settings. Forest Policy and Economics, 97, 146-152. https://doi.org/10.1016/j.forpol.2018.09.015
Maas, K., & Grieco, C. (2017). Distinguishing game changers from boastful charlatans: Which social enterprises measure their impact? Journal of Social Entrepreneurship, 8(1), 110-128. https://doi.org/10.1080/19420676.2017.1304435
Maas, K., & Liket, K. (2011). Social impact measurement: Classification of methods. In R. Burritt, S. Schaltegger, M. Bennett, T. Pohjola & M. Csutora (Eds.), Environmental management accounting and supply chain management (pp. 171-202). Springer.
MacCallum, D. (2009). Social innovation and territorial development. Ashgate Publishing.
Moulaert, F., Martinelli, F., Swyngedouw, E., & Gonzalez, S. (2005). Towards alternative model (s) of local innovation. Urban studies, 42(11), 1969-1990. https://doi.org/10.1080/00420980500279893
Moulaert, F. (2013). The international handbook on social innovation: collective action, social learning and transdisciplinary research. Edward Elgar Publishing.
Mulgan, G. (2010). Measuring social value. Stanford Social Innovation Review, 8(3), 38-43.
Neumeier, S. (2012). Why do social innovations in rural development matter and should they be considered more seriously in rural development research? –Proposal for a stronger focus on social innovations in rural development research. Sociologia Ruralis, 52(1), 48-69. https://doi.org/10.1111/j.1467-9523.2011.00553.x
Neumeier, S. (2017). Social innovation in rural development: identifying the key factors of success. The geographical journal, 183(1), 34-46. https://doi.org/10.1111/geoj.12180
Nicholls, A., Simon, J., & Gabriel, M. (2015). Introduction: Dimensions of social innovation. In A. Nicholls, J. Simon & M. Gabriel (Eds.), New frontiers in social innovation research (pp. 1-26). Palgrave Macmillan.
Niccholls, A. (2015). Synthetic grid: a critical framework to inform the development of social innovation metrics. CRESSI Working papers, 14. Oxford, Creating Economic Space for Social Innovation (CRESSI) project.
Nicholls, A. (2018). A general theory of social impact accounting: Materiality, uncertainty and empowerment. Journal of Social Entrepreneurship, 9(2), 132-153.
Nicholls, A., & Dees, J.G. (2015). Social Innovation. In J. D. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences (pp. 355-361). Elsevier.
Nijnik, M., Secco, L., Miller, D., & Melnykovych, M. (2019). Can social innovation make a difference to forest-dependent communities? Forest Policy and Economics, 100, 207-213. https://doi.org/10.1016/j.forpol.2019.01.001
Novikova, M., de Fátima Ferreiro, M., & Stryjakiewicz, T. (2020). Local Development Initiatives as Promoters of Social Innovation: Evidence from Two European Rural Regions. Quaestiones Geographicae, 39(2), 43-53. https://doi.org/10.2478/quageo-2020-0012
Novikova, M. (2021). Transformative Social Innovation in Rural Areas: Insights from a Rural Development Initiative in the Portuguese Region of Baixo Alentejo. European Countryside, 13(1), 71-90. https://doi.org/10.2478/euco-2021-0005
Noy, C. (2008). Sampling knowledge: The hermeneutics of snowball sampling in qualitative research. International Journal of social research methodology, 11(4), 327-344. https://doi.org/10.1080/13645570701401305
Nyseth, T., & Hamdouch, A. (2019). The transformative power of social innovation in urban planning and local development. Urban Planning, 4(1), 1-6. https://doi.org/10.17645/up.v4i1.1950
Organisation for Economic Cooperation and Development. (2006). The new rural paradigm: Policies and governance. OECD Publishing.
Phills, J. A., Deiglmeier, K., & Miller, D. T. (2008). Rediscovering social innovation. Stanford Social Innovation Review, 6(4), 34-43.
Preskill, H., & Beer, T. (2012). Evaluating social innovation. Available online: https://repository.fteval.at/146/1/2012_Evaluating%20Social%20Innovation.pdf (assessed 11 June 2021).
Ravazzoli, E., Dalla Torre, C., Da Re, R., Marini Govigli, V., Secco, L., Górriz-Mifsud, E., ... & Nijnik, M. (2021). Can social innovation make a change in European and Mediterranean marginalized areas? Social innovation impact assessment in agriculture, fisheries, forestry, and rural development. Sustainability, 13(4), 1823. https://doi.org/10.3390/su13041823
Schuitmaker, T. J. (2012). Identifying and unravelling persistent problems. Technological Forecasting and Social Change, 79(6), 1021-1031. https://doi.org/10.1016/j.techfore.2011.11.008
Tracey, P., & Stott, N. (2017). Social innovation: a window on alternative ways of organizing and innovating. Innovation, 19(1), 51-60. https://doi.org/10.1080/14479338.2016.1268924
Vanclay, F., Baines, J. T., & Taylor, C. N. (2013). Principles for ethical research involving humans: ethical professional practice in impact assessment Part I. Impact assessment and project appraisal, 31(4), 243-253. https://doi.org/10.1080/14615517.2013.850307
Vanclay, F. (2003). International principles for social impact assessment. Impact assessment and project appraisal, 21(1), 5-12. https://doi.org/10.3152/147154603781766491
Vanclay, F. (2020). Reflections on Social Impact Assessment in the 21st century. Impact Assessment and Project Appraisal, 38(2), 126-131. https://doi.org/10.1080/14615517.2019.1685807
Van der Have, R.P. and Rubalcaba, L., 2016. Social innovation research: An emerging area of innovation studies? Research Policy, 45(9), 1923-1935. https://doi.org/10.1016/j.respol.2016.06.010
Westley, F., & Antadze, N. (2010). Making a difference: Strategies for scaling social innovation for greater impact. Innovation Journal, 15(2), 1-19.
Wittmayer, J. M., Backhaus, J., Avelino, F., Pel, B., Strasser, T., Kunze, I., & Zuijderwijk, L. (2019). Narratives of change: How social innovation initiatives construct societal transformation. Futures, 112, 102433. https://doi.org/10.1016/j.futures.2019.06.005
Ziegler, R. (2017). Social innovation as a collaborative concept. Innovation: The European Journal of Social Science Research, 30(4), 388-405. https://doi.org/10.1080/13511610.2017.1348935
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Non Commercial, No Derivatives Attribution 4.0. International (CC BY-NC-ND 4.0.), that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
