How to measure the intervention process? An assessment of qualitative and quantitative approaches to data collection in the process evaluation of organizational interventions

Abildgaard, Johan S., Saksvik, Per Ø. and Nielsen, Karina (2016) How to measure the intervention process? An assessment of qualitative and quantitative approaches to data collection in the process evaluation of organizational interventions. Frontiers in Psychology, 7. ISSN 1664-1078

Preview

PDF (Published manuscript) - Published Version Available under License Creative Commons Attribution. Download (215kB) | Preview

Abstract

The evaluation of organizational interventions targeting employee health and wellbeing has been found to be a challenging task (Murta et al., 2007). The use of process evaluation, defined as the evaluation of “individual, collective or management perceptions and actions in implementing any intervention and their influence on the overall result of the intervention.” Nytrø et al. (2000) has served to increase focus on the evaluation of the specific intervention processes and not only the outcomes. Although several evaluation frameworks (Nielsen and Abildgaard, 2013; Nielsen and Randall, 2013) have been suggested it has proven to be methodologically challenging to evaluate the processes of implementation of organizational interventions (OIs; Nielsen and Randall, 2013). Two distinct approaches to process evaluation data collection are commonly used. One is a quantitative approach where either standardized or intervention-specific questionnaire items are included in a follow-up questionnaire, and are later integrated into statistical models of implementation and effect (e.g., Nielsen et al., 2007; Nielsen and Randall, 2009, 2012). The other is the collection of qualitative data; often specifically as a supplement to quantitative data, using semi-structured interviews with employees and managers, (Dahl-Jørgensen and Saksvik, 2005; Nielsen et al., 2006), observations of intervention activities (Brannan and Oultram, 2012), or long-term field observations (Czarniawska-Joerges, 2007). Qualitative process evaluation has been used extensively to understand the context of interventions outcomes (e.g., Mikkelsen and Saksvik, 1998; Saksvik et al., 2002; Nielsen et al., 2006; Aust et al., 2010). Each data source has its methodological strengths and weaknesses and the concurrent mixed methods use of both quantitative and qualitative approaches has been proposed as a potential middle ground (Dahl-Jørgensen and Saksvik, 2005; Nielsen and Randall, 2013). Mixed methods is here defined “as a method [which] focuses on collecting, analyzing and mixing both quantitative and qualitative data in a single or series of studies. Its central premise it that the use of [both] approaches in combination provides a better understanding of research problems than either approach alone” (Creswell and Plano Clark, 2011, p. 5). Although much is written about evaluation research in general (Lipsey and Cordray, 2000; Rossi et al., 2004; Pawson, 2013), and mixed methods evaluation in general (Rallis and Rossman, 2003; Nastasi et al., 2007) the particularities and methodological considerations of using qualitative and quantitative data in mixed methods based process evaluation have been sparse (Nastasi et al., 2007), particularly concerning the specifics of evaluating OIs (Nielsen and Abildgaard, 2013). Using a case of an OI in the Danish postal service where questionnaires and semi-structured interviews were used for process evaluation data collection, we compare the epistemological properties of both methods and assess the benefits of different ways to collect process information. The aim of the present study is to examine the type of knowledge about the intervention process that may be produced by quantitative and qualitative data and discuss how these sources best can be applied in mixed methods designs. It is hence not a study of different forms of mixed methods designs (for such literature see Nastasi et al., 2007; Teddlie and Tashakkori, 2009; Creswell and Plano Clark, 2011) but instead an assessment of the properties and potential roles of specific data sources in mixed methods OI evaluation. We employ a sequential mixed methods analysis to identify a set of factors in the quantitative data that function as an analytical framework with which we comparatively analyze the qualitative data. This approach will help us accentuate what knowledge about the intervention each data collection methods may provide, and allows us to discuss differences and similarities.

Item Type:	Article
Additional Information:	Copyright © 2016 Abildgaard, Saksvik and Nielsen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Faculty \ School:	Faculty of Social Sciences > Norwich Business School
UEA Research Groups:	Faculty of Social Sciences > Research Groups > Employment Systems and Institutions
Related URLs:	http://journal.frontiersin.org/article/1...
Depositing User:	Pure Connector
Date Deposited:	26 Sep 2016 11:00
Last Modified:	24 May 2023 15:30
URI:	https://ueaeprints.uea.ac.uk/id/eprint/60583
DOI:	10.3389/fpsyg.2016.01380

Downloads

Actions (login required)

View Item