The implications of unconfounding multisource performance ratings

Jackson, Duncan J. R., Michaelides, George ORCID:, Dewberry, Chris, Schwenke, Benjamin and Toms, Simon (2020) The implications of unconfounding multisource performance ratings. Journal of Applied Psychology, 105 (3). 312–329. ISSN 0021-9010

[thumbnail of UnconfoundingRatings2019]
PDF (UnconfoundingRatings2019) - Accepted Version
Download (1MB) | Preview


The multifaceted structure of multisource job performance ratings has been a subject of research and debate for over 30 years. However, progress in the field has been hampered by the confounding of effects relevant to the measurement design of multisource ratings and, as a consequence, the impact of ratee-, rater-, source-, and dimension-related effects on the reliability of multisource ratings remains unclear. In separate samples obtained from 2 different applications and measurement designs (N [ratees] = 392, N [raters] = 1,495; N [ratees] = 342, N [raters] = 2,636), we, for the first time, unconfounded all systematic effects commonly cited as being relevant to multisource ratings using a Bayesian generalizability theory approach. Our results suggest that the main contributors to the reliability of multisource ratings are source-related and general performance effects that are independent of dimension-related effects. In light of our findings, we discuss the interpretation and application of multisource ratings in organizational contexts.

Item Type: Article
Uncontrolled Keywords: 360-degree ratings,bayesian generalizability theory,multisource performance ratings,360-degree feedback,variance-components,perspective,reliability,multisource performance ratings,rater,criterion construct space,job-performance,error,validity,generalizability,applied psychology ,/dk/atira/pure/subjectarea/asjc/3200/3202
Faculty \ School: Faculty of Social Sciences > Norwich Business School
Related URLs:
Depositing User: LivePure Connector
Date Deposited: 25 Jul 2019 02:42
Last Modified: 25 May 2023 14:31
DOI: 10.1037/apl0000434


Downloads per month over past year

Actions (login required)

View Item View Item