Tett, R. P., & Christiansen, N. D. (2007). Personality tests at the crossroads: A response to Morgeson, Campion, Dipboye, Hollenbeck, Murphy, and Schmitt (2007)
Personnel Psychology
Vol 60, pp 967–993.
At a 2004 Society for Industrial and Organizational Psychology (SIOP) panel discussion, Morgeson et al., (2007) discuss the broad topic of personality tests and their acclaimed value in predicting job outcomes for selection purposes in organizations. The result of the panel discussion was a paper published in 2007 where they essentially concluded 2 things:
- Personality tests lack sufficient validity to warrant using them for selection purposes
- Applicant faking in personality tests doesn’t affect the validity of the test and may even be desirable in some situations.
If you have used personality tests or thinking of using them for selection purposes at your organization, your heart probably skipped a beat as you read the conclusions above. Not to worry, this is where the current research paper comes in. Tett and Christiansen (2007) carefully review the claims made by Morgeson et al. and concluded that there is evidence to support personality tests as valid predictors of job performance; thereby making them useful for selection purposes. Now you can breathe fine again. To arrive at their conclusions, they review each of Morgeson et al.’s claims and analyze it with meta-analytic evidence. (A meta-analysis is an examination of data from independent studies on a chosen topic in order to determine overall trends). Below is a summary of Tett and Christiansen’s conclusions.
Terms to Know
Validity: This is a test’s ability to measure what it is supposed to measure. Validity coefficient is the statistical index used to report evidence of validity for interpreting test scores. It also shows the magnitude of the correlation between a predictor (personality test) and an outcome (job performance).
Faking: The practice of skewing answers in an assessment to create a favorable (or otherwise) impression.
Conclusions on Personality Test Usage
Conclusion 1: The strength of personality test validity is increased when certain conditions are considered. Rather than dismiss personality tests as invalid predictors of job performance, more attention should be given to conditions where strong vs. weak or positive vs. negative linkages can be found between the personality traits and job performance. For example, the stronger the personality trait conscientiousness is, the better the outcome may be for a particular job type.
Conclusion 2: Personality test validity is stronger when confirmatory strategies are used to identify job-relevant traits. To explain what that means: When trying to identify which personality traits linked to performance in a given job, one may adopt an exploratory strategy where all the scales are administered to see which ones take hold; or a confirmatory strategy, where the job and the personality traits most related to it are assessed up front. For predicting job performance, confirmatory validity yields better results. An exploratory method yields validity coefficients not related to the job and detracts away from how well the personality test can be used to predict.
Conclusion 3: Personality-oriented job analysis (POJA) yields stronger validity for personality tests. Be sure to perform job analysis if you are thinking of using a personality assessment.
Conclusion 4: There is more predictive power at the sub-scale level of a personality trait than at the broad level. For example, the broad trait of conscientiousness has sub-scales such as organized, detail oriented, and compliance. Evidence shows that the sub-scales have more power to predict job performance than the broad scaled of conscientiousness.
Conclusion 5: When scores are combined on multiple trait measures, they offer incremental validity due to regression analysis. This is a statistical process that uses the predictor (personality test) to explain the percentage of variance in an outcome (job performance). For example, job performance can be explained by multiple different factors such as personality, organizational factors, job type and other unknown factors. Regression analysis can show how much of the variance in job performance can be explained by personality tests. Assuming the variance in job performance is 100%, personality tests can explain 24% of the variance. In the psychology world, any predictor that can explain a statistically significant variability in a construct is useful.
Conclusion 6: Incremental validity is possible by considering how the different traits interact with each other to predict job outcomes. For example, exploring how conscientiousness interacts with agreeableness may help explain more job outcomes than just looking at either trait by itself.
Conclusion 7: Although applicant faking weakens personality test validity, enough trait variance remains to be useful for predicting job performance. Faking is a problem and doesn’t indicate social competence or job success.
Conclusion 8: If applicants are told what trait is being assessed, they tend to show overly desirable responses.
Conclusion 9: Administering items from the same scale as a set may create a bias and faking in the way applicants respond because the items are within the same context. Independent observations of the items are better for the validity of personality tests.
Conclusion 10: Contemporary personality tests have been shown to be psychometrically sound; no research suggests that personality test items are so ambiguous or confusing as to undermine validity.
In summary, commercial self-report personality tests are useful in selection scenarios for predicting job performance when the necessary conditions are considered. Keep in mind that faking weakens but does not destroy meaningful levels of personality test validity. Follow the tips below to gain incremental validity if you want to use personality tests in your organization:
- Conduct a job analysis
- Use confirmatory strategies to link personality traits to job requirements
- Do not reveal what trait is being assessed to the applicants
- Randomize the items when administering the test
- Review results at the sub-scale level
- Perform regression analysis
- Review interactions between the different traits
- Review the strength of the broad individual traits alone rather than the average of all the traits
All the best with your selection strategy!
Article by Ore Osikoya & Brandon Jordan
One thought on “Closing the Scientist-Practitioner Gap: 10 Reasons to Consider Personality Tests for your Selection Strategy”