Performance evaluation systems are important tools that organizations use to motivate employees, provide them with productive feedback, and align their goals with those of the organization. Organizations today are adopting increasingly complex performance evaluation systems that often feature two key elements: 1) They contain multiple performance measures, and 2) they require supervisors to use their subjective judgment.
Using multiple performance measures can help supervisors evaluate their subordinates’ skills and qualities across various dimensions that the organization values. Allowing supervisors to exercise their subjective judgment in performance evaluations can also be beneficial because they can incorporate aspects of their subordinates’ performance that can’t be measured precisely or objectively. A major problem with subjective evaluations is that supervisor biases can come into play.
We wanted to better understand how subtle designs of a subjective performance evaluation system can affect supervisors’ performance evaluations of their employees. Specifically, does it matter whether supervisors use a holistic evaluation system, where supervisors assign a single overall performance rating, or a detailed evaluation system, where supervisors assign separate ratings for each performance measure and an overall rating? We conducted a survey of supervisors and subordinates to get a better understanding of how these two evaluation systems are used and viewed. We designed an experiment to examine whether using a holistic or detailed performance evaluation system will indeed make a difference on supervisors’ assessments (see “Disaggregated versus Holistic Performance Evaluations in a Promotion Setting,” Journal of Management Accounting Research, Fall 2022).
Holistic vs. Detailed Evaluation Systems
Using Amazon’s Mechanical Turk, we screened and surveyed 54 supervisors and 52 subordinates based on their work experience and familiarity with their organization’s performance evaluation system. We learned that approximately 80% of the respondents’ organizations use a detailed evaluation system and 20% use a holistic evaluation system. Both types of evaluation systems are common across different company sizes and industries. The typical evaluation system contains three to four different performance measures, most of which relate to an employee’s current job, but some also relate to the employee’s next job after promotion.
We also learned from the survey results that, compared to a holistic evaluation system, respondents generally prefer a detailed evaluation, which they think is fairer and provides more useful feedback, even though it takes supervisors more time and effort to complete. Interestingly, our respondents, particularly supervisors, think that the two different evaluation systems can potentially lead to different assessments.
We then decided to test the research question of whether using a holistic or detailed performance evaluation system makes a difference on assessments in the context where supervisors must make a promotion decision shortly after assessing their subordinates’ job performance.
Deciding whom to promote is a critical personnel decision that supervisors must often make. What makes promotion decisions particularly fascinating is that they aren’t always tied to employees’ performance in their current job. The nature of an employee’s job and their skill requirement can change drastically after a promotion. For example, when a rank-and-file employee (e.g., engineer, salesperson, or accountant) gets promoted to a manager position, skills like leadership and people management become much more important.
In this case, supervisors can face the dilemma of either promoting their best-performing employee in the current job or the employee best suited for success at the promoted position. The problem with promoting the best current job performer is that the promoted employee could turn out to be ill-suited and incompetent for the new job.
Our Study
With this promotion setting in mind, we conducted an experiment with 120 MBA students in which they took on the role of a retail store manager tasked with evaluating the performance of two sales associates and then deciding which sales associate to promote to an open assistant manager position. We described one sales associate as being better at the current job (i.e., star salesperson with the ability to develop strong relationships with customers) and the other associate as being better suited for promotion (i.e., strong organizational skills and attention to detail). This creates a dilemma in the supervisor’s promotion decision because they must strike a delicate balance between rewarding the best current job performer vs. promoting the employee best suited for the higher-level job.
To test whether the type of evaluation system influences supervisors’ assessments of subordinates’ performance, and whether supervisors’ anticipation of the upcoming promotion matter, we required half of the participants to complete their performance assessments under a holistic evaluation system, whereas the other half completed the same assessments under a detailed evaluation system. Second, we told half of the participants that they’ll make a subsequent promotion decision before they complete their performance assessments, whereas the other half didn’t know about the upcoming promotion until after they completed their performance assessment.
From our experiment, we learned that when supervisors don’t know that a promotion is coming up, the evaluation system doesn’t matter as supervisors give the same overall performance ratings under both a holistic and a detailed system. Yet when supervisors know that a promotion is coming up after they complete their performance assessments, they give different overall ratings under the two systems. Specifically, supervisors give a higher overall rating to the employee who’s best suited for promotion under a holistic system as compared to a detailed system. We found further evidence showing that some supervisors do this strategically because they want to justify promoting that employee later, while others do this subconsciously because they mentally place greater focus on the performance measures that relate to the upcoming promotion as they complete the performance assessments.
Key Takeaways
Our research highlights some of the pros and cons that organizations should consider carefully when designing their performance evaluation systems. In a promotion setting, we find that, on one hand, a holistic system allows supervisors to better align their performance assessments with their desired promotion decision, which can increase the likelihood that they’ll later promote the subordinate best suited for promotion. On the other hand, a holistic system raises fairness issues as it arguably biases against the employee best at the current job whose performance assessment is negatively affected when the supervisor focuses on the upcoming promotion.
Ultimately, whether an organization should adopt a holistic or detailed evaluation system depends on its goals, values, and culture. For example, for an organization that values empowering its managers, a holistic evaluation system could be preferable because it gives managers more flexibility to exercise their discretion and control during the assessment process. Yet for an organization that values equity and employee trust, a detailed evaluation system could be preferable because it reduces supervisors’ real or perceived biases in their assessments and likely provides more informative feedback for employees.
Our survey findings that more organizations appear to use a detailed evaluation system may suggest that maintaining fairness perceptions and providing employee feedback could be the priority for many organizations.
August 2023
