« Back to the January/February 2005 Explorer newsletter

FEATURE

Managerial Assessment in the 21st Century

By Charles Handler, Ph.D.

Much of what I’ve written over the past several years has discussed the use of assessment tools in very broad terms. I’ve spent most of my time discussing basic concepts of assessment rather then drilling into detail about specific types of tools or application to specific jobs or industries. All for a cause: I’ve been trying to give a general understanding of the basic concepts of assessment and to promote the idea that, when used correctly, assessment is a very good thing.

I’m happy to report that over 3 years of hammering away at communicating the importance of assessment while discussing it at a general level seem to have had an impact. So, for 2005, I’m ready to branch out and start discussing some of the subtleties around the use of assessment in different situations. For instance, towards the end of last year, I began writing about the ins and outs of hourly hiring and the role of technology and assessment in helping organizations maximize the value of their hourly hiring processes (embed link to articles).

KICKING OFF THE YEAR

I want to address a very important topic, which is a logical extension of where I left off last year: assessment for managerial level hiring. While the basic idea of collecting information used to help hiring personnel make predictions about an applicant’s ability to perform a job remains the same, in many ways assessment for managerial positions is a different proposition from that used for hourly positions.

This article presents some basic information about the challenges associated with assessing candidates for managerial level positions as well as some ideas about how technology can help organizations to effectively address some of these challenges.

MANAGERIAL vs HOURLY

How is managerial level hiring different from hourly hiring? The obvious difference lies in the fact that managerial level jobs are often quite different from hourly ones. Probably not a surprise. While there are many differences between these two types of jobs, it boils down to one major thing in my mind:

Managerial level jobs are more complex then hourly jobs.

For instance, performance for most hourly jobs, while of critical importance to the organization, often boils down to 2 or 3 key elements.

• Will the employee show up for work?
• Can the employee provide needed levels of customer service?
• Can the employee be trusted?

CHALLENGE #1

While managerial level performance often includes some of the key aspects of hourly hiring, it also requires a whole lot more. I like to use the analogy of a pie chart to explain these differences. For instance, a pie chart outlining the key issues most hourly level performance pie has only a few big pieces. Identify these key pieces and measure each of them in your hiring process and you should be able to field a pretty good hiring system.

On the other hand, the managerial level performance pie has a heck of a lot more pieces. Think about it, managers have to perform a wider variety of tasks and do all of them well to be able to add value. This makes for a lot more pieces of pie. We selection specialists always have to be concerned with how to account for each piece of pie that really matters in the selection system we develop. The more complex the job, the more difficult it becomes to identify the specific pieces of the pie that make the most difference. When we look at this empirically, it is often the case that we find that predicting performance for these jobs is a situation where understanding the whole pie requires more then just understanding provided by each individual slice. Instead, all the pieces of pie contribute to a whole that is hard to measure using any few of them alone.

CHALLENGE #2

A second difficulty in measuring managerial job performance is in the measurement of job performance itself. Success in many jobs, including many hourly ones, boils down to very objective and verifiable information. We I/O Psychologists dream of these situations because they provide a very clean index of performance that can be used to help organizations understand the direct relationship between their selection system and ROI. Unfortunately, when it comes to managerial performance it is often much more difficult to obtain a set of numbers that cleanly summarize performance. This is a direct result of the pie analogy used earlier. Managerial level performance is the summation of many important but harder to define characteristics. For instance, while it is easy enough to develop assessments to measure important factors in managerial performance such as interpersonal skills, situational judgment, decision making; it is much harder to find the truth when attempting to evaluate how to objectively measure these things and determine their role in overall performance.

WORTH THE EFFORT

Just because managerial performance presents some difficulties does not mean that it is not an important thing to measure. This is directly related to another part of the pie analogy in that, as a general rule of thumb, the more pieces of pie that are needed to define performance, the more value the job has to the organization and therefore the more important it is to make good hiring decisions when filling that job.

So, while some elements of the hiring process are the same for hourly and managerial level workers, there are some important differences to note. Accurate prediction for managerial level jobs is not achieved simply by measuring more pieces of pie. It does rely more heavily on the use of assessment tools but it also requires expert judgment to look over assessment results and interpret them relative to job performance. This is known as a “clinical” model of assessment interpretation. This model has been used extensively for higher level jobs for quite some time and has proven to be very effective under the right conditions. One such condition is the expertise and experience of the person who is responsible for interpreting the results of the assessment and making predictions about the individual’s ability to perform the job in question. In managerial assessment the complexity of interpretation requires an expert. This is somewhat different from hourly hiring because it requires the interpretation of much more extensive information in order to draw conclusions regarding the individual’s ability to perform the job. When done well, the extra time and expense of this method is easily justified (but often hard to measure).

TECHNOLOGY AIDS

While both research and practice have proven this model to be very effective, it is often out of the reach of many organizations due to the time and expense that is involved. I believe that technology offers a way to help change this status quo. Technology based expert systems provide organizations with the ability to automate the interpretation of complex managerial assessments and therefore help to facilitate the accuracy of managerial selection decisions. This usually takes the form of a very detailed report outlining the potential strengths and weaknesses of the candidate relative to the job. Such a report is rarely a simple repetition of assessment results, but rather more of an expert opinion on what the combination of assessment results means about the individual in question’s suitability for a particular job. These kinds of judgments require an expert perspective that has been based on years of experience and practice in interpretation of assessment results.

The ability to automate the output of this process is the real value proposition provided by technology. We are now able to leverage the expertise of experienced individuals to create expert systems that provide the same exact type of output found in high touch individualized assessment. The advantage of these systems is their ability to provide the same quality of output while requiring significantly less time and money on the part of the organization.

HUMAN ELEMENT REQUIRED

While automated expert systems for managerial selection offer an excellent way to help make managerial level hiring systems, it is important to remember that these systems were not meant to replace human judgment, only to help provide decision makers with richer information that accurately reflects the subtleties of managerial job performance. That being said, the ability to create expert systems that are able to interpret large amounts of data and synthesize it into a summary report can help move the traditionally more expensive process down to a level that is accessible to many individuals. An additional strength of this model is that it facilitates direct linkages to developmental systems.

As with every other type of hiring, the future of managerial hiring will definitely involve the use of expert systems to help collect and interpret information that can be used to help organizations make more accurate hiring decisions. However, it is important to remember that simply automating the ability to generate useful reports regarding a candidate’s suitability for a job alone is not enough to ensure that your organization is able to systematically make good managerial level hiring decisions.

POINTS TO CONSIDER

Organizations interested in the use of these types of systems should be sure to consider the following:

Understanding the job is key - This may be harder for more complex jobs but without a clear understanding of what is required for performance, it will be very difficult to systematically make good hiring decisions.

Always be systematic - It is important to ensure that hiring occurs via a systematic process in which a variety of pieces of information are collected and used to support decision making. This system must be able to account for the management of candidate information while also providing key pieces of data to those using their expertise to make decisions

Take advantage of automated expert judgment - If assessment is not a core competency for your organization, automated expert systems are a great option. These systems allow organizations to take advantage of the value in high touch assessments for a wider variety of jobs than in the past. This in turn provides a serious advantage with the ability to make sound hiring decisions for more complex jobs.

Don’t leave out the human element - The use of an expert system to create a highly detailed assessment report is a key ingredient in good managerial hiring, but it is not meant to be used as the sole piece of information on which decisions are based. Effective decision making requires relying on the expert judgment of experienced hiring managers and recruiters. Assessment results are meant to help provide these folks with decision making support.

Link assessment results to development - This is a good idea for any type of job however, it is especially important for managerial level jobs. Very few organizations understand the importance of using assessment data collected during the hiring process to help establish immediate developmental plans for new hires. The idea of hiring a managerial level employee who is perfect right out of the box is pretty far fetched. The complexity of these jobs requires the immediate identification of developmental needs and the provision of ongoing coaching and performance planning to help address them.

After all, developing managers offers the organization a great way to identify and groom future leaders.