 |
 |
 |
 |
 |
 |
 |
 |
« Back to the Winter 2006 Explorer newsletter
Results from the 4th Annual Rocket-Hire Online Screening and Assessment Usage Survey
November 2006
By Mark C. Healy & Charles A. Handler
Rocket-Hire
Download a PDF of this article »
Recruiting in the 21st Century represents a unique combination of advertising, email, phone calls, interviews, and completion of various instruments designed to evaluate the qualifications of potential employees. But while much of this has evolved in its administrative complexity, many companies have sought to find out more about applicants than simply where they worked most recently and what college degree they earned. Moreover, change in how applicants present their qualifications, and how organizations evaluate them, is exemplified by the migration of paper-based hiring tools (e.g., personality assessments, skill certifications, and tests of cognitive abilities) to the Internet.
Whereas the fundamental approach and content of many questionnaires and tests has, in most cases, not changed relative to their paper-and-pencil predecessors, the deployment of these tools on a computer has altered both the relationship with applicants and the overall process of acquiring employees. Therefore, it is not particularly an overstatement when commentators and vendors alike suggest that a real revolution in recruitment and hiring is underway.
But where and how are companies implementing this technology? For the last four years, Rocket-Hire has surveyed people professionals who use web-based screening and assessment products to enhance and quantify their hiring processes. The information we have been able to gather from this research has been a valuable part of our efforts to track important trends in the use of screening and assessment tools. Thus, earlier this year, we again asked people professionals to tell us about their use and opinions of typical hiring instruments. This report highlights the major findings of our 4th Annual Survey.
We had three goals for this study, including: 1) Understanding the general rate of usage of web-based screening and assessment, 2) getting information on usage rates for various types of tools; and, 3) determining whether these hiring tools are perceived as effective for the organization. We also explore trends and attempt to understand the limitations perceived by companies seeking to deploy these tools.
Who responded?
This year, 136 recruitment and hiring professionals completed the survey, representing an increase of 51% over last year. This increase in response rate is
significant, in that we feel it is indicative of the fact that screening and assessment is continuing to gain momentum as a best practice amongst those responsible for recruiting, hiring, leadership development, succession planning, training, and placement.
As we have seen in past surveys, respondents were evenly representative of recruiters, recruiting leaders, HR executives, business owners, and hiring managers, and represented a wide variety of organizations and hiring situations. Specifically, 59% were employed by organizations with 500 employees or more, with 34% working for a company with 5,000 or more employees. In contrast, about 20% were involved with recruiting for companies with 50 or fewer employees. In terms of employee acquisition activities, 34% hired more than 500 workers in a given year, but a majority of respondents (66%) made 500 or fewer hires per year.
In terms of interpreting these results, it should be noted that these hirers represent a sophisticated group of users of online technology. They were contacted as regular readers of a website specializing in the dispersal of online recruitment technology (http://www.ere.net), and, in general, are comprised of individuals who have been working on the front lines of this online recruitment revolution. We feel their insights are extremely valuable, however, because they represent current trends in online assessment as well as the specific advancements made in this line of work.
Use of Applicant Tracking Systems
An applicant tracking system (ATS) is designed to replace the vast reams of paper associated with acquiring and tracking candidates in a hiring process. As with previous surveys, a majority of respondents employ an ATS. Specifically, 72% already have one deployed and in use, with another 8% in the process of installing this technology. As can be seen in the table below, companies of all sizes are taking advantage of an ATS, and high rates of adoption indicate that use is fairly mainstream.
Table 1. Use of Applicant Tracking Systems by Company Size
This is an important indicator of overall technical sophistication: After successfully automating the drudgery of sorting and filing away general applicant information, use of more evaluative online tools, specifically screening and assessment, is often the next step in the implementation of a technology-based hiring process. Thus, the higher the adoption rate for ATS, the greater the usage of screening and assessment tools will be.
Use of pre-screening tools
Strictly an administrative process, an ATS alone does not usually include any in-depth evaluation of candidates. However, the following data on use of prescreening and assessment processes does underscore the growing importance of judging job applicants using systematic tools. For purposes of this survey, screening tools are defined as:
Tools that gather information about, or ask candidates to respond to questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.
62% of respondents indicated that they use at least one online screening tool to sort job candidates early in the hiring process. The table below captures the usage rates of each of the most common forms of screening.
Table 2. Usage rates of common prescreening tools
Over the last several years, usage rates for each of these tools are quite consistent. As can be seen above, qualifications screening was the most common form of automated prescreen in use. Driving this trend appears to be the relative simplicity with which basic job (or stated "minimum") qualifications (such as degrees earned, location, and years of experience) may be collected and evaluated. As with ATS’ and resume scanning, the appeal of paper and file folder reduction is very persuasive!
Within these tech-savvy organizations, how ubiquitous is the use of screening technology? As can be seen in Table 3, the extent of the deployment of prescreening technology varied quite a bit as well, with some companies utilizing these tools for all jobs while some use it for one or two alone.
Table 3. Extent of Prescreening Deployment
Use of Assessment tools
In contrast to screening tools, online assessment tools were defined as:
Scientifically based screening tools that look more deeply into a candidate's abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.
The overall usage rate of candidate assessment tools by organizations has grown when compared to the last few years. Although our current sample of people professionals represents a largely different group of individuals compared to last year, the substantial increase in the adoption of most types of tools is difficult to put aside. This trend alone is quite significant as it shows the growing increase in the popularity of these tools.
Table 4. Usage Rates of Common Assessment Tools
In line with previous surveys, specific skills and knowledge certifications, checks of criminal background, and personality inventories are becoming routine. However, this year's results signify a substantial jump in the level of adoption of these instruments. Moreover, the assessment of “fit” or alignment with the culture or style of a job or work environment is a current trend in assessment, underscored by high rates of usage across our sample of companies. Skills testing and certification grew, and remains a pervasive assessment strategy due to the need for certification of technical skills often required by certain IT and computer-related positions. Of the various types of instruments listed in Table 4, these tests are likely the most often deployed on an ‘a la carte’ basis, less often with reference to an overall hiring strategy. Online interviews continue to be less often used, but the relatively low rate of simulations is gradually inching upward. But how extensive are deployments of assessment tools within organizations?
Table 5. Extent of Assessment Deployment
As with prescreening, the variation in the deployment of tests and assessments is wide, with 36% of users indicating that assessment is utilized in the placement of individuals into either all domestic or global jobs. For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.
Table 6. Job Level of Assessment Implementation
Table 6 suggests that this aspect of online assessment has evolved. Specifically, 2006 appears to be the first year that our data show a general pattern of use across all levels of an organization. Nonetheless, executive roles tend to involve more interpersonal methods of selection and placement. However, unlike level in the organization, rates of use for different types of job settings appear to vary considerably. Table 7 displays these broad differences.
Table 7. Rates of Assessment Usage by Job Setting or Function
As we noted in previous surveys, web-based managerial, sales, IT, administrative, and customer service assessments are common, in part, because so many different instruments and recruiting programs have been developed specifically for these sorts of jobs. Moreover, non-online, validated assessments of key skills and competencies required for success in these roles have been used for decades.
Effectiveness of Screening and Assessment Tools
A key goal of this series of studies has been to document the feelings users of online hiring technology have about the effectiveness of these systems. This year, 37% of prescreen users and 63% of assessment users felt their tools added value to their organization. The rest were evenly divided between either not having an opinion, or feeling that web-based hiring tools did not add value.
Understanding how these users have formed their opinions is of even greater interest. Apparently, impressions of effectiveness have been formed through gut reactions and personal feelings, as opposed to actual evaluation, as only a minority of companies in our survey actually evaluated the use of hiring technology. Specifically, only 31% of users of prescreening tools and 30% of assessment users collected metrics to judge the quality of their interventions. Moreover, users of metrics tend to distinguish themselves from the majority. Tables 8 and 9 reveal this contrast:
Table 8. Perceived effectiveness of Prescreening Tools
Table 9. Perceived effectiveness of Assessment
Clearly, collecting metrics helps to answer the question of effectiveness and also appears to lead to impressions of success in the use of web-based hiring systems. A variety of different metrics and measures are in use. Our respondents mentioned: comparisons of assessment scores versus tenure and turnover; validation studies; diversity indicators; standard HR metrics (e.g., time-to-fill, cost-per-hire); and, impressions and opinions of managers and job candidates.
The Future of Online Screening and Assessment
Of those companies not currently using screening or assessment instruments, 62% feel they will implement one or more in the future. For them, Table 10 summarizes the sorts of tools under consideration.
Table 10. Screening and Assessment Tools under Consideration
Consistent with experienced users, qualifications screening, assessment of cultural fit, skill/knowledge assessments, and personality inventories dominate the interest of organizations seeking to implement web-based hiring tools. This rate of growth should actually increase as online hiring technology becomes more standard. But with such an optimistic outlook ahead, what is keeping organizations from adopting online screening and assessment tools?
Obstacles to the Use of Online Screening
Of those seeking to enter the world of online hiring, what is single biggest obstacle to the adoption of prescreening and assessment in their organization?
Table11. Single Biggest Perceived Obstacle to Deployment of Online Tools.
The data in Table 11 are both encouraging and discouraging at the same time. On the one hand, it is very encouraging that the largest percentage of respondents indicated there are no obstacles at this time. This is a first for the authors’ research program, and clearly shows an increasingly positive climate for the use of screening and assessment tools.
On the other hand, it’s the same old song and dance. These data suggest that typical concerns center on the value of spending money and time on advanced assessment, and lack of knowledge regarding the positive implications of valid hiring tools. Thus, in terms of skepticism about benefits and a lack of budget, companies continue to be penny-rich and dollar-poor when it comes to dedicating monetary and human resources to their recruitment and hiring processes. It is quite apparent that many HR executives and recruitment professionals are simply not aware of the benefits of comprehensive screening and assessment. In addition, they probably aren’t familiar with the available safeguards and metrics designed to detect problems when systems are in place. Moreover, this lack of understanding, combined with the dearth of evaluation of most HR and training functions, only clouds real knowledge of what works and what doesn’t when it comes to dedicating resources to new people initiatives.
Survey respondents also tended to mention candidate distortion of qualifications as the biggest threat to the success of their online hiring tools. But inflation of job skills has always been with us. A greater concern is applicant identity: Is the person taking the test really the applicant? This is most likely the most important issue most organizations could have, but it should not limit any one company from deploying web-based hiring technology. Overall, most companies may legitimately cite few or no substantive reasons to limit their use of online recruitment and hiring techniques.
Summary
We offer the following as key findings that help clarify today’s trends in online hiring:
- Most medium-sized and large organizations have adopted or will adopt an ATS, but the use of an ATS does not relate to perceptions of hiring effectiveness.
- Some hiring tools, like qualifications screening, skills testing/certification and personality tests, are becoming standard features of a strategic hiring process. The use of some methods (e.g., online interviewing) is rare.
- Few companies formally assess their prescreening tools or use metrics of any kind as a decision making aid. Those who do tend to be aware of the effectiveness of their hiring systems and perceive greater value in them.
- People professionals report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.
Overall, the results of this 4th version of our survey are the most encouraging yet. They clearly show a growing interest in scientifically-derived hiring tools. Valid, web-based approaches to recruiting, screening, and placing job candidates are fast becoming the norm at leading edge, people-focused organizations. This trend is encouraging and, in these authors’ experience, not particularly surprising. We feel these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools, seeing for themselves how quality hiring tools can improve a workforce and add bench strength.
Unfortunately, a major takeaway for us is that there is still a very widespread failure to adopt the business intelligence mindset required to evaluate the effectiveness of screening and assessment tools. We found a lack of understanding of the value these tools can return as well as budgeting issues the primary obstacles. As long as this deficiency exists, it is not surprising that skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and help ensure an increase in the understanding of how properly implemented screening and assessment tools can impact the bottom line.
* * *