|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Second Annual Rocket-Hire Online Screening and Assessment User Survey Results: Feb/March 2004
Towards the end of 2003 I invited ERE readers to participate in my second annual 20 Questions About Online Screening and Assessment Survey. The goal of this year's survey was to pick up where last year's survey left off by continuing to identify important trends in the usage of online screening and assessment tools and by providing up to date data regarding usage rates for online screening and assessment tools. In order to help provide continuing insight into the development of major trends in the usage of online screening and assessment tools, this year's survey asked many of the same questions as last year's did. However, one major difference between this survey and the one conducted last year is that last year's survey did not provide much in-depth information about the usage of ATSs and screening tools relative to that of assessment tools. In order to help collect this type of information, this year's survey added a few specific questions related to the use of ATSs, separated questions referring to screening tools from those related to assessment tools, and provided questions designed to drill down more deeply into the characteristics of usage for both screening and assessment tools. First of all, a big hand to all of the ERE readers out there who were kind enough to spend some of their valuable time completing my survey. Thanks to you I was able to collect a wealth of information. The survey provided so much good information that it was published by the Electronic Recruiting Exchange (www.erexchange.com) in 2 installments.
For the purposes of the present report, the information from these two installments were combined into one larger document. This document contains all of the results related to the survey; so, without further adieu, on to the results.. Sampling LimitationsBefore I begin discussing my findings, I think it is important for me to bring up the fact that the conclusions reported in this article may have been influenced by several characteristics of the sample. These include:
Despite the possible sampling limitations, the results of this survey offer a lot of meaningful information that have allowed me to identify some interesting trends in the use of online screening technology. Sample CharacteristicsA total of 78 readers completed the survey (as compared to 65 last year). Of these 62 were from the US, 2 were from New Zealand, and 1 was from Australia. The information in Figure 1 summarizes the relative percentages of the job titles held by survey respondents.
The data in Figure 1 indicate that the majority of respondents were either recruiters (36%), HR executives (19%), or Managerial level staffing personnel such as hiring, HR, or, staffing managers (27%). These results are very similar to those obtained last year. As it did last year, this data suggests that respondents represent a highly relevant sample population and should provide a very good source of information on the topics of interest. The information in Figure 2 summarizes the number of persons employed by survey respondents' companies. The data in Figure 2 indicate that 37% of respondents were employed by large companies while the remaining respondents were fairly evenly split between small (24%), medium (23%) and medium/large (15%) size companies. These results were very consistent with those of last year's survey. The information in Figure 3 summarizes the approximate number of hires per year made by respondents' organizations.
The data in Figure 3 indicate a wide range responses in terms of # of hires made per year (from a low of 0 to a high of 16,000). Despite this range, the data indicates that the majority of respondents (61%) made between 0 and 150 hires per year. However, the percentage of respondents indicated making between 500 and 20,000 hires per year was also rather high (30%). Surprisingly, very few respondents (9%) fell into the middle range (151-500 hires per year) in their hiring volume. Additional analyses comparing the relationship between company size and number of hires made per year confirm the expectation that the smallest companies in the sample make fewer hires, while the largest companies make the most hires. However, the hiring patterns of medium sized companies (500-5000 employees) are less clear. Although, as one would expect, the biggest companies tended to make the most hiring decisions and the smallest companies the fewest, results still indicate a significant drop off in hiring amongst mid-sized companies. I feel this can be partially explained by the slow economy during the time the survey was completed, most notably the huge slowdown in hiring that was experienced during 2003. The fact that such a high percentage of respondents (61%) indicated making less than 150 hires per year helps to confirm the fact that overall hiring volumes were low in 2003. The information in Figure 4 summarizes the length of time respondents' companies have used the web to collect information from candidates for the purpose of making staffing and hiring decisions.
The data summarized in Figure 4 indicate that majority of respondents' organizations (56%) have been using the web as part of their hiring process for 1 to 4 years, while another 31% indicate having used the web for over 5 years. In the minority were organizations who either do not use the web for hiring or have done so for less than one year (13%). Additional analyses revealed no relationship between the size of a company and the time they have been using the web for hiring related purposes. Overall, the data indicate that the sample was relatively balanced in terms of respondents' job titles and the size of their companies. Survey respondents represented companies of all sizes and the distribution of respondents amongst various sized organizations was relatively even. The fact that almost half of respondents are recruiters is complimented by the fact that about half of the sample hold managerial or executive-level positions. This suggests that the sample provides a good balance between respondents who are actually using online hiring technology on a daily basis and those who are making decisions about implementing such technology. Sample data also indicate that the majority of respondent's organizations have significant experience using the web for making hiring decisions. Overall, the balance reflected in this sample means that the data represent an excellent cross-section of people who are likely to be using online screening. This suggests that the data summarized here offer excellent insight into high-level trends in the use of online screening and assessment technology. The fact that the sample characteristics are similar to last years' should help to highlight ongoing trends in the usage of these tools. Results: ATS UsageSurvey results indicate that the majority of respondent's companies (64%) currently use an ATS system while another 8% indicate they are in the process of installing an ATS system. Additional analyses examining the usage of ATSs relative to company size revealed the following:
These analyses confirm the notion that larger companies are more likely to have an ATS or be in the process of installing one. The relatively high percentage of usage amongst small and mid-sized companies also provides information that companies of all sizes are using ATS systems. Additional analyses provide the following information regarding the relationship between a company's ATS usage and number of hires they make per year:
Again, these data confirm the idea that companies making more hiring decisions are likely to have an ATS system. They also reinforce the notion that even companies with medium and small hiring volumes (i.e., smaller or medium sized companies) are still likely to be users of ATS systems. Survey data also indicate a very interesting relationship between companies that use ATSs and the percentage of successful hires made by respondents' organizations over the past 2 years. For example amongst those with ATS systems:
These results are very interesting because they clearly indicate that the absence of a relationship between the use of an ATS system and the perceptions of the success of hires made using the system. These results reinforce my strong opinion that, while ATS systems are extremely useful, an ATS system alone is not sufficient for increasing the overall quality of hiring decisions. Overall, these results confirm the fact that the adoption curve for ATSs has reached maturity and that almost all medium to large sized organizations are using some form of ATS system. Another interesting point is the fact that even small organizations are using ATS systems. This widespread usage of ATSs by companies of all sizes suggests that there are ATS products available to suit a wide variety of hiring needs. Of additional interest is the fact that there seems to be little relationship between the usage of an ATS system and the actual quality of hires made using the system. This confirms my opinion that, while ATSs are extremely useful for management of candidate data, they do very little to help organizations predict which candidates will succeed. Results: Current usage of online screening toolsAs with last year's survey, one of the major goals of this survey was to collect data about usage rates and characteristics for online screening and assessment tools. Unlike last year's survey, this year I split screening and assessment into 2 separate sections. I did this in order to help us drill down a bit more deeply into these 2 somewhat distinct types of online staffing tools. This section deals with the usage of online screening tools (online assessment tools will be addressed next month in Part 2 of our results). In the survey we defined these tools as: Tools that gather information about, or ask candidates to respond to questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process. This section of our survey was designed to gather the information needed to answer the following general questions about online screening:
The data summarized in this section provides some interesting answers to these questions. For instance: 59% of respondents indicated that they use some form of automated screening tools. Table 1 summarizes the usage rates for 4 specific types of tools. Section 1, Table 1: Type of Screening Tools Used
Effectiveness of screening toolsThe survey also sought to gather information regarding the perceived effectiveness of screening tools and the usage of metrics to help evaluate their effectiveness. These survey questions also provided some interesting results. For instance:
Additional analyses examining perceptions of screening effectiveness amongst respondents indicating that their organizations DO collect metrics provided some very interesting information. These analyses revealed that:
This information suggests that organizations that take the time to collect metrics have a better understanding of the effectiveness of their screening tools and seem more likely to use tools effectively. This information may seem like a no-brainer but it clearly reinforces the importance of collecting metrics related to the usage of screening tools. Without hard data, it is much harder to know how effective your screening tools are. In my experience it is much more difficult to convince the powers that be to allocate more money for screening tools when you lack the ability to demonstrate the ability of these tools to provide ROI. The information in Table 2 summarizes respondents' opinions regarding the effectiveness of screening tools currently used by their organization. Section 1, Table 2: Perceived Effectiveness of Various Types of Screening Tools
The information in Table 3 suggests that the tools that collect more in-depth information from applicants are more likely to be seen as being effective. I do not think this is a coincidence. Tools that examine applicant characteristics such as personality and biographical history should do a much better job of predicting applicant success then tools that collect only cursory, surface information (i.e., resumes). It will be interesting to see the relationship between perceptions of effectiveness of these screening tools versus that of assessment tools (this information will be provided in Part 2). Finally, respondents were asked to indicate the single biggest problem they have experienced related to the use of screening tools. The most common responses to this question included:
Again, these responses are not surprising to me. I have been hearing these same reasons for several years now. While I agree that screening tools are time consuming and can be hard to set up, I also understand the many benefits of using these tools. As with anything else in life, it is very hard to get something for nothing and screening is no exception. An investment in overcoming the difficulties with setting up screening questions correctly and positioning the value of screening to applicants will pay off handsomely in the end. Conclusions: Part 1I feel that the survey results presented here provide a very good reflection of the major hiring related trends we all lived with over the past few years. These include:
Conclusions: Part 1With the above information as a backdrop, it is now time to dig into the data for part two. Part two will cover the following topics:
Defining Online Assessment ToolsIn order to collect more detailed information about screening and assessment related trends, this year's survey provided separate questions for screening and assessment. In order to help ensure the collection of usable data, we provided operational definitions for both of these terms. In the survey we defined online screening tools as:
While we defined online assessment tools as:
This section deals with trends related to the use of online assessment tools as defined above. The data revealed some very interesting results about the current usage of these tools. Results: Current Usage of Assessment Tools67% of respondents indicated that their organization uses some form of online assessment tools. This represents a slight increase over last year's results however, this is merely an anecdotal observation. There is no way to verify the magnitude of this increase since last year's survey provided a slightly different definitions and utilized a different respondent sample. Only 13% of respondents indicated that they use assessment tools but only in paper and pencil format. When examining the usage of online assessment tools by size of the organization a clear trend emerges. Usage rates somewhat similar for all organizations of 5,000 employees or less. For instance,
However, 44% of organizations with over 5,000 employees indicated using some form of assessment tool when selecting employees. This confirms the fact that larger companies are the biggest consumers of online assessment tools. The sample had approximately even numbers of respondents across all ranges of organizational size, so I feel confident that this trend is not an artifact of the sample (i.e., there were not a disproportionately large number of big companies in the sample). Only 13% of respondents currently using ATS systems and online assessment tools indicated that they have integrated assessment tools into their ATS system. This information indicates that assessment is still a relatively stand alone component of the hiring process. Data on the manner in which assessments are deployed helps to support this conclusion. For instance, The data in Table 1 indicate that assessment tools are still deployed at a local, job specific level rather then globally across many jobs within an organization. Part 2, Table 1: Deployment of Assessment Tools
The data in this table demonstrate the clear fact that most organizations use assessment tools locally for specific jobs rather than as a core component of a strategic staffing program. This is consistent with the fact that these tools are still new to many organizations and are often adopted as part of an effort to fight fires rather then as part of a strategic fire prevention strategy. This indicates huge amount of growth potential for these tools especially given the high percentage of respondents who indicated using assessment tools. However, realizing this growth potential will require using local implementations as case studies that can be used to demonstrate the value of a more broadly based assessment program. Unfortunately, most organizations are not currently following this type of strategy. The survey also collected data related to the specific type of assessment tools that are currently being deployed. The data in Table 2 summarizes this information and provides a summary of the data collected from my 2002 survey to help provide information on increases in usage. Part 2, Table 2: Type of Assessment Tools Currently in Use
These data reveal that, except for skills assessments which seems to have experienced a large increase in usage, the type of assessments being used have remained relatively stable over the last few years. It is important to remember that the samples differed between these two years, a fact that somewhat limits the value of these direct comparisons. The data also revealed that most companies who are using assessment tools used more than one tool. In fact, 67% of those using assessment tools listed above used more than one of these tools. This is consistent with last year's results indicating that very few organizations using assessment tools used only one type of assessment tool. The data do not indicate if these tools are used together as part of the hiring process for one specific job or if they are used separately for different jobs. The data in Table 3 provide an overview of the job levels for which online assessment used. Part 2, Table 3: Assessment Usage by Job Level
These data indicate that online assessment is most commonly used for lower level or less complex jobs. There is a clear trend in the data indicating that as the complexity and thus the value of the job goes up, use of online assessment tools declines. This is consistent with the fact that higher value hiring decisions often include a longer hiring cycle with more in-depth assessment measures. The fact that a large amount of performance at most simpler jobs can be boiled down to a few key factors and the fact that lower level jobs will have the highest applicant volume make these types of jobs a perfect candidate for the use of online assessment tools. This is because these tools can quickly and remotely assess key competencies while also providing the organization with tools needed to manage applicant data. The data in Table 4 summarizes the percentages of responding organizations who are currently using assessment for specific job types. Part 2, Table 4: Online Assessment Usage by Job Type
These data indicate that customer service, sales, call center, and admin jobs are the most commonly use of assessment tools. This is consistent with the information in Table 3 indicating that less complex jobs that have a few key performance factors seem to account for the bulk of online assessment usage. Given the large number of assessment tools available and the fact that assessment tools are useful for all types of jobs, not just lower level ones, there is huge growth potential for the use of assessments to make managerial/professional level hiring decisions. Effectiveness of Assessment ToolsA look at respondents' perceptions of the success of their hiring process provided some very interesting results. Table 5 summarizes respondents' perceptions regarding the effectiveness of their hiring process. This table summarizes these perceptions for both companies that indicated using assessment and those that indicated that they do not use assessments. Part 2, Table 5: Assessment Usage and Perceptions of Hiring Success
These results do not seem to indicate any significant trend demonstrating that there is a direct relationship between the use of assessment tools and perceptions of successful hiring. Before attempting to draw any conclusions from this data it is important to understand that respondents were asked to indicate their perceptions of the effectiveness of their overall hiring process for all jobs rather than their perceptions of success related to the use of specific assessment tools. This fact makes a difference in the interpretations that can be drawn from this data. Also, the data on assessment usage clearly indicate that assessments are most commonly used for specific, local jobs. It is important to note that the survey question of interest here asked about perceptions of effectiveness for the overall hiring process, a fact that limits the generalizability of the data in Table 5. Despite this, I think it is important to share this data because I feel it helps demonstrate the fact that companies are not taking the time to evaluate the effectiveness of their hiring processes. This is especially true when it comes to the evaluation of assessment measures. Survey data regarding respondents' use of metrics to evaluate their assessment process help to reinforce this conclusion. For instance, only 42% of respondents indicated that their companies collect metrics of effectiveness of assessment tools. This is consistent with the data presented in Part 1indicating that only 40% of companies collect metrics to evaluate the effectiveness of their screening programs. It is clear that companies who do not collect metrics are missing out. For instance, survey data indicate that 86% of respondents whose companies use assessment and collect metrics felt that these metrics have been useful in helping to build a business case for the continued or expanded use of assessment tools. These data clearly highlight a trend I discussed in Part 1 of my survey results. That is that companies simply aren't making the effort needed to collect evidence demonstrating the effectiveness of screening and assessment tools and that collecting evidence makes a difference when it comes to understanding the value of online assessment tools. Until companies can demonstrate a clear linkage between the use of these tools and ROI, it will be difficult for them to justify the use of assessment. For those readers interested in gaining a better understanding of what type of metrics companies are using to evaluate the effectiveness of their assessment tools, open ended survey data indicated the following as the most popular type of metrics collected:
Finally, 82% of respondents indicated that they felt the use of screening or assessment has had a positive impact on their organization. In my mind, when viewed along with data indicating the lack of assessment related metrics, this data provides clear evidence of a gap between the perceived value of assessment and the organization's ability to capture this value in concrete terms. In order to close this gap, organizations need to begin closing the gap between the perceived value of assessment and its actual applied value. Results: Future Usage and ObstaclesThis section presents survey data related to some of the obstacles that are preventing organizations from adopting screening and assessment tools and provides a quick look into future usage of online assessment tools. Future UsageSurvey results indicate that 60% of those not currently using screening assessment tools felt they would use these tools in the future. Table 6 provides a summary of the specific types of tools being considered. Part 2, Table 6: Tools Under Consideration
These results are very consistent with the trends in usage over the past few years with qualifications screening, skills assessment, and personality assessment representing some of the most popular tools. The results in Table 7 summarize the manner in which respondents feel they will decide on which specific screening and assessment tools they will use. Part 2, Table 7: Selection Methods
These results seem to indicate that there is no one method that is clearly used most however, it seems that the formal RFP process is a popular option. Obstacles 91% of respondents indicated that they face obstacles to the adoption of screening and assessment within their companies. Table 8 summarizes some specific obstacles and the percentage of respondents indicating the presence of a particular obstacle within their organizations. Part 2, Table 8: Obstacles to the Adoption of Screening and Assessment Tools
These results are consistent with the major trends in the adoption of screening and assessment that I have been discussing over the past few years and are almost identical to the data collected last year. Specifically, organizations are still firmly in the habit of passing over dollars to pick up pennies. Lack of understanding regarding screening and assessment tools and their potential upside in terms of adding ROI via a focus on applicant quality are still the norm. The complexity of these tools and confusion around their correct application is continuing to present a serious obstacle to the adoption of these tools. Summary of Trends Identified in Part TwoThe results of my survey help to develop a profile for the use of online assessment tools. I feel this profile is as follows: The most common user of assessment is a very large organization that uses assessment locally for specific lower level, high volume jobs. These assessment tools are not integrated into the rest of the hiring process, but rather are used in a stand alone fashion. Most consumers of assessment tools are probably using skills or personality assessment tools in a reactionary manner to help them solve a specific problem related to the job performance associated with these jobs. The organization probably does not collect metrics to evaluate the effectiveness of their assessment initiatives. Those organizations that are collecting metrics are much more likely to be able to see the value that these tools can provide. Beyond this, survey data related to the usage of assessment tools provides the following high level information: Assessment usage is slowly rising but it is still not as widespread as that of screening. The fact that assessments are being used mostly for less complex jobs and are deployed locally reflects fact that assessment is being used in a reactionary manner to help put out fires or stop the bleeding rather than as a strategic part of the staffing process. This mindset is not going to help any organizations to build a winning talent acquisition strategy. I feel that companies should be using assessment to help bake quality into their hiring processes instead of gunning for a quick fix. There is a large gap between the perceived value of assessment and company's ability to clearly demonstrate this value. As long as this gap exists, the adoption rate for assessment tools will be relatively slow. Information regarding obstacles related to the adoption of screening and assessment provides a pretty clear indication that the cause of this gap is a lack of understanding regarding how assessment works and how it can create value as well as the failure to establish programs to gather the metrics needed to effectively demonstrate ROI. Overall Trends Identified via Survey DataTaken together, the data discussed in Part 1 and Part 2 of this series reveal some very interesting trends about the current state of online hiring. These trends include: EvolutionCollectively survey results provide a very clear picture of our current evolutionary state when it comes to the use of technology in the hiring process. Specifically, it seems that by now almost everyone is using the web to source candidates (although many companies still have a lot to learn about how to create a good web based branding and sourcing strategy). It also seems clear that ATS usage has moved well past the early adopter stage and is approaching a saturation point. At the present time companies are finding that the value of the ATS is beginning to shift from one of managing candidates to that of helping them focus on determining candidate quality. People are beginning to realize that a good website and an ATS will only take them so far. The ability to evolve the hiring process is going to require measures that help companies impart quality control into their hiring processes. I firmly believe that the next step in the ongoing evolution will be an increased focus on the adoption of quality focused tools such as screening and assessment measures. We currently stand at what I feel is still the early adopter phase when it comes to online screening and assessment tools. I feel that the information gathered via this survey clearly reinforces this position. Quality vs. CostSurvey results clearly indicate that most companies do not yet understand how to build a quality focused hiring process. Organizations do not understand the role of assessment tools in helping them to turn the hiring process into a profit center. Instead, they are still fixated on the up front costs associated with using screening and assessment tools. It is clear that the understanding of profit based human capital decisions still lags far behind that of other types of capital expenditures. Until firms bite the bullet and make an initial investment in quality, this situation is not going to change. Those that are willing to make the investment now will find themselves with a distinct competitive advantage in the near future. EducationAt the present time one of the biggest barriers to a quality focused hiring process is education. The creation of a quality focused hiring process requires the strategic use of scientific tools. Much of the knowledge required to execute a quality strategy is complex and lies outside the core competencies of most HR departments. This can be a difficult obstacle to overcome. In my opinion education and experience are the way to break through barriers related to understanding. The best way to learn about how to use tools that provide a focus on quality is to begin using them. While the learning curve may be steep at first, efforts in this area can help organizations create a knowledge base that can provide a legacy as the organization moves forward with a continued focus on using human capital to increase profits via a more efficient and effective execution of the strategic initiatives that define their existence. |
| Home | Products & Services | Articles/Publications | News & Events | About Us | Buyer’s Guide | Newsletter | Top ^ |
|
Rocket-Hire • Charles Handler • tel. (504) 236-7259 • Media & press inquiries: Donna Lehman / MarketUP • tel. (770) 565-7275 • |
|