Validation Procedure

For use with pre-employment selection tests from

Benchmark Testware’s SkillsProfiler series including:

 

Shop Apprentice

Plant Maintenance Tech

Building Maintennace

Automotive Mechanic

Industrial Electrician

Scale Technician

 

 

In compliance with guidelines for pre-employment testing set out in

"Uniform Guidelines on Employee Selection Procedures (1978)"

Document Number 29 US CFR 1607

Section 1607.15 B Criterion-related Validation Studies

 

 

 

1. Why Validate?

Since 1964, pre-employment testing in the United States has been subject to restrictions imposed under Title VII of the Civil Rights Act. Specifically, an employer is prohibited from using pre-employment tests which have an adverse statistical impact on minority groups unless he has taken measures to demonstrate the business relevance of the results of such tests. "Validation" is the term which describes a study of this kind.

In Canada, to a lesser degree, similar avenues of litigation have been opened to job-seekers under various guises including the Charter of Rights and various provincial labor standards. It is therefore prudent for Canadian employers to take reasonable measures to establish the relevance of their own pre-employment tests.

Aside from legal considerations, a properly-conducted validation study can help an employer determine to what extent the information obtained from testing job applicants is predictive of future job performance, and accordingly what weight should be given to the results of such tests.

2. How are tests validated?

The applicable literature of Industrial Psychology ("Standards for educational and psychological testing", American Psychological Association (1985); "Principles for the validation and use of personnel selection procedures",Society for Industrial and Organizational Psychology. (1987)) recognizes three basic methods of test validation: content-, criterion- and construct-based validation. Content validation involves analyzing the content of the tests and demonstrating that it corresponds to the job tasks as set out in a complete Job Analysis. Construct-validation involves showing that the test measures specific personal characteristics which are shown to be necessary for performance of the job. Criterion validation involves showing a statistical correlation between performance on the test and actual job performance as measured by specific criteria.

Benchmark Testware uses a criterion-based validation method. We have chosen this system because it is readily adaptable to the needs of different clients, and provides us with necessary feedback to continually improve the content of our tests. Of the three validation methods outlined above, it requires the least effort in terms of time and expense on the part of the employer.

The method involves three steps.

1. The employer has several members of his existing work-force write the tests.

2. The workers’ supervisor does a written evaluation of each worker’s job performance.

3. Benchmark Testware does a statistical analysis comparing the worker’s test score with the supervisors’s subjective evaluation.

 

3. VALIDATION PROCEDURE.

1. Administering the tests.

The employer should select a sample group from his existing work force. This may include all the members of the relevant job classification if the department is small. A sample group of six or seven is adequate for statistical purposes; however, a larger group is more desirable. Benchmark Testware supplies the test in computer-administered form. There is no time limit on the tests, but most workers will complete them in less than one hour. The order of the test questions is randomized to reduce the likelihood of information being passed between candidates. Results of all tests are stored in a file called "results.exe" which may be found under "c:\skillpro\qfiles\". This file should be submitted by e-mail to Benchmark Testware at "testware@pangea.ca".

2. Supervisor Evaluation.

To comply with guidelines established by the American Psychological Association for criterion-based validation studies, it is necessary that the workers who write the test be rated by their supervisors according to specific criteria of job performance. Benchmark Testware has therefore developed an evaluation form which breaks down the overall job function into relevant factors. A copy of the evaluation form is attached. You may note that candidates are ranked on a 1-2-3 scale according to 8 different criteria including things such as quality of workmanship, ability to read blueprints, etc. It is MOST important that the ratings be completed by the supervisor BEFORE he has had a chance to see the results of the tests, to ensure that his subjective rating is not influenced by seeing a candidate’s score.

One evaluation form is required for each candidate who writes the test. The forms should be completed and signed by the supervisor, and submitted to Benchmark Testware. If necessary, the forms can be provided by e-mail and returned in the same way.

3. Statistical Analysis.

Benchmark Testware will transfer the results of the tests into a spreadsheet and perform a correlation analysis between the test results and the supervisor evaluations. We also recalculate the scores according to various sub-categories we have established to determine which types of questions within the test provide the best prediction of actual job performance. The results of this analysis will be provided to the client.

4. Validation Studies

The results of several validation studies may be viewed by clicking on the links below:

Shop Apprentice: Study #1

Shop Apprentice: Study #2

Personality Profile: Study #1

Links

 

Back to Main Page

Products and Services

Sample Downloads

Seminars

How to contact us