Many recruitment consultants and recruiting managers conduct interviews. But few optimise the selection process. This blog sets out the role of a very special selection instrument – the work sample or skills test. 'Work samples' is one of four tools, which when used together provide the highest predictive validity of all staff selection methods. The other instruments are the cognitive ability or aptitude test, the structured interview and the personality profile.

What we’re trying to do

Selection as part of the recruitment lifecycle aims to predict which candidates will do best. Selection instruments like structured interviews have what’s called predictive validity – a measure on a scale of 0 to 1 (strictly -1 to +1 but we ignore negative correlations) indicating the quality of prediction of that instrument. The higher the predictive validity, the better. The best all round predictor is the cognitive ability test (including aptitude tests). CAT tests have a predictive validity of around 0.5. Correctly built work samples also have a predictive validity of around 0.5. The overall predictive validity can be incrementally improved over 0.5 to around 0.65 by using several additional instruments together in the selection process. The result is a robust and diverse selection process that is the best that occupational psychology as a discipline can offer.

Development of work samples

The key to getting the prediction validity up is the development of a correctly constructed set of competencies that the job holder will be expected to exhibit. In our case study discussed below the client needed a consultant who would excel at the following job-related competencies:

• Develop project methodologies from client documentation and briefs;
• Model complex wireless systems;
• Present complex technical, economic and legal concepts to clients;
• Write complex documents for presentation to governments and stakeholders;
• Understand, analyse and synthesise wireless networks.

In any selection programme, the HR consultant must determine which instrument is going to test which competency. Only work samples could test these five.

Getting practical

In developing a work sample test for project methodologies we determined that the candidate would perform well if they could take a typical client document and turn that into a methodology for a consulting project. Conveniently, an invitation to tender (ITT) was available covering spectrum re-farming in the province of Macao. This document had six requirements set out in phrases in the ITT like “shall provide recommendations for the re-farming of the spectrum”. We asked the candidates to set out how they’d construct a project to do this. We used the process of ‘structured walk-through’, asking the candidates to think about the method and then walk us through it. The structured walkthrough is an important sub-competency in developing project methodologies. By thinking about the work sample beforehand, we also defined what an excellent candidate performance would be like (and what an adequate or poor performance would be like). Each candidate’s performance was then scored for each competency.

The same thinking was used to develop work samples for the requirement to model complex systems. By asking the candidate to select a model with which they were familiar (such as how the queues behave in a Post Office) and characterise this we could test modelling competency. By asking the candidate to develop this into a presentation and then write it up as might be required in a client report we also tested reporting and presentation competencies. The result was a progressive second interview at which the client and HR consultant conducted work sample tests and a structured interview and scored the candidates’ responses.

Work sample tests in recruitment and selection

The complete recruitment and selection process starts with the development of a valid set of competencies that are converted into first a job description and then into a person profile for the recruitment or search consultant to work with to find candidates. Some competencies can be tested for use in structured interviews, cognitive ability or aptitude tests and elements of personality profile inventories. Other competencies can only be tested for using work sample tests or skills tests. Work samples add to the predictive validity of the overall selection method.

This blog has illustrated how TimelessTime consultants work with clients to develop these competencies and then translate them into work sample tests that probe the candidates’ ability to complete key job-related competencies.

If this blog has interested you and you’d like to discuss how we might help you achieve optimum prediction of candidates’ ability to perform, do call us.