ASSESSING APPLICANTS

Purpose and Introduction

What's New and Different?

Assessing the Factors of Merit

Creating an Effective Assessment Strategy

Screening and Short Listing Options

Assessment Options:

• Oral Interviews: Situational and Behavioural;

• Simulations: In-Baskets , Case Project Files, Role-Plays, Oral Presentations, Assessment Centres, Written Performance, Graphic Representation, Construction, and Identification Tests; 

• Written Tests: Multiple Choice, Short Answer / Completion, Essay / Long Answer, True / False, Checklist / Matching, and Written Performance Tests;

• Past Work Performance Checks: Reference checks, Work Samples, and Performance Reviews; and 

• Physical Tests: Capacity or Aptitude

Options for Rating Applicants
Documenting Assessment Results
Resources, Information and Tools

Purpose of This Guideline

This guideline is designed to be an overview for line managers and supervisors involved in the assessment of applicants in the staffing process. This includes screening, short listing, assessing and rating applicants. In this document you will find links to further resources and information if you would like more details. 

If you require further explanation, have more specific questions, or wish to discuss a specific issue not addressed in this guideline, please contact your Human Resources Consultant.

Other references: 

• Staffing Policy: 4.1 Recruitment, Selection and Appointment ;

• Public Service Act; and 

• Public Service Act Review of Staffing Decisions Regulations.

Introduction

As a manager, you can't afford to hire the wrong person. Job applicants vary widely in their competencies, knowledge, skills, abilities, interests, work styles, and other characteristics.  These differences inevitably will affect the way people perform or behave in various positions.  As these differences are not necessarily obvious, simple observation of a job applicant is not sufficient to predict their success as an employee. 

Using assessment tools based on the job-related competencies, knowledge, skills and abilities to assess applicants is a key step to ensuring that the right person is hired for the job. The information gathered helps assess the fit or match between people and jobs. The guideline, and the accompanying information in the Manager's Toolkit, Assessing Applicants, attempts to provide you with information that is as comprehensive as possible so that you can make informed choices and are equipped to develop an effective assessment strategy when the time comes. 

Tip: Assessment can be a fairly complex issue. Start considering your options as early as possible, ideally when you are establishing your qualifications.

What's New and Different?

As a manager responsible for staffing under a delegated service model, your role is more important than ever. You design and control the staffing process, make all the substantive decisions and are accountable for the outcomes.

Believe it or not, the "system" is flexible! Presented in this guideline are many different options for you to consider. There are others. Nothing in this guideline prescribes the exact nature or type of staffing approach to be used in every case. There are many different variations and combinations that will be effective. Have you ever envisioned a staffing process that is significantly different than what you've experienced in the past? Have you ever wanted to build organizational capacity by incorporating future requirements into your staffing actions? Now you can. As a key decision maker, you have the opportunity to create the type of staffing approach you've always wanted to. Remember also that we are a single employer. Your staffing decision affects your organization and the public service as a whole.

There are many assessment options for you to consider and the choices are yours to make. While most of the assessment options available are not new, many have been underused. Check out some of the more creative options described in the Manager's Toolkit, Assessment Methods. 

There are many different options available to make screening and short listing applications faster and more effective. Resumes and government application forms are not the only ways applicants can tell you about their qualifications. 

Have you ever considered using narrative or behaviourally-anchored rating scales; or establishing a passing standard that is related to a reference group; or using score averaging rather than consensus as a way to reach a final rating decision? Are you curious about what these rating options are and how they can be used? Check out the information and examples on rating applicants, in the Manager's Toolkit, Options for Rating 

You have been hearing the word "competency" more and more lately. Competencies are not new to the public service, however, in 2002; a Dictionary of Competencies was purchased from the Hay Group and adapted for use in the Province of BC. This dictionary has established a common definition and descriptive format for 43 broad behavioural-based competencies that can be used for staffing, employee development and performance management. Focusing on these broader competencies, ensures that employees are able to move successfully throughout the public service. 

Two features of the Dictionary that will be useful tools for staffing are: descriptions of behaviours grouped in levels that form the basis for behavioral-anchored rating scales; and oral interview questions, interpretive guides (secured site) and details for recording information collected from the interview.  

Take this opportunity to improve on past practice. Create an assessment approach that is flexible, innovative, effective, efficient, timely, and most importantly, principle-based and focused on outcomes. We hope this guideline and the information in the Managers Toolkit will help you to do that.

Assessing The Factors of Merit

The factors of merit to be considered are applicants' education, experience, knowledge, skills, and abilities, past work performance, and continuous service. This is not an exhaustive list. If other factors, such as behavioral-based competencies, are identified that contribute to a merit-based appointment, you may also consider them.

You may consider a factor of merit and determine that no specific assessment of that particular factor is required. The only exceptions to this are the factors of past work performance, which is mandatory and must be specifically assessed, and continuous service where a requirement exists within a collective agreement (e.g. BCGEU, Article 12.3c).

Where a new factor is identified and included in the staffing process, or where a particular factor of merit is considered, and decision is made to not assess it, document the rationale for this decision.

Education and Experience 

It may be determined through job analysis that there is no bona-fide requirement for a specific level of academic education or credential. Even where not specified as a requirement in your qualifications, you may consider an applicant's educational credentials and type of experience as options within the range of equivalent combinations of experience, education and training. 

Tip: Requiring completion of grade 12 for most jobs is irrelevant. More bona fide criteria would be job-related training, coursework or experience. 

When you have ascertained that requiring specific educational credentials and types or years of experience is essential, establish acceptable combinations of education, training and experience in advance wherever possible so that you and the members of your staffing team are clear on what criteria will be used for screening. A statement that equivalencies will be considered is then included when advertising the position. 

Sometimes, there are no alternatives to educational credentials because of legislative or policy requirements for licensing or where a single educational credential is the only practical means by which a given competency can be acquired. In those cases, equivalencies are not used. 

Knowledge, Competencies, Skills and Abilities 

Job analysis often results in the discovery that there is no discrete knowledge required for day one on the job. Knowledge, while essential to performing the duties and tasks successfully, is more often a "learn" on the job or underlies the successful demonstration of a competency, skill or ability. Instead of assessing knowledge separately, a more effective measure of high performance is how an applicant applies their knowledge, i.e. how they behave in job-related situations.

In specifying an educational credential or specific type of experience, it can be assumed that the applicant has acquired a particular body of knowledge, therefore, further assessment of that knowledge would be redundant.

Knowledge would only be assessed as a discrete factor where it is determined that it is critical to performing the job duties on day one of the job and where it is an effective predictor of high performance. Similarly, only those competencies, skills and abilities that are critical, required on day one, and effective predictors are assessed in your staffing action.

Past Work Performance

Research evidence suggests strongly that one of the most valid predictors of future performance is past work performance, therefore, past work performance is an essential element in assessing applicants. Past work performance checks serve several purposes: verifying factual data, such as dates of previous jobs and reason for leaving; confirming perceptions or clarifying concerns that arose throughout the staffing action; and assessing competencies, knowledge, skills and abilities.

The overriding principle throughout the PWP assessment process is that information sought must be job related and consistent with the qualifications that have been established for the job. Use an objective and consistent rating scale to help ensure consistency between raters.

For further information, see the Practice Guideline, Assessing Past Work Performance and the Manager's Toolkit, Past Work Performance and Conducting Reference Checks.

Continuous Service

Where applicable, this factor recognizes an employee's continuous service up to the closing date of the staffing action and gives them credit for each completed year of service. There are different approaches used for calculating continuous service depending on whether the vacancy is included in the BCGEU or not.

For detailed requirements, procedures and samples for calculating continuous service see the Manager's Toolkit, Continuous Service

Creating an Effective Assessment Strategy

Following are some key steps and best practice suggestions to guide you in developing an effective assessment strategy. These suggestions pre-suppose that the job analysis has been completed and the qualifications for staffing have been established. 

Form An Assessment Team

Applicants are commonly screened for eligibility, short listed on qualifications, and assessed by an assessment team. Assessment teams may decide to develop the actual assessment tools as a group, or develop them individually and then review them as a group. The assessment team is the group of people you assemble to help you make the right staffing decision.

The typical assessment team:

• Generally consists of three members, but not always.
• At least one member needs to be very familiar with both the organizational and job requirements for the vacant position.  This may be the manager, direct supervisor or a colleague performing the same duties.
• If there is a specific technical element to the job, consider including someone who can assess the technical merits of the candidates.
• Typically one of the team members is the direct supervisor of the position being filled.  This person is often the team leader.
• It is good human resource management practice to consider the experience in conducting assessments and the gender and diversity balance of the team.
• The manager filling the vacant position is responsible for selecting the team and designating the team leader.

Plan, Prepare, Set Deadlines and Assign Work

Once the team has been formed, meet to assign responsibilities and set deadlines for the staffing action. This will ensure that each team member has the opportunity to inform the others of personal schedules and availability, and is aware of the time constraints for the complete process.  Schedule dates for interviews or other assessment tests well in advance to commit you and your team and as close together as possible to make it easier to compare applicants and to complete the process faster.

Identify who on the team will be responsible for creating the assessment tools and related rating scales. Assign responsibility and deadlines.

Break the process down into smaller steps with target dates, i.e. develop a project plan. The project plan enables the team to plan the timing of intermediate steps, it makes the staffing action easier to understand and complete and ensures that unnecessary delays do not affect the outcome. You don't want to lose valuable applicants to other jobs! This will also provide the team leader with a benchmark against which to assess the progress of the assessment phase and determine if tasks are being completed in time, and if the process will finish by the desired date.

Screening and Short Listing Options

There are a number of options for you to consider to assist applicants in submitting their qualifications and to make screening (for eligibility) and short listing (on qualifications) faster and more efficient. Personal resumes, standard application forms, and web-based applications are the most common options in use today.  You may also wish to consider creating supplementary application forms or questionnaires, or inviting applicants to a workplace orientation visit.

Use instruments that can be easily administered to large numbers of applicants and that do not create unnecessary additional work for you or your assessment team.  Whichever method is used, ensure that your instructions are clear and minimize the work for the applicant wherever possible. Labour-intensive or time-consuming application systems will dissuade many applicants from applying.  Don't risk losing that high performer you've been having difficulty recruiting!

For more detailed description of each of the options, including the advantages, disadvantages, most appropriate uses, etc., see the Manager's Toolkit, Assessment Methods and Screening and Short Listing Instruments.

Screening for Eligibility

Screening is the process used to determine who is an eligible applicant. Only eligible applicants are screened in for further consideration in the staffing action.

For every staffing action, all applicants must meet the basic criteria of age (between 15 and 65 years), and eligibility to work in Canada.

Whether other criteria (as follows) apply will vary depending on the nature and scope of the staffing action: in-service eligibility; relocation within 2 years; working relationships; for geographically restricted staffing actions, applicants location; for temporary appointments, employee status and location; or designated group membership for staffing actions that are restricted or preferred in support of workplace diversity.

For further information on eligibility criteria, see the Manager's Toolkit,  Determining Applicant Eligibility.

Short Listing on Qualifications

Prior to applying costly assessment options, use an initial means of "short listing" potential applicants.  In plain language, "short listing" is a term describing the process of determining which applicants you will see for further assessment.  The goal of short listing is to identify those applicants who have the qualifications necessary to do the job. This should be accomplished in the most efficient, job-related, fair, reasonable and consistent manner possible. See the Practice Guideline, Establishing Qualifications and Competencies for definitions of qualifications.

You are responsible for clearly identifying the qualifications in the advertisement and judiciously considering how applicants meet these requirements.  Your assessment of an applicant's qualifications is to be unbiased, and conducted with care and good judgment. Using more than one member of the assessment team to review the applications enhances the perspective.  

Tip: Use common sense when short listing applicants. For example, if an applicant has acquired a certificate, diploma or degree, it is reasonable to assume they met the academic and/or experience pre-requisites for entry into that particular program as established by the academic institution. Applicants should not be screened out simply because they did not state a lower level credential on their resume (e.g. grade 12 where a college diploma or certificate is stated or an undergraduate degree where a Masters degree is stated).

Tip: Applicants with international credentials may meet the educational or training requirements for the job. The International Credential Evaluation Service (ICES) is a service of the British Columbia Institute of Technology that assesses international credentials against provincial or national standards you may have deemed essential for the job. Check the ICES website for further details.

Initial short listing is generally measured on a pass / fail basis. That is, an applicant is either considered further or not.  If the number of applicants remains high after initial short listing, the use of preferred qualifications (if articulated in the advertisement) or assessment tests might be considered. 

Do not change required qualifications (the criteria for short listing) once short listing has started.

For further information, and sample short listing forms, see the Manager's Toolkit, Short List Applicants

Preferred Qualifications

• Is this qualification a requirement of the position or not? Perhaps it is not necessary and should not be included at all. Or, it is essential and therefore a bona fide requirement that all applicants must have.
• Is there a need to distinguish between a large number of applicants who meet all other qualifications?
• Is the preferred qualification reasonable (i.e. a sufficient pool of applicants will have the qualification)?

It may be that, despite the concerns noted above, the most efficient and effective way to reduce the applicant pool to a reasonable and manageable number, is to short list on preferred qualifications. If preferred qualifications are to be used, ensure they are included in the advertisement and/or applicant information packages.

Assessment Options

There are many assessment methods, tools and techniques available to you to use to assess qualifications. Each has strengths and weaknesses. Some tools are more effective than others in certain situations depending on what is being assessed. For example, oral interviews and simulations are best used to assess competencies, skills and abilities, and written tests to assess knowledge or job-specific technical competencies such as written communication. Vigorous reference checking is critical to verify competencies and how the person will function in the workplace rather than simply relying on how an applicant performs in an interview.

Valid and effective assessment tests or tools are those that are: pre-tested; work-related, reasonable and current; clear, concise and appropriate; comprehensive; reliable, measurable and observable; practical; objective and fair.

• Oral Interviews: Situational and Behavioural;

• Simulations: In-Baskets , Case Project Files, Role-Plays, Oral Presentations, Assessment Centres, Written Performance, Graphic Representation, Construction, and Identification Tests; and

• Written Tests: Multiple Choice, Short Answer / Completion, Essay / Long Answer, True / False, Checklist / Matching, and Written Performance Tests.

• Past Work Performance Checks: Reference checks, Work Samples, and Performance Reviews

One other category of test that is used in specialized situations:

• Physical Tests: Capacity or Aptitude

A fuller description of each of the above assessment options, best practice suggestions and uses, advantages and disadvantages, and associated tools such as checklists, samples and templates may be found in the Manager's Toolkit, Assessment Methods.

In addition to deciding which assessment methods you will use, consider also using more than one assessment format to obtain a more complete picture of critical competencies, combining tests, assessing more than one competency with a particular assessment tool and using technology effectively to distribute and administer tests. You may also wish to consider using computer-assisted tests or integrating the use of audio or video taped scenarios or situations into your staffing process. See the Manager's Toolkit, Assessment Methods and Other Assessment Options for more information.

Developing Assessment Tests or Questions

Be efficient and plan. Develop assessment tools early on in the staffing action. If you haven't prepared them prior to the advertisement appearing, make good use of the time during which applications are being received and before the closing date to finalize assessment tools. Then you can move forward quickly with screening, short listing and assessment actions.

You may find it helpful to create a grid or matrix to help you plan out assessment. Laying it out like this in the planning stage will help you identify whether you have placed appropriate emphasis on each competency, knowledge, skill or ability to be assessed, chosen the most appropriate assessment options for the situation or designed a system that involves using too few or two many different types of options and is too complex. 

Tip: Use common sense and consider the applicant when deciding which type and how many assessment tools to use. There is much that you can learn through thorough and effective reference checking or by obtaining samples of work already completed. Consider this as an alternative to developing an overly complex and time-consuming assessment process. 

The Manager's Toolkit, Assessment Methods, contains detailed information on each of these assessment options, samples and blank templates.  The toolkit also includes tips for developing effective questions and tests and information on establishing the level of difficulty for questions and tests.  

Preparing Tests Instructions for Applicants

Write very clear instructions outlining what the applicant is expected to do. Preparing clear test instructions contributes to consistency, objectivity, and fairness. These, in turn, increase validity and reliability.  

Tip: It is critical to communicate openly with the applicants throughout the assessment process. Tell them what to expect in advance. If you are using an assessment method that relies on an applicant to recall past experiences (e.g. behavioural interviewing) provide them information in advance including not only a description of the type of examples of experience that you are looking for but also the competency being assessed. Tell them how much time is allocated, what resources they may call upon and whether they may use any reference materials during the test. Surprise, tension or confusion don't help you learn about an applicant's competencies.

For further information on creating applicant instructions, see the Manager's Toolkit, Assessment Methods and Tips for Developing Applicant Instructions.

Pre-Test! Pre-Test! Pre-Test!

It is very important to pre-test assessment tools wherever possible. Once your assessment test has been developed, ask full-performing staff to answer the interview questions, do the tasks (e.g. in-basket) or complete the written test. By doing this, any errors or problems in language, content, instructions or timing will be identified. This will also help you determine an appropriate standard for qualifying (i.e. passing). Adjust and correct as necessary.

Administering Assessment Tests  

The administration of tests should promote fairness and the perception of fairness. The best general approach is to put yourself in the position of the test taker and specify what you expect to see. Ensuring consistency in administration and monitoring and test security are two key principles to keep in mind.

Occasionally, you may find that during the assessment process, a particular test or question turns out to be problematic (e.g. it has been misunderstood by the first several applicants or is not getting the results you hoped for). If that happens, do not make a change in midstream. It is not possible to know what the impact is on individual applicants if you make a change in the middle of a process Have all applicants complete the test as it was written and then if it is necessary to delete it, or make any adjustments, do so afterwards.  

Reasonable accommodation, as it relates to test administration, refers to adjustments or provisions made to address requests that fall within one of the protected grounds of the Human Rights Code related to a physical or mental disability . Such requests are considered on a case-by-case basis. Examine your test and the process for administering it, to ensure it will not inadvertently result in an individual being denied an opportunity for employment or advancement for reasons unrelated to their ability to do the job. Examples of accommodation during the staffing action include providing or allowing for the use of specialized equipment, providing additional time, modifying the test or administration procedures, and ensuring physical facilities allow for access.

Note: Should you receive a request for accommodation during the staffing action, consult with your Human Resources Consultant to discuss whether it is possible to accommodate the applicant's disability on the job and, if so, to determine an appropriate accommodation for the test.

See the Manager's Toolkit, Assessment Methods and Tips for Effective Test Administration for more information.

Options for Rating Applicants

Having a pre-developed plan for assessing applicants (e.g. rating scales, marking guides, standards for passing) makes it easier and faster for the assessment team to rate applicants. The rating plan is critical to ensuring that assessment results are properly interpreted and objectively and consistently scored.

Not all rating options include assigning points! Some scales are narrative; others are anchored with sample behaviours. There are a number of options for you to consider when establishing rating scales and marking guides, setting standards for passing, and for arriving at a final decision.

• is valid and job-related;
• contains objective benchmarks against which observations can be rated;
• is weighted appropriately relative to the importance of the qualification being assessed; and

• differentiates between effective and ineffective performers.   

Avoid complex, overly-detailed rating schemes that are hard to administer and explain and more prone to errors. 

Rating Scales

• numerical scales use numbers to rate performance;
• narrative scales use adjectives such as "excellent" or "poor" to rate performance and may be used in their narrative form with a "pass" or "fail" being used to identify applicants who qualify or not for further consideration; or
• behaviorally-anchored rating scales (BARS) describe different behaviours that can be observed for a given work situation and anchors them at points on a scale.

Tip: Use the behavioural levels described for each competency in the BC Competency Dictionary to rate applicants! Use the levels to create BARS in either a narrative or numerical format. There is a Behavioural Event Interview Guide for each competency that may be used as is to assess applicants or may be adapted to the BARS format. Interested? For more, see Establishing Rating Scales and Marking Guides

For factors to consider when creating rating scales and marking guides, more detailed information on each of the above types of rating scales and samples see the Manager's Toolkit, Assessment Methods, Options for Rating Applicants and Establishing Rating Scales and Marking Guides.

Setting a Standard for Passing

The ultimate objective of setting a standard for passing an assessment test is the identification of qualified applicant(s). This is the standard of performance that an applicant is expected to display at various stages in a staffing action in order to be considered further. Standards of passing may also be referred to as "cut-off scores" or "pass marks". Identification of a standard for passing is always a question of judgment.

• A performance-related style of standard is set by making a judgment about the test score or the level of the qualification that corresponds to the level of job performance (e.g. on a written knowledge test, 80% of questions are answered correctly; or on a physical test of strength, lifting a weight of 20 kg)
• Group-related standards for passing are set relative to the performance of the applicants in a reference group (e.g. a score in the top 5% of the applicant group; or the top 10 applicants).
• A combined method. Both methods may be used in combination in order to select the highest-ranking applicants while ensuring that they demonstrate a minimum level of performance on the test (e.g. an applicant qualifies if they are in the top 10% of the group and have met a minimum performance-related score of 70%).

For factors to consider when establishing standards for passing and more detailed information on each option, see the Manager's HR Toolkit, Assessment Methods and Setting Standards for Qualifying ("Passing").

Deciding on the Final Rating

Once a particular assessment has been completed, it is now time to rate each applicant's performance against your pre-determined rating scale and marking guide. This should be done with objectivity, fairness, care, due diligence, decisiveness, and exercising good judgment and respect for applicants.

• Score averaging means that team members' individual rating scores of each applicant are tallied and then averaged for a final score.  This method works best when all members of the team have a thorough understanding of the job duties and requirements.  Where significant differences exist in individual ratings, a full discussion gives members the chance to explore each other's reasons for their ratings. If marking guides or rating scales are adjusted as a result of this discussion, record the reasons. An averaged score dilutes any extremes in individual ratings however, this can lead to central tendency (i.e. applicants are rated in the middle).
• Consensus: An assessment team may decide to reach a single final score for each candidate by discussing their individual observations and scores and reaching a consensus.  While this method promotes full discussion, there is a possibility that one dominant opinion may override the input from other team members.

Common Rating Problems

No rating system is perfect. There are a number of pitfalls to avoid that are common to all assessment methods (particularly when rating applicants' oral interview responses) including: rating applicants too high, too low or in the middle; allowing a rating in one area to either positively or negatively affect a rating in another; comparing applicants to each other rather than to the rating scale; rating applicants differently as time passes or with practice; and allowing inadequate scheduling can adversely impact rating.

Recognizing that these types of errors exist will help you to spot them and to follow practices aimed at reducing or eliminating their adverse effect.

For further information and solutions to each of these problems, and tips for conducting effective rating, see the Manager's Toolkit, Assessment Methods and Avoiding Common Rating Problems.

Documenting Assessment Results

It is important to be able to summarize and record in some way the results of the various assessment activities to facilitate decision-making and to be able to quickly provide feedback to applicants. There is no one method of documenting assessment results.

Screening and short listing decisions are typically documented by creating a list, summary form or sheet which identifies the criteria against which applicants are assessed, annotations as to whether the applicant has met the criteria or not, and a record of the final screening decision. Copies of assessment tests, applicant responses and scores are retained in a confidential file. Typically, a list of applicants' names and test scores is created.  The result of the staffing process is also documented either anecdotally or by virtue of a report or list that summarizes all results and records the final decision.  The form and style of the documentation is at the discretion of the assessment team.

Ask yourself if your documentation includes the evidence necessary to support your decisions.  

The Practice Guideline, Documenting the Staffing Action provides more information in this regard.

Sample summaries (applicant summary lists, rating guides, etc.) and instructions and checklists for creating confidential files may be found in the Manager's Toolkit, Assessment Methods.

Tip: In anticipation of being able to provide meaningful feedback to applicants in a timely fashion, as you are rating, make brief notes about strengths noted or key points missing in an applicant's response. These should be specific and objective comments directly related to the content of the assessment test or question and be comments you are confident about and would be comfortable sharing with an applicant during feedback. Remember that a third party in the event of an inquiry may review written material.

Resources, Information and Tools