RM Online - Which is the real person?

More than 40 percent of Fortune 500 companies and government agencies are now using a relatively new employment interviewing technique called "behavior-based structured interviewing." But in spite of all the research data showing this approach to be significantly more likely to distinguish winners from losers in the candidate pool, many companies continue to use the traditional, off-the-cuff approach to developing and delivering interview questions.

"Our interviewers like the unstructured approach to hiring," one personnel director told me. "It makes their job more interesting. You never know what you'll end up discussing with an applicant."

Traditional interviewing has the weight of history on its side - it's "the way we've always done it." Lots of questionable practices have gone on unquestioned under that banner.

Consider the problems with traditional interviewing. Does it matter who interviews you? Of course it does. The judgments of individual interviewers - and their relative clout in the company - can vary widely. If you're lucky, you will get an interviewer who likes you. If not, you're branded as an also-ran. Does the interview environment affect the outcome? Yes, of course.

Note that in the unstructured interview, questions come "out of the blue," according to the whim of the interviewer. They are not formulated in advance in close coordination with job requirements. At best, they are conjured up by the interviewer in an attempt to get an overall impression of the applicant and see how he/she relates and communicates.

At worst, these questions are time-wasters for both the company and the applicant - just talk that yields little valuable information about the candidate's suitability for the position.

Does the length of an interview matter? Yes. Recent research demonstrates that successful candidates invariably are granted longer interviews than unsuccessful candidates.

Will it all come down to the interviewer's gut feeling, a crude thumbs-up or thumbs-down based on personal impressions and vague criteria such as "the kind of person we're looking for" and someone who will be "a good fit"?

It's a lottery, of sorts. The interviewer, the interview site, the length of the interview, the interview questions, the recording of answers and the evaluation of them are all a matter of luck, not planning.

No wonder that traditional interviewing so often produces the "halo effect," in which an interviewer simply hires the person who seems to be wearing a halo similar to the interviewer's. And, given the freedom to make up questions and to evaluate candidate responses as he or she wishes, no wonder the prejudices of an interviewer play such a major role in hiring. You can easily imagine an interviewer who consistently blocks the hiring path for women, or middle-aged men, or overweight applicants, or non-athletic types, or whatever other bias the interviewer applies in the hiring process.

To all such matters of chance, gut feeling and potential bias, the law related to hiring says "'Stop!" Specifically, the Uniform Guidelines arising out of Title VII and EEOC legislation insist that the interview be designed on the basis of specific job requirements. Both the content and method of the interview must be developed to reveal accurately and fairly which candidates are best qualified to fulfill the job requirements determined by the company. Employers must be able to show that interview questions are directly related to these job requirements. In addition, employers must afford each candidate equal treatment.

In creating interview questions, go to those who know the job best: those who do it themselves or supervise its performance, hands-on experts, and get them to list "critical incidents" - specific problems or challenges presented by the job, along with a description of the behavior that solved the problem or met the challenge. For example, here's an analysis of a rental counter position:

Problem: A customer delayed renting because she had a technical question that the counterperson could not answer on the spot.

Successful behavior: The counterperson validated the customer's question, promised to find the answer promptly and followed through in a timely fashion. The customer proceeded with the rental.

This incident illustrates one piece in the process of constructing job-based questions for use in interviews. If an employer knows what behaviors it takes to succeed in a given job, the employer can then develop interview questions and tests to locate people qualified to perform those behaviors. For some positions, the job analysis may involve only a dozen or so critical incidents. For more complex positions - in outside sales, management, accounting and so forth - there may be many critical incidents.

These critical incident descriptions form the empirical basis for all assertions by the employer regarding what it takes to perform a particular job.

Now let's look at developing the job analysis and description. A job analysis panel, usually composed of a personnel manager and operating managers who are thoroughly familiar with the job, meets to organize the many critical incident descriptions into a succinct description of job behaviors.

The panel also considers information sources of particular importance for the job. Must the applicant, for example, be thoroughly familiar with certain state or federal regulations? An operations manual? Computers?

Must the applicant have abilities in decision-making or information-processing?

Are there special physical requirements - coordination, dexterity, strength? Are there sustained periods of unusual job pressure?

What about social skills? Meeting customers in social contexts? Super-vising others? Motivating co-workers?

Must the applicant be available for overtime and undesirable shifts? Ready to make schedule changes and travel plans on short notice?

With the completed job analysis at hand, the expert panel begins to formulate interview questions. These should be drawn from several different types of questions to provide variety and to measure an interviewee's ability to respond to different forms of questions, including the following:

Definitional questions are usually posed in a "What is a ?" or "What does _______ refer to?'' format that requires applicants to demonstrate knowledge of terms, concepts and tools. What is a dado blade?

Causal questions ask "What happens when ?" or "What is the result of ?" They ask an applicant to specify the consequence of some initial act or procedure. What happens when foundation concrete contains too much lime?

Hypothetical questions take the form of "What would you do

if ?" or "What could happen if .?" They test the candidate's ability to handle future situations based on past learning and experience.

Situational questions, which are related in some ways to hypothetical questions, ask the applicant to put himself or herself into a realistic circumstance described in detail by the interviewer. "Here's the situation What would you do?''

Simulational questions do not describe the circumstance or situation verbally to the applicant; instead, the applicant is physically presented with some aspects of the job situation. Simulation questions take this form: "To achieve purpose X, you are now holding Y. Show what you would do to achieve your purpose." Say the candidate is given a container of plumber's putty and asked to show how to use it in installing a sink.

Relational questions ask the applicant to tell, perhaps by role-playing, how he or she would handle interpersonal situations. What would you say or do if a boss unjustly accused you in front of others of loafing on the job?

Explanatory questions follow the format of "Why would you ?" or "How would you explain ?" Why should you drill a pilot hole?

Once questions and their answers have been selected according to the relative importance of job behaviors, the panel must arrange them for uniform delivery to job applicants. Questions are often arranged to follow the course of a typical workday or the work cycle from the beginning to the end of production. Or, questions can be arranged in order of increasing or decreasing importance. In most cases, applicants will respond to questions more clearly and completely if the questions occur in a meaningful arrangement.

Recent studies at Duke University show that job interviewers are often reluctant to ask hard questions, particularly of applicants for whom they have an initial liking. Often in an effort to help out a congenial applicant, an interviewer may glide over such thorny areas as a spotty job history or poor college performance. This quite human tendency on the part of the interviewer leaves an information gap for the company in its hiring process.

For this reason, all questions developed for a structured interview are asked in order and verbatim for each applicant. In structured interviewing, the interviewer can repeat a question if necessary, but not paraphrase it, coach or prompt the applicant, give hints regarding the intent of or possible answers for the question, or otherwise influence the applicant's response. Nor does the interviewer indicate by verbal or nonverbal signals the relative success or failure of the candidate's answer.

Unlike traditional interview results, structured interviewing offers a planned scoring system for each question used in the interview. At the time questions are developed for the interview, job experts work out a rating scale for a continuum of possible answers, ranging from a great answer to a poor one. Such judgments as "great" and "poor" are tied directly to behavioral objectives in the job analysis, not to the flair or style with which the candidate can schmooze.

A great answer is one that reflects probable success in performing the related job task. A marginal answer is one that reflects probable difficulty in performing it. A poor answer indicates probable failure in performing the task.

But who is to say which answers are great, marginal or poor? That judgment call is up to the same job experts who know the job best and are familiar with employees' relative success in accomplishing it. In most cases, rating scales for interview questions are developed in brainstorming sessions by job experts and personnel managers. Great, marginal and poor responses are specified as "anchors" on a five-point scale. Even though applicants may not hit any of these predicted answers on the nose, their responses can still be placed meaningfully at some point on the continuum marked by these anchors.

Here are two examples of interview questions with accompanying rating scales. Note that these are not multiple-choice questions. Only the interviewers see the suggested responses following the question.

Question 1: When putting a piece of machinery back together after a repair job, why would you clean all the parts first?

5 (good) "Particles of dust and dirt can wear moving parts. Must be clean to inspect for wear and damage."

3 (marginal) "Parts will go together easier. Equipment will run better."

1 (poor) "So it will all be clean. I don't know."

Question 2: Many of the tasks involved in this job require the use of a forklift. Please read this 90-word forklift checkout procedure aloud.

5 (good) Reads fluently, pronouncing words correctly.

3 (marginal) Can read most words, but hesitates often.

1 (poor) Reads with great difficulty.

Interviewees are not expected to say the precise words suggested in the anchor responses. Interviewers simply use these benchmarks to determine the appropriate numeric score of the applicant's actual answer.

Panel members mark their scorecards for each question and also take notes on the content of the applicant's answer. After all applicants have been interviewed, the panel begins the task of comparing, compiling and reconciling their scores for individual applicants. If scores between raters differ by more than one number (say, a 5 and a 3), the panel discusses the applicant's response and seeks to bring the scores closer together. Ratings are then averaged for each question and totalled.

This scoring system makes it relatively easy to compare several candidates on the merit of their responses. It goes a long way toward eliminating distortions caused by interviewer bias, differences in questions and interpersonal factors such as attraction or revulsion.

Companies and individual managers have good reason to protect themselves against potential charges of discrimination in hiring. Plaintiffs in such suits have enjoyed a lot of success in court, and awards can run into the millions of dollars. If a hiring discrimination suit is brought against a company, the court will insist on knowing the following information. These items are musts for documention of the interview process:

1. Document the job analysis process. How is the job defined? How did you determine the specific behaviors necessary for performing the job successfully?

2. Document the process by which questions were created. Who participated in their creation? Why were these people deemed competent to create the questions? How does each question relate to a behavior necessary for performing the job? In what ways do the number, type and arrangement of questions reflect the proportionate importance of particular behaviors necessary to perform the job?

3. Document the system by which applicant responses were scored. What is the system? Who created anchoring responses? How do these anchoring responses relate to real levels of success among those actually performing the job? How were raw scores handled statistically? What weighting, if any, was used in the analysis of scores?

4. Document the process of interviewing candidates. How did applicants find out about the job? What were the criteria for choosing those applicants who were invited for interviews? Where and when were interviews conducted? Who served as interviewers? What are their qualifications, especially in relation to this job? How were questions delivered? How were responses noted? How long did interviews last? How did different interviews compare in time, content of questions and method of evaluation?

5. Document applicant responses and scores. Notes taken by interviewers must be easily interpretable in reconstructing the approximate content of an applicant's response.

6. Document the specific process by which one applicant was chosen over others. What factors were involved? What was the weighting of those factors?

7. Document the validity of the interview process. Does the process in fact predict job performance?

8. Document the presence or absence of adverse impact against EEOC-protected groups participating in the interview process. Are such groups being hired at a rate at least 80 percent of that of non-protected groups? Companies will also want to consider recent court standards regarding the ethnic makeup of the surrounding labor pool in determining adverse impact.

This kind of documentation may seem burdensome to managers and their companies, but it's better than trying to fabricate a legally defensible hiring procedure after the company is already facing a discrimination suit.

Examples of Behavior-based Interview Questions

What an applicant has done is a better indicator of future job success than what the applicant believes, feels, thinks or knows. The following questions are useful in getting people to discuss work realities rather than notions or suppositions.

Tell me how you increased teamwork among those you worked with.

Describe what you liked and disliked about how you were managed in previous positions.

Tell about a time when you made what you consider a mistake or a bad decision on the job. How did you handle the situation?

In your past worklife, what kind of people rubbed you the wrong way? How did you respond?

Tell me about a time when you set specific work goals for yourself. How did things turn out?

Describe a time when you had to criticize or discipline the performance of someone who worked with you or for you. How did you handle the situation? What was the result?

Walk me through the major highlights of your career so far and tell me where you want to go next.

Tell me about a work emergency or crisis of some kind in which you were involved. What was your role? What did you do?

We've all felt stress in our work lives. Tell me about work-related situations that cause stress for you. How do you typically handle it?

In your most recent position, what did you learn? How did you apply this learning?

Tell me about a challenge you faced in a previous work situation. How did you respond?

Every manager has to learn to delegate well. Describe a work situation in which you delegated responsibility successfully. Then tell me about a time when your delegation of responsibility did not work well. How did you handle that situation?

What approaches worked best for you in the past in communicating with your boss? Your co-workers? Your subordinates?

Tell me about a time when you took charge as a leader in a work situation without being formally assigned to that role by your boss.

What experiences have you had working with people of different ethnicities, age or physical ability levels?

Tell me about a time when you felt you went beyond the call of duty in helping a customer.

Although probing for more extensive answers from candidates is not allowed in strictly structured interviewing environments, the practice is still widely used. It can be applied fairly if each candidate is given the advantage of approximately the same degree of probing by interviewers.

Please clarify what you mean How did you feel when that happened? Why do you think you reacted as you did? Did you consider other options at the time? Please give me more details. How do you think others felt about your actions at the time? Looking back on the experience, how do you see things now? What was going through your mind when you took that action? Did the outcome of your action satisfy you?

For both legal and financial reasons, the movement is definitely on toward structured, behavior-based interviewing. Managers and personnel officers have to be able to defend their hiring decisions in court, if necessary. That defense becomes almost impossible if hiring criteria, methods and records have been handled in an unplanned and disorganized way.

At the same time, companies want to improve the quality and efficiency of the hiring procedure. A structured hiring process can be evaluated and improved through repeated testing of its components; an unstructured process changes according to the whims of the interviewers. This creates a moving target that is exceedingly hard to hit by those charged with bringing true performers into the company.