JOB AND TEAM DESIGN:MEASUREMENT AND EVALUATION OF JOB AND TEAM DESIGN

MEASUREMENT AND EVALUATION OF JOB AND TEAM DESIGN

Using Questionnaires to Evaluate Job and Team Design One easy and versatile way to measure job design is by using questionnaires or checklists. Job design can then be defined from an operational point of view as ‘‘a set of dimensions of jobs that can be used to describe all jobs, regardless of job content, which influence a wide range of benefits and costs for both the organization and the employee.’’ This method of measuring job-design is also highlighted because it has been used widely in research on job design, especially on the motivational approach.

Several questionnaires exist for measuring the motivational approach to job design (Hackman and Oldham 1980; Sims et al. 1976). Only one questionnaire has been developed that measures all four approaches to job design. A version of that questionnaire is presented in Table 2. It is called the multimethod job-design questionnaire (MJDQ) because of its interdisciplinary emphasis. It yields an evaluation of the quality of a job’s design based on each of the four approaches. Table 2 also includes a rating scale so that it can simply be copied and used without being retyped.

Table 7 presents a scale that can be used to measure team-design characteristics. It can be used to evaluate input and process characteristics of teams. Background information and examples of the use of this measure can be found by Campion et al. (1993 1996).

Questionnaires may be used in several different contexts:

1. When designing new jobs. When a job does not yet exist, the questionnaire is used to evaluate proposed job descriptions, workstations, equipment, and so on. In this role, it often serves as a simple design checklist.

2. When redesigning existing jobs. When a job exists, there is a much greater wealth of infor- mation. Questionnaires can be completed by incumbents, managers, and engineers. Question- naires can be used to measure job design before and after changes are made and to evaluate proposed changes.

3. When diagnosing problem jobs. When problems occur, regardless of the apparent source of the problem, the job-design questionnaire can be used as a diagnostic device to determine whether any problems exist with the design of the jobs.

The administration of questionnaires can be conducted in a variety of ways. Employees can complete them individually at their convenience at their workstation or some other designated area, or they can complete them in a group setting. Group settings allow greater standardization of instruc- tions and provide the opportunity to answer questions and clarify ambiguities. Managers and engi- neers can also complete the questionnaires either individually or in a group session. Engineers and analysts usually find that observation of the job site, examination of the equipment and procedures, and discussions with any incumbents or managers are important methods of gaining information on the job before completing the questionnaires.

Scoring for each job-design approach is usually accomplished by simply averaging the applicable items. Then the scores from different incumbents, managers, or engineers are combined by averaging (Campion 1988; Campion and McClelland 1991). The implicit assumption is that slight differences among respondents are to be expected because of legitimate differences in viewpoint. However, the absolute differences in scores should be examined on an item-by-item basis, and large discrepancies (e.g., more than one point) should be discussed to clarify possible differences in interpretation. It is often useful to discuss each item until a consensus group rating is reached.

The higher the score on a particular job-design scale, the better the quality of the design of the job based on that approach. Likewise, the higher the score on a particular item, the better the design of the job on that dimension. How high a score is needed or necessary cannot be stated in isolation. Some jobs are naturally higher or lower on the various approaches as described previously, and there may be limits to the potential of some jobs. The scores have most value in comparing jobs or alternative job designs rather than evaluating the absolute level of the quality of job design. However, a simple rule of thumb is that if the score for an approach is smaller than three, the job is poorly designed on that approach and should be reconsidered. Even if the average score on an approach is greater than three, examine any individual item scores that are at two or one.

Choosing Sources of Data

1. Incumbents. Incumbents are probably the best source of information if there is an existing job.

In the area of job analysis, incumbents are considered subject matter experts on the content of their jobs. Also, having input into the job design can enhance the likelihood that suggested changes will be accepted. Involvement in such work-related decisions can enhance feelings of participation, thus increasing motivational job design in itself (see item 22 of the motivational scale in Table 2). One should include a large number of incumbents for each job because there can be slight differences in perceptions of the same job due to individual differences. Evidence suggests that one should include all incumbents or at least 10 incumbents for each job (Campion 1988; Campion and McClelland 1991).

2. Managers or supervisors. First-level managers or supervisors may be the next-most knowl- edgeable persons about an existing job. They may also provide information on jobs under development if they have insight into the jobs through involvement in the development process. Differences in perceptions of the same job among managers should be smaller than among incumbents, but slight differences will exist and multiple managers should be used. Evidence suggests that one should include all managers with knowledge of the job or at least three to five managers for each job (Campion 1988; Campion and McClelland 1991).

3. Engineers or analysts. Engineers, if the job has not been developed yet, may be the only source of information because they are the only ones with insight into what the job will eventually look like. But also for existing jobs, an outside perspective by an engineer, analyst, or con- sultant may provide a more objective viewpoint. Again, there can be small differences among engineers, so at least two to five should evaluate each job (Campion and Thayer 1985; Campion and McClelland 1991).

Evaluating Long-Term Effects and Potential Biases

It is important to recognize that some of the effects of job design may not be immediate, others may not be long lasting, and still others may not be obvious. The research has not tended to address these issues directly. In fact, these effects are offered here as potential explanations for some of the incon- sistent findings in the literature. The purpose is to simply put the reader on the alert for the possibility of these effects.

Initially when jobs are designed and employees are new, or right after jobs are redesigned, there may be a short-term period of positive attitudes (often called a ‘‘honeymoon effect’’). As the leg- endary Hawthorne studies indicated, often changes in jobs or increased attention given to workers tends to create novel stimulation and positive attitudes (Mayo 1933). Such transitory elevations in affect should not be mistaken for long-term improvements in satisfaction, as they may wear off over time. In fact, with time the employees may realize that the job is now more important or bigger and should require higher compensation (Campion and Berger 1990). These are only examples to illustrate how dissipating and lagged effects might occur.

Likely candidates for costs that may lag in time include compensation, as noted. Stress and fatigue may also take a while to build up if a job’s mental demands have been increased excessively, and boredom may take a while to set in after a job’s mental demands have been overly decreased. In terms of lagged benefits, productivity and quality are likely to improve with practice and learning on the new job. And some benefits, like reduced turnover, simply take a period of time to estimate accurately.

Benefits that may potentially dissipate with time include satisfaction, especially if the elevated satisfaction is a function of novelty rather than basic changes to the motivating value of the job. Short-term increases in productivity due to heightened effort rather than better design may not last over time. Costs that may dissipate include the training requirements and staffing difficulties. Once the jobs are staffed and everyone is trained, these costs disappear until turnover occurs. So these costs will not go away completely, but they may be less after initial start-up. Dissipating heightened satisfaction but long-term increases in productivity were observed in a recent motivational job- redesign study (Griffin 1989).

Another potential effect that may confuse the proper evaluation of the benefits and costs of job- design is spillover. Laboratory research has shown that job satisfaction can bias employees’ percep- tions of the motivational value of their jobs (O’Reilly et al. 1980). Likewise, the level of morale in the organization can have a spillover effect onto employees’ perceptions of job design in applied settings. If morale is particularly high, it may have an elevating effect on how employees view their jobs; conversely, low morale may have a depressing effect on employees’ views. The term morale refers to the general level of job satisfaction across employees, and it may be a function of many factors, including management, working conditions, and wages. Another factor included that has an especially strong effect on employee reactions to job-design changes is employment security. Obvi- ously, employee enthusiasm for job-design changes will be negative if they view them as potentially decreasing their job security, and every effort should be made to eliminate these fears. The best method of addressing these effects is to be attentive to their potential existence and conduct longi- tudinal evaluations of job design.

Example of an Evaluation of a Job Design

One study is briefly described here as an illustration of a job-redesign project (Campion and Mc- Clelland 1991). It best illustrates the evaluation component of redesign and the value of considering both potential benefits and costs, rather than the implementation and process components of redesign. The setting was a large financial services company. The unit under study processed the paperwork in support of other units that sold the company’s products. Jobs were designed in a mechanistic manner in that separate employees prepared, sorted, coded, computer keyed, and performed other specific functions on the paper flow.

The organization viewed the jobs as perhaps too mechanistically designed. Guided by the moti- vational approach, the project intended to enlarge jobs by combining existing jobs. In so doing, the organization hoped to attain three objectives. First, larger jobs might enhance motivation and satis- faction of employees. Second, larger jobs might increase incumbent feelings of ownership of the work, thus increasing customer service. Third, management recognized that there might be potential costs of enlarged jobs in terms of lost efficiency, and thus every attempt was made to maintain (i.e., avoid decreased) productivity.

As indicated by the third objective, the study considered the consequences of the redesign in terms of all approaches to job design. It was anticipated that the project would increase motivational consequences, decrease mechanistic and perceptual / motor consequences, and have no effect on bi- ological consequences (Table 1).

The evaluation consisted of collecting detailed data on job design and a broad spectrum of po- tential benefits and costs of enlarged jobs. The research strategy involved comparing several varieties of enlarged jobs with each other and with unenlarged jobs. Questionnaire data were collected and focus group meetings were conducted with incumbents, managers, and analysts. The study was repeated at five different geographic sites.

Results indicated that enlarged jobs had the benefits of more employee satisfaction, less boredom, better quality, and better customer service; but they also had the costs of slightly higher training, skill, and compensation requirements. Another finding was that all the potential costs of enlarging jobs were not observed, suggesting that redesign can lead to benefits without incurring every cost in a one-to-one fashion. Finally, the study revealed several improvements to the enlarged jobs.

Example of an Evaluation of a Team Design

This illustration demonstrates the use of multiple sources of data and multiple types of team- effectiveness outcomes. The setting was the same financial services company as in the job-design evaluation above. Questionnaires based on Table 7 were administered to 391 clerical employees and 70 managers on 80 teams (Campion et al. 1993) and to 357 professional workers on 60 teams (Campion et al. 1996) to measure teams’ design characteristics. Thus, two sources of data were used, both team members and managers, to measure the team-design characteristics.

In both studies, effectiveness outcomes included the organization’s satisfaction survey, which had been administered at a different time than the team-design characteristics questionnaire, and man- agers’ judgments of team effectiveness. In the first study, several months of records of teams’ pro- ductivity were also used to measure effectiveness. In the second study, employees’ judgments of team effectiveness, managers’ judgments of team effectiveness measured three months after the first man- agers’ judgements measure, and the average of team members’ most recent performance ratings were also used as outcome measures.

Results indicated that all of the team-design characteristics had positive relationships with at least some of the outcomes. Relationships were strongest for process characteristics. Results also indicated that when teams were well designed according to the team-design approach, they were higher on both employee satisfaction and team-effectiveness ratings.

One final cautionary note regarding evaluation. Different sources (e.g., incumbents, managers) provide different perspectives and should always be included. Collecting data from a single source could lead one to draw different conclusions about a project than if one obtains a broader picture of results by using multiple sources of data.