Download Proposals for papers reporting empirical work should be in the form

An Examination Of The Relationship Between Contextual And
Technical Performance In An Academic Setting
Arthur Poropat
Griffith University
(E-mail: [email protected])
Paper presented at the ESRC Teaching and Learning Research Programme,
First Annual Conference - University of Leicester, November 2000
ABSTRACT
Job Performance is increasingly conceived of as including two components: technical
performance reflecting job requirements and contextual performance which covers nonprescribed activities such as team-work and support.
Within educational settings
performance is still typically tied to assessment measures such as marks which reflect
technical performance. This study therefore examines the relationship between contextual
and technical performance within an educational setting. Participants were students enrolled
within an introductory management subject. In all 205 students both agreed to participate
and provided useable data for this study. Two measures of Contextual Performance were
used. The Computerised Adaptive Rating Scales (CARS) and attendance at tutorials.
Technical performance was measured by marks on two assignments. A factor scale labelled
active support was developed on the basis of CARS. Active support and attendance both
predicted performance on the Essay (0.24 and 0.35 respectively; combined R2 = 0.120) and
the Take-Home Exam (0.15 and 0.32 respectively; combined R2 = 0.085). The significant
role of contextual performance demonstrated implies that methods of teaching should
incorporate socially facilitative aspects as well as more traditional measures. University
teachers can actively promote increased contextual performance through teaching methods
which encourage value alignment. Further exploration of the links between educational and
employment related contextual performance may be of both pedagogical and organisational
value.
INTRODUCTION
Performance is conceived of in a variety of ways in research but it is a key variable within
both organisational and educational settings. Despite this, performance as a construct has
received comparatively little theoretical attention when compared with other variables
relevant to these areas, possibly because researchers tend to emphasise independent
variables (eg individual differences, treatment conditions) or because performance tends to
be controlled by people other than the researcher (Campbell, McCloy, Oppler & Sager,
1993).
Performance has often been operationalised by measures over which an individual has
limited control (eg productivity and efficiency confound organisational constraints) and that
therefore confuse the relationship between predictor variables and criteria. However
performance is more usefully defined as actual behaviour that can be scaled and measured
in terms of proficiency rather than outcomes (Campbell, et al, 1993). Ratings of behaviour
correspond well with this definition and in educational settings ratings such as grades and
marks reflect actual performance to the extent that students and trainees are not subjected to
arbitrary or biased assessment.
Recent models of job performance have tended to move from an assumption that
performance is unitary (a behavioural equivalent to the g factor in IQ) to models which
emphasise separate aspects of performance (Borman, Hanson & Hedge, 1997). Campbell
(1990) and Campbell, Gasser and Oswald (1996) present a multi-factor model of job
performance based on a review of the job performance literature and provide confirmatory
research from military settings. The eight components of job performance identified in their
model are:
1.
2.
3.
4.
5.
6.
7.
8.
Job-Specific Task Proficiency
Non-Job-Specific Task Proficiency
Written And Oral Communication Task Proficiency
Demonstration Of Effort
Maintenance Of Personal Discipline
Facilitation Of Peer And Team Performance
Supervision/Leadership
Management/Administration
Campbell, et al (1996) deny that these components represent orthogonal factors of job
performance and also state that not all of these components are necessarily present in every
job nor that they are the last word in defining the performance domain. However they do
suggest that these components account for most of the variation in job performance
assessments as well as presenting evidence which suggests the relative independence of
the various components and their relevance for performance research. One of the claims
made by Campbell et al (1993) is that this model of job performance appears to be equally
applicable to educational, training and laboratory task performance. In each situation people
are required to perform to certain levels in a dynamic, socially demanding environment.
An alternative and simpler approach to identifying the various aspects of performance grew
out of path analytic studies which suggested the presence of several key components.
Borman & Motowidlo (1993) in particular developed a model based on two key types of
performance - technical and contextual - which are somewhat reminiscent of the venerable
distinction between task and maintenance processes in group research. Technical
performance relates to what Borman & Motowidlo (1993) call the 'technical core' of the
organisation or job, the activities directly or indirectly involved with transforming resources
into products for economic exchange. Technical task activities vary considerably from job to
job, are dependent on knowledge, skills and abilities, and are role-prescribed.
Contextual performance on the other hand involves activities directed at maintaining the
inter-personal and psychological environment that needs to exist to allow the technical core
to operate. As such contextual activities are common to most if not all jobs, are dependent
upon motivations and predispositions such as personality, and are rarely role-prescribed.
Examples of contextual performance are activities such as:
"Volunteering to carry out task activities that are not formally part of the job
Persisting with extra enthusiasm or effort when necessary to complete own task
activities successfully
Helping and cooperating with others
Following organizational rules and procedures even when personally inconvenient
Endorsing, supporting and defending organizational objectives" (Borman &
Motowidlo, 1993, p73).
Borman and Motowidlo's model is simpler but broadly comparable with Campbell et al's
(1996). Components 1, 2 3, and possibly component 8 of Campbell et al's (1996) model are
linked to technical performance, while components 4, 5, 6 and possibly 7 appear to be
examples of contextual performance. It should be noted that the proponents of both these
models freely acknowledge the strengths of the alternative models and the limitations of their
own (Campbell, Gasser & Oswald, 1996; Borman & Motowidlo, 1997). Despite their different
origins these should be seen as complementary models rather than competitive ones.
Other writers have also looked at alternatives to traditional understandings of performance
with concepts that are directly comparable with the technical/contextual distinction.
Organisational citizenship behaviour (Smith, Organ & Near, 1983) was identified as the
contributions made by employees to their fellows and the organisation overall which
extended beyond the requirements of formal roles or job descriptions.
As such
organisational citizenship behaviour has been explicitly linked to contextual performance
(Borman & Motowidlo, 1993; Organ & Ryan, 1995).
Contextual performance and similar variables have been growing in importance in
discussions of organisational performance criteria (Borman, Hanson & Hedge, 1997). In part
this is because variables like contextual performance have been shown to have roughly
equal influence with technical performance on overall performance ratings by supervisors
(Motowidlo & Van Scotter, 1994; Borman, White & Dorsey, 1995). Podsakoff & MacKenzie
(1997) review evidence that the impact of contextual variables on technical performance has
economically significant implications, predicting between 15% and 43% of variance on such
measures as sales, production and quality. Contextual performance has also grown in
theoretical importance as a mediating variable between personality and general ratings of
performance (Konovsky & Organ, 1996; Borman, Hanson & Hedge, 1997; Borman &
Motowidlo, 1997; Motowidlo, Borman, & Schmit, 1997).
Discussion of performance within educational settings has not had the same discussion of
types of performance as that reviewed so far for organisational settings. Mostly performance
has been discussed from the perspective of how best to assess it with the nature of
performance itself being left largely assumed or explicitly tied to educational objectives (cf.
Palomba & Bamba, 1999). When performance is discussed by writers in tertiary education
this is typically done by emphasising its active and 'real world' nature and contrasting
performance with the knowledge and/or skills which underlie it (Moss, 1992; Biggs, 1999).
The multi-variate model of performance described above has not as yet been investigated or
applied to educational settings. This study therefore aims to discover the nature of the
relationship between contextual and technical performance within an educational setting.
METHOD
Sample
Participants were students enrolled within an introductory management subject. All
measures used in the study were conducted as part of the teaching or assessment process,
but inclusion within the analysis for this research was voluntary: students were informed that
participation would have no positive or negative influence on their marks and analyses were
conducted after finalisation of marks. In all 205 students both agreed to participate and
provided useable data for this study.
Measures
Contextual Performance. The measurement of contextual performance is a complex and
developing field, with much work still underway attempting to identify the basic dimensionality
of the construct (Motowidlo, Borman, & Schmit, 1997). Two measures of contextual
performance were used in this study. The Computerised Adaptive Rating Scales or CARS
(Borman, 1999) were developed using the framework established by Motowidlo, Borman and
Schmit (1997). This instrument attempts to measure three dimensions of contextual
performance – Personal Support, Organisational Support and Conscientious Initiative.
Additionally attendance at tutorials was recorded and used as a measure of contextual
performance in line with previous research (eg. Hattrup, O’Connell & Wingate, 1998).
Technical Performance. The measures of technical performance used within this study were
the marks for two pieces of assessment completed within the subject. These pieces were an
individual essay, and a take-home exam consisting of three short essays. The assessors
used standardised criteria for assessing individual pieces of assessment and the participants
were provided with these criteria in advance as part of the learning process (Nightingale,
Wiata, Toohey, Ryan, Hughes & Magin, 1996). The criteria used were use of literature from
relevant content area, logical argument and presentation, which correspond broadly with
Campbell, Gasser and Oswald's (1996) first three components of job performance (JobSpecific Task Proficiency, Non-Job-Specific Task Proficiency, Written And Oral
Communication Task Proficiency). An overall rating or mark was given to each assessment
item on the basis of these criteria.
Procedure
All assessments were conducted as part of the learning process for the subject. The CARS
scales were scored by fellow students after completion of a group project. Attendance was
recorded by tutors at each tutorial and ratings of technical performance were conducted as
assessment items were completed.
RESULTS
Contextual Performance
The CARS scales were relatively low on internal reliability as assessed by Cronbach Alpha
(0.54 to 0.77) as well as being low on inter-rater reliability (0.47 to 0.60). Equally concerning
is the fact that the three scales are highly inter-related, with their inter-correlations ranging
from 0.62 to 0.78.
A principal components analysis and varimax rotation of the items in these scales revealed
that the first factorial component accounted for 42% of the variance. On the basis of this a
new scale consisting of eight of the original 12 items was constructed by eliminating all items
with either low loadings on this first component or high loadings on multiple components.
This scale was labelled active support and had both improved internal reliability (Cronbach
Alpha = 0.86) and inter-rater reliability (r = 0.66) when compared with the original scales.
Technical Performance
Inter-rater reliabilities for assessments were computed using double-blind ratings on marks
for both the essay and the take-home exam. The average inter-rater reliability was 0.85.
Statistical Relationships
The correlations between the various measures of contextual and technical performance are
reported in Table 1. All measures of both contextual and technical performance were
correlated in predicted directions.
Essay
Exam
Attendance
Active Support
0.52**
0.35**
0.24**
Essay
0.32**
0.15*
Exam
0.18**
Attendance
N = 205. * P < 0.05; ** P < 0.01, : P is one-tailed.
Table 1: Correlations between measures of contextual and technical performance.
Table 2 reports the results for the multiple regression of contextual performance measures
on the essay, while Table 3 reports the multiple regression of these measures on the takehome exam. From these analyses it is apparent that both measures of contextual
performance contribute to the statistical prediction of marks on both the essay and the exam,
and that both measures provide a statistically significant independent contribution to the
prediction of the respective criterion variables.
Attendance
Active Support
Beta
.255**
.216**
Partial R
.261**
.224**
Multiple R
R2
.346**
.120**
N = 205. * P < 0.05; ** P < 0.01, : P is one-tailed.
Table 2: Multiple regression of Attendance & Active Support on Essay.
Attendance
Active Support
Beta
.252**
.127*
Partial R
.254**
.131*
Multiple R
R2
.291*
.085*
N = 205. * P < 0.05; ** P < 0.01, : P is one-tailed.
Table 3: Multiple regression of Attendance & Active Support on Exam.
DISCUSSION
This study showed that contextual performance has a significant relationship with technical
performance within an educational setting. Both measures of contextual performance
provide independent contributions to the prediction of the two measures of technical
performance. This relationship is similar to the link observed between contextual and
technical performance within employment settings. Non-prescribed activities are related to
the effective performance of prescribed activities in employment, and appear to be similarly
related in educational settings.
One of the possible explanations for the relationship between contextual performance and
technical performance in this study is that engaging in contextual behaviour would make it
more likely that students would have opportunities to learn. In other words attendance may
increase technical performance by increasing the exposure to the learning environment.
Students who participate more increase their chances of receiving instruction and guidance,
and it would be a great irony if all the effort put into teaching produced no impact upon
student performance.
However the fact that active support correlated with technical performance independently of
attendance suggests that there is more to this relationship than participation in learning
activities. It appears that for some reason people who are more likely to assist others
through actively supporting their efforts are enhancing their own efforts at the same time: a
case of good behaviour providing its own reward. The fact that the personality variable of
conscientiousness has been shown elsewhere to be positively correlated with contextual
performance (Motowidlo, Borman, & Schmit, 1997) suggests that the relationship between
contextual and technical performance may reflect a general willingness to make greater
efforts. There is a need for experimental work to explore the causal nature of this link further.
Apart from research this study has implications for teaching practice. The significant role of
contextual performance demonstrated here implies that methods of teaching and
assessment should focus on socially facilitative aspects of performance as well as more
traditional measures. Not only would this reflect growing trends in organisations (Hattrup et
al, 1998), it also has potential to enhance the success of students within educational settings.
It may be tempting to extend assessment to cover contextual as well as technical
performance along the lines of writers who suggest that students do what gets marked
(Nightingale et al, 1996). However to the extent that marks have a similar impact to other
extrinsic rewards such as money, this may be counter-productive in the long-run for many
students. Although workers who are potentially able to achieve an extrinsic reward from
contextual performance will increase their supportive behaviours, they are likely to drop
these behaviours whenever the reward structure changes (Hogan, Rybicki, Motowidlo &
Borman, 1998). However workers whose values are aligned with organisational values
provide high levels of contextual performance regardless of extrinsic reward (Deckop,
Mangel, & Cirka, 1999). Likewise workers who take pride in their tasks also display higher
levels of contextual performance (Hodson, 1998).
Consequently it is worth considering educational approaches which are better at aligning
student and faculty values such as student-centred learning (Biggs, 1999). This runs
somewhat counter to the growing popularity of competency-based approaches to education
(Hall, 1994; McKenzie, Mitchell, & Oliver, 1995; Smith, 1992). Competency-based-training is
predicated on the assumption that what matters in life generally and work specifically are
clearly defined skills and abilities, and that therefore training and education should likewise
focus on these. Yet contextual performance is by its nature extra-role behaviour (Smith et al,
1983) and consequently difficult to specify and incorporate in clear competency
requirements. Recognition of contextual performance does not negate the value of
competency-based approaches, but does encourage a broader perspective on education
and training.
In conclusion attention to the multi-dimensional nature of performance within educational
settings is justified by this research. Further research on the nature of the relationship is
likely to enhance understanding of the processes which lead to success in learning.
However the findings presented here provide support for a greater emphasis on modelling
and encouraging support and team-work among staff and students. This is likely to lead to
better outcomes for students, a more pleasant environment for staff, and better-educated
graduates for future employers.
REFERENCES
Biggs, J. (1999). Teaching for Quality Learning at University. Buckingham, U. K.: Society for
Research into Higher Education and Open University Press.
Borman, W. (1999). Computerised Adaptive Rating Scales. Personal Communication.
Borman, W. C., & Motowidlo, S. J. (1993). Expanding the Criterion Domain to Include
Elements of Contextual Performance. In N. Schmidt, W. C. Borman, A. Howard, A. Kraut, D.
Ilgen, B. Schneider, & S. Zedeck (Eds.), Personnel Selection in Organizations (pp 71-98).
San Francisco: Jossey-Bass.
Borman, W. C., & Motowidlo, S. J. (1997). Task performance and contextual performance:
The meaning for personnel selection research. Human Performance. 10, 99-109.
Borman, W. C., Hanson, M. A., & Hedge, J. W. (1997). Personnel Selection. Annual
Review of Psychology, 48, 299-337.
Borman, W. C., White, L.A., & Dorsey, S. J. (1993). Effects of ratee task performance and
interpersonal factors on supervisor and peer performance ratings. Journal of Applied
Psychology, 80, 168-177.
Campbell, J. P. (1990). An overview of the army selection and classification project (Project
A). Personnel Psychology, 43, 231-239.
Campbell, J. P., Gasser, M. B., & Oswald, F. L. (1996). The Substantive Nature of Job
Performance Variability. In K. R. Murphy (Ed.), Individual Differences and Behavior in
Organizations (pp 258-299). San Francisco: Jossey-Bass.
Campbell, J. P., McCloy, R. A., Oppler, S. H. & Sager, C. E. (1993). A Theory of
Performance. In N. Schmitt, & W. Borman (Eds.), Employee Selection. New York: JosseyBass.
Deckop, J. R., Mangel, R., & Cirka, C. C. (1999). Getting more than you pay for:
Organizational citizenship behavior and pay-for-performance plans.
Academy of
Management Journal, 42, 420-428.
Hall, W. (1994). Competency-based training and assessment. Leabrook, South Australia:
NCVER.
Hattrup, K., O’Connell, M. S., & Wingate, P. H. (1998). Prediction of multi-dimensional
criteria: Distinguishing task and contextual performance. Human Performance, 11(4), 305319.
Hodson, R. (1998). Pride in task completion and organizational citizenship behaviour:
evidence from the ethnographic literature. Work and Stress, 12, 307-321.
Hogan, J., Rybicki, S.L., Motowidlo, S. J., & Borman, W. C. (1998). Relations between
contextual performance, personality, and occupational advancement. Human Performance,
11, 189-207.
Konovsky, M. A., & Organ, D. W. (1996). Dispositional and contextual deteminants of
organizational citizenship behaviour. Journal of Organizational Behavior, 17, 253-266.
Moss, P. A. (1992). Shifting Conceptions of Validity in Educational Measurement:
Implications for Performance Assessment. Review of Educational Research, 62, 229-258.
McKenzie, P., Mitchell, P., & Oliver, P. (1995). Competence and accountability in education.
Aldershot, U. K: Arena.
Motowidlo, S. J., Borman, W. C., & Schmit, M. J. (1997). A theory of individual differences
in task and contextual performance. Human Performance, 10(2), 71-83.
Nightingale, P., Wiata, I. T., Toohey, S., Ryan, G., Hughes, C., & Magin, D. (1996).
Assessing Learning in Universities. Sydney: University of New South Wales Press.
Organ, D. W., & Ryan, K. (1995). A meta-analytic review of attitudinal and dispositional
predictors of organizational citizenship behavior. Personnel Psychology, 48, 775-802.
Palomba, C. A., & Bamba, T. W. (1999). Assessment Essentials: Planning, Implementing
and Improving Assessment in Higher Education. San Francisco: Jossey-Bass.
Podsakoff, P. M., & MacKenzie, S. B. (1997). Impact of Organizational Citizenship Behavior
on Organizational Performance: A Review and Suggestions for Future Research. Human
Performance, 10, 133-151.
Smith, A. (1992). Training and development in Australia. Sydney: Butterworths.
Smith, C. A., Organ, D. W., & Near, J. P. (1983). Organizational Citizenship Behavior: Its
Nature and Antecedents. Journal of Applied Psychology, 68, 653-663.