Outdoor Education R&E Center

Management & Corporate

Tools & Instruments

LEQ

A Program Evaluation Tool for Measuring the Impacts of Corporate Adventure Training

Acknowledgements: Andy Carter, Karim Haddad & Zoë Herbert (Outward Bound Australia) made significant contributions to the development of this measurement tool.

James Neill
Last updated:
01 Feb 2008

Table of Contents

• Executive summary
• Measuring corporate adventure training outcomes
• Guiding principles for design of this tool
• Developing the list of objectives & items
• Corporate adventure training objectives
• Short & customized versions
• The measurement scale
• References

Downloads

Short version (6 factors, 23 items, self-report)

• Factors and items
• Questionnaire

Long version (13 factors, 48 items, self-report)

• Factors and items
• Questionnaire

Administration guidelines

• Staff
• Participants

Executive Summary

This corporate adventure training program evaluation tool is designed to assist professional program evaluation and program development.  The instrument allows pre- to post assessments on a variety of personal, social and work-place qualities which corporate adventure training programs often try to enhance.  The content of the instrument is customizable from a pool of self-report scales. The objectives emerged from consultation with interested programs and organizations and the research literature.  This paper describes the rationale, development, and items for the tool.

Basically, the focus is on measuring three domains: Personal, Social, and Work-place.  The short version (6 scales, 23 items) is designed for 1 to 2 day adventure training programs.  The long version is customizable from a list of 13 suggested factors and is recommended for more involved corporate adventure training programs such as 3 to 5 day residential programs.  Each factor is measured by participants' response to 3 to 4 statements, rated on an 8 point scale. 

The style of the items and measurement scale derives from the Life Effectiveness Questionnaire (Neill, Marsh, & Richards, 1997), which has been widely used in evaluating the outcomes of experiential training programs, but does not specifically address outcomes relevant to corporate adventure programs.

The design of the instrument is for practical use - the instrument and support materials can be downloaded and whilst some expertise is advantageous, most program managers can effectively manage administration of the instrument, obtain results, and use them for program reviews, client feedback, report writing, etc.

Potential users should bear in mind that approximately half the scales have not been previously tested, thus their reliability and validity and yet to be determined.  Interested users may wish to consult with the author to get more update information prior to using.

Measuring corporate adventure training outcomes

Instrumentation is vital for good quality empirical research (Neill, 2003).  Designing valid measurement of personal, social and work-place changes following intervention processes has attracted hype and debate, but there remains a lack of readily available, valid program evaluation tools.  This paper aims to plug one of those gaps - the evaluation of outcomes typically aimed for in Corporate Adventure Training or Outdoor Management Development.  For more background, explanation of these programs,, etc. go to "About Outdoor Adventure Training for Corporate and Management Development". 

There are many possible research and evaluation tools for applying to corporate adventure interventions, but to date, there is no readily available tool with reasonable psychometric design and user-friendliness available for corporate adventure training programs and program evaluators.  Simon Priest used an abbreviated form of the Team Development Index in his well-known research studies and this is my other recommendation of a tool to use, however it has been notoriously difficult to obtain, get permission to use, etc. Some other possible instruments and places to search are listed on the Tools, Instruments, and Questionnaires page.

One of the other options to consider is how to analyze the data once collected.  For the corporate  version of the LEQ, Excel data entry templates and SPSS syntax are available for the technologically savvy (e.g., faculty, grad students) and a purchasable Access database with automatic reporting is available for those who want a slick system of entry and reporting.

No instrument is perfect, particularly when it is applied in a variety of circumstances.  The current instrument is designed to provide a low-cost, corporate-adventure-training-focused, user-friendly instrument for researching and evaluating typical goals of corporate adventure training programs.

Guiding principles for this evaluation tool and system

The evaluation tool derives from the Life Effectiveness Questionnaire program evaluation system.

This new tool aims to provide for a rigorous and reasonably comprehensive evaluation of the extent to which program objectives are achieved in a cost and time efficient manner. 

Most corporate adventure training programs are conducted by leaders and administrators who do not have a background in research, but who are skilled in report writing and knowledge about their own program.  Thus, as long as program operators are supported in selecting an appropriate evaluation methodology, and with good quality data analysis, they can construct good quality reports to clients, funders or evaluators, or for internal program development.   

This instrumentation is also designed to be user-friendly for participants, program instructors, and administrators.

The self-report tool can be administered pre- (Time 1) and post-program (Time 2) to get a measure of the short-term program impact.  It is recommended, however, that baseline and followup assessments are obtained, as well as observer measures (e.g., by leaders, parents, teachers, and/or peers).

This evaluation tool is also designed to be customizable.  Program evaluators too often use full instruments, when only part of the instrument is relevant to program objectives, for fearing of undermining the overall validity of the questionnaire.  With this instrument, the goal is to get the best of both worlds - psychometric validity and flexibility.  Evaluators can select from the pool of scales to develop a set of items for evaluating the effects of a particular program.

Developing the list of objectives and the item pool

The range of objectives was mainly derived from those typically mentioned in the research literature on corporate adventure training, and the kinds of goals requested by clients and offered by organizations.  Suggestions for other objectives to measure are welcome - email James Neill.

In addition, other self-report tools were examined, such as the Life Effectiveness Questionnaire and the Review of Personal Effectiveness, and several scales were clearly relevant to corporate adventure training and thus were included in the list.  Suggestions are welcome for suggestions of other instrumentation and contribution of scales and measurement items, such as used in graduate theses, are welcome.

For approximately half of the objectives, 3 items were identified from pre-existing instruments.  For the other half, new items were constructed (4 items per scale) - these are pilot scales.  Since corporate adventure training programs are often customized to specific company cultures and needs, no single instrument would serve all needs.  Thus, this instrument is customizable in the selection of scales (outcomes) to be measured.  The results are then quite meaningful, because they're relevant and customized to the program goals.  Nevertheless, users should be warned about potential limitations with all kinds of evaluation tools and methods - research can provide very valuable insight but it can also be a complete waste of time, all depending on how it is managed.

Short version and Customized version

Short version of tool – include 3-4 items for each of 6 outcomes – 18-23  items – 1 page, ~10 minutes.  Ideal for administration on 1-2 day adventure training for corporate groups where the goals are generally aimed at enhancing positive relations amongst team members and some self- and team-awareness.

Customized version - includes 3-4 items for each of 13 factors – 39-48 items, ~15 minutes, pre-1^st day-last day-followup.  Ideal for administration on 3-7 day corporate adventure training programs which have general personal and team development goals, but also a specific tailoring to tackle more advanced or a wider variety of goals.

Corporate Adventure Training Objectives

Personal objectives include self-awareness, time management, initiative taking, leadership ability, and decision making.

Social objectives includes awareness of others, communication skills, being able to work cooperatively in a team, and conflict resolution.

Work-place objectives include building trust amongst staff and positive relations at work.

Table 1. Typical corporate adventure training objectives

The list of factors in Table 1 is by no means definitive.  Other factors for which items have been developed and which could be included are:

References

Neill, J. T. (2003). Selecting tools, instruments, & questionnaires for outdoor education research & evaluation.  Outdoor Education Research & Evaluation Center.

Neill, J. T., Marsh, H. W., & Richards, G. E. (1997). Development and psychometrics of the Life Effectiveness Questionnaire. Sydney: University of Western Sydney.

Richards, G. E , Ellis, L. A., Neill, J. T. (2002). The ROPELOC: Review of Personal Effectiveness and Locus of Control: A comprehensive instrument for reviewing life effectiveness. Paper presented at Self-Concept Research: Driving International Research Agendas, 6-8 August, Sydney.