Creating an Industry-wide Risk Management Practices Survey

Outdoor Ed — Tue, 02 Jun 2009 01:23:56 GMT

Current Revision posted to Articles by Outdoor Ed on 6/1/2009 9:23:56 PM

Creating an Industry-wide Risk Management Practices Survey

risk management, Karen PaisleyAndy Szolosi, Jim Sibthorp, Andy SzolosiKaren Paisley, risk management

by Karen Paisley, Jim Sibthorp, and Andy Szolosi

Abstract

Managing risks in the field is, perhaps, the single most important dimension of running a successful outdoor expeditionary program — a topic that is certainly worthy of dialogue among professionals. In order to have this dialogue, however, we need to develop some sense of a common language regarding field-based hazards and risk management strategies. The purpose of this paper, then, is to describe the process used to create an industry-wide risk management practices survey.

Introduction

In the last decade, the outdoor adventure industry has seen increased growth. The industry's increased growth in popularity can be viewed both from a programmatic and participant perspective within a variety of arenas including educational institutions, therapeutic organizations, youth programs, and commercial outfitters. The problem is that we, as an industry, can't talk about it and don't know how different organizations manage risks. Why not? Among other reasons, some of which are highlighted by the St. Paul (2001) study, many organizations tend to have their own jargon, a language of sorts that is specific to that organization. For example, what do we mean by "risk?" Do we define it in pragmatic terms, such as "the potential for serious injury or death," or in more philosophical terms, such as "an essential program element employed to facilitate desired outcomes?" Or, perhaps even more complicated, what does the term "instructor judgment" conjure in individual risk managers' minds? Without a common vocabulary, any attempts at real communication are limited. In essence, we find ourselves blindly describing and discussing different parts of the proverbial elephant.

In the fall of 2002, the National Outdoor Leadership School (NOLS) approached the University of Utah, as a comparatively impartial player, to develop a risk management taxonomy for outdoor expeditionary programs (OEPs), which we defined as those strips spending two or more nights in the field. The existing relationship between the University of Utah and NOLS has provided a very unique opportunity to conduct action, or applied, research. Action research can be defined as research that is initiated as a result of an immediate response to a specific problem. The intent, then, is that the results and information gained from the study will be immediately applied to how OEPs manage risk. This taxonomy, ideally, would relate risk management strategies and the types of programs that use them: Do programs offering week-long trips rely more or less heavily on instructor judgment than organizations offering 30-day expeditions? Do programs with more stringent participant screenings rely more on participants to self-manage certain risks? Among other applications, such a taxonomy could be incredibly useful for internal self-evaluations, as well as for identifying baseline industry standards.

To develop this taxonomy, the first step was to design a survey for OEPs that would capture a picture of common field hazards, common strategies used to deal with these hazards, and programmatic factors (e.g., program size) that might influence the strategies used to deal with given hazards. We certainly recognize that individual programs experience risks and utilize management strategies that are, in some way, unique to that particular program. The outdoor expeditionary industry is as multi-faceted as its providers, with programs ranging from overnight trips to 30-day expeditions to semesters overseas. However, while acknowledging our diversity, it seems probable that there are some universal concerns and challenges facing the industry as a whole. If we could identify and contain these, we could have a foundation for that meaningful dialogue sought by many risk managers who may feel they are operating in isolation. A risk management vocabulary could not only facilitate communication between individual organizations, but between organizations and insurance agencies, and between organizations and external stakeholders, as well. Further, a common understanding of terms could simplify the internal processes of writing job descriptions and evaluating employees. Therefore, the purpose of this study was to take the first steps toward the creation of an industry-wide risk management taxonomy, to obtain some baseline data, and to foster dialogue among professionals — namely, you.

We, at the U of U, began the study process by reviewing existing literature on risk management. Much of this literature was from other fields (e.g., industrial safety) and was, obviously, not fully relevant to OEPs. While the existing literature offered a poor basis for a common and meaningful language, it did establish a need to further explore the language and terminology employed in OEP risk management.

To address this need, we conducted interviews with some of the industry's risk managers at the 2002 Wilderness Risk Managers Conference (WRMC) in Reno to gain a fundamental overview of industry jargon. We listened to the tape recordings of these interviews and identified two key points: First, while the interviews were fascinating, we weren't much better off than we were before in terms of our literature review. Second, in explanation of the first and as we expected, different individuals held widely varied, often organization-specific definitions of risk management, and these opinions were, in general, held quite strongly. Overall, the results of the interview process suggested a different, more structured approach to the problem. A Delphi process was utilized in order to achieve the needed structure in developing an accurate and meaningful risk management practices survey.

The Delphi Study

Think of the ancient Oracle at Delphi, where confused souls sought the advice of a wise voice. A Delphi study is simply that: a qualitative research technique that seeks to build consensus among a panel of experts, often through several "rounds" of communication. Experts' opinions are sought on an individual basis in response to a specific issue. In this case, opinions were solicited via e-mail on the "issue" of potential survey items addressing our three inter-related concepts: field-based hazards, strategies for managing those hazards, and program characteristics that might be related to the use of those strategies. The individual communication is used to prevent any sense of "peer pressure" from influencing the experts' opinions.

Once all of the responses are received, the research team compiles the initial results, makes any necessary modifications to the material, and sends the information to the experts again. This process is repeated until consensus is achieved. Ideally, in this case, the Delphi study produces an instrument which offers a reasonable amount of confidence in its ability to produce accurate and meaningful results from the people who ultimately respond to the survey.

The Experts

The first step in the study was to identify the panel of experts. Due to the diverse nature of the outdoor industry, we looked for experts to represent a wide range of OEP risk management views. Ultimately, we selected seven individuals as members of the panel, and they represented the following dimensions of the outdoor industry: university programs, industry consultants, "small" (regional) programs, "large" (national) programs, non-profit organizations, residential camping programs, and legal counsel specializing in outdoor recreation law. After being advised of the potential time commitment and other requirements, these representatives agreed to volunteer their time for three rounds of review.

Results

Again, the purpose of this study was to take the first steps toward the creation of an industry-wide risk management taxonomy and to foster dialogue among professionals. The results suggest that we now have a starting point for that vocabulary. The experts provided their express and implied support of the list of 20 final risk management strategies as comprehensive and understandable (see far left column of table 2). Further, they also supported the final list of 15 hazards (see table 1) and provided a starting point for programmatic factors that may influence how organizations choose to deal with risks. While we certainly realize that no list will ever be completely exhaustive, we do have a mutually understood foundation of "universals."

The presentations made in 2003 at the Wilderness Risk Management Conference (WRMC) in Pennsylvania and ICORE in Utah provided a unique opportunity to acquire additional feedback regarding the list of 20 risk management strategies and the list of 15 field-based hazards. Audience members were asked to comment on the language and terms used in each list. In addition, opinions were solicited to determine which risk management strategies were held as the most important in running an OEP (see table 2). The results of this exercise provided an interesting perspective of risk management between the two populations of conference attendees. Audience members at the WRMC reported the Emergency Action Plan to be the most important risk management strategy (12.20%), while ICORE audience members reported Pre-Course Communication as the most vital risk management strategy (14.20%). Other large differences in perspective are highlighted with asterisks in table 2. While these data were not tested for statistically significant differences, the pattern of responses reinforces our belief that different types of programs (defined by various programmatic characteristics such as number of field days, organizational mission, etc.) may manage field-based hazards in different ways. This, in turn, lends support to the idea that the creation of a taxonomy is possible and would be meaningful to practitioners.

With respect to the dialogue so needed by our profession, based on the results of the Delphi process and feedback received at WRMC and ICORE, we were able to create an internet-based survey instrument designed for large-scale, industry-wide use. Individuals who complete this survey for their organization will be able to obtain real-time comparisons to other, similar organizations (based on responses to questions about program characteristics). These results, hopefully, will spark critical examination — both internally and as a profession.

If you would like to participate in this industry-wide effort and to would like to receive a complimentary copy of the final report in the spring of 2004, please visit our web site at http://www.health.utah.edu/prt/rmstudy.

References

Cline, P. & Curtis, R. (2002, November). Risk management for organizations: Keeping the ship afloat. Paper presented at the International Association for Experiential Education Conference, St. Paul, MN.

Kearns, T. & Maughlin, M. (2002, October) Outdoor Campus Engineers: Standards of care at college and university outdoor recreation programs. Paper presented at the International Conference on Outdoor Recreation and Education, Charleston, SC.

The St. Paul Group (2001). Rocky terrain: A look at the risks in the outdoor adventure industry. St. Paul, MN: Author.

Biographies

Karen Paisley and Jim Sibthorp are Assistant Professors in the Department of Parks, Recreation, and Tourism at the University of Utah. Andy Szolosi is a doctoral student in the same department. Inquiries regarding this paper or study should be submitted to Andy Szolosi.

Table 1: Specific field-based hazards

1. Risk inherent in the program activity itself

2. Driving/Transportation

3. Environmental

4. Participant misbehavior

5. Staff incompetence

6. Medical management

7. Lack of participant supervision

8. Poor instruction

9. Equipment malfunction

10. Misalignment of program activity with program policy

11. Inappropriate staff to participant interaction/contact

12. Public interactions

13. Competition with other institutions

14. Poor nutrition and dehydration

15. Poor hygiene

Table 2: Conference Perspectives on Importance of Risk Management Strategies: Percentages of Votes Cast by Strategy

Risk Management Strategies	WRMC (328 total votes)	ICORE (169 total votes)
Field Staff Screening	7.92%	7.69%
Formal Wilderness Medical Training Requirement of Field Staff	8.84%	11.24%
Mentoring & Apprenticeship	.60%	.91%
Field Staff Training*	8.53%	13.60%
Field Staff (Instructor) Judgment	7.62%	6.50%
Supervision of Field Staff	.60%	0.00%
Participant Screening *	6.70%	.60%
Pre-Course Communication *	6.09%	14.20%
Participant Training	.91%	3.55%
Supervision of Participants *	8.23%	.60%
Ratios of Field Staff to Participants	4.88%	6.51%
Emergency Action Plan	12.20%	8.28%
Policies and Procedures	7.93%	11.24%
Critical Incident Stress Debriefing	0.00%	0.00%
Internal Incident Review Procedure	.60%	.60%
External Incident Review Procedure	0.00%	0.00%
Internal Review of Safety Management Protocols	.60%	0.00%
External Review of Safety Management Protocols	.60%	0.00%
Course Documentation *	5.49%	.60%
Course Debriefings	2.13%	2.37%
Venue Evaluation or Location Scouting	6.40%	7.69%

Copyright © 2000 - 2009 Outdoor Ed LLC. All rights reserved. Outdoor Ed LLC is granted full permission to display the article and all associated material. This material may not be reproduced or extracted in any fashion electronic or otherwise without the express permission of the original author.

Creating an Industry-wide Risk Management Practices Survey

Outdoor Ed — Sat, 03 Jan 2009 01:20:50 GMT

Revision 4 posted to Articles by Outdoor Ed on 1/2/2009 8:20:50 PM

Creating an Industry-wide Risk Management Practices Survey

risk management, Karen Paisley, Jim Sibthorp, Andy Szolosi

by Karen Paisley, Jim Sibthorp, and Andy Szolosi

Abstract

Introduction

The Delphi Study

The Experts

Results

References

The St. Paul Group (2001). Rocky terrain: A look at the risks in the outdoor adventure industry. St. Paul, MN: Author.

Biographies

Table 1: Specific field-based hazards

1. Risk inherent in the program activity itself

2. Driving/Transportation

3. Environmental

4. Participant misbehavior

5. Staff incompetence

6. Medical management

7. Lack of participant supervision

8. Poor instruction

9. Equipment malfunction

10. Misalignment of program activity with program policy

11. Inappropriate staff to participant interaction/contact

12. Public interactions

13. Competition with other institutions

14. Poor nutrition and dehydration

15. Poor hygiene

Table 2: Conference Perspectives on Importance of Risk Management Strategies: Percentages of Votes Cast by Strategy

Risk Management Strategies	WRMC (328 total votes)	ICORE (169 total votes)
Field Staff Screening	7.92%	7.69%
Formal Wilderness Medical Training Requirement of Field Staff	8.84%	11.24%
Mentoring & Apprenticeship	.60%	.91%
Field Staff Training*	8.53%	13.60%
Field Staff (Instructor) Judgment	7.62%	6.50%
Supervision of Field Staff	.60%	0.00%
Participant Screening *	6.70%	.60%
Pre-Course Communication *	6.09%	14.20%
Participant Training	.91%	3.55%
Supervision of Participants *	8.23%	.60%
Ratios of Field Staff to Participants	4.88%	6.51%
Emergency Action Plan	12.20%	8.28%
Policies and Procedures	7.93%	11.24%
Critical Incident Stress Debriefing	0.00%	0.00%
Internal Incident Review Procedure	.60%	.60%
External Incident Review Procedure	0.00%	0.00%
Internal Review of Safety Management Protocols	.60%	0.00%
External Review of Safety Management Protocols	.60%	0.00%
Course Documentation *	5.49%	.60%
Course Debriefings	2.13%	2.37%
Venue Evaluation or Location Scouting	6.40%	7.69%

Creating an Industry-wide Risk Management Practices Survey

Outdoor Ed — Sat, 03 Jan 2009 01:10:32 GMT

Revision 3 posted to Articles by Outdoor Ed on 1/2/2009 8:10:32 PM

Creating an Industry-wide Risk Management Practices Survey

risk management, Karen Paisley, Jim Sibthorp, Andy Szolosi

by Karen Paisley, Jim Sibthorp, and Andy Szolosi

Abstract

Introduction

The Delphi Study

The Experts

Results

References

The St. Paul Group (2001). Rocky terrain: A look at the risks in the outdoor adventure industry. <st1:place w:st="on"><st1:city w:st="on">St. St. Paul, MN: Author.</st1:city></st1:place>

Biographies

Karen Paisley and Jim Sibthorp are Assistant Professors in the Department of Parks, Recreation, and Tourism at the <st1:place w:st="on">University University of Utah. Andy Szolosi is a doctoral student in the same department. Inquiries regarding this paper or study should be submitted to Andy Szolosi.</st1:place>

Table 1: Specific field-based hazards

1. Risk inherent in the program activity itself

2. Driving/Transportation

3. Environmental

4. Participant misbehavior

5. Staff incompetence

6. Medical management

7. Lack of participant supervision

8. Poor instruction

9. Equipment malfunction

10. Misalignment of program activity with program policy

11. Inappropriate staff to participant interaction/contact

12. Public interactions

13. Competition with other institutions

14. Poor nutrition and dehydration

15. Poor hygiene

Table 2: Conference Perspectives on Importance of Risk Management Strategies: Percentages of Votes Cast by Strategy

Risk Management Strategies	WRMC (328 total votes)	ICORE (169 total votes)
Field Staff Screening	7.92%	7.69%
Formal Wilderness Medical Training Requirement of Field Staff	8.84%	11.24%
Mentoring & Apprenticeship	.60%	.91%
Field Staff Training*	8.53%	13.60%
Field Staff (Instructor) Judgment	7.62%	6.50%
Supervision of Field Staff	.60%	0.00%
Participant Screening *	6.70%	.60%
Pre-Course Communication *	6.09%	14.20%
Participant Training	.91%	3.55%
Supervision of Participants *	8.23%	.60%
Ratios of Field Staff to Participants	4.88%	6.51%
Emergency Action Plan	12.20%	8.28%
Policies and Procedures	7.93%	11.24%
Critical Incident Stress Debriefing	0.00%	0.00%
Internal Incident Review Procedure	.60%	.60%
External Incident Review Procedure	0.00%	0.00%
Internal Review of Safety Management Protocols	.60%	0.00%
External Review of Safety Management Protocols	.60%	0.00%
Course Documentation *	5.49%	.60%
Course Debriefings	2.13%	2.37%
Venue Evaluation or Location Scouting	6.40%	7.69%

Creating an Industry-wide Risk Management Practices Survey

Outdoor Ed — Sat, 03 Jan 2009 01:09:26 GMT

Revision 2 posted to Articles by Outdoor Ed on 1/2/2009 8:09:26 PM

Creating an Industry-wide Risk Management Practices Survey

risk management, Karen Paisley, Jim Sibthorp, Andy Szolosi

by Karen Paisley, Jim Sibthorp, and Andy Szolosi

Abstract

Introduction

The Delphi Study

The Experts

Results

References

The St. Paul Group (2001). Rocky terrain: A look at the risks in the outdoor adventure industry. <st1:place w:st="on"><st1:city w:st="on">St. Paul, MN: Author.</st1:city></st1:place>

Biographies

Karen Paisley and Jim Sibthorp are Assistant Professors in the Department of Parks, Recreation, and Tourism at the <st1:place w:st="on">University of Utah. Andy Szolosi is a doctoral student in the same department. Inquiries regarding this paper or study should be submitted to Andy Szolosiat andrew.szolosi@health.utah.edu .</st1:place>

Table 1: Specific field-based hazards

1. Risk inherent in the program activity itself

2. Driving/Transportation

3. Environmental

4. Participant misbehavior

5. Staff incompetence

6. Medical management

7. Lack of participant supervision

8. Poor instruction

9. Equipment malfunction

10. Misalignment of program activity with program policy

11. Inappropriate staff to participant interaction/contact

12. Public interactions

13. Competition with other institutions

14. Poor nutrition and dehydration

15. Poor hygiene

Table 2: Conference Perspectives on Importance of Risk Management Strategies: Percentages of Votes Cast by Strategy

Risk Management Strategies	WRMC (328 total votes)	ICORE (169 total votes)
Field Staff Screening	7.92%	7.69%
Formal Wilderness Medical Training Requirement of Field Staff	8.84%	11.24%
Mentoring & Apprenticeship	.60%	.91%
Field Staff Training*	8.53%	13.60%
Field Staff (Instructor) Judgment	7.62%	6.50%
Supervision of Field Staff	.60%	0.00%
Participant Screening *	6.70%	.60%
Pre-Course Communication *	6.09%	14.20%
Participant Training	.91%	3.55%
Supervision of Participants *	8.23%	.60%
Ratios of Field Staff to Participants	4.88%	6.51%
Emergency Action Plan	12.20%	8.28%
Policies and Procedures	7.93%	11.24%
Critical Incident Stress Debriefing	0.00%	0.00%
Internal Incident Review Procedure	.60%	.60%
External Incident Review Procedure	0.00%	0.00%
Internal Review of Safety Management Protocols	.60%	0.00%
External Review of Safety Management Protocols	.60%	0.00%
Course Documentation *	5.49%	.60%
Course Debriefings	2.13%	2.37%
Venue Evaluation or Location Scouting	6.40%	7.69%

Creating an Industry-wide Risk Management Practices Survey

Outdoor Ed — Sat, 03 Jan 2009 01:07:55 GMT

Revision 1 posted to Articles by Outdoor Ed on 1/2/2009 8:07:55 PM

by Karen Paisley, Jim Sibthorp, and Andy Szolosi

Abstract

Introduction

The Delphi Study

The Experts

Results

References

The St. Paul Group (2001). Rocky terrain: A look at the risks in the outdoor adventure industry. <st1:place w:st="on"><st1:city w:st="on">St. Paul, MN: Author.</st1:city></st1:place>

Biographies

Table 1: Specific field-based hazards

1. Risk inherent in the program activity itself

2. Driving/Transportation

3. Environmental

4. Participant misbehavior

5. Staff incompetence

6. Medical management

7. Lack of participant supervision

8. Poor instruction

9. Equipment malfunction

10. Misalignment of program activity with program policy

11. Inappropriate staff to participant interaction/contact

12. Public interactions

13. Competition with other institutions

14. Poor nutrition and dehydration

15. Poor hygiene

Table 2: Conference Perspectives on Importance of Risk Management Strategies: Percentages of Votes Cast by Strategy

Risk Management Strategies	WRMC (328 total votes)	ICORE (169 total votes)
Field Staff Screening	7.92%	7.69%
Formal Wilderness Medical Training Requirement of Field Staff	8.84%	11.24%
Mentoring & Apprenticeship	.60%	.91%
Field Staff Training*	8.53%	13.60%
Field Staff (Instructor) Judgment	7.62%	6.50%
Supervision of Field Staff	.60%	0.00%
Participant Screening *	6.70%	.60%
Pre-Course Communication *	6.09%	14.20%
Participant Training	.91%	3.55%
Supervision of Participants *	8.23%	.60%
Ratios of Field Staff to Participants	4.88%	6.51%
Emergency Action Plan	12.20%	8.28%
Policies and Procedures	7.93%	11.24%
Critical Incident Stress Debriefing	0.00%	0.00%
Internal Incident Review Procedure	.60%	.60%
External Incident Review Procedure	0.00%	0.00%
Internal Review of Safety Management Protocols	.60%	0.00%
External Review of Safety Management Protocols	.60%	0.00%
Course Documentation *	5.49%	.60%
Course Debriefings	2.13%	2.37%
Venue Evaluation or Location Scouting	6.40%	7.69%