Human Factors in Certification
DEDICATION
CONTENTS
SERIES FOREWORD
PREFACE
Acknowledgements
I— INTRODUCTION
1— Optimizing Human Factors Contributions
Examples of Alternative Approaches
Previous New Applications of Human Factors
Categorizations
Human Factors Professional Practices
The Introduction of Human Factors into Certification
Résumé
References
II— PHILOSOPHIES OF HUMAN FACTORS CERTIFICATION
2— The Use of the Systems Approach to Certify Advanced Aviation Technologies
Certification
Of Aviation Technologies
The Systems Approach
What came First, the Product or the Idea?
Working Environment
Money, Money, Money
Toilet Seats and Tool Boxes
Workload
Bottom-Up Approach
Inductive Conclusions
Whole Sum of the Parts
A Bad Product with Good Certification Criteria
A Portrait of the Artist As a Certifier
Conclusion
References
3— The Gains from Certification Are in the Process
Parallels
MANPRINT—Lessons Learned
Standards for Certification
To Certify or Not to Certify?
The Case against Certification
The Case for Certification
Conclusions
References
4— Certifying Human–Machine Systems
Preface
Preamble
Introduction
Certifying Failure
Certifying Stability
Certifying Transition
Why Certification?
Hope for the Future
References
5— Human Factors Requirements in Commercial Nuclear Power Plant Control Rooms
Initial Human Factors Requirements
Requirements for New Design
Concluding Remarks
References
6— A Critical Component for Air Traffic Control Systems
Human Factors in Complex Systems
Certification
Human Factors Roles
References
7— Certification of Tactics and Strategies in Aviation
Typical Operational Concepts
Operational Concept for ATM
Operational Concept for Flight Operations
Planning Theory for Operations
The Players in Operations Management
The Air Transport System
Actors
Relationships Between Actors
The Environment
The Conduct of Operations
Operations in a World Without Planning
Operations in a Goal Oriented World
Fuzziness in the Planning Process
The Feasibility of a Scenario
The Impact of Perturbations
The Impact of the Environment
The Impact of a Lack of Knowledge
Lack of Correlation with Reality
The Need for Planning
Optimization Strategies for Operations Management
Certification Issues
Definitions
The Limitations of Certification
Certification of the Planning Process
Human Factors
Conclusions
General
Planning Theory
OOA
Acknowledgment
References
III— PRACTICAL APPROACHES TO HUMAN FACTORS CERTIFICATION
8— Achieving the Objectives of Certification through Validation: Methodological Issues
A Preliminary Effort to Define the Issues for Human Factors
Existing Guidelines
The Criterion Problem
Identifying the Criteria
Human Performance Models as Sources of Criteria
The Measurement Problem: Identifying the Measurement Strategy
Progressive Acceptance Testing As Human Factors Certification
Implications for the Concept of Certification
Summary Observations
References
9— Quality Assurance and Risk Management: Perspectives on Human Factors Certification of Advanced Av...
Background
The HE Approach to Systems Design
T & E in Systems HE
Identification of Test Parameters
Test Plan
QA Compliance
NATO Defense Research Group Endorsement
Merlin HE
Merlin Specification Rationale
OPAS
Merlin HEPP
Merlin Predictive Analysis
Workload Analysis
Decision Error Analysis
Certification
Government Functions
Certification Authority
Certification Validity
QA
Creative Evaluation
Progressive Acceptance
Certification of Human Behavior
The Government-Furnished Equipment Approach
Cognitive Functions
Aircrew Certification
Skill-, Rule-, and Knowledge-Based Taxonomy
Conclusions
References
10— Certification of Flight Crews for Advanced Flight Control Systems
Variety of Automated Systems in Airplane Operations
Criticality of Automated System Knowledge for Pilot Operators
Industry Threats to Operator Proficiency
New Systems for Automated Cockpits
New Role for Pilots of Advanced Automation
Proposal
Justification
Evaluation for FMS Type Ratings
Summary
References
11— Certify for Success: A Methodology for Human-Centered Certification of Advanced Aviation Systems...
Naturalizing
Stakeholders and Their Concerns
System Evaluation
Evaluation Methodology
Mathematical Analyses
Modeling
Simulation Experiments
Methodology Summary
Certification Issues
Certification Process
Certification Team
Conclusion
Acknowledgments
References
IV— SELECTION AND TRAINING
12— Certification of Training
Academic Accreditation
Professional Licensure/Certification
Training, Training Devices, and Simulation
Trainer Certification and Training Verification
Conclusions
References
13— Presentation of a Swedish Study Program Concerning Recruitment, Selection and Training of Studen...
Background Phase 1
Graduation Rate
Economic Review
Goal Setting
The Research
Job Analysis to Determine the Job Criteria
The Selection Procedure
Methods and Research Procedures
Job Analysis Procedures
Control of Validity
Analysis of the Test Battery Material
Analysis of Relations among Variables
Results
Introduction
Traffic Processing
Coordination
Disturbances and Irregularities
Fluctuating Workload
Personalities and Social Skills
ACC
TMC
TWR
Conclusions
The Construct Validity of the Analysis
Relation between Test Results and Training Outcome
Regression Analyses
Discriminant Analysis
Discussion
The Main Task
The Analysis
The Relation between Test Results and Training Outcome
Conclusions
Future ATS Systems
References
14— Does Human Cognition Allow Human Factors (HF) Certification of Advanced Aircrew Systems?
Cognition and Operation of Advanced Aircraft System
Conceptual Issues
Absent System Functions
Problems in the Appreciation of Unobservable Mental Life
Traditional System Functionality
SCFs
Consolidation of Cognition and Operation
Problems Inherent in HF Specification with Traditional Systems Designs
Traditional Design by DEF STAN 00–25
Design by STANAG 3994
Traditional Approach to HMI
Systems Engineering Models
Specification of System Capability through Its Required Functionality
System Performance
Summary of the Assessment of Functionality
Traditions of Enforced Compromise
Methods Outside the Traditional Compromises
Recapitulation on Design and Methods
Functionality Specification for the RN Merlin Helicopter
FRD of Merlin Specification
Merlin HEPP and Design Requirements
Discussion On Merlin Specification
The Incorporation of Human Tactics and Strategies
Introduction to Tactics and Strategies
Difficulties in Plan Concept and Application
An Avenue toward a New Method of Systems Design
Examination of Similar Systems
Appreciation within New System Design
Certification Issues Revisited
Summary and Conclusion
References
V— PARALLEL VIEWS AND TOPICS
15— Practical Guidelines for Workload Assessment
Questions That Should Be Asked
Factors to Consider in Choosing a Particular Topic
Validity
Reliability
Sensitivity
Diagnosticity
Intrusiveness
Generality, Acceptability, and Applicability
Workload Assessment Techniques
Self-Report Measures
Physiological Measures
Performance
Workload in Applied Settings
Conclusions
References
16— Is There a Role for a "Test Controller" in the Development of New ATC Equipment?
Innovation: A Long Haul
A Comparison with China Lake
Patching It Up
Local Content
Test Controllers
R & D
Preliminary Testing
Full-Scale Deployment
Discussion and Conclusion
Acknowledgments
References
17— Towards a Framework of Human Factors Certification of Complex Man–Machine Systems
Definition of Terms
Problems of Human Factors Certification
What Makes Human Factors Certification So Difficult?
Approach to Human Factors Certification
Model of Human-Centered System Issues
Tools for Human Factors Certification
Conclusions
References
VI— REFLECTIONS OF CERTIFICATION IN AVIATION
18— Successful Management of Programs for Human Factors Certification of Advanced Aviation Technolog...
Definition of HF
Attitudes
Organizational Tasks
Organizational Structure
The Team
Regulation
The Leader and Technological Change
Conclusions
References
19— Evaluation in Context: ATC Automation in the Field
Introduction
CTAS
Field Development and Assessment Process- Overview
Structured Assessments – Methods and Approaches
Issue Identification
Technical Usability
Domain Suitability
User Acceptance
Method Selection
Data Collection
Scenario-Driven Surveys
Shadowing Live Operations and Contextual Interviews
Subjective Ratings
Data Interpretation
Analysis, Inferences and Implications
Identifying Design Deficiencies Surveys
Observations and Interviews
Discoveries and Description of Feature Use
Requirements Definition and Rejuvenation
Conclusions
Acknowledgments
References
20— Integrating Human Factors Knowledge into Certification: The Point of View of the International C...
Background
ICAO\\u0027s Role in Regulation and Implications of Certification
An Overview of the ICAO Annexes
(CNS/ATM) Systems
Discussion of Human Factors and the Certification Process
The ICAO Initiative
The New Human Factors Requirements
Questions for Discussion
Conclusion
References
Appendix A— List of Considerations Which Are Recommended for Use When Determining Humans Factors Asp...
Appendix B— Description of ICAO Annexes
Appendix C— ICAO Annexes Including Human Factors-Related Standards and Recommended Practices (SARPs)...
21— Improving Air Traffic Control by Proving New Tools or by Approving the Joint Human–Machine Syste...
Verification and Validation from a Cognitive Engineering Point of View
Cognitive Engineering
Cognitive Engineering As a Method of Designing New Tools
Cognitive Engineering As a Method of Validating These Tools
A Field Study: The Case of Air Traffic Controllers
Method
What Really Makes the Task Difficult
A Rapid Overview of the Controller\\u0027s Cognitive Model
Making Decisions in a State of Partial Ignorance
Making Decisions in a Fuzzy and Uncertain Environment
Memorization Problems
Cooperative Activity
Conclusion: Validity Domain of All the Mental Processes
Justification of the Tools
Guidelines
Information Filtering
Justification
Design Problems
Description of the Expert System
Discussion
The Extrapolation Function
Justification
Design
Simulation Functions
Memorization Aids
Improving the Management of Cognitive Resources: The Reminder
Justification
Design Problems
Validation Techniques
The First Evaluation Campaign October 1994-April 1995
The Role of the Cognitive Model in Defining the Experimental Protocols
Results
The Reminder
The Cues
The Second Evaluation Campaign October 1997-May 1998
Conclusion
References
22— Certification for Civil Flight Decks and the Human-Computer Interface
Problems with Automation
Civil Aircraft Design and Regulatory Authorities
Role of Human Factors
Contributions of Cognitive Psychology
HCI and the Design Model
HCI Guidelines and Standards
HCI Evaluation Methods: Checklists and Usability Testing
Organizational Implications
Conclusions
References
23— Some Inadequacies of Current Human Factors Certification Process of Advanced Aircraft Technologi...
Current Certification Principles
Certification References
Basic Purpose
Demonstration of Compliance
The Test Pilot Judgment Methodology
Discussion
Some Potential Biases in the Current Human Factors Certification Objectives
The Adequacy of the Compliance Checking Methodology
Conclusion
References
VII— ISSUES IN FUTURE DESIGN AND CERTIFICATION OF COMPLEX SYSTEMS
24— Advanced Automation Glass Cockpit Certification
What Model for Human Error and What Links Between System Design and Accident Risk?
Why Change the Current Procedures of Certification?
The Wariness about the Definition of Human Error
Modeling Relations between Human Error and System Design
New Technologies and Classic Human Factors
Evolution of Theories: Cognitive Ergonomics, Another Way to Consider Ergonomics
Solutions Call for Planning, Anticipation, and Risk Taking
Should Error Analysis Be Restricted to Human–Machine Interaction?
Relation between Human Error and Accident Risk
The Evolving Nature of Pilot Expertise and the Immediate and Definitive Style of Certifying Systems
Practical Outcomes: How to Improve Certification with Human Factors Considerations
When to Bother with Human Factors in the Certification Process?
What to Certify?
What to Measure?
Who Makes the Evaluation?
What Limitations for Human Factors?
What Should Change?
1993–1998: Five Years of Intense Activity in Human Factors for Design and Certification
The Interfaces between Flight Crews and Modern Flight Deck Systems, Report of the FAA HF Team
Closing Notes
References
25— Beware of Agents When Flying Aircraft: Basic Principles behind a Generic Methodology for the Eva...
The Multiple Resources of Automation
Interfacing Functions on Static Worlds: Classical HCI
Cognitive Theories of Interaction
General Principles
Principles for Static Objects
Principles for Functions
Generic Forms of Interfaces
Interfacing Agents on Dynamic Worlds: HCI Goes to the Real World
Cognitive Theories of Interaction
General Principles
Respective Scopes or Competencies and Cooperative Modes.
Principles for Dynamic Worlds or Objects
Principles for Agents
Generic Forms of Interfaces
Conclusion
References
VIII— CONCLUSION
26— Human Factors Certification of Advanced Aviation Technologies: Overview
Relations between Human Factors and Certification
Human Factors Contributions
IsThere a Need for Human Factors Contributions?
Certification As Bias
Certification As Wishful Thinking
Certification As Opinion
Aviation Contexts
Certification As a Minimum
Iterative Certification
Certification As Change
Cognitive Functions
Certification and the Law
Certification and Safety
Applying Human Factors Evidence
Training
Research
References
SUBJECT INDEX
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
Y