Testing and assessment in translation and interpreting studies : a call for dialogue between rese...

Table of contents 5
Introduction. Testing and assessment in translation and interpreting studies 7
References 15
Part I. The development of assessment instruments 17
Using a rubric to assess translation ability 19
Introduction 19
1. Initial considerations 20
1.1 Questions preceding test development 21
1.2 Nature of the test 21
1.3 Validity 22
1.4 Reliability 23
1.5 Test authenticity 26
1.6 Task authenticity 27
2. Defining the test construct 28
3. Review of relevant literature 30
4. An expanded framework for an expanded construct 36
5. Using a rubric 44
6. Levels of performance of a rubric 49
Conclusion 50
Limitations and implications 51
References 51
Moving beyond words in assessing mediated interaction 55
Introduction 55
Interpreter performance-based assessment in healthcare 56
Interpreting as discourse 59
Measuring interactional competence in healthcare interpreters 66
Conclusion 72
References 73
Part II. The development of assessment instruments 77
The perks of norm-referenced translation evaluation 79
1. Introduction 79
2. Research hypothesis 82
3. Method 83
3.1 Participants 83
3.2 Materials and procedure 84
3.3 Data processing 85
4. Results and discussion 86
5. Conclusion: Implications and limitations of the study 93
5.1 Implications 93
5.2 Limitations 94
References 94
Appendix 1. 95
Appendix 2. 97
Appendix 3. 97
Appendix 4. 99
Revisiting Carroll\u2019s scales 101
1. Introduction 101
1.1 Purpose and research questions 102
2. Background 103
2.1 Carroll\u2019s scales 103
2.2 Applying grading scales to interpreting 105
3. Data and method 107
3.1 Adaptation of the scales 107
3.2 Eliciting material 110
3.3 The grading procedure 113
3.4 Measuring significant difference and inter-rater reliability 114
4. Results 115
4.1 Inter-rater reliability 115
4.2 Intelligibility 115
4.3 Informativeness 118
4.4 Spontaneous comments from graders 119
5. Discussion 120
5.1 Limitations 120
5.2 Discussion of the results 121
5.3 Conclusion 123
References 123
Appendix 1. 125
Appendix 2. 126
Meaning-oriented assessment of translations 129
Introduction 129
Background 130
NAATI assessment criteria for translation tests 131
Underlying theories of meaning-oriented assessment criteria 133
Meaning-oriented assessment criteria 141
Applications in a pedagogical context 151
Limitations and further study 157
References 159
Appendix 1 162
Assessing cohesion 165
Introduction 165
Using corpora in translation assessment 166
Methods 170
Discussion 177
Conclusions 185
Note 187
References 187
Assessing software localization 191
Introduction 191
Localization: A process of adaptation 192
What is software and how is it developed? 193
What is software quality? How is it evaluated and measured? 194
Localized software quality: A problem of perspective 195
Localized software quality and quality assessment: The vendor\u2019s (or practitioner\u2019s) perspective 196
Software localization processes 196
Linguistic testing 199
Automatic linguistic testing versus manual linguistic testing 204
Manual linguistic testing: Translation quality in localization 204
Accuracy, equivalence, and consistency in manual linguistic testing: What constitutes an error? 207
Cosmetic testing 216
Functional testing 218
Localization testing from the vendor\u2019s (or practitioner\u2019s) perspective: Implications 219
Localized software quality: The client reviewer\u2019s (end-user\u2019s) perspective 221
Conclusion 224
References 226
Part III. Professional certification 229
The predictive validity of admission tests for conference interpreting courses in Europe 231
Introduction 231
Admissions testing 232
Basic concepts of aptitude testing 234
Development of new aptitude tests 236
Validation of existing admissions tests 239
Admissions tests and final exams at Wilhelm University 240
Discussion: Issues raised 246
Conclusion 247
References 249
Getting it right from the start 253
Standards as critical success factors in assessment 303
Introduction 303
Social interpreting in a context of increasing rationality 304
The certification of social interpreters in Flanders, Belgium 312
The certification exam procedure 313
The professional standard for social interpreters 321
Professional competency profile for graders 327
Conclusion 331
References 332
Appendix 1. 335
Getting it right from the start: Program admission testing of signed language interpreters in Australia 253
Study procedure 267
Study 1: Survey instrument 269
Results and discussion 273
Study 2: Program admission test 279
Results and discussion 289
Conclusion 295
Acknowledgments 296
References 297
Assessing ASL-English interpreters 337
Introduction 337
The Canadian context 338
The Australian context 351
The United States context 352
Comparing testing models in Australia, Canada and the United States 353
The new AVLIC testing model 354
The new four-step testing process 357
Considerations arising from the new testing model 370
Summary and conclusion 374
References 374
Appendix 1 377
Introduction 377
I DISCOURSE STRATEGIES 378
II FORM 378
Appendix 2 379
Introduction 379
I MESSAGE PROCESSING 380
II INTERPRETING SUB-TASKS 380
III MISCUE PATTERNS 381
IV ADDITIONAL OBSERVATIONS 381
References 381
Author index 383
Subject index 387