[ Pobierz całość w formacie PDF ]
.On the other hand, students who are at a relatively high correlation between their TOEFL scoreslinguistic disadvantage in the first term of study might find and criterion measures such as those given above.Withthemselves at less of a disadvantage in subsequent terms.a requirement that is neither too high nor too low, thecorrelation between TOEFL scores and subsequent successAssemble information about the validity of TOEFL testwill be only moderate, and the magnitude of the correlationscore requirements at the institution.will depend on a variety of factors.These factors mayIt is important to establish appropriate standards of language include variability in scores on the criterion measures andproficiency.Given the wide variety of educational programs, the reliability of the raters, if raters are used.it is impossible for TOEFL to design a validity study that is Additional methodological issues should be consideredrelevant for all institutions.Rather, the TOEFL program before conducting a standard-setting or validation study.strongly encourages score users to design and carry out Because language proficiency can change within a relativelyinstitutional validity studies to validate a relationship short time, student performance on a criterion variablebetween the established cut-score range and the levels of should be assessed during the first term of enrollment.performance on classroom tasks.11 Validity evidence may Similarly, if TOEFL scores are not obtained immediatelyprovide support for raising or lowering requirements or for prior to admission, gains or losses in language skills mayretaining current requirements should their legitimacy be reduce the relationship between the TOEFL test and thechallenged.criterion.Another issue is the relationship between subjectAn important source of validity evidence for TOEFL matter or level of study and language proficiency.Not allscores is students performance in English or ESL courses.academic subjects require the same level of languageScores can be compared to such criterion measures as teacher proficiency for acceptable performance in the course.Foror adviser ratings of English proficiency, graded written instance, the study of mathematics may require a lowerpresentations, grades in ESL courses, and self-ratings of proficiency in English than the study of philosophy.English proficiency.However, using data obtained solely Similarly, first-year undergraduates who are required to takefrom individuals who have met a high admissions standard courses in a wide range of subjects may need a differentmay be problematic.If the standard is so high that only those profile of language skills from graduate students enrolledwith a high degree of proficiency are admitted, there may in specialized courses of study.be no relationship between TOEFL scores and criterion Section scores should also be taken into consideration inmeasures.Because there will be little important variability in the setting and validating of score requirements.For fieldsEnglish proficiency among the group, variations in success that require a substantial amount of reading, the Readingscore may be particularly important.For fields that requirelittle writing, the Structure/Writing score may be less11important.Assessment of the relationship of section scoresA separate publication, Guidelines for TOEFL InstitutionalValidity Studies, provides information to assist institutions in theto criterion variables can further refine the process ofplanning of local validity studies.To order this publication, fill outinterpreting TOEFL scores.the form on page 43.To support institutions that wish to conductvalidity studies on their cut scores for computer-based TOEFLtest, the TOEFL program plans to fund a limited number of localvalidation studies.For more information, contact the TOEFLprogram office.Additional information about the setting andvalidation of test score standards is available in a manual byLivingston and Zieky, 1982.22To be useful, data about subsequent performance must be advice from colleges and universities that have had morecollected for relatively large numbers of students over an extensive experience with the TOEFL test.The TOEFLextended period of time.Because the paper-based test program recommends that institutions evaluate their TOEFLwill eventually be eliminated, it might be advisable for score requirements regularly to ensure that they areinstitutions to convert paper-based scores to computer-based consistent with the institutions own academic requirementsscores to establish trends.However, to do this, institutions and the language training resources they offer nonnativemust convert the score of every student, not just the means of speakers of English.certain groups.Institutions that have only begun to require For additional information on score user guidelines,TOEFL scores or that have few foreign applicants may not visit the TOEFL Web site at www.toefl.org/find it feasible to conduct their own validity studies.Such useofscr.html#guidelines.institutions might find it helpful to seek information and23Statistical Characteristics of the TestTable 2 shows the most frequently represented nativeThe Computer-Based Testlanguage groups in the computer-based test group and theirThe computer-based test is conceptually different from thecorresponding percentages in the paper-based test group [ Pobierz całość w formacie PDF ]