TOEIC[R] tests have been used extensively for placement or proficiency decisions at universities in Japan. However, the amount of diagnostic information they make available to individual candidates is fairly limited. In fact, providing such TOEIC[R] examinees/learners with substantial formative feedback has often been deemed impractical. The purpose of this study is to help bridge the gap between the publicized results of official TOEIC[R] tests and additional findings which can be derived from the analyses of individual item-examinee responses to TOEIC[R] Practice Tests, so as to benefit both TOEIC[R] examinees and English learners. Different versions of TOEIC[R] Practice Tests, as developed by the Institute for International Business Communication (IIBC), were administered to a sample of 136 university students in Japan and a separate sample of 86, employing a common set of anchor items to calibrate all the items from different test forms on the same latent scale. A series of analyses were conducted to delineate salient characteristics of 800 items involved. The results and implications of the study were addressed in terms of diagnostic functions of TOEIC[R] tests for university students in Japan.

View full abstract

This study investigated the effects of three types of questions (i.e., literal questions, bridging inference questions, elaborative inference questions) in reading tests for facilitating EFL learners' reading comprehension. In the experimental study, Japanese high school students read two passages with different difficulty levels and their comprehension was evaluated using both immediate and delayed free-written recall tasks. The recall data was analyzed quantitatively and qualitatively. The results indicated that while literal and bridging inference questions increased readers' immediate and delayed recall of the text, elaborative inference questions only increased delayed recall. It was also found that questions were particularly effective in comprehending difficult passage.

View full abstract

Peer assessment is a practical tool for in-class performance testing. In order to investigate its applicability for oral performances, this study examines the extent to which peer assessments by Japanese students correlates with teacher assessments in terms of the following conditions: (a) anonymity of the rater and (b) peer discussion of students' oral performances prior to assessment. Students' oral performances in story retelling tasks were rated by their peers and teacher and these assessments were subsequently compared. The results showed that (1) peer and teacher assessments were significantly correlated when anonymity of raters was preserved and (2) determining ratings in pairs did not increase the correlation between peer and teacher assessments. These results suggest that whether peer assessment is a genuine tool for evaluation remains open to question.

View full abstract

The main purpose of this study is to examine how valid self-assessment (SA) of speech performance can be in a high school setting in Japan. Fukazawa (2009) studied the validity of peer assessment (PA) and found that PA has reasonable degree of validity to support teacher assessment (TA). In the current study, another alternative assessment, SA, is the focus. Fifty-two students and three teachers participated in this study. The validity of SA was investigated from five aspects - the content, substantial, structural, external, and consequential aspect - according to Messick's (1996) concept of validity. The validity of SA in this study means, to be more precise, the validity of interpretation and use based on self-assessment scores. The content and substantial aspects of validity were examined through analyzing a questionnaire, and both aspects were considered to have sufficient validity. The structural aspect was analyzed using Multi-faceted Rasch Measurement, and this aspect of validity was partially supported. In order to examine the external aspect of validity, the scores of SA were compared with that of TA in terms of correlations and mean scores. The result showed insufficient validity from the external aspect. The consequent aspect of validity was investigated analyzing students' comments on SA, and some positive effects were found but validity of this aspect was not sufficient. According to these results, it was concluded that the validity of SA was shown to some extent but not sufficient to use as a part of formal assessment. Pedagogical implications are also suggested.

View full abstract