Reliability
A reliable test provides accurate and consistent results
Test reliability can be viewed from two perspectives:
Student reliability Scorer reliability
1. 对05、06级学生进行问卷调查
05级90名学生，06级89名学生 各有51.1％、77.5％的学生选择重视口语
机考 各有53.3％、49.2％的学生支持采用机考 06级学生有67.2％的学生认为，机考能体
现其口语水平的60-80％
2. 对教师的问卷调查数据分析
收回14份老师问卷 13位老师支持机考，认为能反映出学生口
[准备时间：20 seconds] [答题时间：60 seconds]
2006.2.完成第一批48套试卷，即每一级配 12套试卷，每年淘汰2套，并可升级更新。
2007.2.完成第二批48套试卷，即每一级配 12套试卷，每年淘汰2套，并可升级更新。
可以Hale Waihona Puke 由组建试题 可以增减考试内容
四、口语试题库的信度和效度研究
Hints:
Numerous UFO photos in the world can show UFOs are alien spacecraft.
Many planets outside the solar system are similar to the Earth. So there is a possibility that aliens do exist.
Student receives feedback of results Teacher considers results in planning
subsequent instruction Grades are recorded
口语试题库分级
分为四级，对应教学内容 难度控制
Does the test require the student to know or do something other than what you intended and/or taught?
Does some aspect of the test prevent the student who may know the material from responding correctly?
为什么要口语考试？
Assign grades Improve instruction Motivate students to work Provide feedback to students
如何进行口语考试？ Testing
Objective Subjective
Formative Summative
Characteristics of a Test
Validity Reliability Objectivity Discrimination (applies to norm-
referenced test only) Comprehensiveness Score-Ability
大学英语 教改口语考试
中国科学技术大学 李萌涛
大学英语教改口语考试
一、大学英语教改和口语考试 二、口语考试数据及分析 三、口语试题库 四、口语试题库的信度和效度研究 五、口语评分标准
一、大学英语教改和口语考试
大学英语改革 改革的目标是要全面提高大学生的英语综合实 用能力，特别是强化听说能力。 利用英语口语考试的反驳效应 ，促进口语教学。 学生基数太大，采用口语考试系统实施大规模 的口语测试成为可能。
Nearly all photographs are blurry and many have been proved to be forgeries.
There is no evidence of intelligent aliens living anywhere in our solar system or in outer space.
Student Reliability
Test items are readable and clear Instructions are simple
Group Discussion
Part 4 Discussion [5 minutes, 20 points] Each group is composed of three students – Student A, Student B and Student C.
Topic: UFOs are alien spacecraft from outer space
meet the standard
Norm Referenced Test
Make test intentionally difficult Average score should be about 50% Strong students should tend to score
high and weak students should tend to score low Award As for highest scores, Fs for lowest scores, Cs for average scores
subsequent instruction Grades are not recorded!
Summative Testing
Using measurement tools to conduct evaluation for the purpose of assigning student grades
语水平 92.9%的老师认为，用机考成绩来解释学
生口语能力的可信赖程度达到70~90% 对照机考口语成绩与期末笔试成绩，有70-
80％的相关度
三、口语试题库
四所院校共同编写： 中国科技大学 复旦大学 苏州大学 西南交通大学
多种题型、与教材配套、内容丰富的英语口 语试题库
A valid test is FAIR
Questions about Validity
Does the test actually measure what you intend it to measure?
Did you teach the content and skills that are being tested?
系统支持文本、图形、语音和视频等多种媒 体，具有主题和级别等试题指标。
试题由多所高校优秀英语教师和测试专家开 发。
口语试题库的定位
学业考试(Performance Test), 所测即所学。 主题式(topic-based)，有近似主题，难度、
要求不同。 综合＋听说教材里所涉及的主题。 涵盖全新版和修订版所有主题。 部分题型模拟四、六级口语考试形式。
Formative Testing
Using measurement tools to conduct evaluation for the purpose of improving student performance
Student receives feedback of results Teacher considers results in planning
Student meets or does not meet the standard Competition is between the student and the
skill, knowledge, or ability Grade is based on accomplishment Everybody can earn a passing grade if they
You will have about 5 minutes to discuss on this topic. Each person can only talk for one minute at a time. During the discussion, you may voice your opinion, argue with each other, or ask somebody to clarify his or her points. Your performance will be judged according to your contribution to the discussion.
中国科学技术大学04级网络多媒体 英语教改试点班英语口语考试现场
二、口语考试数据及分析
2004年10月，中国科学技术大学2004级英语教 学改革试点班学生英语口语摸底考试，实考517人。 2005年1月，中国科学技术大学2004级全体本科 生英语口语期末考试，实考1815人。 2005年6月，中国科学技术大学2004级全体本 科生英语口语期末考试，实考1798人。 2005年12月 1813人 2006年6月 1798人 2007年1月 3707人（二个年级）
五、口语评分标准
准则参考性口试（criterion-referenced oral test)
常模参考性口试（norm-referenced oral test)
Criterion-Referenced Test
Measures student performance against predetermined standards
The UFO phenomenon may have existed for over a half-century and UFO sightings are becoming more and more frequent around the world. Some people think these objects are alien spacecraft, although there is no conclusive evidence yet. There is no doubt that unidentified flying objects in the sky do exist. But the question is — what are they? Are UFOs alien spacecraft from other planets?