Thursday, June 20, 2013

A response to Trevor Packer about AP World History Scores

Today, Trevor Packer, the College Board's Head of AP, according to his twitter bio, tweeted the following:

AP World History scores, 2013. 5: 5.7%. 4: 13.5%. 3: 29.4%. 2: 30.4%. 1: 21%. These may shift slightly as late exams are scored.

This year’s AP World History essays earned the lowest scores ever. It appears many students are being rushed into the course.

Out of 9 points possible on each of the 3 AP World History essays, the mean scores were: 2, 1, 1, the lowest essay scores ever on this exam.

113,000 AP World Hist students (60% of all) earned 0/9 pts on Q2 (politics – continuity/change in medieval cultures)

Ideally, AP World History is a 12th grade course, after a standard world history course in an earlier grade. But some 10th graders excel.

The AP World History exam is designed and scored by many college faculty, who ensure it reflects the standards of a college course.

You can see the actual tweets if you go to the feed of @AP_Trevor. 

So the test results indicated that students did universally poorly on the Free Response Questions with mean scores of 2, 1, and 1. Why is that, I wonder? And overall, the vast majority of students who wrote the exam, over 70%, did not earn a score that would garner them college credit!

In my experience, any teacher worth his or her salt can look at student performance on an exam and gauge a number of things. The majority of good teachers, when confronted with a universal fail (because that's what mean scores on the FRQs of 1 and 2 out of 9 are: failure) are faced with one of two causes for that failure. It is either a failure to prepare well, or it is a failure to write a good test. Mr. Packer would indicate by his tweets that it is the former, and not the latter, and that this is due largely to students being "rushed" into taking a course that is ideally taught to 12th grade. On what does he base this assumption? Presumably, since all AP courses must submit a syllabus to the audit process and receive approval to be named "AP," the College Board is aware of the grade levels, content, structure and pedagogy of the class being taught. Is Mr. Packer indicating that the College Board has been rubber stamping courses that are poorly structured? Surely not. Is Mr. Packer indicating that they should not approve this course to be taught to any grade lower than 12th? Why, then, are classes like mine, taught to 11th graders, approved? If the course is for 12th grade, then it would behoove the College Board to limit the courses designated as AP to lower grades by applying an even stricter standard of pedagogy to those syllabi for non-senior classes. But they don't. Perhaps there is not enough free response practice built into the syllabi of the nation's classrooms teaching AP World. But if that were the case, why are classes lacking in sufficient practice time approved? My students write an essay every two weeks, and we spend a lengthy amount of time reviewing the score guides. Is that not enough because they are juniors and not seniors? I don't think that Mr. Packer has a leg to stand on with his justification for the score results based on preparation. This simply is not an acceptable reason for the low rate of success on the Free Response Questions, and the low overall scores unless Mr. Packer is meaning to discredit the audit process. And maybe he is, but since his checks come from the College Board, I rather doubt it.

No, the conclusion we can point to is that the Free Response Questions were poorly constructed. Now, Mr. Packer's defense is that the exam is scored and designed by "many college faculty." This would certainly help to explain why recent questions are written around such esoterica as Cricket and Indian Politics, the mechanization of the textile industries in Japan and India in the late 1800's, and the "continuity and change of politics in medival cultures." Could you write a broader question? When 60% of your students can't even write an acceptable thesis statement for a question and gain one point, Mr. Packer, you have a poorly. written. question. Or you have 113,000 poorly prepared students nation wide. So which is it? Were the 40% who scored higher than a 0 on that question all the 12th graders? I doubt it.

Mr. Packer, the initial test data are telling you that you need to hire a different group of people to write your questions, because the esoteric minutia that interests your "many college faculty" don't result in good questions. I have not seen the FRQ's from this year's exam, and I shall not rely upon my student's reportage to comment on them specifically. But I can feel very safe in assuming that the questions did two things: poorly represent the scope of actual world history in any larger sense (i.e.: they were likely pulled from 600CE-2000, (a mere 1400 years, not even close to the full 8,000 years the course is supposed to address) and that they relied upon students using information about Europe to generate answers. Those are the overwhelming trends of the FRQ's in recent years. Perhaps the "many college faculty" who design and score the course are all specialists in European history? are all a group of modernists? cast offs from the AP European cohort? Of course, when you design a course that is not supposed to deal with Europe and North America, and then you only design the test questions around Europe and North America, those are the results you get. Face it, Mr. Packer, just as happened in 2002 with the first time around, your "many college faculty" wrote a bad test.