Pages

Thursday, June 20, 2013

A response to Trevor Packer about AP World History Scores

Today, Trevor Packer, the College Board's Head of AP, according to his twitter bio, tweeted the following:

AP World History scores, 2013. 5: 5.7%. 4: 13.5%. 3: 29.4%. 2: 30.4%. 1: 21%. These may shift slightly as late exams are scored.

This year’s AP World History essays earned the lowest scores ever. It appears many students are being rushed into the course.

Out of 9 points possible on each of the 3 AP World History essays, the mean scores were: 2, 1, 1, the lowest essay scores ever on this exam.

113,000 AP World Hist students (60% of all) earned 0/9 pts on Q2 (politics – continuity/change in medieval cultures)

Ideally, AP World History is a 12th grade course, after a standard world history course in an earlier grade. But some 10th graders excel.

The AP World History exam is designed and scored by many college faculty, who ensure it reflects the standards of a college course.

You can see the actual tweets if you go to the feed of @AP_Trevor. 

So the test results indicated that students did universally poorly on the Free Response Questions with mean scores of 2, 1, and 1. Why is that, I wonder? And overall, the vast majority of students who wrote the exam, over 70%, did not earn a score that would garner them college credit!

In my experience, any teacher worth his or her salt can look at student performance on an exam and gauge a number of things. The majority of good teachers, when confronted with a universal fail (because that's what mean scores on the FRQs of 1 and 2 out of 9 are: failure) are faced with one of two causes for that failure. It is either a failure to prepare well, or it is a failure to write a good test. Mr. Packer would indicate by his tweets that it is the former, and not the latter, and that this is due largely to students being "rushed" into taking a course that is ideally taught to 12th grade. On what does he base this assumption? Presumably, since all AP courses must submit a syllabus to the audit process and receive approval to be named "AP," the College Board is aware of the grade levels, content, structure and pedagogy of the class being taught. Is Mr. Packer indicating that the College Board has been rubber stamping courses that are poorly structured? Surely not. Is Mr. Packer indicating that they should not approve this course to be taught to any grade lower than 12th? Why, then, are classes like mine, taught to 11th graders, approved? If the course is for 12th grade, then it would behoove the College Board to limit the courses designated as AP to lower grades by applying an even stricter standard of pedagogy to those syllabi for non-senior classes. But they don't. Perhaps there is not enough free response practice built into the syllabi of the nation's classrooms teaching AP World. But if that were the case, why are classes lacking in sufficient practice time approved? My students write an essay every two weeks, and we spend a lengthy amount of time reviewing the score guides. Is that not enough because they are juniors and not seniors? I don't think that Mr. Packer has a leg to stand on with his justification for the score results based on preparation. This simply is not an acceptable reason for the low rate of success on the Free Response Questions, and the low overall scores unless Mr. Packer is meaning to discredit the audit process. And maybe he is, but since his checks come from the College Board, I rather doubt it.

No, the conclusion we can point to is that the Free Response Questions were poorly constructed. Now, Mr. Packer's defense is that the exam is scored and designed by "many college faculty." This would certainly help to explain why recent questions are written around such esoterica as Cricket and Indian Politics, the mechanization of the textile industries in Japan and India in the late 1800's, and the "continuity and change of politics in medival cultures." Could you write a broader question? When 60% of your students can't even write an acceptable thesis statement for a question and gain one point, Mr. Packer, you have a poorly. written. question. Or you have 113,000 poorly prepared students nation wide. So which is it? Were the 40% who scored higher than a 0 on that question all the 12th graders? I doubt it.

Mr. Packer, the initial test data are telling you that you need to hire a different group of people to write your questions, because the esoteric minutia that interests your "many college faculty" don't result in good questions. I have not seen the FRQ's from this year's exam, and I shall not rely upon my student's reportage to comment on them specifically. But I can feel very safe in assuming that the questions did two things: poorly represent the scope of actual world history in any larger sense (i.e.: they were likely pulled from 600CE-2000, (a mere 1400 years, not even close to the full 8,000 years the course is supposed to address) and that they relied upon students using information about Europe to generate answers. Those are the overwhelming trends of the FRQ's in recent years. Perhaps the "many college faculty" who design and score the course are all specialists in European history? are all a group of modernists? cast offs from the AP European cohort? Of course, when you design a course that is not supposed to deal with Europe and North America, and then you only design the test questions around Europe and North America, those are the results you get. Face it, Mr. Packer, just as happened in 2002 with the first time around, your "many college faculty" wrote a bad test.

4 comments:

  1. I wonder if the issue here isn't the whole idea of offering a course called "World History" and deeming it a college-level course. College history departments usually have the sense to break up even their broadest surveys by era and/or region. If the idea is to broaden students' world views, I wonder if we risk achieving the opposite by giving such cursory attention to such a broad topic.

    ReplyDelete
  2. Personally, I found the 2013 FRQs for World History to be acceptable.

    However, I was quite disappointed with the multiple choice questions. About 1/3 of them were passage based, and essentially didn't require any outside knowledge. Now, I don't object to testing that a student can analyze excerpts from documents, as that is an essential part of understanding history. That being said, that's why there's something called a "DBQ." The multiple choice should be testing analysis of trends and information based on solid knowledge accumulated from a year of AP World History. Obviously if a student can answer factual questions correctly, then he has been effectively using his analysis skills the entire year.

    I find it difficult to believe "AP Trevor's" claim that recent AP World History exams (and specifically this year's) have been precisely equal to the pre-reformatting tests after the curve was applied. I suppose it is somehow the natural result of the (apparently extraordinarily) rapid deterioration of education that the percentage of 5s has been rather chopped in half since the revision.

    Skeptical

    ReplyDelete
  3. Hi Todd,

    I've been following a thread on the AP World History Teacher's Facebook group, and there has been a lot of discussion about this year's exam. Trevor Packer does not speak on behalf of the Test Development Committee (TDC). If you take a look at the posting thread, both of the Co-chairs (one is a HS teacher, one is a prof) of the current TDC gave very thoughtful responses to the angst over the exams this year. My own bias is that many of these people on the TDC are personal friends, and I have a high regard for their dedication and commitment to the field of world history. The committee is made up of equal numbers of college professors (one Africanist/gender, one Central Asianist, one Caribbean specialist/internationalist, and the Chief Reader who is a Europeanist) and practicing secondary school teachers who teach the course to a multitude,diverse group of students and have been leaders in the the AP World field. http://apcentral.collegeboard.com/apc/members/exam/2049.html


    Regardless of who they are, if you take a close look at the 2013 FRQs, though difficult, do follow closely with the new Curriculum Frameworks that were released last year. I think it's difficult to be criticizing an exam without taking a close look at yourself. Yes, they were difficult, and I know that my students probably struggled with them. At the AP reading this year, I read the CCOT, and you can't believe how many students wrote about Athens/Sparta and Alexander the Great even though the time frame was 200 - 1000 CE.

    In terms of the MC, we knew that they were moving toward more stimulus based questions, and as the exam shifts in the next few years, there will be more of them. The passages required contextualization, so it did require students to have a good understanding of world history.

    ReplyDelete
  4. There is absolutely NOTHING wrong with offering this course to younger students. At my school, freshmen are the ones who take this course (if they achieve a high enough Explore score) and our school did extremely well. Most of my friends got a 4 or 5; I myself got a 5. Everyone though they had done poorly but really we kicked this test's rear! So no, AP WH does NOT need to be a senior course because there are definitely 9th grades intelligent enough to excel on the test.

    ReplyDelete