Jay Mathews

• Education News • Metro Schools News • Class Struggles Column Archive • Recent Articles by Jay Mathews • Metro news • Talk: Metro message boards • Live Online Transcripts • Subscribe to washingtonpost.com e-mail newsletters

Class Struggle
With Reporter Jay Mathews
Thursday, Dec. 6, 2001

Jay Mathews is an education reporter for The Washington Post, where he has been a local, national, foreign and business correspondent for 29 years. His Class Struggle columns detail local and national education issues. His most recent column, "Understanding the Myth of Grade Inflation" countered the notion that today's students have it easy academically.

Mathews's fourth book, Class Struggle: What's Wrong (and Right) With America's Best Public High Schools, was the first detailed look at the dynamics of elite public high schools. It ranked the nation's most challenging schools and revealed how schools denied many students a chance to take their most demanding courses. The American School Board Journal named it one of the most notable education books of 1998, and the book's Challenge Index has been widely adopted. For more information, read Mathews's 1999 Post Magazine article, "High School and Beyond: The Challenge Index Revisited" and the Washington area high school rankings.

Submit your questions and comments before or during the discussion.

Editor's Note: Washingtonpost.com moderators retain editorial control over Live Online discussions and choose the most relevant questions for guests and hosts; guests and hosts can decline to answer questions.

To read the most recent responses, click "Get New Responses" or select "Automatically Update Page."

washingtonpost.com: Jay, in compiling your latest edition of the Challenge Index for various high schools, were you surprised by anything you found?

Jay Mathews: Thanks again for the opportunity to explain this rating system, and underline why I think it is an important way to look at high schools.
Let's start what DIDN'T surprise me, which is the high level of college-level course participation in high schools in this metropolitan area. Only 5 percent of schools nation wide have a rating as high as 1.000, an average of one AP or IB test for every graduating senior. In the Washington area, 39 percent of schools have reached that high mark. In the nation, 40 percent of high schools have no AP or IB tests at all. Among the 148 Washington area high schools, there was only one that did not give any tests in 2001.
So our overall record is very good, but let's talk about the surprises.
First, the Washington Math Science Tech charter school in DC. Having covered its opening three years ago, I knew it had the potential to have an AP program, and I kept checking to see if they had begun to give tests. It is the first charter school to appear on the list in the Washington area, and I wanted to make sure I got their numbers. I was very much surprised that they debuted so high on the list, with 122 tests in a school that only graduated 67 seniors. As I said in the DC Extra story, none of those tests received scores high enough to receive college credit, but teachers familiar with AP and IB learned long ago that even a failing score on one of these tests, if they student has been trying, is a plus, because they are ready for the trauma of their first college test. And now that MST, as the charter school is called, has jumped into the water, they can start learning to swim better.
I am also surprised that many otherwise fine schools that draw lots of middle class, college-bound kids, particularly in Howard County, still do not show as much AP or IB participation as they seem capable of. I hope to visit some of them in the new year and find out more about this.

Bethesda, MD: I graduated in 1996 with 5 AP courses under my belt - which transferred into more than a year of college credit. I felt completely comfortable in college, and credit much of my success to a wonderful high school experience.

My question concerns the methodology of the challenge index. I took my first AP test (Biology) as a sophomore, the two more as a junior (Psychology and Chemistry) and two as a senior (English and European History). Many of my friends also spread out their AP classes over the last two years of high school. Yet I know many other schools only encouraged students to take AP classes in their senior year. How does the challenge index take into account AP classes taken earlier in high school, rather than just the senior year? Thanks!

Jay Mathews: I am delighted to hear that the courses worked for you, and I apologize for the confusion about how I count courses taken in the junior and sophomore years. We tried to solve that problem by adding this sentence to the introduction of the list: "Tests taken by ALL students, not just seniors, are counted." But the print is sort of small and my 56-year-old eyes have to squint to read it.
The formula for the index rating for a school is to take the total number of AP or IB tests given in May and divide by the number of seniors graduating the next month. I divide by the number of graduating seniors because it is an easy number to get and serves as a useful measure of the relative size of each school. By dividing by the number of seniors, a BIG school like TC Williams in Alexandria with 544 graduates has no unfair advantage over a little school like MST with only 67 graduates.
But it leaves the false impression that I am only counting tests taken by seniors. That is not true. I count the total AP grades reported back to the school by the College Board, or the total IB course-ending exams, and that includes tests taken by seniors and juniors and sometimes even precocious sophomores like you once were.

Binghamton, NY: Hello Mr. Matthews,

Last year, I graduated from Richard Montgomery High as a part of the IB Program. In a recent article, you stated that these magnet programs actually hurt students' chances of being accepted into the elite universities. This is all too true. Unfortunately, I was rejected from all but one of the colleges that I applied to. Can you see any way that this trend can be stoppped and changed so that students are rewarded for undergoing a rigorous magent program like the IB program?

Jay Mathews: One thing I have learned in the more than half century I have been stumbling around the planet is that hard work, such as the work that so many RM students do, is always rewarded, but not always rewarded in the way you expect. You didn't get into the colleges you wanted to, but I suspect you are now discovering all kinds of virtues in the college you are attending, and also finding that the great courses you took at RM are allowing you to make full use of the university resources now at your disposal.
The answer to your good question is in part in a long piece I did for the magazine last April, and am expanding into a book, analyzing and debunking the view that if you dont get into a brand name college, you have put yourself at a disadvantage. If you would like a copy, email me at mathewsj@washpost.com and I will send you one.
In other words, I think you are being rewarded for going to RM, but it may take a while to appreciate HOW you are being rewarded. Surprise rewards take some time to appreciate, but over time you will see that they are just as sweet as the rewards that you thought you were going to get.

Alexandria, VA: Jay,

Allen Flanigan here. I must say I thoroughly enjoyed the interchange between you and Dan McMahon of DeMatha High School. It is reassuring to see that in this age of e-mail flaming and sound bite punditry, that two informed and passionate people can engage in meaningful dialogue, enlighten each other (and others as well) in the process, and actually reach a consensus on important issues.

I want to second Mr. McMahon's implied plea for more nuanced and comprehensive education coverage, and I must agree with his impressions that you bend over backwards to try and provide balanced and comprehensive reporting on education. I appreciate that you have included a perfunctory warning at the beginning of your ratings that the rating is "not a measurement of the overall quality of the school", and I think this warning should be expanded to clearly point out that the index may not be the best indicator of how well schools are challenging students to be prepared for college, and may not even be a valid predictor at all.

As you know, George Cunningham (Educational and Psychological Measurement, 1995, page 54) and many other statisticians have warned against confusing correlation with causation; how does Clifford Adelman know that the strong correlation between AP and IB participation and college performance implies a causal relationship (taking AP or IB actually improves your performance in college)? I suspect he does not.

I would also add to my plea for more nuance in coverage of education topics, a request that you consider broadening your approach to looking at school quality and performance by adopting a different measure, based on actual outcome. Like the Eight Year Study, you could see which schools send greater percentages of graduates to college (across demographic and social categories), and how they fare in their college careers. This would potentially provide more meaningful evidence that schools are not only trying, but succeeding, in preparing their students for college. If we really want to run schools in a logical and business-like way, we should measure the outcome we have decided is important (preparation for college success) directly, instead of inferring it from inputs. Wouldn't you agree?

washingtonpost.com: "Defending the Challenge Index," the transcript of the e-mail exchange between Jay Mathews and Daniel McMahon.

Jay Mathews: An excellent, and astute, question. Correlation, of course, can not be proven to be the same as causation, but a correlation as strong as the one Adelman found, linked with what we know about why kids drop out of college, leaves me and most people I spoken to with a strong belief that he has found something significant.
I should let Cliff's work speak for itself. The report is "Answers in the Tool Box", June 1999, and can be ordered from the Gov. Printing office at 877-433-7827.
We know that many kids drop out of college because they find the work unexpectedly demanding. We know that most kids arrive at college without having taken anything like an AP or IB course. We know that high schools give high grades for work that is no where near college level. So it makes sense that taking a hard course in high school can have an favorable impact that good grades or high SAT scores (given the limits of the SAT) will not have.
But I yearn, as you do, for more research. So does Cliff. We just need better data.
For now, however, the indices you suggest, such as percentage of kids going to college, seem very closely tied, like most other measures of high school, to the income and education of parents. One reason why I like the Challenge Index is that schools with very few upper income parents can still have very high ratings if they change their course policies in a way that seems to help kids. There is not, on the other hand, much they can do to change their average SAT scores.

Huntsville, Alabama: Jay: I enjoy your column very much. However, I'm troubled by your Education Index which relies totally on participation to the exclusion of mastery of material (i.e. percent of students with an AP grade of 3 or higher). Please defend and justify your bias towards participation vs. mastery and how can a parent obtain information on mastery of material. Specifically, percent of students which pass an AP subject exam on a national, state and school district basis.

Jay Mathews: Thanks. I am asked about that facet of the index often, and it takes some explaining.
It is very easy---just by calling the school---to get the relevant mastery figures, which in this case is the percentage of tests graded 3 or above on the 1 to 5 AP scale, or 4 and above on the 1 to 7 IB scale. That is what many people call the passing rate, because those are the scores that have some chance of getting college credit.
If this were a perfect world, and all schools encouraged all motivated and potentially capable students of taking AP or IB courses and required that all students in those courses take the tests, I would be tempted to report mastery---the passing rate----as an important indicator of what kind of job that school is doing.
Unfortunately, most schools these days do the opposite. They view AP and IB courses as appropriate only for A students, and keep everyone else out. So a high passing rate at those schools simply means they have only allowed in kids who, as I said at one point, would probably do well in the course if you just tossed them the textbook and a reading list and gave them the test 8 months later.
Also, some schools let more kids into AP but then discourage all but the best students in the course from taking the tests. thus, their high passing rates disguise a system that does not challenge all kids, and particularly fails to challenge those kids who are going to college but may have the toughest time adjusting to college. They need AP and IB, and as I have said, based on interviews with many of them, even if they flunk one of those tests, they find they are much better prepared for college than if they had not been allowed in the course. They have gone one on one against the academic equivalent of Kobe Bryant, and Kobe has beaten them, but they now have a visceral understanding of what they must do to play in that league.

Durham, NC: Since a number of school districts try to pad their figures by subsidizing the fees for the exams, it seems to me that basing the index on the number of exams taken is a flawed methodology. A better basis would be the number of exams passed, not the number taken.

Jay Mathews: See the last answer, and I don't consider what districts like Fairfax, who pay the testing fees, are doing to be padding. it is quite the opposite. The districts that do NOT do everything they can to get every student to take the tests are padding their passing rate.
The test is an integral part of the AP experience. I learned the power of AP during five years watching Jaime Escalante and other teachers at a very low-income East LA school produce phenomenal results by getting kids involved who thought they had no place in AP. They turned the test into the equivalent of the big football game against Roosevelt High. Kids became excited, motivated, studied like a team, and in 1987 that school had more kids taking the AP calculus exam---with 2/3rds of them passing---than all but 4 high school, public and private, in the country.

Alexandria, Va: Yesterday's Post editorialized in support of efforts to permit to permit the spending of hundreds of thousands, if not millions of dollars at a particular public school if those funds come from that school's community (not just the parents?). I'm afraid of one school having the "Exxon Auditorium," from Exxon parents in its community, while another makes due with a stage in the cafeteria (as some do now). What's a reasonable limit on public/business contributions to schools that add services there unavailable widely throughout that system?

washingtonpost.com: Post Editorial: "Private Funds, Public Schools"

Jay Mathews: I just don't understand why we would want to do anything to limit the amount of resources used to educate kids. If the money is there at EXXON high, lets spend it. We already spend far more federal dollars at low income schools than we do at high income schools. I am all in favor of that. Those schools need more help. But i don't see how you help those low income students by telling the parents at the high income schools that they cannot spend their own money on their kids educations. It is that achievement gap problem, which I wrote about recently. We should be focussing on making every kid better, not focussing on how much further some kids are ahead of other kids.

Robert E Lee, Springfield VA: Our school is in a transition from having AP classes to an IB program. I see it as being headed in the wrong direction. AP is much more widely accepted for credit by US Universities. Do you have an opinion on changes such as this

Jay Mathews: Good question. I did a column on this recently. Email me and I will send you a copy. The colleges are wrong not to give more credit for IB, which in some ways is more challenging and better organized than AP. The colleges are now meeting to review their credit rules, and I am sure they will change.
But credit is not a big issue any more. The College Board still uses the possibility of college credit as a way to sell AP, and they have a point, but most kids who pile up AP and IB credits do not use them. They prefer to take advantage of their full full years of college, and perhaps just use the IB or AP credit to get themselves into a higher level college course.
I have talked to lots of college Dept chairs about IB vs. AP, and it is clear if an IB student can show, through just a quick conversation, that they have already covered the material in a course they want to get credit for, or skip, they will get what they want. It will take the bureaucrats a little longer to change the stupid rules they have at present. And kids in IB courses can do what many do at schools like Richard Montgomery---take both the IB and the AP tests so they are covered for any eventuality.

Washington DC: I am wondering why you do not take into account the scores received on AP tests, as well as the number of tests taken. Not all AP courses are as good as they should be -- at our high school, the AP Probability/Statistics teacher did not know how to pronounce "chi-square" and not a single student in his class scored above a 3 on the AP test. The kids really learned very little statistics....

Jay Mathews: See above answers. One of the many nice things about AP and IB is that such clumsy teaching does not go undiscovered, at least if you are in a district like Fairfax that is smart enough to pay for the tests so that they can require everyone to take them. The record that you have mentioned is a big red flag for the principals, the students, their parents and the teacher. Unfortunately, in most situations, students would have nothing to prove that there was a problem.

Centreville, VA: Jay,

Why isn't Westfield High in Chantilly, not included in your report?

Jay Mathews: If you squint real hard and read the fine print at the end of the list, you will see that it is one of 4 schools that are new and did not have a full senior class in June 2001, so it was unfair to count them. they will be on the list next year, and pretty high, i will bet.

washingtonpost.com: Jay's column comparing AP and IB: IB or Not IB?, Aug. 7, 2001

Washington, DC: Jay, I've written you in the past about your wonderful column and again I applaud you.

I work at an inner-city youth center and I have to admit that what shocked me most about your data is that I am dumbfounded and still cannot fathom the unique circumstances that must take place in some of the DC schools, like Anacostia or Bell Multicultural where some of my youths attend, where 70, 80, 90 and up to 98 percent (!!!) of the students come from households that qualify and apply for free or reduced lunches. Have you considered a further investigation into this possible correlation between poverty and your ingenious ranking?

Thanks again for your column.

Jay Mathews: You have raised a question that has many answers. What i want to do is look very carefully at some of the AP courses taught in those schools, by great teachers, with motivated kids, and find out precisely why so few can pass at an AP level. I suspect it has much to do with what they received in the lower grades, but I want to find out more. Thanks for pushing me in the right direction. What i saw at Garfield High in East LA is different from what I see in some DC schools (a few, as i noted, are doing very well).

Montgomery Cty: Dear Jay: Thank you for watching out for our children's educations. Bravo!

Your recent article, and the question from the RM graduate who wasn't accepted in choice universities, has me confused. My son is in the seventh grade at St. Anselm's Abbey School, which I believe you mentioned once as one of those that throws off the curve because everyone takes at lease two or three AP tests. He is thriving and challenged, but I'm still looking over my shoulder to see if my husband and I should be doing more--or saving some dough while sending him to a county school that might get him into a better college. Your thoughts?

Jay Mathews: You should NOT, i think, try to game the college admissions system. It is not rational, at least when we are talking about a few brand name schools, and it will always break your heart.
Instead, find a high school that your kid likes and that has good teachers and challenging courses, of which St. A's A is a prime example. Then stop worrying. If you have a motivated, hard-working kid, he is going to get into a fine college. The top 100 on any list I have ever seen are just as good as Harvard in any way that would be important for your child's future.
What is important is that your kid is engaged in learning, and enjoying some of what the school has to offer. If he is in a very tough school where the competition for Yale is brutal, so what? He will go to Harvey Mudd, have a fine time, go work for Microsoft and then invent a machine that will allow me to think my words onto this page.

Silver Spring MD: What do you think the effect is of AP state initiatives, where the state encourages AP testing by "encouraging or mandating AP courses in high schools, helping teachers attend AP workshops and seminars, paying — in whole or in part — students' AP Examination fees, and establishing statewide policies with regard to the use of AP Examination grades in state colleges and universities." on the challenge index of schools in a particular area?

DC, for one, has such a program that helps with exam fees.

washingtonpost.com: From www.collegeboard.com: State Initiatives

Jay Mathews: Those all sound like good things to do, although I would have to see the details. Anything that encourages more kids to take the risk of a hard course like AP or IB is good, and paying the test fees, as i have said, is an excellent first step.

Washington DC: I wasn't suggesting that you rank schools according to # of passing AP test grades divided by # of AP tests taken. Rather, why not use # of passing AP grades divided (as you do now) by # of seniors in the school?

Jay Mathews: Because I would be then giving no credit for kids who took the course, worked hard, learned a great deal, prepared themselves for college in a significant way, but failed the exam. They are the most important part of the equation, it seems to me, because those are the sort of kids who are getting no such preparation in most schools today. They are not encouraged to take AP, and if they demand a place anyway, are told they are not good enough.
"You can take that in college," they are told. but when they get to college, they lack the experience with a course that demanding, and they often fail, and at that level, failure means bye-bye college, hello McDonalds.

Durham, NC (again): Schools that promote AP exams only to their top schools doubtless are padding their passing rate, but the design of your index would take that into account -- dividing the number of passing grades by the number of graduating students takes the passing rate (number passed / number taken) entirely out of the calculation.

Schools that emphasize doing well on the exams would have a higher index score. As it is now, higher scores go to schools that promote taking the exam, and not to those that promote taking the exam and doing well on it.

Jay Mathews: See answer above. I wouldnt mind doing what you suggested as a secondary statistic, but my goal here has been to keep it simple. And the statistic I am using has a very clear message----more AP and IB testing is good. I don't think sending mixed messages would be very helpful, and it would make the list much harder to read and explain. But maybe some day....

Waterford,VA: It would seem that number of AP courses offered and the criteria for admission to AP classes would allow further interpretation of the school ratings. Do you have that data and can you publish it?

Jay Mathews: It would be interesting to show the number of kids enrolled in AP, compared to the number who actually take the AP tests. (In IB, if you are in the course, you must take the test.) That would do what you are suggesting, give the reader a sense of how many AP courses the school has, and actually would be a better way of presenting it other than number of courses, because some courses just have a few kids enrolled.
Unfortunately, it is hard getting that figure from all schools, and accurately. Whereas the number of tests they gave, and the number of kids who took them, is printed on the AP grade report they get back from the College Board.
But it is an excellent idea for future reference. It is just that this being a one man show, I stretch myself to my physical limits just collecting the data that I do collect.
The criteria for admission to AP classes is a VERY important factor, but I can think of no way to describe it quantitatively, other than the way I do. And the rules often do not match reality. I have had several schools that told me they let anyone into any AP course they wanted, then spoke to kids and discovered teachers were quietly discouraging those kids whom I think would get the most out of the course because those teachers want to have everyone in their class get a 5.
But these are good thoughts. If you thing of a way to get past the barriers i have described, let me know.

Durham, NC: Mr. Mathews: Earlier in this conversation you argued that the passing rate on AP (and IB) exams was not as good an indicator of challenge level as was total number of tests taken. But how about multiplying the two together, i.e. the total number of test PASSED per senior? Wouldn't this take into account both the level of mastery of the material and the level of success the school has had in encouraging students to take the classes?

Jay Mathews: As I said above, it has possibilities, but I think would muddy the waters. But email me with more details. Multiplying---your use of that word confused me. I dont want anyone to have to do too much arithmetic on this thing, then it because a grand mystery, like the US News college ratings.

Philadelphia, PA: Hi Jay. I have a journalism question. I am a young reporter interested in covering education and I think you do it so well. I am curious as to what you think is a good way for a reporter to approach covering education?

Jay Mathews: I have thought about this a lot. When I was a young reporter, I covered schools like we usually do---covering what the school board decided and occasionally writing about a program that worked, or a great teacher, or some problem a school was having.
Decades later, I realized that is not the way to do it. If I had my way (and my editors would object to this) I would devote very little time to covering the school board and the superintendent, and spend lots of time in classrooms, watching teachers teach, students learn, and trying to figure out what works and what doesnt.
I can do this now because the Post has kindly spared me school board reporting duties, but i think an energetic young reporter could find ways to do both. Every year I have a project that forces me to spend time in schools. This year I am regularly visiting 3 middle schools and studying the way they decide who is ready for algebra in the 8th grade, and how they get kids through the trauma of algebra. If you can work out a system that allows you to do a project like that every year, I think you would find it pays great dividends. Not only are you learning much about the chosen topic, but once you are inside the school, teachers, kids, principals, everyone will be tempted to tell you other interesting things that are happening. Then the job is a joy. Good luck.

Waterford,VA: Following up on my previous question,with perhaps a better idea. How about percentage of graduates taking at least one AP course, and average number taken for those students. This would give at least some sense of the school's effort to challenge a broader range of students.

Jay Mathews: Good idea. I have tried it, and as a comparative measure, rating one school against another, it produces the same order of ranking as the Challenge Index. And the Challenge Index data is easier to get----schools do not, as a matter of course, count how many seniors took how many AP or IB tests. For me todo it, i would have to get the individual student names as score sheet, and that is, and should be, a confidential document.

Jay Mathews: Thanks again for the great questions, many of which have me thinking. Feel free to email me at mathewsj@washpost.com about anything at any time. ---jay

   |      |