Sunday, August 22, 2010

Very limited PSSA information released

Last year by August 11 complete PSSA results by grade (although not by school) had been released by August 11 (see http://www.pps.k12.pa.us/14311059122535553/lib/14311059122535553/Education%20Committee/2009/ed%20comm%20ayp%20pres%20-%208-11-09.pdf)

This year information was essentially just a report on AYP status (despite statistical adjustment and whatever credit is now being given for students "on track" to proficiency the district reports that it "just missed" making AYP) with a few selected "highlights" about specific schools. From the PG:

http://www.post-gazette.com/pg/10228/1080440-100.stm

Apparently the media but not the public received some additional information later in the week; from the PG:

http://www.post-gazette.com/pg/10232/1081360-298.stm

34 comments:

Questioner said...

Here is a link to a district page showing which schools made AYP:

http://www.pps.k12.pa.us/14311059122535553/lib/14311059122535553/Education%20Committee/2010/August/2010-AYP-School-Status.pdf

Of 9-12 schools, the ONLY school to improve rather than continue along the warning/school improvement/ corrective action sequence was Schenley- and this despite being displaced to an inadequate middle school building and dismissed as a school being "phased out." Imagine what the school could have done if, rather than being decimated, it had been given just a little more in the way of resources. What is even more remarkable is that this same pattern of overcoming very difficult circumstances and making much from little holds true across the board for Schenley, from academics to sports to its theatre program. Meanwhile University Prep, which was created for students that would have been in the Schenley feeder pattern rather than its magnet, is already in the "warning" category.

Questioner said...

The district website above includes this note about the "making progress" category:

"Making Progress: A school identified as making progress is one that was previously in either School Improvement or Corrective
Action but has made AYP for one year. If the school fulfills its AYP for a second year, it will exit the improvement system and will be counted among schools meeting AYP targets."

- It appears then that Schenley was the ONLY high school that met its AYP target this year. Meanwhile Allderdice and Carrick, which were heralded as making enough progress that they were not the focus of major changes, progressed respectively from Corrective I to Corrective II and from Corrective II to another year of Corrective II.

Questioner said...

The district assertion in the first PG article cited above that "the district missed the mark in the category of reading by the subgroup of special education students" may create the impression that except for this one category of students the district is meeting proficiency standards.

However, the chart in the second PG article noted above shows that for the seven grade levels tested, in both 2009 and 2010 the ONLY grade to meet the 63% AYP target for reading was the 8th grade. At ever other grade level, fewer than 63% of students are proficient.

Questioner said...

Finally, the PG quotes the superintendent as stating that "There is nothing in these scores at any level that argues for us to continue with business as usual in the secondary schools".

- In other words, poor performance of the high schools is cited as justification for the administration's proposed reforms. However, if performance had improved wouldn't that result also have been cited as justification for the administration's proposed reforms, on the grounds that district initiatives were working? In other words, it's heads they win and tails they win.

And has it really been business as usual over the past 5 years? Wasn't the Pittsburgh Promise supposed to provide the kind of incentive that would motivate students to become proficient? Wasn't CEP intended to remove distruptive students so that other students could learn? What about the new curriculum that was put into place over the past four years ago, and programs like Read 180 and PELA principals? Weren't these changes expected to have an effect on overall PSSA's? If so, and the expected effect did not materialize, is a "trust me" approach really appropriate?

Questioner said...

One more observation- the AYP chart just does not give the impression of greater improvement at the ALA's.

Of the 8 ALA's, four regressed on the warning/school improvement/corrective action scale. In contrast, of the 39 other schools serving elementary grades (K-5, K-8 or 6-8), only four schools regressed on the warning/ school improvement/ corrective action scale. And one of the four ALA's that did improve on the scale, Rooney, is slated to be closed.

Questioner said...

PPG article on the topic of PSSA scores:

http://www.post-gazette.com/pg/10234/1081555-192.stm

Anonymous said...

CONSIDER the following:

In 2003, the Reading "proficiency'" rates for Oliver and Peabody were 31.7% and 26.5% respectively.
The Pa minimum target was 45%.

In 2006, the Reading "proficiency" rates for Oliver and Peabody were 24% and 32% respectively.
The Pa minimum target had risen to 54%.

In 2010, the Reading "proficiency" rate for Oliver and Peabody is 18% and 17% respectively.
The PA minimum target is now 63%.

Thus, we see that over the last seven years our schools have moved further away from the target until these two schools for which we have numbers are more than 80% NOT "proficient" which puts them 45 percentage points BELOW the minimum PA standard. (In 2003, they were only 14 and 18 percentage points BELOW the minimum PA standard.

PPS IS IN SUBSTANTIAL DECLINE WITH NO END IN SIGHT! DROPPING 45% POINTS BELOW PA MINIMUM STANDARDS IS UNCONSCIONABLE FOR ANY SCHOOL OR GROUP OF STUDENTS!

(Next year the minimum target will be 72%. Will our schools drop to 55 percentage points below the minimum standards? What is now in place to prevent that from happening? Another experiment?)

It would be of value for the media to chart the progress (or lack thereof) for our schools from 2003 through 2010. Can we make that request?

Questioner said...

Last year PURE Reform charted the progress from 2000 - 2009 for grades 5, 8 and 11 (the other grades did not take PSSA's back in 2000):

http://purereform.com/PSSAsince2000.html

Using the gradewide information from the PPG article we should be able to update those grade-wide charts. Charts by school would also be interesting, though. The message has been that high school results are "flat", but it sounds like we may well find that the stronger schools have been getting stronger and the weaker schools weaker.

Questioner said...

And yes it would be nice if the media would prepare some of these charts.

The explanation we received is that the reading public has a limited attention span for stories about PSSA's. But one good story on trends in PSSA's (and SAT's) would be useful, especially if reporters didn't let the district brush off declines with explanations such as "it's a different group of students taking the test from year to year".

Anonymous said...

I'm going to be interested in the Schenley actual numbers. A couple of years ago, Westinghouse won accolades as "making progress" when they made it into the 30% above basic/proficient region and as such, I've always taken this specific report with a grain of salt.
I do believe that Schenley was in the 52% area last year and again I will be interested to see figures for this year.
That said, I am in agreement with regards to the callous closing of the school. In retrospect, it foretold the methodology this administration would use in all of its decisions.
Shame.

Questioner said...

Westinghouse did show amazing gains one year, but then inexplicably the next year dropped back to where it had been before the major gains.

Anonymous said...

Questioner, when a high school makes great numbers one year and not the next it can be the direct result of JUST AN EXCEPTIONAL CLASS of 11th graders having been tested. Since 9th and 10th grades don't take PSSA there is no way to judge progress at high school. The teachers know when a class of kids is outstanding because they see them work everyday and can sense a good attitude if one exists. Teachers could likely predict 11th grade PSSA results pretty accurately if anyone cared to ask them. The way the process is operating now, with an 11th grade test, makes it easier to blame high school teachers for poor performance. Please don't anyone tell me 4Sight is a fair substitute for testing at 9th and 10th grades since the kids take those tests less seriously. Perhaps admin should think about renaming them. I am only one parent opinion and my thoughts could be way off.

Questioner said...

But, taking a quick look back at SAT scores the class that did well on PSSA's does not seem to have done especially well on SAT's.

Anonymous said...

That apparently is due to only teaching kids how to do well on PSSA tests.

Anonymous said...

Also, in a school that's smaller, like Westinghouse has been, a seemingly large jump in percentages can be the result of just a few kids doing better -- that is if you've 65-75 kids taking the PSSA in a grade, each kid counts for more than a percentage point.

If your scores are very low (in the 10% range) you can get to 20% by moving 5-8 kids up, and then advertise that you've "doubled" the percentage.

Questioner said...

And some schools definitely do focus attention disproportionately on kids at the "border" between basic and proficient.

Mark Rauterkus said...

With SAT #s in a high school, if fewer kids sit for the SATs, then the numbers can be higher.

If a bunch of kids get pushed to take the SATs (good thing) -- then the results can be worse for a school (bad) vs. past years as many of the kids would just have not shown up.

The numbers tell part of the story. Naratives as well as the numbers should be delivered to better understand. And, finally, criticial reviews of both are necessary too.

Questioner said...

In these particular back to back years the number of kids taking the SAT was almost the same, around 38 - 40 kids.

Questioner said...

The first PG article cited above contains this quote:

"Even so, district officials said that 72 percent -- 43 of 60 schools -- made adequate progress this year."

- That would lead a reader to believe that 43 schools made AYP. But looking at the second PG article, only 32 schools are listed as "Made AYP." So there is an important distinction to be noted if for example we wish to compare this year's results to last year's results: "made adequate progress" is not the same as "Made AYP" but also includes school that are listed as "making progress."

And, of the 43 schools the district credits as "making adequate progress" Schenley is the only 9-12 school. Wouldn't we want to look to see what it is doing right, and maybe rethink the decision to close the school?

Anonymous said...

Remember the word of CAUTION:

BE AWARE that schools can make AYP even though 70 to 80% of the students are NOT proficient on the PSSA. (Ex: A school moved from 17% proficient last year to 23% proficient this year; thus more than 75% of the students are NOT proficient but they "made AYP."

"Making AYP" is a misleading term when you consider that the MAJORITY of students in PPS are NOT meeting the state MINIMUM target of 63% "PROFICIENCY" in READING and 54% "PROFICIENCY" in MATH.

Over the past seven years, most PPS scores have declined by more than 10% instead of improving by more than 20-30% which is the PA expectation.

Again, the MINIMUM standard in PA is Reading 63% and Math 54%.

The PG report indicates that for the few schools reported with actual 2010 scores Peabody and Oliver are over 80% NOT "proficient:"

These same students will now be moved to other schools and as Seniors will this 80% be made "eligible" for graduation? And what about all of the underachieving students already in schools for which we have no data?

The Question is: How do your student's/school's scores compare to the PA state minimum standard?

Questioner said...

The message seems to be that actually meeting the state standards is difficult due to socioeconomic issues, but at least we are making progress since 72% of schools "made adequate progress" (which again should not be confused with "making AYP").

Questioner said...

But if you look at the results from last 2008-2009 32 of 60 schools in the District
(53.3%) ACTUALLY made AYP

(see http://www.pps.k12.pa.us/14311059122535553/lib/14311059122535553/Education%20Committee/2009/ed%20comm%20ayp%20pres%20-%208-11-09.pdf p. 43).

For 2009-2010 32 of 60 schools in the District ACTUALLY made AYP- exactly the same number and percentage as last year.

Anonymous said...

MM responds: I respectfully DISAGREE that the egregious lack of achievement for African American (AA) students (and it is AA students that are being "mis-educated" and thus are underachieving in PPS) is due to socio-economic status or conditions. There is abundant proof that these (AA) students can achieve at levels equal to and even surpassing all other students. We (educators) KNOW enough to educate AA students in equity and excellence. It seems, however, that there is not sufficient BELIEF that it can be done to go forward and accomplish that "responsibility" to do so.

Questioner said...

At the same time this is also the belief the district professes and apparently will require of new hires. The district seems to be saying that it is difficult and they and/or teachers and/or all involved just haven't made it happen yet.

Anonymous said...

When some leader finally has the gumption to say that the problem in education is the lack of support from parents, then perhaps we can move forward as a society. Americans as a rule do not like to look in the mirror to assess blame and yet, urban education's failures come down to the idea that if a child's parents do not see education as valuable--or if a child's parents simply aren't in a visible support role in that child's life---he will not put forth the needed effort because he does not see school as being important.
Roosevelt is a reflection of the state PDE, which is a reflection of the national education chief Duncan, who is a reflection of Obama: cowards who won't tell it like it is because it is not politically correct and might cost them votes and jobs.
Education starts at home. It's just easier to blame teachers.
Two last notes: yeah, there are special classes that come and go and raise PSSA scores. There are special teachers who do, too.
And...who is MM????

Anonymous said...

Educators/teachers are "special" who BELIEVE that thy can educate ALL children___and move forward to DO just that___ without excuses,

Questioner said...

PURE Reform has proposed a couple of ways to address the effects of disengaged parents:

1) Start early, really early, at the time a child is born and work to engage parents and help them to understand how important they are to the success of their child.

2) Create schools with a mix of students that is diverse in every way so that it is easier to absorb and compensate for those families with struggling or disengaged parents.

Questioner said...

Here is a link to a PA Dept of Ed page that defines terms related to AYP, although some concepts are still confusing; for example it indicates that 3 years of results may or may not be used in determining AYP and does not explain when 3 years will be used or how the public will know when 1 year or 3 years are being used:
http://paayp.emetric.net/Home/About#q10

Anonymous said...

The AYP calculations ( as detailed on the website) are done each year and when published on this site for every school in PA, will show the progress this year as compared to the year before____2009 compared to 1010.

Questioner said...

Yes but, the state website says "For additional protection of confidentiality and for increased reliability, AYP results may be based on data from this year only, or on an average of up to three years of data."

A PPS announcement also recently mentioned a plan to use 3 years of data.

Anonymous said...

Let's see how pps uses the data in judging a given teacher's effectiveness.
60 Minutes ought to do an investigative piece about these people.

Anonymous said...

Unless there is a significant change, this year, in the way the way PDE AYP data is reported for each school and district you will see, on the PA AYP, 2009 percentages next to the 2010 percentages with an immediately adjacent column citing the increase or decrease.

Information about the formulas used to determine AYP ie. "safe harbor" or "confidence intervals" etc. will also be included.

The "growth model", when used, is PVAAS, another whole system of projecting data which uses previous years student progress to project future achievement. PVAAS did not originate at PDE but rather in Tennessee. It can be used to track teacher performance. Confidentiality could be an issue depending on how this is processed.

One might call John Weiss at the PDE Bureau of Assessment & Accountability to get further information.

PSSA scores back to the late 1990's are available. PA Standards and the PSSA were developed in 1996 and passed into law in PA in 1999. The PSSA Data began to count for AYP in 2003 with minimum targets changing every 3 years____Reading has moved from 45% to 54% to 63% and next year to 72% and so on . . .

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.