Monday, December 27, 2010

Difficulties measuring value added by individual teachers

On the December "Start a new post," Anonymous wrote:

"Article on problems with latest approach to evaluating teachers

http://www.nytimes.com/2010/12/27/nyregion/27teachers.html?ref=education

This article points out some of the problems with this recent way of evaluating teachers. I've always thought this evaluative approach is silly. A good principal is one who is regularly present, regularly visiting classrooms, regularly discussing with teachers what they're doing in the classrooms. Instead we want this impersonal, statistical tool that -- what? allows principals to not get their hands so dirty? I know, it's an old school, unfashionable approach, and one that takes time and training on the part of the principal. But I think it's also a more common sense solution than the silliness described in this article."

30 comments:

Questioner said...

From the article:

"... on average, students placed for a year with a high-value-added teacher will do better than those placed with a low-value-added teacher. Dr. Staiger placed the improvement at about three percentile points on a typical standardized test."

- A "high-value-added teacher" is defined as being in the top 5% and a "low-value-added teacher" is defined as being in the bottome 5%. That would seem to mean, then, that if a student was in the 20th percentile in kindergarten and a low-value added teacher would leave her at the 20th percentile, but instead she was taught by only the very best teachers for the next 12 years, the student would be in about the 56th percentile.

But under the system described, 70% of teachers are "average" or "above average"- so it seems likely that it may take quite a few years to see even noticeable results from teacher training.

Anonymous said...

And how much control of ALL other variables is necessary for any prediction ( and that is what it is, a prediction) to be realized?

Questioner said...

And the disruption caused by changing teachers may have an a negative effect scores.

Anonymous said...

“If I thought they gave accurate information, I would take them more seriously,” the principal of P.S. 321, Elizabeth Phillips, said about the rankings. “But some of my best teachers have the absolute worst scores,” she said, adding that she had based her assessment of those teachers on “classroom observations, talking to the children and the number of parents begging me to put their kids in their classes.”

Comment from a principal. Even some administrators are skeptical.

Anonymous said...

This was a relatively well-written article; it covered many of the critical issues with perspective and opposing data. Generally, it comes back to the same point. Those with expertise and experience can identify quickly and accurately examples of "good, effective teachers." Those without experience and expertise need to do tons of research which is usually inconclusive, unreliable, and proves whatever the individual researcher decides that it will prove.

The article is comprehensive enough to call all of the current value-added evaluation systems into serious question.

Old Timer said...

And that's the point, anon. I cannot name one current central office administrator who was known as a success in the classroom.
Not one.
If these people do not know good teaching, then they'll rely on what they "research".
More than anything else--building a bridge to all of your students and their many learning styles and personalities is of utmost importance.
People skills.
I can think of one assistant superintendent who knows nothing about people skills and numerous learning walker "leaders" who consciously seek to cause trouble.
Given this, how can we hope to get a fair shake as to what "effectiveness" is all about?

Anonymous said...

There is widespread agreement with the comments you make "Old Timer." It is not likely that a "fair shake" from current central office administrators is coming any time soon. However, they cannot be in all classrooms all of the time. Take charge of your environment _____ students and teachers ____ collaborate. Communicate objectives and goals between and among one another____including the students____ that's a priority!. Create an agreed-upon way to reach those goals and objectives_____it can happen. Get results___ your (collective) way____ with your "solutions."

When observed, put on the expected show for the 'clipboard types' then return to "what works."

Anonymous said...


When observed, put on the expected show for the 'clipboard types' then return to "what works."


This used to be possible. It is no longer possible if you want to keep your job. Between PELA principals, pacing schedules for scripted curriculum and tests written to that scripted curriculum (not by the teachers), it's no longer possible. It really isn't.

Not that I'm saying teachers shouldn't speak up, but I certainly recognize the danger that comes from speaking up. The danger that comes from not doing exactly what you're told is even clearer.

Questioner said...

What is a CBA?

Anonymous said...

I believe a Currriculum Based assessment. aka benchmarks maybe. My son always said too many tests. I will be giving myself away because I once explained to an administrator what teaching is. My son in middle school said "teaching is something a teacher does in between giving tests."

Questioner said...

That is too funny!

See what Parent1 meant about consulting with the experts in our midst?

Anonymous said...

When a school district has to create an "office of teacher effectiveness" and then staffs it with folks who generally have little or no real teaching experience, all under the leadership of a superintendent and a deputy superintendent who have never, ever been school principals, its time to turn out the lights.

Real innovation and results are happening all around Pittsburgh, with a research office, an office of strategic initiatives, an office of teacher effectiveness, PELA, PULSE, VAM, Broad interns, America's Choice, Focus on Results, a ridiculous ration of central administrators per student, etc.

Test scores won't hide behind glossy brochures for long.

Old Timer said...

Anon, I agree with you and have to think that you are being facetious in paragraph 2. However, it is clear that the leadership is cognizant of resident apathy. It is slick enough to churn out public relations pieces that gloss over all of the failures.
To me, it can go on for quite a long time.
Roosevelt established a bureaucracy of incredible proportions, replete with myriad do-nothing administrators who likely have never been in a classroom, or have not been in one for years. Most of these types likely have no clue what it means to teach in an Oliver, or King, or an ALA...and yet, they are going to have a say about what is effective?
I just want to see a prudent, courageous group of people win school board seats and then hire a prudent, courageous and visionary leader of our schools whose first act is to pink slip these charlatans, and hopefully begin with leaders and members of learning walk teams--individuals who are emblematic of the problem and representative of the totalitarian approach. These pretenders should be the first to go---especially learning walk leaders.
As you can tell, I have little patience and less regard for those stealing from the public coffers. Teachers don't need this interference--this static that detracts from instruction. They don't need individuals who consciously seek to find fault and to target teachers. These people largely have the skills to run a cash register at your local Get Go, and little more.

Anonymous said...

Oldtimer,

I too am tired of these charlatans stealing from the public coffers. If change is to occur, it must begin at the board. However, I know of only one person that is willing to challenge an incumbent member. It will take several others to truly effect change. There must be more people out there in the District who are tired of the B---sh-t and are willing run for school board.

Anonymous said...

there is a lot of conversation
about teacher's evaluation,now after you get this data who gets it
and for what purpose,i really have not heard anything on this topic
is it use for measuring on how to improve the academic growth and student achievement in the dist.

Anonymous said...

The information is used to put teachers on improvement plans. Essentially, it is a pink slip hanging over your head for a year. Talk about a threat, if that isn't one, I don't know what is.

In other words, the information is not used to close the achievement gap or to grow student achievement. It is used to keep teachers in line.

Stephanie Tecza said...

QUESTION

How can I find out how many students are at Clayton Academy and at McNaugher and has the population in the two schools increased, decreased or stayed the same over the last couple of years?
Any suggestions???

Old Timer said...

It's probably hard for the average individual to believe that there is an office of compliance within PPS--with tentacles that basically work to go after teachers. It sounds like exaggeration and sounds almost like science fiction.

And yet, the truth is stranger than fiction.

It's your tax dollars at work.

Ask yourselves about why this situation is allowed to exist and then remember: it is what Gates is all about. The money is NOT a philanthropic effort but instead, an effort to rid public schools of teachers who do not fit in line with what Bill and Melinda Gates envision as being "effective".

What you continue to see is big money being used to push a personal agenda.

Where do we live again?

Questioner said...

Here's the McN info: 102 students

http://www.pps.k12.pa.us/1461201026164457277/site/default.asp

An official chart on file from PPS shows McN at 97 in Sep 2008 and 116 in Oct 2009.

Questioner said...

Current Clayton numbers do not seem to be on the PPS website "School Information" section, but the official chart from previous years shows 203 in Sep 2008 and 201 in Oct 2009.

Of course, next year Clayton will apparently become something different, a credit recovery center.

Anonymous said...

ok the evaluation is to improve plans and keep them in line with a bountty hanging over their as a pink slip for what i still did not get a specific answer,on the other hand talk about high-value and
low-value teacher since when we start using this language
how do you expect the community to
understand all of this jargon
let's get to the point on what this all means!!!!!

Anonymous said...

Anon 12/30 12:50,

You perplex me. What does it mean? Does it need to spell out for you? It means that this is private money being spent to destroy public schools not to improve student achievement nor close the achievement gap. It is to destroy the public schools so that privately run or charter schools may replace them. Once in place, the taxpayer will be sattled with the enormous bill of operating and maintaining all the new schools. And more than likely, there will be no improvement in student achievement and an even wider achievement gap. But our youth will have been trained in an ideology that might well destroy us. Have you looked into the Gates foundations, as well as the other billionares?

As for the high and low valued teachers, who cares? The rating system is flawed! Even those so-called high rated teachers have bad days too! It's a blame game. That is all it is, and the teachers are the easiest to blame. Afterall, my dear, adorable son Johnny is an Einstein.

anon said...

I have never taught anyone anything and even I know the evaluation system is no good at all. It will create an anti-collaborative atmosphere, to devolve into fingerpointing and accusation all day long. Principals had better never have had a prior working relationship with anyone they will evaluate since it will color the current performance evaluation. We will soon see just what a small town this is since there seems to be noone without some, even scant, connections. The best advice is keep a journal.

Questioner said...

Anonymous tried to post the following but it did not come through:

"Interesting reading:

http://www.pittsburghlive.com/x/pittsburghtrib/opinion/columnists/guests/s_715659.html

Posted by Anonymous to PURE Reform at December 30, 2010 5:44 PM"

Questioner said...

Weissberg's views seem fairly ridiculous. To address just one point, if the issue is cognitive ability then why are American students doing so much better than the ancestors they left behind?

Old Timer said...

This bears repeating, and is likely something all older teachers could relate:

In almost three decades of teaching, I can only remember one vindictive, venomous and overbearing principal who thought that it was all about her and not the kids, not their achievement and not the process. I can not remember any other principal, vice principal/dean or central office staff ever striking me as adversarial, or as someone who simply was out to "get me".

In the last five years, things have changed.

I can no longer name any principal or central admin type who is NOT a true believer in the Gates process, who actually questions what is being handed down or who sees the kids as the primary focus. (I will not lop VP's and Deans into this, as they have little choice and feel as if their days are numbered, too).

That says it all, doesn't it?

Old Timer said...

Weissberg merely reaffirms what most teachers already know: in a rip and read media, no one is checking the numbers.
This piece should be mandatory reading for all school board members and all bleeding hearts who continually tell us we're not measuring up. In fact, I can think of one assistant superintendent who prides herself on research who I am sure will downplay this report as it casts serious doubts upon her views about curriculum and grading.

The simple point is that we test EVERYONE in American schools where our "competition" does not. If you were going to play football against NA, you'd like to think you were doing it with comparable players, not kids who had never seen a football or had limited exposure.

Here's a loaded question---what do Germany, Japan and other of the countries mentioned do with consistently unruly students? With kids who are not native speakers? With kids who have learning difficulties?

When will media do the legwork on stories?

Anonymous said...

Well, here comes the answer to the diffivulties-measuring value added-by adding new teachers-ridding the difficulties

http://www.thericblog.org/employers/pittsburgh-resident-teachers/


Pittsburgh Resident Teachers | RIC BLOG
By James Doyle

The Teacher Academy Residency with Pittsburgh Public Schools has launched! Hello Friends. As a longtime supporter and friend of the Regio...
The RIC Blog: Internship Resources -

http://www.thericblog.org/

Anonymous said...

PA Bulletin, Doc. No. 11-28

Resolved: That the New Teacher Project and Pittsburgh Public Schools shall report to the State Board of Education and the Department of Education before ...

www.pabulletin.com/secure/data/vol41/41-1/28.html

Approved and delivered-just as the previous post-another blog entry?

Anonymous said...

As far as administrators falling in line with "Broad," one non-Broad administrator told me,"You do what you have to do." There are some non-Broad ones still in PPS. Obviously, if you want to keep your job, you fall in line - or else.