Friday, May 25, 2007

Fraser school ratings invaluable for parents

It's appalling to hear some of the nonsense being talked about the Fraser Institute's recent report on elementary schools across the province.
Kids are in big trouble if they can't read, write and work with numbers. Before they're even out of elementary school, some children - many children - have fallen behind. Catching up is hard.
And some schools, year after year, are doing a significantly worse job than others of ensuring children meet minimum standards for literacy and numeracy. More of the children who go to those schools are starting life behind.
That should alarm us. The public education system is supposed to be the great equalizer. Poor home, inattentive parents, too many moves - all children are supposed to get the chance to learn the basic skills that will let them make the most of their lives.
But it's not happening. The latest report from the institute looked at elementary schools. It's based on the performance of students in Grade 4 and 7 on standardized provincial tests in reading, writing and numerical skills. The report includes information on the percentages of special needs and English as a second language students in schools, factors that can affect test results.
Many schools have great results. That's not surprising; B.C. has an excellent public education system.
But the results also show some big problems. The report, among its measures, includes data on the percentage of students who aren't meeting the province's basic standards for reading, writing and math skills.
That's a critical measure of how many children are being left behind. Up in Prince George, across the public school district, 22 per cent of children were not performing up to the basic standard set by the province. That's worrying in itself.
But there were also dramatic variations. In one school, only a few students - seven per cent - did not meet the basic standard. But at five schools, more than 30 per cent of students failed to meet the standard. At one school, almost half the students weren't able to read, write and do math at the level the government considered acceptable.
Parents - anyone with an interest in children and the future - should welcome that information. If almost half the students in a school aren't meeting standards for basic skills, the school needs help. If a school district has too many schools where students aren't meeting standards, the district needs help.
None of this means the teachers are bad, or the principal, or the program, or that there is something wrong with the children. It means there is a problem. It's not the children's problem; they just need it fixed.
Instead of accepting that, the reaction from some quarters - sadly, especially from some teachers, backed up by Education Minister Shirley Bond - has been to attack the review.
The basic argument is that schools' success doesn't rest on test results. The ability to involve children in a supportive community and help them experience the arts and sport, to make school joyful, those are important too.
That's true. A school that simply churned out students who did well on tests, without gaining the experience of working with others and the joy of learning, would be a failure.
But it is extremely important that children learn the basic skills that will give them a fair chance at life. When the report suggests that that is not happening for a significant number of students in a school, we should pay attention.
That doesn't mean panic at slight differences or one-year swings in results. The number of students can be small and results skewed.
But if two schools are in a district, with similar populations, and one is doing much better at ensuring students acquire skills than the other, then perhaps lessons can be learned.
And if in one of those schools almost half the students aren't getting needed reading, writing and math skills, then we have to do something.
Footnote: Some critics dismiss the report because it comes from the Fraser Institute, which has a mission of promoting privatization. That's reason to look critically at the information, but not to dismiss it blindly. The reason the report is widely publicized is because it responds to the near-total lack of useful information on school performance.

10 comments:

Anonymous said...

Paul,

On the one hand, it is worth noting that the data used by the Fraser Institute are just the province's own data. So the broad conclusions you are drawing about needs in the system are correct.

The problem with the Fraser Institute rankings is that they completely gloss over the socioeconomic context of a given school, instead presenting the information as if it was a choice about which SUV to buy; that is, as if the differential in rankings was attributable to the quality of the teachers and principal, rather than the fact that there are lots of immigrant families or poor families in the neighbourhood.

Having just been through the process of registering for kindergarten, I can assure you that tensions are running high among middle class families. In my case, the local school in East Vancouver had 17 out of 19 kindergarten kids with ESL, and my child is already reading at a very high level due to two years of pre-school and lots of attention at home. My greatest fear was that she would be bored with school and turned off of it due to that early experience. I think others feel the same way.

We were lucky and won a lottery for one of the VSB's special progams. Others go the French immersion route, not because they so love French but because they want some challenge for their child. Yet, for the local school it becomes a self-fulfilling prophesy, as none of the middle-class families want to send their child there. Whereas if everyone sent their child there, and there was adequate support funding for the children who need it, that would likely be a better outcome.

If a political party were to promote a major education push, like transitioning from child care to comprehensive early learning programs, and doubling the K-12 budget so that there were resources to accommodate the ESL and special needs kids, I bet they would win a lot of support among families with children.

Anonymous said...

Paul,

I would treat the Fraser Institute statistical analysis very cautiously. My child attends the school that was ranked the bottom of the Lower Mainland Schools. Interestingly, it's name is Fraser too .. so I'm hoping the school is not in the practice of teaching deception like its apparent namesake.

Also, I would like to preface my comments with the following: I would rate my child's Grade 4 teacher as one of the better teachers that she has encountered.... and I'm glad she has had the opportunity to attend her class. Also, when I recall my 18 yrs of my formal education experience, I think it's the teacher that makes the difference, and the best ones inspire their students by instilling a desire to achieve their best.

Anyway, a close look at the data that Fraser Institute used for this analysis is interesting. In Simon Fraser Elementary Grade 4 class there are 27 students of which 31.8% are ESL students, 9.5% special needs, and 32.3% didn't write the tests - which means only 18 students were tested!!

Compare this to the Fraser Institutes top rated school - Crofton House. Of the 40 students in Grade 4 - none were denoted as ESL or special needs students, and only 0.8% not write the tests, which essentially means that 39.6 students were tested!!

Back to Simon Fraser school's results: What I do know for my child's class, a number of parents withdraw their child from participating in the tests because of their concern with the Fraser Institute's agenda (promoting private over public education) and the CanWest's apparent support of this by devoting considerable space to publishing the results, without any critique of the statistical analysis used.

It is important, if the statistical analysis is to be meaningful, that an accounting of the non participating students be appropriately addressed. I've only limited knowledge in such analysis, but I do know that if it's not, the results can skewed to a point that they're meaningless. It was reported that the Fraser Institute assigned the non-participating students a zero grade ... and if that's case then the Simon Fraser school's ranking is completely meaningless when compared to the results of Crofton House. Would the public realize this ..... I would suspect not.

I rest my case with the following well known Mark Twain saying ... There are three kinds of lies: lies, damned lies, and statistics (or simply ... how accurate statistics can bolster in inaccurate answers).

paul said...

I don't usually join the discussion, since others should have the last word, but. . .
I don't understand the eagerness to frame this as a public school, private school debate, as in the Crofton-Fraser comparisons. That seems a side point.
In the case of Fraser elementary, only 18 students were tested, and five of them weren't able to do math or read and write at the minimum level the privince considers necessary. (Non-testing students weren't give zero and were noted in the reported data.)
That may be a one-year blip. Perhaps flu was going through the classroom on test day.
But surely that's useful information. What do we to reduce the number? The solution might not even lie in the school - perhaps the real need is pre-school classes to help kids be ready.
But ignoring the results is unfair to kids.

Anonymous said...

A teacher who lives in our block teaches at one of the schools that scoe high acording to the Frazer group. Seems the teachers get a chance to decide who they won't be taking as students next year. When she taught at other public schools, they took whomever showed up. WEW have a relation who teaches at a high number private school as well. same deal as to whom is accepted. This is not a level playing field.

Dawn Steele said...

1) You need to distinguish between the usefulness of the FSA tests themselves as an indicator and the way they are used in the Fraser Institute report, which is riddled with flaws. The FSAs on their own don't tell us much, thought they can serve as useful indicators, helping to highlight trends and anomalies that deserve closer scrutiny to check for potential problems. [It's like checking odd freckles for possible cancer, though most will be benign. Likewise, the methodology of the Fraser Institute rankings is fatally flawed in that it incorrectly assumes and labels all freckles as cancerous.]

2) Any standardized testing approach assumes that students are all equal - little identical widgets turned off the assembly line. The authors of BC's School Act understood better. The goal was to allow all students to achieve their individual potential and become useful citizens. More than ever, with classrooms of gifted, average, geeks, jocks, artsy, ESL and special needs students, our schools reflect the true diversity of our society. A "standardised testing" approach to accountability fails to take into account that diversity and therefore tells us little about whether the system is helping every child achieve their unique potential. We need regimes that provide individualized assessment of continuous progress.

3) We all want accountability. it's horrendous but true that kids fall through gaping cracks and leave elementary school unable to read. Trouble is, the kids most at risk are excluded from an accountability regime that now focusses almost solely on standardized testing. All students with intellectual and developmental disabilities are automatically excluded from school performance reporting systems and the Ministry provides no alternate oversight mechanisms to hold schools accountable for their outcomes. [The Ministry doesn't even monitor graduation rates for students with autism, for example. It just assumes they won't graduate, although many, like my son, have been tested and found to be perfectly capable of getting a full Dogwood.]

4) The Fraser Institute ranking is a farce and moreover it is punitive -- harming the same kids it purports to help. My son's mid-Vancouver elementary school was faring relatively poorly on the FI rankings because we have an 85% ESL population. Low scores scared parents off, which meant we were losing ever more resource teachers as our numbers dropped each year, & thus increasingly unable to support all those ESL and special needs kids.

So our enterprising principal started "excusing" all the lowest performing kids -- which is totally permissible under this regime. Voila! The Fraser Institute ranking went up, enrolment is on the rise again and that means more help for all our kids.

5) The results of a single test probably tells you more about how seriously the teacher takes the test (as opposed to her teaching in general), whether a kid is having a good day or not, self confidence vs. nervousness, short-term recall vs. true understanding, etc, etc. I was a phenomenally successful test-taker in school and quite frankly, it didn't get me as far in life as most of my "mediocre" classmates.

If the Fraser Institute truly wanted to help students and families and to reward quality teaching, they would throw out this discredited ranking system and start an open debate about ways of providing real accountability in our schools.

Parent said...

The FSAs tell us far more about the demographics of a school community than they do about teaching effectiveness or school quality, which is why FSA results almost perfectly correlate to property values, income levels, educational attainment of the parents.
When our oldest child attended a school that served a very diverse and often disadvantaged population, teachers responded to kids who were failing/struggling by adding before and after school tutorials and many other forms of support. Many children came to that school with little knowledge of English, and for many their parents were not literate in any language. Few attended preschool. Some kids had never held a book before they started kindergarten. For our eldest, it was an amazing school filled with incredibly enthusiastic and motivating teachers. With that education, our child was able to go on as a successful honour student and get full scholarships and early admission to university. Sadly for many of our child's classmates, parental drug problems, poverty, domestic violence and other issues overwhelmed the at-times Herculian efforts of staff to educate them. That school she went to ranked near the bottom of Vancouver schools in Fraser Institute rankings, which to this day causes our child a lot of grief and resentment after experiencing first hand what an excellent school it was.
Our next two kids attend a different elementary school in a far more affluent community. When students at their school struggle, it is very common to have staff suggest tutoring, Kumon, Reading Foundation etc. Most kids there attended preschool and come from families where university graduation is the norm. This school ranks very high in FSA rankings.
I've believed for years that kids at our present school could play ball all day and still score high on FSAs as they are so well prepared for school, supported at home and have a rich range of extracurricular activities. When they struggle, parents have the means to obtain outside support.
So when these two different groups sit down and write FSAs, what are we really testing? It's an apples and oranges situation and it's getting worse as supports for students with special learning challenges (learning disabilities, autism, ESL) are steadily declining — at least in our district — year by year. A top-notch teacher with a few unsupported kids in his/her class can see the whole program derailed. We are holding teachers "accountable" for many factors that are simply beyond their control by administering a standardized test to groups that are not "standard" at all.
I sincerely believe these tests are misleading and damaging and provide no valuable information.

Dawn Steele said...

"Parent" is right, the Fraser Institute rankings tell us far more about how much the kids had going for them in the first place than about whether schools or teachers are doing a good job teaching the basics -- look at the almost perfect correlation between scores and the parent demographics (and where it's not a perfect match, a closer look will usually show someone "playing" the numbers, as our school did).

The Fraser Institute and everyone else has missed the forest by focussing entirely on the trees! Because what the test results do tell us is that the public education system as a whole is failing to serve as the great equalizer.

But it's not the individual schools and teachers that are the problem, because they are all consistently and equally failing vulnerable kids, no matter what they do. If BC's public schools are failing vulnerable kids province-wide(and there should be a very interesting report out tomorrow from the new Child & Youth Advocate in that regard!) we need to ask what we are doing wrong at a provincial level. Two suggestions:

1) Underfunding of special ed, ESL and other learning supports for vulnerable kids. The Province pays an extra $16,000 for each student with autism, whereas it costs at least 3 times that much to provide special ed services. Schools get little or nothing extra to support the extra needs of kids in foster care. BC has lost 15% of special ed teachers, 18% of ESL teachers and 5% of Aboriginal teachers in the past five years, at a time when student enrolment overall has declined by just 6%, and when enrolment in most Special Ed and ESL categories has actually increased, in some cases dramatically. Vancouver's cutting another $6 million for 2007-08, most of which targets resource teachers who support the most vulnerable kids. Parents are so desperate they're calling for segregated schools again!

2) Accountability (or lack thereof), which explains #1. These kids often aren't counted, or their failures are disguised by averages or excused by demographics. Their parents don't vote or if they do, they don't have the political clout to sway local or provincial electoral outcomes. There are no effective appeal mechanisms and no Ombudsman until the recent appointment of the new Representative for Children & Youth.

Anonymous said...

The vitriol against skills testing is unfortunate. If you can't quantify the results of a program, how are you supposed to determine if it's effective? Demographic factors do not change the fact that this test assesses whether students are acquiring the basic skills they need to function in our society. Considering demographics such as ESL and special needs can help us to design solutions to the problems, but can't be used as an excuse if we're not getting students an adequate education. Teachers need to embrace this type of data - it can be their best friend if used to expose problems and their scope. Instead, teachers (or at least their union) fixate on anecdotes and complain that they have to "teach to the test." As an engineer and scientist, I have to point out that data trumps opinion any time. P.S. - if the test measures what the curriculum expects students to learn, why should giving students a test require any special preparation, assuming they're being taught the proper curriculum in the first place?

Budd Campbell said...

Who prepares the FSA tests? Is it B.C. Ministry of Education bureaucrats, or some American psychological testing company, perhaps the same testing company that prepares the SAT tests so widely used in American schools.

I think it's important to ask to what degree the FSA really is a "standardized" test. If it's prepared by the B.C. Ministry and evaluated only against B.C. students, then the comparison base is necessarily somewhat limited.

There are other international tests of literacy and numeracy which may be a better bet in terms of basic measurement.

Dawn Steele said...

Anon misses the point - it's not skills testing that's the issue, but 1) the approach of standardized testing of non-standard subjects [e.g. testing and comparing the citric acid content of apples and oranges to determine whether they're good fruit] and 2) flawed methodology that specifically excludes or allows fruit that won't pass the test to be excluded; or that allows farmers to pump up their fruit to score better on tests, etc.; and then 3) The Fraser Institute's mis-use of the tests to announce that apple farmers are lazy and inferior because their fruit have lower citric content than those of orange farmers.

We need skills testing. We need better skills testing. And that means re-thinking what attributes we're testing, how we do it, and how we interpret the results.