Measurement Improves School Systems

Measurement is critical to improving organizational performance. It is also central to a lean strategy.

That is why I was pleased to read about a small but growing trend in which greater effort is being made to measure the performance of school systems. A recent article in The New York Times described this trend. Its jumping-off point was a description of the Paterson, N.J., school district, which has been under state control since 1991 because of fiscal mismanagement and poor academics.

It is the latest evolution of Compstat, a widely copied management program pioneered by the New York Police Department in 1994. Paterson is one of a half-dozen school districts around the country that have embraced this confrontational approach, known here as SchoolStat, in an effort to improve school performance and overhaul bureaucracies long seen as bloated, wasteful and unresponsive to the public.

SchoolStat borrows the tactics of the Compstat program — regular, intense meetings in which police officials famously pick apart crime data and, just as often, their subordinates — to analyze police performance and crime trends, and to deploy resources to trouble spots. The school version taps into an ever-expanding universe of data about standardized testing and school operations to establish a system of accountability.

In Maryland, the process has been credited with reducing teacher vacancies and increasing student immunization rates in Baltimore schools. In Montgomery County, Md., it has pushed principals to come up with strategies like encouraging students to take the Preliminary SAT by offering a free pancake breakfast if they attend.

In Jackson, Miss., the state’s largest district has used it to increase food sales in high school cafeterias by adding salads and hot breakfast items, after the data showed that more than one-third of the students were not buying meals. In Philadelphia, where as many as 42 SchoolStat meetings are held each month at all levels in the district, officials say it has helped develop strategies to reduce the number of suspensions, increase attendance and raise standardized test scores…

While school officials have pored over data like test scores for decades, in SchoolStat the information is broken down into unusual detail, not just by school but also often by student, and presented in elaborate charts and graphs so the SchoolStat panel can look for problems or trends that are not evident in routine reports. The data is updated every three to five weeks in “relentless follow-up,” said Michael Kanarek, a district official who plans the meeting.
For instance, when the SchoolStat panel examined the backlog of work orders in the district’s 52 schools last spring, the data specialists created an electronic tracking system to find out when work orders were being completed, and also kept a running tally of the results in weekly and monthly charts. When SchoolStat looked at classroom instruction, it started keeping tabs on the number of visits by instruction support staff who are required to — but do not always — go to schools three times a week.

Once the data has been analyzed, SchoolStat follows a predictable pattern in which assistant superintendents and their staffs are called into meetings to answer questions and explain problems.

It is unfortunate that the Times article, written by Winnie Hu, describes the system as “confrontational,” although I’m sure many school officials view it that way. The article notes that, as always, the cultural challenges loom large.

In some school districts, there has been a backlash against SchoolStat. Bryan Richardson, a former director of Baltimore’s program and now a consultant who helps implement SchoolStat nationwide, said that after Baltimore stepped up its efforts in 2005, the windows in his car were broken on school grounds, and a school employee called him and his staff derogatory names on a local radio show.

“Is it a dramatic shake-up of a school culture? It certainly can be,” Mr. Richardson said. “When you start moving from a culture that rewards relationships to one that rewards results, there are people who feel a sense of diminished importance and loss, and that’s upsetting and makes them angry.”…

Some school districts, in response, have sought to adopt a version of the process that avoids criticizing people or making it part of job evaluations. Instead, these districts seek to provide help to failing schools rather than assess penalties. Questions are typically addressed to a group rather than to individuals.

Questioning administrators about their schools is hardly a new practice, although it is often not as formalized as SchoolStat. Michael E. Glascoe, the Paterson superintendent since 2005, recalled that as a district official in Montgomery County, Md., he would be called to meetings to explain test scores and disciplinary issues.

“We called it ‘hell’ because some of us would go in there and be there for four or five hours,” Dr. Glascoe said.

This kind of approach is valuable. It would be even more valuable if it were part of a broader lean approach, in which lean strategies and tactics were used to identify even more waste and problems, with methods and tools available to achieve improvement.