by James Harvey
Educators and analysts excited about the possibilities of using data analytics to evaluate schooling need to take a deep breath and calm down. Despite the apparent success of analytics, such as the specialized analysis of baseball that was used in the movie Moneyball, it’s not clear the approach can be transferred to other sports, much less classrooms.
In their February Educational Leadership article (“Data: No Deus ex Machina“), Frederick Hess and Jal Mehta made an excellent argument in pointing out that too many policymakers and advocates seem to believe that “this wonderful thing called ‘data’ is going to resolve stubborn, long-standing problems.” (One picky correction: “deus ex machina” means “God from a machine,” not “God in a machine.”)
We should all applaud their observation that it “seems clear that would-be reformers have consistently overestimated the potential of data and have used data in inappropriate and troubling ways.” Their cautions about the challenges standing in the way of effective data usage are robust and well grounded. Although schools are awash in data, there is little understanding of how educators can use it to improve teaching and learning. In a powerful point, they argue that using data makes the job harder, not easier, because interpreting messy and ambiguous data requires human skill and judgment. Nor will data systems replace political debates around schooling or address the problem of data that is collected for one purpose but used for another.
It is a well-argued and nicely written piece. In concert with philanthropist Bill Gates’ recent expression of similar reservations about assessment (“A Fairer Way to Evaluate Teachers,” Washington Post, April 3), their argument is perhaps a harbinger that the tide is turning on crude accountability metrics relying on data and testing.
But like many who’ve taken the first step in a 12-step program, Hess and Mehta find themselves back on a barstool buying another round for the house. They seem beguiled by the possibility of using the same Moneyball approach to evaluate schooling as Oakland A’s general manager Billy Beane used to evaluate baseball players.
Would that it were so easy. While education policy initiatives from Hollywood seem to be all the rage recently, analytical insight into sports data extends far beyond the blockbuster movie starring Brad Pitt as Beane. For seven years, MIT’s Sloan School of Management has convened a sports analytics conference for analysts, sports buffs, researchers, and figures from the cable and television worlds, with their voracious appetites for sports programming. Extending over several days, with research papers, dozens of panels and thousands of participants, the Sloan conference aims to spot the next big thing in sporting data. The big excitement in 2013 seems to have been around a finding in an expensive paper that any high school coach could have offered for free: highly effective soccer players are alert, continuously scanning the entire field for possibilities. “Moneyball” in baseball may have represented the high-water mark for sports analytics.
Reporting on discussions with participants at this year’s seventh annual Sloan conference, journalist Marc Tracy (“Which Sport Is Most Immune to Moneyball?,” New Republic, March 7) concluded that Moneyball approaches might be applicable in a sport like baseball, essentially a duel between a stationary batter and a stationary pitcher, but they flame out in fluid sports such as football, soccer, and hockey, where many participants are expected to move simultaneously while conforming their efforts. Equally to the point, the Moneyball approach seems to work only in situations where the sole objective is winning.
Needless to say, classroom practice involves conforming the activities of many people simultaneously. And it is unlikely to be improved by looking at it through a quantitative lens that values winning at all costs—or considers learning to be a duel between teachers and students.
James Harvey is executive director of the National Superintendents Roundtable and lead author of The Superintendent’s Fieldbook (2nd edition, Corwin Press, 2013).