Analyze this

Yes I know it’s the American spelling – stay with it…

Admittedly not perhaps one of de Niro’s finest celluloid capers, the tension – and comedy –  in ‘Analyze This’ derives from the gap between de Niro’s hard man mafia-boss self-perception and the reality that he secretly needs to talk about his feelings to a psychiatrist. As ‘Analyze This’ is an American film, the solution to mental health issues is of course (the clue’s in the title) – analysis.

analyze thisUnlike De Niro (as I am) I find myself in precisely the reverse situation:  analysis (of the data variety) has left me, a poetry-loving arts graduate, precariously teetering on the abyss of insanity.

As I recently emailed to a member of SLT recently in a data-related exchange …. THEY DIDN’T TELL ME ABOUT ALL THIS DATA SHENANIGANS ON THE PGCE!!!! (yes, in caps lock – and with multiple exclamation marks – I know, totally banned in any lesson of mine, but I told you, insanity, brink of).

And they don’t. Tell you about it on the PGCE. Not enough, anyway. Maybe the kindly teacher-trainers don’t want to crush our dreams too soon: we are idealistic, make-a-difference, creative types, planning lessons based on the deep love and knowledge of our subject in the hopes of inspiring future generations. But by going data-lite they send us trainees to immolation on the pyre of statistics.  A baptism of fire, if you will. (Caps lock and cliches already … oh dear).

So I started teaching English and I loved it. Still do.

But it took me a year or so to realise that to serve my students effectively, to know whether data is relevant, or (should I wish to) ascend education’s greasy pole to any form of leadership, I would need deep knowledge and understanding of statistics.

Now you may argue that I don’t, but really, I do.

I do because without deep understanding of the data tools I am using I am effectively a fraud. To myself, to my students, to my fellow teachers. I genuinely do want to understand the calculation methods behind RAISEOnline, ALPS, APS, Z score, Sig+ Sig-, UMS and all their acronymic counterparts – yes, partly because OMFG, FMSoL, my PRP depends on the spreadsheet – but principally, because I cannot trust the data until I understand how it works.

Take RAISE Online. My students’ progress is judged entirely on this data. I am judged entirely on this data. My pay depends on it.  My school is judged on it. Parents believe in it.  Yet I could not explain to them how it is calculated, or justify whether it is indeed measuring what it purports to measure.  At first, I assumed it was because I’m a mathematically challenged arts graduate, or because despite many kind-hearted attempts to elucidate the joys of RAISEOnline, ALPs and UMS, by sweet, kind, good, well-meaning people, when they were talking I wanted to stick my fingers in my ears and scream:

BUT WHAT IS THE *MEANING* OF THIS?

DO YOU KNOW WHAT YOU’RE MEASURING, REALLY?

WHAT FRESH DYSTOPIAN, DYSFUNCTIONAL, STATISTICAL SPREADSHEET HELL IS THIS?

I told you. CAPS LOCK. Brink of. Maybe it comes of reading too many dystopian novels, but it’s a mistrust which comes of not truly, deeply understanding the mechanisms by which my students, my school and I are judged daily. Reinforced by the jarring mismatch between the depth and breadth of what happens in the classroom and the reductive statistic on the spreadsheet. Yet, as I lack the mathematical knowledge and understanding, I cannot effectively question the mechanisms:

“Several of them would have protested if they could have found the right arguments.” George Orwell, Animal Farm

I knew it made no sense. The data mania was absurd. I just thought I didn’t get it.

And then I read this blog and others like it http://icingonthecakeblog.weebly.com/blog/hammering-nails-in-raiseonlines-coffin

like this – http://www.workingoutwhatworks.com/en-GB/Magazine/2014/10/Statistics_in_educational_research

… and then Carl spoke. And it was good.

http://chronotopeblog.com/2015/04/04/the-mcnamara-fallacy-and-the-problem-with-numbers-in-education/

It’s not just me. Well, okay, it’s just me admitting I don’t understand half of it.

But how many other non-statisticians are out there using statistics to judge others?

“it is not necessary to accept everything as true, one must only accept it as necessary.’  ‘A melancholy conclusion,’ said K. ‘It turns lying into a universal principle.”  Franz Kafka, The Trial

 In the strange data cycle of a teacher’s life, we input data and it is returned to us: ‘Analyze this’ and ‘Analyze that’.  What is this data? Simply a reductive numerical version of the teacher’s nuanced knowledge about the student, because a spreadsheet somewhere requires it.  So someone comes back with the ‘data analysis’ and tells you Amrita or Adam is underperforming and you think ‘I know, because I put the data in that spreadsheet so that you could tell me that’ and you think ‘but I also know who they are and why they are struggling and how to help them: do you?’  No? Shall I fill in another spreadsheet saying how I am helping them?

Yes, the Data Beast of A Thousand Spreadsheets has been created. And now it is used to being fed. And it is hungry.

Abolishing levels seems to have made no difference as yet. Because the beast must be fed. Even when we’re told it’s nil by mouth.

“This work was strictly voluntary, but any animal who absented himself from it would have his rations reduced by half.”  George Orwell, Animal Farm

I understand the need to measure progress. I believe teachers must be accountable for student progress. I appreciate the move away from levels, and grading lessons.  I like the ideas around PLCs or knowledge records with formative comments or qualitative statements to work from and hand over to the next year’s teacher, discuss with parents, or feed back to SLT. I am happy to study further and learn the methodologies, statistical or otherwise, to help me measure progress more effectively.

But as A* to C becomes Progress 8 and Attainment 8 and 1 – 9, here’s a question for statisticians from an English teacher, with an arts background, struggling to make sense of this data-but-no-levels brave new world:  can you measure knowledge, skills, understanding (or even the new sacred cows, British values and character) using numbers?

I’m not being flippant, I genuinely want to know.