You are viewing celandine13

Previous Entry | Next Entry

Errors vs. Bugs and the End of Stupidity


"A pianist has to believe in telekinesis.  You have to believe you have the power to move your fingers with your mind."

I learned that from Phil Cohn, my piano teacher's piano teacher.  Once in a while, when I was in high school, she'd arrange for me to have a master class with him.  He was a diminutive man who looked exactly like Dr. Strangelove, and had a gentle way of guiding your hands and body while you played.  He was very interested in the physicality of piano; he liked to tell stories about students of his who could play any piece upside down, or cross-handed, or one-fingered like Chico Marx.  He had a lot of theories about the process by which we can learn to control our muscle movements.

I wasn't an exceptional pianist, and when I'd play my nocturne for him, there would be a few clinkers.  I apologized -- I was embarrassed to be wasting his time.  But he never seem to judge me for my mistakes.  Instead, he'd try to fix them with me: repeating a three-note phrase, differently each time, trying to get me to unlearn a hand position or habitual movement pattern that was systematically sending my fingers to wrong notes.

I had never thought about wrong notes that way.  I had thought that wrong notes came from being "bad at piano" or "not practicing hard enough," and if you practiced harder the clinkers would go away.  But that's a myth.

In fact, wrong notes always have a cause. An immediate physical cause.   Just before you play a wrong note, your fingers were in a position that made that wrong note inevitable. Fixing wrong notes isn't about "practicing harder" but about trying to unkink those systematically error-causing fingerings and hand motions.  That's where the "telekinesis" comes in: pretending you can move your fingers with your mind is a kind of mindfulness meditation that can make it easier to unlearn the calcified patterns of movement that cause mistakes.

Remembering that experience, I realized that we really tend to think about mistakes wrong, in the context of music performance but also in the context of academic performance.

A common mental model for performance is what I'll call the "error model."  In the error model, a person's performance of a musical piece (or performance on a test) is a perfect performance plus some random error.  You can literally think of each note, or each answer, as x + c*epsilon_i, where x is the correct note/answer, and epsilon_i is a random variable, iid Gaussian or something.  Better performers have a lower error rate c.  Improvement is a matter of lowering your error rate.  This, or something like it, is the model that underlies school grades and test scores. Your grade is based on the percent you get correct.  Your performance is defined by a single continuous parameter, your accuracy.

But we could also consider the "bug model" of errors.  A person taking a test or playing a piece of music is executing a program, a deterministic procedure.  If your program has a bug, then you'll get a whole class of problems wrong, consistently.  Bugs, unlike error rates, can't be quantified along a single axis as less or more severe.  A bug gets everything that it affects wrong.  And fixing bugs doesn't improve your performance in a continuous fashion; you can fix a "little" bug and immediately go from getting everything wrong to everything right.  You can't really describe the accuracy of a buggy program by the percent of questions it gets right; if you ask it to do something different, it could suddenly go from 99% right to 0% right.  You can only define its behavior by isolating what the bug does.

Often, I think mistakes are more like bugs than errors.  My clinkers weren't random; they were in specific places, because I had sub-optimal fingerings in those places.  A kid who gets arithmetic questions wrong usually isn't getting them wrong at random; there's something missing in their understanding, like not getting the difference between multiplication and addition.  Working generically "harder" doesn't fix bugs (though fixing bugs does require work). 

Once you start to think of mistakes as deterministic rather than random, as caused by "bugs" (incorrect understanding or incorrect procedures) rather than random inaccuracy, a curious thing happens.

You stop thinking of people as "stupid."

Tags like "stupid," "bad at ____", "sloppy," and so on, are ways of saying "You're performing badly and I don't know why."  Once you move it to "you're performing badly because you have the wrong fingerings," or "you're performing badly because you don't understand what a limit is," it's no longer a vague personal failing but a causal necessity.  Anyone who never understood limits will flunk calculus.  It's not you, it's the bug.

This also applies to "lazy."  Lazy just means "you're not meeting your obligations and I don't know why."  If it turns out that you've been missing appointments because you don't keep a calendar, then you're not intrinsically "lazy," you were just executing the wrong procedure.  And suddenly you stop wanting to call the person "lazy" when it makes more sense to say they need organizational tools.

"Lazy" and "stupid" and "bad at ____" are terms about the map, not the territory.  Once you understand what causes mistakes, those terms are far less informative than actually describing what's happening. 

These days, learning disabilities are far more highly diagnosed than they used to be. And sometimes I hear the complaint about rich parents, "Suddenly if your kid's getting B's, you have to believe it's a learning disability.  Nobody can accept that their kid is just plain mediocre.  Are there no stupid people left?"  And maybe there's something to the notion that the kid who used to be just "stupid" or "not a great student" is now often labeled "learning disabled." But I want to complicate that a little bit.

Thing is, I've worked with learning disabled kids.  There were kids who had trouble reading, kids who had trouble with math, kids with poor fine motor skills, ADD and autistic kids, you name it.  And these were mostly pretty mild disabilities.  These were the kids who, in decades past, might just have been C students, but whose anxious modern-day parents were sending them to special programs for the learning disabled. 

But what we did with them was nothing especially mysterious or medical.  We just focused, carefully and non-judgmentally, on improving their areas of weakness.  The dyslexics got reading practice.  The math-disabled got worksheets and blocks to count.  Hyperactive kids were taught to ask themselves "How's my motor running today?" and be mindful of their own energy levels and behavior.  The only difference between us and a "regular" school is that when someone was struggling, we tried to figure out why she was struggling and fix the underlying problem, instead of slapping her a bad report card and leaving it at that.

And I have to wonder: is that "special education" or is it just education?

Maybe nobody's actually stupid.  Maybe the distinction between "He's got a learning disability" and "He's just lousy at math" is a false one.  Maybe everybody should think of themselves as having learning disabilities, in the sense that our areas of weakness need to be acknowledged, investigated, paid special attention, and debugged.

This is part of why I think tools like Knewton, while they can be more effective than typical classroom instruction, aren't the whole story.  The data they gather (at least so far) is statistical: how many questions did you get right, in which subjects, with what learning curve over time?  That's important.  It allows them to do things that classroom teachers can't always do, like estimate when it's optimal to review old material to minimize forgetting.  But it's still designed on the error model. It's not approaching the most important job of teachers, which is to figure out why you're getting things wrong -- what conceptual misunderstanding, or what bad study habit, is behind your problems.  (Sometimes that can be a very hard and interesting problem.  For example: one teacher over many years figured out that the grammar of Black English was causing her students to make conceptual errors in math.)

As a matter of self-improvement, I think it can make sense not to think in terms of "getting better" ("better at piano", "better at math," "better at organizing my time").  How are you going to get better until you figure out what's wrong with what you're already doing?  It's really more an exploratory process -- where is the bug, and what can be done to dislodge it?  Dislodging bugs doesn't look like competition, and sometimes it doesn't even look like work.  Mr. Cohn was gentle and playful -- he wasn't trying to get me to "work harder," but to relax enough to change the mistaken patterns I'd drilled into myself. 



Comments

dogofjustice
May. 29th, 2012 07:11 pm (UTC)
I think of statistical knowledge as one of the lowest forms of knowledge, what you use to describe a phenomenon when you do not have or cannot trust more specific insights. A Gaussian error is often just a sum of a moderate number of specific effects, most or all of which can be comprehended. Peter Thiel recently made some related observations in his Startups class.

Thinking of my mistakes as bugs has been second nature for me for as long as I can remember; it did not occur to me that this was unusual. I wonder how common this actually is. "Maybe nobody's actually stupid" strikes me as literally false but usefully provocative; a teacher should calibrate the speed of teaching to the student's abilities--so that systematic debugging is possible--and then work with the student on the actual debugging. Ideally, the student learns to and gets in the habit of debugging their own mistakes most of the time.

I wonder about practical considerations. I've seen a lot of pushback against the "small class sizes = good" conventional wisdom, but this approach does seem to require a small class size, and good teachers of course.

Edit: The debugging model has wide application, but there are quite a few educational problems outside its scope. E.g. if someone is performing poorly in a foreign language class, a teacher might be able to suggest a few study habit adjustments, but there's a significant amount of raw memorization that has to occur, and it's often rational for the student to spend that time on something else even if the foreign language class is required. It's also only partially applicable to creative endeavors like English composition and arguably mathematical proofs. Still, it's a big step forward from what we have now.

Edited at 2012-05-30 04:19 am (UTC)
cavyherd
Feb. 12th, 2014 05:06 pm (UTC)
Well, except. Where is the memorization happening?

If I'm learning Russian, and I have to conjugate a verb to myself silently to get the right person and tense, it takes a hell of a long time to get to the right construction.

If, on the other hand, I play with speaking Russian a lot, read a lot, listen a lot, I eventually develop a feel for how those persons and tenses play in specific contexts.

Rote memorization tends to be verbal/auditory. Being able to speak fluently tends to be more verbal/kinesthetic.

Profile

celandine13
celandine13

Latest Month

July 2013
S M T W T F S
 123456
78910111213
14151617181920
21222324252627
28293031   

Page Summary

Powered by LiveJournal.com
Designed by Tiffany Chow