Log in

No account? Create an account

Previous Entry | Next Entry

Errors vs. Bugs and the End of Stupidity

"A pianist has to believe in telekinesis.  You have to believe you have the power to move your fingers with your mind."

I learned that from Phil Cohn, my piano teacher's piano teacher.  Once in a while, when I was in high school, she'd arrange for me to have a master class with him.  He was a diminutive man who looked exactly like Dr. Strangelove, and had a gentle way of guiding your hands and body while you played.  He was very interested in the physicality of piano; he liked to tell stories about students of his who could play any piece upside down, or cross-handed, or one-fingered like Chico Marx.  He had a lot of theories about the process by which we can learn to control our muscle movements.

I wasn't an exceptional pianist, and when I'd play my nocturne for him, there would be a few clinkers.  I apologized -- I was embarrassed to be wasting his time.  But he never seem to judge me for my mistakes.  Instead, he'd try to fix them with me: repeating a three-note phrase, differently each time, trying to get me to unlearn a hand position or habitual movement pattern that was systematically sending my fingers to wrong notes.

I had never thought about wrong notes that way.  I had thought that wrong notes came from being "bad at piano" or "not practicing hard enough," and if you practiced harder the clinkers would go away.  But that's a myth.

In fact, wrong notes always have a cause. An immediate physical cause.   Just before you play a wrong note, your fingers were in a position that made that wrong note inevitable. Fixing wrong notes isn't about "practicing harder" but about trying to unkink those systematically error-causing fingerings and hand motions.  That's where the "telekinesis" comes in: pretending you can move your fingers with your mind is a kind of mindfulness meditation that can make it easier to unlearn the calcified patterns of movement that cause mistakes.

Remembering that experience, I realized that we really tend to think about mistakes wrong, in the context of music performance but also in the context of academic performance.

A common mental model for performance is what I'll call the "error model."  In the error model, a person's performance of a musical piece (or performance on a test) is a perfect performance plus some random error.  You can literally think of each note, or each answer, as x + c*epsilon_i, where x is the correct note/answer, and epsilon_i is a random variable, iid Gaussian or something.  Better performers have a lower error rate c.  Improvement is a matter of lowering your error rate.  This, or something like it, is the model that underlies school grades and test scores. Your grade is based on the percent you get correct.  Your performance is defined by a single continuous parameter, your accuracy.

But we could also consider the "bug model" of errors.  A person taking a test or playing a piece of music is executing a program, a deterministic procedure.  If your program has a bug, then you'll get a whole class of problems wrong, consistently.  Bugs, unlike error rates, can't be quantified along a single axis as less or more severe.  A bug gets everything that it affects wrong.  And fixing bugs doesn't improve your performance in a continuous fashion; you can fix a "little" bug and immediately go from getting everything wrong to everything right.  You can't really describe the accuracy of a buggy program by the percent of questions it gets right; if you ask it to do something different, it could suddenly go from 99% right to 0% right.  You can only define its behavior by isolating what the bug does.

Often, I think mistakes are more like bugs than errors.  My clinkers weren't random; they were in specific places, because I had sub-optimal fingerings in those places.  A kid who gets arithmetic questions wrong usually isn't getting them wrong at random; there's something missing in their understanding, like not getting the difference between multiplication and addition.  Working generically "harder" doesn't fix bugs (though fixing bugs does require work). 

Once you start to think of mistakes as deterministic rather than random, as caused by "bugs" (incorrect understanding or incorrect procedures) rather than random inaccuracy, a curious thing happens.

You stop thinking of people as "stupid."

Tags like "stupid," "bad at ____", "sloppy," and so on, are ways of saying "You're performing badly and I don't know why."  Once you move it to "you're performing badly because you have the wrong fingerings," or "you're performing badly because you don't understand what a limit is," it's no longer a vague personal failing but a causal necessity.  Anyone who never understood limits will flunk calculus.  It's not you, it's the bug.

This also applies to "lazy."  Lazy just means "you're not meeting your obligations and I don't know why."  If it turns out that you've been missing appointments because you don't keep a calendar, then you're not intrinsically "lazy," you were just executing the wrong procedure.  And suddenly you stop wanting to call the person "lazy" when it makes more sense to say they need organizational tools.

"Lazy" and "stupid" and "bad at ____" are terms about the map, not the territory.  Once you understand what causes mistakes, those terms are far less informative than actually describing what's happening. 

These days, learning disabilities are far more highly diagnosed than they used to be. And sometimes I hear the complaint about rich parents, "Suddenly if your kid's getting B's, you have to believe it's a learning disability.  Nobody can accept that their kid is just plain mediocre.  Are there no stupid people left?"  And maybe there's something to the notion that the kid who used to be just "stupid" or "not a great student" is now often labeled "learning disabled." But I want to complicate that a little bit.

Thing is, I've worked with learning disabled kids.  There were kids who had trouble reading, kids who had trouble with math, kids with poor fine motor skills, ADD and autistic kids, you name it.  And these were mostly pretty mild disabilities.  These were the kids who, in decades past, might just have been C students, but whose anxious modern-day parents were sending them to special programs for the learning disabled. 

But what we did with them was nothing especially mysterious or medical.  We just focused, carefully and non-judgmentally, on improving their areas of weakness.  The dyslexics got reading practice.  The math-disabled got worksheets and blocks to count.  Hyperactive kids were taught to ask themselves "How's my motor running today?" and be mindful of their own energy levels and behavior.  The only difference between us and a "regular" school is that when someone was struggling, we tried to figure out why she was struggling and fix the underlying problem, instead of slapping her a bad report card and leaving it at that.

And I have to wonder: is that "special education" or is it just education?

Maybe nobody's actually stupid.  Maybe the distinction between "He's got a learning disability" and "He's just lousy at math" is a false one.  Maybe everybody should think of themselves as having learning disabilities, in the sense that our areas of weakness need to be acknowledged, investigated, paid special attention, and debugged.

This is part of why I think tools like Knewton, while they can be more effective than typical classroom instruction, aren't the whole story.  The data they gather (at least so far) is statistical: how many questions did you get right, in which subjects, with what learning curve over time?  That's important.  It allows them to do things that classroom teachers can't always do, like estimate when it's optimal to review old material to minimize forgetting.  But it's still designed on the error model. It's not approaching the most important job of teachers, which is to figure out why you're getting things wrong -- what conceptual misunderstanding, or what bad study habit, is behind your problems.  (Sometimes that can be a very hard and interesting problem.  For example: one teacher over many years figured out that the grammar of Black English was causing her students to make conceptual errors in math.)

As a matter of self-improvement, I think it can make sense not to think in terms of "getting better" ("better at piano", "better at math," "better at organizing my time").  How are you going to get better until you figure out what's wrong with what you're already doing?  It's really more an exploratory process -- where is the bug, and what can be done to dislodge it?  Dislodging bugs doesn't look like competition, and sometimes it doesn't even look like work.  Mr. Cohn was gentle and playful -- he wasn't trying to get me to "work harder," but to relax enough to change the mistaken patterns I'd drilled into myself. 


( 38 comments — Leave a comment )
Page 2 of 2
<<[1] [2] >>
Sep. 24th, 2012 07:26 am (UTC)

When giving me a new piece to try, my old drum teacher would make me play it reallllly slowly. He did this because, although he didn't know the name of the phenomenon, he knew that once your muscles and brain 'learn' something it's very difficult to unlearn it. Most of the mistakes I made as a young drum player were due to my muscles "just doing it" even though my brain said other wise, because I'd previously made mistakes whilst rushing through things that then became set in stone.

Lesson: Take it slow and learn it correctly the first time. This applies to everything from drums to politics.
Jorge Cañizales Díaz
Sep. 24th, 2012 01:58 pm (UTC)
Please keep sharing your thoughts.
That was a very very interesting read. I found it because someone submitted it to www.reddit.com/r/programming .
Sep. 24th, 2012 02:50 pm (UTC)
This is a good article, but it would be better if it didn't use "schizophrenia", which is an actual condition that affects some people, as a metaphor for something else. I don't have schizophrenia, but I am bipolar, and I feel othered when people describe themselves as "a little bipolar about bagels" (or whatever) because they like bagels sometimes and dislike them at other times. To them, it's just a metaphor, but to me, it's my life, which they are taking and trivializing for their own point.

I realize that this is part of a quote, but I think you could use the quote while also bracketing the problematic nature of using disabled people's lives as metaphors to prove points that aren't about disability (at least, if you think it's problematic).

I'd like to be able to link this, but I don't feel like I can do so while it begins in an ableist way. Which is too bad, since I think the main point (that it's more useful to think about debugging one's issues than to think of oneself as irreparably "stupid") is an anti-ableist one.
Oct. 3rd, 2012 04:59 pm (UTC)
Ok, good point. It was a distracting word and there's really no reason for me to trivialize real problems over a metaphor.

Oct. 3rd, 2012 05:02 pm (UTC)
Ken Catchpole
Sep. 26th, 2012 09:47 pm (UTC)
Human Error Theory

This is a really great post.

There's a whole science behind why accidents happen that is precisely about this; about understanding what trips you up every day, and rather than "trying harder" working out what goes wrong and building a better way of doing it.

I work with Doctors on precisely this sort of thing, because when things go really wrong, it ends up with the death of the patient and possibly the end of the Doctors career, even though they may have been set up to fail by their training, the equipment they use, or a whole range of other things.

I've just spent the day analysing 1800 very small errors associated with about 90 trauma care patients, looking at trends and common causes. We know that little problems can build to bigger ones, and that telling the Docs to do better won't work - but giving them better information, better communication tools and skills, and new methods to do the same things really does make a difference to saving lives.

A good place to start with the academic side of this is a 1990 book by James Reason called "Human Error".
Keith Albion Borgholthaus
Sep. 27th, 2012 12:09 am (UTC)
A bug in your theory
I got here from one of your friends. I actually enjoy your analysis, however I see a bug. What if something is actually a gift and it is just called a disability?

Most people I know work with what is called the left side of the bell jar. Those who have disabilities and they stop the person from being able to function in society are treated while those with gifts are looked at in funny ways and not really analyzed. I believe that many students with disabilities are actually gifted they just haven't figured out how to use it.

My best example is Sherlock Holmes. He has a habit of looking at everything, and getting bored easily. In our minds this is because his intellect is so high of course it would be bored. The instant I change it to ADHD we view him as having a disability. Before you say that he is fictional Holmes is based on the skills and instincts of an investigator at Scotland yard. Sir Doyle simply added adventure to the character.

So, you mention the people you treat and in the same sentence as autism you mention ADHD. It seems so odd to me. The bug I see is that we are placing some very intelligent people on the left side of the bell curve because we haven't studied the right side well enough.

I enjoyed your article and believe you are very close to figuring out the answer, but I saw this bug and hope you can fix the programming for it. It's missing a bracket or something.
Nov. 21st, 2012 07:19 am (UTC)
I found your LJ through Yvain's. I like your comments on his journal and your entries are pretty interesting. If some of your posts are friends-only, it'd be nice to be added.
Feb. 17th, 2013 05:58 pm (UTC)
Thank you!
This was a wonderful application of systems thinking to education and learning, and the music analogy and how your music teacher thought was a fantastic analogy. Really very good insights. These are the kinds of things that I think apply to nearly every field and every person, and if understood, would help make the world a better place. So by writing this, and by extension hopefully helping some people understand it, thank you for making the world better.
Feb. 12th, 2014 08:58 am (UTC)
As a math professor (who used to take piano lessons, but is now very rusty), I LOVE this entry! I'm tempted to give my students excerpts (with credit, of course). Would that be OK with you?
Feb. 14th, 2014 11:55 am (UTC)
This really hits the nail on the head. Either that, or I'm succumbing to the tendency I frequently criticise of choosing the model that is metaphorically derived from our most advanced technology ;-)

BTW, with regarding to learning music, recorder teacher had an alternative approach. He said to me "You're practising like an amateur, not a professional. An amateur musician plays the piece, then tries to correct their mistakes. A professional musician plays the piece so slowly that there are no mistakes, then gradually increases their speed one BPM at a a time." I suppose that would be not so much debugging as making sure bugs don't creep in in the first place.
R. J. Zimmerman
Nov. 13th, 2014 02:28 pm (UTC)
What a pleasure to read.
Dec. 21st, 2014 07:37 pm (UTC)
I wish I could share this a bazillion times. Thank you.
Page 2 of 2
<<[1] [2] >>
( 38 comments — Leave a comment )