Friday, July 5, 2013

On Being a Sample Size of One

I recently finished reading a statistics “textbook” for fun (“textbook” is in quotes because it was one of those “make learning enjoyable” books, thus diminishing the nerd factor of the achievement). I had always wanted to take a stat course before. Expectation, variance, binomial and Poisson distributions ... I thought it was all pretty cool stuff.

One thing you learn when studying statistics: How to quantify the probability that an observed outcome for a trial/study/experiment is significant. In other words, is it likely that some causal agent A led to some outcome B, or were the study’s results probably due to chance?

One thing that helps in making this determination: sample size. The larger, the better.

Which brings us to my personal knee experiment. What does it mean to be a sample size of one?

Well, from a professional statistician’s viewpoint: not much. A sample size of one is a joke. Where are your control subjects? How could you possibly calculate, say, margin of error?

So what does it mean to say that, in a tightly controlled experiment, I improved the state of my knees by doing x and y, when I was the only subject in the experiment?

Again, statistically: not much.

But there’s another way to look at this.

And that’s by considering the likelihood that a given experiment can actually prove cause and effect.

See, it doesn’t matter if you have 100,000 subjects in a study, if that study is poorly designed and/or poorly executed, making it impossible to isolate cause and effect with a high degree of confidence. The study’s researchers may think that they have reached a conclusion that A leads to B (or doesn’t lead to B), but this is a dangerous sort of illusion if the study is fundamentally flawed. Long-term knee studies of the type, “Activity X is good/bad for knees” are especially prone to this problem. In Saving My Knees, I take a close look at one.

I won’t retrace that ground here. But consider an example to help illustrate what I mean: Suppose you run a one-year study in Smallville to see if daily swimming helps really bad knees. You believe that to be true, but at the end of the trial, you’re puzzled to find the swimming group actually does worse than the control group -- in a statistically significant way too. You publish the results.

“Swimming adversely affects knee joint health,” you proclaim.

But -- what if you discover the Smallville swimming pool is on the sixth floor of a building that has no elevator? And for the subjects in the study, who have really bad knees, climbing all those steps is too much for their joints?

This is the kind of problem you face in a study when important variables aren’t tightly controlled. Granted, my example is a bit extreme and far-fetched, but what’s undeniable is that, in studies that attempt to show “Activity X helps/hurts bad knees,” for about 98 percent of the time (during waking hours), the knees of the subjects will NOT be engaged in Activity X. And what they’re doing during that 98 percent of the time -- whether it’s ill advisedly climbing six flights of stairs or kneeling to scrub a floor or running to catch a public bus -- can be vastly significant and skew the results in a big way.

There are other flaws of long-term knee studies that attempt to show causal relationships, which are independent of sample size. In Chapter 8 of Saving My Knees, I mention some more.

My experiment, on the other hand, was closely observed and very well-controlled when compared to a typical study into what helps/hurts knees. Now you can argue that doesn’t matter -- that I have the constitution of a space alien, or that my patellofemoral pain syndrome was wholly unlike anyone else’s, so what I did to cure my bad knees won’t help others.

Of course, I would disagree. Further, I’d argue that sometimes you can learn a lot more from a one-person experiment that’s very well-conducted than a multi-thousand subject trial that isn’t.

So maybe it’s not so bad after all being a sample size of one.


  1. I did statistics at uni (forgotten it all now), but I'll take N=1 as it gives me hope.

    Have been re-reading bits of your book Richard while on hols in Italy (not a knee friendly place - steep streets & lots of steps).

    I was under the impression your cartilage damage was on the back of your kneecaps (which I've read/been told has more likelihood of healing), but the re-read suggested it was the cartilage on the ends of your femur/tibia (and/or perhaps the meniscus between the bones) which many medicos seem to think has limited healing capacity. Which do you think it was?

    cheers, TriAgain

  2. I thought my main problem was the cartilage under the kneecap. However, I took a look at my MRI scan results which say "slight cartilage irregularity is seen over the patella, postero-inferior part of the medial femoral condyle and the posterior weight bearing part of the medial femoral and tibial condyles." So it appears the cartilage was at least somewhat irregular in multiple places.

    Regarding which stands the best chance of healing: Not sure where you read that it's the patellar region. Changhai Ding et al's "Natural History of Knee Cartilage Defects and Factors Affecting Change" shows 27% and 26% of subjects had decreases in cartilage defect scores for the medial tibiofemoral and lateral tibiofemoral compartments respectively. That figure was only 13% however for the patellar compartment.

    1. Interesting Richard. My MRIs showed significant chondromalacia on the back of he right kneecap & some damage to the femur cartilage. But when the O/S went into remove the torn piece of meniscus in my left knee, he said everything else looked really good. But shortly after both knees succumbed to PFPS (or whatever it might be)

      So it seems something can trigger pain beyond what the physical damage indicates should occur. This really confuses the medicos.

      Regarding the patella region having the best chance of cartilage recovery, I was told that by a Sports Dr who wanted to inject me with PRP, and who had his own chondromalacia successfully treated by having holes drilled in the back of his kneecap. I declined, it seemed too radical at this time.

      cheers, TriAgain

  3. Cool thought. Also there is a baseball player (Chase Utley) who had chondromalacia patella in both knees. It kept him out for of the past 2 seasons but this year he is back full time. He stated in spring training press conference "I think my knee problems are behind me". He didn't do surgery. I'm unsure of what he did, but maybe like you he "saved his knees".