Uncertainty and Probability

Uncertainty

If one has only read this blog, there hasn't been a ton of information provided but that information, on the surface, seems fairly concrete and straightforward. In the initial post, we confirmed the malignancy. In subsequent posts, you were told Dad was undergoing more tests to determine things like the actual size and whether it had spread to local lymph nodes and/or other areas within the stomach. In the previous post, you were told that it, thankfully, has not spread. He now has a chemo regimen which does to his body what chemo usually does to people's bodies.

What is not apparent from the blog is the amount of uncertainty infused in every step of the whole process, haunting every turn, graying any edge of black and white we may hope for.

Cancer, like most unknowns, is a state of affairs we want to get a better picture of and so we administer tests. The only “known” is that something is wrong. Specifically, there was difficulty eating and perceived anemia. The cause of that, however, remained hidden from view. Every doctor’s consultation and every test then became a means to determine the cause of those issues, to unearth the culprit and detail it in all its complexity. We rule out certain things, assess the possibility of other things, etc. etc., all to gain a clearer picture of a very murky reality.

Dad has undergone many tests by now and, while we do know it’s malignant and are fairly certain it’s stomach cancer and hasn’t spread, we are not certain. And for various reasons, the stage is left undetermined. The family, and those close few who have rode the ups and downs with us over the past couple of weeks, and most poignantly Dad, have found this surprising/confusing/annoying. Why do tests not give us the answer? And when they give us answers, why are the answers never clear-cut? What is the point of a test if it can’t tell us what is going on?

Those readers affected by cancer or other medical unknowns are most likely familiar with this awful combination of momentousness and uncertainty.

A more formal way to frame all of this is the following: 1) we believe there to be a state of affairs that is definite, i.e. there is or is not cancer and that cancer is in some particular stage of growth and 2) we administer the testing of various hypotheses to determine the exact nature of that state of affairs to then know how to adjust our actions accordingly to then treat the issue.

But everything that has happened over the past few weeks does not seem to make sense given 1 and 2. After all the tests, we only know it’s malignant and it’s obstructing the junction of the esophagus and stomach. That’s about it. Why did it take 6 tests to determine the kind of cancer and whether it has spread? Why are they still uncertain about the lymph nodes? And connecting all of this, why won’t they stage the cancer?

Staging

Probability is a subset of mathematics that let’s us make better decisions when the given state of affairs or a future state of affairs is unknown; it is a formal methodology for discussing things that are likely or probable, or unlikely or improbable. In matters of uncertainty, it helps us approximate.

Cancer is certainly a state of affairs that involves unknowns and as such probability helps us think about it. Survival rates are given in terms of probability to help us think about the future state of affairs resulting from cancer. Thinking about survival in terms of probabilities, or rates, is probably more intuitive than thinking about whether the current state of affairs is one involving malignant cancer or not. How could it be that we can only ever assign a probability to the reality of cancer? Either there is cancer or there isn’t! Either it is stomach or it isn’t! Either it’s malignant or it isn’t! And yet, that is what happens. At a certain point, a Dr. takes all the relevant data from tests and consultations, and makes a judgment about the cancer and its stage, only with a particular degree of certainty.

One thing we, and our friends and family, want to know is the stage of the cancer, or how far along the cancer is. But the doctors have been reluctant to classify the stage, even after all these tests. We’ve determined malignancy but we want to know the stage of the cancer because staging the cancer gives us a good idea of the survival rates. But what are stages? Cancer growth isn’t like the half-life of a radioactive isotope which decays with a mathematically determinable consistency. We know cancer grows and we more or less know how it grows but we have not developed a method that lets us accurately determine the rate of that growth down to the individual case.

In determining the stage, or how far along the cancer is, there are 3 variables: the main tumor, the spread to lymph nodes, and the spread to other areas within the body. These variables can be assessed along 9, 6, and 2 values respectively. Once these variables are determined, they are combined to determine the stage, from 0 to 4. But those stages have sub-stages, and in most cases, no particular determination of the individual variables necessitates one stage being determined over another. In other words, whether it has spread to lymph nodes, for example, does not mean it is necessarily stage 1B vs. stage 2A or even 2B. Further, it is also the case that, still thinking about the lymph nodes, whether it has or has not spread does not limit it to one stage. So, one could still have stage 2B cancer regardless of the determination of the lymph node variable, so long as other variables have been assessed at specific values.

All of this is a long way of saying staging is a way to think about the progress of the cancer, but it is messy. We can test along the values of the variables, and we can settle on a stage, but even if we’ve settled on the determination of a particular stage, it is never done so with a great deal of certainty.

Staging is usually thought of as buckets the particular case of cancer is or is not in (stage 1, 2, 3, or 4). But as we’ve seen above, it is much more accurate to think of the growth of cancer on a continuous scale from beginning to late stage. Another way to say this is that our categories (4 stages) do not cleanly map the reality of the growth progress of cancer.


Staging Methodology in Light of Probability

Now I want to talk about probability as a way to help us understand the methodology grounding the judgments involved in assessing the various determinations relevant to cancer. I showed above how messy all the determinations are in the case of stomach cancer, but now I want to put that aside and talk simply about the methodology employed in making judgments in situations involving messy determinations.

In determining when something is X or not, we collect as much data as possible and then determine whether X is the case or not. But usually, collecting data only gets us most of the way there. We conduct tests to measure values of the relevant variables but those only work to better or worse degrees and usually don’t provide conclusive results. And so, after collecting the data, we end up at a point where we say, within a statistical framework at least, we believe X to be the case, with a P-value of .05. Here, P-value can be thought of as the likelihood you’re wrong, or that the test was a false positive. In other words, we could say we are 95% sure X is the case. We are forced to say this because, in this case, all our data, combined with the testing framework employed, only gets us close to knowing whether X is the case or not. Of course, being 95% sure of something is a good place to be. Most of us would bet a lot of money if we were 95% likely to win. But it’s not 100%.

It turns out, a lot of things are like the example above. We measure values and acquire data, but we can only be certain up to a degree. Cancer is paradigmatic of this. Tests can produce false positives. So when doctors are trying to determine what stage a given instance of cancer is, they only ever arrive at those determinations with particular levels of certainty. And let’s keep in mind that staging, at least in the case of stomach cancer, is really the result of a combination of 3 variables, which have up to 98 possible combinations. 

The likelihood of a coin coming up heads are 1/2. The likelihood of a coin coming up heads two times in a row are 1/4. So the odds that the tests have gotten the value of each variable correct are small, not only because the individual tests performed only get us an approximate value for a given variable, but also because we are compounding the judgments of multiple tests of multiple variables. Moreover, as we saw above, almost no one value for a particular variable restricts the staging, and the values of the particular variables are only loosely correlated/somewhat independent. 

So if we think back to that continuous scale I mentioned above, suppose the 4 stages are buckets grouping final values from 1-100 on that scale, i.e. values 1-25 are for stage 1, 25-50, for stage 2, etc. Now, if all the values of the individual variables result in a final value of 15, then, in this hypothetical methodology, we feel very confident it’s in stage 1. But, if the final values total 24, we can still say it’s stage 1, but we leave open the possibility it’s actually stage 2, our tests just didn’t give us enough data to accurately assess that to be the case. And I’d hope by now you also understand that me even talking about the staging of cancer as a bucketing of a continuous scale is really just a heuristic and that the distinctions between “stages” are never as clear-cut as going from values of 25 to 26, if only for the fact that the values we’ve arrived at are only believed to be the case with greater or lesser degrees of certainty. In other words, the variables used to help determine the stage have a lot of grey, and the boundaries between the stages themselves have a lot of grey.


Bayesian Probability as a Way to Understand Cancer Testing

My explication above of staging and the methodology that allows a stage to be determined has shown that staging is complicated but the question still remains: why are we testing if the tests can’t help us actually stage the cancer? What are the tests doing if not that? Our common sense thinking about probability leaves us in the pickle of not being able to reconcile 1 and 2 from above; there is a definite state of affairs, and we test to determine that state of affairs, but those tests are not giving us clear answers.

Bayesian probability is another way to think about drawing conclusions in statistical matters. Common sense thinking would say we are testing for X or not X and so a test is either going to confirm or reject X. Bayesian methodology, on the other hand, assigns an initial probability to X and the result of a given test is read in light of that initial probability.

Suppose you came off the street and took a test for cancer and the test came back positive. Also suppose that particular test produces a false positive at a rate of 1 out of every 1000 tests and only 1 in 1000 people have that particular cancer. In this case, WHEN a test happens to say you have cancer, it’s actually just as likely that the test was a false positive as it is that you have cancer. That is because you took a test that has a 1 in 1000 chance of producing a false positive and it was also the case in this situation that 1 in 1000 people have this sort of cancer. But this is why PRIOR information helps us read test results. In a Bayesian framework, if you’re a totally random person coming off the street, with no prior symptoms or history associated with that cancer, as would be in the case above in this paragraph, and the test comes back positive, you really have a 50/50 chance of it accurately diagnosing cancer versus being a false positive, WHEN the test is positive. But in a traditional framework, the 1 in a 1000 chance of having cancer is not factored in when thinking about the fact that the test accurately picks out cancer 999 times out of a 1000.

But of course, most of us don’t take test for cancer randomly. There are usually some symptoms that lead us and doctors to decide to head down the gauntlet of tests required to determine the existence of, and then nature of, the cancer. In my Dad’s case, he had been anemic for awhile, with iron supplements providing little help, and then began to have difficulty in eating.

So why all the tests? Well, tests, once performed, provide us with a better prior probability to then read the next test. And here, this is fairly common sense. If you have 10 tests all saying you don’t have cancer, and the 11th test says you have cancer, you might as well throw it out because it is so much more likely the 11th test is wrong than that the 10 tests are wrong. But Bayesian thinking provides a rigorous framework to reach that conclusion.

Traditional probability would say the same thing but it would be much less certain because it always assumes a purely blind state of affairs. So in that case, you always have a 1 in 1000 chance that the test was a false positive because in traditional probability, you’re always in a random universe. The Bayesian framework does not have a random universe; it always assumes the field is slanted one way. In a Bayesian framework, you could throw the 11th test out even though it’s only a 1 in a 1000 chance it was a false positive. Again, it is because your prior probability is so strong, that the 1 in a 1000 of a false positive is much more likely than the odds that your prior probability is wrong. On the other hand, traditional probability would weigh the initial 10 tests against the fact that a positive test is correct 999 times out of 1000.

It turns out, to spite all this abstract stuff above, this is exactly the framework one doctor used to talk about the potential of a particular test coming back positive. The “final” test Dad underwent down here in Tampa was two parts: 1) a scope of the tumor and stomach, and 2) a sampling of the stomach that’d go off to the lab to see if the tumor was emitting entities in an attempt to spread.

After the test was performed, the doctor said the scope looked so good (there was no sign the tumor was emitting entities), she didn’t need to perform the sampling. And then, and this was the shocking part, she said: “and I wouldn’t know what to do if the sampling came back positive.” Common sense probability would say: What?! If a test comes back positive for the tumor attempting to spread, and it’s extremely unlikely that that test produces a false positive, wouldn’t a positive result for that test be good evidence that it is in fact emitting?!? 

But under a Bayesian framework, her thinking was on solid ground. She was saying, in the very unlikely chance that the test would come back positive, it wouldn’t actually be good evidence that it is attempting to spread, GIVEN all the other evidence she had, and regardless of how unlikely a false positive would be.

And I’d like to point out that this isn’t some abstract thought experience in a text book, or some line of thought without consequence. Her approach led to the very real real-world result of not performing that 2nd part of the test.

And this is where we see the power of probability.

We are limited beings, both in space and in time. Some things are certain, others less so. Where we would like to see black and white, we often are left with grey. Empirical work, gathering evidence, is one way to break down some of those limitations and define a few more of those edges. But even gathering evidence can only go so far and so we use statistics to push those limits out even further, if ever so slightly, and to maybe even remove a bit of the grey in the world, if only occasionally.

Dad started Chemo earlier this week. The stage of his cancer is still unknown.