Peter, thanks again for your posts on this. I'd like to use the COVID example you provided to specify my concerns about Carrier's Rank-Raglan class.
Before responding I'd like to highlight these points:
1. I've read a few times that Bayes Theorem is a reflection on how we should think normally anyway, and it is a formalization of that kind of thinking. Intuitively, the use of the RR class just doesn't sit right for me, as I'll explain below. It may mean I just don't really understand it, of course.
2. I think Gregor/Hanson make a good case that Carrier's RR class actually provides a prior probability for historicity of Jesus being over 90%, so from that perspective I wouldn't mind being wrong!
But those are firm figures. Look at the RR class, with its sampling of heroes who match the criterion of 10 hit points out of 22. Say it is 1 historical hero in 15, so about 7%. That's fine, but the criterion is arbitrary. Imagine adjusting the criterion to 1 hit point and getting a prior, then to 2, and so on all the way up to 22. Some wouldn't provide enough heroes for a reference class so could be excluded, but for the ones that do produce a result the prior would change. That doesn't sit right with me. If I can choose the criterion number that gives the result I prefer, then that doesn't seem Bayesian.
Of course, a probability can appear in the prior or posterior. If it doesn't go into the prior then it goes into the posterior. I don't see how that can be done with RR class. If I can choose a criterion of 9 and that doubles the chance of historicity, then that doubles the overall result (including prior and posterior). Changing the criterion of 10 to 9 should then be reflected somehow in changing the posterior calculation if I understand Bayes correctly. I don't see how it does.
I've been trying to think of a similar scenario with regards to your COVID example. Perhaps something like redefining a positive COVID infection as anything between a cough and lung infection, up to being put onto a ventilator? I'm not sure.
I know that the prior probability can be subjective, and certainly Carrier's criterion that the RR hero has to make at least 10 points out of 22 in order to to included in the reference class is subjective. But then are we looking at the same things when examining the posterior calculations? It would be like saying a positive COVID result means one thing for the prior (say, getting a doctor's certificate to show one has COVID) and another thing for posterior (say, a positive result on a RATS text).
I hope I'm making sense here. To me, the frequency of the RR class as prior - with its criterion of 10 - is just a number that relates to RR heroes. It has nothing to do with the posterior calculations, and shouldn't be combined with them.
Before responding I'd like to highlight these points:
1. I've read a few times that Bayes Theorem is a reflection on how we should think normally anyway, and it is a formalization of that kind of thinking. Intuitively, the use of the RR class just doesn't sit right for me, as I'll explain below. It may mean I just don't really understand it, of course.
2. I think Gregor/Hanson make a good case that Carrier's RR class actually provides a prior probability for historicity of Jesus being over 90%, so from that perspective I wouldn't mind being wrong!
Okay, I agree with all that. But I wonder how the COVID example maps to the example of the RR class? The prior probability is clear: one million samples, 20000 with COVID, so 2% for the prior. And if the number with COVID is much less, then the prior decreases.Consider a simple example of the application of a Bayesian framework: does the patient have the Covid virus? Presumably either the patient has Covid or does not have Covid. However, we can't observe the truth of either statement directly and with apodeictic certitude. We would not be wise to pretend to a certainty or to a kind of direct access to the state of the world that we do not have, simply because of the logical truism of A or not-A here. The use of credences allows modeling how likely the sorts of evidence we have or don't have are believed to imply that a hypothesis is true. A credence refers to an expression of how likely some hypothesis is, on the basis of the available evidence, and that doesn't mean it has to be purely subjective. Consider this example:
https://www.redjournal.org/article/S036 ... 3256-9/pdfWe can further illustrate how Bayes’ theorem and the use of prior information can help to answer important questions using the example of COVID-19 testing. For this example, let’s assume that a test for COVID-19 infection is guaranteed (100% chance) to detect the COVID-19 virus in someone who has the infection (P [test+j virus] = 1.0) and has a 99.9% chance of correctly identifying that someone does not have the virus (or a 0.1% chance of a false positive: P[test- j no virus] = 0.999 and P[test + j no virus] = 0.001).9 That sounds like a high probability, but what we are really interested in is if you have a positive test, how likely is it that you actually have the virus (P[virus j test+]). We can calculate this probability using Bayes’ theorem. As discussed, this probability depends on some prior information—how likely you were to have the virus (ie, P[virus]) before taking the test.
If we assume the prior probability is the prevalence of the virus in the population at the height of the pandemic, 2%, then P(virus) = 0.02. Out of 1 million people, 20,000 will have the virus and 980,000 will not. If we test them all, 20,000 will have a true positive test, 979,020 will have a true negative test, and 980 will have a false positive test (980,000 £ 0.001). Thus, there would be 20,980 positive tests in total (P[test+] = 20,980/1,000,000 = 0.02098), and the chance of having the virus given a positive test could be calculated using Equation 2: P(virus j test +) = (1.0 £ 0.02)/ 0.02098 = 0.953, which can be approximated as 95%. In this setting, if the test is positive, one should believe the test.
However, what if the prevalence of COVID-19 was estimated to be much less, say, 0.2% (P[virus] = 0.002), such as during a period of a governmental stay-at-home order? In this scenario, when we test 1 million people, 2000 will have a true positive test, 997,002 will have a true negative test, and 998 will have a false positive test (998,000 £ 0.001), giving 2998 positive test results, and P(test+) = 2988/1,000,000 = 0.002988. The probability of having the virus after a positive test now becomes P(virus j test +) = (1.0 £ 0.002)/0.002988 = 0.669. In this scenario, one may truly question whether a positive test is diagnostic of infection, because the likelihood of a false positive is approximately 1 in 3.
Therefore, knowledge of the prior information (in this case, the prevalence of COVID-19 in a population) can alter the chance of having a virus when a test is positive from 95% to 67% when the sensitivity and specificity of the test remain unaltered. On the other hand, if we were testing hospitalized patients, for example, the prevalence would be expected to be much higher, and there would be a lower chance of obtaining a false positive result. This simple calculation highlights the importance of taking into account prior information, something a frequentist analysis does not do.
The reasoning is Bayesian: the probability describes a hypothesis, and the evidence is the test, which updates the prior probability to a posterior probability. The strength of the evidence is measured by the specificity of the test, which is itself based (in an objective i.e. data-determined way) on previous observations on how frequently the test returns a positive result for the infected and how frequently the test returns a negative result for the uninfected. The prior probability corresponds to an estimated frequency of occurrence for the population from which the patient is drawn: whether that is randomly from the public or from those that are hospitalized.
At no point does the Bayesian approach nudge up the credence from 95% (or 67% or whatever) to 100% just because of the point that the person either has covid or they don't. The credences of the hypothesis that the patient has covid and the hypothesis that the patient does not have covid sum up to 1, which is what is required for coherence. Neither has to be 1 by itself. Under a probabilistic conception, credences refer to a probability space where the propositions describe different possibilities regarding the state of the world. These are understood as possibilities given the limitations of uncertain knowledge. The probabilities represent how likely it is believed that each possibility represents the actual state of the world. A probability of 1 is just a special case of a credence where it is believed that all mutually exclusive possibilities have an infinitesimal likelihood or are indeed impossible.
But those are firm figures. Look at the RR class, with its sampling of heroes who match the criterion of 10 hit points out of 22. Say it is 1 historical hero in 15, so about 7%. That's fine, but the criterion is arbitrary. Imagine adjusting the criterion to 1 hit point and getting a prior, then to 2, and so on all the way up to 22. Some wouldn't provide enough heroes for a reference class so could be excluded, but for the ones that do produce a result the prior would change. That doesn't sit right with me. If I can choose the criterion number that gives the result I prefer, then that doesn't seem Bayesian.
Of course, a probability can appear in the prior or posterior. If it doesn't go into the prior then it goes into the posterior. I don't see how that can be done with RR class. If I can choose a criterion of 9 and that doubles the chance of historicity, then that doubles the overall result (including prior and posterior). Changing the criterion of 10 to 9 should then be reflected somehow in changing the posterior calculation if I understand Bayes correctly. I don't see how it does.
I've been trying to think of a similar scenario with regards to your COVID example. Perhaps something like redefining a positive COVID infection as anything between a cough and lung infection, up to being put onto a ventilator? I'm not sure.
I know that the prior probability can be subjective, and certainly Carrier's criterion that the RR hero has to make at least 10 points out of 22 in order to to included in the reference class is subjective. But then are we looking at the same things when examining the posterior calculations? It would be like saying a positive COVID result means one thing for the prior (say, getting a doctor's certificate to show one has COVID) and another thing for posterior (say, a positive result on a RATS text).
I hope I'm making sense here. To me, the frequency of the RR class as prior - with its criterion of 10 - is just a number that relates to RR heroes. It has nothing to do with the posterior calculations, and shouldn't be combined with them.
Statistics: Posted by GakuseiDon — Sun Dec 29, 2024 12:51 am