We are already familiar with Bayes, and the theorem that, simply put, expresses the probability that a hypothesis is true as fitness with evidence weighted by its prior probability.
Science is analogous to Bayesian inference, in which both our a priori confidence in a theory and how well it fits new data influence our belief in it. For example, if we have three theories about the bias of a coin (either a 0.6, 0.5, or a 0.4 bias for heads) and we observe five heads in sequence, then Bayes' theorem tells us how we optimally revise our certainty that the coin has a 0.6 head bias.
Importantly, when we lack a clear idea of possible explanations (of the other side of the filter) and try to apply Bayes’ theorem to our reasoning, we are faced with the fact that there is no such thing as an unbiased prior. There is a more sophisticated mathematical reason for this, but we can see it more intuitively in theological arguments for the existence of God.
I was once told by a creationist: “If you were to shake a handful of sand and throw it all up in the air, and if it is all mindlessly random – as you say – then obviously it would be improbable to the point of impossible for it to fall by pure happenstance into something as well-organized as an organism?” The reasoning behind this watchmaker-argument goes that “Given that there is no god, the structure we see would be improbable. Therefore, because the world is so stunningly assembled, we must conclude that there is a conscious God”. Here, the observation that we do exist is our data. Because we have no priors, we will have to invoke equiprobable priors, such that “God exists” and “God does not exist” having 50% each. As for likelihoods, a creationist would say that P(humans|no God) has a vanishingly low probability, like a one-trillionth and P(humans|God) is significantly higher, maybe a millionth.
Theological arguments that are covertly Bayesian depend on what hypotheses they entertain in their prior. Because of the degree of arbitrariness in this selection, quantitative reasoning may in this case be meaningless.
The argument may be problematic because when we assigned 50% each to God/no God, we partitioned the vast space of possible hypotheses in an inevitably biased way. We could fragment “no god” into an indefinite number of alternative hypotheses, such as “Hindu gods”, or that we are a quantum computer simulation, which would be compatible with an orderly universe too. To this type of fundamental questions, in which we have no good conception of the possibility space, probability theory cannot be meaningfully applied.
In philosophy of science, the fact that the hypothesis space is never exhaustively specified is known as “under-determination of theory by data”. This refers to the idea that we can never know whether another theory would account for the evidence equally well. Hence, science cannot be said to constitute truth. This is consistent with Bayesian reasoning, where only hypotheses deemed worthy a priori are investigated. Yes, a demon tampering with your mind could explain your current sensory experience, but this prior probability is so low it becomes negligible.
Interestingly, a similar argument has been advanced by philosopher Hillary Putnam in defense of science, called the “No Miracles” argument. Science’s successes are improbably due to luck, and therefore science must deal in truth. The argument is a conditional probability, where P(science’s success| science is unrelated to truth) is considered low. Bayes’ theorem tells us that this is meaningless unless we consider the base rate, which is the relative frequency of false theories, but again, this incidence cannot be quantified. However, if an evolutionary view of science is taken, the “No Miracles” argument amounts to “Survival of the fittest”. Science is successful, precisely because hypotheses that do not perform well are eliminated.