Firstly, selection effects have the potential to resolve much of the disquietude we feel regarding quantum nondeterminism. If a quantum experiment is repeated, such as firing a photon on a screen with two slits it could pass through – then the relative frequencies of different outcomes (left slit versus right slit) can be predicted, but each individual instance appears to be irreducibly governed by chance, with any “hidden variables” cleverly ruled out. The idea is that the particle passes through both slits in a state of “superposition”, described by a mathematical entity called the “wavefunction”, which states the probability of it being found anywhere in the universe. The wavefunction unfolds deterministically according the Schrödinger equation. Then, upon observation (e.g. by turning on a particle detector behind the slits), the Copenhagen interpretation by Bohr and Heisenberg states that the wavefunction “collapses” into a unique state.
However, in 1957 Hugh Everett pointed out that nothing in the mathematics implies a collapse. Schrödinger’s equation could continue to evolve the wavefunction. The universe does not metaphysically “split” into non-interacting branches – the superposition remains as a single wavefunction – but as soon as it transfers information to something else, like an air molecule, it in effect becomes unobservable. By becoming correlated with the environment, it thus “decoheres”, and because the particle superpositions in neurons decohere faster than they fire, we cannot experience parallelism at macro-scales.
To adapt an analogy by physicist Max Tegmark, we could imagine ourselves being unknowingly cloned into ten copies while asleep, with each clone being placed to wake up in a room with a different number on the wall (ranging from 0 to 9). Upon waking up, then subjectively the number 6 on the wall would seem random, but if we had access to the parallel worlds, then finding that each numeral is represented would make it feel deterministic. Similarly in quantum experiments, we only see one out of all the logically conceivable outcomes contained in the wavefunction. While the “Many worlds” interpretation may not yet be empirically testable, it (bizarrely) has the virtue of parsimony, and it has become increasingly respectable.
According to Everett's many-worlds interpretation, quantum randomness appears non-deterministic because we only have access to one of all logically conceivable universes.
Telescoping up to cosmic scales, awareness of selection effects forms one of the key methodological principles of cosmology. The cosmological origin story is that the early Universe was maximally simple, symmetric, and – at least in some places – low in entropy, but that expansion and consequent temperature fall caused these symmetries to break. According to the “inflationary hypothesis”, a special form of matter accelerated the expansion, causing some regions to inflate more than others. Therefore, in a different kind of multiverse theory, there could logically be an infinite number of other universes beyond the visible horizon, and ours just happened to be one that inflated enough to allow sentient, carbon-based life to evolve.
The Weak Anthropic Principle states that when physicists find some property of cosmos that appears fine-tuned to support sapient life, they should regard this not as a marvellous coincidence, but as a necessity since - in the space of possible universes - only universes with these properties would be compatible with the physical himself!
Point is: regardless of how intrinsically improbable it is, we would necessarily find ourselves in a Universe that can support us. In what is known as the “Weak Anthropic Principle”, the observed universe should not be considered as coming from some unconstrained space of possible universes, but from the life-supporting subset thereof. If this principle is neglected, erroneous conclusions will be drawn. For example, Paul Dirac wanted to revise the law of gravitation in light of a coincidence between a constant of Nature and the age of the Universe, but without that coincidence, there would have been no Paul Dirac there that could be preoccupied with such fine tunings!
Selection effects also figure heavily in discussions regarding the eerie nature of mathematics, and as physicist Eugene Wigner said, its “unreasonable effectiveness” in physical predictions. Newtonian physics gives the impression that we live in a universe of perfect spheres and parabolas. There are many examples of mathematical curiosities that have been collecting dust for decades and suddenly find themselves elegantly applied to some newly discovered phenomenon, and high-energy physics portrays the fundamental laws to be astoundingly tidy and integer-laden.
However, the Universe is more than deep symmetries – it would not be fully specified without its initial conditions. Something in their interaction appears to have generated a cosmos of perplexingly rich structure, dynamical systems and nested hierarchies, and unlike the deep symmetries themselves, their outcome is far from mathematically elegant. With the advent of computers and big data, there is a growing appreciation for just how disorderly the universe is. In biology and sociology, attempts at mathematical formalisms are taken as cartoonish simplifications.
Philosopher Reuben Hersh argues that vision of a clockwork-universe is an illusion arising from how we disproportionately focus on phenomena that are amenable to mathematical modelling. The concept of cardinal numbers (“put in 5 balls, then 3 into a container. The prediction is that it will contain 8 balls”) would break down for water drops or gases. The prediction would hold only if humans went on to invent the concept of mass and volume. The tools were selected based on their predictive abilities. As Mark Twain said: “To a man with a hammer, everything looks like a nail.”
The problem is that there is no way for us to quantify all the “phenomena” to calculate what proportion of these are “orderly”. Nor can we conceive of a possibility space of different, logically consistent laws of physics that were not simple integers, to see just probable our universe is. Instead, we are restricted to the illusion selected for us, by quantum decoherence, post-Big Bang inflation, and our own perceptual affinity for mathematical elegance – forever to wonder what’s on the other side of the filter.