14 DAY TRIAL // How is it possible for a mathematical formula to give rise to such passions and excitement? It is even claimed that there are two kinds of people: bayesians and non-bayesians, i.e. the ones who have seen the light of Bayes' formula and the ones who haven't. It is the formula that it is said to describe how learning functions and how scientific progress takes place. It describes how one can feel comfortable knowing that everything one knows is subjective. Well, it is all true! But when someone first hears such stories and maybe wants to get some insight, all that one usually finds is a blunt mathematical statement that doesn't look very revealing nor exciting. Where's all the fuss coming from? I'll try to describe the formula without the formula.
It is a formula about probabilities, subjective probabilities. A subjective probability is a number that describes how plausible a certain statement seems to you. A subjective probability always depends on your current knowledge (or on your prejudices, it doesn't matter how we call it). Something seems more or less plausible to you because you have a certain world view.
Bayes formula tells what happens to subjective probabilities when one receives some new information. You may find some statement S to have certain plausibility. How does this plausibility change when you receive some new information I?
There are several factors.
Firstly, in what conditions does the plausibility change at all: it depends how credible the new information is - if the new information isn't credible at all, it doesn't make any difference. It depends how relevant the new information is for the statement S. Secondly, if the plausibility changes - in what direction does it change and how much does it change: it depends on how surprising the new information is. If the new information is considered relevant and trustworthy and in the same time it is very surprising, i.e. you wouldn't have expected such information, the plausibility of S changes very much. The direction of change depends on whether S implies the information I, i.e. if the new information is something that supports the belief in the truth of S or, on the contrary, it supports the belief in non-S.
Now, these are not four different, independent factors, there are only two factors. The credibility of information I is the same thing as how surprising I is - if something is too surprising, if it doesn't fit one's world view at all, one simply doesn't trust that information. One piece of information isn't trustworthy in itself, one decides whether to consider it reliable - i.e. this trustworthiness of information is subjective. And, finally, whether the information I supports S or non-S is the same thing with how relevant is I for S. If I supports S then obviously it is relevant for it. On the other hand, if it supports non-S obviously isn't relevant for S, it is relevant for non-S. The point here, again, is that whether some information supports S or non-S is also a subjective factor. Therefore, all we have here are subjective factors: the initial plausibility of S, before information I arrived, the relevance of I as judged by ourselves, and how surprising I is.
Nonetheless, in the mist of all this subjectivity, one is still able to constantly improve one's world view as new information become available. The idea is that no information should be completely discarded; each piece of information should be taken into account and weighted by its subjective relevance. The act of learning can be thus understood as a sequence of constantly adding new information, i.e. as a recurrence of Bayes' formula.
Each step looks like this: One considers one's beliefs, for example belief S and one assigns certain plausibility to S. Then, in the light of everything that one knows, one assigns a certain relevance to information I, and one is more or less surprised to find such a thing. Based on this, one changes the plausibility of S to some new plausibility. One repeats this process for all the statements S, for all the beliefs that one holds.
Bayes' formula provides a quantitative description of this process.