Bayes' Theorem in AI

First proposed by Thomas Bayes, Bayes' Theorem is one of the fundamental theorems in the realm of AI and Bayesian statistics. It can be utilized as a probability tool to effectively find conditional probability. Therefore, it has numerous real-life applications like drug testing, robotics, and machine learning.

Before we dive into the specifics of how Bayes' Theorem can be used in AI, let's first understand what conditional probability actually is.

Conditional probability

Formally put, conditional probability is the probability of an event X occurring provided that an event Y has occurred.

We can simply understand it as the chance of something happening if we know that another event has taken place.

Bayes' Theorem

Bayes' Theorem is the name of the formula put forth for calculating conditional probability.

It is given by the following mathematical expression:

The above formula states that the conditional probability of event $A$ if event $B$ has occured is given by first multiplying these variables:

Probability of event $B$ given $A$
Probability of event $A$

This can then be divided by the probability of event $B$ to obtain the final result.

Explanation

$P (A)$ refers to the likelihood that event $A$ occurs.
$P (B)$ refers to the likelihood that event $B$ occurs.
$P (A | B)$ refers to the likelihood that event $A$ occurs given that event $B$ has occured.
$P (B | A)$ refers to the likelihood that event $B$ occurs given that event $A$ has occured.
$P (A ∩ B)$ refers to the likelihood that events $A$ and $B$ both occur.

Bayes' theorem allows us to show how likely something is to occur through the values of already known occurrences of other events.

Code explanation

Lines 3–5: Suppose we have some known values such as the probability of events A i.e. pOfA, B i.e. pOfB, and B given that A i.e pOfBGivenA .
Lines 7–9: We can calculate the probability of A happening when we know that B has happened by applying Bayes' Theorem through the formula AGivenB = probabilityOfA * BGivenA / probabilityOfB , which is defined in the function bayes. This function simply returns AGivenB.
Line 11: bayes is called to save the value of pOfAGivenB , which is then printed out to us.

Numerical scenario

Let's take System $Y$ , which has a malfunction rate of 0.5%. There is a diagnostic test available for detecting malfunctions in System $Y$ . It correctly identifies 98% of System $Y's$ malfunctions. Also, the test has a false positive rate of 3%. What is the probability of System $Y$ having a malfunction given that the test result is positive?

Solution

We can apply Bayes' theorem to calculate $P(Malfunction|Positive test)$ .

$P(Malfunction) = 0.5 \%= 0.005$

$P(Positive test|Malfunction) = 0.98$

$P(No Malfunction) = 100\% - 0.5\%$

$P(No Malfunction) = 99.5\%$

$P(Positive test|No Malfunction) = 0.03$

Since we don’t have P(Positive Test), we’ll have to calculate it using the rest of the values. For this, we need to consider the probabilities of getting a positive test result in both malfunction and non-malfunction cases.

$P(Positive Test) = (0.005 * 0.98) + (0.995 * 0.03)$

$P(Positive Test) = 0.035$

Plugging in the values in Bayes’ theorem gives us our result:

$P(Malfunction | Positive Test) = (0.005 * 0.98) / 0.035$

$P(Malfunction | Positive Test) = 0.142857$ .

Note: 0.142857 can now be expressed as a percentage. Hence, we can say that the probability of System $Y$ having a malfunction given a positive test result is 14.29%.

Applications in AI

Bayes' Theorem can be put to effective use in numerous fields such as computing, health, and finance.

Let's take a look at three applications of Bayes' Theorem:

Bayesian Optimization

This technique is mainly used for optimizing black box functions. Simply put, this requires changing the parameters of the function until an optimal solution is achieved based on previously observed function evaluations.

Bayesian Networks for decision support

Bayes' Theorem can be used to model decision-making problems in AI by calculating the probability of different outcomes based on available evidence. We can use it to further update the probabilities in the network as new evidence is obtained, leading to more informed decision-making.

Naive Bayes Classifier

Mostly used in machine learning, we apply Bayes' Theorem here for tasks such as text classification and spam filtering. For instance, the probability of an email being spam can be predicted by relating it to various keywords used in that email.

Conclusion

Since we can find an event's conditional probability through Bayes' Theorem, we can use it to predict various events in artificial intelligence or machine learning. We can further lead our calculations to higher precision by adding new observations in our data.