Precision vs. recall vs. accuracy in neural networks

Neural networks are machine learning models created by taking inspiration from biological neurons in the human brain. They consist of multiple different types of layers that perform mathematical operations to generate an output that maps to a decision.

To determine which neural network model is better, it is important to have performance metrics to evaluate the effectiveness of models. We can compare the multiple neural network models to select the optimum one.

In this Answer, we will discuss three evaluation metrics: accuracy, precision, and recall, to assess the performance of neural networks and compare them.

Confusion matrix

First, let us take a look at what a confusion matrix is. A confusion matrix is a table summarizing our model's predictions against the actual class labels in the data set.

Below we can see a sample confusion matrix.

• True positive: The number of examples that belong to a positive class and are predicted as positive.

• True negative: The number of examples that belong to a negative class and are predicted as negative.

• False positive: The number of examples that belong to a negative class but are predicted as positive by the model.

• False negative: The number of examples that belong to a positive class but are predicted as negative by the model.

Precision

Precision measures how accurately a model determines a positive output for given input data.

A high precision value shows that the neural network model avoids false positives. Models with a high precision value are preferred in an environment where the cost of dealing with false positives is high.

The precision of a neural network model can be calculated using the formula below:

Let us take an example; suppose we are working on a neural network for spam detection. We are interested in the model being able to predict spam mail accurately. Precision would be used here to tell us how the model found true positives against all the positive predictions.

Recall

Recall refers to the ability of the neural network model to detect positive class labels.

A model with a high recall value accurately captures the data associated with the positive class labels and steers clear of false negative predictions. Models with high recall scores are necessary when the costs of dealing with false negatives are high.

Recall of a neural network can be calculated using the formula below:

For example, if we are building a neural network model for cancer detection, our focus would be to minimize the number of false negatives. In this case, recall would be an important performance metric.

Accuracy

Accuracy is the percentage of cases properly categorized within the dataset.

A model with a high accuracy score shows that the model correctly classifies the input across all classes. This indicates that accuracy is a measure of the overall correctness of the neural network model in predicting the class label for a given input.

The accuracy of a neural network model can be calculated using the formula below:

For example, we are working on a neural network model to distinguish between different animals. Accuracy will be used to measure the model's performance as it gives the overall correctness of the model.

Comparison discussion

Now that we have gone through the 3 different types of neural network performance metrics, we can see a brief comparison that differentiates them.

Precision measures the proportion of examples correctly classified as having a positive class by the neural network against the total examples with a positive class. Accuracy measures how correctly the neural network predicted the positive and negative classes against all examples. Recall measures the proportion of examples that are positively predicted by the neural network against the total number of positive examples.