Terms Used To Assess Effectiveness of Coronavirus Tests Similar To That For Bank Loan Customers, says Scientist

In a paper sent to PAP, Professor Marek Majewski argues that concepts such as false positives, false negatives, sensitivity and specificity used to describe the effectiveness of tests are used in both medical diagnostics and data analysis, where mathematical models are constructed.

He said: “Imagine a dataset containing information about the history of bank accounts of customers of a certain bank. In this case, a classification model can help answer the question whether given customers will (or will not) pay back the loan they are applying for.

“The situation with diagnostic tests is quite similar. They are designed to answer the question whether the patient belongs to one of two categories: he is infected or not.”

He added that the quality of these models is assessed using the same techniques and concepts. A certain set of data is required, for example one hundred customers or one hundred patients who are known to be (or not to be) able to pay back the loan, or they are known to be infected or not infected. This set is called the test set.

For bank customers, the test set can be built based on customer history data. In the case of medicine, it will be a comparison with a certain, absolutely reliable reference test (the so-called gold standard).

Tests are evaluated by comparing them with this 'golden' test. The test under evaluation is carried out for each person from the 'golden' set and the results are compared with real data. According to Professor Majewski, there are four possible results of such this comparison.

The first is when the test result is positive, according to the test the patient is infected, although in fact the patient is healthy. This case is called a false positive. The second is when the test result is positive and the patient is actually infected and unwell. Then the result is a true positive. In the third case, when the test result is negative, according to the test the patient is healthy, although in fact the patient is actually - a false negative. Finally, the test result is negative and the patient is in fact healthy. This is a true negative result.

The number of observations, for which individual results were obtained, is usually compiled in a table called confusion matrix. Based on the confusion matrix, the parameters of the test are entered. Professor Majewski defined the most important of them as being: 

- Accuracy rate (ACC) means the number of correctly classified observations divided by the total number of observations.

- Error rate means the ratio of falsely classified observations to the total number of observations.

- Sensitivity measures the proportion of true positive classifications relative to the number of all (truly) positive cases. Sensitivity answers the question of what proportion of positives the test detects, or what when testing an infected person, the test will show that the person is infected.

- Test specifity is the number of true negative classifications relative to the number of all (true) negative cases. Specificity tells you what proportion of negatives the test detects, or what the probability is that for a healthy person the test will show that the person is healthy.

- Positive predictive value (PPV) measures the proportion of true positive classifications relative to all positive classifications. Positive predictive value answers the question how many of the positively classified cases have been classified correctly, or if the test result is positive, what is the probability that the subject is infected.

- Negative predictive value (NPV) is the ratio of the number of true negative classifications to all negative classifications. The indicator answers the question: if the test result is negative, what is the probability that the tested person is healthy?

Majewski said: “In medical diagnostics, the interpretation of the last two indicators should be approached with caution because they have been shown to depend on prevalence (the current number of infected persons). For example, as prevalence increases, PPV also increases. But they are still used in data analysis.”

He added that tests with very high accuracy are usually expensive while so-called screening tests are usually cheaper. In his opinion, it makes sense to use screening tests that have high precision in negative prediction. They return the least possible false negative results.

PAP - Science in Poland, Karolina Duszczyk

tr. RL

kol/ agt/ kap/