Specificity, Sensitivity trade-off in Covid-19 antigen test trials
Alice: Hey Bob, How’re you doing in Lockdown?
Bob: Hi Alice, I’m doing fine, you?
Alice: Same Man!! Ssup?
Bob: Nothing man, I was just reading a news article about ICMR gives nod to the antigen-based testing kit for faster diagnosis and stuck at a point, now trying to get the meaning of Sensitivity and Specificity, Do you know about these things?
Alice: Oh yeah man. I got to know these terms in the Machine learning Course while I was in college. I can help you with that.
Bob: Oh yes, please.
Alice: Okay, let’s get the nit & grit of these terms.
In Machine learning, we use different metrics for different tasks. For classification tasks only we’ve different metrics like Accuracy, Precision, Recall, F1 score, etc. It’s completely dependent on your Problem statement which you’re trying to solve. You can refer to this blog to get a good understanding of WHICH TO USE WHEN.
So, first, let me state the problem statement which we have here, viz:
ICMR recommending Antigen-based testing to fast check whether a person is Covid-19 positive or not. So How do we measure the performance of this Diagnosis system?
As you can see its a classification problem i.e. you need to classify a person between two categories either COVID positive or COVID negative. So there are four possibilities for a test (based on what is in Actual and what Diagnosis system predicting) which is shown in the below diagram:
True Positive (TP): the patient has Corona and the test is positive.
False Positive (FP): the patient doesn’t have Corona but the test is positive.
True Negative (TN): the patient doesn’t have Corona and the test is negative.
False Negative (FN): the patient has Corona but the test is negative.
In the Medical domain, Sensitivity (TPR, True Positive Rate) and specificity (TNR, True Negative Rate) are terms used to evaluate a clinical test. Now based on Matrix (apparently called Confusion Matrix and you know it why :P) in the above diagram, one can easily calculate these terms by using below formula:
so as you can see, Sensitivity of a test refers to the ability to correctly identify those patients with the disease, and Specificity of a test refers to the ability to correctly identify those patients without the disease. Below snippet, that will help you in visualizing this process of generating these metrics.
In the above snippet, you can see there is 100 % Sensitivity which means the system is able to correctly identify all people whoever actually had Corona. And it has 50 % Specificity which means it got False Positive Cases (third person from the left).
There is always a trade-off between Sensitivity and Specificity. For example, In Antigen based Testing, ICMR stated this test would be focused on maintaining high specificity and low sensitivity which means the Diagnosis system would be good at ruling out people who don’t have Corona. But on the other side, if the system has very low sensitivity, it means it generates many False Negative results which is a dangerous thing.
In today’s world where modern Diagnostic Systems are using Artificial Intelligence to deliver much faster and accurate results, maintaining a good balance between these terms is a supreme challenge.
Bob: Oh man!! I didn’t know these terms had this much importance. Thank you Alice for explaining it to me.
Alice: yeah man!! We’re in this fight together and I’m feeling so proud that I’ve shared this knowledge with you. Knowledge transfer should be on top to fight this current situation. Stay Safe Bob.
Bob: Completely agree. You too :)
Thanks for the read!
I hope you have a picture of these terms now. It’s wise and smart to have knowledge of these terms, given the current scenario of the COVID-19 pandemic so that we can get a better understanding of the ongoing situation.
Any feedback or suggestion would be welcomed.
