Python-bloggers

Performance measures: The Concordance Correlation Coefficient

This article was first published on Technical Posts Archives - TDS , and kindly contributed to python-bloggers. (You can report issue about the content on this page here)
Want to share your content on python-bloggers? click here.

Wanna become a data scientist within 3 months, and get a guaranteed job? Then you need to check this out !

This post continues the series of posts on performance measures. In our previous article we talked about Cohen’s Kappa. In this post we will talk about the concordance correlation coefficient.

A common way to measure the performance of a regression algorithm is Pearson’s correlation between the true and the predicted values. However, Pearson’s correlation in this case suffers from one drawback. It ignores any bias which might exist between the true and the predicted values. Let me give an example. The figure below shows the relationship between two variables. Variable X is sampled from the standard normal distribution and y=x+10. We can treat X as the target variable and as Y the predicted value of a statistical model.

Trending

The Positive Side of Technology: Leverage AI to Make Smarter Financial Business Decisions

The correlation between these two variables takes the value 1, so it is perfect.

However, what we care about when measuring the agreement between the true and the predicted values, is whether they predict the exact same value. If that is the case, then a point is on the 45 degree line crossing the origin. This is the red line shown below. It is clear that the red line that indicates perfect agreement is parallel to the relationship between x and y.

So what can we do? The concordance correlation coefficient comes to the rescue! The concordance correlation coefficient measures the agreement between two variables. In this case, the value is around 0.02, indicating no agreement between the two variables.

Unfortunately, the concordance correlation coefficient is not widely used in the evaluation of predictive models. I believe this to be an important omission and I would urge any data scientist to start using it for regression modelling.

You can use it in R by downloading the package epiR.

So, next time you run a regression model use both Pearson’s correlation coefficient and the concordance correlation coefficient and check the difference between them. You’ll see that the concordance correlation coefficient is usually more conservative and has a lower value than Pearson’s correlation coefficient. However, it can be a safer measure to use for measuring performance in regression, in order to avoid these few cases where strong bias in the predictions might be an issue.

Also, make sure to check some of my other posts about performance measures in predictive modelling and Cohen’s Kappa.

Wanna become a data scientist within 3 months, and get a guaranteed job? Then you need to check this out !

The Positive Side of Technology: Leverage AI to Make Smarter Financial Business Decisions

Power Skiving for Smaller Precise Robotics Machining

Unraveling the Key Techniques and Best Practices of Regression Testing for Ensuring Long-Term Quality Assurance

Unleashing the Power of ChatGPT: Revolutizing Communication and Streamlining Workflows

To leave a comment for the author, please follow the link and comment on their blog: Technical Posts Archives - TDS .

Want to share your content on python-bloggers? click here.
Exit mobile version