Support for partial AUC was added to fbroc with the version 0.4.0, which has just been accepted on CRAN.

Official changelog

fbroc 0.4.0

New features

• Partial AUCs over both TPR and FPR ranges can be calculated
• You can now adjust text size for plots
• In the ROC plot the (partial) AUC can now optionally be shown instead of confidence regions

Other Changes

• The location of the text showing the performance in the ROC plot has been shifted downwards and
  to the left

Changes in detail

There are only two changes worthwhile mentioning in fbroc 0.4.0. The first one is an option to adjust the text-size when printing out the performance details on the ROC plot. This change was motivated by the text sometimes being too wide for the graph – I observed this effect on my mobile phone.

A more important addition is the ability to handle partial AUC, integrating the part of the ROC curve over a specific FPR or TPR interval. The typical McClish correction for the partial AUC is applied by default. I will talk about it and more details about the partial AUC in a later post.

Plotting the partial ROC area was also a bit of a challenge, as the overlap with the confidence region around the ROC curve makes it difficult. Therefore, when fbroc shows the confidence region, the partial AUC region is only denoted by a pair of dotted lines. By setting

show.conf = FALSE 

in the plotting call when the metric being shown is the partial AUC, the relevant area is shown instead.

As a minor bonus this now also works for the normal AUC.

Implementation

The C++ code for calculating the partial AUC was somewhat tricky, as it needed to work when integrating over both FPR and TPR. As an example, take a look at the function used to integrate over a TPR interval by calculating the area contributed by the part of the ROC between the (i-1)-th and the i-th cutoff.

double pauc_tpr_area(NumericVector &tpr, NumericVector &fpr, 
                     NumericVector &param, int index)
{
  // necessary check to avoid division by zero later
  if (tpr(index - 1) == tpr[index]) return 0; 
  // cases where relevant TPR interval is not included
  if (tpr[index - 1] < param[0]) return 0;
  if (tpr[index] > param[1]) return 0;
  
  double left = std::max(tpr[index], param[0]);
  double right = std::min(tpr[index - 1], param[1]);
  
  double base_val = 1 - fpr[index];
  double slope = (fpr[index] - fpr[index - 1]) / 
                 (tpr[index - 1] - tpr[index]);

  double value_left = base_val + (left - tpr[index]) * slope;
  double value_right = base_val + (right - tpr[index]) * slope;
  
  return (right - left) * (value_left + value_right);
}

The first line excludes a case where the contribution to the partial AUC is zero anyway, because we are looking at a line instead of an area. After using this code segment

double left = std::max(tpr[index], param[0]);
double right = std::min(tpr[index - 1], param[1]);

to account for the case that the area actually contributing is just a slice of the full trapezoid between the TPRs for the (i-1)-th and i-th cutoff when the TPR interval used for the partial AUC does not fully encompass it, the difference would cancel out in the product used for the trapezoid rule in this line here

return (right - left) * (value_left + value_right);

but would lead to NaN numbers when fbroc calculates the slope for the trapezoid rule as follows.

double slope = (fpr[index] - fpr[index - 1]) / 
                 (tpr[index - 1] - tpr[index]);

What’s next

After releasing fbroc 0.4.0 I will first update the shiny interface, before working more on the package itself. I will also first create a shiny interface for the analysis of paired ROC curves – something I originally planned to do before releasing fbroc 0.4.0. As it turned out, I decided to update the package first since I find that more enjoyable.