Questions tagged [roc]
ROC (Receiver Operating Characteristic) curve is a graphical plot comparing the true positive and false positive rates of a classifier as its discrimination threshold is varied.
1,064
questions
126
votes
18
answers
412k
views
How to plot ROC curve in Python
I am trying to plot a ROC curve to evaluate the accuracy of a prediction model I developed in Python using logistic regression packages. I have computed the true positive rate as well as the false ...
- 1,331
85
votes
5
answers
96k
views
Roc curve and cut off point. Python
I ran a logistic regression model and made predictions of the logit values. I used this to get the points on the ROC curve:
from sklearn import metrics
fpr, tpr, thresholds = metrics.roc_curve(...
- 1,869
67
votes
4
answers
128k
views
Simple line plots using seaborn
I'm trying to plot a ROC curve using seaborn (python).
With matplotlib I simply use the function plot:
plt.plot(one_minus_specificity, sensitivity, 'bs--')
where one_minus_specificity and ...
- 3,801
42
votes
5
answers
44k
views
Obtaining threshold values from a ROC curve
I have some models, using ROCR package on a vector of the predicted class percentages, I have a performance object. Plotting the performance object with the specifications "tpr", "fpr" gives me a ROC ...
- 725
42
votes
4
answers
102k
views
ROC for multiclass classification
I'm doing different text classification experiments. Now I need to calculate the AUC-ROC for each task. For the binary classifications, I already made it work with this code:
scaler = StandardScaler(...
- 815
38
votes
2
answers
37k
views
scikit-learn - ROC curve with confidence intervals
I am able to get a ROC curve using scikit-learn with
fpr, tpr, thresholds = metrics.roc_curve(y_true,y_pred, pos_label=1), where y_true is a list of values based on my gold standard (i.e., 0 for ...
- 383
32
votes
3
answers
53k
views
ROC curve from training data in caret
Using the R package caret, how can I generate a ROC curve based on the cross-validation results of the train() function?
Say, I do the following:
data(Sonar)
ctrl <- trainControl(method="cv",
...
- 16.7k
32
votes
4
answers
15k
views
Plotting a ROC curve in scikit yields only 3 points
TLDR: scikit's roc_curve function is only returning 3 points for a certain dataset.
Why could this be, and how do we control how many points to get back?
I'm trying to draw a ROC curve, but ...
- 6,384
29
votes
3
answers
60k
views
Multiple ROC curves in one plot ROCR
Is it possible to plot the roc curve for diffrent classifiers in the same plot using the ROCR package? I've tried:
>plot(perf.neuralNet, colorize=TRUE)
>lines(perf.randomForest)
But I get:
...
- 2,344
28
votes
1
answer
36k
views
How to plot precision and recall of multiclass classifier?
I'm using scikit learn, and I want to plot the precision and recall curves. the classifier I'm using is RandomForestClassifier. All the resources in the documentations of scikit learn uses binary ...
- 1,097
27
votes
1
answer
21k
views
How to interpret almost perfect accuracy and AUC-ROC but zero f1-score, precision and recall
I am training ML logistic classifier to classify two classes using python scikit-learn. They are in an extremely imbalanced data (about 14300:1). I'm getting almost 100% accuracy and ROC-AUC, but 0% ...
- 4,507
24
votes
3
answers
77k
views
R logistic regression area under curve
I am performing logistic regression using this page. My code is as below.
mydata <- read.csv("http://www.ats.ucla.edu/stat/data/binary.csv")
mylogit <- glm(admit ~ gre, data = mydata, family = "...
- 6,228
23
votes
6
answers
77k
views
ROC curve in R using ROCR package
Can someone explain me please how to plot a ROC curve with ROCR.
I know that I should first run:
prediction(predictions, labels, label.ordering = NULL)
and then:
performance(prediction.obj, ...
- 407
22
votes
4
answers
34k
views
How to compute AUC with ROCR package
I have fitted a SVM model and created the ROC curve with ROCR package. How can I compute the Area Under the Curve (AUC)?
set.seed(1)
tune.out=tune(svm ,Negative~.-Positive, data=trainSparse, ...
- 333
21
votes
2
answers
33k
views
ValueError: Data is not binary and pos_label is not specified
I am trying to calculate roc_auc_score, but I am getting following error.
"ValueError: Data is not binary and pos_label is not specified"
My code snippet is as follows:
import numpy as np
from ...
user2374515
20
votes
3
answers
27k
views
plotting ROC in R with ROCR vs pROC
I am plotting ROCs and measuring partial AUC as a metric of ecological niche model quality. As I am working in R, I am using the ROCR and the pROC packages. I'll settle on one to use, but for now, I ...
- 1,630
20
votes
2
answers
2k
views
TPR & FPR Curve for different classifiers - kNN, NaiveBayes, Decision Trees in R
I'm trying to understand and plot TPR/FPR for different types of classifiers. I'm using kNN, NaiveBayes and Decision Trees in R. With kNN I'm doing the following:
clnum <- as.vector(diabetes....
- 5,752
19
votes
2
answers
17k
views
plot.roc for multiclass.roc in pROC package?
I am trying to plot multiclass ROC curves but I have not found anything fruitful in the pROC package. Here's some start code:
data(iris)
library(randomForest)
library(pROC)
set.seed(1000)
# 3-class ...
- 1,281
19
votes
2
answers
20k
views
How to set a threshold for a sklearn classifier based on ROC results?
I trained an ExtraTreesClassifier (gini index) using scikit-learn and it suits my needs fairly. Not so good accuracy, but using a 10-fold cross validation, AUC is 0.95. I would like to use this ...
- 323
17
votes
2
answers
58k
views
How to compute ROC and AUC under ROC after training using caret in R?
I have used caret package's train function with 10-fold cross validation. I also have got class probabilities for predicted classes by setting classProbs = TRUE in trControl, as follows:
...
- 4,896
16
votes
2
answers
23k
views
thresholds in roc_curve in scikit learn
I am referring to the below link and sample, and post the plot diagram from this page where I am confused. My confusion is, there are only 4 threshold, but it seems the roc curve has many data points (...
- 9,979
15
votes
5
answers
53k
views
How to plot multiple ROC curves in one plot with legend and AUC scores in python?
I am building 2 models.
Model 1
modelgb = GradientBoostingClassifier()
modelgb.fit(x_train,y_train)
predsgb = modelgb.predict_proba(x_test)[:,1]
metrics.roc_auc_score(y_test,predsgb, average='macro',...
- 544
14
votes
2
answers
6k
views
Creating a threshold-coded ROC plot in Python
R's ROCR package provides options for ROC curve plotting that will color code and label threshold values along the curve:
The closest I can get with Python is something like
from sklearn.metrics ...
- 46.7k
13
votes
3
answers
5k
views
scikit-learn roc_curve: why does it return a threshold value = 2 some time?
Correct me if I'm wrong: the "thresholds" returned by scikit-learn's roc_curve should be an array of numbers that are in [0,1]. However, it sometimes gives me an array with the first number close to "...
- 2,447
12
votes
1
answer
6k
views
Good ROC curve but poor precision-recall curve
I have some machine learning results that I don't quite understand. I am using python sciki-learn, with 2+ million data of about 14 features. The classification of 'ab' looks pretty bad on the ...
- 4,507
12
votes
1
answer
17k
views
GridSearchCV has no attribute grid.grid_scores_
tried grid.cv_results_ didnt correct problem
from sklearn.model_selection
import GridSearchCV
params = {
'decisiontreeclassifier__max_depth': [1, 2],
'pipeline-1__clf__C': [0.001, 0.1, 100.0]
}...
- 121
12
votes
2
answers
8k
views
How to fix ROC curve with points below diagonal?
I am building receiver operating characteristic (ROC) curves to evaluate classifiers using the area under the curve (AUC) (more details on that at end of post). Unfortunately, points on the curve ...
- 7,584
11
votes
2
answers
19k
views
Using cross validation and AUC-ROC for a logistic regression model in sklearn
I'm using the sklearn package to build a logistic regression model and then evaluate it. Specifically, I want to do so using cross validation, but can't figure out the right way to do so with the ...
- 1,149
11
votes
1
answer
15k
views
DecisionTreeClassifier predict_proba returns 0 or 1
I m trying to use the decision tree classified to identify two classes (renamed 0 and 1) based on certain parameters. I train it using a dataset and then run it on the "test dataset". When I try to ...
- 397
10
votes
2
answers
28k
views
How to fix ValueError: multiclass format is not supported [duplicate]
This is my code and I try to calculate ROC score but I have a problem with ValueError: multiclass format is not supported. I'm already looking sci-kit learn but it doesn't help. In the end, I'm still ...
- 139
9
votes
6
answers
35k
views
sklearn ImportError: cannot import name plot_roc_curve
I am trying to plot a Receiver Operating Characteristics (ROC) curve with cross validation, following the example provided in sklearn's documentation. However, the following import gives an ...
- 2,370
9
votes
2
answers
13k
views
Plot ROC curve from Cross-Validation (training) data in R
I would like to know if there is a way to plot the average ROC Curve from the cross-validation data of a SVM-RFE model generated with the caret package.
My results are:
Recursive feature selection
...
- 349
9
votes
1
answer
40k
views
roc curve with sklearn [python]
I have an understanding problem by using the roc libraries.
I want to plot a roc curve with a python
http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html
I am writing ...
- 91
9
votes
1
answer
13k
views
How can I get The optimal cutoff point of the ROC in logistic regression as a number
I would like to get the optimal cut off point of the ROC in logistic regression as a number and not as two crossing curves.
Using the code below I can get the plot that will show the optimal point but ...
- 2,203
9
votes
1
answer
17k
views
How to calculate AUC with tensorflow?
I've built a binary classifier using Tensorflow and now I would like to evaluate the classifier using AUC and accuracy.
As far as accuracy is concerned, I can easily do like this:
X = tf....
- 1,182
8
votes
2
answers
9k
views
Get optimal threshold with at least 75% sensitivity with pROC in R
I have a dataframe with two columns : score1 which is numeric and truth1 which is boolean.
I want to predict truth1 using score1. To do that, I want a simple linear model, and then ask for a good ...
- 1,809
8
votes
1
answer
21k
views
R get AUC and plot multiple ROC curves together at the same time
I have tried 2 methods to plot ROC curve and get AUC for each ROC curve.
Method 1
- The first method is simple but I don't know how to plot multiple ROC curves together.
I am simply using roc.curve(...
- 3,964
8
votes
2
answers
25k
views
how to calculate roc curves?
I write a classifier (Gaussian Mixture Model) to classify five human actions. For every observation the classifier compute the posterior probability to belong to a cluster.
I want to valutate the ...
- 317
8
votes
0
answers
4k
views
Roc_curve pos_label confusion
I have a question related to the roc_curve from scikit-learn for a deep learning exercise, I have noticed that my data has 1 as the positive label. After my training the testing accuracy is coming ...
- 1,256
7
votes
3
answers
23k
views
Manually calculate AUC
How can I obtain the AUC value having fpr and tpr? Fpr and tpr are just 2 floats obtained from these formulas:
my_fpr = fp / (fp + tn)
my_tpr = tp / (tp + fn)
my_roc_auc = auc(my_fpr, my_tpr)
I know ...
user9370613
7
votes
1
answer
18k
views
ROC curve for classification from randomForest
I am using randomForest package in R platform for classification task.
rf_object<-randomForest(data_matrix, label_factor, cutoff=c(k,1-k))
where k ranges from 0.1 to 0.9.
pred <- predict(...
- 279
7
votes
1
answer
2k
views
Does calibration improve roc score?
I'm studing the effects of performing a calibrated classifier and I read that the aim of calibrating is to make a classifier's prediction more 'reliable'.
With this in mind I think that a calibrated ...
- 628
7
votes
1
answer
12k
views
Computing AUC and ROC curve from multi-class data in scikit-learn (sklearn)?
I am trying to use the scikit-learn module to compute AUC and plot ROC curves for the output of three different classifiers to compare their performance. I am very new to this topic, and I am ...
- 555
7
votes
2
answers
8k
views
Changing scale of the ROC chart
I am using the following code to plot the ROC curve after having run the logistic regression.
fit1 <- glm(formula=GB160M3~Behvscore, data=eflscr,family="binomial", na.action = na.exclude)
prob1=...
- 147
7
votes
2
answers
2k
views
Using ROC AUC score with Logistic Regression and Iris Dataset
What I need is to:
Apply a logistic regression classifier
Report the per-class ROC using the AUC.
Use the estimated probabilities of the logistic regression to guide the construction of the ROC.
...
- 574
7
votes
1
answer
3k
views
How to optimize precision-recall curve instead of AUC-ROC curve in python scikit-learn?
I am asking a follow-up question as suggested from my previous post - Good ROC curve but poor precision-recall curve. I am only using the default setting with Python scikit-learn. It seems like the ...
- 4,507
7
votes
2
answers
20k
views
How to plot a ROC curve with Tensorflow and scikit-learn?
I'm trying to plot the ROC curve from a modified version of the CIFAR-10 example provided by tensorflow. It's now for 2 classes instead of 10.
The output of the network are called logits and take the ...
- 276
7
votes
1
answer
17k
views
Python Matplotlib: Find the corresponding x value of a given y value on a basic plot
I have a plot generated using Matplotlib (it is a precision-recall curve of a histogram originally) and I need to compute the correct x value which corresponds to the y value with y = 0.9. The data ...
- 101
7
votes
1
answer
4k
views
How to extract average ROC curve predictions using ROCR?
The ROCR library in R offer the ability to plot an average ROC curve (right from the ROCR reference manual):
library(ROCR)
library(ROCR)
data(ROCR.xval)
# plot ROC curves for several cross-validation ...
- 599
6
votes
4
answers
8k
views
How to directly plot ROC of h2o model object in R
My apologies if I'm missing something obvious. I've been thoroughly enjoying working with h2o in the last few days using R interface. I would like to evaluate my model, say a random forest, by ...
- 811