I understand the scoring is calling for probabilities, not binary 0,1. How did the scorers/judges get these probabilities? It seems like judges only have access to the actual binary 0,1, unless they are mind readers. What am I not understanding?
Hi, I am unable to understand your question. From my understanding, when you have a prediction of probabilities and you check the AUC under ROC against binary variables, you are going to get a number within 0 and 1. I work in R and I am checking my model’s efficiency in the fashion below.
#Probability Prediction
pred_seasonal <- predict(modFitgbm, val, type = 'prob')$X1
#Check Individual AUC
seasonal<-val$seasonal_vaccine
AUC(pred_seasonal,seasonal)
This gives me a value like 0.83234. That is the AUC my train and validation model is predicting. And I am getting decently close values with the test data so the test in not very far off.
Please let me know if this helps? Thanks!
A resource below which is compact and informative: