【機械学習】モデルの精度を確認するための指標について(Indices as a measure of the model accuracy)

When I explain the rogit model which I made recently, I often use some indecies to explain how my model is perfect (or dreadful).

We can understand easily but difficult to explain to my boss or coleagues, therefore, I would like to summarize some indices I often use.

 

1.AR (Accuracy Ratio) score

AR is used as a index to check how the model can predict answer collectly.

AR is also known as CAP curve as well.

AR is calculated by below.

1.The model calculates that the probability of 1(black) or 0(white) for each data.

After that, we sort the data in descending order regarding calcurated score.

Then, we plot with x-axis and y-axis, x-axis is accumurated by data, y-axis is accumurated by predicted score.

Then , we calcurated the area between the plotted line and 45 degree line.

The score created in these steps are AR score.

 

2.AUC

AUC is an avvribation of the area under the curve.

AUC is calculated as below.

1.The model calculates that the probability of 1(black) or 0(white) for each data.

After that, the true positive rate and the false postive rate is calicurated.

the true positive rate is predicted true data/predicted true + real true

the false positive rate is predicted false/predicted false + real false.

Then,