当损失是均方误差 (MSE) 时,什么函数定义了 Keras 中的准确性?

问题描述:

当损失函数为均方误差时,Accuracy 是如何定义的?是平均绝对百分比误差吗?

How is Accuracy defined when the loss function is mean square error? Is it mean absolute percentage error?

我使用的模型具有输出激活线性,并使用loss= mean_squared_error

The model I use has output activation linear and is compiled with loss= mean_squared_error

model.add(Dense(1))
model.add(Activation('linear'))  # number

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])

输出如下所示:

Epoch 99/100
1000/1000 [==============================] - 687s 687ms/step - loss: 0.0463 - acc: 0.9689 - val_loss: 3.7303 - val_acc: 0.3250
Epoch 100/100
1000/1000 [==============================] - 688s 688ms/step - loss: 0.0424 - acc: 0.9740 - val_loss: 3.4221 - val_acc: 0.3701

那是什么,例如val_acc:0.3250 是什么意思?Mean_squared_error 应该是标量而不是百分比 - 不是吗?那么 val_acc - 均方误差,平均百分比误差还是其他函数?

So what does e.g. val_acc: 0.3250 mean? Mean_squared_error should be a scalar not a percentage - shouldnt it? So is val_acc - mean squared error, or mean percentage error or another function?

来自* MSE 的定义:https://en.wikipedia.org/wiki/Mean_squared_error一个>

From definition of MSE on wikipedia:https://en.wikipedia.org/wiki/Mean_squared_error

MSE 是衡量估算器质量的指标——它始终是非负,值越接近零越好.

The MSE is a measure of the quality of an estimator—it is always non-negative, and values closer to zero are better.

这是否意味着 val_acc: 0.0 的值比 val_acc: 0.325 更好?

Does that mean a value of val_acc: 0.0 is better than val_acc: 0.325?

训练时准确度指标输出的更多示例 - 随着训练次数的增加,准确度会提高.而损失函数 - mse 应该减少.准确度是否为 mse 定义了良好 - 以及它在 Keras 中是如何定义的?

edit: more examples of the output of accuracy metric when I train - where the accuracy is increase as I train more. While the loss function - mse should decrease. Is Accuracy well defined for mse - and how is it defined in Keras?

lAllocator: After 14014 get requests, put_count=14032 evicted_count=1000 eviction_rate=0.0712657 and unsatisfied allocation rate=0.071714
1000/1000 [==============================] - 453s 453ms/step - loss: 17.4875 - acc: 0.1443 - val_loss: 98.0973 - val_acc: 0.0333
Epoch 2/100
1000/1000 [==============================] - 443s 443ms/step - loss: 6.6793 - acc: 0.1973 - val_loss: 11.9101 - val_acc: 0.1500
Epoch 3/100
1000/1000 [==============================] - 444s 444ms/step - loss: 6.3867 - acc: 0.1980 - val_loss: 6.8647 - val_acc: 0.1667
Epoch 4/100
1000/1000 [==============================] - 445s 445ms/step - loss: 5.4062 - acc: 0.2255 - val_loss: 5.6029 - val_acc: 0.1600
Epoch 5/100
783/1000 [======================>.......] - ETA: 1:36 - loss: 5.0148 - acc: 0.2306

您的问题至少有两个单独的问题.

There are at least two separate issues with your question.

从史努比博士的评论和另一个答案中,第一个现在应该很清楚:在回归问题中,准确性毫无意义,例如您的问题;另请参阅 patyork 在 此 Keras 线程中的评论.不管好坏,事实是 Keras 不会保护"您或任何其他用户在您的代码中放置无意义的请求,即您不会收到任何错误,甚至警告,您正在尝试一些没有意义的事情,例如在回归设置中请求准确性.

The first one should be clear by now from the comments by Dr. Snoopy and the other answer: accuracy is meaningless in a regression problem, such as yours; see also the comment by patyork in this Keras thread. For good or bad, the fact is that Keras will not "protect" you or any other user from putting not-meaningful requests in your code, i.e. you will not get any error, or even a warning, that you are attempting something that does not make sense, such as requesting the accuracy in a regression setting.

澄清之后,另一个问题是:

Having clarified that, the other issue is:

由于 Keras 确实返回了准确度",即使在回归设置中,它究竟是什么以及它是如何计算的?

为了说明这一点,让我们回到公共数据集(因为您没有提供有关数据的任何详细信息),即 波士顿房价数据集(本地保存为housing.csv),运行一个简单的实验如下:

To shed some light here, let's revert to a public dataset (since you do not provide any details about your data), namely the Boston house price dataset (saved locally as housing.csv), and run a simple experiment as follows:

import numpy as np
import pandas
import keras

from keras.models import Sequential
from keras.layers import Dense

# load dataset
dataframe = pandas.read_csv("housing.csv", delim_whitespace=True, header=None)
dataset = dataframe.values
# split into input (X) and output (Y) variables
X = dataset[:,0:13]
Y = dataset[:,13]

model = Sequential()
model.add(Dense(13, input_dim=13, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model asking for accuracy, too:
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])

model.fit(X, Y,
     batch_size=5,
     epochs=100,
     verbose=1)

与您的情况一样,模型拟合历史记录(此处未显示)显示损失减少,准确度大致增加.现在让我们使用适当的 Keras 内置函数评估同一训练集中的模型性能:

As in your case, the model fitting history (not shown here) shows a decreasing loss, and an accuracy roughly increasing. Let's evaluate now the model performance in the same training set, using the appropriate Keras built-in function:

score = model.evaluate(X, Y, verbose=0)
score
# [16.863721372581754, 0.013833992168483997]

score 数组的确切内容取决于我们在模型编译期间所请求的内容;在我们这里的例子中,第一个元素是损失 (MSE),第二个元素是准确度".

The exact contents of the score array depend on what exactly we have requested during model compilation; in our case here, the first element is the loss (MSE), and the second one is the "accuracy".

此时,我们来看看metrics.py 文件:

At this point, let us have a look at the definition of Keras binary_accuracy in the metrics.py file:

def binary_accuracy(y_true, y_pred):
    return K.mean(K.equal(y_true, K.round(y_pred)), axis=-1)

因此,在 Keras 生成预测 y_pred 后,它首先对它们进行四舍五入,然后检查其中有多少与真实标签 y_true 相等,在此之前得到平均值.

So, after Keras has generated the predictions y_pred, it first rounds them, and then checks to see how many of them are equal to the true labels y_true, before getting the mean.

让我们用普通的 Python & 复制这个操作在我们的例子中的 Numpy 代码,其中真正的标签是 Y:

Let's replicate this operation using plain Python & Numpy code in our case, where the true labels are Y:

y_pred = model.predict(X)
l = len(Y)
acc = sum([np.round(y_pred[i])==Y[i] for i in range(l)])/l
acc
# array([0.01383399])

好吧,宾果游戏!这实际上与上面的 score[1] 返回的值相同...

Well, bingo! This is actually the same value returned by score[1] above...

长话短说:由于您(错误地)在模型编译中请求 metrics=['accuracy'],Keras 将 尽力让你满意,并且会返回一些准确度";确实,如上所示计算,尽管这在您的设置中完全没有意义.

To make a long story short: since you (erroneously) request metrics=['accuracy'] in your model compilation, Keras will do its best to satisfy you, and will return some "accuracy" indeed, calculated as shown above, despite this being completely meaningless in your setting.

在很多设置中,Keras 在幕后执行相当无意义的操作,而不给用户任何提示或警告;我碰巧遇到的其中两个是:

There are quite a few settings where Keras, under the hood, performs rather meaningless operations without giving any hint or warning to the user; two of them I have happened to encounter are:

  • Giving meaningless results when, in a multi-class setting, one happens to request loss='binary_crossentropy' (instead of categorical_crossentropy) with metrics=['accuracy'] - see my answers in Keras binary_crossentropy vs categorical_crossentropy performance? and Why is binary_crossentropy more accurate than categorical_crossentropy for multiclass classification in Keras?

完全禁用 Dropout,在极端情况下,当有人要求 dropout 率为 1.0 时 - 请参阅我在 Keras 中 rate=1 的丢弃行为(丢弃所有输入单元)不符合预期>

Disabling completely Dropout, in the extreme case when one requests a dropout rate of 1.0 - see my answer in Dropout behavior in Keras with rate=1 (dropping all input units) not as expected