Keras神经网络中val-acc和预测精度之间的巨大差异

我有一个用于在Keras中建立NN模型的数据集，我从该数据集中获取了2000行作为验证数据，应该在.predict函数中添加这2000行.

I have a dataset that I used for making NN model in Keras, i took 2000 rows from that dataset to have them as validation data, those 2000 rows should be added in .predict function.

我为Keras NN写了一个代码，现在它可以正常工作，但是我注意到一些对我来说很奇怪的东西.它为我提供了超过83％的非常好的准确性，损失约为0.12，但是当我要使用看不见的数据(即2000行)进行预测时，它只能预测平均65％的正确率. 当我添加Dropout层时，只会降低准确性.

I wrote a code for Keras NN and for now it works good, but I noticed something that is very strange for me. It gives me very good accuracy of more than 83%, loss is around 0.12, but when I want to make a prediction with unseen data (those 2000 rows), it only predicts correct in average of 65%. When I add Dropout layer, it only decreases accuracy.

然后我添加了EarlyStopping，它使我的准确度达到了86％左右，损失大约为0.10，但是当我用看不见的数据进行预测时，最终的预测准确度还是67％.

Then I have added EarlyStopping, and it gave me accuracy around 86%, loss is around 0.10, but still when I make prediction with unseen data, I get final prediction accuracy of 67%.

这是否意味着模型可以在87％的情况下做出正确的预测?我的逻辑是，如果我在.predict函数中添加100个样本，那么该程序应该对87/100个样本或该范围内的某个位置(可以说超过80个)做出了很好的预测?我曾尝试在我的.predict函数中添加100、500、1000、1500和2000个样本，它总是对65-68％的样本做出正确的预测.

Does this mean that model made correct prediction in 87% of situations? Im going with a logic, if I add 100 samples in my .predict function, that program should make good prediction for 87/100 samples, or somewhere in that range (lets say more than 80)? I have tried to add 100, 500, 1000, 1500 and 2000 samples in my .predict function, and it always make correct prediction in 65-68% of the samples.

那是为什么，我做错了什么? 我尝试使用层数，节点数，具有不同的激活功能和不同的优化程序，但是它只能将结果更改1-2％. 我的数据集看起来像这样:

Why is that, am I doing something wrong? I have tried to play with number of layers, number of nodes, with different activation functions and with different optimizers but it only changes the results by 1-2%. My dataset looks like this:

DataFrame shape (59249, 33)
x_train shape (47399, 32)
y_train shape (47399,)
x_test shape (11850, 32)
y_test shape (11850,)
testing_features shape (1000, 32)

这是我的NN模型:

model = Sequential()
model.add(Dense(64, input_dim = x_train.shape[1], activation = 'relu')) # input layer requires input_dim param
model.add(Dropout(0.2))
model.add(Dense(32, activation = 'relu'))
model.add(Dropout(0.2))
model.add(Dense(16, activation = 'relu'))
model.add(Dense(1, activation='sigmoid')) # sigmoid instead of relu for final probability between 0 and 1

# compile the model, adam gradient descent (optimized)
model.compile(loss="binary_crossentropy", optimizer= "adam", metrics=['accuracy'])


# call the function to fit to the data training the network)
es = EarlyStopping(monitor='val_loss', min_delta=0.0, patience=1, verbose=0, mode='auto')
model.fit(x_train, y_train, epochs = 15, shuffle = True, batch_size=32, validation_data=(x_test, y_test), verbose=2, callbacks=[es])

scores = model.evaluate(x_test, y_test)
print(model.metrics_names[0], round(scores[0]*100,2), model.metrics_names[1], round(scores[1]*100,2))

这些是结果:

Train on 47399 samples, validate on 11850 samples
Epoch 1/15
 - 25s - loss: 0.3648 - acc: 0.8451 - val_loss: 0.2825 - val_acc: 0.8756
Epoch 2/15
 - 9s - loss: 0.2949 - acc: 0.8689 - val_loss: 0.2566 - val_acc: 0.8797
Epoch 3/15
 - 9s - loss: 0.2741 - acc: 0.8773 - val_loss: 0.2468 - val_acc: 0.8849
Epoch 4/15
 - 9s - loss: 0.2626 - acc: 0.8816 - val_loss: 0.2416 - val_acc: 0.8845
Epoch 5/15
 - 10s - loss: 0.2566 - acc: 0.8827 - val_loss: 0.2401 - val_acc: 0.8867
Epoch 6/15
 - 8s - loss: 0.2503 - acc: 0.8858 - val_loss: 0.2364 - val_acc: 0.8893
Epoch 7/15
 - 9s - loss: 0.2480 - acc: 0.8873 - val_loss: 0.2321 - val_acc: 0.8895
Epoch 8/15
 - 9s - loss: 0.2450 - acc: 0.8886 - val_loss: 0.2357 - val_acc: 0.8888
11850/11850 [==============================] - 2s 173us/step
loss 23.57 acc 88.88

这是为了预测:

#testing_features are 2000 rows that i extracted from dataset (these samples are not used in training, this is separate dataset thats imported)

prediction = model.predict(testing_features , batch_size=32)

res = []
for p in prediction:
    res.append(p[0].round(0))


# Accuracy with sklearn - also much lower 
acc_score = accuracy_score(testing_results, res)
print("Sklearn acc", acc_score)    

result_df = pd.DataFrame({"label":testing_results,
                          "prediction":res})


result_df["prediction"] = result_df["prediction"].astype(int)

s = 0
for x,y in zip(result_df["label"], result_df["prediction"]):
    if x == y:
        s+=1

print(s,"/",len(result_df))
acc = s*100/len(result_df)
print('TOTAL ACC:', round(acc,2))

问题是...现在我获得sklearn 52％和my_acc 52％的准确性. 为什么验证的准确性要低得多?

The problem is...now I get accuracy with sklearn 52% and my_acc 52%. Why do I get such low accuracy on validation, when it says that its much larger?