私は現在マルチラベルの画像分類をkerasのCNNを用いて行っています。
また、kerasのaccuracyだけでなくscikit-learnの様々な評価方法(Recall, Precision, F1 scoreそしてAccuracy)を用いて精度の再確認を行いました。
結果としてkerasで算出したAccuracyは約90%を示すのに、scikit-learnはどれも60%前後しか示しません。
これは私にはなぜだかわからないので誰か教えてください。
kerasの計算がおかしいのでしょうか?
kerasとscikit-learnのaccuracyの算出に違いがあるのですか?
何か不足があれば付け足します
データの読み込みなどは、こちらを真似ています。
modelは, mobilenetV2を使用してFine-tuningさせています。
#input_tensorの定義
input_tensor = Input(shape=(img_width, img_height, 3))
base_model = MobileNetV2(include_top=False, weights='imagenet')
#model.summary()
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation = 'relu')(x)
x = Dropout(0.5)(x)
predictions = Dense(6, activation = 'sigmoid')(x)
# ネットワーク定義
model = Model(inputs = base_model.input, outputs = predictions)
print("{}層".format(len(model.layers)))
sgd = optimizers.SGD(lr=0.001, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(optimizer=sgd, loss="binary_crossentropy", metrics=["acc"])
history = model.fit(X_train, y_train, epochs=50, validation_data=(X_val, y_val), batch_size=64, verbose=2)
# モデル評価
def model_evaluate():
score = model.evaluate(X_test, y_test, verbose = 1)
print("evaluate loss: {[0]:.4f}".format(score))
print("evaluate acc: {[1]:.1%}".format(score))
model_evaluate()
Kerasの精度(Accuracy)は約90%
scikit-Learnの処理
from sklearn.metrics import precision_score, recall_score, f1_score, accuracy_score
thresholds=[0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
y_pred = model.predict(X_test)
predictions=model.predict(X_test)
for val in thresholds:
print("For threshold: ", val)
pred=predictions.copy()
pred[pred>=val]=1
pred[pred<val]=0
precision = precision_score(y_test, pred, average='micro')
recall = recall_score(y_test, pred, average='micro')
f1 = f1_score(y_test, pred, average='micro')
print("Micro-average quality numbers")
print("Precision: {:.4f}, Recall: {:.4f}, F1-measure: {:.4f}".format(precision, recall, f1))
Scikit-Learnの結果
For threshold: 0.1
Micro-average quality numbers
Precision: 0.3776, Recall: 0.8727, F1-measure: 0.5271
For threshold: 0.2
Micro-average quality numbers
Precision: 0.4550, Recall: 0.8033, F1-measure: 0.5810
For threshold: 0.3
Micro-average quality numbers
Precision: 0.5227, Recall: 0.7403, F1-measure: 0.6128
For threshold: 0.4
Micro-average quality numbers
Precision: 0.5842, Recall: 0.6702, F1-measure: 0.6243
For threshold: 0.5
Micro-average quality numbers
Precision: 0.6359, Recall: 0.5858, F1-measure: 0.6098
For threshold: 0.6
Micro-average quality numbers
Precision: 0.6993, Recall: 0.4707, F1-measure: 0.5626
For threshold: 0.7
Micro-average quality numbers
Precision: 0.7520, Recall: 0.3383, F1-measure: 0.4667
For threshold: 0.8
Micro-average quality numbers
Precision: 0.7863, Recall: 0.2132, F1-measure: 0.3354
For threshold: 0.9
Micro-average quality numbers
Precision: 0.8987, Recall: 0.1016, F1-measure: 0.1825