Deep learning ——（5） Data imbalance class_weight

In the process of classifying images or other data , When the number of samples in each category is not equal or even very different —— The data is unbalanced , This situation will make model In the training process, one batch All the samples taken in belong to the same category , The features learned by samples of the same category are all equal , Caused on the training set loss Low value , But the accuracy of the verification set is not high .
You can calculate loss Add when the value is class_weight, The specific process is as follows ：

from sklearn.utils.class_weight import compute_class_weight
import numpy as np

classes=[190,90,121,57,28]

label = np.zeros(train_num)
for i in range(classes[0]):
    label[i]=0
for i in range(classes[1]):
    label[classes[0]+i]=1
for i in range(classes[2]):
    label[classes[0]+classes[1]+i]=2
for i in range(classes[3]):
    label[classes[0]+classes[1]+classes[2]+i]=3
for i in range(classes[4]):
    label[classes[0]+classes[1]+classes[2]+classes[3]+i]=4

class_weights=compute_class_weight('balanced',np.unique(label),label)
class_weights=torch.tensor(class_weights,dtype=torch.float) #  Data imbalance , stay loss Used in the calculation class_weight, Give each category a weight 
loss_function = nn.CrossEntropyLoss(class_weights)

Calculated after loss that will do , The above code is CPU There is no problem running on the computer , But put it in GPU Running up will report an error , as follows ：

RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'weight' in call to _thnn_nll_loss_forward

The reason for the error is that cpu Run up ,device Is directly cpu, Direct use tensor Just go . But in GPU Be sure to run loss Parameters in class_weight Load into cuda.

Solution

Add .to(device), for example ：

loss_function = nn.CrossEntropyLoss(class_weights.to(device))

886, Let yourself form the good habit of recording casually ~