PyTorch - Hello World - MNIST手寫數字辨識

對於 PyTorch 基礎操作差不多了解後，就是要來實作MNIST手寫數字辨識的練習。
且利用 MNIST 可以先初階的練習使用 Linear NN 跑一遍，接著再替換成卷積層(convnet)跑一遍。
中間再加些 regularization ，基本用法應該都能練習一遍，是個很好的練習。

1. Import Libraries

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt

2. 資料預處理

這邊使用 keras.datasets 直接載入 MNIST data，方便練習

from keras.datasets import mnist
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()

normalization

X_train = X_train.astype('float32') / 255
X_test = X_test.astype('float32') / 255

利用 sklearn，將 X_train 切出 0.2 比例的 Validation_data

features_train, features_test, targets_train, targets_test = train_test_split(X_train, Y_train, test_size = 0.2, random_state = 42)

接著將切好的data 通通轉成 torch 的 tensor 形式，
後面丟進 TensorDataset 需要符合 torch 的 tensor 形式。

featuresTrain = torch.from_numpy(features_train)
targetsTrain = torch.from_numpy(targets_train).type(torch.LongTensor) # data type is long

featuresTest = torch.from_numpy(features_test)
targetsTest = torch.from_numpy(targets_test).type(torch.LongTensor) # data type is long

torch.utils.data.TensorDataset(data_tensor, target_tensor)，可參閱 PyTorch 文檔，用意就是將input 數據與目標 output 打包。

# Pytorch train and test TensorDataset
train = torch.utils.data.TensorDataset(featuresTrain,targetsTrain)
test = torch.utils.data.TensorDataset(featuresTest,targetsTest)

超參數設定:

# Hyper Parameters
# batch_size, epoch and iteration
LR = 0.01
batch_size = 100
n_iters = 10000
num_epochs = n_iters / (len(features_train) / batch_size)
num_epochs = int(num_epochs)

torch.utils.data.TensorDatasetDataLoader(dataset, batch_size=1, shuffle=False,...)，可參閱 PyTorch 文檔，為數據加載器。組合數據集和採樣器，並在數據集上提供單進程或多進程迭代器。如需要打亂數據則將 shuffle=True。

# Pytorch DataLoader
train_loader = torch.utils.data.DataLoader(train, batch_size = batch_size, shuffle = True)
test_loader = torch.utils.data.DataLoader(test, batch_size = batch_size, shuffle = True)

3. 建立模型

先做簡單的練習，所以這邊只使用Linear NN，每層皆使用Relu激活函數。後面再練習CNN。

定義Model:

class Linear_NN_Model(nn.Module):
    def __init__(self, input_dim, output_dim):
        super(Linear_NN_Model, self).__init__()
        self.nn1 = nn.Linear(input_dim, 256)
        self.nn2 = nn.Linear(256, 128)
        self.fc = nn.Linear(128, output_dim)
    def forward(self, x):
        x = F.relu(self.nn1(x))
        x = F.relu(self.nn2(x))
        out = self.fc(x)
        return out

將模型 print 出，選擇優化器 Adam，loss function : CrossEntropyLoss() (在多分類任務中使用)

input_dim = 28*28 # size of image px*px
output_dim = 10  # labels 0,1,2,3,4,5,6,7,8,9
model = Linear_NN_Model(input_dim, output_dim)
print(model)
optimizer = torch.optim.Adam(model.parameters(), lr=LR)
loss_func = nn.CrossEntropyLoss()   # the target label is not one-hotted

4. 訓練模型

4-1. 先簡單跑過，這邊只記錄traing loss，並將結果畫出。

# Traning the Model
training_loss = []
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        # 1.Define variables
        train = Variable(images.view(-1, 28*28))
        labels = Variable(labels)
        # 2.Clear gradients
        optimizer.zero_grad()
        # 3.Forward propagation
        outputs = model(train)
        # 4.Calculate softmax and cross entropy loss
        loss = loss_func(outputs, labels)
        # 5.Calculate gradients
        loss.backward()
        # 6.Update parameters
        optimizer.step()
    # 7.store loss / epoch
    training_loss.append(loss.data)
    print('Train Epoch: {} Traing_Loss: {}'.format(epoch, loss.data))

Train Epoch: 0 Traing_Loss: 0.019337305799126625
...
...
...
Train Epoch: 19 Traing_Loss: 0.0003676938940770924

# visualization
plt.plot(range(epoch+1), training_loss, 'b-', label='Training loss')
plt.title('Training loss')
plt.xlabel('Number of epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()

4-2. 邊訓練邊預測，並將結果畫出。

我想知道在每一個epoch訓練模型過後，當下的驗證accuracy是如何，修改代碼如下:

從# Traning the Model以下進行修改：

# Traning the Model
training_loss = []
val_accuracy = []  #for store val_acc
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        # 1.Define variables
        train = Variable(images.view(-1, 28*28))
        labels = Variable(labels)
        # 2.Clear gradients
        optimizer.zero_grad()
        # 3.Forward propagation
        outputs = model(train)
        # 4.Calculate softmax and cross entropy loss
        loss = loss_func(outputs, labels)
        # 5.Calculate gradients
        loss.backward()
        # 6.Update parameters
        optimizer.step()
    # 7.store loss / epoch
    training_loss.append(loss.data)
    
    #Predictions / epoch
    correct = 0
    total = 0
    for images, labels in test_loader:
        # 1.Define variables
        test = Variable(images.view(-1, 28*28))
        # 2.Forward propagation
        outputs = model(test)
        # 3.Get predictions from the maximum value
        predicted = torch.max(outputs.data, 1)[1]
        # 4.Total number of labels
        total += len(labels)
        # 5.Total correct predictions
        correct += (predicted == labels).float().sum()
    #6.store val_acc / epoch
    accuracy = 100 * correct / float(total)
    val_accuracy.append(accuracy)
    print('Train Epoch: {}/{} Traing_Loss: {} val_accuracy: {:.6f}%'.format(epoch+1, num_epochs, loss.data , accuracy))

Train Epoch: 1/20 Traing_Loss: 0.1850169599056244 val_accuracy: 95.141670%
...
...
...
Train Epoch: 20/20 Traing_Loss: 0.14025935530662537 val_accuracy: 96.758331%

# visualization
plt.plot(range(epoch+1), training_loss, 'b-', label='Training loss')
plt.title('Training loss')
plt.xlabel('Number of epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()
plt.plot(range(epoch+1), val_accuracy, 'g-', label='Val_accuracy')
plt.title('Val_accuracy')
plt.xlabel('Number of epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.show()

4-3. 最終目標，將Train_loss,Train_acc,Val_loss,Val_acc 都畫出來，達到如 Keras 的 history 的功能。

從# Traning the Model以下進行修改：

# Traning the Model
#history-like list for store loss & acc value
training_loss = []
training_accuracy = []
validation_loss = []
validation_accuracy = []
for epoch in range(num_epochs):
    #training model & store loss & acc / epoch
    correct_train = 0
    total_train = 0
    for i, (images, labels) in enumerate(train_loader):
        # 1.Define variables
        train = Variable(images.view(-1, 28*28))
        labels = Variable(labels)
        # 2.Clear gradients
        optimizer.zero_grad()
        # 3.Forward propagation
        outputs = model(train)
        # 4.Calculate softmax and cross entropy loss
        train_loss = loss_func(outputs, labels)
        # 5.Calculate gradients
        train_loss.backward()
        # 6.Update parameters
        optimizer.step()
        # 7.Get predictions from the maximum value
        predicted = torch.max(outputs.data, 1)[1]
        # 8.Total number of labels
        total_train += len(labels)
        # 9.Total correct predictions
        correct_train += (predicted == labels).float().sum()
    #10.store val_acc / epoch
    train_accuracy = 100 * correct_train / float(total_train)
    training_accuracy.append(train_accuracy)
    # 11.store loss / epoch
    training_loss.append(train_loss.data)
    
    #evaluate model & store loss & acc / epoch
    correct_test = 0
    total_test = 0
    for images, labels in test_loader:
        # 1.Define variables
        test = Variable(images.view(-1, 28*28))
        # 2.Forward propagation
        outputs = model(test)
        # 3.Calculate softmax and cross entropy loss
        val_loss = loss_func(outputs, labels)
        # 4.Get predictions from the maximum value
        predicted = torch.max(outputs.data, 1)[1]
        # 5.Total number of labels
        total_test += len(labels)
        # 6.Total correct predictions
        correct_test += (predicted == labels).float().sum()
    #6.store val_acc / epoch
    val_accuracy = 100 * correct_test / float(total_test)
    validation_accuracy.append(val_accuracy)
    # 11.store val_loss / epoch
    validation_loss.append(val_loss.data)
    print('Train Epoch: {}/{} Traing_Loss: {} Traing_acc: {:.6f}% Val_Loss: {} Val_accuracy: {:.6f}%'.format(epoch+1, num_epochs, train_loss.data, train_accuracy, val_loss.data, val_accuracy))

# visualization
plt.plot(range(epoch+1), training_loss, 'b-', label='Training_loss')
plt.plot(range(epoch+1), validation_loss, 'g-', label='validation_loss')
plt.title('Training & Validation loss')
plt.xlabel('Number of epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()
plt.plot(range(epoch+1), training_accuracy, 'b-', label='Training_accuracy')
plt.plot(range(epoch+1), validation_accuracy, 'g-', label='Validation_accuracy')
plt.title('Val_accuracy')
plt.xlabel('Number of epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.show()

完成!!
可以看到訓練loss 比驗證loss低，訓練精度比驗證精度高。

大致上完成了，將以上代碼寫成def 方便後續使用，這邊略…。