Classification Example with PyTorch

     Classification tasks are fundamental in machine learning, involving the categorization of input data into distinct classes or categories based on their features. In this tutorial, we'll learn how to implement data classification using PyTorch deep learning framework.

    We'll cover the following topics:

1. Introduction to classification
   
2. Preparing data
   
3. Building the classifier model
4. Training the model
5. Prediction and accuracy check
   
6. Conclusion 
7. Source code listing
   

     Let's get started.

    Please note that this tutorial provides a basic understanding of implementing data classification with PyTorch. Keep in mind that parameters and model definitions may need adjustments when working with larger datasets.

 

Introduction to classification

    Classification is a supervised learning technique in machine learning and statistics used to categorize data points into predefined classes or categories based on their features. It involves training a model on a labeled dataset, where each data point is associated with a specific class or category. The goal of classification is to learn a mapping from input features to class labels, allowing the model to make accurate predictions on unseen data.

    In classification tasks, the input data is typically represented as a set of features, and the output is a discrete class label or category. The model learns patterns and relationships in the input data to classify new instances into one of the predefined classes. Common examples of classification tasks include spam email detection, sentiment analysis, image recognition, and medical diagnosis.

 

Preparing data

    We'll begin by loading the necessary libraries for this tutorial.

 

import torch

import torch.nn as nn

from torch.utils.data import TensorDataset, DataLoader

from sklearn.datasets import load_iris

from sklearn.model_selection import train_test_split

from sklearn.metrics import classification_report

 

    We use the Iris dataset and load it using the load_iris() function from scikit-learn. We check the size of the dataset, which contains 150 samples with 4 features. The number of features is important for defining the input size of our model later on.

 

# Load the Iris dataset

iris = load_iris()

X, y = iris.data, iris.target

 

print(X.shape)

(150, 4) 

 

    Next, we define model parameters and hyperparameters.

• input_size defines the input size of the data X. 
• hidden_size is size of hidden layer. The hidden layer is an intermediate layer between the input and output layers.
• num_classes is the number of classes in target variable y. We can calculate it with len(set(y)).
• learning_rate defines the learning rate for the optimization algorithm used to train the neural network. 
• batch_size is the batch size for mini-batch gradient descent during training. 
• num_epochs specifies the number of epochs or iterations over the entire dataset during training.
  

 

# Define model parameters

input_size = X.shape[1]

hidden_size = 100

num_classes = len(set(y))

learning_rate = 0.01

batch_size = 32

num_epochs = 100

 

    Then we split data into train and test parts. Here, we use 20 percent of data as test data.

 

# Split the data into train and test sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) 

     The input data needs to be converted to PyTorch tensors. Below, we convert the training data into tensors and create PyTorch dataset and DataLoader for train set.

 

# Convert the data into PyTorch tensors

X_train_tensor = torch.tensor(X_train, dtype=torch.float32)

y_train_tensor = torch.tensor(y_train, dtype=torch.long)

# Create PyTorch datasets and dataloaders for train

train_dataset = TensorDataset(X_train_tensor, y_train_tensor)

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True) 

 

Building the classifier model

     We'll define a Classifier class using PyTorch's nn.Linear module. The model class consists of the first fully connected linear layer, ReLU activation function, and the second fully connected linear layer.
    The forward() method is for the forward pass through the neural network. It takes input and output tensor data.
    We define the train() method for training the model. This method requires a DataLoader for the training dataset, a loss function, an optimizer, and the number of training epochs.
    The predict() method is used for prediction.  

 

# Define the neural network model

class Classifier(nn.Module):

def __init__(self, input_size, hidden_size, num_classes):

super(NeuralNetwork, self).__init__()

# Define the first fully connected layer

self.fc1 = nn.Linear(input_size, hidden_size)

# Define the ReLU activation function

self.relu = nn.ReLU()

self.fc2 = nn.Linear(hidden_size, num_classes)

# Forward pass through the neural network.

def forward(self, x):

# Pass the input through the first fully connected layer

out = self.fc1(x)

# Apply the ReLU activation function

out = self.relu(out)

# Pass the output through the second fully connected layer

out = self.fc2(out)

return out

# Define functions for training

def train(self, train_loader, criterion, optimizer, num_epochs):

# Iterate over each epoch

for epoch in range(num_epochs):

# Iterate over each batch in the training data

for inputs, labels in train_loader:

# Zero the gradients

optimizer.zero_grad()

# Forward pass: compute predicted outputs by passing inputs to the model

outputs = self(inputs)

# Calculate the loss

loss = criterion(outputs, labels)

# Backward pass: compute gradient of the loss with respect to model parameters

loss.backward()

# Update the parameters

optimizer.step()

    

# Method for prediction

def predict(self, inputs):

with torch.no_grad():

# Forward pass through the model

outputs = self(inputs)

# Get the predicted class labels

_, predicted = torch.max(outputs, 1)

return predicted

 

 

Training the model

    Next, we instantiate the Classifier model, providing the input size, hidden layer size, and number of classes. This creates a neural network model with the specified architecture. 

    The loss function is defined to compute the error between the predicted and actual class labels. The nn.CrossEntropyLoss() is commonly used for classification tasks where the output can belong to multiple classes. We use stochastic gradient descent (torch.optim.SGD) as the optimization algorithm with a specified learning rate (lr), which is responsible for updating the model parameters during training to minimize the loss.

 

# Train the model

model = Classifier(input_size, hidden_size, num_classes)

criterion = nn.CrossEntropyLoss()

optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

model.train(train_loader, criterion, optimizer, num_epochs)

 

Prediction and accuracy check

    After training, we predict the test data using the trained model. First, we convert the test data to PyTorch tensor type. Then, we pass this tensor to the trained model for prediction. The returned prediction data is converted to a numpy array type for further analysis.

    Next, we calculate the prediction accuracy by comparing the predicted labels with the actual labels from the test set. The accuracy is computed as the ratio of correctly predicted samples to the total number of samples in the test set. Finally, we print the accuracy and generate a classification report to evaluate the performance of the model on the test data. The classification report provides metrics such as precision, recall, F1-score, and support for each class label, offering insights into the model's performance across different classes.

 

# Convert X_test to PyTorch tensor

X_test_tensor = torch.tensor(X_test, dtype=torch.float32)

# Predict using the trained model

y_pred = model.predict(X_test_tensor)

# Print predictions and accuracy

y_pred_np = np.array(y_pred)

# Calculate accuracy

correct = sum(y_test == y_pred_np)

accuracy = correct / len(y_test)

print("Accuracy:", accuracy)

print(classification_report(y_test, y_pred_np))

 

The result is as follows.

 

Accuracy: 0.9666666666666667
precision recall f1-score support

0 1.00 1.00 1.00 10
1 1.00 0.89 0.94 9
2 0.92 1.00 0.96 11

accuracy 0.97 30
macro avg 0.97 0.96 0.97 30
weighted avg 0.97 0.97 0.97 30 

 

Conclusion

 

    In this tutorial, we learned how to classify data with using PyTorch neural network model. We used Iris dataset, built a linear regression model, trained it on the training data, and evaluated its performance on the test data. The full source code is listed below. 

 

Source code listing

 

import torch.nn as nn

from torch.utils.data import TensorDataset, DataLoader

from sklearn.datasets import load_iris

from sklearn.model_selection import train_test_split

from sklearn.metrics import classification_report

import numpy as np

# Load the Iris dataset

iris = load_iris()

X, y = iris.data, iris.target

print(X.shape)

# Define model parameters

input_size = X.shape[1]

hidden_size = 100

num_classes = len(set(y))

learning_rate = 0.01

batch_size = 32

num_epochs = 100

# Split the data into train and test sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Convert the data into PyTorch tensors

X_train_tensor = torch.tensor(X_train, dtype=torch.float32)

y_train_tensor = torch.tensor(y_train, dtype=torch.long)

# Create PyTorch datasets and dataloaders for train and test sets

train_dataset = TensorDataset(X_train_tensor, y_train_tensor)

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# Define the neural network model

class NeuralNetwork(nn.Module):

def __init__(self, input_size, hidden_size, num_classes):

super(NeuralNetwork, self).__init__()

# Define the first fully connected layer

self.fc1 = nn.Linear(input_size, hidden_size)

# Define the ReLU activation function

self.relu = nn.ReLU()

self.fc2 = nn.Linear(hidden_size, num_classes)

# Forward pass through the neural network.

def forward(self, x):

# Pass the input through the first fully connected layer

out = self.fc1(x)

# Apply the ReLU activation function

out = self.relu(out)

# Pass the output through the second fully connected layer

out = self.fc2(out)

return out

# Define functions for training

def train(self, train_loader, criterion, optimizer, num_epochs):

# Iterate over each epoch

for epoch in range(num_epochs):

# Iterate over each batch in the training data

for inputs, labels in train_loader:

# Zero the gradients

optimizer.zero_grad()

# Forward pass: compute predicted outputs by passing inputs to the model

outputs = self(inputs)

# Calculate the loss

loss = criterion(outputs, labels)

# Backward pass: compute gradient of the loss with respect to model parameters

loss.backward()

# Update the parameters

optimizer.step()

# Method for prediction

def predict(self, inputs):

with torch.no_grad():

# Forward pass through the model

outputs = self(inputs)

# Get the predicted class labels

_, predicted = torch.max(outputs, 1)

return predicted

# Train the model

model = NeuralNetwork(input_size, hidden_size, num_classes)

criterion = nn.CrossEntropyLoss()

optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

model.train(train_loader, criterion, optimizer, num_epochs)

# Convert X_test to PyTorch tensor

X_test_tensor = torch.tensor(X_test, dtype=torch.float32)

# Predict using the trained model

y_pred = model.predict(X_test_tensor)

# Print predictions and accuracy

y_pred_np = np.array(y_pred)

# Calculate accuracy

correct = sum(y_test == y_pred_np)

accuracy = correct / len(y_test)

print("Accuracy:", accuracy)

print(classification_report(y_test, y_pred_np))

 

References: