## Pages

### Linear Regression with PyTorch

Linear regression is a fundamental supervised learning technique used for predicting a continuous target variable based on one or more input features. In this tutorial, we'll learn how to implement linear regression using PyTorch deep learning framework. We'll cover the following topics:

1. Introduction to linear regression
2. Preparing data
3. Building the linear regression model
4. Training the model
5. Evaluating the model
6. Conclusion
7. Source code listing

Let's get started.

Introduction to linear regression

Linear regression is a simple yet powerful statistical method used for modeling the relationship between two or more variables. It assumes a linear relationship between the input features and the target variable. The model aims to find the best-fitting line that minimizes the difference between the observed and predicted values.

Preparing data

import torch
import torch.nn as nn
import numpy as np
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt

We'll use a synthetic regression dataset generated using the make_regression function from scikit-learn. The dataset contains 400 samples with 2 input features and a noise level of 20.

# Generate synthetic regression data
x_data, y_data = datasets.make_regression(n_samples=400, n_features=2, noise=20, random_state=4)

Then we split data into train and test parts. Here, we use 20 percent of data as test data.

# Split data into training and testing sets
xtrain, xtest, ytrain, ytest = train_test_split(x_data, y_data, test_size=0.2)

The input data needs to be converted to PyTorch tensors. Below, we convert the training and test input data into tensors.

# Convert data to PyTorch tensors
X_train_ts = torch.from_numpy(xtrain.astype(np.float32))
y_train_ts = torch.from_numpy(ytrain.astype(np.float32)).view(-1, 1)
x_test_ts = torch.from_numpy(xtest.astype(np.float32))

Building the linear regression model

We'll define a simple linear regression class using PyTorch's nn.Linear module. The model class consists of a single linear layer without any activation function. Inside the constructor, a linear transformation layer (nn.Linear) is created and assigned to the self.linear attribute of the object. The forward method defines the forward pass of the model. It takes an input tensor x as input and passes it through the linear layer (self.linear) to compute the output.

# Define the linear regression model
class LinearRegressionModel(nn.Module):
def __init__(self, input_size, output_size):
super(LinearRegressionModel, self).__init__()
self.linear = nn.Linear(input_size, output_size)

def forward(self, x):
return self.linear(x)

Next, we define model parameters and hyperparameters.

Model parameters:

• input_size: This represents the number of features in the input data.
• output_size: This specifies the dimensionality of the output, which is usually 1 for regression tasks since we are predicting a single target variable.

Hyperparameters:

• learning_rate: The learning rate determines the step size at which the model parameters are updated during training.
• num_epochs: The number of epochs refers to the number of times the entire dataset is passed forward and backward through the model during training.

# Model parameters and hyperparameters
input_size = X_train_ts.shape[1]
output_size = 1
learning_rate = 0.01
num_epochs = 200

Training the model

Next, we'll train the linear regression model on the training data. We'll define the loss function (mean squared error) and the optimizer (Stochastic Gradient Descent).

# Instantiate the model, loss function, and optimizer
model = LinearRegressionModel(input_size, output_size)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

To train the model, the dataset is iterated for a fixed number of epochs. An epoch is one complete pass through the entire training dataset.

Inside the loop, we perform a forward pass through the model using the training data (X_train_ts). This generates predicted values (y_predicted) for the target variable. We then calculate the loss between the predicted values (y_predicted) and the actual target values (y_train_ts). The loss represents how far off the model's predictions are from the true values.

After computing the loss, we perform a backward pass through the model to compute gradients with respect to each parameter. This is done using the backward() method on the loss tensor. Then, we use the optimizer to update the model parameters based on these gradients, using the step() method.

We print the current value of the loss every 10 epochs to monitor the training progress and ensure that the loss is decreasing over time.

# Training loop
for epoch in range(num_epochs):
# Forward pass
y_predicted = model(X_train_ts)

# Compute loss
loss = criterion(y_predicted, y_train_ts)

# Backward pass and optimization
loss.backward()
optimizer.step()

# Print loss every 10 epochs
if (epoch + 1) % 10 == 0:
print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')

Evaluating the model

After training, we'll evaluate the model's performance on the test data. The torch.no_grad() is a context manager that ensures PyTorch operations do not track gradients inside this context manager. This is useful during model evaluation, where we do not need to compute gradients during inference. By disabling gradient tracking, we save memory and computational resources.

Then, we calculate the mean squared error and visualize the actual and predicted data on a graph.

# Evaluation
test_pred = model(x_test_ts).detach().numpy()

# Calculate Mean Squared Error
mse = mean_squared_error(ytest, test_pred)
print("MSE: %.4f" % mse)

# Plot actual vs. predicted
x = range(len(ytest))
plt.scatter(x, ytest, color='red', marker='o', label='Actual')
plt.plot(x, test_pred, color='blue', linewidth=2, label='Predicted')
plt.xlabel('X')
plt.ylabel('y')
plt.title('Linear Regression')
plt.legend()
plt.grid()
plt.show()

The result is as follows.

Epoch [10/200], Loss: 9233.3242
Epoch [20/200], Loss: 6646.3257
Epoch [30/200], Loss: 4822.3564
Epoch [40/200], Loss: 3533.9141
Epoch [50/200], Loss: 2622.0962
Epoch [60/200], Loss: 1975.6768
Epoch [70/200], Loss: 1516.6350
Epoch [80/200], Loss: 1190.1326
Epoch [90/200], Loss: 957.5449
Epoch [100/200], Loss: 791.6177
Epoch [110/200], Loss: 673.0816
Epoch [120/200], Loss: 588.2905
Epoch [130/200], Loss: 527.5629
Epoch [140/200], Loss: 484.0189
Epoch [150/200], Loss: 452.7618
Epoch [160/200], Loss: 430.3016
Epoch [170/200], Loss: 414.1467
Epoch [180/200], Loss: 402.5165
Epoch [190/200], Loss: 394.1366
Epoch [200/200], Loss: 388.0937
MSE: 401.0717

Conclusion

In this tutorial, we learned how to implement linear regression using PyTorch. We prepared a synthetic regression dataset, built a linear regression model, trained it on the training data, and evaluated its performance on the test data. The full source code is listed below.

Source code listing

import torch
import torch.nn as nn
import numpy as np
from sklearn import datasets
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
from sklearn.metrics import mean_squared_error

# Generate synthetic regression data
x_data, y_data = datasets.make_regression(n_samples=400, n_features=2, noise=20, random_state=4)

# Split data into training and testing sets
xtrain, xtest, ytrain, ytest = train_test_split(x_data, y_data, test_size=0.2)

# Convert data to PyTorch tensors
X_train_ts = torch.from_numpy(xtrain.astype(np.float32))
y_train_ts = torch.from_numpy(ytrain.astype(np.float32)).view(-1, 1)
x_test_ts = torch.from_numpy(xtest.astype(np.float32))

# Define the linear regression model
class LinearRegressionModel(nn.Module):
def __init__(self, input_size, output_size):
super(LinearRegressionModel, self).__init__()
self.linear = nn.Linear(input_size, output_size)

def forward(self, x):
return self.linear(x)

# Model parameters and hyperparameters
input_size = X_train_ts.shape[1]
output_size = 1
learning_rate = 0.01
num_epochs = 200

# Instantiate the model, loss function, and optimizer
model = LinearRegressionModel(input_size, output_size)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

# Training loop
for epoch in range(num_epochs):
# Forward pass
y_predicted = model(X_train_ts)

# Compute loss
loss = criterion(y_predicted, y_train_ts)

# Backward pass and optimization
loss.backward()
optimizer.step()

# Print loss every 10 epochs
if (epoch + 1) % 10 == 0:
print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')

# Evaluation
test_pred = model(x_test_ts).detach().numpy()

# Calculate Mean Squared Error
mse = mean_squared_error(ytest, test_pred)
print("MSE: %.4f" % mse)

# Plot actual vs. predicted
x = range(len(ytest))
plt.scatter(x, ytest, color='red', marker='o', label='Actual')
plt.plot(x, test_pred, color='blue', linewidth=2, label='Predicted')
plt.xlabel('X')
plt.ylabel('y')
plt.title('Linear Regression')
plt.legend()
plt.grid()
plt.show()

References: