Multi-output Regression Example with Keras LSTM Network in Python

   Multi-output regression data can be fitted and predicted by the LSTM network model in Keras deep learning API. This type of data contains more than one output value for given input data.
LSTM (Long Short-Term Memory) network is a type of recurrent neural network and used to analyze sequence data. In this tutorial, we'll briefly learn how to fit and predict multi-output regression data with Keras LSTM model.
The post covers:
  1. Preparing the data
  2. Defining and fitting the model
  3. Predicting and visualizing the results
  4. Source code listing
We'll start by loading the required libraries of Python and Keras API for this tutorial.

from keras.models import Sequential
from keras.layers import Dense, LSTM
from numpy import array
from numpy.random import uniform
from numpy import hstack
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

Preparing the data

We'll create a multi-output dataset for this tutorial. It is randomly generated data with a simple rule. You can check the logic of data generation in a function below. There are three inputs and two outputs in this dataset.

def create_data(n):
 x1 = array([i/100+uniform(-1,3) for i in range(n)]).reshape(n,1)
 x2 = array([i/100+uniform(-3,5)+2 for i in range(n)]).reshape(n,1)
 x3 = array([i/100+uniform(-6,5)-3 for i in range(n)]).reshape(n,1)

 y1= [x1[i]-x2[i]+x3[i]+uniform(-2,2) for i in range(n)]
 y2= [x1[i]+x2[i]-x3[i]+5+uniform(-1,3) for i in range(n)]
 X = hstack((x1, x2, x3))
 Y = hstack((y1, y2))
 return X, Y

Now, we'll set n sample numbers and generate the dataset. You can draw X or Y data in a plot to check it visually.

x, y = create_data(n=400)
 
plt.plot(y)
plt.show()

Knowing the input data dimension is important when we build the neural network model. The defined input data dimension in a model needs to match with the dimension training input data. We'll check the x data dimensions.

print(x.shape)
(400, 3)

The LSTM model input dimension requires the third dimension that will be the number of the single input row. We'll reshape the x data.

x = x.reshape(x.shape[0], x.shape[1], 1)
print("x:", x.shape, "y:", y.shape)
(400, 3, 1) 

Next, we'll define the input and output data dimensions from x and data.

in_dim = (x.shape[1], x.shape[2])
out_dim = y.shape[1]
 
print(in_dim)
(3, 1) 
print(out_dim)
2 

Now, we can split the data into the train and test parts.

xtrain, xtest, ytrain, ytest=train_test_split(x, y, test_size=0.15)


Defining and fitting the model

We'll start by defining the sequential model. The sequential model contains LSTM layers with ReLU activations, Dense output layer,  and Adam optimizer with MSE loss function. We'll set the input dimension in the first layer and output dimension in the last layer of the model.

model = Sequential()
model.add(LSTM(64, input_shape=in_dim, activation="relu"))
model.add(Dense(out_dim))
model.compile(loss="mse", optimizer="adam"
 
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm_5 (LSTM)                (None, 64)                16896     
_________________________________________________________________
dense_364 (Dense)            (None, 2)                 130       
=================================================================
Total params: 17,026
Trainable params: 17,026
Non-trainable params: 0
_________________________________________________________________ 

We'll fit the model with train data.
 
model.fit(xtrain, ytrain, epochs=100, batch_size=12, verbose=0)


Predicting and visualizing the results

Finally, we'll predict test data and check the mean squared error rate.

ypred = model.predict(xtest)
 
print("y1 MSE:%.4f" % mean_squared_error(ytest[:,0], ypred[:,0]))
print("y2 MSE:%.4f" % mean_squared_error(ytest[:,1], ypred[:,1])) 

The result can be visualized as below.

x_ax = range(len(xtest))
plt.title("LSTM multi-output prediction")
plt.scatter(x_ax, ytest[:,0],  s=6, label="y1-test")
plt.plot(x_ax, ypred[:,0], label="y1-pred")
plt.scatter(x_ax, ytest[:,1],  s=6, label="y2-test")
plt.plot(x_ax, ypred[:,1], label="y2-pred")
plt.legend()
plt.show()

 
   In this tutorial, we've briefly learned how to fit and predict multi-output regression data with the keras LSTM network model. The full source code is listed below.


Source code listing

from keras.models import Sequential
from keras.layers import Dense, LSTM
from numpy import array
from numpy.random import uniform
from numpy import hstack
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

def create_data(n):
 x1 = array([i/100+uniform(-1,3) for i in range(n)]).reshape(n,1)
 x2 = array([i/100+uniform(-3,5)+2 for i in range(n)]).reshape(n,1)
 x3 = array([i/100+uniform(-6,5)-3 for i in range(n)]).reshape(n,1)

 y1= [x1[i]-x2[i]+x3[i]+uniform(-2,2) for i in range(n)]
 y2= [x1[i]+x2[i]-x3[i]+5+uniform(-1,3) for i in range(n)]
 X = hstack((x1, x2, x3))
 Y = hstack((y1, y2))
 return X, Y

x, y = create_data(n=400)
plt.plot(y)
plt.show()

print(x.shape)

x = x.reshape(x.shape[0], x.shape[1], 1)
print("x:", x.shape, "y:", y.shape)

in_dim = (x.shape[1], x.shape[2])
out_dim = y.shape[1]
print(in_dim)
print(out_dim)

xtrain, xtest, ytrain, ytest=train_test_split(x, y, test_size=0.15)
print("xtrain:", xtrain.shape, "ytrian:", ytrain.shape)

model = Sequential()
model.add(LSTM(64, input_shape=in_dim, activation="relu"))
model.add(Dense(out_dim))
model.compile(loss="mse", optimizer="adam")
model.summary()

model.fit(xtrain, ytrain, epochs=100, batch_size=12, verbose=0)

ypred = model.predict(xtest)
print("y1 MSE:%.4f" % mean_squared_error(ytest[:,0], ypred[:,0]))
print("y2 MSE:%.4f" % mean_squared_error(ytest[:,1], ypred[:,1]))

x_ax = range(len(xtest))
plt.title("LSTM multi-output prediction")
plt.scatter(x_ax, ytest[:,0],  s=6, label="y1-test")
plt.plot(x_ax, ypred[:,0], label="y1-pred")
plt.scatter(x_ax, ytest[:,1],  s=6, label="y2-test")
plt.plot(x_ax, ypred[:,1], label="y2-pred")
plt.legend()
plt.show()


No comments:

Post a Comment