LSTM (Long Short-Term Memory) network is a type of recurrent neural network and used to analyze sequence data. In this tutorial, we'll briefly learn how to fit and predict multioutput regression data with Keras LSTM model.
The post covers:
- Preparing the data
- Defining and fitting the model
- Predicting and visualizing the results
- Source code listing
from keras.models import Sequential from keras.layers import Dense, LSTM from numpy import array from numpy.random import uniform from numpy import hstack import matplotlib.pyplot as plt from sklearn.model_selection import train_test_split from sklearn.metrics import mean_squared_error
Preparing the data
We'll create a multioutput dataset for this tutorial. It is randomly generated data with a simple rule. You can check the logic of data generation in a function below. There are three inputs and two outputs in this dataset.
def create_data(n): x1 = array([i/100+uniform(-1,3) for i in range(n)]).reshape(n,1) x2 = array([i/100+uniform(-3,5)+2 for i in range(n)]).reshape(n,1) x3 = array([i/100+uniform(-6,5)-3 for i in range(n)]).reshape(n,1) y1= [x1[i]-x2[i]+x3[i]+uniform(-2,2) for i in range(n)] y2= [x1[i]+x2[i]-x3[i]+5+uniform(-1,3) for i in range(n)] X = hstack((x1, x2, x3)) Y = hstack((y1, y2)) return X, Y
Now, we'll set n sample numbers and generate the dataset. You can draw X or Y data in a plot to check it visually.
x, y = create_data(n=400)
plt.plot(y) plt.show()
Knowing the input data dimension is important when we build the neural network model. The defined input data dimension in a model needs to match with the dimension training input data. We'll check the x data dimensions.
print(x.shape)
(400, 3)
The LSTM model input dimension requires the third dimension that will be the number of the single input row. We'll reshape the x data.
x = x.reshape(x.shape[0], x.shape[1], 1) print("x:", x.shape, "y:", y.shape)
(400, 3, 1)
Next, we'll define the input and output data dimensions from x and data.
in_dim = (x.shape[1], x.shape[2]) out_dim = y.shape[1]
print(in_dim)
(3, 1)
print(out_dim)
2
Now, we can split the data into the train and test parts.
xtrain, xtest, ytrain, ytest=train_test_split(x, y, test_size=0.15)
Defining and fitting the model
We'll start by defining the sequential model. The sequential model contains LSTM layers with ReLU activations, Dense output layer, and Adam optimizer with MSE loss function. We'll set the input dimension in the first layer and output dimension in the last layer of the model.
model = Sequential() model.add(LSTM(64, input_shape=in_dim, activation="relu")) model.add(Dense(out_dim)) model.compile(loss="mse", optimizer="adam")
model.summary()
_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm_5 (LSTM) (None, 64) 16896 _________________________________________________________________ dense_364 (Dense) (None, 2) 130 ================================================================= Total params: 17,026 Trainable params: 17,026 Non-trainable params: 0 _________________________________________________________________
We'll fit the model with train data.
model.fit(xtrain, ytrain, epochs=100, batch_size=12, verbose=0)
Predicting and visualizing the results
Finally, we'll predict test data and check the mean squared error rate.
ypred = model.predict(xtest)
print("y1 MSE:%.4f" % mean_squared_error(ytest[:,0], ypred[:,0])) print("y2 MSE:%.4f" % mean_squared_error(ytest[:,1], ypred[:,1]))
The result can be visualized as below.
x_ax = range(len(xtest)) plt.title("LSTM multi-output prediction") plt.scatter(x_ax, ytest[:,0], s=6, label="y1-test") plt.plot(x_ax, ypred[:,0], label="y1-pred") plt.scatter(x_ax, ytest[:,1], s=6, label="y2-test") plt.plot(x_ax, ypred[:,1], label="y2-pred") plt.legend() plt.show()
In this tutorial, we've briefly learned how to fit and predict multioutput regression data with the keras LSTM network model. The full source code is listed below.
Source code listing
from keras.models import Sequential from keras.layers import Dense, LSTM from numpy import array from numpy.random import uniform from numpy import hstack import matplotlib.pyplot as plt from sklearn.model_selection import train_test_split from sklearn.metrics import mean_squared_error def create_data(n): x1 = array([i/100+uniform(-1,3) for i in range(n)]).reshape(n,1) x2 = array([i/100+uniform(-3,5)+2 for i in range(n)]).reshape(n,1) x3 = array([i/100+uniform(-6,5)-3 for i in range(n)]).reshape(n,1) y1= [x1[i]-x2[i]+x3[i]+uniform(-2,2) for i in range(n)] y2= [x1[i]+x2[i]-x3[i]+5+uniform(-1,3) for i in range(n)] X = hstack((x1, x2, x3)) Y = hstack((y1, y2)) return X, Y x, y = create_data(n=400) plt.plot(y) plt.show() print(x.shape) x = x.reshape(x.shape[0], x.shape[1], 1) print("x:", x.shape, "y:", y.shape) in_dim = (x.shape[1], x.shape[2]) out_dim = y.shape[1]
print(in_dim)
print(out_dim) xtrain, xtest, ytrain, ytest=train_test_split(x, y, test_size=0.15) print("xtrain:", xtrain.shape, "ytrian:", ytrain.shape) model = Sequential() model.add(LSTM(64, input_shape=in_dim, activation="relu")) model.add(Dense(out_dim)) model.compile(loss="mse", optimizer="adam") model.summary() model.fit(xtrain, ytrain, epochs=100, batch_size=12, verbose=0) ypred = model.predict(xtest) print("y1 MSE:%.4f" % mean_squared_error(ytest[:,0], ypred[:,0])) print("y2 MSE:%.4f" % mean_squared_error(ytest[:,1], ypred[:,1])) x_ax = range(len(xtest))
plt.title("LSTM multi-output prediction")
plt.scatter(x_ax, ytest[:,0], s=6, label="y1-test") plt.plot(x_ax, ypred[:,0], label="y1-pred") plt.scatter(x_ax, ytest[:,1], s=6, label="y2-test") plt.plot(x_ax, ypred[:,1], label="y2-pred") plt.legend() plt.show()
No comments:
Post a Comment