RNN Example with Keras SimpleRNN in Python


   Recurrent Neural Network models can be easily built in a Keras API.  In this tutorial, we'll learn how to build an RNN model with a keras SimpleRNN() layer. For more information about it, please refer to this link.
   The post covers:
  • Generating sample dataset
  • Preparing data (reshaping)
  • Building a model with SimpleRNN
  • Predicting and plotting results
We'll start by loading the required libraries.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN

Generating sample dataset

For test purpose, we'll generate simple sequence data.

N = 1000    
Tp = 800    

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)
df.head()
        0
0  0.224671
1  0.490856
2  0.567755
3  0.974433
4  0.805705 
 
plt.plot(df)
plt.show() 
 


Next, we'll split 'df' dataset into a training and test parts.

values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]


Preparing data (reshaping)

RNN model requires a step value that contains n number of elements as an input sequence. Here, we define it as a 'step'. This is an important part of RNN so let's see an example:
x has the following sequence data.
x = [1,2,3,4,5,6,7,8,9,10]
for step=1, x input  and its y prediction becomes:
x  y
1  2
2  3
3  4
4  5
..
9  10
for step=3, x and y contain:
x         y
1,2,3   4
2,3,4   5
3,4,5   6
4,5,6   7
...
7,8,9   10

As you have noticed the sizes of x input and y output become different. We'll fix it by adding step size into the training and test data.

step = 4
# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))

Next, we'll convert test and train data into the matrix with step value as it has shown above example.

# convert into dataset matrix
def convertToMatrix(data, step):
 X, Y =[], []
 for i in range(len(data)-step):
  d=i+step  
  X.append(data[i:d,])
  Y.append(data[d,])
 return np.array(X), np.array(Y)

trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)

Finally, we'll reshape trainX and testX to fit with Keras model. RNN model requires three-dimensional input data. You can see the shape of testX below.

trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))
trainX.shape
(800, 1, 4) 
 

Building a model with SimpleRNN

Next, we create the keras Sequential model.

# SimpleRNN model
model = Sequential()
model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
model.add(Dense(8, activation="relu")) 
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn_26 (SimpleRNN)    (None, 32)                1184      
_________________________________________________________________
dense_50 (Dense)             (None, 8)                 264       
_________________________________________________________________
dense_51 (Dense)             (None, 1)                 9         
=================================================================
Total params: 1,457
Trainable params: 1,457
Non-trainable params: 0
_________________________________________________________________

Predicting and plotting result

We'll fit model with trainX data and predict testX data.

model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)

Next, we'll check the loss

trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)
0.35334232926368714 


Finally, we check the result in a plot. A vertical line in a plot identifies a splitting point between training and test part.

index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show() 



    In this post, we've learned how to fit and predict sequence data with keras SimpleRNN model. The full source code is listed below.
   Thank you for reading!

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN

# convert into dataset matrix
def convertToMatrix(data, step):
 X, Y =[], []
 for i in range(len(data)-step):
  d=i+step  
  X.append(data[i:d,])
  Y.append(data[d,])
 return np.array(X), np.array(Y)

step = 4
N = 1000    
Tp = 800    

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)
df.head()

plt.plot(df)
plt.show()

values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]

# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))
 
trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

model = Sequential()
model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
model.add(Dense(8, activation="relu")) 
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()

model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)

trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)

index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show()

2 comments:
  1. Hey,

    Nice example, it was helpful. How would it be if the input data consisted of many features (let's say 40) and not just one ?

    ReplyDelete
    Replies
    1. You are welcome! You need to create combined X array data (contains all features x1, x2, ..) for your training and prediction. It goes like this;
      x1, x2, y
      2, 3, 3
      3, 4, 4
      2, 4, 4
      3, 5, 5
      4, 6, 6

      Here, each window contains 3 elements of both x1 and x2 series.
      2, 3,
      3, 4,
      2, 4, =>4

      3, 4,
      2, 4,
      3, 5, => 5

      2, 4,
      3, 5,
      4, 6, => 6

      Delete