Pages

RNN Example with Keras SimpleRNN in Python

Recurrent Neural Network models can be easily built in a Keras API.  In this tutorial, we'll learn how to build an RNN model with a keras SimpleRNN() layer. For more information about it, please refer to this link.
The post covers:
• Generating sample dataset
• Preparing data (reshaping)
• Building a model with SimpleRNN
• Predicting and plotting results

```import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN```

Generating sample dataset

For this tutorial, we'll generate simple sequence data.

```N = 1000
Tp = 800

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)
```        0
0  0.224671
1  0.490856
2  0.567755
3  0.974433
4  0.805705 ```
` `
```plt.plot(df)
plt.show() ```
` `

Next, we'll split 'df' dataset into a training and test parts.

```values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]
```

Preparing data (reshaping)

RNN model requires a step value that contains n number of elements as an input sequence. Here, we define it as a 'step'. This is an important part of RNN so let's see an example:
x has the following sequence data.
x = [1,2,3,4,5,6,7,8,9,10]
for step=1, x input  and its y prediction become:
x  y
1  2
2  3
3  4
4  5
..
9  10
for step=3, x and y contain:
x         y
1,2,3   4
2,3,4   5
3,4,5   6
4,5,6   7
...
7,8,9   10

As you have noticed the sizes of x input and y output become different. We'll fix it by adding step size into the training and test data.

`step = 4`
```# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))```

Next, we'll convert test and train data into the matrix with step value as it has shown above example.

```# convert into dataset matrix
def convertToMatrix(data, step):
X, Y =[], []
for i in range(len(data)-step):
d=i+step
X.append(data[i:d,])
Y.append(data[d,])
return np.array(X), np.array(Y)
```

```trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)```

Finally, we'll reshape trainX and testX to fit with the Keras model. RNN model requires three-dimensional input data. You can see the shape of testX below.

```trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))```
```trainX.shape
(800, 1, 4) ```
` `

Building a model with SimpleRNN

Next, we create the keras Sequential model.

```# SimpleRNN model
model = Sequential()
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
simple_rnn_26 (SimpleRNN)    (None, 32)                1184
_________________________________________________________________
dense_50 (Dense)             (None, 8)                 264
_________________________________________________________________
dense_51 (Dense)             (None, 1)                 9
=================================================================
Total params: 1,457
Trainable params: 1,457
Non-trainable params: 0
_________________________________________________________________
```

Predicting and plotting the result

We'll fit model with trainX data and predict testX data.

```model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)```

Next, we'll check the loss

```trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)```
`0.35334232926368714 `

Finally, we check the result in a plot. A vertical line in a plot identifies a splitting point between the training and the test part.

```index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show() ```

In this post, we've learned how to fit and predict sequence data with the keras SimpleRNN model. The full source code is listed below.

```import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN

# convert into dataset matrix
def convertToMatrix(data, step):
X, Y =[], []
for i in range(len(data)-step):
d=i+step
X.append(data[i:d,])
Y.append(data[d,])
return np.array(X), np.array(Y)

step = 4
N = 1000
Tp = 800

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)

plt.plot(df)
plt.show()

values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]

# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))

trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

model = Sequential()
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()

model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)

trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)

index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show()```

1. Hey,

Nice example, it was helpful. How would it be if the input data consisted of many features (let's say 40) and not just one ?

1. You are welcome! You need to create combined X array data (contains all features x1, x2, ..) for your training and prediction. It goes like this;
x1, x2, y
2, 3, 3
3, 4, 4
2, 4, 4
3, 5, 5
4, 6, 6

Here, each window contains 3 elements of both x1 and x2 series.
2, 3,
3, 4,
2, 4, =>4

3, 4,
2, 4,
3, 5, => 5

2, 4,
3, 5,
4, 6, => 6

2. Hello!

Very good example, it showed step by step how to implement a RNN. I am struggling to reuse your knowledge and build a Jordan network.

I am attempting to translate your Sequential to Functional API but summary shows different network. :(

This is what I am doing:

visible = Input(shape=(None, step))
rnn = SimpleRNN(units=32, input_shape=(1,step))(visible)
hidden = Dense(8, activation='relu')(rnn)
output = Dense(1)(hidden)
_model = Model(inputs=visible, outputs=output)
_model.compile(loss='mean_squared_error', optimizer='rmsprop')
_model.summary()

By using same data input, I can have some result, but then, when predicting, I am not sure how Tensorflow does its recurrence. I would like to use only one output as input, then, what should I change?

Could you help me out, please?

3. Hello again!

I am trying very hard to understand how I build a RNN with the following features
1. x1, x2 and x3 are input signals that are measurements.
2. o1, o2 are outputs from the last prediction of the NN and o is the actual output

x1, x2, x3, o1, o2 --> o
2, 3, 3, 10, 9, 11,
3, 4, 4, 11, 10, 12,
2, 4, 4, 12, 11, 13,
3, 5, 5, 13, 12, 14,
4, 6, 6, 14, 13, 15,

3. how do I train and predict?