## Pages

### RNN Example with Keras SimpleRNN in Python

Recurrent Neural Network models can be easily built in a Keras API.  In this tutorial, we'll learn how to build an RNN model with a keras SimpleRNN() layer. For more information about it, please refer to this link.
The post covers:
• Generating sample dataset
• Preparing data (reshaping)
• Building a model with SimpleRNN
• Predicting and plotting results
We'll start by loading the required libraries.

```import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN```

Generating sample dataset

For this tutorial, we'll generate simple sequence data.

```N = 1000
Tp = 800

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)
df.head()```
```        0
0  0.224671
1  0.490856
2  0.567755
3  0.974433
4  0.805705 ```
` `
```plt.plot(df)
plt.show() ```
` `

Next, we'll split 'df' dataset into a training and test parts.

```values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]
```

Preparing data (reshaping)

RNN model requires a step value that contains n number of elements as an input sequence. Here, we define it as a 'step'. This is an important part of RNN so let's see an example:
x has the following sequence data.
x = [1,2,3,4,5,6,7,8,9,10]
for step=1, x input  and its y prediction become:
x  y
1  2
2  3
3  4
4  5
..
9  10
for step=3, x and y contain:
x         y
1,2,3   4
2,3,4   5
3,4,5   6
4,5,6   7
...
7,8,9   10

As you have noticed the sizes of x input and y output become different. We'll fix it by adding step size into the training and test data.

`step = 4`
```# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))```

Next, we'll convert test and train data into the matrix with step value as it has shown above example.

```# convert into dataset matrix
def convertToMatrix(data, step):
X, Y =[], []
for i in range(len(data)-step):
d=i+step
X.append(data[i:d,])
Y.append(data[d,])
return np.array(X), np.array(Y)
```

```trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)```

Finally, we'll reshape trainX and testX to fit with the Keras model. RNN model requires three-dimensional input data. You can see the shape of testX below.

```trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))```
```trainX.shape
(800, 1, 4) ```
` `

Building a model with SimpleRNN

Next, we create the keras Sequential model.

```# SimpleRNN model
model = Sequential()
model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
model.add(Dense(8, activation="relu"))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
simple_rnn_26 (SimpleRNN)    (None, 32)                1184
_________________________________________________________________
dense_50 (Dense)             (None, 8)                 264
_________________________________________________________________
dense_51 (Dense)             (None, 1)                 9
=================================================================
Total params: 1,457
Trainable params: 1,457
Non-trainable params: 0
_________________________________________________________________
```

Predicting and plotting the result

We'll fit model with trainX data and predict testX data.

```model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)```

Next, we'll check the loss

```trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)```
`0.35334232926368714 `

Finally, we check the result in a plot. A vertical line in a plot identifies a splitting point between the training and the test part.

```index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show() ```

In this post, we've learned how to fit and predict sequence data with the keras SimpleRNN model. The full source code is listed below.
Thank you for reading!

```import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN

# convert into dataset matrix
def convertToMatrix(data, step):
X, Y =[], []
for i in range(len(data)-step):
d=i+step
X.append(data[i:d,])
Y.append(data[d,])
return np.array(X), np.array(Y)

step = 4
N = 1000
Tp = 800

t=np.arange(0,N)
x=np.sin(0.02*t)+2*np.random.rand(N)
df = pd.DataFrame(x)
df.head()

plt.plot(df)
plt.show()

values=df.values
train,test = values[0:Tp,:], values[Tp:N,:]

# add step elements into train and test
test = np.append(test,np.repeat(test[-1,],step))
train = np.append(train,np.repeat(train[-1,],step))

trainX,trainY =convertToMatrix(train,step)
testX,testY =convertToMatrix(test,step)
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

model = Sequential()
model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu"))
model.add(Dense(8, activation="relu"))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='rmsprop')
model.summary()

model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2)
trainPredict = model.predict(trainX)
testPredict= model.predict(testX)
predicted=np.concatenate((trainPredict,testPredict),axis=0)

trainScore = model.evaluate(trainX, trainY, verbose=0)
print(trainScore)

index = df.index.values
plt.plot(index,df)
plt.plot(index,predicted)
plt.axvline(df.index[Tp], c="r")
plt.show()```

#### 9 comments:

1. Hey,

Nice example, it was helpful. How would it be if the input data consisted of many features (let's say 40) and not just one ?

1. You are welcome! You need to create combined X array data (contains all features x1, x2, ..) for your training and prediction. It goes like this;
x1, x2, y
2, 3, 3
3, 4, 4
2, 4, 4
3, 5, 5
4, 6, 6

Here, each window contains 3 elements of both x1 and x2 series.
2, 3,
3, 4,
2, 4, =>4

3, 4,
2, 4,
3, 5, => 5

2, 4,
3, 5,
4, 6, => 6

2. Hello!

Very good example, it showed step by step how to implement a RNN. I am struggling to reuse your knowledge and build a Jordan network.

I am attempting to translate your Sequential to Functional API but summary shows different network. :(

This is what I am doing:

visible = Input(shape=(None, step))
rnn = SimpleRNN(units=32, input_shape=(1,step))(visible)
hidden = Dense(8, activation='relu')(rnn)
output = Dense(1)(hidden)
_model = Model(inputs=visible, outputs=output)
_model.compile(loss='mean_squared_error', optimizer='rmsprop')
_model.summary()

By using same data input, I can have some result, but then, when predicting, I am not sure how Tensorflow does its recurrence. I would like to use only one output as input, then, what should I change?

Could you help me out, please?

3. Hello again!

I am trying very hard to understand how I build a RNN with the following features
1. x1, x2 and x3 are input signals that are measurements.
2. o1, o2 are outputs from the last prediction of the NN and o is the actual output

x1, x2, x3, o1, o2 --> o
2, 3, 3, 10, 9, 11,
3, 4, 4, 11, 10, 12,
2, 4, 4, 12, 11, 13,
3, 5, 5, 13, 12, 14,
4, 6, 6, 14, 13, 15,

3. how do I train and predict?

1. This may help you
https://www.datatechnotes.com/2020/01/multi-output-multi-step-regression.html

4. How to tell if this network is Elman or Jordan? I mean, these two are simple recurrent networks, right?
In the Keras documentation it is only explained that are "Fully-connected RNN where the output is to be fed back to input".

5. How does one modify your code if your data has several features, not just one?

6. Hi, nice example - I am trying to understand nns... why did you put a Dense layer with 8 units after the RNN?

7. Hi, Thank you for this simple yet meaningful example. I have a request: How to pictorially show the SimpleRNN model used here? If you can give some info or link or something in this regard, it will be very helpful.
Regards, Manoranjan Dash