The post covers:

- Generating sample dataset
- Preparing data (reshaping)
- Building a model with SimpleRNN
- Predicting and plotting results

import pandas as pd import numpy as np import matplotlib.pyplot as plt from keras.models import Sequential from keras.layers import Dense, SimpleRNN

**Generating sample dataset**

For this tutorial, we'll generate simple sequence data.

N = 1000 Tp = 800 t=np.arange(0,N) x=np.sin(0.02*t)+2*np.random.rand(N) df = pd.DataFrame(x) df.head()

0 0 0.224671 1 0.490856 2 0.567755 3 0.974433 4 0.805705

plt.plot(df) plt.show()

Next, we'll split 'df' dataset into a training and test parts.

values=df.values train,test = values[0:Tp,:], values[Tp:N,:]

**Preparing data (reshaping)**

RNN model requires a step value that contains n number of elements as an input sequence. Here, we define it as a 'step'. This is an important part of RNN so let's see an example:

x has the following sequence data.

x = [1,2,3,4,5,6,7,8,9,10]

for step=1, x input and its y prediction become:

x y

1 2

2 3

3 4

4 5

..

9 10

for step=3, x and y contain:

x y

1,2,3 4

2,3,4 5

3,4,5 6

4,5,6 7

...

7,8,9 10

As you have noticed the sizes of x input and y output become different. We'll fix it by adding step size into the training and test data.

step = 4

# add step elements into train and test test = np.append(test,np.repeat(test[-1,],step)) train = np.append(train,np.repeat(train[-1,],step))

Next, we'll convert test and train data into the matrix with step value as it has shown above example.

# convert into dataset matrix def convertToMatrix(data, step): X, Y =[], [] for i in range(len(data)-step): d=i+step X.append(data[i:d,]) Y.append(data[d,]) return np.array(X), np.array(Y)

trainX,trainY =convertToMatrix(train,step) testX,testY =convertToMatrix(test,step)

Finally, we'll reshape trainX and testX to fit with the Keras model. RNN model requires three-dimensional input data. You can see the shape of testX below.

trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1])) testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

trainX.shape (800, 1, 4)

**Building a model with SimpleRNN**

Next, we create the keras Sequential model.

# SimpleRNN model model = Sequential() model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu")) model.add(Dense(8, activation="relu")) model.add(Dense(1)) model.compile(loss='mean_squared_error', optimizer='rmsprop') model.summary() _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= simple_rnn_26 (SimpleRNN) (None, 32) 1184 _________________________________________________________________ dense_50 (Dense) (None, 8) 264 _________________________________________________________________ dense_51 (Dense) (None, 1) 9 ================================================================= Total params: 1,457 Trainable params: 1,457 Non-trainable params: 0 _________________________________________________________________

**Predicting and plotting the result**

We'll fit model with trainX data and predict testX data.

model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2) trainPredict = model.predict(trainX) testPredict= model.predict(testX) predicted=np.concatenate((trainPredict,testPredict),axis=0)

Next, we'll check the loss

trainScore = model.evaluate(trainX, trainY, verbose=0) print(trainScore)

0.35334232926368714

Finally, we check the result in a plot. A vertical line in a plot identifies a splitting point between the training and the test part.

index = df.index.values plt.plot(index,df) plt.plot(index,predicted) plt.axvline(df.index[Tp], c="r") plt.show()

In this post, we've learned how to fit and predict sequence data with the keras SimpleRNN model. The full source code is listed below.

Thank you for reading!

import pandas as pd import numpy as np import matplotlib.pyplot as plt from keras.models import Sequential from keras.layers import Dense, SimpleRNN # convert into dataset matrix def convertToMatrix(data, step): X, Y =[], [] for i in range(len(data)-step): d=i+step X.append(data[i:d,]) Y.append(data[d,]) return np.array(X), np.array(Y) step = 4 N = 1000 Tp = 800 t=np.arange(0,N) x=np.sin(0.02*t)+2*np.random.rand(N) df = pd.DataFrame(x) df.head() plt.plot(df) plt.show() values=df.values train,test = values[0:Tp,:], values[Tp:N,:] # add step elements into train and test test = np.append(test,np.repeat(test[-1,],step)) train = np.append(train,np.repeat(train[-1,],step)) trainX,trainY =convertToMatrix(train,step) testX,testY =convertToMatrix(test,step) trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1])) testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1])) model = Sequential() model.add(SimpleRNN(units=32, input_shape=(1,step), activation="relu")) model.add(Dense(8, activation="relu")) model.add(Dense(1)) model.compile(loss='mean_squared_error', optimizer='rmsprop') model.summary() model.fit(trainX,trainY, epochs=100, batch_size=16, verbose=2) trainPredict = model.predict(trainX) testPredict= model.predict(testX) predicted=np.concatenate((trainPredict,testPredict),axis=0) trainScore = model.evaluate(trainX, trainY, verbose=0) print(trainScore) index = df.index.values plt.plot(index,df) plt.plot(index,predicted) plt.axvline(df.index[Tp], c="r") plt.show()

Hey,

ReplyDeleteNice example, it was helpful. How would it be if the input data consisted of many features (let's say 40) and not just one ?

You are welcome! You need to create combined X array data (contains all features x1, x2, ..) for your training and prediction. It goes like this;

Deletex1, x2, y

2, 3, 3

3, 4, 4

2, 4, 4

3, 5, 5

4, 6, 6

Here, each window contains 3 elements of both x1 and x2 series.

2, 3,

3, 4,

2, 4, =>4

3, 4,

2, 4,

3, 5, => 5

2, 4,

3, 5,

4, 6, => 6

Hello!

ReplyDeleteVery good example, it showed step by step how to implement a RNN. I am struggling to reuse your knowledge and build a Jordan network.

I am attempting to translate your Sequential to Functional API but summary shows different network. :(

This is what I am doing:

visible = Input(shape=(None, step))

rnn = SimpleRNN(units=32, input_shape=(1,step))(visible)

hidden = Dense(8, activation='relu')(rnn)

output = Dense(1)(hidden)

_model = Model(inputs=visible, outputs=output)

_model.compile(loss='mean_squared_error', optimizer='rmsprop')

_model.summary()

By using same data input, I can have some result, but then, when predicting, I am not sure how Tensorflow does its recurrence. I would like to use only one output as input, then, what should I change?

Could you help me out, please?

Hello again!

ReplyDeleteI am trying very hard to understand how I build a RNN with the following features

1. x1, x2 and x3 are input signals that are measurements.

2. o1, o2 are outputs from the last prediction of the NN and o is the actual output

x1, x2, x3, o1, o2 --> o

2, 3, 3, 10, 9, 11,

3, 4, 4, 11, 10, 12,

2, 4, 4, 12, 11, 13,

3, 5, 5, 13, 12, 14,

4, 6, 6, 14, 13, 15,

3. how do I train and predict?

This may help you

Deletehttps://www.datatechnotes.com/2020/01/multi-output-multi-step-regression.html

How to tell if this network is Elman or Jordan? I mean, these two are simple recurrent networks, right?

ReplyDeleteIn the Keras documentation it is only explained that are "Fully-connected RNN where the output is to be fed back to input".