Pages

Multi-output Regression Example with MultiOutputRegressor in Python

We studied many methods of multioutput regression analysis with Keras in previous posts. In this tutorial, we'll learn how to fit and predict multioutput regression data with scikit-learn's MultiOutputRegressor class. Multioutput data contains more than one target labels for a given x input data. The tutorial covers:
1. Preparing the data
2. Defining the model
3. Predicting and visualizing the result
4. Source code listing
We'll start by loading the required libraries for this tutorial.

```from numpy import array, hstack, math
from numpy.random import uniform
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.multioutput import MultiOutputRegressor```

Preparing the data

First, we 'll create a multi-output dataset for this tutorial. It is randomly generated data with some rules below. There are three inputs and two outputs in this dataset. We'll plot the generated data to check it visually.

```def create_data(n):
x1=array([math.sin(i)*(i/10)+uniform(-5,5) for i in range(n)]).reshape(n,1)
x2=array([math.cos(i)*(i/10)+uniform(-9,5) for i in range(n)]).reshape(n,1)
x3=array([(i/50)+uniform(-10,10) for i in range(n)]).reshape(n,1)

y1 = [x1[i]+x2[i]+x3[i]+uniform(-1,4)+15 for i in range(n)]
y2 = [x1[i]-x2[i]-x3[i]-uniform(-4,2)-10 for i in range(n)]
X = hstack((x1, x2, x3))
Y = hstack((y1, y2))
return X, Y

n = 300
X, Y = create_data(n)```
` `
```f = plt.figure()
plt.title("Xs input data")
plt.plot(X)
plt.xlabel("Samples")
plt.title("Ys output data")
plt.plot(Y)
plt.xlabel("Samples")
plt.show()```

Next, we'll split the dataset into the train and test parts and check the data shapes.

```xtrain, xtest, ytrain, ytest=train_test_split(X, Y, test_size=0.15)
print("xtrain:", xtrain.shape, "ytrian:", ytrain.shape)```
`xtrain: (255, 3) ytrian: (255, 2) `
`print("xtest:", xtest.shape, "ytest:", ytest.shape)`
`xtest: (45, 3) ytest: (45, 2) `

Defining the model

We'll define the model with the MultiOutputRegressor class of sklearn. As an estimator, we'll implement GradientBoostingRegressor with default parameters and then we'll include the estimator into the MultiOutputRegressor class. You can check the parameters of the model by the print command.

```gbr = GradientBoostingRegressor()
model = MultiOutputRegressor(estimator=gbr)
print(model)```
` `
Now, we can fit the model with train data and check the training score.

```model.fit(xtrain, ytrain)
score = model.score(xtrain, ytrain)
print("Training score:", score)```
`Training score: 0.9952671502749106`

Predicting and visualizing the result

We'll predict the test data with a trained model and check the MSE rate for both y1 and y2 outputs.

```ypred = model.predict(xtest)
print("y1 MSE:%.4f" % mean_squared_error(ytest[:,0], ypred[:,0]))```
`y1 MSE:10.9138 `
`print("y2 MSE:%.4f" % mean_squared_error(ytest[:,1], ypred[:,1]))`
`y2 MSE:10.8929 `

Finally, we'll visualize the results in the plot and check them visually.

```x_ax = range(len(xtest))
plt.plot(x_ax, ytest[:,0], label="y1-test", color='c')
plt.plot(x_ax, ypred[:,0], label="y1-pred", color='b')
plt.plot(x_ax, ytest[:,1], label="y2-test", color='m')
plt.plot(x_ax, ypred[:,1], label="y2-pred", color='r')
plt.legend()
plt.show()```

In this tutorial, we've briefly learned how to MultiOutputRegressor class in Python. We've trained the multioutput dataset and predicted test data.

Source code listing

```from numpy import array, hstack, math
from numpy.random import uniform
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.multioutput import MultiOutputRegressor```
` `
```def create_data(n):
x1=array([math.sin(i)*(i/10)+uniform(-5,5) for i in range(n)]).reshape(n,1)
x2=array([math.cos(i)*(i/10)+uniform(-9,5) for i in range(n)]).reshape(n,1)
x3=array([(i/50)+uniform(-10,10) for i in range(n)]).reshape(n,1)

y1 = [x1[i]+x2[i]+x3[i]+uniform(-1,4)+15 for i in range(n)]
y2 = [x1[i]-x2[i]-x3[i]-uniform(-4,2)-10 for i in range(n)]
X = hstack((x1, x2, x3))
Y = hstack((y1, y2))
return X, Y

n = 300
X, Y = create_data(n)```
` `
```f = plt.figure()
plt.title("Xs input data")
plt.plot(X)
plt.xlabel("Samples")
plt.title("Ys output data")
plt.plot(Y)
plt.xlabel("Samples")
plt.show()```
` `
```xtrain, xtest, ytrain, ytest=train_test_split(X, Y, test_size=0.15)
print("xtrain:", xtrain.shape, "ytrian:", ytrain.shape)
print("xtest:", xtest.shape, "ytest:", ytest.shape)

model = MultiOutputRegressor(estimator=gbr)
print(model)

model.fit(xtrain, ytrain)
score = model.score(xtrain, ytrain)
print("Training score:", score)

ypred = model.predict(xtest)
print("y1 MSE:%.4f" % mean_squared_error(ytest[:,0], ypred[:,0]))
print("y2 MSE:%.4f" % mean_squared_error(ytest[:,1], ypred[:,1]))

x_ax = range(len(xtest))
plt.plot(x_ax, ytest[:,0], label="y1-test", color='c')
plt.plot(x_ax, ypred[:,0], label="y1-pred", color='b')
plt.plot(x_ax, ytest[:,1], label="y2-test", color='m')
plt.plot(x_ax, ypred[:,1], label="y2-pred", color='r')
plt.legend()
plt.show() ```