DataTechNotes Year in Review: 2019

I have no special talent. I am only passionately curious. 

- Albert Einstein

   The year 2019 is coming to an end and now it is time to look back and evaluate our work during the year. First of all, I am grateful for everything this year. I am especially thankful that I've accomplished my publication target for the year and I am honored to write my year-end post to my readers.

    In 2019, I published 52 articles on datatechnotes.com. I could keep the persistence in my publications. I believe that the quality of the tutorials has improved compared to the last year.

   This year, we have been discussed various topics on machine learning, deep learning, and data analysis. Particularly, we've tackled the regression analysis, accuracy, NLP, clustering, and deep learning topics. Examples in Python have been increased, about seventy percent of the tutorials were written in Python.

   I'll keep writing the tutorials on topics of machine learning and deep learning next year too. Topics may include NLP, GANs, LSTM networks, and other interesting concepts of data science.

Multioutput Regression Example with Keras LSTM Network in Python

   Multioutput regression data can be fitted and predicted by the LSTM network model in Keras deep learning API. This type of data contains more than one output value for given input data.
LSTM (Long Short-Term Memory) network is a type of recurrent neural network and used to analyze sequence data. In this tutorial, we'll briefly learn how to fit and predict multioutput regression data with Keras LSTM model.
The post covers:

1. Preparing the data
2. Defining and fitting the model
3. Predicting and visualizing the results
4. Source code listing

We'll start by loading the required libraries of Python and Keras API for this tutorial.

from keras.models import Sequential
from keras.layers import Dense, LSTM
from numpy import array
from numpy.random import uniform
from numpy import hstack
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

How to Fit Regression Data with CNN Model in Python

   Convolutional Neural Network (CNN) models are mainly used for two-dimensional arrays like image data. However, we can also apply CNN with regression data analysis. In this case, we apply a one-dimensional convolutional network and reshape the input data according to it. Keras provides the Conv1D class to add a one-dimensional convolutional layer into the model.
   In this tutorial, we'll learn how to fit and predict regression data with the CNN 1D model with Keras in Python. The tutorial covers:

1. Preparing the data
2. Defining and fitting the model
3. Predicting and visualizing the results
4. Source code listing

We'll start by loading the required libraries for this tutorial.

from sklearn.datasets import load_boston
from keras.models import Sequential
from keras.layers import Dense, Conv1D, Flatten
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt

Multi-output Regression Example with Keras Sequential Model

   Multi-output regression data contains more than one output value for a given input data. We can easily fit and predict this type of regression data with Keras neural networks API. In this tutorial, we'll learn how to fit multi-output regression data with Keras sequential model in Python. The post covers:

1. Preparing the data
2. Defining the model
3. Predicting and visualizing the result
4. Source code listing

We'll start by loading the required Python modules for this tutorial.

from keras.models import Sequential
from keras.layers import Dense
from numpy import array
from numpy.random import uniform
from numpy import hstack
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

Preparing the data

Data preparation is an import part of this tutorial. Once you can prepare your data in a correct format, the simple sequential model can handle the remaining part of the problem. We'll create a multi-output dataset for this tutorial. It is randomly generated data with some rules. You can check the logic of data generation in the below function. There are three inputs and two outputs in this dataset.

Understanding Optimizers in Neural Networks with Keras

   To improve the accuracy and reduce the loss, we need to train the neural networks by using optimization algorithms. The optimizers are one of the main components of model training. Neural network optimization is a process to fit the model with training data by adjusting the weights to get the best performance. 

   In this tutorial, we'll briefly learn some of the mainly used optimizers such as SGD, RMSProp, Adam, Adagrad, Adamax, and their implementations in neural network training with Keras API. The post covers:

1. A brief about optimizers: SGD, RMSProp, Adam, Adagrad, Adamax
2. Implementing optimizer with Keras
3. Source code listing