How to apply Neural Networks for Classification using TensorFlow? Well, Neural Networks for classification is a process of grouping data points into classes using training data. The goal is to assign new data points to the correct class based on the similarity of the data points. There are many well-known Python libraries for Neural Network classification and one of them is TensorFlow. TensorFlow is an open-source software library for Deep Neural Networks. It can create models for image recognition, natural language processing, and more. In this article, we will be using TensorFlow for classification purposes. We will apply Neural network classification to binary and multi-class classifications.
If you have any questions, please let us know through comments, and we will try our best to help you.
What is TensorFlow?
TensorFlow is a sophisticated Machine Learning software library created by Google. It allows for the creation of custom algorithms for a variety of purposes, including image and video recognition, natural language processing, and predictive analytics. TensorFlow is also capable of running on a variety of platforms, including CPUs, GPUs, and TPUs. It provides an extensive API that makes it easy to develop complex Machine, Learning models. In addition, TensorFlow offers a number of high-level APIs that make it possible to quickly develop and train models.
The word TensorFlow is made of two words, i.e., Tensor and Flow, where the “Tensor” is simply a multidimensional array, and the “Flow” is used to define the flow of data in operation.
Important Features of TensorFlow
TensorFlow has the following important features:
- TensorFlow is an open-source library that allows rapid and easier calculations in machine learning.
- We can execute TensorFlow applications on various platforms such as Android, Cloud, IOS, and various architectures such as CPUs and GPUs. This allows it to be executed on various embedded platforms.
- TensorFlow has its own designed hardware to train the neural models known as Cloud TPUs.
- TensorFlow provides a fast debugging method. Tensor Board works with the graph to visualize its working using its dashboard.
- It works with multidimensional arrays with the help of a data structure tensor which represents the edges in the flow graph.
- TensorFlow provides a defined level of abstraction by reducing the code length and cutting the development time.
- We can build and train ML models easily using intuitive high-level APIs like Keras with eager execution, which makes for immediate model iteration and easy debugging.
- TensorFlow provides the process of resolving complex topologies with the support of Keras API and data input pipelines.
- TensorFlow provides a defined level of abstraction by reducing the code length and cutting the development time.
Getting Started With Neural Networks For Classification Using TensorFlow
As we discussed earlier, we will be using Neural Networks for Classification using TensorFlow to classify binary and multi-class problems. The Architecture of the Neural Network for both of the classifications is nearly the same with few changes. For example, in the case of binary classification, the output layer has only one node as shown below:
Applying any activation function on the input layer is not allowed as this layer just takes the input and transfers it to the next layer. We can apply any activation function except softmax in the hidden layer. However, in the case of binary classification, the activation function on the output layer should be sigmoid as it gives the probability of an item occurring.
The Architecture of the multi-class classification problem is a little bit different from the binary one. The nodes in the output layer should be equal to the number of output classes. For example, if we have a classification dataset having 3 output categories, then their neural network architecture will be:
Similar to binary classification, no activation function can be applied to the input layer. While any activation function except softmax can be applied to the hidden layer. However, for the output layer, the activation function should be softmax.
Now let us understand how we can use TensorFlow to implement Neural Networks and solve classification problems.
Binary Classification Using TensorFlow
We will now use TensorFlow to classify a binary classification problem. In this section, we will be using a dataset about bank marking campaigns based on phone calls. The output class includes whether or not the clients will subscribe to their services. You can download the dataset from this link.
Now, before going to the implementation part, it is necessary to install all the required Python modules that we will be using in this article.
You can open the terminal and run the following commands to install the required modules.
pip install tensorflow
pip install pandas
pip install numpy
pip install matplotlibOnce the above modules are installed successfully, we can then move to the implementation part.
Exploring the Dataset
Let us first import the dataset using the pandas module.
# importing pandas module
import pandas as pd
# importing dataset
data = pd.read_csv('bank-full.csv', sep=';')
# heading
data.head()Output:
As you can see, most of the rows have object-type values. We can only feed numeric values to the Neural Networks, so we will convert the object type to numeric values using the labeling encoding method.
# importing the module
from sklearn import preprocessing
# creating labing encoding object
label_encoder = preprocessing.LabelEncoder()
# Encode labels in multiple columns
data['job']= label_encoder.fit_transform(data['job'])
data['marital']= label_encoder.fit_transform(data['marital'])
data['education']= label_encoder.fit_transform(data['education'])
data['default']= label_encoder.fit_transform(data['default'])
data['housing']= label_encoder.fit_transform(data['housing'])
data['housing']= label_encoder.fit_transform(data['housing'])
data['loan']= label_encoder.fit_transform(data['loan'])
data['contact']= label_encoder.fit_transform(data['contact'])
data['month']= label_encoder.fit_transform(data['month'])
data['poutcome']= label_encoder.fit_transform(data['poutcome'])
data['y']= label_encoder.fit_transform(data['y'])You may also like the One Hot Encoding method:
As you can see, we first imported the preprocessing module and then applied label encoding to convert the object type values to numeric values. Let us use the info() method to check the data types of each row.
As it is a binary classification problem, let us also plot the bar plot of the output classes. We will use matplotlib module to plot the graph.
# importing matplotlib
import matplotlib.pyplot as plt
# plotting
fig = plt.figure(figsize = (10, 5))
# Bar plot
plt.bar(['No',"Yes"], data.y.value_counts(), color ='blue',
width = 0.8)
# labeling
plt.xlabel("Classes")
plt.ylabel("Values")
plt.show()Output:
As you can see, we have only two output values.
Splitting the Dataset
Before feeding the dataset to the model, we need to split the dataset into training and testing datasets, so that we can easily evaluate the performance of our model.
But before it, let us divide the dataset into output and input values.
# splitting dataset
X = data.drop('y', axis=1)
y = data['y']Now, we will split the inputs and outputs into testing and training parts. We will use the sklearn module to split the dataset.
We will assign value 1 to the random state.
# importing the module
from sklearn.model_selection import train_test_split
# splitting the dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=1)Now we will move to build Neural Networks.
Building Binary Neural Network For Classification Using TensorFlow
Let us first import all the required modules to build a Neural Network.
# importing required module for Neural Networks for Classification using TensorFlow
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import InputLayer
from tensorflow.keras import SequentialNow the very first step is to initialize the model.
# define Neural Networks for Classification using TensorFlow model
model = Sequential()The next step is to build the input layer. Remember that the nodes in the input layer should be equal to the number of input variables. In our case, there is a total of 16 input variables, so the number of nodes will be 16 in the input layer.
# adding input layer with 16 nodes
model.add(InputLayer(16))As a practice, we will add a hidden layer with 15 nodes as the next layer of the neural network and apply the ReLU activation function.
# adding hidden layer with 10 nodes
model.add(Dense(10, activation='relu', kernel_initializer='he_normal'))Now, we will add the last layer, the output layer, and will specify the number of nodes to be 1 as it is a binary classification problem.
# adding output layer
model.add(Dense(1, activation='sigmoid'))We have now completed building the neural network.
Compiling and Training the Model
Once we create the model, the very next step is to compile the model. Let us now compile the model. We will use adam as an optimizer and binary_crossentropy as a loss function.
# compile Neural Networks for Classification using TensorFlow model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])Once the compilation is complete, we will then move to the training of the model. We will fix the epochs to be 50. An epoch means training the neural network with all the training data for one cycle. You can change the number of epochs to get an optimum solution.
# fit Neural Networks for Classification using TensorFlow model
model.fit(X_train, y_train, epochs=50)It will take some time to train the model. Once the training is complete, we can then use the testing data to find the accuracy of the model.
# evaluate Neural Networks for Classification using TensorFlow model
evaluate = model.evaluate(X_test, y_test)
print('Test Accuracy:', evaluate[1])Output:
0.88
As you can see, we get an accuracy score of 88% which means the model was able to classify 88% of the testing data correctly
Multi-class Classification Using TensorFlow
Now we will jump into the multiclass classification using TensorFlow. The steps are pretty much similar to the binary one, we will not spend too much time explaining each of the steps again and again. The only difference is in the output layer which we will focus on in this section.
In this section, we will be using a dataset created by a higher education institution related to students enrolled in different undergraduate degrees, such as agronomy, design, education, nursing, journalism, management, social service, and technologies. The data is used to build classification models to predict students’ dropout and academic success. So, there are three output categories; enrolled, dropout, and graduate. You can read about the dataset and download it from this link.
Importing dataset
For multi-class classification, we need to have a dataset where there should be more than two possible output categories. Again we will use the Pandas module to import the dataset.
# importing the dataset
dataset = pd.read_csv('dataset.csv', sep=";")
# heading of dataset
dataset.head()Output:
In this dataset, everything is in numeric values, except the output class. So, now we will encode the target class using label encoding.
# creating labing encoding object
label_encoder = preprocessing.LabelEncoder()
# Encode labels in multiple columns
dataset['Target']= label_encoder.fit_transform(dataset['Target'])No, the dataset is ready to feed to the model
Splitting the Dataset
The purpose of splitting the dataset is to have testing and training parts so that we can use the training part to train the model and then use the testing part in order to evaluate the performance of the model. Let us first divide the dataset into input and output values.
# dividing the dataset
X = dataset.drop('Target', axis=1)
y=dataset['Target']The next step is to split the data set into testing and training parts.
# splitting the dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=1)As you can see, we have assigned 30% of the data to the testing part and the remaining 70% to the training part.
Building Neural Network for Multi-classification Using TensorFlow
As our dataset is now ready, we will start building the mode. We will initialize the model, then add a hidden layer with 10 nodes.
# importing required module
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import InputLayer
from tensorflow.keras import Sequential
# defineNeural Networks for Classification using TensorFlow model
model = Sequential()
# adding input layer with 16 nodes
model.add(InputLayer(34))
# adding hidden layer with 10 nodes
model.add(Dense(10, activation='relu', kernel_initializer='he_normal'))Now we will add the output layer. Notice that we will add three nodes to the output layer and apply the softmax activation function.
# adding output layer
model.add(Dense(3, activation='softmax'))Once the neural network is built, the next step is to compile the model.
Compiling and Training the Model
Once, we complete building the neural network model, the next step is to compile the model and train it using the training dataset. This time we will fix the epoch value to 50.
# compile the Neural Networks for Classification using TensorFlow model
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
# fit the model
model.fit(X_train, y_train, epochs=50)Let us now find the accuracy of the model using the testing dataset.
# evaluate Neural Networks for Classification using TensorFlow
evaluate = model.evaluate(X_test, y_test)
print('Test Accuracy:', evaluate[1])Output:
0.76As you can see, we get an accuracy score of 76% which means the model was able to classify 76% of the testing data correctly.
Summary
In this article, we learned how we can use Neural Networks for Classification using TensorFlow. Moreover, we discussed how we can use TensorFlow to build a neural network for binary and multiclass classification problems. The classification problem is a problem where we have categorical output possible values. These categorical values can either be binary or multi. In the case of binary classification, we use the sigmoid function on the output layer while in the case of multi-class classification, we use the softmax activation function on the output layer.