Heatmap Using Python Visualization in Simple Steps

Do you want to visualize your data with a heatmap using Python? Stay with us and follow the step-by-step visualization of data with heatmap using Python.

A heat map helps us visualize density. For example, in the case of web design and analysis, it helps us to visualize how far people scroll on our site, where they click, and even sometimes where they’re looking. In this article, we will discuss how we can visualize data with heatmap using Python and will interpret the results. Moreover, we will also visualize data on a Google map using heatmaps for different time periods using a slider.

You may also like Hexabin Visualization in Python.

Why Visualization is Important?

Data visualization is the practice of translating information into a visual context, such as a map or graph, to make data easier for the human brain to understand and pull insights from. The main goal of data visualization is to make it easier to identify patterns, trends, and outliers in large data sets.

Data visualization is essential to assist businesses in quickly identifying data trends, which would otherwise be a hassle. The pictorial representation of data sets allows analysts to visualize concepts and new patterns. With the increasing surge in data every day, making sense of the quintillion bytes of data is impossible without Data Proliferation, which includes data visualization.

Here are some of the main points why visualization is considered to be an important step.

• It helps to analyze data in a better way. Analyzing reports helps business stakeholders focus on the areas that require attention. Visual mediums help analysts understand the key points needed for their business
• It helps to take decisions faster. Humans process visuals better than any tedious tabular forms or reports. If the data communicates well, decision-makers can quickly take action based on the new data insights, accelerating decision-making, and business growth simultaneously
• It helps us to make sense of complicated datasets. Data visualization allows business users to gain insight into their vast amounts of data. It benefits them to recognize new patterns and errors in the data. Making sense of these patterns helps the users pay attention to areas that indicate red flags or progress.
• Data visualization strengthens the impact of messaging for your audiences and presents the data analysis results in the most persuasive manner.

What is a Heatmap?

A heatmap is a graphical representation of data that uses a system of color coding to represent different values. Heatmaps are used in various forms of analytics but are most commonly used to visualize the density of data points in a specific area.

In the upcoming sections, we will use various Python modules to visualize datasets using heatmap.

Heatmap Using Python

Now, we will dive into the implementation part and practically visualize datasets using a heatmap. Before going to the implementation part, make sure that you already have installed the following required modules as we will be using them later.

• matplotlib
• plotly
• folium
• geopandas
• NumPy
• calplot

You can get access to all the datasets that we will be using in this article and the Jupyter Notebook from my GitHub account.

To get started with visualization and understand what a heatmap looks like, we will randomly generate a data set and then visualize that randomly generated dataset using a heatmap.

# importing required modules
import matplotlib.pyplot as plt
import numpy as np


# size of plot
plt.figure(figsize=(10, 6))

# Generate Data
data = np.random.rand(10,6)

#Heatmap using Python
plt.pcolor(data)
plt.show()

Output:

As you can see, we get a map with different colors which represent the intensity of data points at a specific location.

Visualizing Correlation Matrix Using the Heatmap

A correlation matrix is a table that displays the correlation coefficients for different variables. The matrix depicts the correlation between all the possible pairs of values in a table. It is a powerful tool to summarize a large dataset and identify and visualize patterns in the given data. We will not use a heatmap to visualize a correlation matrix.

But first lets us import the dataset.

# importing the required modules
import pandas as pd


# importing dataset
data = pd.read_csv("house.csv")

# printing the heading
data.head()

Output:

Now, we will use a heatmap to find the correlation matrix. We will use Seaborn to visualize the heatmap.

# importing the module
import seaborn as sns

# setting the size of the figure
sns.set(rc={'figure.figsize':(12,8)})

# Heatmap using Python
sns.heatmap(data.corr(), cmap="YlGnBu", annot=True);

Output:

As you can see, each color represents correlations. The white color shows a highly negative correlation and the blue dark blue color shows a highly positive correlation.

Visualizing Time Series with Heatmap

Time series data also referred to as time-stamped data, is a sequence of data points indexed in time order. Time-stamped is data collected at different points in time. These data points typically consist of successive measurements made from the same source over a time interval and are used to track change over time. In this section, we will learn and interpret the heatmap of time series data.

The most common time-series visualization that you all have seen is the visualization of the contributions to your GitHub account. For example, it looks like this:

We can also visualize our time series data in a similar way as shown above using the cotplot module.

As you can guess to visualize your dataset using a cotplot, you need three things.

1. An array of dates
2. array of days
3. date with events

We will use NumPy and pandas module to create the above-required datasets. So, let us first create our dataset.

# creating dataset
all_days = pd.date_range('1/1/2019', periods=350, freq='D')
days = np.random.choice(all_days, 1000)
events = pd.Series(np.random.randn(len(days)), index=days)

Once, we prepare our dataset for the visualization, we can then use the Cotplot module to visualize it.

# importing the module
import calplot

# Heatmap using Python
calplot.calplot(events)

Output:

The different colors show the intensity of an event.

Visualizing Heatmaps on a Google Map Using Python

Now, we will learn how we can use heatmaps to visualize the data on Google Maps. We will use plotly and folium plot heatmap. We will also learn how we can save the heatmap in HTML format and open the heatmap in any browser. Moreover, we will also learn how we can add a slider to the heatmap to see what the looks like over different time periods.

Heatmap Using Plotly on Google Map

Before visualizing the heatmap, let us first import the dataset.

# importing dataset
heatmap = pd.read_csv('heatMap.csv')

# head of the Heatmap using Python
heatmap.head()

Output:

As you can see, we have locations in the form of latitude and longitude of the event. We will use these locations and plot a heatmap on Google map using Plotly.

Let us first visualize the overall dataset. We will use the density_mapbox function to visualize the heatmap. This function takes the data, latitude and longitude values, and other various parameters to make the heatmap attractive.

# importing the dataset
import plotly.express as px

# creating Heatmap using Python
fig = px.density_mapbox(heatmap, lat='latitude', lon='longitude', radius=10, zoom=10,center ={'lat': 42.88231, 'lon' :74.56664},
                        mapbox_style="stamen-terrain")
fig.show()

Output:

As you can see, the yellow part shows the high intensity of the dataset and the purple color shows the low intensity. The good thing about plotly plots is that they are interactive and you can zoom the map as much as you want.

Creating Heatmap in HTML Format

Now, we will learn how we can create an HTML formatted heatmap so that we can use send the map directly to the clients and they can easily see the visualizations by opening it in any browser. We will use the folium module to create an HTML-formatted heat map. To make the visualization more interactive, we will also use a slider based on per hour.

Let us first add the hour section to our dataset.

# importing the required module
import regex as re

# creating fuction to convert the into hours
def get_hour(tf):
    hour = re.findall('\d\d:\d\d:\d\d', tf)
    return int(hour[0][:2])

# calling the function
heatmap['time'] = heatmap['created_at'].apply(get_hour)

# heading
heatmap.head()

Output:

As you can see, we have added a new column that shows the time at which the event was created. Now, in the next part, we have to combine the dataset with each hour together.

# creating an empty list
lat_long_list = []

# using for loop to iterate through each of hour
for i in heatmap['time'].unique():
    temp=[]
    for index, instance in heatmap[heatmap['time'] == i].iterrows():
        temp.append([instance['latitude'],instance['longitude']])
    lat_long_list.append(temp)

Now, there is one final step before going to visualize the heatmap with the slider. And that step is to specify the slider. In our case, we will use 24 hours and shows data for each hour.

# I had added a random dat of 14/02/2022 for the formating. The important thing is I am chaning the hours step by step for the slider
date_strings =['14/02/2022, 00:00:00','14/02/2022, 01:00:00','14/02/2022, 02:00:00','14/02/2022, 03:00:00','14/02/2022, 04:00:00','14/02/2022, 05:00:00','14/02/2022, 06:00:00','14/02/2022, 07:00:00','14/02/2022, 08:00:00','14/02/2022, 09:00:00','14/02/2022, 10:00:00','14/02/2022, 11:00:00',
'14/02/2022, 12:00:00','14/02/2022, 13:00:00','14/02/2022, 14:00:00','14/02/2022, 15:00:00','14/02/2022, 16:00:00','14/02/2022, 17:00:00','14/02/2022, 18:00:00','14/02/2022, 19:00:00','14/02/2022, 20:00:00','14/02/2022, 21:00:00','14/02/2022, 22:00:00','14/02/2022, 23:00:00']

Finally, we will import folium and plot the above dataset as a heatmap in HTML format with a slider.

# importing folium
import folium
from folium.plugins import HeatMapWithTime

#Choosing the map type 
m = folium.Map(location=[42.88231,74.56664],zoom_start = 10)
#Plot it on the map
HeatMapWithTime(lat_long_list,radius=5,auto_play=True,position='bottomright',name="cluster",index=date_strings,max_opacity=0.7).add_to(m)
# Display Heatmap using Python
m

Output:

You can use the slider to see the dataset during different hours.

Finally, you can save the heat map in HTML format by running the following code.

# saving the Heatmap using Python in HTML format
m.save('heatmap.html')

This will save the heatmap in HTML format.

Matplotlib and Heatmap Using Python

Now we will discuss various Python modules that can be used to plot a simple heatmap for the given data. We already had discussed, seaborn, Plotly, and folium so we will not again repeat those modules.

First, let us import the dataset and do a little preprocessing to make it suitable for the visualization with heatmap.

# importing the dataset
passangers = pd.read_csv('AirPassengers.csv')

# printing the heading
passangers.head()

Output:

So, to make a heatmap, we need to separate the month and year and store them in separate columns

# converting the Month to pandas datetine
passangers['Month'] =  passangers['Month'].apply(pd.to_datetime)

# selecting year
passangers['year']= passangers['Month'].dt.year

# selecting month
passangers['month'] = passangers['Month'].dt.month

# heading of the dataset
passangers.head()

Output:

Now, we can remove the column of Month.

# removing the column
passangers.drop('Month', axis=1, inplace=True)

And as a final step of preprocessing, we will convert the pandas data frame into matrix form,

# converting to matrix form
passengers_matrix = passangers.pivot("month", "year", "#Passengers")

# printing
passengers_matrix

Output:

Now, our dataset is ready to be visualized using a heatmap.

Heatmap Using Python in Matplotlib Module

Let us now use matplotlib module to visualize the heatmap of the above dataset.

# importing the modules
import matplotlib.ticker as ticker
import matplotlib.cm as cm
import matplotlib as mpl


# setting up for plotting
fig = plt.figure()
fig, ax = plt.subplots(1,1, figsize=(12,8))

# plotting Heatmap using Python
heatplot = ax.imshow(passengers_matrix, cmap='BuPu')
ax.set_xticklabels(passengers_matrix.columns)
ax.set_yticklabels(passengers_matrix.index)

# labeling 
tick_spacing = 1
ax.xaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))
ax.yaxis.set_major_locator(ticker.MultipleLocator(tick_spacing))
ax.set_title("Heatmap of Flight Density from 1949 to 1961")
ax.set_xlabel('Year')
ax.set_ylabel('Month')

Output:

So the darker purple color shows the high rate of passengers while the white color shows the low rate of passengers.

Summary

A heat map is a two-dimensional representation of information with the help of colors. Heat maps can help the user visualize simple or complex information. Heat maps are used in many areas such as defense, marketing, and understanding consumer behavior. In this article, we learn how to visualize heatmap using various datasets. We learn how to visualize time series data, location-based data, and data having demands using heatmaps.