Fill missing values in Pandas

The “fillna” function in Pandas not only can replace missing values with a given constant value, like in this example:

1
2
3
4
import pandas as pd
import numpy as np
df = pd.DataFrame([[np.nan], [2], [np.nan], [0]])
df
A dataframe with missing values
A dataframe with missing values
1
df.fillna(47)
Missing values replaced with a constant
Missing values replaced with a constant

You can also replace a missing value with the next (or previous) value in the data frame!

1
df.fillna(method = "ffill")
Missing values filled with the previous existing value.
Missing values filled with the previous existing value.

Note that the first value cannot be replaced because nothing is preceding it.

You can also use the value of the next row to fill a missing value.

1
df.fillna(method = "bfill")
Missing values filled with the next existing value.
Missing values filled with the next existing value.

Did you enjoy reading this article?
Would you like to learn more about leveraging AI to drive growth and innovation, software craft in data engineering, and MLOps?

Subscribe to the newsletter or add this blog to your RSS reader (does anyone still use them?) to get a notification when I publish a new essay!

Newsletter

Do you enjoy reading my articles?
Subscribe to the newsletter if you don't want to miss the new content, business offers, and free training materials.

Bartosz Mikulski

Bartosz Mikulski

  • MLOps engineer by day
  • AI and data engineering consultant by night
  • Python and data engineering trainer
  • Conference speaker
  • Contributed a chapter to the book "97 Things Every Data Engineer Should Know"
  • Twitter: @mikulskibartosz
  • Mastodon: @mikulskibartosz@mathstodon.xyz
Newsletter

Do you enjoy reading my articles?
Subscribe to the newsletter if you don't want to miss the new content, business offers, and free training materials.