Pandas Data Cleaning: Remove rows with empty data or missing values

Example of removing rows.

  1. Use dropna() that when you don’t care which rows, just drop them.
df.dropna(axis='rows')
def get_empty_rows(df):
indices = []
for i, s in df.iterrows():
for v in s.values:
if str(v).lower() == 'nan':
if i not in indices:
indices.append(i)

return indices
def get_empty_rows(dataset):
# Nested 'for' loop and unique value list of 'set'
return list(set([i for i, s in dataset.iterrows() for v in s.values if str(v) == 'nan']))
list_of_empty_rows = get_empty_rows(df)
df.drop(list_of_empty_rows)
index_list = df[df['price'] < 20].indexindex_list = df[df[‘name’] == np.nan].indexindex_list = df[df['email'].isnull()].indexdf.drop(index_list)
df[df.isnull().any(axis=1)]

Drop rows that with particular colums contain missing value:

df.dropna(subset=['costPrice', 'SuggesedPrice'])

--

--

--

I occasionally write about software, web, blockchain, machine learning, random thoughts.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Seattle Car Accidents Severity Prediction

Weekly picks by Ali Alohali — مختارات اسبوعيه بواسطة علي العوهلي — Issue #12

What’s The Best API To Get Aluminum Rates In Indian Rupee?

What Is The Most Accurate URL Classification API For SSP?

Pandas — DataFrames

LOTI: Weeknote 41

Slide showing title: “from Covid to recovery”

6 Things To Keep In Mind While Writing SOP For A Data Science MS Programme

Does More Data Equal Better Health?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Weiming Chen

Weiming Chen

I occasionally write about software, web, blockchain, machine learning, random thoughts.

More from Medium

Beginner Data Analysis with Pandas

Learning pandas by analyzing a kaggle data set

Data Wrangling with Pandas-101

Groupby Pandas