Pandas Data Cleaning: Remove rows with empty data or missing values

Example of removing rows.

  1. Use dropna() that when you don’t care which rows, just drop them.
df.dropna(axis='rows')
def get_empty_rows(df):
indices = []
for i, s in df.iterrows():
for v in s.values:
if str(v).lower() == 'nan':
if i not in indices:
indices.append(i)

return indices
def get_empty_rows(dataset):
# Nested 'for' loop and unique value list of 'set'
return list(set([i for i, s in dataset.iterrows() for v in s.values if str(v) == 'nan']))
list_of_empty_rows = get_empty_rows(df)
df.drop(list_of_empty_rows)
index_list = df[df['price'] < 20].indexindex_list = df[df[‘name’] == np.nan].indexindex_list = df[df['email'].isnull()].indexdf.drop(index_list)
df[df.isnull().any(axis=1)]

Drop rows that with particular colums contain missing value:

df.dropna(subset=['costPrice', 'SuggesedPrice'])

--

--

--

I occasionally write about software, web, blockchain, machine learning, random thoughts.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Visualizing Populism

Well, here’s another nice mess you’ve gotten me into

Starting Data Science and Analytics from scratch || The experience of an undergrad

________ is a Japanese term meaning continuous improvement.

Go for a Mattress that Fits OnesRequirements https://t.co/2FEBSUCgrN

Generate Fake Data with Python

Who is classier? “The University of Michigan or Michigan State University.

A (Much) Better Approach to Evaluate Your Machine Learning Model

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Weiming Chen

Weiming Chen

I occasionally write about software, web, blockchain, machine learning, random thoughts.

More from Medium

DIABETES DATASET

Is Pandas the Only Library to Open Datasets?

How to perform Exploratory Data Analysis (EDA) and clean your data for model training?

EDA with examples and codes on the Demographic Data