Archive for August 15, 2019

Python: Pandas Lesson 11

August 15, 2019 2 comments

Learning : Pandas Lesson 11
Subject: Zoo Management System – P5

In this post we will cover another three functions, let’s start.

Missing Data: A commun problems we will face in any data file is Missing data, and if we are doing data analysis this may ledd us to a real problem. In this section we will cover only the missing data and will not go through any filling techniques.

Missing Data Report

def missing_data(df):

clear() # To clear the terminal.

print(‘\n\n *** Missing Data Report ***\n’)

print(‘ There are {} rows and {} columns.’.format(df.shape[0],df.shape[1]))

print(‘\n Number of missing data in each columns are:\n’)

total_miss = 0 # Counter

for each in df:

print(‘\n We have {} missing data in {} column.’.format(df[each].isnull().sum(),each))

total_miss = total_miss + df[each].isnull().sum()

print(‘\n Total missing data :’,total_miss)

print(‘\n The missing data represent {}% of the DataFrame.’.format((df.size*total_miss)/100))

input(‘\n\n\n\n ** Press any key to continue .. . . ‘)

This is a very simple missing data report, but doing the job for-now, we cover some part of missing data in lesson 5 read it Here.

Add New Record: I divide the editing function into three parts or three functions, Add new record, Edit a record and delete a record.
First we will work on Add New Record, from the name we know that this function will allow the user to add a record to the DataFrame, first we will get all columns in the dataframe, also we will get the maximum value in the ID column and increase it by one, then we will ask the user to enter the data for each column, and we are assuming that the user will (or MUST) enter a valid data since we agree that we will not have any validation test in this version of the application, also if the user just press ‘Enter’ thats mean NA data so we will add NaN to the field.
So let’s start..

Add New Record
def add_new_record(old_df):

clear() # To clear the terminal.

# First we will fitch the columns from the df

col_list = []

for each in old_df.columns :



# Get max id and increase it by 1

next_id = old_df[‘id’].max()+1


# let user enter the new record.

print(‘\n Enter the data for each field then press Enter.\n’ )

print(‘ If you just press Enter NaN will be entered.’)

for each in col_list:

if each !=’id’:

print(‘ Enter data for ‘,each)

new_row.update({each:(input(‘ : ‘))})


old_df = old_df.append([new_row])

for each in col_list :

if (old_df.loc[old_df[‘id’] == next_id, each][0]) ==”:

(old_df.loc[old_df[‘id’] == next_id,[each]]) = float(‘NaN’)

print(‘\n New Record added successfully..\n’)

# print out last 5 rows to show the new record.

print(‘\n The new record in the df..\n ”,old_df.tail(5))

global df # Reset the df as global variable

df = old_df

input(‘\n\n\n\n ** Press any key to continu .. . . ‘)

Save : At this point we will jump to the last function, simply after adding data to our DataFrame we need to keep this change, we need to save the new record so we can retrieve it next time we run the application. So to do this we will re-write the csv file again.

Save the DataFrame

def save_the_df (df):

clear() # To clear the terminal.


df.to_csv(file_name, encoding=’utf-8′, index=False)

print(‘\n File saved .. ‘)

input(‘\n\n\n\n ** Press any key to continue .. . . ‘)

Now, if we close the application then re-open it again we will have the new records that we add. We just save the file and will not have any option to rename it.

In Next Post.. in coming post we will cover deleting and editting functions, also I will upload the full code file to the portal so any one can download it as python file.

:: Pandas Lessons Post ::

Lesson 1 Lesson 2 Lesson 3 Lesson 4
Lesson 5 Lesson 6 Lesson 7 Lesson 8
Lesson 9 Lesson 10 Lesson 11

Follow me on Twitter..