Archive

Posts Tagged ‘covid19’

Python: Covid19 Function Upgrade 1

April 16, 2020 Leave a comment


Learning : Using Beautifulsoup in Python
Subject: Python & Covid19 Function ‘Data By Country Upgrade

In last post (Python and Covid19) we use the Beautifulsoup and other libraries to grab COVID19 data from the internet and display it on the screen using our Main-Menu and some functions, and we said that some of those functions need some enhancement. In this post we will do some upgrade to one of the functions ‘def covid19_in_country()‘ and we will add the version number to it’s name as (def covid19_in_country_v02()).

In the previous Version of the function, we list down all the countries name with there indexes and ask the user to enter the index number of a country to get it’s COVID19 Data. However in this version we will give the user an option to select wither to:
1. Select the Country Name from a list.
2. Write the Country Name.

So if the user go-for option [1] then this will be as version 1, and if the user select option [2] then we will provide the opportunity to Enter Full or Part of a country Name and we will search for similar input (in the country list) and list them. Here is a screen shot of the RUN-Time ..


First we will write a code to create a list of Countries and there Indexes, this new function will called ‘def country_list()’ and it will return a list variable named ‘c_name_list’, here is the code for it.

# Function to create Country List

def country_list() :
    c_name_list=[]
    total_rows = soup.find_all('th',class_='covid-total-row')
    Total_Countries = int((total_rows)[0].b.text)
    for x in range (2,Total_Countries) :
            try:
                c_name_list.append([x,trs[x].find('a',title=re.compile('2020 coronavirus pandemic in *')).text])

            except:
                pass # If any errors, just pass.
    return c_name_list


Now we will split a part that (displaying the Country Data) from the main function ‘covid19_in_country()’ to be as an objet so we can call it any time with passing the country index to it. Here is the code..

# Code to display COVID19 data on the screen.

def show_counry_data(user_c_index) :
    # This will print the country name.
    print('\n\n     ',trs[int(user_c_index)].find('a',title=re.compile('2020 coronavirus pandemic in *')).text)
    # This will print the country information.
    tds = trs[int(user_c_index)].find_all('td')
    print('        Country Rank is: ',int(user_c_index)-1)
    print('        Cases:',tds[0].text.strip())
    print('        Deths:',tds[1].text.strip())
    print('        Recoveries:',tds[2].text.strip())
   



No we will re-write the main function of covid19_in_country() and we will add version number to it so it will be “covid19_in_country_v02()“. Here is the code ..

# covid19_in_country_v02()
 

def covid19_in_country_v02() :
    """
    Version: 02.13.4.2020
    """
    os.system('clear')
    print('\n    Covid19 Country Data.')
    print('\n    ',last_update,'\n')

    while True:

        print('\n   Do you want to:' )
        print('   1. Select the Country name from a list.' )
        print('   2. Write the Country Name.')
        country_as = input('\n   Select from the Menu [1,2]: ')
        if country_as in ['1','2'] :
            break
        else:
            print('\n    You Must Select [1 or 2] ... Try Again .. ')

    c_name_list = country_list()
    if country_as =='1' :
    # First we will print-out the list of counries name.
        for cou in range (0,(len(c_name_list)),4) :
            try:
                print ('    ',c_name_list[cou],c_name_list[cou+1],c_name_list[cou+2],c_name_list[cou+3])
            except:
                pass # if any error just continue.

        while True :
            user_c_index = (input('\n\n    Enter the Index number of the Country to see it''s Data. [e to Exit] '))
            try :
                show_counry_data(user_c_index)         
            except :
                if ((str(user_c_index)) in ['e','E']) :
                    return
                else :
                    print('\n    You must Enter a Number or ''e''/''E'' to exit')


    if country_as =='2' :
        
        print('\n   You Select the option to Enter a Country Name.')
        search_country = input('   Enter the Country Name (or Part of it): ')       
        print('  You enter: ',search_country)
        c_indexes=[]
        for x in range (0,(len(c_name_list))) :
            try :
                if (search_country.casefold()) in (c_name_list[x][1]).casefold() :
                    c_indexes.append([c_name_list[x][0],c_name_list[x][1]])
            except:
                 pass
        if (len(c_indexes)==0) :
            print('\n   We can''t find your search, this may due to:')
            print('   1. Spelling error.')
            print('   2. Country Name drops for some technical error.')
            input('\n    ... Press any key ...'')
            return

        else :
            print('\n   We fond {} matches with your search, Enter the Index Number for the\n   one you are looking for: '.format(len(c_indexes)))
            print('\n\n ',c_indexes)

            show_counry_data(int(input('\n   Enter the Counrty Index Now: ')))

    input('   ... Press Any Key ...')



This application and the code was written as a matter of training and learning. The owner/creator is not responsible for any missing or wrong data may exist in the file and/or the Data. (the code and/or the data-sets).



Note That: The file will only contain the changes in the function, NOT the Full COVID19 APP.

To Download my Python code (.py) files Click-Here




Follow me on Twitter..




By: Ali Radwani




Python and Covid-19

April 13, 2020 1 comment


Learning : Using Beautifulsoup in Python
Subject: Grab Data from a Website Using Python

In this article we will use Python codes to fetche some data from a website, the Data we are looking for is about Covid-19, to do this we can use an API but here I am using web crawler with Beautifulsoup. So let’s start.

Introduction: First we need a Data Source web-Page, if we search the web we will find a lot of pages there I select this one (en.m.wikipedia.org/wiki/2019%E2%80%9320_coronavirus_pandemic_by_country_and_territory) Source:wikipedia.org, you need a basic knowledge about HTML to understand the stag’s. After reading the web-page source code we can found the following:
1. The HTML code structure contain 9 Tables.
2. The Data about Covid-19 is in Table No. 2.
3. The Table consist of several ‘tr’ Tag. In HTML ‘tr’ tag is Row.
4. Each ‘tr’ consist of several ‘td’ Tag. In HTML ‘td’ tag is Colunm.
5. Each ‘td’ may contain other tags or information, and some of those information are what we are looking for. We will talk about this later in bellow.

Now we will talk about our Python Application. We will write a Python Application that will display a Menu on the screen and ask the user to select from the Menu, and based on the user selection we will run a function to perform something, in our case we will have a Menu with four choice of selection as:
1. The Totals in the world.
2. The covid-19 by countries.
3. Covid-19 Table.
9. Exit.

To start writing our code, First we need to import some library we here is the code..

# To import needed Libraries

import requests, os, re
from bs4 import BeautifulSoup

The Menu: As we mentioned we will have a Main-Menu to help the users in there selections. Here is the code:

# COVID-19 APP MAIN MENU

def covid19_menu() :

    while True :
        # This will Clear the terminal
        os.system('clear')  
        print ('\n    =====[ COVID-19 APP MAIN MENU ]=====')
        print('    1. The Total in the World.')
        print('    2. The covid-19 by Countries')
        print('    3. Covid-19 Table. ')
        print('    9. Exit')

        user_input = input('\n    Enter Your choice: ')

        if user_input in ['1','2','3','9'] :
            if user_input =='1' :
                covid19_world_total()

            if user_input =='2' :
                covid19_in_country()

            if user_input =='3' :
                covid19_table()

            if user_input =='9' :
                return

        else :
            print (' You Must select from the above Menu.')
            input('\n      ... Press any key ...')



Now we will work on First function, before that we will write this two lines of code to set my_url (the webpage we want to read), and calling the BeautifulSoup for it, so all the webpage source will be as text in variable soup:

# To load the page in a variable soup
my_url ='https://en.m.wikipedia.org/wiki/2019%E2%80%9320_coronavirus_pandemic_by_country_and_territory'
source = requests.get(my_url, headers = headers).text
soup = BeautifulSoup(source, 'lxml')


Now our first function will get the last update of the data (Date and Time), after searching the web-site code, I found that the page contain 19 ‘P’ tag’s and the last update statement is in fifth Paragraph, but in case that the page may updated with more data or the page structure changed, we will write a code to search for it. Here is the function to get the Data last Update statement.

# Function to get Last update statement

def get_last_update(): 
    update_date_time = soup.find_all('p')
    # I found the date in P = 5, but i will search other the 19 P's
    for thep in range (0,19) : 
        if 'UTC on' in update_date_time[thep].text :
            last_update='The Data Updated on ' + update_date_time[thep].text[6:32]
            return last_update
        else:
            thep = thep + 1 


Now, our function will get the four set of information for Grand Totals in the world as Total Countries has COVID-19, Total Cases of COVID-19, Total Deaths by COVID-19 and Total Recoveries from COVID-19. Here is the function

# Function to get the covid19 World Totals

def covid19_world_total():
    os.system('clear')
    print('\n    ',last_update,'\n')
    # This section will fetch th Total Rows.
    total_rows = soup.find_all('th',class_='covid-total-row')
    # We know that total_rows has 4 lines, so we will access them as individual.
    print('\n   ::: TOTAL NUMBERS :::')
    print('   Total Countries has COVID-19:   ',(total_rows)[0].b.text)
    print('   Total Cases of COVID-19:        ',(total_rows)[1].b.text)
    print('   Total Deaths by COVID-19:       ',(total_rows)[2].b.text)
    print('   Total Recoveries from COVID-19: ',(total_rows)[3].b.text)
    input('\n    ... Press any Key ...')
RUN-Time:


Second function we will work on is covid19_table, in this function we will print-out a table of covid19 contains countries Name, cases, Deaths and Recoveries sorted by Number of cases. In this version of application we will display only 10 records.


Enhancement: In a Coming version the user will select the number of records also the sorting type.

# Function to print-out the Covid19 cases.     
def covid19_table(top =10):
    """
    This Function will print-out the countries table sorted by Number of cases.
    If the user did not select number of countries then defult will be first 10.
    """
    os.system('clear')
    print('\n   The List of Countries has Covid-19, Sorted by Total Cases..')
    print('   ',last_update,'\n')
    
    r_spc = 30
    tab_offset = (r_spc-1)
    # To print the Table Header.
    print('-'*r_spc,'+','-'*20, '+','-'*21,'+','-'*20)
    print(' '*8,'{0:<20}  |       {1:<15}|      {2:<17}|     {3:<20}'.format('Countries','Cases','Deths','Recoveries'))
    print('-'*r_spc,'+','-'*20, '+','-'*21,'+','-'*20)
    
    droped_data =[]
    c = 2
    while c <= top :
        try:
            # This will print the country name.
            contry_name = trs[c].find('a',title=re.compile('2020 coronavirus pandemic in *'))
            tds = trs[c].find_all('td')
            c_offset = len(contry_name.text.strip())
            print('   ',c-1,'-',contry_name.text.strip(),' '*(tab_offset - c_offset),(tds[0].text).strip(),' '*(30 -(20 + len((tds[0].text).strip())) +10) ,(tds[1].text).strip(),' '*(30 -(20 + len((tds[1].text).strip())) +12),(tds[2].text).strip())
            
        except :
            droped_data.append([c])
        c = c + 1    
    print('\n\n    We have {} Droped Country(ies) due to some error'.format(len(droped_data)))
    input('\n   .. Press  Any Key ... ')


RUN-TIME


In the last Function, we will let the user to select the Country name then we will display the Covid19 information regarding his selection. So to perform this action, First we will create a list of all Countries Name, and display the list on the screen asking the user to select (Enter) the index number of any Country, then we search for that country name and grab it’s Covid19 Data. Here is the code..

# Function to grab Covid19 Data by country. 
  
def covid19_in_country() :

    os.system('clear')
    print('\n    Covid19 Country Data.')
    print('\n    ',last_update,'\n')

    # First we will build a list of counries name.
    c_name_list=[]
    total_rows = soup.find_all('th',class_='covid-total-row')
    Total_Countries = int((total_rows)[0].b.text)
    drops_c = 0
    for x in range (2,Total_Countries) :
        try:
            c_name_list.append([x,trs[x].find('a',title=re.compile('2020 coronavirus pandemic in *')).text])

        except:
            drops_c +=1

    print('    We miss {} Countries due to some errors.'.format(drops_c))
    print('\n    Enter the Country Number to show it''s Data.')
    for cou in range (0,(len(c_name_list)),4) :
        try:
            print ('    ',c_name_list[cou],c_name_list[cou+1],c_name_list[cou+2],c_name_list[cou+3])
        except:
            pass # If there is an error just pass.

    while True :
        user_c_index = (input('\n\n    Enter the Index number of the Country to see it''s Data. [e to Exit] '))
        try:
            if (isinstance(int(user_c_index), int)) :
                # This will print the country name.
                print('\n\n     ',trs[int(user_c_index)].find('a',title=re.compile('2020 coronavirus pandemic in *')).text)

                # This will print the country information. 
                tds = trs[int(user_c_index)].find_all('td')
                print('        Country Rank is: ',int(user_c_index)-1)
                print('        Cases:',tds[0].text.strip())
                print('        Deths:',tds[1].text.strip())
                print('        Recoveries:',tds[2].text.strip())
        except :
            if ((str(user_c_index)) in ['e','E']) :
                return
            else :
                print('\n    You must Enter a Number or ''e''/''E'' to exit')

    input('   ... Press Any Key ...')



This application and the code was written as a matter of training and learning. The owner/creator is not responsible for any missing or wrong data may exist in the file and/or the Data. (the code and/or the data-sets).



To Download my Python code (.py) files Click-Here




Follow me on Twitter..




By: Ali Radwani