Archive

Archive for the ‘Learning’ Category

Python: SQlite Project – P2

January 23, 2020 Leave a comment


Learning : Python and Sqlite3
Subject: Sqlite3, Database functions ” Employee App” P2

In Part1 of this project (Click to Read) we create the database and set the connection, also we create an Employee table with very basic fields and also we wrote a dummy_data() function to Insert some records into the table. And to make the application usable we wrote the Main-Men function and we test it with selecting to display the records that we have.

Today we will write other functions from our Menu. INSERT NEW EMPLOYEE: To Insert new employee we will ask the user to input or to fill the fields we have in our table such as First Name, Last Name and the Salary. .Let’s see the code ..

 # Insert Function

def insert_emp ():
   
    os.system("clear")
    print("\n\n ======== INSERT NEW RECORD ========")
    if input("\n Do you want to enter new employee data press. (y,n) ") in ["Y","y"] :
        f_name = input("    Enter the first name: ")
        l_name = input("    Enter the last name: ")
        p_pay = input("    Enter the salary: ")

        c.execute ("INSERT INTO emp (fname,lname,pay) VALUES(:fname,:lname, :pay)",{"fname":f_name,"lname":l_name, "pay":p_pay})
        db_conn.commit()
        print(input ("\n  One record has been Inserted. .. Press any key .. ."))
    else :
        print(input ("\n  Ok .. you don't want to enter any data. .. Press any key .. .")) 


So, if we select to Insert a new Employee and we Enter First name as : Jacob Last Name as: Noha also we we set the salary to 3200 and press Enter, as in this page ..

Then if we select to show all data we have in the database, we can see the new record added..



DELETE AN EMPLOYEE: To Delete or remove an employee from the database, First we will print-out all the records on the screen and ask the user to enter the ID_ number of the employee he want to delete. As shown here ..

In the above example we select ID number 3 to be deleted and press enter, the system will show the record and ask to confirm the deletion and wait for ‘Y’ to be pressed, then the record will be deleted.
..Here is the code..


In this post we cover the INSERT AND DELETE of the records from the database, in the next post we will cover the SEARCH AND EDIT Functions also some search conditions like salary range and group-by command.



Follow me on Twitter..




By: Ali Radwani




Python: SQlite Project – P1

January 21, 2020 1 comment


Learning : Python and Sqlite3
Subject: Sqlite3, Database functions ” Employee App” P1

One of the most important thing about writing applications is to save or store some Data for later use. This data could be simple as setting information or basic user information or can be a full records about something you need to keep, such as health data, or employee contacts or other projects data. To store a huge data we use a Data-Base, some databases are free and open to be downloaded from the internet, one of them is SQLITE3, in Python we can use the Sqlite for small and personal projects. In this post We will use the Sqlite to write a simple project for Employees.

As that our goal is to learn how to use Sqlite and python code to track our employees, and to keep every thing as simple as we can, we will collect only three piece of information that’s will be First Name, Last Name and the Salary.

Functions: In any application there must be several functions to complete our works and tasks that we need to perform, in our Employee System we need to perform these tasks:

1. Show the Data we have.
2. Insert New Employee.
3. Delete an Employee.
4. Editing Employee information.
5. Search for Employee.

This is the most important functions in any application, we will start working on the system and see how things goes on.

First we MUST Creating the data base and set the connection, here is the code to do this and we will call our database as test.db.

 # Create the database. 

import sqlite3, os

db_conn = sqlite3.connect ("test.db")    # set the data-base name

c = db_conn.cursor()  # set the connection


To create the Employee table we will write this code and run it only ONE Time.

 # Create the Employee Table. 

def create_tabels_() :    # to create tables.

    # employee table
    sql_s= "CREATE TABLE if not exists emp (emp_id INTEGER PRIMARY KEY AUTOINCREMENT,
fname text,lname text, pay integer)" c.execute(sql_s) db_conn.commit() print(input('\n .. Employee TABLE created.. Press any key .. '))


Since we are learning and playing with our code, we may need to drop the table for some reasons, so here is the code to Drop the table we will re-call the function if we need-so.

 # Function to DROP a Table. 

def drop_table(tname):

    c.execute("DROP TABLE {}".format(tname))
    db_conn.commit()


Now after the creating of the Table we need to feed it with some data so we can see the records. To do so we will run a function called dummy_data.

 # Function to INSERT Dummy data into the Employee Table. 


def dummy_data():
    """
        This Function will Insert 4 Dummy rows in the temp table, first record will set the emp_id to 1, the other
        record the emp_id will be AUTOINCREMENT.

        This Function to be run one time only.

    """
# First record will have the emp_id set as 1, other records will be AUTOINCREMENT.
    c.execute ("INSERT INTO emp (emp_id, fname,lname,pay) VALUES(:emp_id, :fname,
:lname, :pay)",{"emp_id":1,"fname":"James","lname":"Max", "pay":"2000"}) c.execute ("INSERT INTO emp (fname,lname,pay) VALUES(:fname,:lname, :pay)",
{"fname":"Robert","lname":"Ethan", "pay":"1500"}) c.execute ("INSERT INTO emp (fname,lname,pay) VALUES(:fname,:lname, :pay)",
{"fname":"Jack","lname":"Leo", "pay":"890"}) c.execute ("INSERT INTO emp (fname,lname,pay) VALUES(:fname,:lname, :pay)",
{"fname":"Sophia","lname":"Jack", "pay":"320"}) db_conn.commit() print(input('\n Dummy Data has been INSERTED\n\n .. Press any key .. '))


Main Menu To use the application we need a Menu to jump between the tasks in the app. Here is the Main-Menu, it will return the user selection.

 # The Main Menu. 

def menu():
    os.system("clear")
    print("\n\n      ::: The Menu :::")

    print("   1. Show the Data.")
    print("   2. Insert a New  Employee.")
    print("   3. Delete an Employee.")
    print("   4. Edit/Change employee data. ")
    print("   5. Search.")
    print("   6. Setting.")
    print("   7. Data-Base Information.")
    print("   9. Exit. ")

    uinput = input("\n     Enter a selection: ")

    return uinput


Here is the loop for the Menu and the user selection until (9. Exit) will be selected.

 # The Main Menu. 

while True :

    uinput = menu()

    if uinput == '1' :
        show_data()

    elif uinput =='2' :
        insert_emp ()

    elif uinput =='3' :
        delete_record()

    elif uinput =='4' :
        print("Edit")

    elif uinput =='5' :
        search_emp()

    elif uinput =='6' :
        setting_menu()

    elif uinput =='7' :
        #print("DataBase Information.")
        get_db_info()

    elif uinput =='9' :
        break

    else: # If the user select something out of the menu (Numbers or Character)
        print("  You need to select from the list")


Now we remember that we run the dummy_data() function (above) so we have four records in our Employee Table, so if we want to see the records we will select first option in the Main Menu: 1. Show the data. this will call a function called show_data() as in this screen shot.



The screen prompt will wait for an input of the number that present the task we want. So if we select No. 2 then we will get all the records in the table as this .. .



And here is the code behind this function..


Done with Part 1, in part 2 we will cover more functions to Search and Add records to the Table.



Follow me on Twitter..




By: Ali Radwani




Python: My Fake Data Generator P-6

December 26, 2019 1 comment


Learning : Python: Functions, Procedures and documentation
Subject: About fake data P-6: (Fake Food List)

In the last post the function we create def get_animal_name() get it’s data or let’s say select a random animal names from a dictionary data-set after loading the data from json file through URL, and I upload the Python functions on my Download Page so you can use it. Also I am working on standardizing my data-sets json file and adding information key for each file.

In this post we will write the code to print-out a random selected Food. The function will ask the user if he/she wants random list of Fruit, Vegetables or Meals. The Food data-set is growing up and i am adding more items to it, from the last update, the Meals contains: breakfast from UK, France and USA and Planning to add Dinners and Lunches recipes from more countries.

Let’s Start .. First we will load the json file, here is the code..

 #Calling and loading the json data-set file 

 #Importing libraries
import json , requests, random 


fname = "https://raw.githubusercontent.com/Ali-QT/Ideas-and-Plan/master/foods.json"

def call_json_url(fname): 
 #    """ 
    Function to load the json file from URL. 
    Argument : str  : fname 
    return   : dict : data 
    """ 
    req = requests.get(fname) 
    cont = req.content 
    data = json.loads(cont) 
    return data  # retuning the data-set

data = call_json_url(fname)


Now, we have several other functions in this application, first we will list them and then we will see the code in each one.
So here is the list:

  • get_type_index (adata,akey,subkey,aval)
    Function to return a list with index numbre of akey (atype).
    Argument: dic : adata, str : atype
    Return : list : ind_list
  • get_f_or_v(adata,atype)
    Function to print-out the Fruits or Vegetables as the user ask for.
    Argument: dict : adata, str : atype
  • get_meals (adata)
    Function to print-out the Meal as user will select. The user can select the Type of Meal: Breakfast, Dinner and Lunch. Also the user can select the countries of the Meals.

    Argument: dict: adata

  • def mmenu()
    This is the main menu to help the user to select whatever he/she want to be ptint-out.

Main Menu: Very short easy menu of four selection number as following:

 #The menu function


def mmenu() :
    """
        This is the main menu to help the user to select whatever want to be ptint-out.
    
    
    """
    print('\n\n  In the Food data-set we have several keys, here they are:')
    print('  Fruits, Vegetables and Meals. This application will select a random items\n  of your selection to print it on the screen.')
    print('  also you can select the number of items you want.\n ')
    print('  First, What type of Food do you want? ')
    print('   1. Fruits.')
    print('   2. Vegetables.')
    print('   3. Meals')
    print('   4. To exsit enter Q or q')

    ft = input('  Select one item:')
    if ft =='1' :
        get_f_or_v(data,'1')

    elif ft =='2' :
        get_f_or_v(data,'2')
    elif ft == '3' :
        get_meals (data)
    elif ft in ['q','Q']:
        print('Close')
        return 'q'
    else :
        print('\n   We only have three items') 
      


If the user select (1 or 2 ), then we will call the get_f_or_v(data,the user selection), the first argument is the data-set, the second one is the user selection between Fruit, Vegetables or Meal. Here is the function ..

 #Calling and Menu 

def get_f_or_v(adata,atype):
"""
       Function to print-out the Fruits or Vegetables as the user ask for.
       Argument:   dict   :  adata
                     str  :  atype      
    """
    if atype == '1': ft = 'fruits'  
    else: ft = 'vegetables'
    fts = len(adata[ft])
    while True :
        s = input('\n  You select {}, we have list of {} {}, So how many\n  {} you want to print: '.format(ft,fts,ft,ft))
     
        if int(s) < fts :
            break
        else : 
            print('  Enter a number in range 0 to {}.'.format(fts))
    print('Random {} names are:'.format(ft))
    for x in range (int(s)) :
        ind = random.randint(1,fts) 
        print('   ',adata[ft][ind]['name'])
    
    input('\n    ... Press any key ... ')


Last function we will see here is to get random Meals, I am loading the data from a json file that i create, for-now we have a breakfast only from UK, USA and France. If the user select the Meals from first menu, then another short menu will pop-up here it is ..

 #Calling get Meals 

def get_meals (adata) :	
	    """
	        Function to print-out the Meal as user will select. The user can select the Type of Meal: Breakfast, Dinner and Lunch.  
	        Also the user can select the countries of the Meals. 
	        
	        Argument:    dict: adata
	    """
	
    ft = 'meals'
    fts = len(adata[ft])
	 
    while True :
        s = input('\n  You select {}, we have list of {} {}, So how many\n  {} you want to print: '.format(ft,fts,ft,ft))
     
        if int(s) < fts :
            break
        else : 
            print('  Enter a number in range 0 to {}.'.format(fts))
        
    print('\n  For Meals we have Breakfast from three counties:')
    print(' 1. UK.')
    print(' 2. USA.')
    print(' 3. France.')
    co = input(' Select a country:')
    if co == '1' : co ='UK'
    elif co == '2' : co ='USA';
    else : co ='FR' 
          
    ind_list = get_type_index(adata,ft,'country',co) 
    
    for x in range (0,int(s)):
        ind = random.choice(ind_list)
        print(adata[ft][ind]['name'],'is a ', adata[ft][ind]['time'],'from ',adata[ft][ind]['country'])
  
    input('\n    ... Press any key ... ')


Notes that in def get_meals() I use a hard-coded keys for the menu. I am working on a function to read any json file and get a list of all keys and sub-keys in any level, then giving the user the ability to walk-through all the data. This will be in coming post.



:: Fake Function List ::

Function Name Description
Color To return a random color code in RGB or Hex.
Date To return a random date.
Mobile To return a mobile number.
Country To return a random country name.
City To return a random City name.
ID To return X random dig as ID.
Time To return random time.
Car’s Brand
file_name file name: list for fake file names.
Creatures Random animal names of a certain type: Mammals, Birds, Insect, Reptiles
Foods To return a random list of foods

Done



Follow me on Twitter..




By: Ali Radwani




Python: My Fake Data Generator P-4

December 19, 2019 3 comments


Learning : Python: Functions, Procedures and documentation
Subject: About fake data P-4: (Fake Dates)

The fourth function of our Fake Data Generator will be the date function, from it’s name this one will generate a FAKE Date in yyyy/mm/dd format. The function will have one argument (go_back) for range, the max-limit is current date (today) and mini-limit will be (1/1/1900), if the user did’t pass any thing for (go_back) then the range is (from current to 1/1/1900) and if the user pass (X) then the range will be ((current date) to current – X_YEARS). Also in dates we need to take care of Leap Years, In leap year, the month of February has 29 days instead of 28. To solve this in our function we can use two ways, first one (the easy way) we know that we are generating a random numbers for months and days; so we can say if the month is February, then days can’t be more than 28. But if we want this thing to be more realistic we need to add more conditions such as :
1. The year can be evenly divided by 4.
2. If the year can be evenly divided by 100, it is NOT a leap year, unless the year is also evenly divisible by 400. Then it is a leap year (February has 29 days).



'''
10/12/2019
By: Ali Radwani
To get Fake Date.

'''

import random, datetime

def fdate(go_back = 0): 
    """      
        ###   Fake Date Generator V.01  ###
        Date: 10.12.2019, By: Ali Radwani

        This function will generate and return a fake date in string format.
        The function accept one int argument go_pback.
        If go_past = X, and current year - X is less than 1900 then
        the range of FAKE time will be (current year to current year - X).
    
        If NO argument passed to the function, then default limit set to 1900. 

        Default limits:  Date are from current (today) and back to 1900.

        Import: random, datetime
    
        Argument: int : go_back to set the upper limit of the date
        
        Return: str: dd/mm/yyyy  

    """

    # Get current year. 
    c_year = datetime.datetime.today().year

    # set the maximum year limit.
    if go_back > 0 :
        max_y_limit = c_year - go_back
    else :
        max_y_limit = 1900

    if max_y_limit < 1900 :
        max_y_limit = 1900

    yy = random.randint(max_y_limit, c_year)

    mm = random.randint(1,12)

    if mm in [1,3,5,7,8,10,12] :
        dd = random.randint(1,31)
    elif mm in [4,6,9,11]:
        dd = random.randint(4,30)

    else :
        # IF the month is February (2) 
        if (yy % 4 == 0 ) or ((yy % 100 == 0)and (yy % 400 == 0)):
            # It is a leap year February has 29 days.
            dd = random.randint(1,29)

        else : # it is NOT a leap year February has 28 days.
            dd = random.randint(1,28)

    d = (str(dd) +'/'+ str(mm)+'/' + str(yy))
    
    return (str(dd) +'/'+ str(mm)+'/' + str(yy))


# To check the output.
for x in range (30):
    print(fdate())


<
Here is a screenshot of the code, also available on the Download Page . . .



:: Fake Function List ::

Function Name Description
Color To return a random color code in RGB or Hex.
Date To return a random date.
Mobile To return a mobile number.
Country To return a random country name.
City To return a random City name.
ID To return X random dig as ID.
Time To return random time.
Car’s Brand
file_name

Done



Follow me on Twitter..




By: Ali Radwani




Python: My Fake Data Generator P-3

December 17, 2019 3 comments


Learning : Python: Functions, Procedures and documentation
Subject: About fake data P-3: (Fake Time)

The third function of our Fake Data Generator will be the Time function, Fake Time is very easy to implement, all we need is call random function two times, one for minutes (0,60) and another for hours (0,12 or 0,23) based on the argument s (style).

Let’s start: First we need to Import random, the function def ftime() will take one integer argument (s) represent the time style.
If the s = 12 then the time format will be regular start from 1 and end at 12, the number will be generated randomly using random.randint, also we will select random.choice([‘ AM’,’ PM’]) to be added to the time and return it back.
If the s = 24 or nothing been passed then the time format will start from 0 to 23 (Military Time Format). Another random integer (0,60) to be generated as minutes.


'''
  Fake Data Generator 
  Function for:  Fake Time
  Ali Radwani
  11/12/2019
'''

import random

def ftime(s = 24):

    """
        ###   Fake Time Generator   ###
        Date: 11.12.2019, By: Ali Radwani

        This function will generate a fake time, the
        function accept one int argument s.

        If s = 12 function return Regular time format, 
        If s = 24 function return military time format 
        (the 24 format system).

        If No argument passes then default time system
        format will be 24 system (military time)

        Argument: int : s, if No argument then default is 24.

        Return: str : ftimes
    """

    m = str(random.randint(0,60))
    if (len(m)) == 1 :
            m = '0' + str (m)

    if s == 12 :
        h = str(random.randint(1,12))
        if (len(h)) == 1:
            h = '0' + h
    else :
        h = str(random.randint(0,23))
        if (len(h)) == 1:
            h = '0' + h

    ftimes = str(h) + ':' + str(m)
    if s == 12 :
        ftimes = ftimes + random.choice([' AM',' PM'])

    return ftimes

# Testing the function.
for x in range (10):
    print (ftime(12))




:: Fake Function List ::

Function Name Description
Color To return a random color code in RGB or Hex.
Date To return a random date.
Mobile To return a mobile number.
Country To return a random country name.
City To return a random City name.
ID To return X random dig as ID.
Time To return random time.
Car’s Brand
Foods

Done



Follow me on Twitter..




By: Ali Radwani




python: Fake Data-set

December 9, 2019 1 comment


Learning : Python to generate fake data-set
Subject: About Fake data library

Most of the time when we working on a project, we need to test our procedures and functions with some data. In most cases we need just dummy data such as dates, names, address .. and so-on.

Last week, I was reading on the net and i fond an article about generating fake data using a library in PHP (PHP is a Computer Programming Language) so I start to find if we have one in Python! and the answer is YES there is a library that we can import called ‘Fake’. I start to work on it and discover it. This post is about the Fake Data-set Library.

The library called ‘Faker’ and we need to install it in our python environment, i use : pip install Faker to install it. In it’s documentation we can use some properties like : name, city, date, job .. and others. So if we want to generate a fake name we write this:

# Using lib:fake to generate fake name

print(fake.name()) 
[Output]: Victoria Campbell

Here is a screen-shot from Jupyter notbook screen.


To generate more than one name we can use for loop as:

# Using lib:fake to generate (X) fake name

for x in range (10) :
    print(fake.name())
[Output]: Jared Hawkins
Michael Reid
Ricky Brown
Mary Tyler
Kristy Dudley
Karen Cain
Jennifer Underwood
Desiree Jensen
Carla Rivera
Brandon Cooper


Other properties that we can use are :address, company, job, country, date_time and many other, and with all this we can create a data-set full of fake data.

So if we want to create a fake data-set contain:
Name, Date-of-birth, Company, Job, Country as one person data we will use it like this:

# Using lib:fake to generate (X) person fake data
# Data-set contain: Name, Date-of-birth, Company, Job, Country
p_count = 1
for x in range (p_count):
    print('Name:',fake.name())
    print('DOB:',fake.date())
    print('Company:',fake.company())
    print('Job:',fake.job())
    print('country:',fake.country())


[Output]: 
Name: Crystal Mcconnell
DOB: 2002-09-30
Company: Bailey LLC
Job: Insurance underwriter
country: Pakistan


Now if we want to store the person data in a dictionary type variable and use it later, we can do this as following:

# Using lib:fake to generate (X) person fake data and store it in a dictionary 
people_d ={}
p_count = 5
for x in range (p_count):
    ID = x
    people_d[ID]={'name':fake.name(),'date':fake.date(),'company':fake.company(),'job':fake.job(),'country':fake.country()}

# To print-out the people_d data-set.
for x in people_d :
    print(people_d[x])


Just in case we want a complicated ID we can use a random function (8-dig) integer, or combining two fake numbers such as (fake.zipcode() and fake.postcode()) just to make sure that we will not have a duplicate ID.

Using fake library will help a lot, and it has many attributes and properties that can be inserted in a data-set. For more information on this document you may read it here: Fake Library



Follow me on Twitter..




By: Ali Radwani




Python: Machine Learning – Part 3

December 3, 2019 Leave a comment


Learning :Python and Machine Learning Part 3
Subject: Implementation and saving ML-Model

After creating a data-set and use it to train a ML model and make sure that it works fine and give a height accuracy predictions (Click here to read: Python and Machine Learning Part 2 ), we may or say we need to keep this model trained and re-use it on any actual data. In many real-life ML to training the model may take time with huge train data in image recognition or voice recognition models, so we need to keep the model trained even if we exit the application. To do this in sklearn we will use the “Model persistence” document page and use the joblib serialization.

First we need to import joblib , also import so to print out the file name and the path, we will use two functions in joblib (dump and load) in save_trained_model we will use the dump. Her is the code.

 # Function to save a trained ML-Model

  import joblib, os  # To Import  joblib and os
  
  def save_trained_model(model_name):
    
      print('\n  You select to save the trained ML model.')
      ml_name = input('  Enter a file name: ')
      joblib.dump(model_name, ml_name)
      print('\n  --> ML Model been saved.\n')
      print('   File Name is :',ml_name)  # To print-out the file name 
      print('   File Path is :',os.path.abspath(ml_name))  # To print-out the file path
      print('\n\n Do you want to save the ML trained Model? (Y,N): ' )
      if input('') in ['y','Y'] :
        save_trained_model(ML_trained_model)


Now after we save our trained ML-Model we want to load it and use it in our ML program without training our machine. I will use the function new_test_data() from part 2 and pass the ML trained model to it. And to do this, first we need to load the trained ML-Mode. So let’s do it.

 # Function to load trained ML-Model
  
def load_ML_Model(ML_filename):
    the_trained_model= joblib.load(ML_filename)
    
    return the_trained_model

# we call the function in the main application code.
ML_model = load_ML_Model(ML_t_model_filename)
 


And now we will call our new_test_data() function and pass ML_model to see the prediction.

 # Function to load trained ML-Model

  
def new_test_data(ML_model):
    print('\n\n====================================================')
    print('---------  START PREDICTION  for New Data Set ---------')
    print('\n   In this function a new data set will be generated, ')
    print('  and a trained ML-Model for "mouse on the coordinate plane" ')
    print('  will be loaded from the disk. So we will not train the Model.')
    #print('  So we will not train the Model. ')
    #print('  will use the IF loops.')
    
    new_data_size = 1000 
    new_data_range = 100
    print('\n\n  The new data range is {}, and the new data size is {}.'.format(new_data_range,new_data_size))
    
    # generate new data 
    new_test_data1= []
    for x in range (new_data_size):
        new_test_data1.append([round(random.uniform(-new_data_range,new_data_range),2),round(random.uniform(-new_data_range,new_data_range),2)])
    
    print('\n  This is the prediction for the New Data set..\n')
    # Do prediction using ML_model.
    prediction = ML_model.predict(new_test_data1)
    cot = 0
    # check the predictions accuracy .
    for i in range (len(prediction)) :
        if prediction[i] =='Up_r':
          if ((new_test_data1[i][0]) > 0 and (new_test_data1[i][1]) > 0) :
            cot = cot + 1
        elif  prediction[i] =='Up_l':
          if ((new_test_data1[i][0])  0) :
            cot = cot + 1
        elif  prediction[i] =='D_r':
          if ((new_test_data1[i][0]) > 0 and (new_test_data1[i][1]) < 0) :
            cot = cot + 1
        elif  prediction[i] =='D_l':
          if ((new_test_data1[i][0]) < 0 and (new_test_data1[i][1]) < 0) :
            cot = cot + 1
        
    print('\n  We count {} correct prediction out of {} Instances.'.format(cot,(new_data_size)))
    print('\n  The Accuracy is:',round((cot/len(prediction))*100,3),'%')

 




Follow me on Twitter..




By: Ali Radwani