Archive

Posts Tagged ‘Lesson’

Python: SQlite Project – P2

January 23, 2020 Leave a comment


Learning : Python and Sqlite3
Subject: Sqlite3, Database functions ” Employee App” P2

In Part1 of this project (Click to Read) we create the database and set the connection, also we create an Employee table with very basic fields and also we wrote a dummy_data() function to Insert some records into the table. And to make the application usable we wrote the Main-Men function and we test it with selecting to display the records that we have.

Today we will write other functions from our Menu. INSERT NEW EMPLOYEE: To Insert new employee we will ask the user to input or to fill the fields we have in our table such as First Name, Last Name and the Salary. .Let’s see the code ..

 # Insert Function

def insert_emp ():
   
    os.system("clear")
    print("\n\n ======== INSERT NEW RECORD ========")
    if input("\n Do you want to enter new employee data press. (y,n) ") in ["Y","y"] :
        f_name = input("    Enter the first name: ")
        l_name = input("    Enter the last name: ")
        p_pay = input("    Enter the salary: ")

        c.execute ("INSERT INTO emp (fname,lname,pay) VALUES(:fname,:lname, :pay)",{"fname":f_name,"lname":l_name, "pay":p_pay})
        db_conn.commit()
        print(input ("\n  One record has been Inserted. .. Press any key .. ."))
    else :
        print(input ("\n  Ok .. you don't want to enter any data. .. Press any key .. .")) 


So, if we select to Insert a new Employee and we Enter First name as : Jacob Last Name as: Noha also we we set the salary to 3200 and press Enter, as in this page ..

Then if we select to show all data we have in the database, we can see the new record added..



DELETE AN EMPLOYEE: To Delete or remove an employee from the database, First we will print-out all the records on the screen and ask the user to enter the ID_ number of the employee he want to delete. As shown here ..

In the above example we select ID number 3 to be deleted and press enter, the system will show the record and ask to confirm the deletion and wait for ‘Y’ to be pressed, then the record will be deleted.
..Here is the code..


In this post we cover the INSERT AND DELETE of the records from the database, in the next post we will cover the SEARCH AND EDIT Functions also some search conditions like salary range and group-by command.



Follow me on Twitter..




By: Ali Radwani




Python: My Fake Data Generator P-7

January 19, 2020 Leave a comment


Learning : Python: Functions, Procedures and documentation
Subject: About fake data P-7: (Fake File Name)

In this post we will write a function to generate a file name. Each file name consist of two part, first one is the name which present (or should present) a meaning part, then there is a dot (.) then mostly three characters showing the file type or extension.

Scope of work: Our files names will have 4 syllables all to gather will be one file name. Each syllables will be loaded in a variable as shown ..
1. fext: for Files extensions such as: (doc, jpeg, pdf, bmp …. and so on)
2. name_p1: Is a noun such as (customers, visitors, players .. . and so on )
3. name_p2: will be an nouns, adjective or characteristics such as (name, index, table .. .. and so on)
4. Then we will add a random integer number between (1,30) to give the file a version, or a number.

All parts or syllables will be as one file name.

Let’s Work: First we need to load the File name syllables from the json file called : file_dict.json

 # Loading the json from a url 

import json , requests, random

fname = "https://raw.githubusercontent.com/Ali-QT/Ideas-and-Plan/master/file_dict.json"

def call_json_url(fname):
    """
    Function to load the json file from URL.
    Argument: str :fname
    
    Return : dict: data
    """
    req = requests.get(fname)
    cont = req.content
    data = json.loads(cont)
    return data

fdict = call_json_url(fname)

# Save each syllables into variable. 
f_ext = fdict["fext"]
f_p1_name = fdict["name_p1"]
f_p2_name = fdict["name_p2"]


Now we will write the function that will generate the file name:

 # Function to generate the file name 

def generate_fname():
    """
    Function to generate a Fake files name.
    
    File Name consist of four syllables, Two names, a random number and an extension.
    First two syllables of the file name will be selected randomly from a dictuenary stored in a json       file.
    
    Return : str : f_file_name

    To read the information key in the json file use this code.
    ------ CODE TO READ DATA-SET INFORMATION --------------
     	for each in fdict["information"]:
          print(each,":",fdict["information"])

    ---END OF CODE ------------------------------------------
    """
    fp1 = (random.choice (f_p1_name)["n_p1"])
    fp2 = (random.choice (f_p2_name)["n_p2"])
    fp3 = (random.choice (f_ext)["ext"])

    f_file_name = (fp1 + "_" + fp2 + "_" + str(random.randint(1,30)) + "." + fp3)

    return f_file_name


Last thing we just will call the function for X numbers of files Name we want.

 # Generate 15 file Name. 

for x in range (15):
    generate_fname()

[Output]:

 
kids_name_15.ico
speakers_list_1.asp
cars_photos_27.csv
students_database_26.xml
kids_details_27.html
animals_index_10.mov
speakers_parameters_17.csv
drivers_name_8.doc
males_attributes_16.mov
players_sketches_11.py
animals_sketches_3.wav
cars_details_12.css
animals_list_17.txt
flowers_parameters_4.doc
players_database_28.log





:: Fake Function List ::

Function Name Description
Color To return a random color code in RGB or Hex.
Date To return a random date.
Mobile To return a mobile number.
Country To return a random country name.
City To return a random City name.
ID To return X random dig as ID.
Time To return random time.
Car’s Brand
file_name file name: list for fake file names.
Creatures Random animal names of a certain type: Mammals, Birds, Insect, Reptiles
Foods To return a random list of foods



Follow me on Twitter..




By: Ali Radwani




Python: My Fake Data Generator P-4

December 19, 2019 3 comments


Learning : Python: Functions, Procedures and documentation
Subject: About fake data P-4: (Fake Dates)

The fourth function of our Fake Data Generator will be the date function, from it’s name this one will generate a FAKE Date in yyyy/mm/dd format. The function will have one argument (go_back) for range, the max-limit is current date (today) and mini-limit will be (1/1/1900), if the user did’t pass any thing for (go_back) then the range is (from current to 1/1/1900) and if the user pass (X) then the range will be ((current date) to current – X_YEARS). Also in dates we need to take care of Leap Years, In leap year, the month of February has 29 days instead of 28. To solve this in our function we can use two ways, first one (the easy way) we know that we are generating a random numbers for months and days; so we can say if the month is February, then days can’t be more than 28. But if we want this thing to be more realistic we need to add more conditions such as :
1. The year can be evenly divided by 4.
2. If the year can be evenly divided by 100, it is NOT a leap year, unless the year is also evenly divisible by 400. Then it is a leap year (February has 29 days).



'''
10/12/2019
By: Ali Radwani
To get Fake Date.

'''

import random, datetime

def fdate(go_back = 0): 
    """      
        ###   Fake Date Generator V.01  ###
        Date: 10.12.2019, By: Ali Radwani

        This function will generate and return a fake date in string format.
        The function accept one int argument go_pback.
        If go_past = X, and current year - X is less than 1900 then
        the range of FAKE time will be (current year to current year - X).
    
        If NO argument passed to the function, then default limit set to 1900. 

        Default limits:  Date are from current (today) and back to 1900.

        Import: random, datetime
    
        Argument: int : go_back to set the upper limit of the date
        
        Return: str: dd/mm/yyyy  

    """

    # Get current year. 
    c_year = datetime.datetime.today().year

    # set the maximum year limit.
    if go_back > 0 :
        max_y_limit = c_year - go_back
    else :
        max_y_limit = 1900

    if max_y_limit < 1900 :
        max_y_limit = 1900

    yy = random.randint(max_y_limit, c_year)

    mm = random.randint(1,12)

    if mm in [1,3,5,7,8,10,12] :
        dd = random.randint(1,31)
    elif mm in [4,6,9,11]:
        dd = random.randint(4,30)

    else :
        # IF the month is February (2) 
        if (yy % 4 == 0 ) or ((yy % 100 == 0)and (yy % 400 == 0)):
            # It is a leap year February has 29 days.
            dd = random.randint(1,29)

        else : # it is NOT a leap year February has 28 days.
            dd = random.randint(1,28)

    d = (str(dd) +'/'+ str(mm)+'/' + str(yy))
    
    return (str(dd) +'/'+ str(mm)+'/' + str(yy))


# To check the output.
for x in range (30):
    print(fdate())


<
Here is a screenshot of the code, also available on the Download Page . . .



:: Fake Function List ::

Function Name Description
Color To return a random color code in RGB or Hex.
Date To return a random date.
Mobile To return a mobile number.
Country To return a random country name.
City To return a random City name.
ID To return X random dig as ID.
Time To return random time.
Car’s Brand
file_name

Done



Follow me on Twitter..




By: Ali Radwani




Python: My Fake Data Generator P-3

December 17, 2019 3 comments


Learning : Python: Functions, Procedures and documentation
Subject: About fake data P-3: (Fake Time)

The third function of our Fake Data Generator will be the Time function, Fake Time is very easy to implement, all we need is call random function two times, one for minutes (0,60) and another for hours (0,12 or 0,23) based on the argument s (style).

Let’s start: First we need to Import random, the function def ftime() will take one integer argument (s) represent the time style.
If the s = 12 then the time format will be regular start from 1 and end at 12, the number will be generated randomly using random.randint, also we will select random.choice([‘ AM’,’ PM’]) to be added to the time and return it back.
If the s = 24 or nothing been passed then the time format will start from 0 to 23 (Military Time Format). Another random integer (0,60) to be generated as minutes.


'''
  Fake Data Generator 
  Function for:  Fake Time
  Ali Radwani
  11/12/2019
'''

import random

def ftime(s = 24):

    """
        ###   Fake Time Generator   ###
        Date: 11.12.2019, By: Ali Radwani

        This function will generate a fake time, the
        function accept one int argument s.

        If s = 12 function return Regular time format, 
        If s = 24 function return military time format 
        (the 24 format system).

        If No argument passes then default time system
        format will be 24 system (military time)

        Argument: int : s, if No argument then default is 24.

        Return: str : ftimes
    """

    m = str(random.randint(0,60))
    if (len(m)) == 1 :
            m = '0' + str (m)

    if s == 12 :
        h = str(random.randint(1,12))
        if (len(h)) == 1:
            h = '0' + h
    else :
        h = str(random.randint(0,23))
        if (len(h)) == 1:
            h = '0' + h

    ftimes = str(h) + ':' + str(m)
    if s == 12 :
        ftimes = ftimes + random.choice([' AM',' PM'])

    return ftimes

# Testing the function.
for x in range (10):
    print (ftime(12))




:: Fake Function List ::

Function Name Description
Color To return a random color code in RGB or Hex.
Date To return a random date.
Mobile To return a mobile number.
Country To return a random country name.
City To return a random City name.
ID To return X random dig as ID.
Time To return random time.
Car’s Brand
Foods

Done



Follow me on Twitter..




By: Ali Radwani




Python: My Fake Data Generator P-2

December 15, 2019 3 comments


Learning : Python: Functions, Procedures and documentation
Subject: About fake data P-2: (Fake ID)

Before we start i’d like to mention that with our last fcolor() function we write some comments in the first part of the function between three double quote(“””), and if we load the function and call help() as help(fcolor()) we will get that information on the python console as a help as in screen shot.


In this post we will write a function to generate a fake ID number, for ID’s there could be several styles, sometime we just want a random number without any meaning; just X number of random digits. Most of the time we need this number to be mean-full based on certain rules. For example, in Banks they may use some digits that indicate the branch. In sport club, they may include the date … and so-on.

Here we will write a function called key_generator(), the function will take two arguments (dig, s) dig is your key digits number, s is the style, if s = d then the first 6 digits of the key will be the date as ddmmyy + random digits, and if s = anything else or s not passed then the key will be as default (just x-digits). Let’s see the code.

First the summary or say information about the function:

def key_generator(dig, s = 'n'):
    """
       ### Date: 8/12/2019, By: Ali Radwani ###
       Summary:
            This function will generate x-digit key randomly.
            If the argument s = 'd' or 'D' then the key is two part, first (6) digits
            are date as ddmmyy then x-digit random numbers.

            If the argument s anything else than ['d','D'] or no argument passes, then the key
            is random numbers without any meaning.

            The numbers will randomly be selected in range of (10 to 99).

            import: random, datetime

            Argument: int: dig: The number of digits for the key.
                 str: s  : The key style (with date or just random numbers)

            return: int: the_key
    """


Now, if the user pass s=’d’ then part of the key will be the current date, to do this we will call the datetime function in python and split it into dd,mm,yy. Here is the key_generator() function.

def key_generator(dig, s = 'n'):
    """
       ### Date: 8/12/2019, By: Ali Radwani ###
       Summary:
            This function will generate x-digit key randomly.
            If the argument s = 'd' or 'D' then the key is two part, first (6) digits
            are date as ddmmyy then x-digit random numbers.

            If the argument s anything else than ['d','D'] or no argument passes, then the key
            is random numbers without any meaning.

            The numbers will randomly be selected in range of (10 to 99).

            import: random, datetime

            Argument: int: dig: The number of digits for the key.
                 str: s  : The key style (with date or just random numbers)

            return: int: the_key
    """
    the_key=''
    if s in ['d','D'] :
        d = str(datetime.date.today())
        dd = d[8:10]
        mm = d[5:7]
        yy = d[2:4]
        the_key = dd + mm + yy
        for x in range (dig):
            the_key = the_key + str( random.randint(10,99))
        return int(the_key[:(dig + 6)])
        
    else :
        for x in range (dig):
            the_key = the_key + str( random.randint(10,99))

        return int(the_key[:dig])


In next Fake Data function we will try to write one to generate the date. It will be published on next Sunday.



:: Fake Function List ::

Function Name Description
Color To return a random color code in RGB or Hex.
Date To return a random date.
Mobile To return a mobile number.
Country To return a random country name.
City To return a random City name.
ID To return X random dig as ID.
Time To return random time.

Done



Follow me on Twitter..




By: Ali Radwani




Python: Machine Learning – Part 1

November 27, 2019 1 comment


Learning :Python and Machine Learning
Subject: Requirements, Sample and Implementation

Machine Learning: I will not go through definitions and uses of ML, I think there is a lot of other posts that may be more informative than whatever i will write. In this post I will write about my experience and learning carve to learn and implement ML model and test my own data.

The Story: Two, three days ago I start to read and watch videos about Machine Learning, I fond the “scklearn” site, from there I create the first ML to test an Iris data-set and then I wrote a function to generate data (my own random data) and test it with sklearn ML model.

Let’s start ..

Requirements:

1. Library to Import: To work with sklearn models and other functions that we will use, we need to import coming libraries:

import os # I will use it to clear the terminal.

import random # I will use it to generate my data-set.

import numpy as np

import bunch # To create data-set as object

from sklearn import datasets

from sklearn import svm

from sklearn import tree

from sklearn.model_selection import train_test_split as tts

2. Data-set: In my learning steps I use one of sklearn data-set named ” Iris” it store information about a flower called ‘Iris’. To use sklear ML Model on other data-sets, I create several functions to generate random data that can be passed into the ML, I will cover this part later in another post.
First we will see what is the Iris dataset, this part of information is copied from sklearn site.

::Iris dataset description ::
dataset type: Classification
contain: 3 classes, 50 Samples per class (Total of 150 sample)
4 Dimensionality
Features: real, positive

The data is Dictionary-like object, the interesting attributes are:
‘data’: the data to learn.
‘target’: the classification labels.
‘target_names’: the meaning of the labels.
‘feature_names’: the meaning of the features.
‘DESCR’: the full description of the dataset.
‘filename’: the physical location of iris csv.

Note: This part helps me to write me data-set generating function, that’s why we import the Bunch library to add lists to a data-set so it will appear as an object data-set, so the same code we use for Iris data-set will work fine with our data-set. In another post I will cover I will load the data from csv file and discover how to create a such file..

Start Writing the code parts: After I wrote the code and toned it, I create several functions to be called with other data-set and not hard-code any names in iris data-set. This way we can load other data-set in easy way.


The Code

 # import libraries 

import numpy as np
from sklearn import datasets
#from sklearn import svm
from sklearn import tree
from sklearn.model_selection import train_test_split as tts
import random, bunch


Next step we will load the iris dataset into a variable called “the_data”

 # loading the iris dataset. 

the_data = datasets.load_iris() 


From the above section “Iris dataset description” we fond that the data is stored in data, and the classification labels stored in target, so now we will store the data and the target in another two variables.

 # load the data into all_data, and target in all_labels. 
all_data= the_data.data 
all_labels = the_data.target   


We will create an object called ‘clf’ and will use the Decision Tree Classifier from sklearn.

 #  create Decision Tree Classifier 

clf = tree.DecisionTreeClassifier()


In Machine Learning programs, we need some data for training and another set of data for testing before we pass the original data or before we deploy our code for real data. The sklearn providing a way or say function to split a given data into two parts test and train. To do this part and to split the dataset into training and test I create a function that we will call and pass data and label set to it and it will return the following : train_data, test_data, train_labels, test_labels.

 #  Function to split a data-set into training and testing data. 

def get_test_train_data(data,labels):

  train_data, test_data, train_labels, test_labels = tts(data,labels,test_size = 0.1)
  return train_feats, test_feats, train_labels, test_labels


After splitting the data we will have four list or say data-sets, we will pass the train_data and the train_labels to the train_me() function, I create this function so we can pass the train_data, train_labels and it will call the (clf.fit) from sklearn. By finishing this part we have trained our ML Model and is ready to test a sample data. But first let’s see the train_me() function.

 #  Function train_me() will pass the train_data to sklearn Model. 

def train_me(train_data1,train_labels1):
  clf.fit(train_data1,train_labels1)
  print('\n The Model been trained. ')


As we just say, now we have a trained Model and ready for testing. To test the data set we will use the clf.predict function in sklearn, this should return a prediction labels list as the ML Model think that is right. To check if the predictions of the Model is correct or not also to have the percentage of correct answers we will count and compare the prediction labels with the actual labels in the test_data that we have. Here is the code for get_prediction()

 #  get_prediction() to predict the data labels. 

def get_prediction(new_data_set,test_labels2,accu):

  print('\n This is the prediction labels of the data.\n')

  # calling prediction function clf.predict
  prediction = clf.predict(new_data_set)
  print('\n prediction labels are : ',prediction,len(prediction))
  
  # print the Accuracy
  if accu == 't' :
    cot = 0
    for i in range (len(prediction)) :
      print(prediction[i] , new_data_set[i],test_labels2[i])
      if [prediction[i]] == test_labels2[i]:
        cot = cot + 1
    print('\ncount :',cot)
    print('\n The Accuracy:',(cot/len(prediction))*100,'%')


The accuracy value determine if we can use the model in a real life or tray to use other model. In the real data scenario, we need to pass ‘False’ flag for accu, because we can’t cross check the predicted result with any data, we can try to check manually for some result.

End of part 1: by now, we have all functions that we can use with our data-set, in coming images of the code and run-time screen we can see that we have a very high accuracy level so we can use our own data-set, and this will be in the coming post.

Result screen shot after running the Iris dataset showing high accuracy level.



Follow me on Twitter..





Python: Circle Packing

November 17, 2019 Leave a comment


Circle Packing Project
Subject: Draw, circles, Turtle

Definition: In geometry, circle packing is the study of the arrangement of circles on a given surface such that no overlapping occurs and so that all circles touch one another. Wikipedia

So, we have a canvas size (w,h) and we want to write a code to draw X number of circles in this area without any overlapping or intersecting between circles. We will write some functions to do this task, thous functions are:
1. c_draw (x1,y1,di): This function will take three arguments x1,y1 for circle position and di as circle diameter.

2. draw_fram(): This function will draw the frame on the screen, we set the frame_w and frame_h as variables in the setup area in the code.

3. c_generator (max_di): c_generator is the circles generating function, and takes one argument max_di presenting the maximum circles diameter. To generate a circle we will generate three random numbers for x position, y position and for circle diameter (max_di is the upper limit),also with each generating a while loop will make sure that the circle is inside the frame, if not regenerate another one.

4. can_we_draw_it (q1,di1): This is very important, to make sure that the circle is not overlapping with any other we need to use a function call (hypot) from math library hypot return the distance between two points, then if the distance between two circles is less than the total of there diameters then the two circles are not overlaps.



So, lets start coding …

First: the import and setup variables:


from turtle import *
import random
import math

# Create a turtle named t:
t =Turtle()
t.speed(0)
t.hideturtle()
t.setheading(0) 
t.pensize(0.5)
t.penup()

# frame size
frame_w = 500 
frame_h = 600 

di_list = [] # To hold the circles x,y and diameters


Now, Drawing the frame function:


def draw_fram () :

t.penup()

t.setheading(0)

t.goto(-frame_w/2,frame_h/2)

t.pendown()

t.forward(frame_w)

t.right(90)

t.forward(frame_h)

t.right(90)

t.forward(frame_w)

t.right(90)

t.forward(frame_h)

t.penup()

t.goto(0,0)


Now, Draw circle function:


def c_draw (x1,y1,di):

t.goto(x1,y1)

t.setheading(-90)

t.pendown()

t.circle(di)

t.penup()


This is Circles generator, we randomly select x,y and diameter then checks if it is in or out the canvas.


def c_generator (max_di):

falls_out_frame = True

while falls_out_frame :

x1 = random.randint(-(frame_w/2),(frame_w/2))

y1 = random.randint(-(frame_h/2),(frame_h/2))

di = random.randint(3,max_di)

# if true circle is in canvas

if (x1-di > ((frame_w/2)*-1)) and (x1-di < ((frame_w/2)-(di*2))) :

if (y1 ((frame_h/2)-(di))*-1) :

falls_out_frame = False

di_list.append([x1-di,y1,di])


With each new circle we need to check the distances and the diameter between new circle and all circles we have in the list, if there is an overlap then we delete the new circle data (using di_list.pop()) and generate a new circle. So to get the distances and sum of diameters we use this code ..

 # get circles distance

    cs_dis = math.hypot(((last_cx + last_cdi) - (c_n_list_x + c_n_list_di)) , (last_cy - c_n_list_y))
    di_total = last_cdi + c_n_list_di


To speed up the generation of right size of circles I use a method of counting the trying times of wrong sizes, that’s mean if the circles is not fit, and we pop it’s details from the circles list we count pops, if we reach certain number then we reduce the upper limits of random diameter of the new circles we generate. Say we start with max_di = 200, then if we pop for a number that divide by 30 (pop%30) then we reduce the max_di with (-1) and if we reach max_di less then 10 then max_di = 60. and we keep doing this until we draw 700 circles.


# if di_list pops x time then we reduce the randomization upper limits 
  if (total_pop % 30) == 0:
    max_di = max_di - 1
    if max_di < 10 :
      max_di = 60


Here are some output circles packing ..




With current output we reach the goal we are looking for, although there is some empty spaces, but if we increase the number of circles then there will be more time finding those area with random (x,y,di) generator, I am thinking in another version of this code that’s will cover:
1. Coloring the circles based on the diameter size.
2. A method to fill the spaces.



Follow me on Twitter..