30 days of Data Engineering: Day 2

Welcome back, peeps!

If you are looking for Day 1, Please visit my previous blog.

30 days of Data Engineering: Day 1

This is Day 2 of 30 days of the Data Engineering series where we will be covering Basic Python for Data Engineering.

Credit: wavelabs.ai/data/

2. Basic Python for Data Engineering

So, Let’s get started with Day 2 —
We will be covering below Python topics with a hands-on coding exercise

1. Python Data types
2. Python strings
3. String Methods
4. Python F Strings
5. Operators in Python
6. Chaining Comparison Operators with Logical Operators
7. Python Lists
8. List Comprehensions
9. Python Dictionaries
10. Python Tuples
11. Python Sets
12. Conditional Statements & Loops in Python
13. Break and Continue Statement
14. Input Output in Python
15. Python Object-Oriented Programming
16. Attributes and Class in Python
17. Functions in Python
18. Private Variables in Python
19. Global, Local, and Non-Local Variables in Python
20. First Class functions in Python
21. __import__() function
22. Tuple Unpacking with Python Functions
23. Static Variables and Methods in Python
24. Lambda Functions in Python
25. Map and Filter Functions in Python

Open up colab/jupyter notebook and start coding.
Let’s dive in!

Python is a high-level, most widely used multi-purpose, easy-to-read programming language.

It is —

• Interpreted − Python is processed at runtime by the interpreter. i.e you do not need to compile your program before executing it
• Interactive − You can interact with the interpreter directly to write your programs
• Portable — It can run on a wide range of hardware Platforms
• Object-Oriented − Python supports Object-Oriented style and Procedural Paradigms
• Used as a scripting language or can be compiled to byte-code for building large applications
• It has a simple structure and a clearly defined syntax
• Known as Beginner’s Language − It’s a great language for the beginner-level programmers
• Source code is comparatively easy-to-maintain
• It provides very high-level dynamic data types and supports dynamic type checking.
• Has a huge collection of the standard library

Popular applications for Python:

• Web Development
• Data Science — including machine learning, data analysis, and data visualization
• Scripting
• Game Development
• CAD Applications
• Embedded Applications

Python Data Types

Data Types represent the kind of value variables can hold and what all operations can be performed on a particular data.

Pic Credit: GeeksForGeeks

Implementation —

#Int data Type
var = 5
print(var)
print(type(var))

Output —

5
<class 'int'>

Implementation —

#Float data type
pvar2 = 0.1234
print(var2)
print(type(var2))

Output —

0.1234
<class 'float'>

Implementation —

#Complex numbers data type
x = 2j
print(x)
print(type(x))

Output —

2j
<class 'complex'>

Implementation —

#String Data Type
str_one = "Hello World"
print(str_one)
print(type(str_one))

Output —

Hello World
<class 'str'>

Implementation —

#list Data type
list1 = ["car","bike"]
print(list1)
print(type(list1))

Output —

['car', 'bike']
<class 'list'>

Implementation —

#Tuple Data Type
tup= ("car","bike","bus")
print(tup)
print(type(tup))

Output —

('car', 'bike', 'bus')
<class 'tuple'>

Implementation —

#Dictionary Data type
dict_one = {"Name": "Steve", "Location": "NewYork"}
print(dict_one)
print(type(dict_one))

Output —

{'Name': 'Steve', 'Location': 'NewYork'}
<class 'dict'>

Implementation —

#Set Data type
set_one = set({"Hello","world","Hello"})
print(set_one)
print(type(set_one))

Output —

{'world', 'Hello'}
<class 'set'>

Implementation —

#Frozen Set
f_one = frozenset({"Hello","World","Hello"})
print(f_one)
print(type(f_one))

Output —

frozenset({'Hello', 'World'})
<class 'frozenset'>

Implementation —

#Boolean Data Type
b = True
print(b)
print(type(b))

Output —

True
<class 'bool'>

Implementation —

#Byte Data Type
byte_one = b"World"
print(type(byte_one))
print(byte_one)

Output —

<class 'bytes'>
b'World'p

Python Strings

Strings are arrays of bytes representing Unicode characters.

• Strings in python are surrounded by either single quotation marks or double quotation marks.

‘today’ is the same as “today”.
Example:
var1 = ‘Hello’
var2 = “Complete Python Course”

• You can assign a multiline string to a variable by using three quotes.

Implementation —

#String
str1 = "Welcome to complete Python Course"
str2 = 'Welcome to the complete Python Course'
str3 = """This is a
       multiline 
       String"""

Output —

Welcome to complete Python Course
Welcome to the complete Python Course
This is a
       multiline 
       Stringpp

Indexing and Slicing with Strings —

• In python, Indexing is used to access individual characters of a string
• Square brackets are used to access the character of the string using Index
• The index starts with 0
• -1 refers to the last character, -2 refers to the second last character, and so on

Example:
var[3]
var[-2]

• In python, Slicing is used to access a range of characters in the string
• The slicing operator colon (:) is used

Example:
var[1:4]
var[:4]

Implementation —

# Indexing : String starts with 0th Index
s= "Captain America"
print(s[4])

Output —

a

Implementation —

#Slicing : slice(start, stop, step), it returns a sliced object containing elements in the given range
s= "Captain America"
print(s[1:5:2])

Output —

at

Implementation —

#Reverse
print(s[::-1])

Output —

aciremA niatpaC

String Methods

• istitle(): Checks for Titlecased String
• isupper(): returns if all characters are uppercase characters
• join(): Returns a Concatenated String
• ljust(): returns left-justified string of given width
• lower(): returns lowercased string
• lstrip(): Removes Leading Characters
• maketrans(): returns a translation table
• replace(): Replaces Substring Inside
• rfind(): Returns the Highest Index of Substring
• rjust(): returns right-justified string of given width
• rstrip(): Removes Trailing Characters
• split(): Splits String from Left
• splitlines(): Splits String at Line Boundaries
• startswith(): Checks if String Starts with the Specified String
• strip(): Removes Both Leading and Trailing Characters
• swapcase(): swap uppercase characters to lowercase; vice versa
• title(): Returns a Title Cased String
• translate(): returns the mapped character string
• upper(): returns uppercased string
• zfill(): Returns a Copy of The String Padded With Zeros
• capitalize(): Converts the first character to Capital Letter
• casefold(): converts to case-folded strings
• center(): Pads string with the specified character
• count(): returns occurrences of a substring in a string
• encode(): returns encoded string of given string
• endswith(): Checks if String Ends with the Specified Suffix
• expandtabs(): Replaces Tab character With Spaces
• find(): Returns the index of the first occurrence of the substring
• format(): formats string into the nicer output
• format_map(): Formats the String Using Dictionary
• index(): Returns Index of Substring
• isalnum(): Checks Alphanumeric Character
• isalpha(): Checks if All Characters are Alphabet
• isdecimal(): Checks Decimal Characters
• isdigit(): Checks Digit Characters
• isidentifier(): Checks for Valid Identifier
• islower(): Checks if all Alphabets in a String are Lowercase
• isnumeric(): Checks Numeric Characters
• isprintable(): Checks Printable Character
• isspace(): Checks Whitespace Characters

Implementation —

#capitalize() method : Returns a copy of the string with its first #character capitalized and the rest lowercased
a = "complete python course" 
print(a.capitalize())

Output —

Complete python course

Implementation —

#centre(width[, fillchar])   : Returns the string centered in a string of length width
a = "Python" 
b = a.center(10, "*")
print(b)

Output —

**Python**

Implementation —

# casefold() method : Returns a casefolded copy of the string. Casefolded strings may be used for caseless matching
a = "PYTHON"
print(a.casefold())

Output —

python

Implementation —

# count(sub[, start[, end]]) : Returns the number of non-overlapping occurrences of substring (sub) in the range [start, end]
a = "Welcome to complete Python Course"
print(a.count("c"))
print(a.count("o"))
print(a.count("Python"))

Output —

2
5
1

Implementation —

# endswith(suffix[, start[, end]]) : Returns True if the string ends with the specified suffix, otherwise it returns False
a = "Watermelon"
print(a.endswith("s"))
print(a.endswith("melon"))

Output —

False
True

Implementation —

# find(sub[, start[, end]]) : Returns the lowest index in the string where substring sub is found within the slice s[start:end]
a = "Exercise"
print(a.find("r"))
print(a.find("e"))

Output —

3
2

Implementation —

# index(sub[, start[, end]]) : Similar to find function, except that it raises a ValueError when the substring is not found
a = "Continent"
print(a.index("i"))
print(a.index("C"))
print(a.index("nent"))

Output —

4
0
5

Implementation —

# isalnum() : Returns True if all characters in the string are alphanumeric, else returns False
c = "456"
d = "$*%!!**"
print(c.isalnum())
print(d.isalnum())

Output —

True
False

Implementation —

# isalpha() : Returns True if all characters in the string are alphabetic, else returns False
c = "456"
d = "Python"
print(c.isalpha())
print(d.isalpha())

Output —

False
True

Implementation —

# isdecimal() : Returns True if all characters in the string are decimal characters, else returns False
c = u"\u00B10"
x = "10"
print(c.isdecimal())
print(x.isdecimal())

Output —

False
True

Implementation —

# isdigit() : Returns True if all characters in the string are digits, else returns False
c = "4567"
d = "1.65"
print(c.isdigit())
print(d.isdigit())

Output —

True
False

Implementation —

# join(iterable) : Returns a string which is the concatenation of the strings in iterable. 
# A TypeError will be raised if there are any non-string values in iterable
a = ","
print(a.join("CD"))

Output —

C,D

Implementation —

# partition(sep) : Splits the string at the first occurrence of sep, and returns a 3-tuple containing the part before the separator, the separator itself, and the part after the separator
a = "Complete.Python-course"
print(a.partition("-"))
print(a.partition("."))

Output —

('Complete.Python', '-', 'course')
('Complete', '.', 'Python-course')

Implementation —

# split(sep=None, maxsplit=-1) : Returns a list of the words in the string, using sep as the delimiter strip. 
# If maxsplit is given,at most maxsplit splits are done.If maxsplit is not specified or -1, then there is no limit on the number of splits.
a = "Welcome,,Friends,"
print(a.split(","))

Output —

['Welcome', '', 'Friends', '']

Implementation —

# strip([chars]) : Returns a copy of the string with leading and trailing characters removed. The chars argument is a string specifying the set of characters to be removed
a = "***Python***"
print(a.strip("*"))

Output —

Python

Implementation —

# swapcase() : Returns a copy of the string with uppercase characters converted to lowercase and vice versa
a = "Hi Homies"
print(a.swapcase())

Output —

hI hOMIES

Implementation —

# zfill(width) : Returns a copy of the string left filled with ASCII 0 digits to make a string of length width
a = "-124"
print(a.zfill(6))

Output —

-00124

Implementation —

# lstrip([chars]) : Return a copy of the string with leading characters removed. The chars argument is a string specifying the set of characters to be removed.
a = "*****Python-----"
print(a.lstrip("*"))

Output —

Python-----

Implementation —

# rindex(sub[, start[, end]]) : Just like rfind() but raises ValueError when the substring sub is not found
a = "Hi World"
print(a.rindex("d"))
print(a.rindex("W"))

Output —

7
3

Python F Strings

• Python F-String is used to embed python expressions inside string literals for formatting, using minimal syntax.
• It’s an expression that’s evaluated at the run time.
• They have the f prefix and use {} brackets to evaluate values.
• f-strings are faster than %-formatting and str.format()

Syntax —

f “string_variable”

• To format and output an expression in the formatted way, you should use curly braces {}

Implementation —

def max_no(x,y):
    return x if x>y else y

f_no= 12
s_no =25
print(f'Max of {f_no} and {s_no} is {max(f_no,s_no)}')

Output —

Max of 12 and 25 is 25

Implementation —

from decimal import Decimal
width = 4
round_point = 2
value = Decimal('12.39065')
print(f'result:{value:{width}.{round_point}}')

Output —

result:  12

Operators in Python

• In python, operators are used for performing some operations on variables and values

Arithmetic operators : +, — , *, /, //, %, **
Logical operators : and, or, not
Identity operators : is, is not
Membership operators : in , not in
Bitwise operators : &, |, ^,~, << , >>
Assignment operators : =, +=, -=, *=,/= , %=, //=, **=, &=, |=, ^=, >>=, <<=
Comparison operators : ==, !=, > , <, >=, <=

• Ternary operators are operators that evaluate things based on a condition being true or false

Syntax : [true] if [expression] else [false]

• Operator overloading can be implemented in Python

Example:
a & b
a >> 2
a is not b
‘b’ in list1

Implementation —

#Arithmetic Operators
x=10
y=4

#Addition
print("Addition:",x+y)
#Subtraction
print("Subtraction:",x-y)
#Multiply
print("Multiply: ",x*y)
#Division
print("Division:",x/y)
#Modulus
print("Modulus:",x%y)
#Floor Division
print("Floor Division:",x//y)
#Exponent
print("Exponent:",x**y)

Output —

Addition: 14
Subtraction: 6
Multiply:  40
Division: 2.5
Modulus: 2
Floor Division: 2
Exponent: 10000

Implementation —

#Comparison Operator  : Result is either True or False
x=5
y=3

#Greater than
print("Greater than:",x>y)
#Less than
print("Greater than:",x<y)
#Greater than equal to 
print("Greater than equal to:",x>=y)
#less than equal to
print("Less than:",x<=y)
#Not equal to 
print("Not equal to:",x!=y)
#Equal to
print("Equal to:",x==y)

Output —

Greater than: True
Greater than: False
Greater than equal to: True
Less than: False
Not equal to: True
Equal to: False

Implementation —

#Logical Operators : and, or, not [Result is either True or False]
x= True
y= False

#And
print("And result:",(x and y))
#Or
print("Or result:",(x or y))
#Not
print("Not result:",(not y))

Output —

And result: False
Or result: True
Not result: True

Implementation —

# Bitwise operators
x = 1001
y = 1010

#And
print("And result:",(x & y))
#Or
print("Or result:",(x | y))
#Not
print("Not result:",(~y))
#Xor
print("XOR result:",(x^y))
#Bitwise right shift
print("Bitwise right shift result:",(x>>2))
#Bitwise left shift
print("Bitwise left shift result:",(x<<2))

Output —

And result: 992
Or result: 1019
Not result: -1011
XOR result: 27
Bitwise right shift result: 250
Bitwise left shift result: 4004

Implementation —

# Assignment operators : used in Python to assign values to variables
x=5
x+=5
x-=2
x*=2
x**=2

Implementation —

# Identity Operator : is and is not are the identity operators in Python
x=5
y=5
z='a'
print("Is operator result:", (x is y))
print("Not is operator result:", (y is not z))

Output —

Is operator result: True
Not is operator result: True

Implementation —

#Membership operator : in operator
x = 'Python Course'
print('y' in x)
print('a' in x)

Output —

True
False

Chaining Comparison Operators with Logical Operators

• In python, to check more than two conditions, we implement chaining where two or more operators are chained together as shown in the example below

if x < y < z :
{…..}

• By associativity and precedence in Python, all comparison operations have the same priority. Resultant values of Comparisons yield boolean values such as either True or False
• When chaining the comparison operators, the sequence can be arbitrary.

Example:
x > y <= c is equivalent to x > y and y <= c

Implementation —

#Chaining Comparison operators with Logical operators
a, b, c, d, e, f, g = 10, 15, 2, 1, 45, 25, 19
e1 = a <= b < c > d < e is not f is g
e2 = a is d < f is c
print(e1)
print(e2)

Output —

False
False

Python Lists

• One of the most versatile data types in Python, Lists are used to store multiple items (homogeneous or non-homogeneous) in a single variable.
• Place the items inside the square brackets[]
• Items can be of any data type
• Lists are defined as objects with the data type ‘list’
• Items are ordered, changeable, and allow duplicate values
• list() constructor can be used when creating a new list
• To access values in lists, use the square brackets for slicing along with the index to obtain item value available at a particular index
• Items inside the list are indexed, the first item has an index [0], the second item has an index [1], etc

Example:
var=[1, 2 , ‘car’, ‘sunday’ , 3.14]
empty_list = []
items = list((“apple”, “banana”, “grapes”))

Implementation —

#Create a List
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
print(list_one)

Output —

['sunday', 'monday', 'tuesday', 'wednesday', 'thursday']

Implementation —

#List Length
print(len(list_one))

Output —

5

Implementation —

# List with different data types
list_two = ['abc',67,True,3.14,"female"]
print(list_two)

Output —

['abc', 67, True, 3.14, 'female']

Implementation —

#type() with List
print(type(list_two))

Output —

<class 'list'>

Implementation —

#list() constructor to make a List
list_cons = list(("hello","World","Beautiful","Day"))
print(list_cons)

Output —

['hello', 'World', 'Beautiful', 'Day']

Implementation —

# nested list
list_nest= ["hello",[8,4,6],['World']]
print(list_nest)

Output —

['hello', [8, 4, 6], ['World']]

Implementation —

#slice lists in Python : Use the slicing operator :(colon)
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
print(list_one[1:4])

Output —

['monday', 'tuesday', 'wednesday']

Implementation —

#Add/Change List Elements : use the assignment operator = to change an item
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_one[3] = 'friday'
print(list_one)

Output —

['sunday', 'monday', 'tuesday', 'friday', 'thursday']

Implementation —

# Appending and Extending lists in Python : Use the append() or extend() method
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_one.append('friday')
print(list_one)
#extend
list_one.extend(['saturday'])
print(list_one)

Output —

['sunday', 'monday', 'tuesday', 'wednesday', 'thursday', 'friday']
['sunday', 'monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday']

Implementation —

# Concatenating and repeat lists : use + operator to concate two lists and use * operator to repeat lists
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
print(list_one + [0,1,2,3,4])

#repeat operation 
print(['a','b']*2)

Output —

['sunday', 'monday', 'tuesday', 'wednesday', 'thursday', 0, 1, 2, 3, 4]
['a', 'b', 'a', 'b']

Implementation —

# Delete/Remove List Elements : delete one or more items or entire list using the keyword del
del list_one[2]
print(list_one)
#remove method : remove the given item or pop() method to remove an item at the given index location
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_one.remove("tuesday")
print(list_one)
#pop method
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_one.pop(2)
print("Pop result:", list_one)

Output —

['sunday', 'monday', 'thursday']
['sunday', 'monday', 'wednesday', 'thursday']
Pop result: ['sunday', 'monday', 'wednesday', 'thursday']

Implementation —

# index() method : Returns the index of the first matched item
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
print(list_one.index("tuesday"))

Output —

2

Implementation —

# sort() method: Sort items in a list in ascending order
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_one.sort()
print(list_one)

Output —

['monday', 'sunday', 'thursday', 'tuesday', 'wednesday']

Implementation —

# reverse() : Reverse the order of items in the list
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_one.reverse()
print(list_one)

Output —

['thursday', 'wednesday', 'tuesday', 'monday', 'sunday']

Implementation —

# copy(): Returns a shallow copy of the list
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_two = list_one.copy()
print(list_two)

Output —

['sunday', 'monday', 'tuesday', 'wednesday', 'thursday']

Implementation —

#Membership : check if an item exists in a list or not, using the keyword in
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
print('tuesday' in list_one)

Output —

True

Implementation —

# insert() method : insert item at a desired location
list_one = ["sunday","monday","tuesday","wednesday","thursday"]
list_one.insert(2,'friday')
print(list_one)

Output —

['sunday', 'monday', 'friday', 'tuesday', 'wednesday', 'thursday']

List Comprehensions

• In python, list comprehensions are used to create a new list based on the values of an existing list most elegantly and shortly.
• A list comprehension consists of an expression followed by for statement inside square [] brackets.

Example:
new_list = [x for x in list1 if “a” in x]

Implementation —

sqr = [2**x for x in range(20)]
print(sqr)

Output —

[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536, 131072, 262144, 524288]

Python Dictionaries

• In python, the dictionary is an unordered collection of data values in which data values are stored in key:value pairs
• Created by placing a sequence of elements within curly {} braces, separated by ‘, ‘
• Values can be of any datatype and can be duplicated, whereas keys are immutable and can’t be repeated
• Can be created by the built-in function dict()
• Dictionaries are defined as objects with the data type ‘dict
• dict() constructor can be used when creating a new dict
• To access values in dict, use the keys
• Key Value format makes dictionary one of the most optimized and efficient data types in Python

Example:
var={ ‘first_day’: ‘sunday’ , ‘second_day’: ‘monday’, ‘third_day’: ‘tuesday’}
empty_dict = {}
class dict(**kwarg)

Implementation —

#Create a Dictionary
#empty dictionary
dict_emp = {}

#dict with items
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(dict_one)

Output —

{0: 'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}

Implementation —

#Accessing Elements from Dictionary : Using keys or get() method
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(dict_one[1])

#get() method
print(dict_one.get(2))

Output —

monday
tuesday

Implementation —

# Length of Dictionary
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(len(dict_one))

Output —

5

Implementation —

#Changing and Adding Dictionary elements: add new items or change the value of existing items using an '=' operator
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}

#change element
dict_one[2] = 'friday'
print("After changing the element", dict_one)

#Add element
dict_one[5] = 'saturday'
print("After adding the element :", dict_one)

Output —

After changing the element {0: 'sunday', 1: 'monday', 2: 'friday', 3: 'wednesday', 4: 'thursday'}
After adding the element : {0: 'sunday', 1: 'monday', 2: 'friday', 3: 'wednesday', 4: 'thursday', 5: 'saturday'}

Implementation —

#Removing elements from Dictionary : Use the pop() or popitem() method
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(dict_one.pop(2))

#popitem : remove an arbitrary item and return (key,value)
print(dict_one.popitem())

Output —

tuesday
(4, 'thursday')

Implementation —

# remove all items : using clear method
dict_one.clear()
print(dict_one)

Output —

{}

Implementation —

#fromkeys(seq[, t]): Returns a new dictionary with keys from seq and value equal to t
subjects = {}.fromkeys(['Computer Science','Space Science','Math','English'])
print(subjects)

Output —

{'Computer Science': None, 'Space Science': None, 'Math': None, 'English': None}

Implementation —

#items() method : displays a list of dictionary's (key, value) tuple pairs
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(dict_one.items())

Output —

dict_items([(0, 'sunday'), (1, 'monday'), (2, 'tuesday'), (3, 'wednesday'), (4, 'thursday')])

Implementation —

#keys() method : displays a list of all the keys in the dictionary
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(dict_one.keys())

Output —

dict_keys([0, 1, 2, 3, 4])

Implementation —

#values() method : displays a list of all the values in the dictionary
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(dict_one.values())

Output —

dict_values(['sunday', 'monday', 'tuesday', 'wednesday', 'thursday'])

Implementation —

#setdefault() method : returns the value of a key. If not there, it inserts key with a value to the dictionary
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
element = dict_one.setdefault(3)
print(element)

#If key not present 
element = dict_one.setdefault(6)
print("If key is not present:", dict_one.items())

Output —

wednesday
If key is not present: dict_items([(0, 'sunday'), (1, 'monday'), (2, 'tuesday'), (3, 'wednesday'), (4, 'thursday'), (6, None)])

Implementation —

#Nested Dictionaries
people = {
              "subject": {0:"Maths",1:"English",3:"Science"},
              "marks": {0:42,1:36,2: 78},
              "Age": {0:12,1:34,2:19}
         }

print(people["marks"][0])
print(people["Age"][2])

Output —

42
19

Implementation —

#sorted(): Return a new sorted list of keys in the dictionary
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(sorted(dict_one))

Output —

[0, 1, 2, 3, 4]

Implementation —

#Iterate through dictionay
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
for i in dict_one.items():
    print(i)

Output —

(0, 'sunday')
(1, 'monday')
(2, 'tuesday')
(3, 'wednesday')
(4, 'thursday')

Implementation —

# Dictionary Comprehension
cubes = {x: x*x*x for x in range(10)}
print(cubes)

Output —

{0: 0, 1: 1, 2: 8, 3: 27, 4: 64, 5: 125, 6: 216, 7: 343, 8: 512, 9: 729}

Implementation —

# update() method : updates the dictionary with the elements from another dictionary object or from any other key/value pairs
dict1 ={0:"zero",4:"four",5:"five"}
dict2={2:"two"}

# updates the value of key 2
dict1.update(dict2)
print(dict1)

Output —

{0: 'zero', 4: 'four', 5: 'five', 2: 'two'}

Implementation —

# Membership Test : check if a key is in a dictionary or not using the keyword 'in'
dict_one = {0:'sunday', 1: 'monday', 2: 'tuesday', 3: 'wednesday', 4: 'thursday'}
print(0 in dict_one.keys())

Output —

True

Python Tuples

• In python, a tuple is a collection of objects which is ordered and immutable
• Created by placing a sequence of elements within a round () braces, separated by ‘, ‘
• Values can be of any datatype
• Concatenation of tuples can be done by the use of the ‘+’ operator
• Tuples are just like lists except that the tuples are immutable i.e cannot be changed

Example:
var= (4, ‘Hello’, 5, ‘World’)
empty_tuple = ()

Implementation —

# Tuple with items
tup= (10,"Hello",3.14,"a")
print(tup)
print(type(tup))

Output —

(10, 'Hello', 3.14, 'a')
<class 'tuple'>

Implementation —

# Negative Indexing : index of -1 refers to the last item, -2 to the second last item and so on
tup = (10,"Hello",3.14,"a")
print(tup[-2])

#Reverse the tuple
print(tup[::-1])

Output —

3.14
('a', 3.14, 'Hello', 10)

Implementation —

#concatenation and repeat in Tuples concatenation using + operator
tup = (10,"Hello",3.14,"a")
print(tup + (50,60))

#repeat using * operator
print(tup * 2)

Output —

(10, 'Hello', 3.14, 'a', 50, 60)
(10, 'Hello', 3.14, 'a', 10, 'Hello', 3.14, 'a')

Implementation —

#membership : check if an item exists in a tuple or not, using the keyword in
tup = (10,"Hello",3.14,"a")
print(10 in tup)
print("World" in tup)

Output —

True
False

Implementation —

#Iterate through Tuple : use for loop to iterate through each item in a tuple
tup = (10,"Hello",3.14,"a")
for i in tup:
    print(i)

Output —

10
Hello
3.14
a

Implementation —

#Nested Tuple
nest_tup = ((10,"Hello",3.14,"a"), (70,(8,"Mike")))
print(nest_tup)a,b = nest_tup
print(a)
print(b)

Output —

((10, 'Hello', 3.14, 'a'), (70, (8, 'Mike')))
(10, 'Hello', 3.14, 'a')
(70, (8, 'Mike'))

Implementation —

#Enumerate : use enumerate function
tup = (10,"Hello",3.14,"a")
for i in enumerate(tup):
    print(i)

Output —

(0, 10)
(1, 'Hello')
(2, 3.14)
(3, 'a')

Python Sets

• In python, a set is a collection of objects which is both unindexed and unordered
• Sets make sure that there are no duplicate elements in the sequence of the items
• Created by using the built-in set() function with an iterable object by placing the items inside curly {} braces, separated by ‘,’
• Items can be added to the set by using the built-in add() function
• Items can be accessed by looping through the set using loops or using the ‘in’ keyword
• Items can be removed from the set by using built-in remove()

Example:
var= set([“Hello”, “People”, “Hello”])

Implementation —

#Create Set
set_one = {10, 20, 30, 40}
print(set_one)

#Create set from list using set() 
set_two = set([10, 20, 30, 40, 30, 20])
print(set_two)

Output —

{40, 10, 20, 30}
{40, 10, 20, 30}

Implementation —

# Removing elements : Use the methods discard(), pop() and remove()
set_one = {100, 70, 40, 10, 80, 20, 60, 30}

#discard() method
set_one.discard(100)
print("After discard:",set_one)

#remove() method
set_one.remove(40)
print("After removing element:", set_one)

#pop() method
set_one.pop()
print("After removing element:", set_one)

Output —

After discard: {70, 40, 10, 80, 20, 60, 30}
After removing element: {70, 10, 80, 20, 60, 30}
After removing element: {10, 80, 20, 60, 30}

Implementation —

#Set operations 
X = {10, 20, 30, 40, 50}
Y = {40, 50, 60, 70, 80}
Z = {20, 30, 100, 50, 10}

#Union : Union of X, Y, Z is a set of all elements from all three #sets using | operator or union() method
print("Set Union:", X|Y|Z)

#Intersection :Intersection of X, Y, Z is a set of all elements from #all three sets using & operator or intersection()
print("Set Intersection:", X&Y&Z)

#Difference : Difference of X, Y is a set of all elements from both #sets using - operator or difference()
print("Set Difference:", X-Y)

# Symmetric Difference : Symmetric Difference of X, Y, Z is a set of #all elements from all three sets using ^ operator or #symmetric_difference()            
print("Set Symmetric Difference:", X^Y^Z)

Output —

Set Union: {100, 70, 40, 10, 80, 50, 20, 60, 30}
Set Intersection: {50}
Set Difference: {10, 20, 30}
Set Symmetric Difference: {80, 50, 100, 70, 60}

Implementation —

#enumerate : Returns an enumerate object which contains the index #and value for all the items of the set as a pair
set_one = {10, 20, 30, 40, 50, 30}
for i in enumerate(set_one):
    print(i)

Output —

(0, 40)
(1, 10)
(2, 50)
(3, 20)
(4, 30)

Implementation —

#Frozenset : set which has the characteristics of a set, but its elements cannot be changed once assigned
X = frozenset([10, 20, 30, 40, 50])
Y = frozenset([40, 50, 60, 70, 80])
print(X.union(Y))

Output —

frozenset({70, 40, 10, 80, 50, 20, 60, 30})

Conditional Statements & Loops in Python

Conditional Statements —

• In python, If, Elif, and Else statements are used to facilitate decision making i.e when we want to execute a code only if a certain condition is satisfied.
• In the example below, the program evaluates the test expression and will execute statement(s) only if the test expression is True. If the test expression1 is False, then it checks elif test expression2 and if that’s also False, then at last it goes to else and executed the else statement.

if test_expression1:
statement(s)
elif test_expression2:
statements(s)
else :
statement

• Python interprets non-zero values as True. None and 0 are interpreted as False.
• The if block can have only one else block but there can be multiple elif blocks.

While loops —

• In python, while loop is used to traverse/iterate over a block of code as long as the test condition is true
• In the example below, test_expression is checked first. The body of the loop is entered only if the test_expression evaluates to True. The loop continues till the test condition becomes False.

Syntax:
while test_expression:
Do this

Example:
while i <= 10:
sum = sum — i
i = i+1

For Loops and Range function —

• In python, for loop is used to traverse/iterate over a sequence (list, tuple, string, etc)
• In the example below, i is the variable that takes the value of the item inside the sequence on each iteration. The loop continues until we reach the last item in the sequence

Syntax:
for i in sequence:

Do this

• Range (range()) is used to generate a sequence of numbers where the syntax of the range is range(start, stop, step_size)

Example:
for i in range(len(list1)):
print(“The element is “, list1[i])

Implementation —

price = 200
if price > 120:
    print("Price is greater than 120")
elif price == 120:
    print("Price is 120")
elif price < 120:
    print("Price is less than 120")
else:
    print("Exit")

Output —

Price is greater than 120

Implementation —

# If, Elif and Else one liner
x = 200
var = { x < 190: "Condition one satisfied", x != 200: "Condition two satisfied"}.get(True, "Condition third satisfied") 
print(var)

Output —

Condition third satisfied

Implementation —

#while with else statement
i = 1
while i < 6:
  print(i)
  i += 1
else:
  print("i is no longer less than 6")

Output —

1
2
3
4
5
i is no longer less than 6

Implementation —

# for loop with range function
days =['sunday','monday','tuesday']
for i in range(len(days)):
    print("Today is", days[i])

Output —

Today is sunday
Today is monday
Today is tuesday

Break and Continue Statement

• In python, we can use a break statement when we want to terminate the current loop without checking the test condition
• Once terminated using the break statement, the control of the program goes to the statement immediately after the body of the loop

Example :
for value in list1:
if value == “t”:
break
print(value)

• In python, we can use a continue statement when we want to skip the rest of the code for the current loop iteration

Example :
for value in list1:
if value == “t”:
continue
print(value)

Implementation —

#break statement
count = 0
while True:
    print(count)
    count += 1
    if count >= 10:
        breakprint('exit')

Output —

0
1
2
3
4
5
6
7
8
9
exit

Implementation —

#Continue statement
for x in range(15):
    if x % 2 == 0:
        continue
    print(x)

Output —

1
3
5
7
9
11
13

Input Output in Python

• In Python, there are two inbuilt functions to read the input from the user
• raw_input( ) function: reads one line from user input and returns it as a string
• input( ) function: Similar to raw_input, except it evaluates the user expression
• For multiple user inputs, we can use —
  split() method

List comprehension

• In python, the output is using the print() function
• String literals in print() statements are used to format the output

\n: Add blank new line
“”: To print an empty line.

• end keyword is used to print specific content at the end of the execution of the print() function

Example:
num= input(“Enter the number: “)
str = raw_input(“Enter your input: “)
print(“Python” , end = ‘**’)

Implementation —

# input
num = int(input('Enter a number: '))

Output —

Enter a number: 50

Implementation —

num = 5
print('The value of num is', num)
print("The value is %d" %num)

Output —

The value of num is 5
The value is 5

Python Object-Oriented Programming

• Python is a multi-paradigm programming language and supports Object Oriented programming. In Python, everything is an object. An object has two characteristics: Attributes and Behavior
• The principles of an object-oriented programming system are —
  — Class
  — Object
  — Method
  — Inheritance
  — Polymorphism
  — Encapsulation

Details of the principles of an object-oriented programming system are:

• Class and constructor — It’s a blueprint for the object. In python, we use the class keyword to define a class. Class constructor is used to assigning the values to the data members of the class when an object of the class is created.
• Object — It’s an instantiation of a class.
• Method — It’s a function that is associated with an object
• Inheritance — Specifies that the child object acquires all the properties and behaviors of the parent object.
• Polymorphism — Refers to functions having the same names but carrying different functionalities.
• Encapsulation — To prevent data from direct modification, we can restrict access to methods and variables in python

Attributes and Class in Python

• In Python, a class is a blueprint of the object
• To define a class, we use the keyword “class” following the class name and semicolon —
  class class_name:
  Body of the class
• Object — It’s an instantiation of a class. The object instance contains real data or value
  To create an instance:
  obj1 = class_name()
• Class constructor — to assign the values to the data members of the class when an object of the class is created, we use the constructor. The __init__() method is called constructor method
  class class_name:
  def __init__(self, parameters):
  self.param1 = parameters
• Instance attributes refer to the attributes inside the constructor method. Class attributes refer to the attributes outside the constructor method
• Method — It’s a function that is associated with an object, used to describe the behavior of the objects
  def method_name(self)

Implementation —

#Class implementation
class cat:
    def __init__(self, cat_name, cat_breed):
        self.name = cat_name
        self.age = cat_breed

#Class Attribute and Instance Attribute
class emp:
    x = 10      #class attribute
    def __init__(self):
        self.name = 'Steve'
        self.salary = 10000
  
    def display(self):
        print(self.name)
        print(self.salary)
  
obj_emp = emp()
print("Dictionary conversion:", vars(obj_emp))

Output —

Dictionary conversion: {'name': 'Steve', 'salary': 10000}

Functions in Python

In python, functions are a convenient way to divide your code into useful blocks, allowing us to order our code, make it more readable, reuse it, define interfaces, and save a lot of time

• A function can be called multiple times to provide modularity and reusability to the python program
• We can easily track a large python program when it is divided into multiple functions
• There are mainly two types of functions —
  — User-defined functions: Defined by the user to perform the specific task
  — Built-in functions: Functions that are pre-defined
• You can define the function using the def keyword
• Arguments are the information that’s passed into the function
• The return statement is used to return the value. A function can have only one return
• Once created, we can call the function using the function name followed by the parenthesis

Example :
def my_function(parameters):
print(“My first Function”)
return expression

#calling the function
my_function()

Private Variables in Python

In python, a variable is a named location used to store or hold the value/data in the memory.

• When we create a variable inside a function, it is local by default
• We create Private Variables by using the underscore _ before a named prefix
• This is quite helpful for letting subclasses override methods without breaking intraclass method calls
• There are three ways private variables can be implemented :
  1.) _Single Leading Underscores
  2.) __Double Leading Underscores
  3.) __Double leading and Double trailing underscores__
• __name__ is a special built-in variable that points to the name of the current module

Example :
_program
__var
__len__

Implementation —

#Single Underscore (_)
class test:
    def __init__(self,num):
        self._num= num

    def _numfunc(self):
        print("Hello")

obj=test(156)
obj._numfunc()#_ attributes can be accessed as normal variables

Implementation —

#Double Underscore (__)
class test:
    def __init__(self,num):
        self.__num= num
    def display(self):
        print("__num = {}".format(self.__num))

obj=test(156)
obj.display()

Implementation —

#Trailing Underscore(n_)
class Test:
    def __init__(self,num):
        #To avoid clash with python keyword 
        self.num_= num

Global, Local, and Non-Local Variables in Python

When we create a variable inside a function, it is local by default. When we define a variable outside of a function, it is global by default

• Nonlocal variables are used in nested functions whose local scope is not defined
• Global variables are those variables that are defined and declared outside a function and we can use them inside the function or we can use the global keyword to make a variable global inside a function

Example :
global variable_name

Implementation —

#Local Variable
def test():
    l = "local_variable"
    return l

var=test()
print(var)

Output —

local_variable

Implementation —

#Global Variable
var = 10
def test():
    var = 20
    print("local variable x:", var)

val=test()
print("global variable x:", var)

Output —

local variable x: 20
global variable x: 10

Implementation —

#Non Local Variable
def test():
    var = "local_variable"
    def test_one():
        nonlocal var
        var = "non_local_variable"
        print("Non Local value:", var)
    test_one()
    print("Outer value:", var)
test()

First Class functions in Python

In python, a function is First Class Function, if it supports all of the properties of a First Class object such as —
1.) It is an instance of an Object type
2.) Pass First Class Function as argument of some other functions
3.) Return Functions from other function
4.) Store Functions in lists, sets, or some other data structures
5.) Functions can be stored as a variable

• In Python, a function can be assigned as a variable that is used without function parentheses
• Functions are objects you can pass as arguments to other functions
• First-class functions allow you to pass around behavior and abstract
• Closure functions — Functions are nested and they can carry forward the parent function’s state with them

Example:
def func1(func):
var = func(‘Hello World’)
print(var)

Implementation —

#Implementation 2
def outer(a):   # Outer function
    def inner(b):   # Inner function
        return b+10
    return inner(a)a = 10
var = outer(a) 
print(var)

Output —

20

__import__() function

In python, the inbuilt __import__() function helps to import modules in runtime

Syntax —

__import__(name, globals, locals, fromlist, level)

• name: Name of the module to import.
• globals: Dictionary of global names used to determine how to interpret the name in a package context.
• locals: Dictionary of local names used to determine how to interpret the name in a package context.
• fromlist: It gives the names of objects or submodules that should be imported from the module given by name.
• level: level specifies whether to use absolute or relative imports.
• To import a module by name, use importlib.import_module()

Example:
_var = __import__(‘spam.ham’, globals(), locals(), [], -1)

Implementation —

#implementation
#fabs() method is defined in the math module which returns the absolute value of a number
math_score = __import__('math', globals(), locals(), [], 0)
print(math_score.fabs(-17.4))

Output —

17.4

Tuple Unpacking with Python Functions

In Python, tuples are immutable data types. Python offers a very powerful tuple assignment tool that maps right-hand side arguments into left-hand side arguments i.e mapping is known as unpacking of a tuple of values into a normal variable.

• During the unpacking of the tuple, the total number of variables on the left-hand side should be equivalent to the total number of values in a given tuple
• It uses a special syntax to pass optional arguments (*args) for tuple unpacking

Example:
days = (“sunday”, “monday”, “tuesday”,”wednesday”,”thursday”)
(day1, day2, *day) = days

Implementation —

def result(a, b):
    return a + b

# function with normal variables
print(result(100, 200))
 
# A tuple is created
c = (100, 300)
 
# Tuple is passed
# function unpacked them
 
print (result(*c))

Output —

300
400

Static Variables and Methods in Python

In Python, Static variables are the variables that belong to the class and not to objects.

• Static variables are shared amongst objects of the class
• Python allows providing the same variable name for a class/static variable and an instance variable

Static Method -

• In Python, Static methods are the methods that are bound to the class rather than an object of the class
• Static Methods are called using the class name and not the objects of the class
• Static methods are bound to the class
• There are two ways of defining a static method:
  1.) @staticmethod
  2.) staticmethod()

Example :
class class_name:
@staticmethod
def function_name(parameters):
print(var)

Implementation —

class test:
    static_variable = 25 

# Access through class
print(test.static_variable) 

# Access through an instance
ins = test()
print(ins.static_variable) 

# Change within an instance
ins.static_variable = 14
print(ins.static_variable)
print(test.static_variable)

Output —

25
25
14
25

Implementation —

# Static Method : Use @staticmethod
class sample_shape:

    @staticmethod
    def msgg(msg):
        print(msg)
        print("Triangles")
        
sample_shape.msgg("Welcome to sample shape class")

Output —

Welcome to sample shape class
Triangles

Lambda Functions in Python

In Python, Lambda is used to create small anonymous functions using the “lambda” keyword and can be used wherever function objects are needed. It can have any number of arguments but only one expression

Syntax:
lambda argument(s): expression

• It can be used inside another function
• In Python, normal functions are defined using the def keyword, and anonymous functions are defined using the lambda keyword
• Whenever we require a nameless function for a short period of time, we use lambda functions

Example:
var = lambda x: x * 5

Implementation —

#A lambda function that adds 10 to the number passed in as an argument, and print the result
x = lambda a, b, c : a * b + c
print(x(5, 6, 8))

Output —

38

Map and Filter Functions in Python

In Python, Map allows you to process and transform the items of the iterable or collections without using a for loop.

• In Python, the map() function applies the given function to each item of a given iterable construct (i.e lists, tuples, etc) and returns a map object

Syntax:
map(function, iterable)

• In Python, the filter() function returns an iterator when the items are filtered through a function to test if the item is true or not

Syntax:
filter(function, iterable)

Example:
result = map(lambda x: x+x, numbers)
var = filter(function_name, sequence)

Implementation —

#map() implementation
numbers = [1,2,3,4,5]
strings = ['s', 't', 'x', 'y', 'z']
mapped_list = list(map(lambda x, y: (x, y), strings, numbers))
print(mapped_list)

Output —

[('s', 1), ('t', 2), ('x', 3), ('y', 4), ('z', 5)]

Implementation —

#map() implementation
days = ['Sunday', 'Monday', 'Tuesday', 'Wednesday']
mod_days = list(map(str.swapcase, days))
print(mod_days)

Output —

['sUNDAY', 'mONDAY', 'tUESDAY', 'wEDNESDAY']

Implementation —

#filter() implementation
marks = [95, 40, 68, 95, 67, 61, 88, 23, 38, 92]
def stud(score):
    return score < 33fail_list = list(filter(stud, marks))
print(fail_list)

Output —

[23]

That’s it for Day 2!

Please visit my next blog for Day 3 and more.

30 days of Data Engineering: Day 3

Follow for more updates & Stay tuned!
Keep learning and coding