Python 3 Reference

Python is a programming language which has gained quite attraction for people working with data; especially in the finance industry. One reason why is its vast amounts of open source libraries for data manipulation and analysis such as Pandas, NumPy and TensorFlow. Python is also very strict in syntax so you are almost guaranteed to have neat code because if you are missing even one space your editor will complain.

Requirements
Syntax
Variables, Objects and Values
Conditionals
Loops
Operators
Regular Expressions
Exceptions
Functions
Classes
String Methods
Containers
File I/O
Databases
Modules

Requirements?

You will need to install a version of Python 3 from the installer found here. You will also need to a text editor and two popular ones as of this time are sublime. and atom. Once those are installed you are ready to begin.

Syntax

Main

The main function is used so that functions can be invoked no matter where the function is defined. We need an if statement to insure that the main function gets called if one exists in the script.

#!/usr/bin/python3
# auto exec main function if exists

def main():
  print("Hello World!")

if __name__ == "__main__": main()

Whitespace

The whitespace tell the compiler that the statements are associated with the preceding function or conditional. Traditionally there are 4 spaces for indentation. There are no character braces ({}) like there are in other languages to specify a block.

# this is correct
def main():
  print("Hello World!")
  print("Line 2")

# if you have one line to exec you can put all on one line
# you can put spaces after :
def main():print("This will work")

#....
if __name__ == "__main__": main()

Comments

To comment use the # symbol for one line comments. Other languages have multi line comments; however, in python they should be avoided.

# this is a comment

Assignment

Assigning values is just like other languages by using the = operator. You also don’t need to specify the type of variable being created; like in ruby.

You can see the class of the variable by using the type(x) function. Also you can mass assign variable in a single line.

Conditionals Structure

The conditionals are similarly structured to how they are in other languages. Use if, elif, else to evaluate statements

a, b = 1, 2

if a < b:
  print("first statement")
elif a > b:
  print("second statement")
else:
  print("third statement")

You can use ternary operation but instead of ? and : you use if and else.

a, b = 1, 2
x = "is True" if a < b else "is False"

Function Structure

To create a function you must specify the def keyword followed by the name of the function and then brackets () or with arguments (x). You can also specify default value for an argument (a=3).

# prints out a + b
def add(a, b):
  print(a + b)

Objects

You need a class before you can instantiate and object. Every class will have an init method which is called each time you create an object for that class.

# create person class
class Person:
  def __init__(self, name="John Smith"):
    self.name = name

  def getName(self):
    return self.name

# create a person object    
pratik = Person()
pratik = Person("Pratik Shah")
print(pratik.getName())

Variables, Objects & Values

Everything in Python 3 is an object so it contains an ID, Type and Value.

Mutable or Immutable

Some object can in Python can change their values which would render them as mutable. If they cannot then they are immutable. You can tell if and object is immutable or not by looking to see if it’s ID has changed.

Immutable - numbers, strings, tuples.

Mutable - lists, dictionaries, and other objects depending on how they are implemented.

Numbers

There are two types of numbers: integers and floats. When you divide the return value is a float unless you round by using // or use round(expression, rounding_digits).

You can also get the remainder of a number by using % modulus. If you want to convert floating point to integer you can typecast by using int(x) or vice versa float(x).

Strings

Strings are immutable objects that can be created using single or double quotes 'string' or "string". You can also use escape characters within a string. If you want the string on one line you place an r before the string; r"this is a raw string". This is good for regular expressions.

You can also do string interpolation with variables by using the {} along with the .format(var) method.

Lists and Tuples (arrays)

Tuples are like lists except they are an immutable object which cannot be changed. They are created by using () ex: tup = (1, 2, 3, 4, 5). Good for a fixed list of data.

Lists aka arrays are mutable and you can add and delete data. They are created by using the [] ex: arr = [1, 2, 3, 4, 5].

Dictionaries (hash)

Dictionaries are like hashes which contain key value pairs. They are created by h = { 'key': 'value' }.

Boolean

A boolean object has the value of either True or False. They are immutable objects so values that are true will always have same id and vice versa if they are false.

x, y = 5, 15

x == y  # will return false
x > y   # will return false
x < y   # will return true

# you can specify true false directly
t = true

Conditionals

If Elif Else Statements

Use if keyword followed by a statement to run a block of code if the statement evaluates to True. You can add an else statement which executes a block of code in the event that the statement in the if evaluates to False

Use elif to evaluate multiple set of conditions

Switch Case

Python does’t have a switch case statement but you can combine a dictionary and a simple variable to create something similar.

# use dictionary to create cases and variable to store choice
# use the get method to get return the value and create a default value if choice does not exist
select = dict(
  one = 1,
  ten = 10,
  hundred = 100,
  thousand = 1000
)
choice = 'hundred'
print(select.get(choice, 'error'))

Ternary

# for simple if else statements condensed into one line
x = True
print("X is true" if x == True else "X is false")

Loops

While

The while loop loops through until the expression evaluates to false. While loop ex: while x < 15.

# fibonacci sequence up to 10
x, y = 0, 1
while y < 10:
  print(y, end=' ')
  x, y = y, x + y

For

for loops allows you to to iterate through an object’s values one by one. For loop ex; for i in cards. For introduces the loop, variable i represents each value one by one and cards will be object you are looping through. You can loop through almost any object.

x = [1, 2, 'text', 4.57]
for i in x:
  print(i, end = '\n')

For with Counter (OG style)

You can use the enumerate method to create and index while looping. for index, line in enumerate(array).

for index, value in enumerate([1, 2, 3, 4, 5]):
  print(index, value, end = '\n')

Break & Continue

If you want to skip an occurrence in a loop use continue. If you want to break out of the loop completely use break. Python also has an else statement which gets run once the loop condition becomes False.

Operators

Arithmetic

+ add
- subtract
* multiply
/ divide
// divides and floors number
* returns remainder
divmod(5, 3) will return both number and remainder
+=, -=, *=, /=, //= short hand

Comparison

The comparison operators will either return True or False

< less than
> greater than
<= less than or equal to
>= greater than or equal to
== equal to
!= not equal to
is and is not will compare ids of Objects

Boolean

True and False are part of boolean class

and both statments must be true
or one of the two statements must be true

Container with Slice

You can use slices or ranges to return a specific starting and end point in an object. The last point is non inclusive.

Operator Precedence

Basic order of operations work when adding, subtracting etc… Use () to evaluate a block that you need first. Here is a link to Python’s Order of Operations

Regular Expressions

RegEx Module

Regular expressions helps match patterns in text. In Python you use the re module to implement regular expressions. The module must be imported first import re. To use it you simply use method re.compile(r'\d') to create a pattern and re.search(regex, text) to search for the pattern within the text.

Searching & Replacing

To search and replace text use the sub method sub(search_text, replace_val, text).

RegEx Compiling

To precompile the regex you will be looking for use the compile(pattern) method and set it to a variable ex: pattern = re.compile('Pratik')

Exceptions

Exceptions are use to handle error control. You can use try and catch to execute statements and retrieve errors if any. You can create your own exceptions with raise

try:
    file = open("filename")
except IOError as e:
    print(e)
else:
    for l in file:print(l)

Handling Exceptions

Use try and except for code that may return errors.

# this will catch all error
try:
    file = open("errrrror.txt")
except:
    print("File open error")
else:
    for l in file: print(l)

#-----------------------------

# you can also specify a specific error type
except IOError as e:

Raising Exceptions

You can create your own custom error messages by using raise

# check to see if the filename ends with .doc
if filename.endswith('.doc'):
  fh = open(filename)
else:
  raise FileExtension('Filename must end with .doc')

try:
    file = open("errrrror.txt")
except FileExtension as e:
    print("Bad filename", e)
else:
    for l in file: print(l)

Functions

Defining a Function

To create a function you must first use the def keyword followed by the name of the function and parenthesis with optional arguments, if any, ex: def add(x, y). If you would like to create the function signature without any statements you can use the pass keyword so that the interpreter will skip over it.

# run main function on load
def main():
    testfunc()
    add(5, 6)

# function without arguments
def testfunc():
    print("this is a test function")

# function with arguments
def add(x, y):
    print(x + y)

if __name__ == "__main__": main()

Arguments

If you don’t know the number of arguments that may be passed you can pass *args which will then give you access to the values through a tuple.

# function with unknown arguments
def add(*args):
    total = 0
    for i in args:
      total += i
    print(total)

add(1, 2, 3, 4, 5)

Named Functions

Use named arguments **kwargs to specify default values and access via dictionary.

def main():
    numbers(one = 1, ten = 10, hundred = 100)

# function with named arguments
def numbers(**kwargs):
  print(kwargs['one'], kwargs['ten'])

main()

Return Values from Functions

Use return keyword followed by object or variable

def main():
    numbers()

# function with named arguments
def numbers():
    return range(100)

main()

Generator Function

The generator function allows you to continue running statements after the yield keyword each time it runs until the loop becomes false.

def main():
    for i in numbers(0, 25, 5):
      print(i, end=' ')

# function with named arguments
def numbers(start, stop, step):
    i = start
    while i <= stop:
      yield i
      i += step

main()

Classes

Classes and Objects

Classes are how you create your own objects. A class is the blueprint for an object. An object is an instance of a class.

# create human class
class Human:
    def talk(self):
        print('Hello!')

    def run(self):
        print('Zoom zoom!')

# create an instance from Human class        
pratik = Human()
pratik.talk()
pratik.run()

Methods

Call an attribute of an object by using the . operator. The keyword self refers to the instance of the object. You can create a constructor method def __init__(self) which auto executes when a new instance of the class is created.

# create human class
class Human:
    def __init__(self, name, age, height):
      self.name = name
      self.age = age
      self.height = height

    def info(self):
        print("My name is {} and I am {} years old and am {} tall".format(self.name, self.age, self.height))

# create an instance from Human class        
pratik = Human("Pratik", "25", "5\'7")
pratik.info()

Object Data

After creating attributes for an object you can create methods to get and set values of each attribute. A better design would be to take in a hash of values and mass assign it.

# create human class
class Human:
    def __init__(self, name, age, height):
      self.name = name
      self.age = age
      self.height = height

    def info(self):
        print("My name is {} and I am {} years old and am {} tall".format(self.name, self.age, self.height))

    def get_name(self):
      return self.name

    def set_name(self, name):
      self.name = name

pratik = Human("Pratik", "25", "5\'7")
pratik.info()
print(pratik.get_name())

scalability refactoring

def __init__(self, **kwargs):
    self.variables = kwargs

def set_variable(self, k, v):
    self.variables[k] = v

def get_variable(self, k):
    return self.variables.get(k, None)

Inheritance

A class can use methods of another class via inheritance. To inherit you simply drop the class as an argument to the inheriting class ex: def Human(Mammal). If you have conflicting method signatures the child will override the parent unless you use the super().methodName().

# create mammal class
class Mammal:
    def walk(self): print('I walk normally.')
    def talk(self): print('I can speak!')

# human class inherits from Mammal
class Human(Mammal):
    def __init__(self, name, age, height):
      self.name = name
      self.age = age
      self.height = height

    def info(self):
        print("My name is {} and I am {} years old and am {} tall".format(self.name, self.age, self.height))

    def get_name(self):
      return self.name

    def set_name(self, name):
      self.name = name

    # The super().method() will call the parent
    def walk(self):
      super().walk()
      print('Zoom Zoom!')

pratik = Human("Pratik", "25", "5\'7")
pratik.info()
print(pratik.get_name())
pratik.walk()
pratik.talk()

Polymorphism

Transforms one class as if it was another class. Also known is duck typing. This is good to create one super class to be used by multiple sub classes.

class Dog:
  def bark(self):
    print('Wooof!')

  def walk(self):
    print('Runs!')

  def fur(self):
    print('Lots of it.')

class Cat:
  def bark(self):
    print('Meeeoooowww!')

  def walk(self):
    print('Slow!')

  def fur(self):
    print('Silky smooth')

bud = Dog()
whisk = Cat()

for obj in (bud, whisk):
  obj.bark()
  obj.walk()
  obj.fur()

# create method where it takes an obj
def dog_or_cat_doesnt_matter(dog):
  dog.bark()
  dog.walk()

# put cat obj inside method for dog obj
dog_or_cat_doesnt_matter(whisk)

Using Generators

You can create a class that is iterate-able. You need to specify in the class the method def __iter__(self).

class inclusive_range:
    def __init__(self, *args):
      numargs = len(args)
      if numargs < 1: raise TypeError('Requires 1 or more arguments')
      elif numargs == 1:
        self.stop = args[0]
        self.start = 0
        self.step = 1
      elif numargs == 2:
        (self.start, self.stop) = args
        self.step = 1
      elif numargs == 3:
        (self.start, self.stop, self.step) = args
      else: raise TypeError('Expected 3 arguments, got {}'.format(numargs))

    # allows class to iterate
    def __iter_(self):
      i = self.start
      while i <= self.stop:
          yield i
          i += self.step

Decorators

Special functions that return other functions. Using @property before a getter or setter method turns it into a decorator. Other decs are @prop.setter or @prop.deleter. This will give the control to the object and you don’t have to call the method each time simply access the property and set a new value.

@property
def name(self):
    return self.properties.get('name', None)

@name.setter
def name(self, name):
    self.name = name

# you can now do this
pratik = Human()

pratik.name = "Pratik"

String Methods

Object of Type String & Methods

.format(var) use format method to do string interpolation
.capitalize() will capitalize the first letter
.upper() will capitalize all letters
.lower() will lowercase all letters
.swapcase() will alter the case of the letter
.find('str') will return first index of the letter found
.replace('str', 'str2') replaces and returns a new string (new ID)
.strip() removes all whitespaces
.rstrip() removes whitespaces at end of string
.isalnum() checks to see if all characters are alphanumeric (t/f)
.isalpha() checks for characters a-z
.isdigit() checks to see if text are all digits
.isprintable() checks to see if all characters are printable

String Format

# this will display b twice and a once.
a, b = 12, 4
'this {1} will display {1} {0}'

# you can even use variables and drop in a dictionary in the format
d = dict( pratik = a, john = b)
'my name is {pratik} and he is {john}'.format(**d)

Split and Join Strings

.split() will split each word one by one
.split('split text') will split each time based on “split text”
'join divider'.join(arr_to_join) will join string and place “join divider” in between each word.

Containers

Tuples and Lists Methods

To create a tuple you must include a , or else it will return an integer ex: t = (1,)

len(list) gets length of collection
min(list) gets smallest data
max(list) gets largest data
x in list returns t/f if x is in tuple
x not in list returns t/f if x is not in tuple
.count(x) returns occurrence of x
.index(x) returns the first index x is found at
.append(x) appends x to list (NO WORK ON TUPPLE)
.extend(x) appends x to end of list
.insert(i, v) inserts v into index i
.remove(v) removes first occurrence of v
del arr[i] deletes arr data at index i
.pop() removes last element in list
.pop(i) removes element at index i

Dictionaries Methods

dict( name = v, name2, v2) creates dictionary
'one' in hash returns true/false if key is in dictionary
h.get('two') gets value from dictionary
h.get('two', default) gets value from dictionary else returns default value
del h['one'] deletes dictionary key value
h.pop(key) deletes dictionary key value

File I/O

Opening Files

To open files use open('text.txt') and it will send back file object.

r read only mode
w write only mode
a append mode
r+ read and write
rt text file mode
rb binary mode

# alternate options r = read only, w = write, a = append to end of file.
# r+ for read and write
f = open('text.txt')
for line in f.readlines():
  print(line, end = '')

Reading and Writing Files

For small text files you can copy line by line

infile = open('lines.txt', 'r')
outfile = open('new.txt', 'w')
for line in infile:
    print(line, file = outfile, end = '')

Use buffer for large text files to copy in chunks

buffersize = 50000
infile = open('lines.txt', 'r')
outfile = open('new.txt', 'w')
buffer = infile.read(buffersize)
while len(buffer):
    outfile.write(buffer)
    print('.', end = '')
    buffer = infile.read(buffersize)

Reading and Writing Binary Files

You must open the file in binary mode but also use buffer

buffersize = 50000
infile = open('pics.jpg', 'rb')
outfile = open('new.jpg', 'w'b)
buffer = infile.read(buffersize)
while len(buffer):
    outfile.write(buffer)
    print('.', end = '')
    buffer = infile.read(buffersize)

Databases

SQLite 3 Database

Python works well with SQLite 3 for databases and even contains a built in class for it: import sqlite3. Create a db by using sqlite3.connect('name.db') and use the .execute('text command') command to drop, add and select data. Use commit() to save the records to the db. To get rows back in an object form use the command sqlite3.Row.

Modules

Standard Library

Python has quite a few standard library modules that you can use which can be found here DOCS.

sys allows you to get back system specs
- .version_info sends back python version
- .platform sends back OS kernal version
os send back operating system information
- .name gets back OS name
- .getenv(var_name) gets back environment variable
- .getcwd get working directory path
- .name gets back OS name
- .urandom(n) gets random n bytes of letters
urllib.request used to get url pages
- .urlopen(url) get back http webpage object (must convert from binary)
random
- randint(start, stop) gets back random number from and to start stop
- shuffle(x) shuffles around an array/list
datetime
- datetime.now() gets back time in year, month, day, hour, minute, seconds, microseconds

Third Party PyPi Modules

List of modules. The modules installed are located in site-packages

Table of Contents

Requirements?

Syntax

Main

Whitespace

Comments

Assignment

Conditionals Structure

Function Structure

Objects

Variables, Objects & Values

Mutable or Immutable

Numbers

Strings

Lists and Tuples (arrays)

Dictionaries (hash)

Boolean

Conditionals

If Elif Else Statements

Switch Case

Ternary

Loops

While

For

For with Counter (OG style)

Break & Continue

Operators

Arithmetic

Comparison

Boolean

Container with Slice

Operator Precedence

Regular Expressions

RegEx Module

Searching & Replacing

RegEx Compiling

Exceptions

Handling Exceptions

Raising Exceptions

Functions

Defining a Function

Arguments

Named Functions

Return Values from Functions

Generator Function

Classes

Classes and Objects

Methods

Object Data

Inheritance

Polymorphism

Using Generators

Decorators

String Methods

Object of Type String & Methods

String Format

Split and Join Strings

Containers

Tuples and Lists Methods

Dictionaries Methods

File I/O

Opening Files

Reading and Writing Files

Reading and Writing Binary Files

Databases

SQLite 3 Database

Modules

Standard Library

Third Party PyPi Modules