Python Modules and Packages

In this tutorial, you will master all about python modules and python packages, the two programming techniques that ease modular programming. Also will cover the topics like how to reload a module, how to import modules from a package etc with examples.

Modular programming is a design technique used in software programming. It emphasizes the technique of splitting a large program into discrete, tiny, compatible subprograms or modules. The modular approach offers the following listed benefits to a program.

• Provides simplicity to program
• Easy to maintain and debug
• Allows reusability of codes
• Avoids code redundancy

Python constructs that boost modularity are Functions, Modules, and Packages. In the previous tutorial, we have learned all about functions in python.

What are the Modules in python?

A module is a simple file consisting of definitions and executable statements. Creating a python module is not a difficult task. All you have to do is :

• create a python file
• Name the file and prepend with .py extension.

For instance, in the below example mymod.py is a module with one variable L  and two functions prt() and sum().

L=['Red','Green','Blue']
print('Colour List is',L)

def prt(name):
 print('Hello',name)

def sum(a,b):
 total = a+b
 print(total) 

A module can contain several objects like variables, functions, class, etc.

How to import modules in python?

A module can be accessed to another module or to an interactive interpreter using the keyword import. The below example shows how to import our previously defined module.

import mymod  #imports the module mymod.py  

Correspondingly we can import builtin modules present in python also. Before that let's learn how to check the list of built-in modules available in python. To get the list of built-in modules uses any of the two methods in the below example.

>>> import sys
>>> print(sys.builtin_module_names)
 
Or 

>>> help('modules') 

The output will be a list of built-in modules in python as shown below.

('_abc', '_ast', '_bisect', '_blake2', '_codecs', '_codecs_cn', '_codecs_hk', '_codecs_iso2022', '_codecs_jp', '_codecs_kr', '_codecs_tw', '_collections', '_contextvars', '_csv', '_datetime', '_functools', '_heapq', '_imp', '_io', '_json', '_locale', '_lsprof', '_md5', '_multibytecodec', '_opcode', '_operator', '_pickle', '_random', '_sha1', '_sha256', '_sha3', '_sha512', '_signal', '_sre', '_stat', '_statistics', '_string', '_struct', '_symtable', '_thread', '_tracemalloc', '_warnings', '_weakref', '_winapi', '_xxsubinterpreters', 'array', 'atexit', 'audioop', 'binascii', 'builtins', 'cmath', 'errno', 'faulthandler', 'gc', 'itertools', 'marshal', 'math', 'mmap', 'msvcrt', 'nt', 'parser', 'sys', 'time', 'winreg', 'xxsubtype', 'zlib')

To access the content or objects in the module we use the dot operator (.). Now let’s check out how to import the contents from our formerly defined module mymod.py using the dot operator.

>>> import mymod      #imports the module mymod
>>> mymod.prt('Chris')  #imports the function prt in module mymod
Hello Chris
>>> mymod.sum(10,20)    #imports the function sum in module mymod
30

Going forward we will learn several other ways to import modules and their contents.

1. The import statement

   Using keyword import and dot operator we can also import the built-in functions in python, the same way we did for the user-defined function. An example is given below to show the use of the import statement.

   import math
   print('Factorial of 5 is :',math.factorial(5)) 
   

   Factorial of 5 is : 120
   

   Here in this example, the built-in module math is imported and its object pi is accessed using the dot operator.

2. Import with renaming

   It is also possible to rename the module once it is imported and use that name to access its contents. Following example shows the renaming of the module.

   import math as X
   print('Factorial of 5 is :',X.factorial(5))
   

   Factorial of 5 is : 120
   

   Here the module math is renamed as X and we use X to access the object pi.

3. from...import statement

   Another way of importing content is by specifying the name of the content and the module in one line. Here we don't need the dot operator to import the content.

   from math import factorial
   print('Factorial of 5 is :',factorial(5)) 
   

   Factorial of 5 is : 120
   

   This example tells us that module object pi is imported from the module math.

4. from...import* statement

   Assume the case where you need to import multiple contents from the same module. In such situation from...import* is the perfect solution. Here we can access all the definitions in the modules. * indicates the access to all the contents or definitions available in the module.

   from math import *
   print('Factorial of 5 is :',factorial(5))
   

   Factorial of 5 is : 120
   

   Even though this is an easy way to import definitions ,it is not the best programming practice as it curbs the code readability

Python module search path

We are now familiar with various methods to import contents from a module, So what will the interpreter do once a module gets imported ? The interpreter will initially check if  the module is available in a built-in module. If it is not found then the python will search for the system search path defined in the sys.path ,which contains a list of directories.
The following example gives you the idea of sys.path

import sys
print(sys.path) 

['', 'C:\\Users\\Programs\\Python\\Python38-32\\Lib\\idlelib', 'C:\\Users\\Programs\\Python\\Python38-32\\python38.zip', 'C:\\Users\\Programs\\Python\\Python38-32\\DLLs', 'C:\\Users\\Programs\\Python\\Python38-32\\lib', 'C:\\Users\\Programs\\Python\\Python38-32', 'C:\\Users\\Programs\\Python\\Python38-32\\lib\\site-packages']

The search follows the order as listed below.

1. Current directory
2. The environment variable, PYTHONPATH
3. The installation dependent default path

Sometimes you may get errors when trying to import a user-defined module. The reason for this is the improper location of the file. So you have to make sure that the module file is placed in any of the above three(current directory, python path or default path).

You can also discover the location of the module using the file attribute after it is imported :

import mymod
mymod.__file__

'C:\\Users\\Programs\\Python\\Python38-32\\mymod.py'

The dir() function

The dir() is a python built-in function which returns an alphabetically sorted list of defined names in a namespace. This list consists of all the modules(built-in and user-defined), variables and functions which were included in the module. The following example gives you the instance of the use of dir() function.

import mymod
dir(mymod)

['L', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'prt', 'sum']

Here the names that start with an underscore are default attributes in the module. For instance, __name__ attribute provides the name of the module.L,prt, sum is the objects or user-defined attributes in the module mymod.

Reloading a module

One of the significant notes on the module is that python imports modules only once per interpreter session. Let's understand the below example

import mymod
Colour List is ['Red', 'Green', 'Blue']
import mymod
import mymod

In the above example, you can find that the executable statement is executed only once and the print statements in the subsequent imports are not ignored. This indicates that a module is imported only once.

If you made any change to the  module contents or if you  want to re-execute the statement then you can either restart the python interpreter or use the function reload(). The reload function resides in the module importlib and hence you need to import that first.

The syntax is quite simple as shown below:

reload(module_name)  

Where module_name is the name of the module you wish to reload.  The print statement executes after reloading as shown in the below example

import importlib
importlib.reload(mymod) 

Colour List is ['Red', 'Green', 'Blue']

What are packages in python

So far you have learned what a module is and how to import modules and the objects in it. Now we are going to cognize with packages which will help us to organize the codes for the applications in a well-structured manner.

In simple language, Packages are collections of modules or packages itself. Packages inside a package is known as a subpackage. Packages help to store directories and modules in an organized manner preferably in a hierarchical structure. This will enable efficient importing when the size of the application grows bigger with the use of numerous modules.

We can store similar kinds of modules in one package and different modules in yet another package. Doing this helps to avoid collision between modules in a package and collision between global variables within a module as well.

Creating a package is quite as simple as creating a folder on the computer. We would have the main folder containing subfolders and files. Analogously, a package contains sub-packages and modules. The below visualization is an example of the hierarchical structure of the package ‘P’ with two sub packages sp1 and sp2. sp1 contains two modules m1 and m2 while sp2 has module m3.

The hierarchical structure of the package

Below shows the different ways of importing sub-packages and modules from a package. All we need is to use a dot notation to import the modules or sub-packages inside a package.

import p   # imports package P
import p.sp1,p.sp2 # imports  subpackages from package P
import p.sp1.m1,p.sp2 # imports modules in subpackages 
import p.sp1.m2 as X  
from p.sp1.m2 import object  # imports contents in a module 

Package Initialization

Python considers a directory as a package when it contains the file named _init_. Literally, a _init_.py is used to initialize the package object or the global variable. This file can be kept empty too. The general form of the package is depicted in the below picture.

The general form of the package

How to import modules from a package

We have seen the general syntax for importing modules from a package. Let us elaborate that to give you a clear cut concept. For that, we have modified our mymod module example into a package and let see how package helps in modular programming.

We have a package name ‘mypackage’ which contains __init__.py file and two subfolders -print and calc. Each sub-package contain modules. Print sub-package contains two modules namely,

• list_print - prints a list.
• string_print - prints a string.

while calc sub-package contains __init__.py file and a function named sum_calc which performs the addition of two numbers.

Given below is a visualization of the aforementioned example.

Now let's see what's inside these modules.

L=['Red','Green','Blue']
print('Colour List is',L) 

def prt(name):
 print('Hello',name) 

def sum(a,b):
 total = a+b
 print('Total is :',total) 

Different importing methods with example

1. Using the import statement

   import mypackage.print.list_print.L 
   

   Colour List is ['Red', 'Green', 'Blue']

2. Import with renaming

   import mypackage.print.String_print as X
   X.prt(‘Chris’)
   

   Hello,Chris

3. Using from...import statement

   from mypackage.print import sum_calc
   sum_calc.sum(10,20) 
   

   Total is :30

We have come across module* which indicates that we can access all definitions inside a module. Similarly, we can access all modules in a package using * notation, but the point to be remembered is that it should contain a list called  __all__  in the __init__.py file.  when the statement from import * is confronted, _all_ attribute takes a list of modules that should be imported.