In this tutorial, you will master everything about generators in python. You will explore what generators are, how to create and manipulate generators, the significance of keyword yield, how useful generators are in python, etc with the aid of simple examples.
From our previous tutorial, you have gained the knowledge for creating iterators in python which enables you to understand the working of loops at the back end. The use of the
iter() method and
next method, stop Iteration exception, etc makes the code more elongated. Moreover, it seems counterintuitive to memorize the facts for a large data set or programs and automaticity is more welcoming in such cases for effective execution.
Generators, in python programming, are a special kind of function or expression which generates or yields results one at a time on iterating over a generator object. One important feature that makes a generator more efficient than an iterator is the utilization of memory space. Generators usually operate on virtual sequences and are not necessarily required to store the whole sequence in memory.
In python, the two cool ways to create generators are using :
The use of generator expression is one way of building generators in python. This method is more convenient for small data sets like lists in python. Generator expression follows the syntax of list comprehension and hence we can use generator expression in areas where list comprehensions are utilized with a bonus of less memory consumption. The series or sequence are maintained virtually and no need to keep the whole sequence in the memory.
To understand the concept of generator expression more clearly let us examine the following simple and elegant example which shows how list and generator differ from each other.
Str_list = ['Yellow','Orange','Red'] # list comprehension list = [x for x in Str_list] # print the type print(type(list)) # Iterate over items in list and print -1st time for item in list: print(item) # Iterate over items in list and print -2nd time for item in list: print(item)
Yellow Orange Red Yellow Orange Red
Str_list = ['Yellow','Orange','Red'] # Creating a generator using generator expression gen = (x for x in Str_list) # Print the type print(type(gen)) # Iterate over items in generator object and print -1st time for item in gen: print(item) # Iterate over items in generator object and print -2nd time for item in gen: print(item)
Yellow Orange Red
From the above code snippet, you can observe the following:
Str_listwhich contains strings of colors like Yellow, Orange, and Red.
for list object and
for generator object.
len()method to check the length of the list and generator we created.
len(list)will give you 3 as an outcome as the list contains 3 items but
len(gen)will throw a Type Error stating generator object has no length.
The Keyword yield plays a vital role in building a generator function in python. The statement that contains the keyword yield is referred to as yield statement. A yield statement in the generator function is used to control the flow of execution of a function as done by the return statement in the normal function.
To make it more clear let us create a generator function using yield to print a list of colors.
colours = ["Yellow", "Orange", "Red"] def print_colours(colours): for c in colours: yield c colour_generator = print_colours(colours) for c in colour_generator: print(c)
Yellow Orange Red
In the above code snippet, we have created a generator function print_colours with the help of a yield statement. The colour_generator is the object of the generator function which stores the special iterator named generator returned by the generator function. To print the elements inside a generator we have to either use the loops or next() function, here we use ‘for loop’ and hence all values in the generator listed at once.
Before discussing on the difference examine the below codes to print the list of colors:
clr_list =["Yellow", "Orange", "Red"] def print_colours(clrs): for c in clrs: return c C_List = print_colours(clr_list) print(C_List)
['Yellow', 'Orange', 'Red']
clr_list =["Yellow", "Orange", "Red"] def print_colours(clrs): for c in clrs: yield c C_gen = print_colours(clr_list) print(C_gen) print(next(C_gen)) print(next(C_gen)) print(next(C_gen)) print(next(C_gen))
Yellow Orange Red Traceback (most recent call last): File "gen_ex.py", line 62, in print(next(C_gen)) StopIteration
When you observe both code snippets you will see the below findings.
Now we are well acquainted with generators and how it is working behind the scene. The use of
next() function is not a fair practice since we need to mention it multiple times. Loops can overrule the next() function since it is more convenient. See the below example to reverse a list.
# generator to reverse a list def reverse_list(clr_list): length = len(clr_list) for i in range(length-1, -1, -1): yield clr_list[i] # using for loop to reverse the list for list in reverse_list(["Yellow", "Orange", "Red"]): print(list)
Red Orange Yellow
In this script, the execution starts with calling the generator function reverse_list inside the for a loop. The generator function will reverse the given list with the aid of
range() function. The range() function inside the for loop is used to get the index of the list in reverse order. Thus we are yielding list elements in the reverse order. Since the function call is written inside the for loop, it executes till the last element is encountered.
Note: Python generators work well with all iterables.
The two main advantage of using generators is: